perf: record the Woodpecker G4 measured baseline
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
ci/woodpecker/push/main Pipeline was successful
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
ci/woodpecker/push/main Pipeline was successful
The manual .woodpecker/performance.yml workflow ran on the designated fixed agent (platform_id woodpecker-agent-1): 5 warmups + 20 samples, all 9 workloads correctness-verified, CPU 0 pinned from the container cpuset. Two runs differed by -10.9%..+7.1% p50 and -1.9% RSS. The node is a shared Docker container with the powersave governor and loadavg ~5.7-6.1, so the release gate is WARN, not PASS, and comparisons are same-platform only. Store both marqdo.perf.v2 reports and the SHA-bound release manifest. The manifest is DO_NOT_RELEASE only because strict_vm remains UNVERIFIED; Linux, Windows, macOS and performance have real same-SHA results.
This commit is contained in:
1 parent
1199c88e3e
commit
c11c1c858a
7 files changed
+660
-25
No files matched your search
+15
-12
@@ -80,9 +80,9 @@ The intermediate "text migrated, implementation pending" record is kept as
|
||||
[`evidence/2026-11-21-semantic-docs-migration.md`](evidence/2026-11-21-semantic-docs-migration.md)
|
||||
and is superseded by the green gate.
|
||||
|
||||
**Not claimed:** the **remote** G3 security artifact and a **MEASURED** G4
|
||||
performance report are still missing. The Linux/Windows/macOS native matrix is
|
||||
green at `5fcc111`, and the G3 mechanism is PASS in a local disposable VM, but
|
||||
**Not claimed:** the **remote** G3 security artifact is still missing. The
|
||||
Linux/Windows/macOS native matrix is green, G4 is measured on the designated
|
||||
Woodpecker agent, and the G3 mechanism is PASS in a local disposable VM, but
|
||||
the RFC does not promote a local VM run to a remote gate. See §0.1, §4 and
|
||||
[`../COMPATIBILITY.md`](../COMPATIBILITY.md). The release decision remains
|
||||
**DO NOT RELEASE**.
|
||||
@@ -115,15 +115,15 @@ Executed evidence:
|
||||
| Linux fmt/clippy/test/build (Woodpecker) | **PASS** | pipelines `#9`–`#12`, `#14`, `#16`, `#17`, `#21`–`#25` on the series; `#13`/`#15`/`#19` were superseded/cancelled by the next push, not failures |
|
||||
| Linux + Windows + macOS native matrix (GitHub) | **PASS** | [run 30](https://github.com/cflmy/marqdo-cli/actions/runs/44896671929) on `5fcc111`: `fmt + clippy + test (Linux x86_64)`, `runtime contract (windows-latest)`, `runtime contract (macos-latest)`, and schema consumers all PASS |
|
||||
| G3 strict isolation | **PASS (local disposable VM) / remote UNVERIFIED** | `5fcc111`, `dev` uid 1000, QEMU/TCG, 7/7 + strict JSON/inert-`@knowledge`; [`evidence/2026-11-21-strict-gc-5fcc111.md`](evidence/2026-11-21-strict-gc-5fcc111.md) |
|
||||
| G4 performance | **SMOKE / UNVERIFIED** | nine workloads `VERIFIED` on the dev container, 1 warmup + 3 samples; [`evidence/2026-11-21-perf-smoke-0d7e11b.md`](evidence/2026-11-21-perf-smoke-0d7e11b.md) |
|
||||
| G4 performance | **MEASURED / WARN (provisional)** | Woodpecker agent 1, `1199c88`: 5 warmups + 20 samples, 9/9 `VERIFIED`, CPU 0 pinned; two runs -10.9%..+7.1% p50; shared powersave host caveat; [`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md) |
|
||||
|
||||
Release decision: **DO NOT RELEASE.** Linux, Windows and macOS native jobs are
|
||||
green on the same SHA, and the strict isolation mechanism PASSes locally as a
|
||||
normal user. The remaining release gates are the **remote** G3 artifact whose
|
||||
`tested_sha` matches and a **MEASURED** G4 report on the fixed node; neither
|
||||
remote claim is made here. The RFC explicitly requires the remote G3 artifact
|
||||
for a release, so the local VM PASS is recorded but not promoted to a release
|
||||
gate.
|
||||
green on the same SHA, the strict isolation mechanism PASSes locally as a
|
||||
normal user, and G4 is `MEASURED`/`WARN` on the designated Woodpecker agent.
|
||||
The remaining release gate is the **remote** G3 artifact whose `tested_sha`
|
||||
matches; the RFC explicitly requires it (and the local VM PASS is not promoted
|
||||
to a remote claim). The perf gate is provisional because the agent is a shared
|
||||
powersave host; see [`performance.md`](performance.md).
|
||||
|
||||
The manifest contract is in
|
||||
[`evidence/ci/README.md`](evidence/ci/README.md); `tools/ci/release_evidence.py`
|
||||
@@ -488,9 +488,12 @@ user-written allocator is explicitly **not** claimed.
|
||||
- [`evidence/2026-11-21-strict-gc-5fcc111.md`](evidence/2026-11-21-strict-gc-5fcc111.md)
|
||||
— RFC-CI-0001 series G-C: local disposable VM, `dev` uid 1000, 7/7 plus the
|
||||
strict JSON and inert-`@knowledge` contracts at `5fcc111`; the earlier `512b9f5`/`0d7e11b` runs are historical.
|
||||
- [`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md)
|
||||
— G4 `MEASURED` provisional baseline on the designated Woodpecker agent
|
||||
(`1199c88`): 9/9 correctness-verified, 5 warmups + 20 samples, repeatability
|
||||
deltas, and the shared-host/powersave caveat.
|
||||
- [`evidence/2026-11-21-perf-smoke-0d7e11b.md`](evidence/2026-11-21-perf-smoke-0d7e11b.md)
|
||||
— `marqdo.perf.v2` smoke: nine workloads correctness-verified, explicitly
|
||||
not a G4 release baseline.
|
||||
— earlier `marqdo.perf.v2` smoke; historical, not the release baseline.
|
||||
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
|
||||
L1/L2 limits.
|
||||
|
||||
|
||||
@@ -0,0 +1,246 @@
|
||||
{
|
||||
"schema": "marqdo.perf.v2",
|
||||
"source_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6",
|
||||
"platform_id": "woodpecker-agent-1",
|
||||
"environment_approved": true,
|
||||
"binary": {
|
||||
"path": "target/release/marqdo",
|
||||
"sha256": "sha256:15ead6261c7500eb3fbcb7c9f60b76abf85b4006883dd23850f74a43af5b18c0",
|
||||
"profile": "release",
|
||||
"cargo_lock_sha256": "sha256:5b9754a3ff7f56ca0158d70772365f02dcbfbc1ebbd97a55ee09d441c2d2af01",
|
||||
"rustc": "rustc 1.100.0 (0ef1bc386 2026-11-09)\nbinary: rustc\ncommit-hash: 0ef1bc386059596dada8fb65b50425aab2ea637a\ncommit-date: 2026-11-09\nhost: x86_64-unknown-linux-gnu\nrelease: 1.100.0\nLLVM version: 23.1.5",
|
||||
"cargo": "cargo 1.100.0 (3acc79c0c 2026-11-05)"
|
||||
},
|
||||
"environment": {
|
||||
"uname": "Linux-7.0.0-31-generic-x86_64-with-glibc2.36",
|
||||
"kernel": "7.0.0-31-generic",
|
||||
"machine": "x86_64",
|
||||
"cpu_model": "12th Gen Intel(R) Core(TM) i7-12700H",
|
||||
"governor": "powersave",
|
||||
"loadavg_start": "6.13 2.82 1.58 1/2879 2892",
|
||||
"cpu_list": "0",
|
||||
"python": "3.11.2",
|
||||
"loadavg_end": "6.13 2.82 1.58 1/2879 2892"
|
||||
},
|
||||
"recipe": {
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"timeout_seconds": 120.0
|
||||
},
|
||||
"workloads": [
|
||||
{
|
||||
"name": "json-parser",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/62-json-array-parser.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/62-json-array-parser.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 6.6149,
|
||||
"p95_ms": 6.838,
|
||||
"mad_ms": 0.1795,
|
||||
"min_ms": 5.7414,
|
||||
"max_ms": 7.786,
|
||||
"cpu_user_ms_median": 3.945,
|
||||
"cpu_sys_ms_median": 1.8365,
|
||||
"max_rss_kb_median": 17676
|
||||
},
|
||||
{
|
||||
"name": "allocator-free-list",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 4.5207,
|
||||
"p95_ms": 4.5971,
|
||||
"mad_ms": 0.0323,
|
||||
"min_ms": 4.4644,
|
||||
"max_ms": 4.6091,
|
||||
"cpu_user_ms_median": 2.322,
|
||||
"cpu_sys_ms_median": 1.5715,
|
||||
"max_rss_kb_median": 17676
|
||||
},
|
||||
{
|
||||
"name": "http-parser",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/71-http-parser.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/71-http-parser.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 4.5041,
|
||||
"p95_ms": 4.5946,
|
||||
"mad_ms": 0.0347,
|
||||
"min_ms": 4.3909,
|
||||
"max_ms": 4.6033,
|
||||
"cpu_user_ms_median": 3.0935,
|
||||
"cpu_sys_ms_median": 1.031,
|
||||
"max_rss_kb_median": 17676
|
||||
},
|
||||
{
|
||||
"name": "table-sort",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/65-table-sort-stdlib.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/65-table-sort-stdlib.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 2.366,
|
||||
"p95_ms": 2.4724,
|
||||
"mad_ms": 0.0254,
|
||||
"min_ms": 2.3177,
|
||||
"max_ms": 2.4961,
|
||||
"cpu_user_ms_median": 0.9655,
|
||||
"cpu_sys_ms_median": 0.9815,
|
||||
"max_rss_kb_median": 17676
|
||||
},
|
||||
{
|
||||
"name": "hello-check",
|
||||
"kind": "check",
|
||||
"source": "examples/hello.mq.md",
|
||||
"argv": [
|
||||
"check",
|
||||
"examples/hello.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 4.8822,
|
||||
"p95_ms": 4.9401,
|
||||
"mad_ms": 0.0396,
|
||||
"min_ms": 4.4905,
|
||||
"max_ms": 4.9744,
|
||||
"cpu_user_ms_median": 2.067,
|
||||
"cpu_sys_ms_median": 2.014,
|
||||
"max_rss_kb_median": 17676
|
||||
},
|
||||
{
|
||||
"name": "bytecode-vm",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/63-memory-bytecode-vm.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/63-memory-bytecode-vm.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 2.3634,
|
||||
"p95_ms": 2.4443,
|
||||
"mad_ms": 0.0384,
|
||||
"min_ms": 2.2816,
|
||||
"max_ms": 3.4082,
|
||||
"cpu_user_ms_median": 1.1155,
|
||||
"cpu_sys_ms_median": 1.073,
|
||||
"max_rss_kb_median": 17676
|
||||
},
|
||||
{
|
||||
"name": "stdlib-print-demo",
|
||||
"kind": "run",
|
||||
"source": "stdlib/demo.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"stdlib/demo.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 11.9034,
|
||||
"p95_ms": 12.0538,
|
||||
"mad_ms": 0.1354,
|
||||
"min_ms": 10.8367,
|
||||
"max_ms": 12.095,
|
||||
"cpu_user_ms_median": 8.1425,
|
||||
"cpu_sys_ms_median": 2.826,
|
||||
"max_rss_kb_median": 17676
|
||||
},
|
||||
{
|
||||
"name": "factorial-run",
|
||||
"kind": "call",
|
||||
"source": "examples/factorial.mq.md",
|
||||
"argv": [
|
||||
"call",
|
||||
"examples/factorial.mq.md",
|
||||
"数学.阶乘",
|
||||
"--arg",
|
||||
"n=5",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 2.3949,
|
||||
"p95_ms": 2.4406,
|
||||
"mad_ms": 0.0244,
|
||||
"min_ms": 2.3354,
|
||||
"max_ms": 2.4659,
|
||||
"cpu_user_ms_median": 0.0,
|
||||
"cpu_sys_ms_median": 1.743,
|
||||
"max_rss_kb_median": 17676
|
||||
},
|
||||
{
|
||||
"name": "matrix-multiply",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/70-matrix-multiply.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/70-matrix-multiply.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 2.5671,
|
||||
"p95_ms": 2.6745,
|
||||
"mad_ms": 0.0744,
|
||||
"min_ms": 2.4391,
|
||||
"max_ms": 2.7945,
|
||||
"cpu_user_ms_median": 0.48,
|
||||
"cpu_sys_ms_median": 1.4085,
|
||||
"max_rss_kb_median": 17676
|
||||
}
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"baseline_comparison": []
|
||||
}
|
||||
@@ -0,0 +1,246 @@
|
||||
{
|
||||
"schema": "marqdo.perf.v2",
|
||||
"source_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6",
|
||||
"platform_id": "woodpecker-agent-1",
|
||||
"environment_approved": true,
|
||||
"binary": {
|
||||
"path": "target/release/marqdo",
|
||||
"sha256": "sha256:15ead6261c7500eb3fbcb7c9f60b76abf85b4006883dd23850f74a43af5b18c0",
|
||||
"profile": "release",
|
||||
"cargo_lock_sha256": "sha256:5b9754a3ff7f56ca0158d70772365f02dcbfbc1ebbd97a55ee09d441c2d2af01",
|
||||
"rustc": "rustc 1.100.0 (0ef1bc386 2026-11-09)\nbinary: rustc\ncommit-hash: 0ef1bc386059596dada8fb65b50425aab2ea637a\ncommit-date: 2026-11-09\nhost: x86_64-unknown-linux-gnu\nrelease: 1.100.0\nLLVM version: 23.1.5",
|
||||
"cargo": "cargo 1.100.0 (3acc79c0c 2026-11-05)"
|
||||
},
|
||||
"environment": {
|
||||
"uname": "Linux-7.0.0-31-generic-x86_64-with-glibc2.36",
|
||||
"kernel": "7.0.0-31-generic",
|
||||
"machine": "x86_64",
|
||||
"cpu_model": "12th Gen Intel(R) Core(TM) i7-12700H",
|
||||
"governor": "powersave",
|
||||
"loadavg_start": "5.66 2.95 1.78 1/2879 2892",
|
||||
"cpu_list": "0",
|
||||
"python": "3.11.2",
|
||||
"loadavg_end": "5.66 2.95 1.78 2/2879 2892"
|
||||
},
|
||||
"recipe": {
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"timeout_seconds": 120.0
|
||||
},
|
||||
"workloads": [
|
||||
{
|
||||
"name": "json-parser",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/62-json-array-parser.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/62-json-array-parser.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 5.8948,
|
||||
"p95_ms": 6.8515,
|
||||
"mad_ms": 0.0947,
|
||||
"min_ms": 5.7445,
|
||||
"max_ms": 6.916,
|
||||
"cpu_user_ms_median": 3.449,
|
||||
"cpu_sys_ms_median": 2.215,
|
||||
"max_rss_kb_median": 17348
|
||||
},
|
||||
{
|
||||
"name": "allocator-free-list",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 4.8416,
|
||||
"p95_ms": 4.8886,
|
||||
"mad_ms": 0.0191,
|
||||
"min_ms": 4.7903,
|
||||
"max_ms": 4.8906,
|
||||
"cpu_user_ms_median": 2.8555,
|
||||
"cpu_sys_ms_median": 0.9645,
|
||||
"max_rss_kb_median": 17348
|
||||
},
|
||||
{
|
||||
"name": "http-parser",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/71-http-parser.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/71-http-parser.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 4.5,
|
||||
"p95_ms": 4.6073,
|
||||
"mad_ms": 0.0327,
|
||||
"min_ms": 4.3314,
|
||||
"max_ms": 11.9118,
|
||||
"cpu_user_ms_median": 3.003,
|
||||
"cpu_sys_ms_median": 1.0625,
|
||||
"max_rss_kb_median": 17348
|
||||
},
|
||||
{
|
||||
"name": "table-sort",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/65-table-sort-stdlib.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/65-table-sort-stdlib.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 2.3841,
|
||||
"p95_ms": 2.461,
|
||||
"mad_ms": 0.0276,
|
||||
"min_ms": 2.2836,
|
||||
"max_ms": 2.6838,
|
||||
"cpu_user_ms_median": 0.992,
|
||||
"cpu_sys_ms_median": 0.992,
|
||||
"max_rss_kb_median": 17348
|
||||
},
|
||||
{
|
||||
"name": "hello-check",
|
||||
"kind": "check",
|
||||
"source": "examples/hello.mq.md",
|
||||
"argv": [
|
||||
"check",
|
||||
"examples/hello.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 4.4992,
|
||||
"p95_ms": 4.5766,
|
||||
"mad_ms": 0.024,
|
||||
"min_ms": 4.4585,
|
||||
"max_ms": 4.6144,
|
||||
"cpu_user_ms_median": 3.0545,
|
||||
"cpu_sys_ms_median": 1.0385,
|
||||
"max_rss_kb_median": 17348
|
||||
},
|
||||
{
|
||||
"name": "bytecode-vm",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/63-memory-bytecode-vm.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/63-memory-bytecode-vm.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 2.3718,
|
||||
"p95_ms": 2.4136,
|
||||
"mad_ms": 0.0312,
|
||||
"min_ms": 2.3153,
|
||||
"max_ms": 2.4141,
|
||||
"cpu_user_ms_median": 1.119,
|
||||
"cpu_sys_ms_median": 1.119,
|
||||
"max_rss_kb_median": 17348
|
||||
},
|
||||
{
|
||||
"name": "stdlib-print-demo",
|
||||
"kind": "run",
|
||||
"source": "stdlib/demo.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"stdlib/demo.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 10.9128,
|
||||
"p95_ms": 12.0283,
|
||||
"mad_ms": 0.0445,
|
||||
"min_ms": 10.8624,
|
||||
"max_ms": 12.0607,
|
||||
"cpu_user_ms_median": 7.8275,
|
||||
"cpu_sys_ms_median": 2.9985,
|
||||
"max_rss_kb_median": 17348
|
||||
},
|
||||
{
|
||||
"name": "factorial-run",
|
||||
"kind": "call",
|
||||
"source": "examples/factorial.mq.md",
|
||||
"argv": [
|
||||
"call",
|
||||
"examples/factorial.mq.md",
|
||||
"数学.阶乘",
|
||||
"--arg",
|
||||
"n=5",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 2.3768,
|
||||
"p95_ms": 2.4606,
|
||||
"mad_ms": 0.041,
|
||||
"min_ms": 2.2335,
|
||||
"max_ms": 2.492,
|
||||
"cpu_user_ms_median": 0.868,
|
||||
"cpu_sys_ms_median": 0.9075,
|
||||
"max_rss_kb_median": 17348
|
||||
},
|
||||
{
|
||||
"name": "matrix-multiply",
|
||||
"kind": "run",
|
||||
"source": "tests/conformance/semantics/70-matrix-multiply.mq.md",
|
||||
"argv": [
|
||||
"run",
|
||||
"tests/conformance/semantics/70-matrix-multiply.mq.md",
|
||||
"--json"
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"warmups": 5,
|
||||
"samples": 20,
|
||||
"order_seed": "6a94f839b1b21291",
|
||||
"median_ms": 2.356,
|
||||
"p95_ms": 2.4275,
|
||||
"mad_ms": 0.0471,
|
||||
"min_ms": 2.2738,
|
||||
"max_ms": 2.429,
|
||||
"cpu_user_ms_median": 0.9515,
|
||||
"cpu_sys_ms_median": 0.9825,
|
||||
"max_rss_kb_median": 17348
|
||||
}
|
||||
],
|
||||
"correctness": "VERIFIED",
|
||||
"status": "MEASURED",
|
||||
"baseline_comparison": []
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
# G4 performance baseline — Woodpecker agent 1
|
||||
|
||||
Status: **MEASURED (provisional baseline); designated fixed agent, shared host
|
||||
caveat.**
|
||||
Date: 2026-11-21. Code SHA:
|
||||
`1199c88e3e461f0443f99c5bf528ddd6a24aa2c6` (CI-only commit; Rust code equals
|
||||
`a9cc1729e1f6344e1c7a4bd58bc9e4acff4efd9e`).
|
||||
|
||||
This is the remote Woodpecker G4 evidence for the designated performance
|
||||
node. The repository has one Woodpecker agent (`agent_id 1`); it is the fixed
|
||||
node. It is **not** a physically isolated performance host: it is a Docker
|
||||
container on a shared machine, the CPU governor is `powersave`, and the host
|
||||
load average was about 5.7–6.1 during both runs. The numbers are valid only for
|
||||
same-`platform_id` regression tracking; do not compare them with another
|
||||
machine, another container, QEMU/TCG, or a future dedicated node.
|
||||
|
||||
## Fixed node and recipe
|
||||
|
||||
```text
|
||||
platform_id woodpecker-agent-1
|
||||
runner_kind Woodpecker Docker agent (agent_id 1)
|
||||
container hostname c9301248e4c7
|
||||
kernel 7.0.0-31-generic
|
||||
arch x86_64
|
||||
CPU 12th Gen Intel(R) Core(TM) i7-12700H
|
||||
cpuset (allowed) 0-19
|
||||
pinned CPU 0 (taskset -c 0)
|
||||
governor powersave
|
||||
source_sha 1199c88e3e461f0443f99c5bf528ddd6a24aa2c6
|
||||
binary sha256:15ead6261c7500eb3fbcb7c9f60b76abf85b4006883dd23850f74a43af5b18c0
|
||||
Cargo.lock sha256:5b9754a3ff7f56ca0158d70772365f02dcbfbc1ebbd97a55ee09d441c2d2af01
|
||||
rustc rustc 1.100.0 (0ef1bc386 2026-11-09)
|
||||
recipe 5 warmups + 20 samples per workload, release profile,
|
||||
SHA-derived interleaved order, 9 workloads
|
||||
status MEASURED
|
||||
correctness VERIFIED for all 9 workloads
|
||||
```
|
||||
|
||||
Both runs passed the immutable expected wire digests in
|
||||
`tools/perf/expected/`; a wrong output would have been `CORRECTNESS_FAIL` and
|
||||
its timing would not have been reported.
|
||||
|
||||
## Runs
|
||||
|
||||
| Run | Woodpecker pipeline | Start loadavg | Notes |
|
||||
|---|---|---|---|
|
||||
| 1 | [#31](https://ci.cflmy.cn/repos/1/pipeline/31) | 6.13 | first baseline |
|
||||
| 2 | [#32](https://ci.cflmy.cn/repos/1/pipeline/32) | 5.66 | repeatability check |
|
||||
|
||||
Reports (full `marqdo.perf.v2` JSON):
|
||||
|
||||
* [`2026-11-21-perf-woodpecker-agent-1-run31.json`](2026-11-21-perf-woodpecker-agent-1-run31.json)
|
||||
* [`2026-11-21-perf-woodpecker-agent-1-run32.json`](2026-11-21-perf-woodpecker-agent-1-run32.json)
|
||||
|
||||
## Results (run 31, p50 / p95 ms)
|
||||
|
||||
| Workload | p50 | p95 | MAD | RSS median (KiB) |
|
||||
|---|---:|---:|---:|---:|
|
||||
| `hello-check` | 4.8822 | 4.9401 | 0.0396 | 17676 |
|
||||
| `factorial-run` | 2.3949 | 2.4406 | 0.0244 | 17676 |
|
||||
| `allocator-free-list` | 4.5207 | 4.5971 | 0.0323 | 17676 |
|
||||
| `json-parser` | 6.6149 | 6.8380 | 0.1795 | 17676 |
|
||||
| `table-sort` | 2.3660 | 2.4724 | 0.0254 | 17676 |
|
||||
| `bytecode-vm` | 2.3634 | 2.4443 | 0.0384 | 17676 |
|
||||
| `matrix-multiply` | 2.5671 | 2.6745 | 0.0744 | 17676 |
|
||||
| `http-parser` | 4.5041 | 4.5946 | 0.0347 | 17676 |
|
||||
| `stdlib-print-demo` | 11.9034 | 12.0538 | 0.1354 | 17676 |
|
||||
|
||||
Repeatability: run 32 p50 deltas versus run 31 ranged from **-10.89% to
|
||||
+7.10%**, all below the tool's 15% investigation threshold. Peak RSS was
|
||||
17348 vs 17676 KiB (-1.9%). This supports using run 31 as the first baseline;
|
||||
it does not replace the RFC's multi-day baseline requirement.
|
||||
|
||||
## Gate classification
|
||||
|
||||
`tools/ci/release_evidence.py` records this gate as **WARN**, not PASS, for
|
||||
`1199c88`: the report is `MEASURED` with verified output, but the node is a
|
||||
shared, powersave-governed agent and this is the first same-day baseline. The
|
||||
release manifest is consequently `DO_NOT_RELEASE` only because `strict_vm` is
|
||||
still `UNVERIFIED`; see
|
||||
[`ci/1199c88e3e461f0443f99c5bf528ddd6a24aa2c6/manifest.json`](ci/1199c88e3e461f0443f99c5bf528ddd6a24aa2c6/manifest.json).
|
||||
|
||||
## How to rerun
|
||||
|
||||
```bash
|
||||
curl -X POST \
|
||||
-H "Authorization: Bearer $MARQDO_WOODPECKER_TOKEN" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"branch":"master","workflow":"performance","message":"G4 baseline"}' \
|
||||
"https://ci.cflmy.cn/api/repos/1/pipelines"
|
||||
```
|
||||
|
||||
The workflow is manual/master-only and never runs PR code. It builds one
|
||||
release binary, pins CPU 0 from the container's allowed cpuset, runs
|
||||
`runner.py --no-build --warmups 5 --samples 20 --approved-node --platform-id
|
||||
woodpecker-agent-1 --cpu-list 0`, and prints the report between
|
||||
`G4_REPORT_BEGIN`/`G4_REPORT_END` in the step log.
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"schema": "marqdo.release-gates.v1",
|
||||
"target_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6",
|
||||
"recorded_utc": "2026-11-21T12:06:06Z",
|
||||
"gates": {
|
||||
"linux": {
|
||||
"state": "PASS",
|
||||
"tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6",
|
||||
"run_url": "https://ci.cflmy.cn/repos/1/pipeline/30",
|
||||
"recorded_utc": "2026-11-21T12:06:06Z"
|
||||
},
|
||||
"windows": {
|
||||
"state": "PASS",
|
||||
"tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6",
|
||||
"run_url": "https://github.com/cflmy/marqdo-cli/actions/runs/44901625043",
|
||||
"recorded_utc": "2026-11-21T12:06:06Z"
|
||||
},
|
||||
"macos": {
|
||||
"state": "PASS",
|
||||
"tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6",
|
||||
"run_url": "https://github.com/cflmy/marqdo-cli/actions/runs/44901625043",
|
||||
"recorded_utc": "2026-11-21T12:06:06Z"
|
||||
},
|
||||
"strict_vm": {
|
||||
"state": "UNVERIFIED",
|
||||
"tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6",
|
||||
"run_url": "https://github.com/cflmy/marqdo-cli/actions/workflows/security-linux.yml",
|
||||
"recorded_utc": "2026-11-21T12:06:06Z"
|
||||
},
|
||||
"performance": {
|
||||
"state": "WARN",
|
||||
"tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6",
|
||||
"run_url": "https://ci.cflmy.cn/repos/1/pipeline/31",
|
||||
"recorded_utc": "2026-11-21T12:06:06Z"
|
||||
}
|
||||
},
|
||||
"release_decision": "DO_NOT_RELEASE",
|
||||
"reasons": [
|
||||
"strict_vm: UNVERIFIED"
|
||||
]
|
||||
}
|
||||
+9
-7
@@ -36,12 +36,14 @@ Rules enforced by the tool:
|
||||
* Development-container smoke run (2026-11-21, `--warmups 1 --samples 3
|
||||
--allow-smoke`): all nine workloads `VERIFIED`/`SMOKE`, no correctness
|
||||
failures. Stored as a smoke artifact, **not** a release baseline.
|
||||
* Remote Woodpecker G4: the repository now has a manual
|
||||
[`.woodpecker/performance.yml`](../.woodpecker/performance.yml) workflow on
|
||||
the designated fixed agent (`platform_id: woodpecker-agent-1`). It runs
|
||||
5 warmups / 20 samples, pins the first CPU in the container's
|
||||
`Cpus_allowed_list`, and records the hardware/toolchain manifest. Trigger it
|
||||
with the Woodpecker API using `"workflow": "performance"`; do not run it on
|
||||
PR code. Its report is the G4 baseline for this node.
|
||||
* Remote Woodpecker G4: **MEASURED provisional baseline** at `1199c88`.
|
||||
The manual [`.woodpecker/performance.yml`](../.woodpecker/performance.yml)
|
||||
workflow ran on the designated fixed agent (`platform_id:
|
||||
woodpecker-agent-1`): 5 warmups / 20 samples, 9/9 correctness verified,
|
||||
pinned CPU 0 from the container cpuset. Two same-day runs differed by
|
||||
-10.9%..+7.1% p50. The agent is a shared Docker container with the
|
||||
`powersave` governor and loadavg ~5.7–6.1, so the gate is **WARN**, not
|
||||
PASS, and comparisons are valid only on this `platform_id`. Evidence:
|
||||
[`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md).
|
||||
* GitHub G4: `.github/workflows/performance-linux.yml` remains available for a
|
||||
physically separate node.
|
||||
Reference in new issue
Block a user