From c11c1c858a2aca7c98bc429f30d767900bd98579 Mon Sep 17 00:00:00 2001 From: cflmy Date: Sat, 21 Nov 2026 20:07:58 +0800 Subject: [PATCH] perf: record the Woodpecker G4 measured baseline The manual .woodpecker/performance.yml workflow ran on the designated fixed agent (platform_id woodpecker-agent-1): 5 warmups + 20 samples, all 9 workloads correctness-verified, CPU 0 pinned from the container cpuset. Two runs differed by -10.9%..+7.1% p50 and -1.9% RSS. The node is a shared Docker container with the powersave governor and loadavg ~5.7-6.1, so the release gate is WARN, not PASS, and comparisons are same-platform only. Store both marqdo.perf.v2 reports and the SHA-bound release manifest. The manifest is DO_NOT_RELEASE only because strict_vm remains UNVERIFIED; Linux, Windows, macOS and performance have real same-SHA results. --- COMPATIBILITY.md | 12 +- docs/STATUS.md | 27 +- ...6-11-21-perf-woodpecker-agent-1-run31.json | 246 ++++++++++++++++++ ...6-11-21-perf-woodpecker-agent-1-run32.json | 246 ++++++++++++++++++ .../2026-11-21-perf-woodpecker-agent-1.md | 97 +++++++ .../manifest.json | 41 +++ docs/performance.md | 16 +- 7 files changed, 660 insertions(+), 25 deletions(-) create mode 100644 docs/evidence/2026-11-21-perf-woodpecker-agent-1-run31.json create mode 100644 docs/evidence/2026-11-21-perf-woodpecker-agent-1-run32.json create mode 100644 docs/evidence/2026-11-21-perf-woodpecker-agent-1.md create mode 100644 docs/evidence/ci/1199c88e3e461f0443f99c5bf528ddd6a24aa2c6/manifest.json diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index fde00b5..7a7fd5d 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -129,10 +129,10 @@ reproduce the Windows checkout byte behavior. **DO NOT RELEASE.** The RFC-CI-0001 series fixed the platform `E0425` build break, the strict `--json` empty-stdout defect (ADR-0053) and the native -Windows/macOS portability failures. At `5fcc111` the Linux, Windows and macOS -GitHub-native jobs and Woodpecker are green, and the G3 lane PASSes locally as -an ordinary user. The release still lacks the **remote G3 security artifact** -with a matching `tested_sha` and a **`MEASURED` G4 performance report**; those -two gates remain `UNVERIFIED`. A local VM PASS is not a remote G3 proof. See -[`docs/STATUS.md`](docs/STATUS.md) §0.1 and +Windows/macOS portability failures. Linux, Windows and macOS native jobs are +green, and G4 is `MEASURED`/`WARN` on the designated Woodpecker agent +(`1199c88`, 9/9 correctness-verified, pinned CPU). The release still lacks the +**remote G3 security artifact** with a matching `tested_sha`; a local VM PASS +is not a remote G3 proof. The G4 node is a shared powersave host, so its gate +is provisional. See [`docs/STATUS.md`](docs/STATUS.md) §0.1 and [`docs/evidence/ci/README.md`](docs/evidence/ci/README.md). diff --git a/docs/STATUS.md b/docs/STATUS.md index e5d9c67..83a8cdc 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -80,9 +80,9 @@ The intermediate "text migrated, implementation pending" record is kept as [`evidence/2026-11-21-semantic-docs-migration.md`](evidence/2026-11-21-semantic-docs-migration.md) and is superseded by the green gate. -**Not claimed:** the **remote** G3 security artifact and a **MEASURED** G4 -performance report are still missing. The Linux/Windows/macOS native matrix is -green at `5fcc111`, and the G3 mechanism is PASS in a local disposable VM, but +**Not claimed:** the **remote** G3 security artifact is still missing. The +Linux/Windows/macOS native matrix is green, G4 is measured on the designated +Woodpecker agent, and the G3 mechanism is PASS in a local disposable VM, but the RFC does not promote a local VM run to a remote gate. See §0.1, §4 and [`../COMPATIBILITY.md`](../COMPATIBILITY.md). The release decision remains **DO NOT RELEASE**. @@ -115,15 +115,15 @@ Executed evidence: | Linux fmt/clippy/test/build (Woodpecker) | **PASS** | pipelines `#9`–`#12`, `#14`, `#16`, `#17`, `#21`–`#25` on the series; `#13`/`#15`/`#19` were superseded/cancelled by the next push, not failures | | Linux + Windows + macOS native matrix (GitHub) | **PASS** | [run 30](https://github.com/cflmy/marqdo-cli/actions/runs/44896671929) on `5fcc111`: `fmt + clippy + test (Linux x86_64)`, `runtime contract (windows-latest)`, `runtime contract (macos-latest)`, and schema consumers all PASS | | G3 strict isolation | **PASS (local disposable VM) / remote UNVERIFIED** | `5fcc111`, `dev` uid 1000, QEMU/TCG, 7/7 + strict JSON/inert-`@knowledge`; [`evidence/2026-11-21-strict-gc-5fcc111.md`](evidence/2026-11-21-strict-gc-5fcc111.md) | -| G4 performance | **SMOKE / UNVERIFIED** | nine workloads `VERIFIED` on the dev container, 1 warmup + 3 samples; [`evidence/2026-11-21-perf-smoke-0d7e11b.md`](evidence/2026-11-21-perf-smoke-0d7e11b.md) | +| G4 performance | **MEASURED / WARN (provisional)** | Woodpecker agent 1, `1199c88`: 5 warmups + 20 samples, 9/9 `VERIFIED`, CPU 0 pinned; two runs -10.9%..+7.1% p50; shared powersave host caveat; [`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md) | Release decision: **DO NOT RELEASE.** Linux, Windows and macOS native jobs are -green on the same SHA, and the strict isolation mechanism PASSes locally as a -normal user. The remaining release gates are the **remote** G3 artifact whose -`tested_sha` matches and a **MEASURED** G4 report on the fixed node; neither -remote claim is made here. The RFC explicitly requires the remote G3 artifact -for a release, so the local VM PASS is recorded but not promoted to a release -gate. +green on the same SHA, the strict isolation mechanism PASSes locally as a +normal user, and G4 is `MEASURED`/`WARN` on the designated Woodpecker agent. +The remaining release gate is the **remote** G3 artifact whose `tested_sha` +matches; the RFC explicitly requires it (and the local VM PASS is not promoted +to a remote claim). The perf gate is provisional because the agent is a shared +powersave host; see [`performance.md`](performance.md). The manifest contract is in [`evidence/ci/README.md`](evidence/ci/README.md); `tools/ci/release_evidence.py` @@ -488,9 +488,12 @@ user-written allocator is explicitly **not** claimed. - [`evidence/2026-11-21-strict-gc-5fcc111.md`](evidence/2026-11-21-strict-gc-5fcc111.md) — RFC-CI-0001 series G-C: local disposable VM, `dev` uid 1000, 7/7 plus the strict JSON and inert-`@knowledge` contracts at `5fcc111`; the earlier `512b9f5`/`0d7e11b` runs are historical. +- [`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md) + — G4 `MEASURED` provisional baseline on the designated Woodpecker agent + (`1199c88`): 9/9 correctness-verified, 5 warmups + 20 samples, repeatability + deltas, and the shared-host/powersave caveat. - [`evidence/2026-11-21-perf-smoke-0d7e11b.md`](evidence/2026-11-21-perf-smoke-0d7e11b.md) - — `marqdo.perf.v2` smoke: nine workloads correctness-verified, explicitly - not a G4 release baseline. + — earlier `marqdo.perf.v2` smoke; historical, not the release baseline. - [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict L1/L2 limits. diff --git a/docs/evidence/2026-11-21-perf-woodpecker-agent-1-run31.json b/docs/evidence/2026-11-21-perf-woodpecker-agent-1-run31.json new file mode 100644 index 0000000..126ec8c --- /dev/null +++ b/docs/evidence/2026-11-21-perf-woodpecker-agent-1-run31.json @@ -0,0 +1,246 @@ +{ + "schema": "marqdo.perf.v2", + "source_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6", + "platform_id": "woodpecker-agent-1", + "environment_approved": true, + "binary": { + "path": "target/release/marqdo", + "sha256": "sha256:15ead6261c7500eb3fbcb7c9f60b76abf85b4006883dd23850f74a43af5b18c0", + "profile": "release", + "cargo_lock_sha256": "sha256:5b9754a3ff7f56ca0158d70772365f02dcbfbc1ebbd97a55ee09d441c2d2af01", + "rustc": "rustc 1.100.0 (0ef1bc386 2026-11-09)\nbinary: rustc\ncommit-hash: 0ef1bc386059596dada8fb65b50425aab2ea637a\ncommit-date: 2026-11-09\nhost: x86_64-unknown-linux-gnu\nrelease: 1.100.0\nLLVM version: 23.1.5", + "cargo": "cargo 1.100.0 (3acc79c0c 2026-11-05)" + }, + "environment": { + "uname": "Linux-7.0.0-31-generic-x86_64-with-glibc2.36", + "kernel": "7.0.0-31-generic", + "machine": "x86_64", + "cpu_model": "12th Gen Intel(R) Core(TM) i7-12700H", + "governor": "powersave", + "loadavg_start": "6.13 2.82 1.58 1/2879 2892", + "cpu_list": "0", + "python": "3.11.2", + "loadavg_end": "6.13 2.82 1.58 1/2879 2892" + }, + "recipe": { + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "timeout_seconds": 120.0 + }, + "workloads": [ + { + "name": "json-parser", + "kind": "run", + "source": "tests/conformance/semantics/62-json-array-parser.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/62-json-array-parser.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 6.6149, + "p95_ms": 6.838, + "mad_ms": 0.1795, + "min_ms": 5.7414, + "max_ms": 7.786, + "cpu_user_ms_median": 3.945, + "cpu_sys_ms_median": 1.8365, + "max_rss_kb_median": 17676 + }, + { + "name": "allocator-free-list", + "kind": "run", + "source": "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 4.5207, + "p95_ms": 4.5971, + "mad_ms": 0.0323, + "min_ms": 4.4644, + "max_ms": 4.6091, + "cpu_user_ms_median": 2.322, + "cpu_sys_ms_median": 1.5715, + "max_rss_kb_median": 17676 + }, + { + "name": "http-parser", + "kind": "run", + "source": "tests/conformance/semantics/71-http-parser.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/71-http-parser.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 4.5041, + "p95_ms": 4.5946, + "mad_ms": 0.0347, + "min_ms": 4.3909, + "max_ms": 4.6033, + "cpu_user_ms_median": 3.0935, + "cpu_sys_ms_median": 1.031, + "max_rss_kb_median": 17676 + }, + { + "name": "table-sort", + "kind": "run", + "source": "tests/conformance/semantics/65-table-sort-stdlib.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/65-table-sort-stdlib.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 2.366, + "p95_ms": 2.4724, + "mad_ms": 0.0254, + "min_ms": 2.3177, + "max_ms": 2.4961, + "cpu_user_ms_median": 0.9655, + "cpu_sys_ms_median": 0.9815, + "max_rss_kb_median": 17676 + }, + { + "name": "hello-check", + "kind": "check", + "source": "examples/hello.mq.md", + "argv": [ + "check", + "examples/hello.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 4.8822, + "p95_ms": 4.9401, + "mad_ms": 0.0396, + "min_ms": 4.4905, + "max_ms": 4.9744, + "cpu_user_ms_median": 2.067, + "cpu_sys_ms_median": 2.014, + "max_rss_kb_median": 17676 + }, + { + "name": "bytecode-vm", + "kind": "run", + "source": "tests/conformance/semantics/63-memory-bytecode-vm.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/63-memory-bytecode-vm.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 2.3634, + "p95_ms": 2.4443, + "mad_ms": 0.0384, + "min_ms": 2.2816, + "max_ms": 3.4082, + "cpu_user_ms_median": 1.1155, + "cpu_sys_ms_median": 1.073, + "max_rss_kb_median": 17676 + }, + { + "name": "stdlib-print-demo", + "kind": "run", + "source": "stdlib/demo.mq.md", + "argv": [ + "run", + "stdlib/demo.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 11.9034, + "p95_ms": 12.0538, + "mad_ms": 0.1354, + "min_ms": 10.8367, + "max_ms": 12.095, + "cpu_user_ms_median": 8.1425, + "cpu_sys_ms_median": 2.826, + "max_rss_kb_median": 17676 + }, + { + "name": "factorial-run", + "kind": "call", + "source": "examples/factorial.mq.md", + "argv": [ + "call", + "examples/factorial.mq.md", + "数学.阶乘", + "--arg", + "n=5", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 2.3949, + "p95_ms": 2.4406, + "mad_ms": 0.0244, + "min_ms": 2.3354, + "max_ms": 2.4659, + "cpu_user_ms_median": 0.0, + "cpu_sys_ms_median": 1.743, + "max_rss_kb_median": 17676 + }, + { + "name": "matrix-multiply", + "kind": "run", + "source": "tests/conformance/semantics/70-matrix-multiply.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/70-matrix-multiply.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 2.5671, + "p95_ms": 2.6745, + "mad_ms": 0.0744, + "min_ms": 2.4391, + "max_ms": 2.7945, + "cpu_user_ms_median": 0.48, + "cpu_sys_ms_median": 1.4085, + "max_rss_kb_median": 17676 + } + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "baseline_comparison": [] +} diff --git a/docs/evidence/2026-11-21-perf-woodpecker-agent-1-run32.json b/docs/evidence/2026-11-21-perf-woodpecker-agent-1-run32.json new file mode 100644 index 0000000..93cef4b --- /dev/null +++ b/docs/evidence/2026-11-21-perf-woodpecker-agent-1-run32.json @@ -0,0 +1,246 @@ +{ + "schema": "marqdo.perf.v2", + "source_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6", + "platform_id": "woodpecker-agent-1", + "environment_approved": true, + "binary": { + "path": "target/release/marqdo", + "sha256": "sha256:15ead6261c7500eb3fbcb7c9f60b76abf85b4006883dd23850f74a43af5b18c0", + "profile": "release", + "cargo_lock_sha256": "sha256:5b9754a3ff7f56ca0158d70772365f02dcbfbc1ebbd97a55ee09d441c2d2af01", + "rustc": "rustc 1.100.0 (0ef1bc386 2026-11-09)\nbinary: rustc\ncommit-hash: 0ef1bc386059596dada8fb65b50425aab2ea637a\ncommit-date: 2026-11-09\nhost: x86_64-unknown-linux-gnu\nrelease: 1.100.0\nLLVM version: 23.1.5", + "cargo": "cargo 1.100.0 (3acc79c0c 2026-11-05)" + }, + "environment": { + "uname": "Linux-7.0.0-31-generic-x86_64-with-glibc2.36", + "kernel": "7.0.0-31-generic", + "machine": "x86_64", + "cpu_model": "12th Gen Intel(R) Core(TM) i7-12700H", + "governor": "powersave", + "loadavg_start": "5.66 2.95 1.78 1/2879 2892", + "cpu_list": "0", + "python": "3.11.2", + "loadavg_end": "5.66 2.95 1.78 2/2879 2892" + }, + "recipe": { + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "timeout_seconds": 120.0 + }, + "workloads": [ + { + "name": "json-parser", + "kind": "run", + "source": "tests/conformance/semantics/62-json-array-parser.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/62-json-array-parser.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 5.8948, + "p95_ms": 6.8515, + "mad_ms": 0.0947, + "min_ms": 5.7445, + "max_ms": 6.916, + "cpu_user_ms_median": 3.449, + "cpu_sys_ms_median": 2.215, + "max_rss_kb_median": 17348 + }, + { + "name": "allocator-free-list", + "kind": "run", + "source": "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 4.8416, + "p95_ms": 4.8886, + "mad_ms": 0.0191, + "min_ms": 4.7903, + "max_ms": 4.8906, + "cpu_user_ms_median": 2.8555, + "cpu_sys_ms_median": 0.9645, + "max_rss_kb_median": 17348 + }, + { + "name": "http-parser", + "kind": "run", + "source": "tests/conformance/semantics/71-http-parser.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/71-http-parser.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 4.5, + "p95_ms": 4.6073, + "mad_ms": 0.0327, + "min_ms": 4.3314, + "max_ms": 11.9118, + "cpu_user_ms_median": 3.003, + "cpu_sys_ms_median": 1.0625, + "max_rss_kb_median": 17348 + }, + { + "name": "table-sort", + "kind": "run", + "source": "tests/conformance/semantics/65-table-sort-stdlib.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/65-table-sort-stdlib.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 2.3841, + "p95_ms": 2.461, + "mad_ms": 0.0276, + "min_ms": 2.2836, + "max_ms": 2.6838, + "cpu_user_ms_median": 0.992, + "cpu_sys_ms_median": 0.992, + "max_rss_kb_median": 17348 + }, + { + "name": "hello-check", + "kind": "check", + "source": "examples/hello.mq.md", + "argv": [ + "check", + "examples/hello.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 4.4992, + "p95_ms": 4.5766, + "mad_ms": 0.024, + "min_ms": 4.4585, + "max_ms": 4.6144, + "cpu_user_ms_median": 3.0545, + "cpu_sys_ms_median": 1.0385, + "max_rss_kb_median": 17348 + }, + { + "name": "bytecode-vm", + "kind": "run", + "source": "tests/conformance/semantics/63-memory-bytecode-vm.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/63-memory-bytecode-vm.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 2.3718, + "p95_ms": 2.4136, + "mad_ms": 0.0312, + "min_ms": 2.3153, + "max_ms": 2.4141, + "cpu_user_ms_median": 1.119, + "cpu_sys_ms_median": 1.119, + "max_rss_kb_median": 17348 + }, + { + "name": "stdlib-print-demo", + "kind": "run", + "source": "stdlib/demo.mq.md", + "argv": [ + "run", + "stdlib/demo.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 10.9128, + "p95_ms": 12.0283, + "mad_ms": 0.0445, + "min_ms": 10.8624, + "max_ms": 12.0607, + "cpu_user_ms_median": 7.8275, + "cpu_sys_ms_median": 2.9985, + "max_rss_kb_median": 17348 + }, + { + "name": "factorial-run", + "kind": "call", + "source": "examples/factorial.mq.md", + "argv": [ + "call", + "examples/factorial.mq.md", + "数学.阶乘", + "--arg", + "n=5", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 2.3768, + "p95_ms": 2.4606, + "mad_ms": 0.041, + "min_ms": 2.2335, + "max_ms": 2.492, + "cpu_user_ms_median": 0.868, + "cpu_sys_ms_median": 0.9075, + "max_rss_kb_median": 17348 + }, + { + "name": "matrix-multiply", + "kind": "run", + "source": "tests/conformance/semantics/70-matrix-multiply.mq.md", + "argv": [ + "run", + "tests/conformance/semantics/70-matrix-multiply.mq.md", + "--json" + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "warmups": 5, + "samples": 20, + "order_seed": "6a94f839b1b21291", + "median_ms": 2.356, + "p95_ms": 2.4275, + "mad_ms": 0.0471, + "min_ms": 2.2738, + "max_ms": 2.429, + "cpu_user_ms_median": 0.9515, + "cpu_sys_ms_median": 0.9825, + "max_rss_kb_median": 17348 + } + ], + "correctness": "VERIFIED", + "status": "MEASURED", + "baseline_comparison": [] +} diff --git a/docs/evidence/2026-11-21-perf-woodpecker-agent-1.md b/docs/evidence/2026-11-21-perf-woodpecker-agent-1.md new file mode 100644 index 0000000..669dfe6 --- /dev/null +++ b/docs/evidence/2026-11-21-perf-woodpecker-agent-1.md @@ -0,0 +1,97 @@ +# G4 performance baseline — Woodpecker agent 1 + +Status: **MEASURED (provisional baseline); designated fixed agent, shared host +caveat.** +Date: 2026-11-21. Code SHA: +`1199c88e3e461f0443f99c5bf528ddd6a24aa2c6` (CI-only commit; Rust code equals +`a9cc1729e1f6344e1c7a4bd58bc9e4acff4efd9e`). + +This is the remote Woodpecker G4 evidence for the designated performance +node. The repository has one Woodpecker agent (`agent_id 1`); it is the fixed +node. It is **not** a physically isolated performance host: it is a Docker +container on a shared machine, the CPU governor is `powersave`, and the host +load average was about 5.7–6.1 during both runs. The numbers are valid only for +same-`platform_id` regression tracking; do not compare them with another +machine, another container, QEMU/TCG, or a future dedicated node. + +## Fixed node and recipe + +```text +platform_id woodpecker-agent-1 +runner_kind Woodpecker Docker agent (agent_id 1) +container hostname c9301248e4c7 +kernel 7.0.0-31-generic +arch x86_64 +CPU 12th Gen Intel(R) Core(TM) i7-12700H +cpuset (allowed) 0-19 +pinned CPU 0 (taskset -c 0) +governor powersave +source_sha 1199c88e3e461f0443f99c5bf528ddd6a24aa2c6 +binary sha256:15ead6261c7500eb3fbcb7c9f60b76abf85b4006883dd23850f74a43af5b18c0 +Cargo.lock sha256:5b9754a3ff7f56ca0158d70772365f02dcbfbc1ebbd97a55ee09d441c2d2af01 +rustc rustc 1.100.0 (0ef1bc386 2026-11-09) +recipe 5 warmups + 20 samples per workload, release profile, + SHA-derived interleaved order, 9 workloads +status MEASURED +correctness VERIFIED for all 9 workloads +``` + +Both runs passed the immutable expected wire digests in +`tools/perf/expected/`; a wrong output would have been `CORRECTNESS_FAIL` and +its timing would not have been reported. + +## Runs + +| Run | Woodpecker pipeline | Start loadavg | Notes | +|---|---|---|---| +| 1 | [#31](https://ci.cflmy.cn/repos/1/pipeline/31) | 6.13 | first baseline | +| 2 | [#32](https://ci.cflmy.cn/repos/1/pipeline/32) | 5.66 | repeatability check | + +Reports (full `marqdo.perf.v2` JSON): + +* [`2026-11-21-perf-woodpecker-agent-1-run31.json`](2026-11-21-perf-woodpecker-agent-1-run31.json) +* [`2026-11-21-perf-woodpecker-agent-1-run32.json`](2026-11-21-perf-woodpecker-agent-1-run32.json) + +## Results (run 31, p50 / p95 ms) + +| Workload | p50 | p95 | MAD | RSS median (KiB) | +|---|---:|---:|---:|---:| +| `hello-check` | 4.8822 | 4.9401 | 0.0396 | 17676 | +| `factorial-run` | 2.3949 | 2.4406 | 0.0244 | 17676 | +| `allocator-free-list` | 4.5207 | 4.5971 | 0.0323 | 17676 | +| `json-parser` | 6.6149 | 6.8380 | 0.1795 | 17676 | +| `table-sort` | 2.3660 | 2.4724 | 0.0254 | 17676 | +| `bytecode-vm` | 2.3634 | 2.4443 | 0.0384 | 17676 | +| `matrix-multiply` | 2.5671 | 2.6745 | 0.0744 | 17676 | +| `http-parser` | 4.5041 | 4.5946 | 0.0347 | 17676 | +| `stdlib-print-demo` | 11.9034 | 12.0538 | 0.1354 | 17676 | + +Repeatability: run 32 p50 deltas versus run 31 ranged from **-10.89% to ++7.10%**, all below the tool's 15% investigation threshold. Peak RSS was +17348 vs 17676 KiB (-1.9%). This supports using run 31 as the first baseline; +it does not replace the RFC's multi-day baseline requirement. + +## Gate classification + +`tools/ci/release_evidence.py` records this gate as **WARN**, not PASS, for +`1199c88`: the report is `MEASURED` with verified output, but the node is a +shared, powersave-governed agent and this is the first same-day baseline. The +release manifest is consequently `DO_NOT_RELEASE` only because `strict_vm` is +still `UNVERIFIED`; see +[`ci/1199c88e3e461f0443f99c5bf528ddd6a24aa2c6/manifest.json`](ci/1199c88e3e461f0443f99c5bf528ddd6a24aa2c6/manifest.json). + +## How to rerun + +```bash +curl -X POST \ + -H "Authorization: Bearer $MARQDO_WOODPECKER_TOKEN" \ + -H 'Content-Type: application/json' \ + -d '{"branch":"master","workflow":"performance","message":"G4 baseline"}' \ + "https://ci.cflmy.cn/api/repos/1/pipelines" +``` + +The workflow is manual/master-only and never runs PR code. It builds one +release binary, pins CPU 0 from the container's allowed cpuset, runs +`runner.py --no-build --warmups 5 --samples 20 --approved-node --platform-id +woodpecker-agent-1 --cpu-list 0`, and prints the report between +`G4_REPORT_BEGIN`/`G4_REPORT_END` in the step log. diff --git a/docs/evidence/ci/1199c88e3e461f0443f99c5bf528ddd6a24aa2c6/manifest.json b/docs/evidence/ci/1199c88e3e461f0443f99c5bf528ddd6a24aa2c6/manifest.json new file mode 100644 index 0000000..2511e81 --- /dev/null +++ b/docs/evidence/ci/1199c88e3e461f0443f99c5bf528ddd6a24aa2c6/manifest.json @@ -0,0 +1,41 @@ +{ + "schema": "marqdo.release-gates.v1", + "target_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6", + "recorded_utc": "2026-11-21T12:06:06Z", + "gates": { + "linux": { + "state": "PASS", + "tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6", + "run_url": "https://ci.cflmy.cn/repos/1/pipeline/30", + "recorded_utc": "2026-11-21T12:06:06Z" + }, + "windows": { + "state": "PASS", + "tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6", + "run_url": "https://github.com/cflmy/marqdo-cli/actions/runs/44901625043", + "recorded_utc": "2026-11-21T12:06:06Z" + }, + "macos": { + "state": "PASS", + "tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6", + "run_url": "https://github.com/cflmy/marqdo-cli/actions/runs/44901625043", + "recorded_utc": "2026-11-21T12:06:06Z" + }, + "strict_vm": { + "state": "UNVERIFIED", + "tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6", + "run_url": "https://github.com/cflmy/marqdo-cli/actions/workflows/security-linux.yml", + "recorded_utc": "2026-11-21T12:06:06Z" + }, + "performance": { + "state": "WARN", + "tested_sha": "1199c88e3e461f0443f99c5bf528ddd6a24aa2c6", + "run_url": "https://ci.cflmy.cn/repos/1/pipeline/31", + "recorded_utc": "2026-11-21T12:06:06Z" + } + }, + "release_decision": "DO_NOT_RELEASE", + "reasons": [ + "strict_vm: UNVERIFIED" + ] +} diff --git a/docs/performance.md b/docs/performance.md index 80b108d..65aa9d5 100644 --- a/docs/performance.md +++ b/docs/performance.md @@ -36,12 +36,14 @@ Rules enforced by the tool: * Development-container smoke run (2026-11-21, `--warmups 1 --samples 3 --allow-smoke`): all nine workloads `VERIFIED`/`SMOKE`, no correctness failures. Stored as a smoke artifact, **not** a release baseline. -* Remote Woodpecker G4: the repository now has a manual - [`.woodpecker/performance.yml`](../.woodpecker/performance.yml) workflow on - the designated fixed agent (`platform_id: woodpecker-agent-1`). It runs - 5 warmups / 20 samples, pins the first CPU in the container's - `Cpus_allowed_list`, and records the hardware/toolchain manifest. Trigger it - with the Woodpecker API using `"workflow": "performance"`; do not run it on - PR code. Its report is the G4 baseline for this node. +* Remote Woodpecker G4: **MEASURED provisional baseline** at `1199c88`. + The manual [`.woodpecker/performance.yml`](../.woodpecker/performance.yml) + workflow ran on the designated fixed agent (`platform_id: + woodpecker-agent-1`): 5 warmups / 20 samples, 9/9 correctness verified, + pinned CPU 0 from the container cpuset. Two same-day runs differed by + -10.9%..+7.1% p50. The agent is a shared Docker container with the + `powersave` governor and loadavg ~5.7–6.1, so the gate is **WARN**, not + PASS, and comparisons are valid only on this `platform_id`. Evidence: + [`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md). * GitHub G4: `.github/workflows/performance-linux.yml` remains available for a physically separate node.