Files
marqdo-cli/docs/performance.md
T
cflmy c11c1c858a
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
ci/woodpecker/push/main Pipeline was successful
perf: record the Woodpecker G4 measured baseline
The manual .woodpecker/performance.yml workflow ran on the designated fixed
agent (platform_id woodpecker-agent-1): 5 warmups + 20 samples, all 9
workloads correctness-verified, CPU 0 pinned from the container cpuset. Two
runs differed by -10.9%..+7.1% p50 and -1.9% RSS. The node is a shared Docker
container with the powersave governor and loadavg ~5.7-6.1, so the release gate
is WARN, not PASS, and comparisons are same-platform only.

Store both marqdo.perf.v2 reports and the SHA-bound release manifest. The
manifest is DO_NOT_RELEASE only because strict_vm remains UNVERIFIED; Linux,
Windows, macOS and performance have real same-SHA results.
2026-11-21 20:07:58 +08:00

50 lines
2.4 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Performance measurements (G4)
Status: the sampler and correctness corpus are implemented; **no release
performance baseline exists yet**. Do not cite a development-container or
TCG/QEMU number as a release gate.
## Three separate measurements
| Class | Goal | Host |
|---|---|---|
| Language core throughput | interpreter, expressions, memory/containers/stdlib algorithms | fixed G4 Linux node |
| CLI experience | cold `check`/`run`/`inspect`, parsing, stdlib loading | fixed G4 Linux node |
| Strict overhead | probe, re-exec, verification and strict run time vs pure | separate isolated performance node |
The G3 security VM can report correctness, not release performance: TCG time is
dominated by emulation and is not comparable with native or KVM.
## Tool
`tools/perf/runner.py` (schema `marqdo.perf.v2`) builds one release binary,
records its SHA256/Cargo.lock/rustc/source SHA, verifies every workload's exact
wire digest **before** timing, then reports warmups/samples, p50, p95, MAD,
CPU user/sys, peak RSS and the environment manifest. See
[`tools/perf/README.md`](../tools/perf/README.md).
Rules enforced by the tool:
* a wrong workload output is `CORRECTNESS_FAIL`; its timing is never reported;
* fewer than 5 warmups / 20 samples is `SMOKE`, not release evidence;
* `--approved-node` is required for `MEASURED`;
* a baseline from another `platform_id` or profile is refused;
* p50 +15% / RSS +20% are investigation thresholds, not automatic rollbacks.
## Current evidence
* Development-container smoke run (2026-11-21, `--warmups 1 --samples 3
--allow-smoke`): all nine workloads `VERIFIED`/`SMOKE`, no correctness
failures. Stored as a smoke artifact, **not** a release baseline.
* Remote Woodpecker G4: **MEASURED provisional baseline** at `1199c88`.
The manual [`.woodpecker/performance.yml`](../.woodpecker/performance.yml)
workflow ran on the designated fixed agent (`platform_id:
woodpecker-agent-1`): 5 warmups / 20 samples, 9/9 correctness verified,
pinned CPU 0 from the container cpuset. Two same-day runs differed by
-10.9%..+7.1% p50. The agent is a shared Docker container with the
`powersave` governor and loadavg ~5.7–6.1, so the gate is **WARN**, not
PASS, and comparisons are valid only on this `platform_id`. Evidence:
[`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md).
* GitHub G4: `.github/workflows/performance-linux.yml` remains available for a
physically separate node.