Files
marqdo-cli/docs/performance.md
T
cflmy c11c1c858a
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
ci/woodpecker/push/main Pipeline was successful
perf: record the Woodpecker G4 measured baseline
The manual .woodpecker/performance.yml workflow ran on the designated fixed
agent (platform_id woodpecker-agent-1): 5 warmups + 20 samples, all 9
workloads correctness-verified, CPU 0 pinned from the container cpuset. Two
runs differed by -10.9%..+7.1% p50 and -1.9% RSS. The node is a shared Docker
container with the powersave governor and loadavg ~5.7-6.1, so the release gate
is WARN, not PASS, and comparisons are same-platform only.

Store both marqdo.perf.v2 reports and the SHA-bound release manifest. The
manifest is DO_NOT_RELEASE only because strict_vm remains UNVERIFIED; Linux,
Windows, macOS and performance have real same-SHA results.
2026-11-21 20:07:58 +08:00

2.4 KiB
Raw Blame History

Performance measurements (G4)

Status: the sampler and correctness corpus are implemented; no release performance baseline exists yet. Do not cite a development-container or TCG/QEMU number as a release gate.

Three separate measurements

Class Goal Host
Language core throughput interpreter, expressions, memory/containers/stdlib algorithms fixed G4 Linux node
CLI experience cold check/run/inspect, parsing, stdlib loading fixed G4 Linux node
Strict overhead probe, re-exec, verification and strict run time vs pure separate isolated performance node

The G3 security VM can report correctness, not release performance: TCG time is dominated by emulation and is not comparable with native or KVM.

Tool

tools/perf/runner.py (schema marqdo.perf.v2) builds one release binary, records its SHA256/Cargo.lock/rustc/source SHA, verifies every workload's exact wire digest before timing, then reports warmups/samples, p50, p95, MAD, CPU user/sys, peak RSS and the environment manifest. See tools/perf/README.md.

Rules enforced by the tool:

  • a wrong workload output is CORRECTNESS_FAIL; its timing is never reported;
  • fewer than 5 warmups / 20 samples is SMOKE, not release evidence;
  • --approved-node is required for MEASURED;
  • a baseline from another platform_id or profile is refused;
  • p50 +15% / RSS +20% are investigation thresholds, not automatic rollbacks.

Current evidence

  • Development-container smoke run (2026-11-21, --warmups 1 --samples 3 --allow-smoke): all nine workloads VERIFIED/SMOKE, no correctness failures. Stored as a smoke artifact, not a release baseline.
  • Remote Woodpecker G4: MEASURED provisional baseline at 1199c88. The manual .woodpecker/performance.yml workflow ran on the designated fixed agent (platform_id: woodpecker-agent-1): 5 warmups / 20 samples, 9/9 correctness verified, pinned CPU 0 from the container cpuset. Two same-day runs differed by -10.9%..+7.1% p50. The agent is a shared Docker container with the powersave governor and loadavg ~5.7–6.1, so the gate is WARN, not PASS, and comparisons are valid only on this platform_id. Evidence: evidence/2026-11-21-perf-woodpecker-agent-1.md.
  • GitHub G4: .github/workflows/performance-linux.yml remains available for a physically separate node.