Files
marqdo-cli/tools/perf/README.md
T
cflmy d2267e912a
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was canceled
perf: add correctness-verified marqdo.perf.v2 sampler
Replace the three-run baseline profile with tools/perf/runner.py: one fixed
release binary, per-child rusage via posix_spawn/wait4, 5 warmups and 20
samples by default, SHA-derived interleaved order, p50/p95/MAD, CPU user/sys
and peak RSS. Every warmup output is normalized and compared with a committed
wire digest before timing; a mismatch is CORRECTNESS_FAIL, never a fast
sample. Short or unapproved-host runs are SMOKE, and baselines from another
platform_id/profile are refused.

Add workloads.json, expected digests for nine existing workloads, a dedicated
fixed-node workflow (manual, self-hosted, requires --cpu-list), and
docs/performance.md. bench.py stays as a compatibility shim. This is step 5 of
the CI/security RFC (B08).
2026-11-21 17:28:05 +08:00

2.1 KiB

Performance sampler (marqdo.perf.v2)

runner.py measures a fixed release binary and refuses to report timings for a workload whose wire output did not match the committed expected digest.

Rules

  • Build one release binary with cargo build --release --locked -p marqdo-cli and keep it fixed for the whole run. The report records its SHA256, the Cargo.lock digest, rustc -vV, source SHA and profile.
  • Correctness comes first: every warmup run is parsed and hashed. A mismatch is CORRECTNESS_FAIL; its faster wall time is never reported.
  • Release evidence needs at least 5 warmups and 20 samples per workload with --approved-node on a dedicated, stable performance node. Shorter runs are SMOKE and must not be cited as a release gate.
  • Do not compare timings across different platform_ids, CPU models, kernels, toolchains or load conditions. --baseline refuses a different platform_id/profile.
  • Avoid TCG/QEMU for final performance numbers; use native or KVM with fixed CPU affinity (--cpu-list) and an administrator-approved governor.
  • The suggested investigation thresholds are p50 +15% and peak RSS +20%; they are not automatic rollback rules. Investigate before freezing a hard gate.

Commands

# Record/review expected wire digests after an intentional behaviour change.
python3 tools/perf/runner.py --record-expected --warmups 1 --samples 1 \
  --allow-smoke --only hello-check

# Non-release smoke run on the development host.
python3 tools/perf/runner.py --warmups 1 --samples 3 --allow-smoke \
  --platform-id local-unapproved --out /tmp/marqdo-perf-smoke.json

# Release measurement on the dedicated node.
python3 tools/perf/runner.py --warmups 5 --samples 20 --approved-node \
  --platform-id linux-x86_64-fixed-node-01 --cpu-list 2,3 \
  --out perf/2026-11-21.json

Expected digests live in expected/<workload>.json and are versioned with the source. Running --record-expected rewrites them: inspect the diff and commit it only when the output change is the intended one.

bench.py remains only as a compatibility shim.