Files
marqdo-cli/docs/ci.md
T
cflmy c8049d51bf
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/main Pipeline failed
ci(ext): ship provider worker in release archives, add Windows compile gate and evidence validator tests
- Woodpecker test: prebuild provider helper; cross-check windows target
- Woodpecker build: release-build marqdo-cli + marqdo-provider-worker, assert helper
- GitHub release candidate: package helper as sibling and smoke assert it
- release-evidence: validate committed external-extension manifest or report UNVERIFIED
- tools/ci: unit tests for verify_extension_contracts manifest validation
- docs/ci: document the CI/CD changes
2026-11-22 01:03:52 +08:00

6.0 KiB

CI/CD

Current setup

The default Docker pipeline runs on push and pull request:

test:
  rustup component add clippy rustfmt
  apt install curl/ca-certificates/python3 + Node.js 22
  cargo fmt --all -- --check
  cargo clippy --workspace --all-targets -- -D warnings
  cargo test --workspace --locked
  python3 -m unittest discover -s tools/ci -p "test_*.py"

build (depends on test):
  cargo build --release --locked -p marqdo-cli

Node.js 22 is required because the optional foreign-kernel tests use ES-module syntax; Debian bookworm's Node 18 fails those tests.

Language contract note. cargo test --workspace includes conformance fixtures that execute .mq.md source under RFC-LANG-0002. The implementation, stdlib digest and corpus are green at ffd1cc9; a red executable-source suite is a real regression, not an expected migration state. Do not "fix" it by restoring the old mapper or regenerating goldens; see STATUS.md. The pipeline definition itself (fmt/clippy/test/build, security lane, platform gaps) is unchanged.

What this pipeline does not cover

  • Strict security lane: needs a disposable VM/self-hosted runner with Bubblewrap/AppArmor namespace support. See security-vm.md and security-gate.md. The Docker pipeline is not an OS-isolation proof.
  • Windows/macOS native lanes: the GitHub-hosted .github/workflows/ci.yml matrix does exist and does run on windows-latest and macos-latest. The Woodpecker instance itself has no Windows/macOS agent, so a Woodpecker-only green pipeline cannot substitute for those native jobs. A remote Windows/macOS PASS requires the GitHub matrix (or an equivalent real native runner) on the same SHA.
  • Performance baseline (G4): needs a fixed Linux node; see performance.md. Development-container or TCG/QEMU timings are SMOKE, never release evidence.
  • Release publication/signing: no publication job is defined. The release-evidence job produces a SHA-bound manifest artifact; it does not tag or upload a release.

RFC-EXT-0001 CI/CD additions

  • The Woodpecker test lane builds marqdo-provider-worker before the workspace tests so the provider CLI tests use the real helper, and adds a cargo check --workspace --all-targets --target x86_64-pc-windows-gnu compile gate (native Windows/macOS runtime lanes stay on the GitHub matrix).
  • The Woodpecker build lane and the GitHub release-candidate lane build marqdo and marqdo-provider-worker and package both as siblings; the provider transport resolves only that sibling (no PATH fallback), so an archive without it would silently lose provider support.
  • tools/ci/test_verify_extension_contracts.py covers the external-extension evidence validator in the existing python3 -m unittest discover -s tools/ci step.
  • The release-evidence job validates docs/evidence/extension/<SHA>/manifest.json when it is committed for the target SHA and prints UNVERIFIED otherwise; generating it is the mandatory release action described in docs/releasing.md §7, not a substitute for G3/G4/native lanes.

Local reproduction

export PATH="$HOME/.cargo/bin:$PATH"
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace --locked
cargo build --release --locked -p marqdo-cli

Local API/trigger

The ignored .secrets/dev.env may contain a Woodpecker API token for local queries/triggers. The token is not needed by the pipeline itself and must never be committed.

External extension evidence (RFC-EXT-0001 PR-07/PR-08)

A release candidate must also prove that a frozen CLI binary can use extensions from physically separate repositories without modifying or rebuilding this workspace. This is a new necessary condition; it does not replace the security/platform/performance gates above.

Required inputs:

  • a release build of this commit: cargo build --release --locked -p marqdo-cli;
  • the provider helper shipped next to it for the same target: marqdo-provider-worker (the provider transport resolves only that sibling; no PATH fallback);
  • a clean checkout of the external fixture repository https://git.cflmy.cn/cflmy/marqdo-extension-conformance.

Generation and validation:

CLI="$(realpath target/release/marqdo)"
python3 tools/ci/verify_extension_contracts.py \
  --cli "$CLI" \
  --fixtures /path/to/marqdo-extension-conformance
python3 tools/ci/verify_extension_contracts.py \
  --validate docs/evidence/extension/<40-hex-source-SHA>/manifest.json

The generator hashes the CLI before and after the external suite, records the source SHA and fixture repo SHA, and refuses to write evidence for a changed binary, a dirty fixture checkout, a FAIL, or an unexplained UNSUPPORTED/SKIPPED result. The external suite itself (tests/run_all.py in that repository) uses only $MARQDO_BIN (absolute), never Cargo, and covers: pinned-Git .mq.md package, worker ABI plugin, unknown foreign kernel, provider v2 worker, and an independent Analysis/Trace consumer.

The schema is evidence/extension/schema.json; the generator is tools/ci/verify_extension_contracts.py. A manifest whose tested_sha is not the release SHA is invalid. Platform-specific results must be reported as UNSUPPORTED (for example provider worker on non-linux/x86_64); a Linux PASS never substitutes for a macOS/Windows lane.