Files
marqdo-cli/docs/ci.md
T
cflmy c8049d51bf
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/main Pipeline failed
ci(ext): ship provider worker in release archives, add Windows compile gate and evidence validator tests
- Woodpecker test: prebuild provider helper; cross-check windows target
- Woodpecker build: release-build marqdo-cli + marqdo-provider-worker, assert helper
- GitHub release candidate: package helper as sibling and smoke assert it
- release-evidence: validate committed external-extension manifest or report UNVERIFIED
- tools/ci: unit tests for verify_extension_contracts manifest validation
- docs/ci: document the CI/CD changes
2026-11-22 01:03:52 +08:00

136 lines
6.0 KiB
Markdown

# CI/CD
## Current setup
- Git remote: `https://git.cflmy.cn/cflmy/marqdo-cli.git`
- Woodpecker: `https://ci.cflmy.cn`
- Pipeline definition: [`.woodpecker/main.yml`](../.woodpecker/main.yml)
- Current hardening plan/RFC:
[`MARQDO_CI_SECURITY_PLATFORM_PERFORMANCE_RFC_20261121.md`](MARQDO_CI_SECURITY_PLATFORM_PERFORMANCE_RFC_20261121.md)
- Credentials are local-only and ignored; see
[development-credentials.md](development-credentials.md). Do not put tokens in
pipeline files or commit messages.
The default Docker pipeline runs on push and pull request:
```text
test:
rustup component add clippy rustfmt
apt install curl/ca-certificates/python3 + Node.js 22
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace --locked
python3 -m unittest discover -s tools/ci -p "test_*.py"
build (depends on test):
cargo build --release --locked -p marqdo-cli
```
Node.js 22 is required because the optional foreign-kernel tests use ES-module
syntax; Debian bookworm's Node 18 fails those tests.
> **Language contract note.** `cargo test --workspace` includes conformance
> fixtures that execute `.mq.md` source under
> [RFC-LANG-0002](architecture/RFC-LANG-0002-unified-semantics.md). The
> implementation, stdlib digest and corpus are green at `ffd1cc9`; a red
> executable-source suite is a real regression, not an expected migration
> state. Do not "fix" it by restoring the old mapper or regenerating goldens;
> see [`STATUS.md`](STATUS.md). The pipeline definition itself
> (fmt/clippy/test/build, security lane, platform gaps) is unchanged.
## What this pipeline does not cover
- **Strict security lane**: needs a disposable VM/self-hosted runner with
Bubblewrap/AppArmor namespace support. See [security-vm.md](security-vm.md)
and [security-gate.md](security-gate.md). The Docker pipeline is not an
OS-isolation proof.
- **Windows/macOS native lanes**: the GitHub-hosted
`.github/workflows/ci.yml` matrix **does exist and does run** on
`windows-latest` and `macos-latest`. The Woodpecker instance itself has no
Windows/macOS agent, so a Woodpecker-only green pipeline cannot substitute
for those native jobs. A remote Windows/macOS PASS requires the GitHub
matrix (or an equivalent real native runner) on the same SHA.
- **Performance baseline (G4)**: needs a fixed Linux node; see
[performance.md](performance.md). Development-container or TCG/QEMU timings
are `SMOKE`, never release evidence.
- **Release publication/signing**: no publication job is defined. The
`release-evidence` job produces a SHA-bound manifest artifact; it does not
tag or upload a release.
## RFC-EXT-0001 CI/CD additions
* The Woodpecker **test** lane builds `marqdo-provider-worker` before the
workspace tests so the provider CLI tests use the real helper, and adds a
`cargo check --workspace --all-targets --target x86_64-pc-windows-gnu`
compile gate (native Windows/macOS runtime lanes stay on the GitHub matrix).
* The Woodpecker **build** lane and the GitHub **release-candidate** lane build
`marqdo` **and** `marqdo-provider-worker` and package both as siblings; the
provider transport resolves only that sibling (no PATH fallback), so an
archive without it would silently lose provider support.
* `tools/ci/test_verify_extension_contracts.py` covers the external-extension
evidence validator in the existing `python3 -m unittest discover -s tools/ci`
step.
* The release-evidence job validates
`docs/evidence/extension/<SHA>/manifest.json` when it is committed for the
target SHA and prints `UNVERIFIED` otherwise; generating it is the mandatory
release action described in `docs/releasing.md` §7, not a substitute for
G3/G4/native lanes.
## Local reproduction
```bash
export PATH="$HOME/.cargo/bin:$PATH"
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace --locked
cargo build --release --locked -p marqdo-cli
```
## Local API/trigger
The ignored `.secrets/dev.env` may contain a Woodpecker API token for local
queries/triggers. The token is not needed by the pipeline itself and must never
be committed.
## External extension evidence (RFC-EXT-0001 PR-07/PR-08)
A release candidate must also prove that a **frozen** CLI binary can use
extensions from physically separate repositories without modifying or
rebuilding this workspace. This is a new necessary condition; it does **not**
replace the security/platform/performance gates above.
Required inputs:
* a release build of this commit: `cargo build --release --locked -p marqdo-cli`;
* the provider helper shipped next to it for the same target:
`marqdo-provider-worker` (the provider transport resolves only that sibling;
no `PATH` fallback);
* a clean checkout of the external fixture repository
`https://git.cflmy.cn/cflmy/marqdo-extension-conformance`.
Generation and validation:
```bash
CLI="$(realpath target/release/marqdo)"
python3 tools/ci/verify_extension_contracts.py \
--cli "$CLI" \
--fixtures /path/to/marqdo-extension-conformance
python3 tools/ci/verify_extension_contracts.py \
--validate docs/evidence/extension/<40-hex-source-SHA>/manifest.json
```
The generator hashes the CLI before and after the external suite, records the
source SHA and fixture repo SHA, and refuses to write evidence for a changed
binary, a dirty fixture checkout, a `FAIL`, or an unexplained
`UNSUPPORTED`/`SKIPPED` result. The external suite itself
(`tests/run_all.py` in that repository) uses only `$MARQDO_BIN` (absolute),
never Cargo, and covers: pinned-Git `.mq.md` package, worker ABI plugin,
unknown foreign kernel, provider v2 worker, and an independent Analysis/Trace
consumer.
The schema is [`evidence/extension/schema.json`](evidence/extension/schema.json);
the generator is `tools/ci/verify_extension_contracts.py`. A manifest whose
`tested_sha` is not the release SHA is invalid. Platform-specific results must
be reported as `UNSUPPORTED` (for example provider worker on non-linux/x86_64);
a Linux PASS never substitutes for a macOS/Windows lane.