marqdo-extension-conformance
An independent, black-box conformance repository for the Marqdo CLI extension
surface. It is physically separate from marqdo-cli: this repository contains
its own sources, its own copy of the public C ABI header, and never links a Rust
crate, a Marqdo workspace path, or a private interpreter structure.
The suite is executed against one frozen Marqdo CLI binary supplied as an
absolute path through MARQDO_BIN. It never runs cargo run, cargo build, or
copies sources into the CLI workspace. The binary's SHA-256 is recorded before
and after the run and must be unchanged.
The contract and exact protocol versions are in CONTRACT.md.
The implementation of the language itself is not redefined here: every
.mq.md file uses the single current Markdown→program mapping (RFC-LANG-0002,
@exec/@knowledge heading modes).
What is exercised
| Area | Directory | Mechanism |
|---|---|---|
External .mq.md package with cross-module functions, pinned Git dependency, lock/sync, offline cache |
pure-lib/, consumer/ |
marqdo lock / sync / check / run; git dependency pinned to a full commit SHA |
| Callable plugin, multiple exports across different prefixes, worker transport | plugin/ |
plugin.yaml + C ABI v1 artifact; run --plugin --allow-plugin |
| Foreign kernel with a completely unknown language tag | kernel/ |
kernel.json + standalone executable speaking the foreign JSON wire; run --kernel --allow-foreign |
Resource provider v2 for an unknown kind (demo.sensor), framed-JSON worker protocol |
provider/ |
provider.json (schema v2) + C ABI provider + standalone worker; CLI --provider/--allow-provider support is not present in the current frozen binary (see known gaps) |
| Independent CLI tool consumer of Analysis v2 / Trace v2 | tool/ |
tool.json + stdlib-only Python consumer; tool install then the tool's own top-level command |
| Frozen-binary and evidence integrity | tests/, evidence/, scripts/ |
verify_frozen_cli.py, run_all.py, record_evidence.py, JSON Schema validation |
Quick start
export MARQDO_BIN=/absolute/path/to/frozen/marqdo # required; never hardcoded
sha256sum "$MARQDO_BIN" # record the frozen binary
# optional: pin the expected frozen binary digest for an evidence run
export MARQDO_EXPECTED_SHA256=<64 lowercase hex>
python3 tests/verify_frozen_cli.py # records path + sha256
./plugin/build.sh # cc builds the C ABI plugin fixtures
./provider/build.sh # cc builds the C ABI provider + worker
python3 tests/run_all.py # full matrix + evidence/results.json
tests/run_all.py rebuilds native fixtures when needed, runs the positive and
negative suites, verifies the CLI digest did not change, regenerates
evidence/results.json, and validates it against
evidence/extension-evidence.schema.json.
Each suite can also be run alone:
python3 tests/test_positive.py
python3 tests/test_negative.py
Exit codes: 0 when every mandatory case passes (or is honestly
UNSUPPORTED); 1 when a mandatory case fails. Cases marked
known_gap: true are real divergences from the published contract in the
current frozen binary; they are reported as FAIL in the matrix and evidence,
and only --strict makes them affect the exit code. They are documented in
CONTRACT.md §7.
Isolation and safety
- A C ABI plugin is not a sandbox. The fixture uses the worker transport so
the native library runs in a child process, but that child still runs as the
current user.
--mode strictis expected to refuse external execution and is tested as a negative case. - A foreign kernel is a child process with the host user's privileges. It is not a sandbox either.
- The capability/authorization model is never bypassed: every effectful
extension call needs its explicit CLI grant (
--allow-plugin,--allow-foreign,--allow-provideronce available). checkis expected to be side-effect free: the plugin fixture contains a C constructor that writes a marker file whenever the shared library is loaded, and a test asserts the marker is absent aftercheckand present afterrun.
Reproducibility notes
- Native artifacts are rebuilt per machine; their SHA-256 is written into the
generated
build/manifests and recorded in the evidence file. - The Git dependency is not a floating branch: tests create a temporary
local Git repository (
git init+ commit), take the full 40-hex commit SHA, and putfile://URL + fullrevinto the generated consumer manifest. SettingMARQDO_FIXTURE_GIT_URL/MARQDO_FIXTURE_GIT_REVoverrides this to demonstrate a remote URL with the same full-SHA pinning. evidence/results.jsonis machine-generated; never hand-edit or hand-claim aPASS.
Known CLI gaps in the current frozen binary
These are recorded as FAIL (with known_gap: true) or UNSUPPORTED in
evidence/results.json, with exact command, exit code, stderr and the expected
contract. See CONTRACT.md §7.
check --plugindoes not statically reject an unknown plugin export; the document passescheckand only fails at runtime withsymbol.unknown.check --pluginaccepts positional plugin arguments; the runtime rejects them withruntime.type. The contract requires support or a static rejection.- The frozen CLI has no
--provider/--allow-providerflags, so the external Resource Provider cannot be activated through the normal CLI. Its implementation (C ABI v1 + ADR-0035 framed-JSON worker) is complete and self-tested here; the CLI cases areUNSUPPORTEDwith a precise repro.
License
The test fixtures and scripts in this repository are provided under the MIT
license (see LICENSE). The copied public header
include/marqdo_plugin_abi_v1.h is the canonical frozen Marqdo Plugin ABI v1
header and is reproduced verbatim for third-party interoperability.