marqdo-extension-conformance

An independent, black-box conformance repository for the Marqdo CLI extension surface. It is physically separate from marqdo-cli: this repository contains its own sources, its own copy of the public C ABI header, and never links a Rust crate, a Marqdo workspace path, or a private interpreter structure.

The suite is executed against one frozen Marqdo CLI binary supplied as an absolute path through MARQDO_BIN. It never runs cargo run, cargo build, or copies sources into the CLI workspace. The binary's SHA-256 is recorded before and after the run and must be unchanged.

The contract and exact protocol versions are in CONTRACT.md. The implementation of the language itself is not redefined here: every .mq.md file uses the single current Markdown→program mapping (RFC-LANG-0002, @exec/@knowledge heading modes).

What is exercised

Area Directory Mechanism
External .mq.md package with cross-module functions, pinned Git dependency, lock/sync, offline cache pure-lib/, consumer/ marqdo lock / sync / check / run; git dependency pinned to a full commit SHA
Callable plugin, multiple exports across different prefixes, worker transport plugin/ plugin.yaml + C ABI v1 artifact; run --plugin --allow-plugin
Foreign kernel with a completely unknown language tag kernel/ kernel.json + standalone executable speaking the foreign JSON wire; run --kernel --allow-foreign
Resource provider v2 for an unknown kind (demo.sensor), framed-JSON worker protocol provider/ provider.json (schema v2) + C ABI provider + standalone worker; CLI --provider/--allow-provider support is not present in the current frozen binary (see known gaps)
Independent CLI tool consumer of Analysis v2 / Trace v2 tool/ tool.json + stdlib-only Python consumer; tool install then the tool's own top-level command
Frozen-binary and evidence integrity tests/, evidence/, scripts/ verify_frozen_cli.py, run_all.py, record_evidence.py, JSON Schema validation

Quick start

export MARQDO_BIN=/absolute/path/to/frozen/marqdo   # required; never hardcoded
sha256sum "$MARQDO_BIN"                              # record the frozen binary

# optional: pin the expected frozen binary digest for an evidence run
export MARQDO_EXPECTED_SHA256=<64 lowercase hex>

python3 tests/verify_frozen_cli.py          # records path + sha256
./plugin/build.sh                           # cc builds the C ABI plugin fixtures
./provider/build.sh                         # cc builds the C ABI provider + worker
python3 tests/run_all.py                    # full matrix + evidence/results.json

tests/run_all.py rebuilds native fixtures when needed, runs the positive and negative suites, verifies the CLI digest did not change, regenerates evidence/results.json, and validates it against evidence/extension-evidence.schema.json.

Each suite can also be run alone:

python3 tests/test_positive.py
python3 tests/test_negative.py

Exit codes: 0 when every mandatory case passes (or is honestly UNSUPPORTED); 1 when a mandatory case fails. Cases marked known_gap: true are real divergences from the published contract in the current frozen binary; they are reported as FAIL in the matrix and evidence, and only --strict makes them affect the exit code. They are documented in CONTRACT.md §7.

Isolation and safety

  • A C ABI plugin is not a sandbox. The fixture uses the worker transport so the native library runs in a child process, but that child still runs as the current user. --mode strict is expected to refuse external execution and is tested as a negative case.
  • A foreign kernel is a child process with the host user's privileges. It is not a sandbox either.
  • The capability/authorization model is never bypassed: every effectful extension call needs its explicit CLI grant (--allow-plugin, --allow-foreign, --allow-provider once available).
  • check is expected to be side-effect free: the plugin fixture contains a C constructor that writes a marker file whenever the shared library is loaded, and a test asserts the marker is absent after check and present after run.

Reproducibility notes

  • Native artifacts are rebuilt per machine; their SHA-256 is written into the generated build/ manifests and recorded in the evidence file.
  • The Git dependency is not a floating branch: tests create a temporary local Git repository (git init + commit), take the full 40-hex commit SHA, and put file:// URL + full rev into the generated consumer manifest. Setting MARQDO_FIXTURE_GIT_URL / MARQDO_FIXTURE_GIT_REV overrides this to demonstrate a remote URL with the same full-SHA pinning.
  • evidence/results.json is machine-generated; never hand-edit or hand-claim a PASS.

Known CLI gaps in the current frozen binary

These are recorded as FAIL (with known_gap: true) or UNSUPPORTED in evidence/results.json, with exact command, exit code, stderr and the expected contract. See CONTRACT.md §7.

  1. check --plugin does not statically reject an unknown plugin export; the document passes check and only fails at runtime with symbol.unknown.
  2. check --plugin accepts positional plugin arguments; the runtime rejects them with runtime.type. The contract requires support or a static rejection.
  3. The frozen CLI has no --provider / --allow-provider flags, so the external Resource Provider cannot be activated through the normal CLI. Its implementation (C ABI v1 + ADR-0035 framed-JSON worker) is complete and self-tested here; the CLI cases are UNSUPPORTED with a precise repro.

License

The test fixtures and scripts in this repository are provided under the MIT license (see LICENSE). The copied public header include/marqdo_plugin_abi_v1.h is the canonical frozen Marqdo Plugin ABI v1 header and is reproduced verbatim for third-party interoperability.

S
Description
Independent black-box conformance fixtures for the Marqdo CLI extension surface (RFC-EXT-0001)
Readme MIT
91 KiB
0 Stars 1 Watchers 0 Forks
Languages
C 49.4%
Python 44.5%
Shell 6.1%