- Add a host-managed helper process speaking framed JSON (4-byte BE length) with ping/open/read/write/close/shutdown, max 1 MiB frames and structured error mapping (ADR-0035). - Add StdHost::register_worker_resource_provider using the same protocol, kind-conflict and name-grant checks as the in-process provider; pure/strict refuse provider kinds. - Tests spawn a Python helper and cover round-trip, grant/mode denial and missing-program failure; no parser/Value/runtime/HostOp change. - Update ADR/spec/status/README/changelog and evidence; C ABI binding remains open. Gate: fmt, clippy -D warnings, 433 tests passed.
Marqdo conformance and integration test corpus
This directory holds the P0/P1 conformance corpus required by
docs/DEVELOPMENT_BLUEPRINT.md §13/§14. The Rust harnesses live in
crates/cli/tests/ (they use the workspace libraries directly and the
marqdo binary via CARGO_BIN_EXE_marqdo); the fixtures are plain files here
so they can be rendered, diffed and reviewed like Markdown documents.
Everything here is data. Nothing in this directory modifies the repository, and no test commits, copies or rewrites files outside a throw-away temp directory.
Layout
tests/
├── conformance/
│ ├── ast/ 160 × *.mq.md + sibling *.ast.json goldens
│ └── semantics/ 73 × *.mq.md + sibling *.expected.json goldens
├── negative/
│ ├── *.mq.md 52 × compile-time hard-error fixtures + *.expected.json
│ ├── warnings/ 6 × inert-executable warning fixtures + *.expected.json
│ └── runtime/ 21 × runtime-error fixtures + *.expected.json
└── README.md this file
Rust harnesses (all in crates/cli/tests/):
| Harness | What it proves |
|---|---|
conformance_ast.rs |
The frozen marqdo.md.v1 profile parses each fixture into exactly the checked-in AST (node kinds, attributes, literals, byte spans, order). |
conformance_semantics.rs |
Each semantic fixture compiles through marqdo_frontend, is assembled into marqdo_ir::Program and executed with marqdo_runtime + marqdo_host::StdHost; returned wire value and program stdout match. |
negative.rs |
Invalid executable shapes are hard errors (never prose fallbacks), exact sorted diagnostic-code lists; warnings and runtime error codes are asserted in their own suites. |
properties.rs |
Blueprint §14.2 prose invariance: inserting/removing pure narration paragraphs never changes the executable function IR (modulo source spans) and diagnostics stay error-free. |
integration_cli.rs |
Real CLI contract: init, check, run, call, inspect, fmt, multi-module imports/cycles, lock/sync --locked, plugin inspection and explicitly trusted plugin invocation. |
Running
Use the workspace cargo cache/target as documented in AGENTS.md:
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-cli # all conformance + integration harnesses
cargo test --workspace # whole workspace (includes crate unit tests)
Individual harnesses:
cargo test -p marqdo-cli --test conformance_ast
cargo test -p marqdo-cli --test conformance_semantics
cargo test -p marqdo-cli --test negative
cargo test -p marqdo-cli --test properties
cargo test -p marqdo-cli --test integration_cli
The power/control-flow slice is pinned by:
- AST fixtures 151–160: code-span vs tight/spaced
**, multi-backtick spans, inert prose/code fences, branch/loop headers and math blocks; - semantic fixtures 37–44: right-associative power, unary-minus precedence, parentheses, float negative exponents, Strong+CodeSpan assignment, branch, loop and inert prose;
- negative fixtures 49–52: missing power/condition operands and executable nodes swallowed by a condition paragraph;
- runtime negatives 11–12: negative integer exponent and exponent budget;
crates/marqdo-frontend/src/payload.rsspan tests andcrates/cli/tests/integration_cli.rsCLI span/E2E assertions.
The addressable-memory slice (RFC-CM P1, ADR-0022) is pinned by:
- semantic fixtures 45–47: sharing a region by handle, per-region isolation,
memory.size,memory.growreturning the old count, preserved old bytes and zero-filled new pages; - runtime negatives 13–18: out-of-bounds and huge offsets, byte values outside
0..=255, index assignment on aList, growth beyond the declared maximum and negative page counts; tests/negative/runtime/15-index-assign-non-memory.mq.mdalso guards the v1 value semantics ofList(it must not become shared mutable memory);crates/marqdo-runtime/src/memory.rsunit tests (aliasing, atomic failed growth, forged/stale handles, page arithmetic) andcrates/marqdo-runtime/tests/interpreter.rshost-boundary/budget tests;crates/marqdo-value/src/wire.rsproves live handles are not transferable;examples/memory-buffer.mq.mdand theexamples_r6CLI test prove the.mq.mdsurface end to end.
Partial RFC-CM P2 (compute closure) is demonstrated, not certified, by:
- semantic fixtures 48–50: a big-endian u32 codec and bump allocator, three allocated nodes linked and traversed, and a failed allocation that leaves the allocator header unchanged. These documents use only language builtins and control flow; no Rust algorithm is added for them.
- The full P2 gate also needs a free-list allocator with reuse/coalescing,
tree/graph structures, a recoverable
Resultmodel and an independent stdlib repository. Those are not implemented.
CM-A04 structure primitives (ADR-0023) are pinned by:
- semantic fixtures 51–54: value semantics of
list.append/map.set/map.remove(inputs unchanged, keys sorted,map.has/map.get), Unicode scalar round-trip throughtext.codepoints/text.from_codepoint(including invalid scalars →None),table.columns/table.row/table.cellprojection, and a.mq.md过滤function that rebuilds a filtered GFM-style table without mutating the input or adding a Rust table algorithm; - runtime negatives 19–21:
list.appendon a Map, table row out of bounds, and a non-Textmap.getkey; crates/marqdo-runtime/src/builtins_impl.rs; no algorithm builtin such as sort/filter/join/JSON/CSV is added.
CM-A05 recoverable results and the first-fit allocator (ADR-0024) are pinned by:
- semantic fixture 55: a
.mq.mdfirst-fit/free-list allocator with 8-byte block headers, free-block reuse, block splitting, address-adjacent coalescing (a second region fills its bump space, so only the merged block can satisfy a 40-byte request), a too-large request returning{ok:false, code:"alloc.no_space", ...}, and payload writes that show the reused blocks do not overlap; - the
Resultshape is an ordinary Map; noResultvalue type, exception,try/catchor propagation syntax is added. The ADR records the success/failure field contract and the fatal-vs-recoverable boundary; - semantic fixture 56: a
Memory-backed binary search tree with 16-byte nodes, recursive insertion and recursive in-order traversal built fromlist.concat; the returned root/left/right offsets and sorted values pin node layout and traversal; - semantic fixture 57: a
.mq.mdCSV single-row splitter usingtext.codepoints,text.from_codepoint,list.appendandlist.concat; it preserves empty fields and does not pretend to support quoted or escaped CSV; - semantic fixture 58: an open-addressing hash table with linear probing,
collision handling (keys 5 and 37), value update, not-found
None, element count, and a full-tablehash.fullResult in a second region; - semantic fixture 59: a
.mq.mdUTF-8 encoder that writes 1–4 byte sequences intoMemoryfromtext.codepoints, returning the byte length and bytes; - semantic fixture 60: a validating
.mq.mdUTF-8 decoder overMemorywith explicitutf8.truncated,utf8.invalid_continuation,utf8.invalid_scalarandutf8.invalid_leadfailures; valid ASCII, 2/3/4-byte and mixed text is decoded; - semantic fixture 61: a
Memory-backed graph adjacency list with 16-byte nodes and 8-byte edges, head-inserted edges and a neighbor traversal; no Rust graph container or newValuetype; - semantic fixture 62: a recursive
.mq.mdJSON parser/formatter for nested arrays and objects, strings with standard escapes including\uXXXX, ints and floats (including exponents),true/false/null, whitespace,nextpositions, compact output andResulterrors (json.unexpected_eof,json.invalid_array,json.invalid_object,json.invalid_string,json.invalid_number,json.invalid_literal,json.unexpected_token); - semantic fixture 63: a
Memory-backed stack bytecode VM with instruction decode, PUSH/ADD/MUL/JMP/JZ/HALT, a stack region and two programs (arithmetic and conditional branch); no Rust VM or interpreter loop was added; - semantic fixture 64:
.mq.mdbig-endian 16/32/64-bit encoders/decoders and LEB128 varint encode/decode, including theu64maximum and byte-level round-trips; - semantic fixture 65: a
.mq.mdtable sort built ontable.row,table.columns,table.cell,表andlist.append; insertion sort rebuilds a new table and leaves the input table unchanged; - semantic fixture 67: a
.mq.mdtable equality join over two tables, usingtable.row/columns/cell,list.concat,list.appendand表; both input tables remain unchanged; - semantic fixture 68: a quoted/escaped CSV single-row parser using Unicode
scalar primitives;
""is a literal quote and commas inside quotes are preserved. Multi-line fields and whitespace trimming are out of scope; - semantic fixture 69: table group by a Text key column into a
Mapof value lists, usingtable.row/cell,map.has/get/setandlist.append; - semantic fixture 70: Int matrix multiply over list-of-list row-major data, covering C12's layout/iteration shape without a numeric backend;
- fixture 55 additionally has a second-free detection path: freeing an
already-free block returns
{ok:false, code:"alloc.double_free"}.
P5 TCP endpoint v1 (ADR-0034) is pinned by two host tests in
crates/marqdo-host/tests/resource_streams.rs (loopback echo with a grant;
grant/pure/strict/DNS refusal) and
crates/cli/tests/resource_streams.rs::tcp_stream_round_trips_through_mqmd_with_a_network_grant
(--allow-net 127.0.0.1).
RFC-CM CM-A09 worker provider transport (ADR-0035) is pinned by
crates/marqdo-host/tests/worker_provider.rs: a Python helper is spawned,
pinged, opened/read/written/closed through framed JSON; missing grant, pure,
strict and missing-program cases are rejected. The helper is trusted, not a
sandbox.
RFC-CM CM-A09 in-process provider v1 (ADR-0033) is pinned by the three
registered_provider_streams_* / provider_* tests in
crates/marqdo-host/tests/resource_streams.rs: a mock provider is registered
and read/written through resource.* with a project-mode name grant; missing
grant, pure/strict, bad protocol version, reserved kinds and duplicate kinds
are rejected.
RFC-CM P2 closure (C01–C06, C11, C12) is pinned by
tools/pack-stdlib.py is the CM-A10 deterministic package builder (not a test;
its descriptor is checked in under docs/evidence/).
tools/perf/bench.py is the P6 baseline harness (not a test; it is not part of
the cargo gate).
crates/cli/tests/p2_closure.rs: 21 semantic fixtures run through
run --mode pure --json, sources are scanned for foreign/plugin/import
shortcuts, and value/stdout are compared with the goldens; the stdlib table
self-test also runs in pure mode.
RFC-CM CM-A08 table stdlib (ADR-0032) is pinned by
crates/cli/tests/stdlib_table.rs, which runs stdlib/tests/table.mq.md and
checks select/filter/sort/append/set/join/group results plus original-table
immutability.
RFC-CM CM-A07 Compat-C (ADR-0031) is pinned by
crates/marqdo-runtime/tests/interpreter.rs::print_without_the_bundled_stdlib_is_a_structured_error
and the absence of any HostOp::Print/op_print reference. The semantic
harness mirrors the loader's stdlib injection for direct-runtime fixtures.
RFC-CM CM-A07 Compat-B (ADR-0030) is pinned by
crates/cli/tests/print_routing.rs:
- a print program's trace contains
resource.writeand noprinteffect; - stdlib modules are injected only for programs that call
打印/print; - all existing semantic/CLI print goldens still pass byte-identically.
RFC-CM CM-A10 bundled stdlib (ADR-0028) is pinned by
crates/cli/tests/stdlib_bundle.rs (6 tests):
- explicit
std:ioimport resolves the verified bundle and runs; - Analysis records
project.stdlib.version/digest/instance_id; - a tampered
io.mq.mdis rejected withstdlib digest mismatchbefore any module executes; - a missing manifest is a hard error;
- the bundle has no
foreign./plugin.calls; - the CLI binary hash is unchanged after running a
std:ioproject (frozen-CLI closure).
RFC-CM CM-A07 Compat-A (ADR-0027) is pinned by:
stdlib/{format,text,io}.mq.mdandstdlib/demo.mq.md;crates/cli/tests/examples_r6.rs::mqmd_stdlib_print_output_matches_builtin_print, which runs the demo and assertsio.写值and builtin打印lines are byte-identical for a nested List/Map/None value and a Table;- the demo covers Unit/None/Bool, Int, Float, Text, List, Map, Table,
Formula, Code and FunctionRef; PluginHandle formatting uses the same
observation members but needs a plugin to construct a value.
打印is routed to the stdlib by ADR-0030 Compat-B when the bundle is present; this demo still calls the explicitio.写值to compare paths.
RFC-CM CM-A06 resource streams (ADR-0025) are pinned by:
- semantic fixture 66:
.mq.mdUTF-8 text encoded intoMemory, written to astdio/stdoutresource, closed, then re-used to prove{ok:false,code:"resource.closed"}; the expectedstdoutis the decoded text; crates/marqdo-host/tests/resource_streams.rs(9): pure-mode stdout write, output-budget atomicity, strict refusal, project-file chunking/EOF, missing grant, traversal rejection and unsupported rights/kinds;crates/cli/tests/resource_streams.rs: realmarqdo runwith--allow-read data.txt, three chunked reads intoMemoryplus EOF, the same document failing withpermission.deniedwithout the grant, and a write-then-read program with--allow-write/--allow-read out.binproving the file is created only atresource.close, and a chunked.mq.mdfile copy (resource.read→resource.write) that reproducesdata.txtinout.bin;crates/marqdo-host/src/resource.rsunit tests for forged/closed handles, stdout read/file write unsupported, and chunked EOF.
Fixture discovery is directory-based and sorted by file name, so the order is deterministic on every platform. The AST and semantic suites fail loudly if the corpus shrinks below 100 / 25 fixtures, and the negative suite fails below 30 compile-time hard-error fixtures.
AST goldens and UPDATE_GOLDENS
*.ast.json files are the pretty-printed canonical AST view
(marqdo_markdown::parse(source).root), exactly as marqdo inspect --ast --json
would print it. They pin parser version, extension flags and span arithmetic.
Regenerate only for an intentional, ADR-backed profile change (the profile
is frozen in spec/markdown-profile.md):
UPDATE_GOLDENS=1 cargo test -p marqdo-cli --test conformance_ast
The update run rewrites every golden, prints how many it rewrote and passes. Re-run without the variable to verify, and review the golden diff: a golden change is a language change, not a bug fix.
A fixture without a golden is an error, and a golden without a fixture is an error (stale corpus output), so add/remove both together.
Semantic goldens
Each *.expected.json has the shape:
{
"function": "<name passed to marqdo_runtime::run>",
"args": { "<name>": <wire value> },
"value": <wire value>,
"stdout": "..."
}
argsandvalueuse the typed wire encoding frommarqdo_value::to_wire({"type":"int","decimal":"42"},{"type":"text","value":"…"}, …).stdoutis the program output captured from the host (never diagnostics).- Optional
"grants": ["foreign"]enablesforeign.call; that fixture is skipped with a printed notice whenpython3is unavailable.
All values were produced by running the implementation and then checked by
hand against the fixture source. UPDATE_SEMANTICS_GOLDENS=1 exists for
authoring convenience, but it must not be used blindly: rewriting a semantic
golden without verifying the value would defeat the purpose of the corpus.
Negative fixtures
Compile-time (tests/negative/*.mq.md):
{"codes": ["markdown.invalid_construct"]}
The expected list is the exact sorted multiset of error diagnostic codes
produced by marqdo_frontend::analyze_source. Warnings emitted by the same
document are ignored here. Every listed code is additionally asserted to have
error severity, and every fixture must produce at least one error — invalid
executable shapes never fall back to prose.
Warnings (tests/negative/warnings/*.mq.md):
{"severity": "warning", "codes": ["markdown.inert_executable"]}
These documents must compile without errors; the sorted multiset of warning
codes must match exactly. return/strong/lists/block quotes outside a
function body are inert content (warning), not hard errors.
Runtime (tests/negative/runtime/*.mq.md):
{"function": "main", "args": {}, "codes": ["runtime.division_by_zero"]}
These documents compile, then must fail at runtime with the expected stable
error code. Optional keys: "grants": ["foreign"] and "stack_limit": 24
(used for the recursion-depth fixture so the budget — not the host process
stack — is what fails).
CLI integration notes
integration_cli.rs never leaves the repository dirty:
- temporary projects are created under
std::env::temp_dir()and removed on drop; - the demo C plugin is built in a temp copy of
examples/plugin-demo(cc -shared -fPIC -I <repo>/include), and thesha256in the copiedplugin.yamlis refreshed there. The repo copy and its.so(gitignored) are never touched; - tests that need
python3orccprint a skip notice and return when the tool is unavailable.
Notes discovered while authoring (all since fixed)
The fixtures initially pinned three observations that were reported and then fixed in the implementation; the goldens below reflect the fixed behaviour:
- wire floats round-trip exactly (
serde_json/float_roundtripis enabled), so Pythonmath.pi * 9keeps28.274333882308138; - the default interpreter stack budget is 128 frames so a deeply recursive
document returns a structured
resource.exhaustedinstead of exhausting the native debug-build stack (--stackcan raise it deliberately); - an external-link call form (
> [x](https://example.com)()) is rejected withref.unresolved, the same code used for other unresolvable call targets.