Update status, compatibility, changelog, README, architecture overview,
ADR status, migration guide, contributor/CI/security/release notes and
the per-file inventory from 'migration pending' to the verified state:
- language SHA ffd1cc9: fmt/clippy clean, workspace 501 passed / 0 failed
/ 2 ignored, stdlib digest regenerated and pack-verified
- green evidence report added; the prior intermediate evidence is marked
superseded
- inventory regenerated: 1015 classified files, 0 pending
- explicit non-claims: Windows/macOS native lanes and the disposable-VM
strict lane were not re-run at this SHA, so the release decision remains
DO NOT RELEASE
Verified: cargo test -p marqdo-cli --test docs_consistency (4/4) and
git diff --check.
Marqdo conformance and integration test corpus
This directory holds the P0/P1 conformance corpus required by
RFC-LANG-0002 and docs/DEVELOPMENT_BLUEPRINT.md §13/§14. The Rust harnesses live in
crates/cli/tests/ (they use the workspace libraries directly and the
marqdo binary via CARGO_BIN_EXE_marqdo); the fixtures are plain files here
so they can be rendered, diffed and reviewed like Markdown documents.
Everything here is data. Nothing in this directory modifies the repository, and no test commits, copies or rewrites files outside a throw-away temp directory.
Language state (RFC-LANG-0002). The corpus uses the single current syntax and the executable suites are green at
ffd1cc9(501 workspace tests).conformance_astis unaffected by the semantic change (the Markdown profile is unchanged).tests/negative/warnings/01-inferred-signaturepins the optionalsem.inferred_signaturelint as enabled for a function that actually has inferred parameters.sem.no_entry_functionis arunentry-selection diagnostic, so it is pinned by the CLI integration testrun_reports_sem_no_entry_function_without_exec_main, not bycrates/cli/tests/negative.rs. Do not regenerate goldens or restore the old mapper to make a failing suite pass. See../docs/STATUS.mdand../docs/migration/README.md.
Layout
tests/
├── conformance/
│ ├── ast/ 160 × *.mq.md + sibling *.ast.json goldens
│ └── semantics/ 73 × *.mq.md + sibling *.expected.json goldens
├── negative/
│ ├── *.mq.md 47 × compile-time hard-error fixtures + *.expected.json
│ ├── warnings/ 2 × warning-model/control fixtures + *.expected.json
│ └── runtime/ 21 × runtime-error fixtures + *.expected.json
└── README.md this file
Rust harnesses (all in crates/cli/tests/):
| Harness | What it proves |
|---|---|
conformance_ast.rs |
The frozen marqdo.md.v1 profile parses each fixture into exactly the checked-in AST (node kinds, attributes, literals, byte spans, order). |
conformance_semantics.rs |
Each semantic fixture compiles through marqdo_frontend, is assembled into marqdo_ir::Program and executed with marqdo_runtime + marqdo_host::StdHost; returned wire value and program stdout match. |
negative.rs |
Current-syntax diagnostics are hard errors (never prose fallbacks): exact sorted sem.*/symbol.*/call.*/frontmatter.* code lists, the sem.inferred_signature warning model with inert-knowledge controls, and runtime error codes — each asserted in its own suite. |
properties.rs |
Blueprint §14.2 prose invariance: inserting/removing pure narration paragraphs never changes the executable function IR (modulo source spans) and diagnostics stay error-free. |
integration_cli.rs |
Real CLI contract: init, check, run, call, inspect, fmt, multi-module imports/cycles, lock/sync --locked, plugin inspection and explicitly trusted plugin invocation. |
Running
Use the workspace cargo cache/target as documented in AGENTS.md:
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-cli # all conformance + integration harnesses
cargo test --workspace # whole workspace (includes crate unit tests)
Individual harnesses:
cargo test -p marqdo-cli --test conformance_ast
cargo test -p marqdo-cli --test conformance_semantics
cargo test -p marqdo-cli --test negative
cargo test -p marqdo-cli --test properties
cargo test -p marqdo-cli --test integration_cli
The power/control-flow slice is pinned by:
- AST fixtures 151–160: code-span vs tight/spaced
**, multi-backtick spans, inert prose/code fences, branch/loop headers and math blocks; - semantic fixtures 37–44: right-associative power, unary-minus precedence, parentheses, float negative exponents, Strong+CodeSpan assignment, branch, loop and inert prose;
- negative fixtures 18–23: malformed payload statements (missing condition,
trailing colon,
end if, emptyreturn, multi-statement payload, badparamssyntax); fixtures 24–29 and 46 pin control-stack pairing, including the heading boundary that reportssem.unclosed_controlandsem.unmatched_endindependently andelseunder an openwhile; - runtime negatives 11–12: negative integer exponent and exponent budget;
crates/marqdo-frontend/src/payload.rsspan tests andcrates/cli/tests/integration_cli.rsCLI span/E2E assertions.
The addressable-memory slice (RFC-CM P1, ADR-0022) is pinned by:
- semantic fixtures 45–47: sharing a region by handle, per-region isolation,
memory.size,memory.growreturning the old count, preserved old bytes and zero-filled new pages; - runtime negatives 13–18: out-of-bounds and huge offsets, byte values outside
0..=255, index assignment on aList, growth beyond the declared maximum and negative page counts; tests/negative/runtime/15-index-assign-non-memory.mq.mdalso guards the current value semantics ofList(it must not become shared mutable memory);crates/marqdo-runtime/src/memory.rsunit tests (aliasing, atomic failed growth, forged/stale handles, page arithmetic) andcrates/marqdo-runtime/tests/interpreter.rshost-boundary/budget tests;crates/marqdo-value/src/wire.rsproves live handles are not transferable;examples/memory-buffer.mq.mdand theexamples_r6CLI test prove the.mq.mdsurface end to end.
Partial RFC-CM P2 (compute closure) is demonstrated, not certified, by:
- semantic fixtures 48–50: a big-endian u32 codec and bump allocator, three allocated nodes linked and traversed, and a failed allocation that leaves the allocator header unchanged. These documents use only language builtins and control flow; no Rust algorithm is added for them.
- The full P2 gate also needs a free-list allocator with reuse/coalescing,
tree/graph structures, a recoverable
Resultmodel and an independent stdlib repository. Those are not implemented.
CM-A04 structure primitives (ADR-0023) are pinned by:
- semantic fixtures 51–54: value semantics of
list.append/map.set/map.remove(inputs unchanged, keys sorted,map.has/map.get), Unicode scalar round-trip throughtext.codepoints/text.from_codepoint(including invalid scalars →None),table.columns/table.row/table.cellprojection, and a.mq.md过滤function that rebuilds a filtered GFM-style table without mutating the input or adding a Rust table algorithm; - runtime negatives 19–21:
list.appendon a Map, table row out of bounds, and a non-Textmap.getkey; crates/marqdo-runtime/src/builtins_impl.rs; no algorithm builtin such as sort/filter/join/JSON/CSV is added.
CM-A05 recoverable results and the first-fit allocator (ADR-0024) are pinned by:
- semantic fixture 55: a
.mq.mdfirst-fit/free-list allocator with 8-byte block headers, free-block reuse, block splitting, address-adjacent coalescing (a second region fills its bump space, so only the merged block can satisfy a 40-byte request), a too-large request returning{ok:false, code:"alloc.no_space", ...}, and payload writes that show the reused blocks do not overlap; - the
Resultshape is an ordinary Map; noResultvalue type, exception,try/catchor propagation syntax is added. The ADR records the success/failure field contract and the fatal-vs-recoverable boundary; - semantic fixture 56: a
Memory-backed binary search tree with 16-byte nodes, recursive insertion and recursive in-order traversal built fromlist.concat; the returned root/left/right offsets and sorted values pin node layout and traversal; - semantic fixture 57: a
.mq.mdCSV single-row splitter usingtext.codepoints,text.from_codepoint,list.appendandlist.concat; it preserves empty fields and does not pretend to support quoted or escaped CSV; - semantic fixture 58: an open-addressing hash table with linear probing,
collision handling (keys 5 and 37), value update, not-found
None, element count, and a full-tablehash.fullResult in a second region; - semantic fixture 59: a
.mq.mdUTF-8 encoder that writes 1–4 byte sequences intoMemoryfromtext.codepoints, returning the byte length and bytes; - semantic fixture 60: a validating
.mq.mdUTF-8 decoder overMemorywith explicitutf8.truncated,utf8.invalid_continuation,utf8.invalid_scalarandutf8.invalid_leadfailures; valid ASCII, 2/3/4-byte and mixed text is decoded; - semantic fixture 61: a
Memory-backed graph adjacency list with 16-byte nodes and 8-byte edges, head-inserted edges and a neighbor traversal; no Rust graph container or newValuetype; - semantic fixture 62: a recursive
.mq.mdJSON parser/formatter for nested arrays and objects, strings with standard escapes including\uXXXX, ints and floats (including exponents),true/false/null, whitespace,nextpositions, compact output andResulterrors (json.unexpected_eof,json.invalid_array,json.invalid_object,json.invalid_string,json.invalid_number,json.invalid_literal,json.unexpected_token); - semantic fixture 63: a
Memory-backed stack bytecode VM with instruction decode, PUSH/ADD/MUL/JMP/JZ/HALT, a stack region and two programs (arithmetic and conditional branch); no Rust VM or interpreter loop was added; - semantic fixture 64:
.mq.mdbig-endian 16/32/64-bit encoders/decoders and LEB128 varint encode/decode, including theu64maximum and byte-level round-trips; - semantic fixture 65: a
.mq.mdtable sort built ontable.row,table.columns,table.cell,表andlist.append; insertion sort rebuilds a new table and leaves the input table unchanged; - semantic fixture 67: a
.mq.mdtable equality join over two tables, usingtable.row/columns/cell,list.concat,list.appendand表; both input tables remain unchanged; - semantic fixture 68: a quoted/escaped CSV single-row parser using Unicode
scalar primitives;
""is a literal quote and commas inside quotes are preserved. Multi-line fields and whitespace trimming are out of scope; - semantic fixture 69: table group by a Text key column into a
Mapof value lists, usingtable.row/cell,map.has/get/setandlist.append; - semantic fixture 70: Int matrix multiply over list-of-list row-major data, covering C12's layout/iteration shape without a numeric backend;
- fixture 55 additionally has a second-free detection path: freeing an
already-free block returns
{ok:false, code:"alloc.double_free"}.
P5 TCP endpoint v1 (ADR-0034) is pinned by two host tests in
crates/marqdo-host/tests/resource_streams.rs (loopback echo with a grant;
grant/pure/strict/DNS refusal) and
crates/cli/tests/resource_streams.rs::tcp_stream_round_trips_through_mqmd_with_a_network_grant
(--allow-net 127.0.0.1).
RFC-CM CM-A09 C ABI worker binding (ADR-0038) is pinned by
crates/marqdo-provider-worker/tests/worker_binding.rs, which compiles a C
provider, runs it through the worker helper and the host transport, and checks
missing-library fail-closed startup.
RFC-CM CM-A09 C ABI provider (ADR-0036) is pinned by
crates/marqdo-host/tests/c_provider.rs, which compiles a small C shared
library, registers it, round-trips resource.*, checks missing-library,
wrong-ABI-version and pure/strict fail-closed behavior; the package-manifest
test installs a C provider from provider.json.
RFC-CM CM-A09 worker provider transport (ADR-0035) is pinned by
crates/marqdo-host/tests/worker_provider.rs: a Python helper is spawned,
pinged, opened/read/written/closed through framed JSON; missing grant, pure,
strict and missing-program cases are rejected. The helper is trusted, not a
sandbox.
RFC-CM CM-A09 in-process provider v1 (ADR-0033) is pinned by the three
registered_provider_streams_* / provider_* tests in
crates/marqdo-host/tests/resource_streams.rs: a mock provider is registered
and read/written through resource.* with a project-mode name grant; missing
grant, pure/strict, bad protocol version, reserved kinds and duplicate kinds
are rejected.
RFC-CM P2 closure (C01–C06, C11, C12) is pinned by
tools/pack-stdlib.py is the CM-A10 deterministic package builder (not a test;
its descriptor is checked in under docs/evidence/).
tools/perf/bench.py is the P6 baseline harness (not a test; it is not part of
the cargo gate).
crates/cli/tests/p2_closure.rs: 21 semantic fixtures run through
run --mode pure --json, sources are scanned for foreign/plugin/import
shortcuts, and value/stdout are compared with the goldens; the stdlib table
self-test also runs in pure mode. Negative cases prove foreign.call is
permission.denied and plugin.invoke fails in the pure lane.
RFC-CM CM-A08 table stdlib (ADR-0032) is pinned by
crates/cli/tests/stdlib_table.rs, which runs stdlib/tests/table.mq.md and
checks select/filter/sort/append/set/join/group results plus original-table
immutability.
RFC-CM CM-A07 Compat-C (ADR-0031) is pinned by
crates/marqdo-runtime/tests/interpreter.rs::print_without_the_bundled_stdlib_is_a_structured_error
and the absence of any HostOp::Print/op_print reference. The semantic
harness mirrors the loader's stdlib injection for direct-runtime fixtures.
RFC-CM CM-A07 Compat-B (ADR-0030) is pinned by
crates/cli/tests/print_routing.rs:
- a print program's trace contains
resource.writeand noprinteffect; - stdlib modules are injected only for programs that call
打印/print; - all existing semantic/CLI print goldens still pass byte-identically.
RFC-CM CM-A10 bundled stdlib (ADR-0028) is pinned by
crates/cli/tests/stdlib_bundle.rs (6 tests):
- explicit
std:ioimport resolves the verified bundle and runs; - Analysis records
project.stdlib.version/digest/instance_id; - a tampered
io.mq.mdis rejected withstdlib digest mismatchbefore any module executes; - a missing manifest is a hard error;
- the bundle has no
foreign./plugin.calls; - the CLI binary hash is unchanged after running a
std:ioproject (frozen-CLI closure).
RFC-CM CM-A07 Compat-A (ADR-0027) is pinned by:
stdlib/{format,text,io}.mq.mdandstdlib/demo.mq.md;crates/cli/tests/examples_r6.rs::mqmd_stdlib_print_output_matches_builtin_print, which runs the demo and assertsio.写值and builtin打印lines are byte-identical for a nested List/Map/None value and a Table;- the demo covers Unit/None/Bool, Int, Float, Text, List, Map, Table,
Formula, Code and FunctionRef; PluginHandle formatting uses the same
observation members but needs a plugin to construct a value.
打印is routed to the stdlib by ADR-0030 Compat-B when the bundle is present; this demo still calls the explicitio.写值to compare paths.
RFC-CM CM-A06 resource streams (ADR-0025) are pinned by:
- semantic fixture 66:
.mq.mdUTF-8 text encoded intoMemory, written to astdio/stdoutresource, closed, then re-used to prove{ok:false,code:"resource.closed"}; the expectedstdoutis the decoded text; crates/marqdo-host/tests/resource_streams.rs(17): pure-mode stdin read, stdout/stderr separation, output-budget atomicity, strict refusal of stdin/stderr/file, project-file chunking/EOF, missing grant, traversal rejection and unsupported rights/kinds;crates/cli/tests/resource_streams.rs: realmarqdo runwith--allow-read data.txt, three chunked reads intoMemoryplus EOF, piped stdin line reads throughstd:io/console, separate stdout/stderr capture, the same document failing withpermission.deniedwithout the grant, and a write-then-read program with--allow-write/--allow-read out.binproving the file is created only atresource.close, and a chunked.mq.mdfile copy (resource.read→resource.write) that reproducesdata.txtinout.bin;crates/marqdo-host/src/resource.rsunit tests for forged/closed handles, stdout read/file write unsupported, and chunked EOF.
Fixture discovery is directory-based and sorted by file name, so the order is deterministic on every platform. The AST and semantic suites fail loudly if the corpus shrinks below 100 / 25 fixtures; the warning and runtime suites fail if their directories are empty; the negative suite fails below 30 compile-time hard-error fixtures.
AST goldens and UPDATE_GOLDENS
*.ast.json files are the pretty-printed canonical AST view
(marqdo_markdown::parse(source).root), exactly as marqdo inspect --ast --json
would print it. They pin parser version, extension flags and span arithmetic.
Regenerate only for an intentional, ADR-backed profile change (the profile
is frozen in spec/markdown-profile.md):
UPDATE_GOLDENS=1 cargo test -p marqdo-cli --test conformance_ast
The update run rewrites every golden, prints how many it rewrote and passes. Re-run without the variable to verify, and review the golden diff: a golden change is a language change, not a bug fix.
A fixture without a golden is an error, and a golden without a fixture is an error (stale corpus output), so add/remove both together.
Semantic goldens
Each *.expected.json has the shape:
{
"function": "<name passed to marqdo_runtime::run>",
"args": { "<name>": <wire value> },
"value": <wire value>,
"stdout": "..."
}
argsandvalueuse the typed wire encoding frommarqdo_value::to_wire({"type":"int","decimal":"42"},{"type":"text","value":"…"}, …).stdoutis the program output captured from the host (never diagnostics).- Optional
"grants": ["foreign"]enablesforeign.call; that fixture is skipped with a printed notice whenpython3is unavailable.
All values were produced by running the implementation and then checked by
hand against the fixture source. UPDATE_SEMANTICS_GOLDENS=1 exists for
authoring convenience, but it must not be used blindly: rewriting a semantic
golden without verifying the value would defeat the purpose of the corpus.
Negative fixtures
All three suites use the single current language contract
(docs/architecture/RFC-LANG-0002-unified-semantics.md,
spec/mapping.md, spec/expression.ebnf): @exec/@knowledge heading
modes, interchangeable Strong/Emph/inline-Code one-statement payloads,
and explicit if/else/while/end/return/params. Ordinary Markdown
lists, block quotes, tables and emphasis are not errors: they are inert
containers/knowledge with an empty execution projection. The v1
markdown.invalid_construct shape errors ("ordered list must have a
condition", "block quote must be a call", "Strong must not nest Emph", "empty
heading") were removed with the old mapping; markdown.inert_executable
warnings no longer exist either.
Compile-time (tests/negative/*.mq.md, 45 fixtures):
{"codes": ["sem.unclosed_control"]}
The expected list is the exact sorted multiset of error diagnostic codes
produced by marqdo_frontend::analyze_source. Warnings emitted by the same
document are ignored here. Every listed code is additionally asserted to have
error severity, and every fixture must produce at least one error — an
invalid Exec payload never falls back to prose.
Coverage by group:
| Group | Fixtures | Codes |
|---|---|---|
| heading modes and unit identity | 01–07, 44–45 | sem.invalid_mode_marker, sem.empty_unit_name, sem.duplicate_unit_name, sem.exec_without_function |
| payload extraction | 08–17 | sem.payload_nested (nested Emph/Strong/Link/math/strikethrough/line break), sem.empty_payload (whitespace-only Strong/Emph) |
| statement grammar | 18–23 | sem.syntax |
| control stack and heading boundary | 24–29 | sem.unmatched_else, sem.duplicate_else, sem.unmatched_end, sem.unclosed_control |
| signatures | 30–32 | sem.signature_position |
| version selection | 33–34 | sem.unsupported_version_directive |
kept non-sem errors |
35–43 | symbol.unknown, call.arity, symbol.uninitialized, frontmatter.invalid |
Fixture 29 crosses a heading boundary on purpose: the opener's segment reports
sem.unclosed_control and the later ### @exec segment reports
sem.unmatched_end; both are asserted, and neither is "repaired" across the
boundary (spec/mapping.md §7).
sem.no_entry_function is a run entry-selection diagnostic: it is produced
when the CLI cannot find an Exec H2 main. crates/cli/tests/negative.rs only
calls marqdo_frontend::analyze_source and marqdo_runtime::run, and the
latter reports an unknown function for a missing name; there is no CLI-entry
suite in that harness, so this corpus cannot pin the code without a harness
change. It is recorded here as a known gap rather than replaced by a weaker
assertion (see spec/references.md §7 and spec/mapping.md §8).
v1 → current reclassification
| v1 fixtures | v1 expectation | Current disposition |
|---|---|---|
01–08 nested Strong/Emph children |
markdown.invalid_construct |
kept as 08–15 with sem.payload_nested |
| 09–12 empty/parse-error carriers | markdown.invalid_construct, expr.parse |
empty carriers kept as 16–17 (sem.empty_payload); the unqualified expression-parse cases are replaced by the explicit sem.syntax statements in 18–23 |
| 13–24 list/quote shape errors | markdown.invalid_construct, expr.parse |
removed: ordinary lists/quotes are valid inert Markdown now; the positive control is warnings/02-knowledge-and-containers-are-inert |
| 25–27 link-form calls (external/cross-module/anchor) | ref.unresolved, module.unresolved |
removed: link-form calls no longer exist; ref.unresolved/module.unresolved are now check --refs/loader diagnostics outside this harness (spec/references.md §7) |
| 28–29 duplicate function/namespace names | markdown.invalid_construct |
kept as 44–45 with sem.duplicate_unit_name (duplicate canonical name, spec/references.md §2.3) |
| 30–31 empty function/namespace headings | markdown.invalid_construct |
replaced by 04–05 with sem.empty_unit_name on an actual mode declaration |
| 32–40 unknown/call/signature/frontmatter errors | symbol.unknown, call.arity, symbol.uninitialized, expr.parse, frontmatter.invalid |
kept 35–43 (frontmatter.*, symbol.*, call.arity, symbol.uninitialized); malformed-signature cases moved to sem.syntax/sem.signature_position (23, 30–32) |
| 41–48 frontmatter/link/assignment/power shapes | frontmatter.invalid, expr.parse |
frontmatter cases kept as 40–42; link/quote/assignment shapes removed with the old mapping; power/condition cases replaced by 18–29 |
| 49–52 power/condition operands swallowed by a branch condition | expr.parse, markdown.invalid_construct |
removed: executable nodes inside list items are now independent payloads. Replacement negatives are the sem.syntax (18–23) and control-stack (24–29) fixtures |
Warnings (tests/negative/warnings/*.mq.md, 2 fixtures):
{"severity": "warning", "codes": ["sem.inferred_signature"]}
These documents must compile without errors; the sorted multiset of warning
codes must match exactly. Fixture 01 pins the sem.inferred_signature lint for
an Exec H2 that relies on free-variable inference. Fixture 02 is a
zero-diagnostic control: it proves that ordinary ordered lists, bullet lists
and block quotes are valid inert Markdown even inside an @exec section, and
that an explicit params(...) declaration (allowed after prose) suppresses the
inferred-signature warning. The lint is specified as optional
(spec/mapping.md §6, "may be enabled"); these fixtures pin it enabled, so if
an implementation leaves it off the warning suite — not the hard-error corpus —
is what must be revisited.
Runtime (tests/negative/runtime/*.mq.md, 21 fixtures):
{"function": "main", "args": {}, "codes": ["runtime.division_by_zero"]}
These documents compile, then must fail at runtime with the expected stable
error code. Sources were migrated to the current syntax (explicit ## @exec
functions, params(...) where a function has arguments, payload calls and
explicit if/end), but the genuine runtime expectations were preserved:
arithmetic type/zero-division, index/field/memory bounds, call.not_callable,
call.arity through a ref function value, non-Bool branch condition,
capability (permission.denied for a foreign kernel in pure mode), unsupported
math subset, recursion budget and the addressable-memory/collection errors.
Optional keys: "grants": ["foreign"] and "stack_limit": 24 (used for the
recursion-depth fixture so the budget — not the host process stack — is what
fails).
CLI integration notes
integration_cli.rs never leaves the repository dirty:
- temporary projects are created under
std::env::temp_dir()and removed on drop; - the demo C plugin is built in a temp copy of
examples/plugin-demo(cc -shared -fPIC -I <repo>/include), and thesha256in the copiedplugin.yamlis refreshed there. The repo copy and its.so(gitignored) are never touched; - tests that need
python3orccprint a skip notice and return when the tool is unavailable.
Notes discovered while authoring (all since fixed)
Two observations were reported and then fixed in the implementation; the goldens reflect the fixed behaviour:
- wire floats round-trip exactly (
serde_json/float_roundtripis enabled), so Pythonmath.pi * 9keeps28.274333882308138; - the default interpreter stack budget is 128 frames so a deeply recursive
document returns a structured
resource.exhaustedinstead of exhausting the native debug-build stack (--stackcan raise it deliberately).
The old link-form block-quote call (> [x](https://example.com)()) no longer
exists: a Markdown link is a static reference, so a call is now an ordinary
expression payload (spec/references.md §6.2).