- add check --emit with a deterministic, read-only semantic projection: package-aware modules, signatures, normalized bodies, refs, calls, diagnostics and source digests - remove stale analysis artifacts on compile errors; atomic writes - add run/call --trace-file with interpreter call/branch/loop/return events and host effect events under one sequence; redact values and record error status - add Analysis/Trace v1 specs, ADR-0015 and an independent Python consumer - add contract tests for determinism, stale artifacts, trace ordering/status and secret redaction
15 KiB
Marqdo Contributor Guide
This guide is for people changing the Marqdo CLI workspace. It assumes you have
read AGENTS.md (the non-negotiable architectural rules) and
docs/DEVELOPMENT_BLUEPRINT.md §0/§1/§11/§13/§14. Formal contracts live in
spec/; decisions live in docs/adr/.
1. Repository map
marqdo-cli/
├── Cargo.toml workspace (15 crates, resolver 2)
├── AGENTS.md operator rules for contributors/agents
├── README.md user-facing quickstart and limits
├── docs/
│ ├── DEVELOPMENT_BLUEPRINT.md architecture baseline
│ ├── NEXT_PHASE_IMPLEMENTATION.md proposed M0.6 plan (Proposed, not implemented)
│ ├── EXTENSIBILITY_AUDIT.md source-backed E01–E16 audit for M0.6
│ ├── STATUS.md implementation matrix (P0–P7)
│ ├── contributor-guide.md this file
│ └── adr/0001..0015-*.md accepted decisions
├── spec/
│ ├── markdown-profile.md frozen parser profile `marqdo.md.v1`
│ ├── mapping.md AST → program roles
│ ├── expression.ebnf executable payload grammar
│ ├── evaluation.md values, numbers, control flow, builtins
│ ├── references.md imports/references/calls/ref()
│ ├── value-wire.md typed wire_version 1 values
│ ├── security.md modes, capabilities, budgets, trust
│ ├── package-format.md marqdo.yaml / marqdo.lock / init
│ ├── plugin-abi.md C ABI v1, SDK, host lifecycle
│ ├── math.md Formula subset and math.* semantics
│ ├── foreign.md Python/Node kernel protocol
│ ├── tool-v1.md generic external Tool Protocol v1
│ ├── analysis-v1.md public Analysis v1 projection
│ └── trace-v1.md public Trace v1 event stream
├── crates/
│ ├── marqdo-span/ Span, LineCol, LineIndex (leaf)
│ ├── marqdo-value/ Value, diagnostics, errors, wire encoding
│ ├── marqdo-expression/ lexer/parser/AST for executable payloads
│ ├── marqdo-markdown/ comrak profile, canonical AST, front matter
│ ├── marqdo-ir/ Module/Unit/Function/Stmt IR, builtin table
│ ├── marqdo-frontend/ units, map, signature, infer, calls, resources
│ ├── marqdo-math/ Formula parser/evaluator (exact rationals)
│ ├── marqdo-foreign/ Python/Node one-shot kernel supervisor
│ ├── marqdo-sandbox/ modes + platform capability detection
│ ├── marqdo-plugin-abi/ C ABI mirrors, manifest, artifact, loader, wire
│ ├── marqdo-plugin-sdk/ Rust plugin SDK and `mq_plugin_entry!`
│ ├── marqdo-host/ CapabilityHost, budgets, FS, math/foreign/plugin adapters
│ ├── marqdo-runtime/ tree-walk interpreter, value ops, builtins
│ ├── marqdo-package/ marqdo.yaml / marqdo.lock types + discovery
│ └── cli/ composition root: flags, commands, loader, output
├── include/marqdo_plugin_abi_v1.h frozen C ABI v1 header
├── examples/ hello, factorial, math, references, python-foreign,
│ multi-module/, plugin-demo/
├── tests/
│ ├── conformance/ast/ *.mq.md + *.ast.json AST goldens
│ ├── conformance/semantics/ *.mq.md + *.expected.json execution goldens
│ ├── negative/ compile-time + runtime hard-error fixtures
│ ├── README.md corpus layout and golden workflow
│ ├── integration/ properties/ security/ abi/ reserved dirs (see STATUS)
└── crates/cli/tests/ Rust harnesses for the corpus and CLI
2. Dependency direction (must stay acyclic)
span -> serde
value -> span
expression -> span
markdown -> span, comrak (no semantic crate!)
ir -> span, expression
frontend -> markdown, expression, ir, value
host -> value, math, foreign, sandbox
runtime -> ir, value, host, expression
math -> value
foreign -> value
plugin-abi -> value
plugin-sdk -> plugin-abi
package -> serde only
sandbox -> serde only
cli -> everything (composition root)
Rules (from blueprint §11.1, adjusted by ADR-0001):
markdownowns the only Markdown parser; it must not depend on semantic crates, the host or the runtime. The single exception to "zero Marqdo deps" ismarqdo-span, which is a dep-free leaf shared by all layers.frontend/ir/runtimenever depend oncli.runtimenever parses Markdown and never performs I/O directly: all effects go throughmarqdo_host::Host.hostnever depends on the CLI; kernels/plugins are adapted here.- The CLI is the composition root: flag parsing, project loading, rendering and process exit. It must not become a second interpreter.
- Check the graph with
cargo tree(orcargo tree -i <crate>);runtime -> cli,markdown -> frontend, and dependency cycles are review blockers.
3. Adding a feature or fixture (workflow)
The project uses a fixture-first workflow (blueprint §13/§14, AGENTS § "Development workflow"). The expected sequence is:
- Find the rule. Identify the affected spec section and blueprint section.
If the change alters observable language semantics, package/lock format,
ABI, authorization defaults or public CLI behavior, write/supersede an ADR
first (
docs/adr/, template in blueprint §16.2). - Write a failing fixture. Add a small
.mq.mdpositive and/or negative case under the right directory:- AST shape:
tests/conformance/ast/NNN-name.mq.md+ goldenNNN-name.ast.json; - semantics:
tests/conformance/semantics/NNN-name.mq.md+NNN-name.expected.json(typed wireargs/value+ capturedstdout); - hard error:
tests/negative/NNN-name.mq.md+{"codes":[...]}; - inert-executable warning:
tests/negative/warnings/; - runtime error:
tests/negative/runtime/with{"function","args","codes"}(optionalgrants,stack_limit). Run the relevant harness (see §6) and confirm it fails for the expected reason.
- AST shape:
- Update the spec.
spec/*.mdis normative. If implementation and spec disagree, fix whichever is wrong and note the tradeoff. Do not silently invent behavior in code and leave the spec stale. - Implement the smallest module. Keep the layers separated: parser
(
marqdo-markdown), semantic mapping (marqdo-frontend), IR (marqdo-ir), interpreter (marqdo-runtime), effects (marqdo-host), composition (cli). Never add a second Markdown parser, a regex-based syntax decision, or a host bypass. - Refresh goldens deliberately (§4) and run the full gate (§6).
- Write the tradeoff/limitations in
docs/STATUS.mdwhen a P-level claim changes, and in the PR description (paths, commands run, skipped tests).
If a fixture reveals an implementation bug, fix the code, not the expected behavior. Negative fixtures must assert hard errors — never weaken them to warnings or prose fallbacks.
4. Golden update workflow
AST goldens (tests/conformance/ast/*.ast.json) are the serialized canonical
AST from marqdo_markdown::parse(source).root. Regenerate only for an
intentional, ADR-backed profile change:
UPDATE_GOLDENS=1 cargo test -p marqdo-cli --test conformance_ast
The update run rewrites every golden, prints the count and passes. Re-run without the variable and review the diff: a golden change is a language change, not a bug fix. A fixture without a golden and a golden without a fixture are both errors.
Semantic goldens (*.expected.json): UPDATE_SEMANTICS_GOLDENS=1 exists for
authoring convenience, but never use it blindly — verify every value against the
fixture source first. Invalid, inconsistent or unverified expected values defeat
the corpus.
Negative goldens are hand-written expected code lists (exact sorted multiset for compile-time errors; warning suites; runtime suites). Do not auto-generate them.
Adding/removing a fixture and its golden must happen in the same change.
Fixtures are data: tests must not commit, copy or rewrite files outside a
throw-away temp directory. tests/README.md is the in-repo reference for corpus
counts, harness responsibilities and known observations.
5. Coding conventions
- Rust edition 2021,
rust-version = 1.82, workspace version0.1.0. unsafeis forbidden in pure crates (marqdo-math,marqdo-expressionuse#![forbid(unsafe_code)]); the ABI/loader crates contain the onlyunsafecode and every block has a SAFETY comment explaining the contract.- No panics on untrusted input: Markdown, expressions, LaTeX, wire data,
manifests and foreign output all produce structured errors.
unwrap()/expect()are acceptable only for genuinely impossible invariants or in tests. - Stable diagnostic/runtime codes are part of the contract; human messages
are not. Add new codes to
crates/marqdo-value/src/diagnostic.rsand map categories inerror.rs. - Values cross layers via
marqdo_value::Valueand the typed wire format (spec/value-wire.md); never serializeIntas a JSON number. check/inspectmust remain side-effect free: no execution, no foreign process, no plugindlopen, no file writes.- Prose and executable nodes mix freely; prose never causes an action.
- Formatting:
rustfmt.tomlis authoritative; keepcargo fmt --checkclean.
6. Build, test and quality gate
The workspace is offline-cached. Use the workspace cache as documented in
tests/README.md / AGENTS.md:
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
Commands:
cargo build -p marqdo-cli # binary: target/debug/marqdo
cargo test --workspace # all crate unit + integration tests
cargo test -p marqdo-cli # conformance corpus + CLI integration tests
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
Harness map (crates/cli/tests/):
| Harness | Proves |
|---|---|
conformance_ast.rs |
frozen profile parses fixtures into checked-in AST (kinds, attributes, literals, spans, order) |
conformance_semantics.rs |
frontend + IR + runtime + host produce expected wire value/stdout |
negative.rs |
invalid executable shapes are hard errors; warning and runtime suites |
properties.rs |
prose-invariance (pure narrative edits do not change executable IR modulo spans) |
integration_cli.rs |
real CLI contract: init/check/run/call/inspect/fmt, multi-module, lock/sync, plugin trust and invocation |
Crate-level tests worth knowing:
marqdo-expression— grammar/priority/nesting/limit tests;marqdo-math/tests/kernel.rs— exact arithmetic, bounds, equations, unsupported constructs;marqdo-foreign/tests/kernels.rs— real python3/node (skips if absent), timeout, output caps, env, wire protocol;marqdo-plugin-abi/tests/loader_unix.rs— compiles the C demo withcc, loads it, ABI-mismatch/negative cases (skips withoutcc);marqdo-runtime/tests/interpreter.rs— recursion, loops, returns, budgets, ambiguity;marqdo-hostunit tests — capabilities, project FS, budgets.
Tests that need Python/Node/cc print a skip notice and pass when unavailable;
record such skips in the PR.
7. Running examples
From the repository root (after cargo build -p marqdo-cli):
B=target/debug/marqdo
$B run examples/hello.mq.md
$B call examples/factorial.mq.md 数学.阶乘 --arg n=5 # prints 120
$B check examples/references.mq.md
$B inspect examples/references.mq.md --refs
$B inspect examples/hello.mq.md --ast --json
$B run examples/math.mq.md # has no main: use call
$B call examples/math.mq.md 数学.计算圆面积
$B run examples/python-foreign.mq.md --allow-foreign # needs python3
$B run examples/multi-module/index.mq.md
(cd examples/multi-module && ../../$B lock && ../../$B sync --locked)
# Plugin demo (build the artifact + refresh its sha256 first)
./examples/plugin-demo/build.sh
$B plugin inspect examples/plugin-demo
$B run examples/plugin-demo/plugin-call.mq.md \
--plugin examples/plugin-demo --trust-native-plugin --allow-plugin demo.solver
Notes: run defaults to index.mq.md and executes main; call requires a
file and function; root-only examples (for example math.mq.md) define no
main. --mode strict is expected to refuse to run (ADR-0004).
8. Docs, ADRs and commit policy
spec/= normative contracts;docs/= architecture/decisions/status;README.md= user quickstart;AGENTS.md= contributor rules.docs/NEXT_PHASE_IMPLEMENTATION.mdanddocs/EXTENSIBILITY_AUDIT.mdare the M0.6 planning/evidence pair. The plan is Proposed: it supersedes the blueprint only on command-surface scope, extension registration, plugin addressing, package identity, repository decoupling and release gates. Language semantics remain normative inspec/+ Accepted ADRs + tests, andspec/tool-v1.mdis a candidate protocol until an ADR and implementation tests land. Never cite these proposed documents as evidence that a feature exists.- Changes to language semantics, AST mapping, wire format, plugin ABI, package/lock format, authorization defaults or public CLI behavior require an Accepted ADR (or superseding one) plus spec updates and tests. Use the blueprint §16.2 template: Context / Decision / Alternatives / Markdown AST shape / Compatibility / Security / Testing / Migration.
- ADRs are immutable once Accepted: add a new ADR with
Status: Superseded by ADR-NNNNrather than rewriting history (typo fixes aside). - Docs must describe what the code does. When implementation and blueprint disagree, document the implementation and flag the divergence explicitly as a limitation/open item; never invent behavior.
- Do not edit generated/vendored state, and never modify tests to hide a regression.
- Commit in reviewable units with a message stating the affected spec/ADR and
the commands run. Do not commit built plugin artifacts (
*.sois gitignored), the cargo cache, or temp projects. - There is no git-commit automation in this workspace; contributors commit manually after review.
9. Known gotchas
.cache/is the offline cargo cache/target used in this environment; it is not part of the deliverable.- Foreign availability (
python3/node) and a C compiler (cc) determine which tests run; a skipped test is not a passed feature. - The strict sandbox, plugin worker isolation and package registry are not
implemented. Do not write docs or tests that pretend they are — see
docs/STATUS.mdandspec/security.md. - Runtime error spans in the CLI currently resolve against the entry file only; imported-module error locations are approximate (ADR-0006).
- The semantic corpus may pin current quirks (for example the external-link call code, or a float's last digit). A fixture diff for such a quirk is a deliberate change, not a free update.