Files
marqdo-cli/docs/contributor-guide.md
T
cflmy bbeb1de768 docs(status): record the landed RFC-LANG-0002 implementation
Update status, compatibility, changelog, README, architecture overview,
ADR status, migration guide, contributor/CI/security/release notes and
the per-file inventory from 'migration pending' to the verified state:

- language SHA ffd1cc9: fmt/clippy clean, workspace 501 passed / 0 failed
  / 2 ignored, stdlib digest regenerated and pack-verified
- green evidence report added; the prior intermediate evidence is marked
  superseded
- inventory regenerated: 1015 classified files, 0 pending
- explicit non-claims: Windows/macOS native lanes and the disposable-VM
  strict lane were not re-run at this SHA, so the release decision remains
  DO NOT RELEASE

Verified: cargo test -p marqdo-cli --test docs_consistency (4/4) and
git diff --check.
2026-11-21 12:31:38 +08:00

358 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Marqdo Contributor Guide
This guide is for people changing the Marqdo CLI workspace. It assumes you have
read `AGENTS.md` (the non-negotiable architectural rules), the current language
authority [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
with its normative summaries `spec/mapping.md` + `spec/expression.ebnf`, and
`docs/DEVELOPMENT_BLUEPRINT.md` §0/§1/§11/§13/§14 (the language-mapping sections
there are historical and defer to the RFC). Decisions live in `docs/adr/` at
their date.
> **Implementation state:** RFC-LANG-0002 is implemented by the frontend,
> `marqdo-expression`, IR lowering, runtime and bundled stdlib. The workspace
> gate is green at `ffd1cc9`; status and evidence are in
> [`STATUS.md`](STATUS.md); migration history and the inventory are in
> [`migration/README.md`](migration/README.md).
## 1. Repository map
```text
marqdo-cli/
├── Cargo.toml workspace (15 crates, resolver 2)
├── AGENTS.md operator rules for contributors/agents
├── README.md user-facing quickstart and limits
├── docs/
│ ├── DEVELOPMENT_BLUEPRINT.md architecture baseline (language-mapping sections historical)
│ ├── architecture/RFC-LANG-0002-unified-semantics.md current language authority
│ ├── STABLE_RELEASE_IMPLEMENTATION.md authoritative next-phase/release plan
│ ├── STATUS.md current evidence, work packages, blockers
│ ├── migration/ syntax-migration rules + file inventory
│ ├── security-gate.md G-C probe evidence (strict stays closed)
│ ├── releasing.md artifact/checksum/smoke/tag procedure
│ ├── CORE_STABILIZATION_AND_ECOSYSTEM_GATE.md historical stabilization spec
│ ├── NEXT_PHASE_IMPLEMENTATION.md historical M0.6 plan
│ ├── EXTENSIBILITY_AUDIT.md source-backed E01–E16 audit
│ ├── history/ archived status snapshots
│ ├── contributor-guide.md this file
│ └── adr/0001..0052-*.md decisions at their date (ADR-0052 = current language mapping)
├── spec/
│ ├── markdown-profile.md frozen parser profile `marqdo.md.v1`
│ ├── mapping.md current heading-mode/payload/control-flow mapping
│ ├── expression.ebnf statement/payload grammar (keywords)
│ ├── evaluation.md values, numbers, control flow, builtins
│ ├── references.md imports/references/calls/ref()
│ ├── value-wire.md typed wire_version 1 values
│ ├── security.md modes, capabilities, budgets, trust
│ ├── package-format.md marqdo.yaml / marqdo.lock / init
│ ├── plugin-abi.md C ABI v1, SDK, host lifecycle
│ ├── math.md Formula subset and math.* semantics
│ ├── foreign.md Python/Node kernel protocol
│ ├── tool-v1.md generic external Tool Protocol v1
│ ├── analysis-v2.md public Analysis v2 projection (v1 historical)
│ ├── trace-v2.md public Trace v2 projection (v1 historical)
│ └── trace-v1.md public Trace v1 event stream
├── crates/
│ ├── marqdo-span/ Span, LineCol, LineIndex (leaf)
│ ├── marqdo-value/ Value, diagnostics, errors, wire encoding
│ ├── marqdo-expression/ lexer/parser/AST for executable payloads
│ ├── marqdo-markdown/ comrak profile, canonical AST, front matter
│ ├── marqdo-ir/ Module/Unit/Function/Stmt IR, builtin table
│ ├── marqdo-frontend/ units/modes, map, payloads, signature, infer, calls
│ ├── marqdo-math/ Formula parser/evaluator (exact rationals)
│ ├── marqdo-foreign/ Python/Node one-shot kernel supervisor
│ ├── marqdo-sandbox/ modes + platform capability detection
│ ├── marqdo-plugin-abi/ C ABI mirrors, manifest, artifact, loader, wire
│ ├── marqdo-plugin-sdk/ Rust plugin SDK and `mq_plugin_entry!`
│ ├── marqdo-host/ CapabilityHost, budgets, FS, math/foreign/plugin adapters
│ ├── marqdo-runtime/ tree-walk interpreter, value ops, builtins
│ ├── marqdo-package/ marqdo.yaml / marqdo.lock types + discovery
│ └── cli/ composition root: flags, commands, loader, output
├── include/marqdo_plugin_abi_v1.h frozen C ABI v1 header
├── examples/ hello, factorial, math, references, python-foreign,
│ multi-module/, plugin-demo/
├── tests/
│ ├── conformance/ast/ *.mq.md + *.ast.json AST goldens
│ ├── conformance/semantics/ *.mq.md + *.expected.json execution goldens
│ ├── negative/ compile-time + runtime hard-error fixtures
│ ├── README.md corpus layout and golden workflow
│ ├── integration/ properties/ security/ abi/ reserved dirs (see STATUS)
└── crates/cli/tests/ Rust harnesses for the corpus and CLI
```
## 2. Dependency direction (must stay acyclic)
```text
span -> serde
value -> span
expression -> span
markdown -> span, comrak (no semantic crate!)
ir -> span, expression
frontend -> markdown, expression, ir, value
host -> value, math, foreign, sandbox
runtime -> ir, value, host, expression
math -> value
foreign -> value
plugin-abi -> value
plugin-sdk -> plugin-abi
package -> serde only
sandbox -> serde only
cli -> everything (composition root)
```
Rules (from blueprint §11.1, adjusted by ADR-0001):
* `markdown` owns the only Markdown parser; it must not depend on semantic
crates, the host or the runtime. The single exception to "zero Marqdo deps"
is `marqdo-span`, which is a dep-free leaf shared by all layers.
* `frontend`/`ir`/`runtime` never depend on `cli`.
* `runtime` never parses Markdown and never performs I/O directly: all effects
go through `marqdo_host::Host`.
* `host` never depends on the CLI; kernels/plugins are adapted here.
* The CLI is the composition root: flag parsing, project loading, rendering and
process exit. It must not become a second interpreter.
* Check the graph with `cargo tree` (or `cargo tree -i <crate>`); `runtime ->
cli`, `markdown -> frontend`, and dependency cycles are review blockers.
## 3. Adding a feature or fixture (workflow)
The project uses a fixture-first workflow (RFC-LANG-0002, blueprint §13/§14,
AGENTS § "Development workflow"). The expected sequence is:
1. **Find the rule.** Identify the affected section of RFC-LANG-0002 /
`spec/` (or, for architecture-only changes, the relevant blueprint section).
If the change alters observable language semantics, package/lock format,
ABI, authorization defaults or public CLI behavior, write/supersede an ADR
first (`docs/adr/`, template in blueprint §16.2). There is one active
language mapping: do not add a second mapper, a per-file syntax selector,
`--legacy` or a parallel syntax spec.
2. **Write a failing fixture.** Add a small `.mq.md` positive and/or negative
case under the right directory:
* AST shape: `tests/conformance/ast/NNN-name.mq.md` + golden
`NNN-name.ast.json`;
* semantics: `tests/conformance/semantics/NNN-name.mq.md` +
`NNN-name.expected.json` (typed wire `args`/`value` + captured `stdout`);
* hard error: `tests/negative/NNN-name.mq.md` + `{"codes":[...]}`;
* inert-executable warning: `tests/negative/warnings/`;
* runtime error: `tests/negative/runtime/` with `{"function","args","codes"}`
(optional `grants`, `stack_limit`).
Run the relevant harness (see §6) and confirm it fails for the expected
reason.
3. **Update the spec.** `spec/*.md` is normative. If implementation and spec
disagree, fix whichever is wrong and note the tradeoff. Do not silently
invent behavior in code and leave the spec stale.
4. **Implement the smallest module.** Keep the layers separated: parser
(`marqdo-markdown`), semantic mapping (`marqdo-frontend`), IR
(`marqdo-ir`), interpreter (`marqdo-runtime`), effects (`marqdo-host`),
composition (`cli`). Never add a second Markdown parser, a regex-based
syntax decision, or a host bypass.
5. **Refresh goldens deliberately** (§4) and run the full gate (§6).
6. **Write the tradeoff/limitations** in `docs/STATUS.md` when a P-level claim
changes, and in the PR description (paths, commands run, skipped tests).
### 3.1 Syntax rules for fixtures and examples (RFC-LANG-0002)
Every `.mq.md` fixture, example, stdlib module and documentation fence must use
the single current syntax:
* heading modes `@exec` / `@knowledge` with ancestor inheritance; only an Exec
H2 is callable;
* `Strong`, `Emph` and inline `Code` as interchangeable one-statement payload
carriers;
* `if`/`else`/`while`/`end`/`return`/`params(...)` keywords inside the payload;
* lists, block quotes and paragraphs as inert containers (no implicit
branch/loop/call);
* no version directive, `marqdo_semantics` field or old-signature code span.
Do not reintroduce the superseded mapping (`Strong`=execute, `Emph`=return,
ordered list=branch, bullet list=loop, block quote=call) in new fixtures. Old
shape negatives that are now valid inert Markdown must be replaced with
negatives for the `sem.*` codes in `spec/mapping.md` §8; genuine runtime
negatives keep their behavior and only change source spelling. Executable-source
tests must stay green; do not weaken assertions, delete failing tests or refresh
goldens just to make a change pass. Record each migrated active file in
[`migration/README.md`](migration/README.md) / the migration inventory.
If a fixture reveals an implementation bug, fix the code, not the expected
behavior. Negative fixtures must assert hard errors — never weaken them to
warnings or prose fallbacks.
## 4. Golden update workflow
AST goldens (`tests/conformance/ast/*.ast.json`) are the serialized canonical
AST from `marqdo_markdown::parse(source).root`. Regenerate **only** for an
intentional, ADR-backed profile change:
```bash
UPDATE_GOLDENS=1 cargo test -p marqdo-cli --test conformance_ast
```
The update run rewrites every golden, prints the count and passes. Re-run
without the variable and review the diff: **a golden change is a language
change, not a bug fix**. A fixture without a golden and a golden without a
fixture are both errors.
Semantic goldens (`*.expected.json`): `UPDATE_SEMANTICS_GOLDENS=1` exists for
authoring convenience, but never use it blindly — verify every value against the
fixture source first. Invalid, inconsistent or unverified expected values defeat
the corpus. During the RFC-LANG-0002 migration do not regenerate goldens to make
red executable-source tests pass: the target spec is ahead of the frontend, and
a golden may only change with a stated semantic reason. See
[`STATUS.md`](STATUS.md).
Negative goldens are hand-written expected code lists (exact sorted multiset for
compile-time errors; warning suites; runtime suites). Do not auto-generate them.
Adding/removing a fixture and its golden must happen in the same change.
Fixtures are data: tests must not commit, copy or rewrite files outside a
throw-away temp directory. `tests/README.md` is the in-repo reference for corpus
counts, harness responsibilities and known observations.
## 5. Coding conventions
* Rust edition 2021, `rust-version = 1.82`, workspace version `0.1.0`.
* `unsafe` is forbidden in pure crates (`marqdo-math`, `marqdo-expression` use
`#![forbid(unsafe_code)]`); the ABI/loader crates contain the only
`unsafe` code and every block has a SAFETY comment explaining the contract.
* No panics on untrusted input: Markdown, expressions, LaTeX, wire data,
manifests and foreign output all produce structured errors. `unwrap()`/
`expect()` are acceptable only for genuinely impossible invariants or in
tests.
* Stable diagnostic/runtime **codes** are part of the contract; human messages
are not. Add new codes to `crates/marqdo-value/src/diagnostic.rs` and map
categories in `error.rs`.
* Values cross layers via `marqdo_value::Value` and the typed wire format
(`spec/value-wire.md`); never serialize `Int` as a JSON number.
* `check`/`inspect` must remain side-effect free: no execution, no foreign
process, no plugin `dlopen`, no file writes.
* Prose and executable nodes mix freely; prose never causes an action.
* Formatting: `rustfmt.toml` is authoritative; keep `cargo fmt --check` clean.
## 6. Build, test and quality gate
The workspace is offline-cached. Use the workspace cache as documented in
`tests/README.md` / AGENTS.md:
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
```
Commands:
```bash
cargo build -p marqdo-cli # binary: target/debug/marqdo
cargo test --workspace # all crate unit + integration tests
cargo test -p marqdo-cli # conformance corpus + CLI integration tests
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
```
Harness map (`crates/cli/tests/`):
| Harness | Proves |
|---|---|
| `conformance_ast.rs` | frozen profile parses fixtures into checked-in AST (kinds, attributes, literals, spans, order) |
| `conformance_semantics.rs` | frontend + IR + runtime + host produce expected wire value/stdout |
| `negative.rs` | invalid executable shapes are hard errors; warning and runtime suites |
| `properties.rs` | prose-invariance (pure narrative edits do not change executable IR modulo spans) |
| `integration_cli.rs` | real CLI contract: init/check/run/call/inspect/fmt, multi-module, lock/sync, plugin trust and invocation |
Crate-level tests worth knowing:
* `marqdo-expression` — grammar/priority/nesting/limit tests;
* `marqdo-math/tests/kernel.rs` — exact arithmetic, bounds, equations, unsupported constructs;
* `marqdo-foreign/tests/kernels.rs` — real python3/node (skips if absent), timeout, output caps, env, wire protocol;
* `marqdo-plugin-abi/tests/loader_unix.rs` — compiles the C demo with `cc`, loads it, ABI-mismatch/negative cases (skips without `cc`);
* `marqdo-runtime/tests/interpreter.rs` — recursion, loops, returns, budgets, ambiguity;
* `marqdo-host` unit tests — capabilities, project FS, budgets.
Tests that need Python/Node/`cc` print a skip notice and pass when unavailable;
record such skips in the PR.
## 7. Running examples
From the repository root (after `cargo build -p marqdo-cli`):
```bash
B=target/debug/marqdo
$B run examples/hello.mq.md
$B call examples/factorial.mq.md 数学.阶乘 --arg n=5 # prints 120
$B check examples/references.mq.md
$B inspect examples/references.mq.md --refs
$B inspect examples/hello.mq.md --ast --json
$B run examples/math.mq.md # has no main: use call
$B call examples/math.mq.md 数学.计算圆面积
$B run examples/python-foreign.mq.md --allow-foreign # needs python3
$B run examples/multi-module/index.mq.md
(cd examples/multi-module && ../../$B lock && ../../$B sync --locked)
# Plugin demo (builds a host-specific manifest under build/)
./examples/plugin-demo/build.sh
$B plugin inspect examples/plugin-demo/build
$B run examples/plugin-demo/plugin-call.mq.md \
--plugin examples/plugin-demo/build --trust-native-plugin --allow-plugin demo.solver
```
Notes: `run` defaults to `index.mq.md` and executes `main`; `call` requires a
file and function; root-only examples (for example `math.mq.md`) define no
`main`. `--mode strict` is expected to refuse to run (ADR-0004). The examples
implement RFC-LANG-0002 syntax and run on the current CLI (subject to the
capability flags shown above). See [`STATUS.md`](STATUS.md).
## 8. Docs, ADRs and commit policy
* `spec/` = normative contracts; `docs/` = architecture/decisions/status;
`README.md` = user quickstart; `AGENTS.md` = contributor rules.
* Language semantics are normative only in RFC-LANG-0002 (`docs/architecture/`)
and its in-place summaries in `spec/`; the language-mapping sections of the
blueprint are historical. Do not add a second active syntax spec.
`docs/CORE_STABILIZATION_AND_ECOSYSTEM_GATE.md`,
`docs/NEXT_PHASE_IMPLEMENTATION.md` and
`docs/NEXT_RELEASE_CI_CONVERGENCE.md` are **Historical/Superseded** planning
input (their banners link the RFC); `docs/STABLE_RELEASE_IMPLEMENTATION.md` is
the authoritative next-phase/release plan. None describes implemented
behavior. `spec/tool-v1.md` is a candidate protocol until an ADR and
implementation tests land. Never cite a proposed or historical document as
evidence that a feature exists.
* Changes to language semantics, AST mapping, wire format, plugin ABI,
package/lock format, authorization defaults or public CLI behavior require an
**Accepted ADR** (or superseding one) plus spec updates and tests. Use the
blueprint §16.2 template: Context / Decision / Alternatives / Markdown AST
shape / Compatibility / Security / Testing / Migration.
* ADRs are immutable once Accepted: add a new ADR with
`Status: Superseded by ADR-NNNN` rather than rewriting history (typo fixes
aside).
* Docs must describe what the code does. When implementation and spec
disagree, document the implementation and flag the divergence explicitly as
a limitation/open item; never invent behavior. Right now `spec/` is ahead of
the frontend, so say that explicitly rather than claiming the new syntax
works.
* Do not edit generated/vendored state, and never modify tests to hide a
regression.
* Commit in reviewable units with a message stating the affected spec/ADR and
the commands run. Do not commit built plugin artifacts (`*.so` is
gitignored), the cargo cache, or temp projects.
* There is no git-commit automation in this workspace; contributors commit
manually after review.
## 9. Known gotchas
* `.cache/` and `.cargo-home/` are local caches used on hosts with a read-only
`$HOME`; neither is part of the deliverable.
* Foreign availability (`python3`/`node`) and a C compiler (`cc`) determine
which tests run; a skipped test is not a passed feature. Windows Native ABI
is explicitly unsupported, not a pass.
* Isolated worker plugins exist (crash/address-space isolation + brokered
`fs.read`), but they are **not** an OS sandbox. The strict sandbox and a
package registry remain unimplemented; strict fails closed. See
`docs/security-gate.md` and `spec/security.md`.
* RFC-LANG-0002 is implemented end to end. Keep `spec/`, the frontend,
`marqdo-expression`, stdlib and fixtures in lockstep; do not reintroduce the
old mapping, a per-file syntax selector or stale goldens.
* Runtime errors resolve against their owning module (ADR-0017); do not
reintroduce entry-file fallback.
* The semantic corpus may pin current quirks (for example the external-link
call code, or a float's last digit). A fixture diff for such a quirk is a
deliberate change, not a free update.