docs: formal specs, ADRs, STATUS matrix, contributor guide and honest README

Complete the documentation set for the M0.5 toolchain, grounded in the code:

- spec/references.md: import/reference/call edges, identities, loader rules,
  anchors, `ref` resolution/ambiguity, `check --refs` validation.
- spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal
  string rule, special floats, limits and exact float round-trip.
- spec/security.md: modes (strict refuses without OS isolation), capabilities,
  path rules, budgets, tracing, foreign process caveats, plugin trust.
- spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync,
  unsupported registry/git/environment work.
- spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact
  rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the
  implemented CLI plugin-invocation flow with its limitations.
- spec/math.md and spec/foreign.md: the implemented math subset and the
  foreign request/response/timeout/capability contract.
- docs/adr/0001..0007: accepted decisions for the span foundation, inert
  executables outside function bodies, ref ambiguity (S-07), strict refusal,
  AST normalization, CLI contract and BigInt/typed wire (S-08).
- docs/contributor-guide.md: repo map, workflow, fixture/golden updates,
  quality gate, ADR/commit policy.
- docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18
  Alpha checklist (conclusion: not yet Alpha-complete), including known
  divergences rather than overclaims.
- README.md rewritten from "design-only starter" to the working implementation
  with quickstart, implemented phases, deliberate limits and doc index.
- AGENTS.md: updated pointers/workflow only; architectural rules unchanged.
- Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside
  functions, unsupported math.simplify/differentiate).
This commit is contained in:
cflmy committed 2026-11-20 10:56:26 +08:00
1 parent fae0cd4b18
commit 0116160399
19 files changed
+3767 -30

No files matched your search

+286
View File
@@ -0,0 +1,286 @@
# Marqdo Contributor Guide
This guide is for people changing the Marqdo CLI workspace. It assumes you have
read `AGENTS.md` (the non-negotiable architectural rules) and
`docs/DEVELOPMENT_BLUEPRINT.md` §0/§1/§11/§13/§14. Formal contracts live in
`spec/`; decisions live in `docs/adr/`.
## 1. Repository map
```text
marqdo-cli/
├── Cargo.toml workspace (15 crates, resolver 2)
├── AGENTS.md operator rules for contributors/agents
├── README.md user-facing quickstart and limits
├── docs/
│ ├── DEVELOPMENT_BLUEPRINT.md architecture baseline
│ ├── STATUS.md implementation matrix (P0–P7)
│ ├── contributor-guide.md this file
│ └── adr/0001..0007-*.md accepted decisions
├── spec/
│ ├── markdown-profile.md frozen parser profile `marqdo.md.v1`
│ ├── mapping.md AST → program roles
│ ├── expression.ebnf executable payload grammar
│ ├── evaluation.md values, numbers, control flow, builtins
│ ├── references.md imports/references/calls/ref()
│ ├── value-wire.md typed wire_version 1 values
│ ├── security.md modes, capabilities, budgets, trust
│ ├── package-format.md marqdo.yaml / marqdo.lock / init
│ ├── plugin-abi.md C ABI v1, SDK, host lifecycle
│ ├── math.md Formula subset and math.* semantics
│ └── foreign.md Python/Node kernel protocol
├── crates/
│ ├── marqdo-span/ Span, LineCol, LineIndex (leaf)
│ ├── marqdo-value/ Value, diagnostics, errors, wire encoding
│ ├── marqdo-expression/ lexer/parser/AST for executable payloads
│ ├── marqdo-markdown/ comrak profile, canonical AST, front matter
│ ├── marqdo-ir/ Module/Unit/Function/Stmt IR, builtin table
│ ├── marqdo-frontend/ units, map, signature, infer, calls, resources
│ ├── marqdo-math/ Formula parser/evaluator (exact rationals)
│ ├── marqdo-foreign/ Python/Node one-shot kernel supervisor
│ ├── marqdo-sandbox/ modes + platform capability detection
│ ├── marqdo-plugin-abi/ C ABI mirrors, manifest, artifact, loader, wire
│ ├── marqdo-plugin-sdk/ Rust plugin SDK and `mq_plugin_entry!`
│ ├── marqdo-host/ CapabilityHost, budgets, FS, math/foreign/plugin adapters
│ ├── marqdo-runtime/ tree-walk interpreter, value ops, builtins
│ ├── marqdo-package/ marqdo.yaml / marqdo.lock types + discovery
│ └── cli/ composition root: flags, commands, loader, output
├── include/marqdo_plugin_abi_v1.h frozen C ABI v1 header
├── examples/ hello, factorial, math, references, python-foreign,
│ multi-module/, plugin-demo/
├── tests/
│ ├── conformance/ast/ *.mq.md + *.ast.json AST goldens
│ ├── conformance/semantics/ *.mq.md + *.expected.json execution goldens
│ ├── negative/ compile-time + runtime hard-error fixtures
│ ├── README.md corpus layout and golden workflow
│ ├── integration/ properties/ security/ abi/ reserved dirs (see STATUS)
└── crates/cli/tests/ Rust harnesses for the corpus and CLI
```
## 2. Dependency direction (must stay acyclic)
```text
span -> serde
value -> span
expression -> span
markdown -> span, comrak (no semantic crate!)
ir -> span, expression
frontend -> markdown, expression, ir, value
host -> value, math, foreign, sandbox
runtime -> ir, value, host, expression
math -> value
foreign -> value
plugin-abi -> value
plugin-sdk -> plugin-abi
package -> serde only
sandbox -> serde only
cli -> everything (composition root)
```
Rules (from blueprint §11.1, adjusted by ADR-0001):
* `markdown` owns the only Markdown parser; it must not depend on semantic
crates, the host or the runtime. The single exception to "zero Marqdo deps"
is `marqdo-span`, which is a dep-free leaf shared by all layers.
* `frontend`/`ir`/`runtime` never depend on `cli`.
* `runtime` never parses Markdown and never performs I/O directly: all effects
go through `marqdo_host::Host`.
* `host` never depends on the CLI; kernels/plugins are adapted here.
* The CLI is the composition root: flag parsing, project loading, rendering and
process exit. It must not become a second interpreter.
* Check the graph with `cargo tree` (or `cargo tree -i <crate>`); `runtime ->
cli`, `markdown -> frontend`, and dependency cycles are review blockers.
## 3. Adding a feature or fixture (workflow)
The project uses a fixture-first workflow (blueprint §13/§14, AGENTS §
"Development workflow"). The expected sequence is:
1. **Find the rule.** Identify the affected spec section and blueprint section.
If the change alters observable language semantics, package/lock format,
ABI, authorization defaults or public CLI behavior, write/supersede an ADR
first (`docs/adr/`, template in blueprint §16.2).
2. **Write a failing fixture.** Add a small `.mq.md` positive and/or negative
case under the right directory:
* AST shape: `tests/conformance/ast/NNN-name.mq.md` + golden
`NNN-name.ast.json`;
* semantics: `tests/conformance/semantics/NNN-name.mq.md` +
`NNN-name.expected.json` (typed wire `args`/`value` + captured `stdout`);
* hard error: `tests/negative/NNN-name.mq.md` + `{"codes":[...]}`;
* inert-executable warning: `tests/negative/warnings/`;
* runtime error: `tests/negative/runtime/` with `{"function","args","codes"}`
(optional `grants`, `stack_limit`).
Run the relevant harness (see §6) and confirm it fails for the expected
reason.
3. **Update the spec.** `spec/*.md` is normative. If implementation and spec
disagree, fix whichever is wrong and note the tradeoff. Do not silently
invent behavior in code and leave the spec stale.
4. **Implement the smallest module.** Keep the layers separated: parser
(`marqdo-markdown`), semantic mapping (`marqdo-frontend`), IR
(`marqdo-ir`), interpreter (`marqdo-runtime`), effects (`marqdo-host`),
composition (`cli`). Never add a second Markdown parser, a regex-based
syntax decision, or a host bypass.
5. **Refresh goldens deliberately** (§4) and run the full gate (§6).
6. **Write the tradeoff/limitations** in `docs/STATUS.md` when a P-level claim
changes, and in the PR description (paths, commands run, skipped tests).
If a fixture reveals an implementation bug, fix the code, not the expected
behavior. Negative fixtures must assert hard errors — never weaken them to
warnings or prose fallbacks.
## 4. Golden update workflow
AST goldens (`tests/conformance/ast/*.ast.json`) are the serialized canonical
AST from `marqdo_markdown::parse(source).root`. Regenerate **only** for an
intentional, ADR-backed profile change:
```bash
UPDATE_GOLDENS=1 cargo test -p marqdo-cli --test conformance_ast
```
The update run rewrites every golden, prints the count and passes. Re-run
without the variable and review the diff: **a golden change is a language
change, not a bug fix**. A fixture without a golden and a golden without a
fixture are both errors.
Semantic goldens (`*.expected.json`): `UPDATE_SEMANTICS_GOLDENS=1` exists for
authoring convenience, but never use it blindly — verify every value against the
fixture source first. Invalid, inconsistent or unverified expected values defeat
the corpus.
Negative goldens are hand-written expected code lists (exact sorted multiset for
compile-time errors; warning suites; runtime suites). Do not auto-generate them.
Adding/removing a fixture and its golden must happen in the same change.
Fixtures are data: tests must not commit, copy or rewrite files outside a
throw-away temp directory. `tests/README.md` is the in-repo reference for corpus
counts, harness responsibilities and known observations.
## 5. Coding conventions
* Rust edition 2021, `rust-version = 1.82`, workspace version `0.1.0`.
* `unsafe` is forbidden in pure crates (`marqdo-math`, `marqdo-expression` use
`#![forbid(unsafe_code)]`); the ABI/loader crates contain the only
`unsafe` code and every block has a SAFETY comment explaining the contract.
* No panics on untrusted input: Markdown, expressions, LaTeX, wire data,
manifests and foreign output all produce structured errors. `unwrap()`/
`expect()` are acceptable only for genuinely impossible invariants or in
tests.
* Stable diagnostic/runtime **codes** are part of the contract; human messages
are not. Add new codes to `crates/marqdo-value/src/diagnostic.rs` and map
categories in `error.rs`.
* Values cross layers via `marqdo_value::Value` and the typed wire format
(`spec/value-wire.md`); never serialize `Int` as a JSON number.
* `check`/`inspect` must remain side-effect free: no execution, no foreign
process, no plugin `dlopen`, no file writes.
* Prose and executable nodes mix freely; prose never causes an action.
* Formatting: `rustfmt.toml` is authoritative; keep `cargo fmt --check` clean.
## 6. Build, test and quality gate
The workspace is offline-cached. Use the workspace cache as documented in
`tests/README.md` / AGENTS.md:
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
```
Commands:
```bash
cargo build -p marqdo-cli # binary: target/debug/marqdo
cargo test --workspace # all crate unit + integration tests
cargo test -p marqdo-cli # conformance corpus + CLI integration tests
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
```
Harness map (`crates/cli/tests/`):
| Harness | Proves |
|---|---|
| `conformance_ast.rs` | frozen profile parses fixtures into checked-in AST (kinds, attributes, literals, spans, order) |
| `conformance_semantics.rs` | frontend + IR + runtime + host produce expected wire value/stdout |
| `negative.rs` | invalid executable shapes are hard errors; warning and runtime suites |
| `properties.rs` | prose-invariance (pure narrative edits do not change executable IR modulo spans) |
| `integration_cli.rs` | real CLI contract: init/check/run/call/inspect/fmt, multi-module, lock/sync, plugin trust and invocation |
Crate-level tests worth knowing:
* `marqdo-expression` — grammar/priority/nesting/limit tests;
* `marqdo-math/tests/kernel.rs` — exact arithmetic, bounds, equations, unsupported constructs;
* `marqdo-foreign/tests/kernels.rs` — real python3/node (skips if absent), timeout, output caps, env, wire protocol;
* `marqdo-plugin-abi/tests/loader_unix.rs` — compiles the C demo with `cc`, loads it, ABI-mismatch/negative cases (skips without `cc`);
* `marqdo-runtime/tests/interpreter.rs` — recursion, loops, returns, budgets, ambiguity;
* `marqdo-host` unit tests — capabilities, project FS, budgets.
Tests that need Python/Node/`cc` print a skip notice and pass when unavailable;
record such skips in the PR.
## 7. Running examples
From the repository root (after `cargo build -p marqdo-cli`):
```bash
B=target/debug/marqdo
$B run examples/hello.mq.md
$B call examples/factorial.mq.md 数学.阶乘 --arg n=5 # prints 120
$B check examples/references.mq.md
$B inspect examples/references.mq.md --refs
$B inspect examples/hello.mq.md --ast --json
$B run examples/math.mq.md # has no main: use call
$B call examples/math.mq.md 数学.计算圆面积
$B run examples/python-foreign.mq.md --allow-foreign # needs python3
$B run examples/multi-module/index.mq.md
(cd examples/multi-module && ../../$B lock && ../../$B sync --locked)
# Plugin demo (build the artifact + refresh its sha256 first)
./examples/plugin-demo/build.sh
$B plugin inspect examples/plugin-demo
$B run examples/plugin-demo/plugin-call.mq.md \
--plugin examples/plugin-demo --trust-native-plugin --allow-plugin demo.solver
```
Notes: `run` defaults to `index.mq.md` and executes `main`; `call` requires a
file and function; root-only examples (for example `math.mq.md`) define no
`main`. `--mode strict` is expected to refuse to run (ADR-0004).
## 8. Docs, ADRs and commit policy
* `spec/` = normative contracts; `docs/` = architecture/decisions/status;
`README.md` = user quickstart; `AGENTS.md` = contributor rules.
* Changes to language semantics, AST mapping, wire format, plugin ABI,
package/lock format, authorization defaults or public CLI behavior require an
**Accepted ADR** (or superseding one) plus spec updates and tests. Use the
blueprint §16.2 template: Context / Decision / Alternatives / Markdown AST
shape / Compatibility / Security / Testing / Migration.
* ADRs are immutable once Accepted: add a new ADR with
`Status: Superseded by ADR-NNNN` rather than rewriting history (typo fixes
aside).
* Docs must describe what the code does. When implementation and blueprint
disagree, document the implementation and flag the divergence explicitly as
a limitation/open item; never invent behavior.
* Do not edit generated/vendored state, and never modify tests to hide a
regression.
* Commit in reviewable units with a message stating the affected spec/ADR and
the commands run. Do not commit built plugin artifacts (`*.so` is
gitignored), the cargo cache, or temp projects.
* There is no git-commit automation in this workspace; contributors commit
manually after review.
## 9. Known gotchas
* `.cache/` is the offline cargo cache/target used in this environment; it is
not part of the deliverable.
* Foreign availability (`python3`/`node`) and a C compiler (`cc`) determine
which tests run; a skipped test is not a passed feature.
* The strict sandbox, plugin worker isolation and package registry are not
implemented. Do not write docs or tests that pretend they are — see
`docs/STATUS.md` and `spec/security.md`.
* Runtime error spans in the CLI currently resolve against the entry file only;
imported-module error locations are approximate (ADR-0006).
* The semantic corpus may pin current quirks (for example the external-link
call code, or a float's last digit). A fixture diff for such a quirk is a
deliberate change, not a free update.