docs: formal specs, ADRs, STATUS matrix, contributor guide and honest README
Complete the documentation set for the M0.5 toolchain, grounded in the code: - spec/references.md: import/reference/call edges, identities, loader rules, anchors, `ref` resolution/ambiguity, `check --refs` validation. - spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal string rule, special floats, limits and exact float round-trip. - spec/security.md: modes (strict refuses without OS isolation), capabilities, path rules, budgets, tracing, foreign process caveats, plugin trust. - spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync, unsupported registry/git/environment work. - spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the implemented CLI plugin-invocation flow with its limitations. - spec/math.md and spec/foreign.md: the implemented math subset and the foreign request/response/timeout/capability contract. - docs/adr/0001..0007: accepted decisions for the span foundation, inert executables outside function bodies, ref ambiguity (S-07), strict refusal, AST normalization, CLI contract and BigInt/typed wire (S-08). - docs/contributor-guide.md: repo map, workflow, fixture/golden updates, quality gate, ADR/commit policy. - docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18 Alpha checklist (conclusion: not yet Alpha-complete), including known divergences rather than overclaims. - README.md rewritten from "design-only starter" to the working implementation with quickstart, implemented phases, deliberate limits and doc index. - AGENTS.md: updated pointers/workflow only; architectural rules unchanged. - Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside functions, unsupported math.simplify/differentiate).
This commit is contained in:
1 parent
fae0cd4b18
commit
0116160399
19 files changed
+3767
-30
No files matched your search
+217
@@ -0,0 +1,217 @@
|
||||
# Marqdo Implementation Status (M0.5)
|
||||
|
||||
Snapshot: **2026-11-20**, workspace version `0.1.0`, plugin ABI `1.0`,
|
||||
`wire_version` 1, Markdown profile `marqdo.md.v1`.
|
||||
|
||||
This file maps the blueprint's phases (`docs/DEVELOPMENT_BLUEPRINT.md` §13) to
|
||||
concrete evidence in the repository and lists honest open items. It is a status
|
||||
report, not a promise. "Done" means implemented **for the documented subset**
|
||||
and covered by tests; it does not mean the blueprint's full ambition is
|
||||
complete.
|
||||
|
||||
Reproduce the evidence:
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test --workspace
|
||||
cargo test -p marqdo-cli
|
||||
cargo fmt --check
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
```
|
||||
|
||||
## 1. Phase matrix
|
||||
|
||||
| Phase | Status | Main evidence |
|
||||
|---|---|---|
|
||||
| **P0** Formal specs + parser goldens | **Done** | `spec/markdown-profile.md`, `spec/mapping.md`, `spec/expression.ebnf`, `spec/evaluation.md`; frozen profile test `crates/marqdo-markdown/src/profile.rs`; 150 AST fixtures + goldens in `tests/conformance/ast/`; 48 compile-error fixtures in `tests/negative/`; `inspect --ast`; `conformance_ast.rs`/`negative.rs` harnesses. |
|
||||
| **P1** Interpreter, expressions, control flow | **Done** | `marqdo-expression`, `marqdo-ir`, `marqdo-frontend`, `marqdo-runtime`; factorial/countdown examples run without Python (`examples/factorial.mq.md`, `examples/hello.mq.md`); 36 execution goldens in `tests/conformance/semantics/`; runtime tests `crates/marqdo-runtime/tests/interpreter.rs`; prose invariance `crates/cli/tests/properties.rs`. |
|
||||
| **P2** Modules, references, CLI experience | **Partial** | Imports/ModuleId/UnitId/anchor/cycle detection + `run/call/check/inspect/fmt` (`crates/cli`), multi-module example and `multi_module_imports_and_cycles` CLI test; `check --refs` reference validation (all loaded documents, on-demand static analysis of link targets) and `inspect --refs` static graph. Missing: cross-module link calls, rename/interface-lock tooling, canonical AST-preserving `fmt` (see §3). |
|
||||
| **P3** Package environment | **Partial** | `marqdo.yaml`/`marqdo.lock` types+validation (`marqdo-package`), `init`, `add`/`remove` (manifest editing), deterministic `lock`, `sync --locked`; CLI test `lock_and_sync_locked_detect_dependency_tampering`; `examples/multi-module`. Missing: git resolution, transitive deps, registry, `tree/update/vendor`, `--offline`, environment materialization. |
|
||||
| **P4** Security core and isolation | **Partial** | `CapabilityHost`, capabilities at execution points (console and math included), project-rooted FS with traversal rejection, budgets, effect trace, pure/project modes, **strict refuses without OS isolation** (`marqdo-sandbox`); CLI test `strict_mode_refuses_without_isolation`; host unit tests. Missing: OS isolation adapter, plugin worker, network enforcement. |
|
||||
| **P5** Math and Foreign | **Partial (documented subset)** | `marqdo-math` exact-rational formula kernel for the grammar in `spec/math.md`; `math.parse/eval/substitute` host ops; `marqdo-foreign` Python/Node one-shot kernels with typed `foreign.call`; tests `crates/marqdo-math/tests/kernel.rs`, `crates/marqdo-foreign/tests/kernels.rs`; examples `math.mq.md`, `python-foreign.mq.md`. Missing: `simplify`/`differentiate`, matrices/series/symbolic algebra, foreign sessions, Python environment management. |
|
||||
| **P6** Plugin ABI and developer experience | **Partial** | Frozen `include/marqdo_plugin_abi_v1.h`; C ABI mirrors/layout assertions, version negotiation, `plugin.yaml` validation, target triple + sha256 verification, wire envelopes, native loader (`marqdo-plugin-abi`); Rust SDK macro (`marqdo-plugin-sdk`); C demo (`examples/plugin-demo`); CLI `plugin inspect` and explicitly trusted in-process invocation (`--plugin`, `--trust-native-plugin`, `--allow-plugin`); tests `plugin_inspect_and_explicit_trusted_invocation`, `crates/marqdo-plugin-abi/tests/loader_unix.rs`. Missing: isolated worker transport, plugin capability bridge, installation, Rust plugin example crate. |
|
||||
| **P7** Performance, release, CI | **Partial** | `.github/workflows/ci.yml` runs fmt/clippy/workspace tests plus CLI example smoke tests (including plugin inspect and invocation) on Linux x86_64. Missing: benchmarks, bytecode backend, cross-platform matrix, signed releases and supply-chain checks. Tree-walk interpreter only. |
|
||||
|
||||
## 2. Feature evidence (selected commands)
|
||||
|
||||
| Command | Verified behavior |
|
||||
|---|---|
|
||||
| `marqdo init demo` | creates `marqdo.yaml` (`>=0.1,<1`), `index.mq.md`, `.gitignore` (`.marqdo/`); refuses existing manifest (exit 1). |
|
||||
| `marqdo check index.mq.md` | static only; 0 when no error diagnostics (warnings allowed); 1 on errors; JSON `command/success/errors/diagnostics`. |
|
||||
| `marqdo run index.mq.md` | executes `main` through the Rust interpreter/host; program stdout on stdout; exit 1 on runtime/compile/strict errors. |
|
||||
| `marqdo call file fn --arg n=5` | typed constant arguments; prints the non-`Unit` return value. |
|
||||
| `marqdo inspect file --ast/--units/--signature/--ir/--refs/--platform --json` | static dashboards; `--platform` reports strict unavailable. |
|
||||
| `marqdo fmt [--check]` | normalizes CRLF and trailing newlines only; exit 1 if `--check` detects a change. |
|
||||
| `marqdo plugin inspect dir` | validates manifest + verifies sha256; never `dlopen`s (`loaded: false`). |
|
||||
| `marqdo run doc --plugin dir --trust-native-plugin --allow-plugin demo.solver` | calls `solver.add(a=20,b=22)` and prints `42`; without trust/grant it exits 1. |
|
||||
| `marqdo lock` / `sync --locked` | deterministic path-dependency lock; `--locked` fails on manifest/dependency drift. |
|
||||
|
||||
Test corpus at this snapshot (`tests/README.md`):
|
||||
|
||||
* 150 AST fixtures + goldens (`tests/conformance/ast/`);
|
||||
* 36 semantic execution fixtures + goldens (`tests/conformance/semantics/`);
|
||||
* 48 compile-time hard-error fixtures, 6 inert-executable warning fixtures,
|
||||
10 runtime-error fixtures (`tests/negative/`);
|
||||
* `crates/cli/tests/` harnesses: `conformance_ast`, `conformance_semantics`,
|
||||
`negative`, `properties`, `integration_cli`;
|
||||
* crate tests across all 15 crates, including real `python3`/`node` foreign
|
||||
tests (skipped if unavailable) and a compiled C plugin test (skipped without
|
||||
`cc`).
|
||||
|
||||
At this snapshot `cargo test -p marqdo-cli` passes all 21 harness tests
|
||||
(2 CLI unit + AST/semantics/negative/properties/integration) and
|
||||
`cargo test --workspace` passes. The fixture counts above match
|
||||
`tests/README.md`; empty `tests/integration`, `tests/properties`,
|
||||
`tests/security` and `tests/abi` directories are reserved for future
|
||||
workstreams — the active corpus lives in `tests/conformance`, `tests/negative`
|
||||
and `crates/cli/tests`.
|
||||
|
||||
## 3. Open items by phase
|
||||
|
||||
### P0
|
||||
|
||||
* Wire floats now round-trip exactly: `serde_json`'s `float_roundtrip`
|
||||
feature is enabled and a Python `math.pi * 9` fixture pins the exact value
|
||||
(`spec/value-wire.md` §3).
|
||||
* Profile change safety relies on the frozen-options unit test; no CI gate
|
||||
regenerates goldens (intentional).
|
||||
|
||||
### P1
|
||||
|
||||
* The blueprint asks for a draft Turing-completeness argument. The language
|
||||
supports mutable locals, branch-first-match, per-item `while` loops and
|
||||
recursion with bounded budgets; a formal encoding write-up is still missing.
|
||||
* The default cooperative stack budget is 128 frames so very deep recursion
|
||||
yields a structured `resource.exhausted` instead of exhausting the debug-build
|
||||
native stack; `--stack` raises it deliberately. A true guard-page/iterative
|
||||
interpreter hardening is still open.
|
||||
|
||||
### P2
|
||||
|
||||
* `fmt` is intentionally minimal (line endings/trailing newlines only), not the
|
||||
blueprint's "canonical AST-equivalent formatting"; no semantic formatting,
|
||||
AST-equivalence check or render-preserving rewrite exists.
|
||||
* Reference validation is opt-in: `check --refs` (and `inspect --refs`) resolve
|
||||
links; plain `check` does not fail on broken links because links are static
|
||||
knowledge references rather than execution inputs.
|
||||
* Link-form calls cannot cross modules; cross-module calls use imported alias
|
||||
expressions. The blueprint's `> [f](./other.mq.md#f)(...)` example is not
|
||||
supported.
|
||||
* `ref("#...")` is module-local and does not accept a selector for units with
|
||||
multiple resources (it errors with `ref.ambiguous`, ADR-0003).
|
||||
* No rename tooling, interface lock/ABI diffing, or incremental cache.
|
||||
* Duplicate/nested project boundaries are not diagnosed (nearest
|
||||
`marqdo.yaml` wins).
|
||||
|
||||
### P3
|
||||
|
||||
* Git dependencies can be declared (`git` + `rev`, validated) but cannot be
|
||||
resolved or locked: `lock` errors "git dependencies are not supported by this
|
||||
build".
|
||||
* No transitive dependency graph, conflict resolution, registry, publishing,
|
||||
signing, `tree`, `update`, `vendor`, `--offline`, or `lock --locked`.
|
||||
* No environment materialization (`.marqdo/env`, caches, kernels, sessions);
|
||||
`sync` verifies but never installs or rewrites.
|
||||
* `add`/`remove` are shallow manifest editors; adding a git dependency succeeds
|
||||
at the manifest level even though `lock` will later refuse it.
|
||||
* JSON errors from `add`/`remove`/`lock`/`sync` are plain stderr even with
|
||||
`--json`.
|
||||
|
||||
### P4
|
||||
|
||||
* **Strict mode cannot run at all**: no OS isolation adapter (Linux
|
||||
namespaces/seccomp/cgroups, Windows Job Object/appcontainer, macOS sandbox)
|
||||
and `worker_processes: false`; `run --mode strict` exits 1 (ADR-0004).
|
||||
* `console` capability is always on; `print` is not capability-checked.
|
||||
* Network capability (`net_hosts`, `check_net`) exists but no host operation
|
||||
performs network I/O and there is no `--allow-net` flag.
|
||||
* Write-path symlink TOCTOU caveat (`spec/security.md` §2.1); symlink/race
|
||||
hardening open.
|
||||
* Budgets do not cover every allocation path; the interpreter's containers are
|
||||
bounded only by fuel and available memory.
|
||||
* Effect trace does not record argument values or paths (privacy-positive, but
|
||||
less diagnostic detail); trace is opt-in and implicitly on for `--json`.
|
||||
* `plugin_handle` wire values are structurally reconstructable, not
|
||||
unforgeable capability tokens.
|
||||
|
||||
### P5
|
||||
|
||||
* Math is a deliberately limited calculator subset: no `simplify`,
|
||||
`differentiate`, matrices, sums/integrals, trig/log functions, units,
|
||||
assumptions or theorem proving. `math.simplify`/`math.differentiate` are
|
||||
host ops that always return `math.unsupported`.
|
||||
* Equation solving is affine-only, single target, target must occur exactly
|
||||
once; other shapes are `math.unsupported`.
|
||||
* Division by zero inside a formula maps to `runtime.type` (via
|
||||
`MathErrorKind::Type`), not `runtime.division_by_zero`; documented divergence.
|
||||
* Foreign kernels are one-shot only: no sessions, state bridge, Python/Node
|
||||
environment management, or network policy for child processes.
|
||||
* Foreign child processes are terminated only when the direct child is killed;
|
||||
grandchildren are not tracked.
|
||||
* Foreign per-call timeout (10 s) and output cap (1 MiB) are not CLI-configurable.
|
||||
|
||||
### P6
|
||||
|
||||
* No isolated plugin worker (length-prefixed IPC) or Wasm transport; only
|
||||
explicitly trusted in-process native loading. A native plugin crash is a host
|
||||
crash (`spec/plugin-abi.md` §1/§9).
|
||||
* No plugin capability bridge from the CLI: plugins are loaded with no
|
||||
capability handler, so every plugin `request_capability` is denied.
|
||||
* Plugin calls are not statically validated; unknown namespaces are runtime
|
||||
`symbol.unknown`. Positional plugin arguments are rejected.
|
||||
* One namespace per plugin (first export prefix); multi-prefix manifests are not
|
||||
reachable beyond the first prefix.
|
||||
* No plugin installation/package management; `plugins:` in `marqdo.yaml` is
|
||||
declarative only. No Rust plugin example crate (SDK has doc examples/tests).
|
||||
* No hot unload/reload (ABI v1 design).
|
||||
|
||||
### P7
|
||||
|
||||
* Performance work is untouched: no benchmark suite, no bytecode backend or
|
||||
differential tests against the tree-walker, no parse/compile caching.
|
||||
* No CI workflows (`.github/` is empty of workflow files); no
|
||||
Windows/macOS/Linux matrix, sanitizers/fuzz gates, signing, release
|
||||
automation or supply-chain scanning.
|
||||
* No published version, no compatibility matrix beyond the frozen ABI/format
|
||||
documents.
|
||||
|
||||
## 4. Blueprint §18 "Alpha Definition of Done" mapping
|
||||
|
||||
| §18 item | Status | Evidence / gap |
|
||||
|---|---|---|
|
||||
| Run/call non-trivial recursive, branch, loop, call, parameter-inference programs without Python | **Met** | factorial/recursion (`examples/factorial.mq.md`, runtime + semantic goldens), countdown loop, multi-module alias call; interpreter is native Rust. |
|
||||
| `.mq.md` readable under the chosen CommonMark/GFM/Math profile; standard AST behavior covered by goldens | **Met** | frozen `marqdo.md.v1`, 150 AST goldens, profile option test, `spec/markdown-profile.md`. |
|
||||
| No heuristic downgrade of execution semantics; accurate source locations for invalid execution nodes | **Mostly met** | hard errors with spans for function-body executable shapes and negative corpus. Gaps: inert executable outside functions is a warning by design (ADR-0002); cross-module runtime error spans resolve against the entry file only; some lazy nested nodes are inert without a warning. |
|
||||
| `check`/`inspect` side-effect free; imports, calls and static references verifiable across files | **Mostly met** | static load/analysis only; imports + cycle detection; cross-module calls validated; `check --refs` validates references across loaded documents and statically analyses link targets; `inspect --refs` prints the entry graph. |
|
||||
| Minimal manifest, lock file, path/Git dependencies and environment sync | **Partial** | manifest/lock/path works, `sync --locked` verifies; git resolution, transitive deps, environment materialization and `--offline` missing. |
|
||||
| All host side effects capability-authorized; untrusted code/native plugins cannot masquerade as strict sandbox | **Mostly met** | capability checks at execution points; strict refuses; native plugins require explicit trust and are documented as unsandboxed. Gaps: network enforcement not implemented (no network builtin is exposed), no worker isolation. |
|
||||
| Formula computational subset explicit/testable; Foreign I/O, timeout and permissions explicit | **Met for the documented subset** | `spec/math.md`, `spec/foreign.md`; math kernel tests; foreign tests (timeout, protocol, output cap, env); permission gating tests/smoke. Simplification/differentiation explicitly unsupported. |
|
||||
| Plugin C ABI v1, Rust SDK, C example, isolated worker basic path, ABI compat error tests | **Partial** | ABI/SDK/C example/loader/ABI-error tests exist; **isolated worker path missing**. |
|
||||
| CI has at least one full research pipeline: Markdown function → Math → Foreign → Plugin → controlled output | **Partial** | CI now runs fmt/clippy/tests and CLI smoke tests; individual math/foreign/plugin paths are covered by examples and tests, but no single end-to-end pipeline document/test exists yet. |
|
||||
| Examples, README, AGENTS and contributor docs allow a first change without legacy context | **Met** | `README.md`, `AGENTS.md`, this guide, `docs/STATUS.md`, specs, ADRs, `tests/README.md`, examples. |
|
||||
|
||||
**Conclusion: the Alpha definition of done is not fully met.** P0/P1 and the
|
||||
documented P5 subset are solid; P2/P3/P4/P6 are functional but partial; P7 is
|
||||
absent. The blocking items for an Alpha claim are the strict sandbox/worker
|
||||
isolation story (P4/P6), git/environment package management (P3), and a CI
|
||||
end-to-end pipeline (P7).
|
||||
|
||||
## 5. What is deliberately limited (do not overclaim)
|
||||
|
||||
* Only a single, frozen Markdown profile (`marqdo.md.v1`); changing parser
|
||||
options/versions is a language change.
|
||||
* Strict mode is a refusal, not an isolation implementation. `project` mode is
|
||||
cooperative and in-process.
|
||||
* Native plugins are trusted in-process code; there is no isolated plugin
|
||||
worker, and a plugin crash takes down the host.
|
||||
* Ints are arbitrary precision; finite floats are binary64 and wire round-trip
|
||||
is exact (`float_roundtrip` enabled), but decimal-vs-binary formatting still
|
||||
applies when displaying values.
|
||||
* Math is a limited exact-rational calculator, not a CAS or a prover.
|
||||
* Foreign kernels are optional, one-shot, unsandboxed child processes.
|
||||
* `fmt` is minimal; `check` does not yet validate Markdown links; link-form
|
||||
calls cannot cross modules.
|
||||
* Package management is path-only in practice; `marqdo.yaml` capabilities are
|
||||
requests, never grants.
|
||||
@@ -0,0 +1,139 @@
|
||||
# ADR-0001: Crate graph and the `marqdo-span` foundation
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
|
||||
## Context
|
||||
|
||||
`docs/DEVELOPMENT_BLUEPRINT.md` §11.1 fixes the dependency direction for the
|
||||
workspace and states: "`markdown ->` zero other Marqdo crate; responsible for a
|
||||
stable view of the standard AST." The implemented workspace instead has
|
||||
fifteen crates, and `marqdo-markdown` depends on `marqdo-span`.
|
||||
|
||||
Why the divergence was necessary:
|
||||
|
||||
* Every layer needs identical byte spans: the Markdown AST
|
||||
(`crates/marqdo-markdown/src/ast.rs`), expression AST
|
||||
(`crates/marqdo-expression`), IR (`crates/marqdo-ir`), diagnostics and runtime
|
||||
errors (`crates/marqdo-value`), effect traces (`crates/marqdo-host`), plugin
|
||||
errors (`crates/marqdo-plugin-abi`) and the CLI renderer.
|
||||
* A `Span` must be the same type across boundaries or every layer needs
|
||||
conversion functions. Conversions are exactly where off-by-one bugs and
|
||||
broken source locations appear, and they would make `Expr`/`Stmt`
|
||||
serde output inconsistent between `inspect --ast` and `inspect --ir`.
|
||||
* The blueprint's "zero Marqdo dependencies" rule exists to keep the parser
|
||||
independent of the semantic layers. A tiny, effect-free span crate preserves
|
||||
that property while satisfying the shared-data-contract requirement.
|
||||
|
||||
## Decision
|
||||
|
||||
1. Add `crates/marqdo-span` as a **leaf** crate with no Marqdo dependencies and
|
||||
only `serde` as an external dependency. It defines:
|
||||
* `Span { start: usize, end: usize }` — half-open byte range into the
|
||||
original UTF-8 source, with `merge`, `shift`, `contains`, `len`;
|
||||
* `LineCol { line, column }` — 1-based, byte columns;
|
||||
* `LineIndex` — precomputed line starts for byte→line/column conversion.
|
||||
2. Treat `Span` as part of the toolchain's shared data contract. Crates that
|
||||
produce or consume source locations depend on `marqdo-span` directly; they
|
||||
do **not** wrap it in local span types.
|
||||
3. `marqdo-markdown` depends on `marqdo-span` (and `comrak`) but on **no
|
||||
semantic crate**. It still owns only the canonical AST view, front matter
|
||||
extraction and source maps; it performs no mapping, resolution or execution.
|
||||
4. Keep the rest of the blueprint's direction:
|
||||
`value -> span`, `expression -> span`, `ir -> span + expression`,
|
||||
`markdown -> span + comrak`, `frontend -> markdown + expression + ir + value`,
|
||||
`host -> value + math + foreign + sandbox`, `runtime -> ir + value + host +
|
||||
expression`, `cli -> all` (composition root). There is no `runtime -> cli`
|
||||
edge and no cycle; `cargo tree` is expected to stay acyclic.
|
||||
5. Do not introduce the blueprint's separate `sema` or `environment` crates in
|
||||
M0.5: semantic analysis lives in `marqdo-frontend`
|
||||
(`analyze`, `calls`, `infer`, `signature`, `units`, `map`), and package/
|
||||
environment formatting lives in `marqdo-package` plus the CLI. Splitting
|
||||
them is a refactor, not a semantic change.
|
||||
|
||||
Actual dependency graph (workspace `Cargo.toml` plus crate manifests):
|
||||
|
||||
```text
|
||||
span serde
|
||||
value span, serde, serde_json, num-bigint, num-traits
|
||||
expression span, serde, num-bigint, unicode-ident
|
||||
markdown span, comrak, serde, serde_yaml, unicode-normalization
|
||||
ir span, expression, serde
|
||||
frontend span, markdown, expression, ir, value, sha2,
|
||||
unicode-normalization, unicode-ident
|
||||
math value, num-bigint, num-rational, num-traits, serde
|
||||
foreign span, value, serde_json, thiserror
|
||||
sandbox serde
|
||||
plugin-abi span, value, serde, serde_json, serde_yaml, thiserror,
|
||||
libloading, sha2, hex
|
||||
plugin-sdk plugin-abi, serde, serde_json
|
||||
host span, value, math, foreign, sandbox, serde, serde_json,
|
||||
thiserror, glob
|
||||
runtime span, value, ir, expression, host, num-bigint,
|
||||
num-rational, num-traits, thiserror
|
||||
package serde, serde_yaml
|
||||
cli span, value, expression, markdown, ir, frontend, math,
|
||||
foreign, plugin-abi, host, runtime, package, sandbox,
|
||||
clap, serde, serde_json, walkdir, hex, sha2
|
||||
```
|
||||
|
||||
## Alternatives
|
||||
|
||||
1. **Put `Span` in `marqdo-value`.** Then `marqdo-markdown` would depend on the
|
||||
runtime value model, which is a semantic layer and pulls in `num-bigint` and
|
||||
`serde_json`. Worse layering than a dedicated leaf.
|
||||
2. **Duplicate `Span` in each crate and convert at boundaries.** Avoids a crate
|
||||
but guarantees conversion code, mismatched serde shapes and duplicated
|
||||
tests; rejected.
|
||||
3. **Define `Span` in `marqdo-markdown` and let others depend on it.** Makes
|
||||
the parser a de-facto foundation crate and lets non-Markdown layers depend
|
||||
on `comrak`'s dependencies; rejected.
|
||||
4. **Use a third-party range type.** Adds a dependency and gives no benefit
|
||||
over ~170 lines; rejected.
|
||||
5. **Keep the blueprint's literal "markdown has zero Marqdo deps".** Would
|
||||
force either duplication or a span type in the parser that cannot be shared
|
||||
with diagnostics; rejected in favor of a documented, narrow exception.
|
||||
|
||||
## Markdown AST shape
|
||||
|
||||
No syntax change. Every `MdNode` still has `span: Span` in the original file
|
||||
(`crates/marqdo-markdown/src/ast.rs`), and the JSON inspection view is
|
||||
`{"kind":"heading","level":2,"span":{"start":0,"end":7},...}`. Spans are
|
||||
half-open byte offsets, BOM-rebased, and byte columns for diagnostics.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* This is a **new repository**; there is no legacy compatibility burden.
|
||||
* Public Rust API: moving `Span` later is a workspace-wide API change but has
|
||||
no language/ABI/package-format impact. It would require updating every
|
||||
`crates/*/Cargo.toml`.
|
||||
* Stable observable contracts (AST JSON `span`, diagnostic JSON `span`, IR
|
||||
`span`) are unchanged by the crate boundary. Changing span semantics
|
||||
(e.g. line/column units) requires a spec change and ADR.
|
||||
* The divergence from blueprint §11.1 is recorded here; future crates should
|
||||
use `marqdo-span` rather than introducing another location type.
|
||||
|
||||
## Security
|
||||
|
||||
No new capability or effect. `marqdo-span` is `serde`-only, performs no I/O and
|
||||
cannot execute anything. Byte offsets are derived from already-parsed source;
|
||||
`LineIndex::line_col` clamps out-of-range offsets instead of panicking.
|
||||
|
||||
## Testing
|
||||
|
||||
* `crates/marqdo-span/src/lib.rs` tests multi-byte UTF-8 line/column mapping,
|
||||
`merge`/`shift`/`contains`, and clamping.
|
||||
* `crates/marqdo-markdown/src/profile.rs` tests BOM rebasing and that every AST
|
||||
node carries a byte span; frontend tests assert diagnostics point at the
|
||||
expected source line/column.
|
||||
* `cargo test --workspace` must pass; `cargo tree` must remain acyclic (no
|
||||
automated CI gate yet — this is a P7 item).
|
||||
|
||||
## Migration & rollout
|
||||
|
||||
Immediate: the crate exists and all layers already use it. Future work may
|
||||
split the frontend into `sema`/`environment` crates; that must preserve the
|
||||
graph above (`runtime` must never depend on `cli`, `markdown` must never depend
|
||||
on semantic crates). If a second span representation is ever needed (for
|
||||
example for generated code), add it as a conversion in `marqdo-span`, not as a
|
||||
per-crate duplicate.
|
||||
@@ -0,0 +1,140 @@
|
||||
# ADR-0002: Executable nodes execute only inside level-2 function bodies
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
|
||||
## Context
|
||||
|
||||
The blueprint (§1.2 execution context, §3.2 control flow, unresolved issue
|
||||
**S-03**) establishes that `Strong` (execute), `Emph` (return), ordered lists
|
||||
(branch), bullet lists (loop) and block quotes (call) have program meaning
|
||||
**inside a function body**, i.e. inside the range of a level-2 heading. It
|
||||
leaves the module top level under-specified: if a document has `**x = 1**`
|
||||
before any function, should that be a hard error, a silent no-op, or an
|
||||
explicit top-level statement?
|
||||
|
||||
Two failure modes had to be avoided:
|
||||
|
||||
1. **Silent execution at module scope.** There is no function frame, no return
|
||||
target and no entry point, so top-level effects would be surprising and
|
||||
would make `check`/`inspect` harder to reason about.
|
||||
2. **Hard error.** Ordinary Markdown documents routinely use `**bold**`,
|
||||
`*italic*`, `1. ordered` and `> quotes` in prose. Treating every such node
|
||||
outside a function as a fatal compile error would make the language hostile
|
||||
to normal documentation and would break the "renders normally" goal.
|
||||
|
||||
The chosen baseline was: executable nodes only execute inside function bodies;
|
||||
elsewhere they are ordinary document content, but this must not happen
|
||||
silently.
|
||||
|
||||
## Decision
|
||||
|
||||
1. A **function body** is the child-block range of a `heading(level=2)` up to
|
||||
the next heading of level ≤ 2 (`crates/marqdo-frontend/src/units.rs`,
|
||||
`FunctionScope.body_start..body_end`). Executable roles are assigned only
|
||||
inside those ranges by `crates/marqdo-frontend/src/map.rs`.
|
||||
2. Outside every function body, `strong`, `emph`, `list` and `block_quote`
|
||||
nodes are **inert document content**. They are not mapped to IR, are not
|
||||
executed, and prose is not inspected for keywords.
|
||||
3. The frontend emits a **warning** `markdown.inert_executable` for every
|
||||
would-be execution node found outside a function body. The document still
|
||||
compiles and runs; only `check`'s diagnostic output reflects the warning.
|
||||
The warning is produced once per top-level node and has a hint to move the
|
||||
construct under a `## function` heading.
|
||||
* `block_quote` → "a block quote invocation ... is inert document content";
|
||||
* `list` → "a list control structure ...";
|
||||
* `paragraph` → the first `strong`/`emph` found in the paragraph
|
||||
("an execution statement" / "a return statement").
|
||||
* warnings are emitted only while scanning the document's direct top-level
|
||||
blocks: headings, tables, code/math blocks and other non-paragraph blocks
|
||||
produce none. A top-level paragraph is walked recursively, so a
|
||||
`strong`/`emph` nested under a link or strikethrough inside that paragraph
|
||||
can still produce the inert warning. Nodes inside a function body produce
|
||||
no warning (they are mapped or rejected by the strict executable-shape
|
||||
rules instead).
|
||||
4. Warnings do not change exit codes: `check` exits 0 when there are no error
|
||||
diagnostics even if warnings exist; `run`/`call` execute normally. JSON
|
||||
`check` output sets `success = (errors == 0)` and includes the warning
|
||||
entries in `diagnostics`.
|
||||
|
||||
Rationale for the nodes covered: these are exactly the block-level constructs
|
||||
that `map.rs` would otherwise turn into statements if a function body existed.
|
||||
Inline nodes inside a paragraph are covered by one paragraph-level warning
|
||||
rather than one per node, which keeps normal prose readable.
|
||||
|
||||
## Alternatives
|
||||
|
||||
1. **Hard error for every executable-looking node outside a function.** Rejected
|
||||
(see context); it forbids emphasis in ordinary Markdown and makes writing a
|
||||
README a compile failure.
|
||||
2. **Silent inert behaviour with no diagnostic.** Rejected: an author could
|
||||
believe top-level `**x = 1**` had executed. The blueprint explicitly wants
|
||||
invalid execution intent surfaced.
|
||||
3. **Execute top-level nodes in an implicit module frame.** Rejected: no
|
||||
defined entry semantics, no return target, would make imported modules
|
||||
potential side-effect roots (violating "modules register definitions only").
|
||||
4. **Compile-time error only when the payload parses as a statement.** Rejected:
|
||||
this is the forbidden heuristic ("parses as code → execute") from blueprint
|
||||
§1.3; role must come from node type and position, not from guessing.
|
||||
5. **Warning per AST node.** Rejected as noisy for nested content; one warning
|
||||
per top-level construct is enough to point at the problem.
|
||||
|
||||
## Markdown AST shape
|
||||
|
||||
No new node kinds. Relevant structure:
|
||||
|
||||
```text
|
||||
document
|
||||
heading(level=2) "f" <-- opens a function body
|
||||
paragraph -> strong(text "x = 1") <-- executes (Stmt::Assign)
|
||||
paragraph (before any heading)
|
||||
strong(text "x = 1") <-- inert; emits markdown.inert_executable
|
||||
list(ordered) <-- inert outside function; warning
|
||||
block_quote <-- inert outside function; warning
|
||||
```
|
||||
|
||||
A level-2 heading without a preceding level-1 heading is a module-root function
|
||||
(`## main`) and its body executes normally. A `###` section inside the function
|
||||
range stays inside the function body; headings never execute.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Warning code `markdown.inert_executable` is part of the CLI diagnostic
|
||||
contract. Renaming it requires an ADR and a CLI version note.
|
||||
* Making this a hard error later would be a breaking language change: documents
|
||||
that currently compile (with warnings) would stop. If that is ever desired it
|
||||
needs a migration plan (e.g. an opt-in strict lint).
|
||||
* No package/ABI/IR shape change: inert nodes are simply absent from the IR.
|
||||
* Prose invariance is preserved: editing narrative text outside functions
|
||||
cannot change the IR.
|
||||
|
||||
## Security
|
||||
|
||||
No effect: inert nodes never reach the interpreter, host or kernels. The
|
||||
warning reduces the chance of a false belief that a side effect happened,
|
||||
which is a security-relevant honesty property. A document cannot cause an
|
||||
execute-by-accident through top-level prose.
|
||||
|
||||
## Testing
|
||||
|
||||
* `crates/marqdo-frontend/src/units.rs::tests::
|
||||
executable_outside_function_warns_but_does_not_execute` asserts the warning
|
||||
and that the function index is still built.
|
||||
* `units.rs` tests for function ranges, nested sections, root-level `## main`
|
||||
and duplicate/empty headings.
|
||||
* `crates/marqdo-frontend/src/analyze.rs` tests hard errors for invalid
|
||||
executable shapes inside function bodies (the companion rule: when a node
|
||||
*is* executable, its shape is strictly checked).
|
||||
* Conformance/negative fixtures under `crates/cli/tests` and `tests/`
|
||||
(work in progress by the fixtures workstream) are expected to cover the
|
||||
warning and the inert behavior; `spec/mapping.md` §4 documents the contract.
|
||||
|
||||
## Migration & rollout
|
||||
|
||||
M0.5 ships with warnings. Rollout guidance:
|
||||
|
||||
* document authors move executable constructs under `## function` headings;
|
||||
* tooling should show the warning in editors (the diagnostic already has a
|
||||
precise span);
|
||||
* if a future release makes inert execution a hard error, add a deprecation
|
||||
window and a `check` flag so existing documents can be migrated.
|
||||
@@ -0,0 +1,157 @@
|
||||
# ADR-0003: `ref` resolution order and `ref.ambiguous` (S-07)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
|
||||
## Context
|
||||
|
||||
`ref("#anchor")` is the only way to turn a heading range into an inert
|
||||
resource value (`Formula` or `Code`) or a `FunctionRef`. The blueprint
|
||||
(§5.2, unresolved issue **S-07**) is explicit that a heading range can contain
|
||||
several formulas or code fences, and that the implementation must **never
|
||||
silently pick the first one**:
|
||||
|
||||
> 一个范围存在多段公式时,默认对象选择必须是明确规则(例如禁止无选择器,要求
|
||||
> `math.formulas(ref(...))` 列出候选;不选"第一条")。
|
||||
|
||||
The failing example is easy to write:
|
||||
|
||||
```markdown
|
||||
## f
|
||||
|
||||
### 公式
|
||||
|
||||
$$a = 1$$
|
||||
|
||||
$$b = 2$$
|
||||
|
||||
**x = ref("#公式")**
|
||||
```
|
||||
|
||||
If `ref` returned `a = 1`, the program would be non-deterministic with respect
|
||||
to document structure (insert an earlier formula and the meaning changes) and
|
||||
would hide a genuine authoring error.
|
||||
|
||||
At the same time the common case — one formula per heading — must stay simple;
|
||||
requiring a selector for every reference would make `examples/math.mq.md`
|
||||
awkward.
|
||||
|
||||
## Decision
|
||||
|
||||
`crates/marqdo-runtime/src/builtins_impl.rs::resolve_ref` implements this
|
||||
algorithm:
|
||||
|
||||
1. The target must be a `Text` beginning with `#`; otherwise
|
||||
`symbol.unknown` ("must be a `#heading` anchor"). `ref` is not a URL
|
||||
resolver and does not cross modules.
|
||||
2. The fragment is looked up in the **current module's anchor map**
|
||||
(`Module::anchors`, GitHub-style slugs computed in
|
||||
`crates/marqdo-frontend/src/units.rs`). `ref` does not fall back to
|
||||
canonical unit names and does not search imported modules.
|
||||
3. Collect the resources attributed to the resolved unit (innermost unit by
|
||||
span, `crates/marqdo-frontend/src/resources.rs`).
|
||||
* **Exactly 1** → return `Formula` or `Code` (with `unit` and `span`
|
||||
provenance).
|
||||
* **0** → if a function's unit equals the resolved unit, return
|
||||
`FunctionRef { module, name }`; otherwise `symbol.unknown`.
|
||||
* **≥ 2** → return **`ref.ambiguous`**:
|
||||
"`#...` contains N resources; select one explicitly instead of relying on
|
||||
the first". Never choose a resource.
|
||||
4. Function references are callable through the normal call path; formulas and
|
||||
code remain inert until `math.eval`/`foreign.*` is explicitly called (those
|
||||
are separately capability-checked).
|
||||
|
||||
Resource attribution is deterministic: a resource belongs to its innermost
|
||||
enclosing heading unit, resources keep document order, and a resource is a
|
||||
Markdown `math`/`math_block` or `code_block` node (inline code is not a
|
||||
resource).
|
||||
|
||||
The `ref.ambiguous` category is `ErrorCategory::RefAmbiguous` with the stable
|
||||
code `ref.ambiguous` (`crates/marqdo-value/src/error.rs`).
|
||||
|
||||
## Alternatives
|
||||
|
||||
1. **First resource wins** (blueprint's rejected baseline). Rejected by S-07:
|
||||
document edits would silently change program meaning, and there is no
|
||||
diagnostic for the author.
|
||||
2. **Always require a selector argument**
|
||||
(`ref("#f", selector="second")`). Rejected for M0.5: no selector syntax or
|
||||
addressing scheme is specified yet, and the single-resource case is by far
|
||||
the common one. A future ADR can add an explicit selector while keeping the
|
||||
ambiguity rule.
|
||||
3. **Error on every heading with ≥ 2 resources at compile time**, even if never
|
||||
referenced. Rejected: a document may legitimately contain several formulas
|
||||
for prose and never `ref` them; only the reference is ambiguous.
|
||||
4. **Return a list of all candidates.** Rejected for v1: it changes the return
|
||||
type and forces every caller to handle a selector; a dedicated API
|
||||
(`math.formulas(ref(...))`) would be a new spec, not a silent widening.
|
||||
5. **Cross-module `ref`.** Rejected for v1: the wire/identity model and
|
||||
`ModuleId` resolution live in the loader; `ref` is a pure runtime builtin
|
||||
over the current module. Cross-module resources are an open item.
|
||||
|
||||
## Markdown AST shape
|
||||
|
||||
```text
|
||||
heading(level=2) "f"
|
||||
heading(level=3) "公式"
|
||||
math_block "/$$ a = 1 $$/" -> Resource::Formula { unit: <#f/公式> }
|
||||
math_block "/$$ b = 2 $$/" -> Resource::Formula { unit: <#f/公式> }
|
||||
paragraph -> strong("x = ref(\"#公式\")")
|
||||
```
|
||||
|
||||
Anchors are separate from unit paths: `#公式` is a GitHub slug for the heading
|
||||
name, while the canonical `UnitId` is `<module>#f/公式`. Colliding slugs get
|
||||
`-1`/`-2` suffixes; canonical ids never do.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* `ref.ambiguous` is a stable runtime code; callers may branch on it.
|
||||
* The single-resource and function-reference cases are frozen. Adding a
|
||||
selector must be additive (new argument or new builtin) and must not change
|
||||
the ≥ 2 error into a silent choice.
|
||||
* `ref` target syntax (`#fragment`) is frozen; changing it to accept file paths
|
||||
would need a new ADR.
|
||||
* No package/ABI/wire change: formula/code values already carry
|
||||
`unit`/`span`/`content_hash`.
|
||||
|
||||
Example behavior, verified by
|
||||
`crates/marqdo-runtime/tests/interpreter.rs::ambiguous_ref_resource_is_an_error`:
|
||||
|
||||
```text
|
||||
$ marqdo run index.mq.md # heading with two $$...$$ blocks and ref("#公式")
|
||||
runtime error[ref.ambiguous]: `#公式` contains 2 resources; select one explicitly instead of relying on the first
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
* `ref` is a pure builtin: no host call, no capability required, no side
|
||||
effect. It only returns inert values or a function reference.
|
||||
* An inert `Formula`/`Code` does not execute. Evaluation/execution requires a
|
||||
separate explicit call (`math.eval`, `foreign.run/call`), which is capability
|
||||
checked at the host boundary.
|
||||
* Refusing ambiguity prevents a class of "wrong formula executed" mistakes,
|
||||
which matters for scientific documents where formulas are evidence.
|
||||
|
||||
## Testing
|
||||
|
||||
* `crates/marqdo-runtime/tests/interpreter.rs::ambiguous_ref_resource_is_an_error`
|
||||
drives a real document through frontend + runtime and asserts the
|
||||
`ref.ambiguous` code.
|
||||
* `crates/marqdo-frontend/src/resources.rs` tests attribution to innermost
|
||||
functions, multiple resources kept in one unit, and stable SHA-256 code
|
||||
hashes.
|
||||
* `crates/marqdo-markdown`/`frontend` tests cover anchors and slug collisions.
|
||||
* `examples/math.mq.md` is the positive single-formula case; it runs under
|
||||
`marqdo run` only via a function (the example itself defines callable
|
||||
functions, no `main`).
|
||||
* Missing anchor/not-a-function cases are covered by the `symbol.unknown`
|
||||
paths; conformance fixtures are being added by the fixtures workstream.
|
||||
|
||||
## Migration & rollout
|
||||
|
||||
No migration needed for documents with exactly one resource per referenced
|
||||
heading. Documents that relied on "first formula" behavior (there are none in
|
||||
the examples) must split headings or wait for an explicit selector API. If a
|
||||
selector is added later, keep `ref("#...")` behavior unchanged and introduce
|
||||
the selector as a new argument/builtin, with tests for both the ambiguous and
|
||||
the selected case.
|
||||
@@ -0,0 +1,133 @@
|
||||
# ADR-0004: Strict mode refuses to run without OS isolation
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
|
||||
## Context
|
||||
|
||||
Blueprint §9 defines three execution modes and one non-negotiable rule:
|
||||
|
||||
> `strict`: untrusted documents/code/plugins — independent restricted process,
|
||||
> OS-level resource and syscall/file/network restrictions; if unavailable,
|
||||
> **refuse to run**.
|
||||
|
||||
A cooperative in-process capability check cannot contain hostile native code
|
||||
or an unrestricted child process. If `strict` silently degraded to `project`,
|
||||
users would believe they had isolation they did not have. That is a security
|
||||
bug, not a missing feature.
|
||||
|
||||
M0.5 implements capability checks, budgets and process supervision but **no OS
|
||||
isolation adapter** (no Linux namespaces/seccomp/cgroups, no Windows Job
|
||||
Object/appcontainer, no macOS sandbox), and the isolated plugin worker
|
||||
transport is not implemented either.
|
||||
|
||||
## Decision
|
||||
|
||||
1. `crates/marqdo-sandbox` is honest about platform support:
|
||||
* `PlatformCapabilities::detect()` always reports
|
||||
`StrictIsolation::Unavailable { reason }` in this build, with a
|
||||
platform-specific reason (Linux: "no OS isolation adapter
|
||||
(namespaces/seccomp/cgroups) is implemented in this build"; macOS/Windows/
|
||||
other targets have analogous messages);
|
||||
* `worker_processes: false`.
|
||||
2. `marqdo_sandbox::enforce(mode, platform)`:
|
||||
* `pure` and `project` → `Ok(())`;
|
||||
* `strict` → `Err(SandboxError::StrictUnavailable { reason })` unless
|
||||
`platform.strict.is_available()`.
|
||||
3. `run`/`call` call `enforce` **before** constructing the host, loading
|
||||
plugins or executing anything. On failure they print
|
||||
"refusing to run in strict mode: ..." (or a JSON `{"success":false,
|
||||
"error":...}` envelope) and exit 1.
|
||||
4. There is **no fallback**: the CLI never downgrades a requested strict run to
|
||||
project mode. `SandboxMode::parse("strict")` succeeds (the mode name is
|
||||
valid); only execution is refused.
|
||||
5. `inspect --platform` exposes the detection report so users can see why:
|
||||
`{"os":"linux","arch":"x86_64","strict":{"state":"unavailable","reason":...},
|
||||
"worker_processes":false}`.
|
||||
6. The rest of the system is labelled accordingly: `project` mode is
|
||||
cooperative, in-process, and is not a sandbox; foreign kernels are
|
||||
process-isolated but not confined; native plugins run in-process and require
|
||||
explicit trust (ADR-0006/security spec).
|
||||
|
||||
## Alternatives
|
||||
|
||||
1. **Best-effort strict** (run with the same checks as project). Rejected: it
|
||||
directly violates the blueprint's "cannot provide → refuse" rule and hides a
|
||||
missing security boundary.
|
||||
2. **Warn and continue.** Rejected for the same reason; a warning is not a
|
||||
sandbox, and CI/agents may not read warnings.
|
||||
3. **Implement a Linux namespaces/seccomp adapter now.** Out of scope for
|
||||
M0.5: it needs platform-specific code, integration tests, resource limits,
|
||||
CI on multiple kernels, and a way to supervise the worker protocol. This is
|
||||
P4 work and belongs in its own ADR when implemented.
|
||||
4. **Use Wasm/WASI for strict.** Attractive for plugins later, but it does not
|
||||
sandbox arbitrary Python/Node processes or native `.so` files, and the
|
||||
runtime is not implemented. Rejected for now.
|
||||
5. **Claim strict via the process isolation of foreign kernels.** Rejected:
|
||||
a child process inherits the user's privileges and filesystem/network
|
||||
access; changing cwd and clearing env is hygiene, not confinement.
|
||||
|
||||
## Markdown AST shape
|
||||
|
||||
No language or AST change. A strict run is rejected before any document is
|
||||
executed (after static loading/diagnostics); the CLI contract is:
|
||||
|
||||
```text
|
||||
$ marqdo run index.mq.md --mode strict
|
||||
error: refusing to run in strict mode: no OS isolation adapter
|
||||
(namespaces/seccomp/cgroups) is implemented in this build. Strict mode
|
||||
requires verified OS-level isolation and never falls back to an
|
||||
unrestricted process.
|
||||
$ echo $?
|
||||
1
|
||||
```
|
||||
|
||||
(Exact wording comes from
|
||||
`crates/marqdo-sandbox/src/lib.rs::SandboxError::display`.)
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Accepting `--mode strict` as a valid *name* is part of the CLI contract; the
|
||||
refusal is a documented behavior, not a parse error. A future release that
|
||||
implements isolation flips the detection report and starts executing strict
|
||||
runs — that is the intended upgrade path.
|
||||
* Exit code 1 (runtime/security refusal) is stable; usage errors are 2.
|
||||
* `PlatformCapabilities` JSON shape (`state: available|unavailable`) is
|
||||
versioned by documentation and should not change without an ADR.
|
||||
* No package/ABI/wire impact.
|
||||
|
||||
## Security
|
||||
|
||||
* The guarantee "requested strict → refused rather than downgraded" is
|
||||
enforced in code (`enforce`), not documentation.
|
||||
* Users cannot accidentally believe untrusted docs are sandboxed by this
|
||||
build; the platform report is the evidence.
|
||||
* Project mode remains explicitly non-sandboxed. Native plugins (in-process)
|
||||
and foreign child processes are recorded as trust decisions, not isolation.
|
||||
* Secrets: strict refusal happens before plugin loading or foreign spawning,
|
||||
so no untrusted code is touched by a refused command.
|
||||
|
||||
## Testing
|
||||
|
||||
* `crates/marqdo-sandbox/src/lib.rs::tests::
|
||||
strict_refuses_when_isolation_is_unavailable` asserts the refusal and that
|
||||
`pure`/`project` still succeed.
|
||||
* `mode_names_round_trip` covers parsing/printing.
|
||||
* CLI behavior (`--mode strict` exit 1, message) is covered by manual/CLI
|
||||
smoke checks; a CLI integration test should be added with the conformance
|
||||
harness.
|
||||
* `inspect --platform` JSON is exercised manually and in the CLI help/example
|
||||
documentation; it should get a golden test in the fixture workstream.
|
||||
|
||||
## Migration & rollout
|
||||
|
||||
When an OS isolation adapter lands:
|
||||
|
||||
1. implement a `StrictIsolation::Available { mechanism }` report and a real
|
||||
supervisor/worker;
|
||||
2. add integration tests for filesystem/network/process restrictions and
|
||||
resource limits per platform, plus the platform capability matrix;
|
||||
3. update `spec/security.md`, this ADR (or supersede it), `docs/STATUS.md` and
|
||||
the CLI docs;
|
||||
4. only then change `PlatformCapabilities::detect`; do not add a flag that
|
||||
bypasses the refusal before that work is done.
|
||||
@@ -0,0 +1,174 @@
|
||||
# ADR-0005: AST normalization (Setext headings, task items, code-span ticks, strikethrough)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
|
||||
## Context
|
||||
|
||||
The language derives program meaning from the **canonical AST**, never from
|
||||
source spelling (`spec/markdown-profile.md`, `spec/mapping.md` §1). The parser
|
||||
(`comrak =1.0.0`, frozen options) normalizes several syntaxes before Marqdo
|
||||
sees them, and a few normalizations are lossy:
|
||||
|
||||
* `# H` and Setext `H\n===` both become a heading node; the source spelling is
|
||||
lost unless recorded;
|
||||
* GFM task list items (`- [x] text`) produce a task-specific item node;
|
||||
* inline code spans are stored as a literal without the backtick delimiters,
|
||||
but source spans include them;
|
||||
* `~~strikethrough~~` is enabled by the profile and wraps arbitrary inlines.
|
||||
|
||||
The language needs a single, documented answer per question, because node type
|
||||
and structural position decide executable roles.
|
||||
|
||||
## Decision
|
||||
|
||||
The profile (`marqdo.md.v1`) and the canonical AST
|
||||
(`crates/marqdo-markdown/src/ast.rs`, `src/profile.rs`) freeze these rules:
|
||||
|
||||
1. **Setext headings are headings.** `# name` and a Setext level-1/2 underline
|
||||
both produce `heading { level, setext }`; `setext` is inspection-only and
|
||||
**never affects semantics**. `## name` and a Setext level-2 heading are both
|
||||
functions. The profile keeps `parse.ignore_setext = false`.
|
||||
2. **Task-list items are `item` nodes with a task flag.**
|
||||
`item { task: Some(true|false) }` for GFM task items, `task: None`
|
||||
otherwise. Node type stays `item`, so list structure (ordered vs bullet)
|
||||
still determines branch/loop roles; the checkbox flag is inert data and does
|
||||
not change execution. A task list item in an executable position still
|
||||
requires its condition header like any other item.
|
||||
3. **Code spans carry delimiter tick counts.** Inline `code { ticks }` records
|
||||
the number of backticks, and the source span includes them; the literal is
|
||||
the inner payload. This lets the frontend map expression payload spans back
|
||||
to exact source bytes for diagnostics. "Ticks" is not user-visible program
|
||||
meaning.
|
||||
4. **Strikethrough is inert/lazy.** `strikethrough` is an inert wrapper: it is
|
||||
never traversed for execution, its content is prose, and a `strong`/`emph`
|
||||
inside it does not execute. It is not an error to write it; the language
|
||||
simply does not use formatting as a control-flow mechanism here. (A code
|
||||
span or math inside it is likewise inert.)
|
||||
5. **Unmodelled nodes are preserved** as `other { tag }` with children and raw
|
||||
literal. They are inert but never silently dropped from `inspect --ast`.
|
||||
6. **Literal payloads are kept** on inert leaves: `text`, `code`, `code_block`,
|
||||
`math`, `math_block`, `html_block`, `html_inline`, `front_matter`.
|
||||
7. **Other frozen normalizations:**
|
||||
* `**x**`/`__x__` and `*x*`/`_x_` are the same node kinds (`strong`/`emph`);
|
||||
* ordered delimiters `1.`/`1)` and bullets `-`/`*`/`+` are the same
|
||||
`list`/`item` structure; role depends on `ordered`, not marker;
|
||||
* tight vs loose lists only affect rendering (`tight` is recorded, inert);
|
||||
* `link`/`image` keep `url`/`title`; `table` keeps column alignments,
|
||||
`table_row` keeps `header`; `heading` keeps `level`/`setext`;
|
||||
* a UTF-8 BOM at offset 0 is stripped and all spans are re-based onto the
|
||||
original file;
|
||||
* front matter is recognized only at the document start (after BOM), with
|
||||
`---` opening and `---`/`...` closing; a later `---` is a thematic break
|
||||
or Setext underline.
|
||||
|
||||
These rules are part of profile id `marqdo.md.v1`. Changing them (or bumping
|
||||
the comrak version/options) requires a profile id bump, updated goldens and this
|
||||
ADR being superseded.
|
||||
|
||||
## Alternatives
|
||||
|
||||
1. **Distinguish Setext vs ATX headings semantically.** Rejected: identical AST
|
||||
structure outside the marker must have identical meaning (`## H` and Setext
|
||||
H2 are the same function). Keeping `setext` only for inspection avoids
|
||||
silently diverging programs when a document changes style.
|
||||
2. **Introduce a separate `task_item` node kind.** Rejected: it would require
|
||||
every list-role check to special-case two item kinds and could be used to
|
||||
hide a control structure behind a checkbox. Normalizing to `item { task }`
|
||||
keeps list structure authoritative.
|
||||
3. **Strip the tick information and store only the literal.** Rejected: payload
|
||||
spans returned by the expression parser are relative to the concatenated
|
||||
payload; without delimiter lengths, error spans cannot be mapped back to
|
||||
file bytes precisely.
|
||||
4. **Traverse strikethrough content for execution.** Rejected: it is a lazy
|
||||
region by profile; executing inside it would make formatting semantics
|
||||
surprising ("struck-through code runs"). It would also create a second way
|
||||
to hide executable nodes.
|
||||
5. **Drop unknown parser nodes.** Rejected: `inspect --ast` must show what the
|
||||
parser produced; dropping syntax hides profile drift.
|
||||
6. **Preserve raw source and re-parse with a custom parser.** Rejected by the
|
||||
project's core rule: one standard Markdown parser, no second Markdown AST.
|
||||
|
||||
## Markdown AST shape
|
||||
|
||||
Frozen JSON shapes (`inspect --ast --json`):
|
||||
|
||||
```json
|
||||
{"kind":"heading","level":2,"setext":false,"span":{"start":0,"end":7},
|
||||
"children":[{"kind":"text","span":{"start":3,"end":7},"literal":"阶乘"}]}
|
||||
```
|
||||
|
||||
```json
|
||||
{"kind":"list","ordered":false,"tight":true,"span":{...},
|
||||
"children":[{"kind":"item","task":true,"span":{...},"children":[...]}]}
|
||||
```
|
||||
|
||||
```json
|
||||
{"kind":"code","ticks":2,"span":{"start":10,"end":16},"literal":"a * b"}
|
||||
```
|
||||
|
||||
```json
|
||||
{"kind":"strikethrough","span":{...},"children":[{"kind":"text",...}]}
|
||||
```
|
||||
|
||||
Attribute order in the JSON is fixed by the serializer; goldens are regenerated
|
||||
only through the explicit update workflow.
|
||||
|
||||
### 5.1 Known interaction to keep in mind
|
||||
|
||||
Inside a function body, a `**strong**` (or an ordered/bullet list) nested under
|
||||
`strikethrough` is inert and never executes; because the mapper does not
|
||||
traverse strikethrough, it also produces no warning. Outside function bodies,
|
||||
the `markdown.inert_executable` scan walks a top-level paragraph recursively,
|
||||
so such a nested node may still produce the inert warning even though it would
|
||||
not execute if the paragraph were inside a function. This is consistent with
|
||||
"formatting never hides execution", but an author who intends execution must
|
||||
not wrap it in strikethrough. Warnings for lazy-region content inside functions
|
||||
are a possible future refinement; no behavior change is planned without an ADR.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Profile id `marqdo.md.v1` is frozen with comrak `=1.0.0` and the exact
|
||||
extension flags; the unit test
|
||||
`crates/marqdo-markdown/src/profile.rs::tests::profile_options_are_frozen`
|
||||
asserts the option table (tables, tasklist, strikethrough, math dollars,
|
||||
front matter; all other extensions disabled).
|
||||
* Adding an AST field is a JSON output change for `inspect --ast`; it needs a
|
||||
goldens update and a profile note. Changing a semantic rule (e.g. executing
|
||||
inside strikethrough) is a language change and requires an ADR.
|
||||
* No package/ABI/wire impact.
|
||||
|
||||
## Security
|
||||
|
||||
* The lazy-region rules ensure inline formatting inside headings, tables,
|
||||
links, code, math, HTML and front matter never executes — a context-aware
|
||||
visitor is used (`crates/marqdo-frontend/src/map.rs` only descends from
|
||||
paragraphs into `strong`/`emph`, and ignores every other inline node).
|
||||
* Strikethrough cannot be used to smuggle executable content; it is inert.
|
||||
* Normalizing task items to `item` prevents a checkbox from hiding a
|
||||
branch/loop header; the condition-header check still applies.
|
||||
* No new capabilities or host calls.
|
||||
|
||||
## Testing
|
||||
|
||||
* `crates/marqdo-markdown/src/profile.rs`:
|
||||
* `profile_options_are_frozen` — exact extension/parse option table;
|
||||
* `setext_headings_are_heading_nodes` — Setext produces `heading { setext: true }`;
|
||||
* `tables_and_task_lists_are_supported` — task items carry `task: Some(_)`;
|
||||
* span/BOM tests.
|
||||
* Frontend mapping tests use code spans as conditions and payloads, exercising
|
||||
`ticks`/payload span mapping.
|
||||
* `spec/markdown-profile.md` §5/§6 is the normative JSON shape; AST goldens
|
||||
under `crates/cli/tests` (fixtures workstream) regenerate only with the
|
||||
documented update command.
|
||||
* Conformance fixtures are expected to include `**a**` vs `__a__`, Setext vs
|
||||
ATX, task/list variants and strikethrough inertness.
|
||||
|
||||
## Migration & rollout
|
||||
|
||||
No migration for documents: Setext and ATX were always the same function under
|
||||
this profile. When comrak is upgraded or the extension set changes, bump the
|
||||
profile id, regenerate AST goldens, re-run all specs/ADRs that reference node
|
||||
shapes, and document the change here (superseding this ADR if a rule changes).
|
||||
Code-span `ticks` and `task` are additive inspection fields; keep them optional
|
||||
in consumers that parse the JSON.
|
||||
@@ -0,0 +1,225 @@
|
||||
# ADR-0006: CLI output, exit-code and JSON contract
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
|
||||
## Context
|
||||
|
||||
CI, scripts and coding agents need a stable CLI contract, not just human
|
||||
output. The blueprint (§12) requires: all commands available with `--json`,
|
||||
non-zero error exit codes, program stdout kept separate from diagnostics, and
|
||||
machine-readable error codes. The implemented CLI must be documented precisely
|
||||
because scripts will depend on stream separation and exit status.
|
||||
|
||||
Implementation: `crates/cli/src/main.rs` (clap definitions, process exit),
|
||||
`crates/cli/src/commands.rs` (behavior), `crates/cli/src/output.rs`
|
||||
(diagnostic/JSON rendering), `crates/cli/src/args.rs` (`--arg` constants).
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Commands and flags (implemented surface)
|
||||
|
||||
| Command | Arguments / flags | What it does |
|
||||
|---|---|---|
|
||||
| `init [dir]` | — | create `marqdo.yaml`, `index.mq.md`, `.gitignore`; refuses an existing manifest |
|
||||
| `run [file]` | `--mode`, `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin`, `--fuel`, `--stack`, `--max-output`, `--timeout`, `--plugin`, `--trust-native-plugin`, `--json`, `--trace` | execute `main` (or default `index.mq.md`) |
|
||||
| `call <file> <function>` | `--arg NAME=VALUE` (repeatable) plus all `run` flags | execute one named function |
|
||||
| `check [file]` | `--json` | static check; never executes |
|
||||
| `inspect [file]` | `--ast`, `--units`, `--signature`, `--ir`, `--refs`, `--platform`, `--json` | static inspection; never executes |
|
||||
| `fmt [file]` | `--check` | normalize CRLF and trailing newlines |
|
||||
| `plugin inspect <dir-or-yaml>` | `--json` | validate manifest + verify artifact hash; never loads code |
|
||||
| `add <name>` | `--path`, `--git`, `--rev`, `--json` | edit `marqdo.yaml` |
|
||||
| `remove <name>` | `--json` | edit `marqdo.yaml` (and prune a stale lock entry) |
|
||||
| `lock` | `--json` | write deterministic `marqdo.lock` |
|
||||
| `sync` | `--locked`, `--json` | verify the lock is current |
|
||||
|
||||
There is no `test`, `tree`, `update`, `vendor`, `add` for plugins, `kernel`
|
||||
command or `--project`/`--offline` flag in this build. `run` defaults the
|
||||
file to `index.mq.md`; `check`/`inspect`/`fmt` do too. `call` requires an
|
||||
explicit file path plus a function name.
|
||||
|
||||
`--mode` is only on `run`/`call` and accepts `pure|project|strict`
|
||||
(default `project`). `--trust-native-plugin`/`--plugin` are only meaningful on
|
||||
`run`/`call`. `inspect --platform` still requires a document argument (or
|
||||
defaults to `index.mq.md`), because project loading happens first.
|
||||
|
||||
### 2. Exit codes
|
||||
|
||||
| Code | Meaning |
|
||||
|---|---|
|
||||
| `0` | success (for `check`: no **error** diagnostics; warnings do not fail) |
|
||||
| `1` | a checked document has errors; a runtime/security failure; strict refusal; plugin load/inspect failure; lock/sync mismatch; `fmt --check` would reformat; `init` found an existing manifest; `add` duplicate; `remove` missing dependency; `sync` stale lock |
|
||||
| `2` | usage error (clap), unreadable entry file, invalid `--mode` name, invalid capability combination (e.g. `--allow-read` in `pure`), invalid `--arg` value, unreadable manifest for project commands, unreadable file for `fmt` |
|
||||
|
||||
Codes are stable contract. Diagnostic **codes** (for example
|
||||
`markdown.invalid_construct`, `expr.parse`, `symbol.unknown`,
|
||||
`permission.denied`, `ref.ambiguous`) are stable; message wording is not.
|
||||
|
||||
### 3. Stream separation
|
||||
|
||||
* **Program output** (`打印`/`print`) goes to stdout in text mode. In JSON mode
|
||||
it is embedded as the `stdout` string; it is never interleaved with JSON.
|
||||
* **Diagnostics and effect traces** go to stderr in text mode.
|
||||
* `run` in text mode writes only program output to stdout (no implicit return
|
||||
value). `call` in text mode additionally prints the returned value (using
|
||||
`Value::display`) when it is not `Unit`.
|
||||
* `check` text mode: diagnostics to stderr; `checked N module(s): ok` to stdout
|
||||
on success; a `check failed: ...` summary to stderr on failure.
|
||||
* `inspect` prints pretty JSON-ish structures to stdout; load errors to stderr.
|
||||
* JSON mode prints exactly one pretty JSON object to stdout per invocation.
|
||||
`add`/`remove`/`lock`/`sync` still print errors as plain `error: ...` on
|
||||
stderr even with `--json` (JSON success envelope only) — an inconsistency to
|
||||
fix before the contract is declared final.
|
||||
|
||||
### 4. JSON output shapes
|
||||
|
||||
`check`:
|
||||
|
||||
```json
|
||||
{"command":"check","success":true,"errors":0,
|
||||
"diagnostics":[{"file":"index.mq.md","module":"index.mq.md",
|
||||
"line":3,"column":5,"severity":"error","code":"expr.parse",
|
||||
"message":"...","hints":["..."]}]}
|
||||
```
|
||||
|
||||
`inspect`:
|
||||
|
||||
```json
|
||||
{"command":"inspect","module":"index.mq.md",
|
||||
"ast":{...},"units":[...],"signatures":[...],"ir":[...],
|
||||
"refs":[...],"platform":{...}}
|
||||
```
|
||||
|
||||
Only the requested sections are present. With no section flags, a `summary`
|
||||
with `modules`, `functions`, `units`, `resources` is emitted instead.
|
||||
|
||||
`run`/`call` (success):
|
||||
|
||||
```json
|
||||
{"command":"run","success":true,
|
||||
"value":{"type":"text","value":"..."},
|
||||
"stdout":"program stdout text",
|
||||
"trace":[{"op":"print","module":"index.mq.md",
|
||||
"span":{"start":43,"end":57},"duration_micros":3,"result":"ok"}]}
|
||||
```
|
||||
|
||||
Note: `call --json` also emits `"command":"run"` in this build (shared
|
||||
implementation). Consumers should accept both names or key on the presence of
|
||||
`diagnostics`/`value`; this is a known quirk.
|
||||
|
||||
`run`/`call` (failure): `{"command":"run","success":false,"stdout":...,
|
||||
"error":{"code":"...","message":"...","span":{"line":..,"column":..,"file":..},
|
||||
"frames":[...]},"trace":[...]}`. Runtime error spans are currently resolved
|
||||
against the **entry file** only, even when the error came from an imported
|
||||
module; frames carry function names, not module ids. This is a known precision
|
||||
limitation.
|
||||
|
||||
`plugin inspect`:
|
||||
|
||||
```json
|
||||
{"command":"plugin.inspect","success":true,"plugin":"demo.solver",
|
||||
"version":"0.1.0","abi":"^1.0","abi_compatibility":"compatible",
|
||||
"target":"x86_64-unknown-linux-gnu",
|
||||
"artifact":{"file":"solver.so","path":"...","sha256":"...","verified":true},
|
||||
"exports":[{"name":"solver.add","signature":"($a: Int, $b: Int) -> Int"}],
|
||||
"loaded":false}
|
||||
```
|
||||
|
||||
`load_project` failures with `--json` print `{"success":false,"error":"..."}`
|
||||
without a `command` field.
|
||||
|
||||
There is no `json_schema_version` field yet. The blueprint asks for a
|
||||
versioned JSON schema; until it exists, treat these keys as the stable subset
|
||||
and ignore unknown keys. Adding keys is not considered breaking; renaming or
|
||||
removing one is.
|
||||
|
||||
### 5. `--arg` values
|
||||
|
||||
`--arg NAME=VALUE` parses `VALUE` with the **expression grammar restricted to
|
||||
constants** (`crates/cli/src/args.rs`): integers, floats, double-quoted text
|
||||
(escapes supported), `true`/`false`/`None`, lists, records, and unary `-`/`+`
|
||||
over constants. Calls and identifiers are rejected ("must be a literal
|
||||
constant"). Text must be double-quoted; there is no bare-word fallback.
|
||||
|
||||
### 6. Determinism and safety of read-only commands
|
||||
|
||||
* `check`/`inspect` never execute project code (ADR context; see
|
||||
`spec/security.md` §7).
|
||||
* `plugin inspect` never loads plugin code.
|
||||
* `fmt` only rewrites the file when not `--check`; exit 1 when `--check`
|
||||
detects a difference.
|
||||
|
||||
## Alternatives
|
||||
|
||||
1. **Use exit code 1 for all failures.** Rejected: scripts need to distinguish
|
||||
usage errors (2) from document/runtime failures (1); it also mirrors common
|
||||
CLI conventions and clap's own usage exit code.
|
||||
2. **Print JSON even for usage errors.** Rejected for now: clap owns usage
|
||||
output before our code runs; wrapping it would mean intercepting clap
|
||||
errors. Documented as a limitation.
|
||||
3. **Interleave program output and JSON.** Rejected: it makes JSON
|
||||
unparseable. Program stdout is embedded as a string instead.
|
||||
4. **Return values from `run` on stdout in text mode.** Rejected: `run` is the
|
||||
batch entry; implicit value printing would pollute program output. `call`
|
||||
prints the return value exactly when non-`Unit`.
|
||||
5. **Human-only messages with numeric exit codes only.** Rejected: diagnostics
|
||||
and runtime errors have stable machine codes in both text and JSON.
|
||||
|
||||
## Markdown AST shape
|
||||
|
||||
The CLI itself does not change the AST. What it exposes:
|
||||
|
||||
* `inspect --ast` = canonical `MdNode` tree (see ADR-0005);
|
||||
* `inspect --units` = `UnitInfo[]` (`marqdo-ir`);
|
||||
* `inspect --signature` = `{name, local_name, signature: explicit|inferred,
|
||||
params:[{name,type,has_default}]}`;
|
||||
* `inspect --ir` = `Function[]` with `Stmt`/`Expr` and spans;
|
||||
* `inspect --refs` = link edge array (`spec/references.md` §4);
|
||||
* `inspect --platform` = `PlatformCapabilities` JSON.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Exit codes, stream separation and the documented JSON keys are the public
|
||||
CLI contract. Changing them requires this ADR to be superseded and a
|
||||
changelog entry.
|
||||
* The old "design-only starter" README listed planned commands (`add`,
|
||||
`lock && sync`, `plugin inspect`); all listed commands except a versioned
|
||||
JSON schema are now implemented as described here.
|
||||
* Diagnostic code renames are breaking for CI; message changes are not.
|
||||
* No package/ABI/wire changes.
|
||||
|
||||
## Security
|
||||
|
||||
* `--json`/`--trace` enable effect tracing; traces contain op/module/span/
|
||||
duration/result summary, never argument values (no secrets) — see
|
||||
`spec/security.md` §4.
|
||||
* JSON mode embeds program stdout, which is program-controlled data; consumers
|
||||
must not treat it as trusted.
|
||||
* Refusing to run with compile errors happens before host creation, so JSON
|
||||
failure output cannot leak effects.
|
||||
* Strict-mode refusal and capability denials produce stable codes for CI.
|
||||
|
||||
## Testing
|
||||
|
||||
* CLI behavior is currently covered by manual smoke checks and library-level
|
||||
tests; the fixtures workstream is adding CLI integration tests under
|
||||
`crates/cli/tests` (goldens/negative cases) and `tests/`.
|
||||
* `crates/cli/src/args.rs` unit tests cover typed `--arg` parsing and rejection
|
||||
of non-constant values.
|
||||
* `crates/cli/src/output.rs` has no unit tests; diagnostic JSON is exercised
|
||||
through check/run smoke tests.
|
||||
* Add contract tests for: exit codes per failure class, stdout/stderr
|
||||
separation, JSON key presence, `call --json` command naming, and runtime
|
||||
error span from a non-entry module before calling the schema "final".
|
||||
|
||||
## Migration & rollout
|
||||
|
||||
1. Keep the current shapes stable while adding tests.
|
||||
2. Add a `schema_version` to JSON envelopes and document the versioned schema;
|
||||
then document that unknown keys may be added (forward compatible) while
|
||||
existing keys are frozen.
|
||||
3. Fix the `call --json` `"command":"run"` quirk and JSON errors in
|
||||
`add/remove/lock/sync`; both are additive/corrective and should be
|
||||
announced in the changelog.
|
||||
4. If a command is renamed or an exit code changes, update this ADR, the
|
||||
README quickstart, `docs/STATUS.md` and CI scripts in the same change.
|
||||
@@ -0,0 +1,178 @@
|
||||
# ADR-0007: Arbitrary-precision `Int` and the typed wire encoding (S-08)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
|
||||
## Context
|
||||
|
||||
The blueprint (§3.1 and unresolved issue **S-08**) requires a numeric model
|
||||
that does not silently lose precision at language, JSON/ABI and math
|
||||
boundaries. It recommends arbitrary-precision integers for `Int` and type tags
|
||||
plus string forms for large integers on the wire.
|
||||
|
||||
Three independent places could lose information:
|
||||
|
||||
1. **Language arithmetic.** A fixed-width `i64` would make `Int` silently
|
||||
overflow or wrap. Factorials, powers and scientific constants routinely
|
||||
exceed 64 bits. The expression grammar already has arbitrary-length integer
|
||||
literals (`spec/expression.ebnf`).
|
||||
2. **JSON / wire.** JSON has one number type (IEEE-754 double in most parsers).
|
||||
Encoding `12345678901234567890` as a JSON number loses digits in
|
||||
JavaScript/Go consumers; encoding `1` as a number makes `Int` and `Float`
|
||||
indistinguishable.
|
||||
3. **Math kernel.** The exact-rational engine needs an integer type that is
|
||||
the *same* type as `Value::Int`, or every formula/binding crossing the host
|
||||
boundary needs a lossy conversion.
|
||||
|
||||
The chosen dependency set (`Cargo.toml`) is:
|
||||
|
||||
```toml
|
||||
num-bigint = { version = "0.4", features = ["serde"] }
|
||||
num-rational = { version = "0.4", features = ["serde"] }
|
||||
num-traits = "0.2"
|
||||
```
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. `Int` is arbitrary precision (`num-bigint::BigInt`)
|
||||
|
||||
* `Value::Int(BigInt)` (`crates/marqdo-value/src/value.rs`) is the only integer
|
||||
representation in the runtime and the expression AST
|
||||
(`crates/marqdo-expression/src/ast.rs`).
|
||||
* Expression integer literals are arbitrary-precision decimal; there is no
|
||||
fixed-width parsing step.
|
||||
* Arithmetic (`crates/marqdo-runtime/src/value_ops.rs`):
|
||||
* `+ - *` on two `Int` are exact `BigInt` operations;
|
||||
* `/` **always** produces `Float` (exact ratio converted to binary64), so
|
||||
there is no integer division surprise;
|
||||
* `//` and `%` require `Int` operands; floor division is mathematical floor
|
||||
and `a % b == a - b * (a // b)`, with the sign following the divisor;
|
||||
* mixed `Int`/`Float` arithmetic produces `Float`; converting an `Int`
|
||||
larger than 2^53 may lose precision (documented, not silently exported as
|
||||
an `Int`);
|
||||
* `Int`/`Float` comparisons are **exact**: the float is treated as the exact
|
||||
binary rational it is (`BigRational::from_float`), so
|
||||
`9007199254740993 > 9007199254740992.0` is `true`;
|
||||
* `**` on `Int` is exact exponentiation, with a negative exponent rejected
|
||||
as a type error and guards `MAX_EXACT_EXPONENT = 100_000` and
|
||||
`MAX_POWER_BITS = 1_000_000` (`resource.exhausted`).
|
||||
* There is no truthiness and no implicit string↔number conversion; `整数`/`int`
|
||||
performs explicit conversions and rejects non-integral floats.
|
||||
|
||||
### 2. `num-bigint` 0.4 is pinned to match `num-rational` 0.4
|
||||
|
||||
`num-rational`'s generic `Ratio<T>` is instantiated as
|
||||
`Ratio<num_bigint::BigInt>` in `crates/marqdo-math/src/ast.rs`
|
||||
(`pub type Rational = Ratio<BigInt>`). The two crates must resolve to the same
|
||||
`num-bigint` version, otherwise `BigInt` values from the value model cannot be
|
||||
passed to `Ratio` without conversion (or a duplicate type). The workspace pins
|
||||
both at the `0.4` line and the lockfile resolves one `num-bigint`. Upgrading
|
||||
either crate requires checking that this shared-type invariant still holds and
|
||||
running the math + runtime test suites.
|
||||
|
||||
### 3. The wire encoding never sends an integer as a JSON number
|
||||
|
||||
Canonical encoding (`crates/marqdo-value/src/wire.rs`; full contract in
|
||||
`spec/value-wire.md`):
|
||||
|
||||
```json
|
||||
{"type":"int","decimal":"12345678901234567890"}
|
||||
```
|
||||
|
||||
* Every value is a JSON object with a `"type"` tag.
|
||||
* `Int` uses a base-10 string. All digits, sign and magnitude survive any JSON
|
||||
consumer including JavaScript.
|
||||
* `Float` uses a JSON number only when finite; `NaN`, `Infinity`,
|
||||
`-Infinity` use explicit `"special"` tags.
|
||||
* JSON numbers appear only in non-value positions (`span.start/end`,
|
||||
`plugin_handle.id`).
|
||||
* Decoding is validated: unknown tags, missing fields, invalid decimals, wrong
|
||||
types and limits are `WireError`s, never coercions.
|
||||
* The same encoding is used for plugin ABI request/response values, foreign
|
||||
kernel stdin `input`, foreign typed return values and `run --json` values.
|
||||
|
||||
### 4. Math results preserve exactness where possible
|
||||
|
||||
The math kernel keeps exact rationals and returns `Value::Int` exactly when the
|
||||
exact result is an integer, otherwise `Value::Float`. Decimal literals are
|
||||
exact rationals (`0.25` = `1/4`), and `\frac{1}{3} * 3` is exactly `1`. Constants
|
||||
(`\pi`, `\e`) and non-perfect square roots introduce binary64 approximations and
|
||||
the result is a `Float` (`spec/math.md` §3).
|
||||
|
||||
## Alternatives
|
||||
|
||||
1. **`i64`/`u64` integers with overflow errors.** Rejected: common programs
|
||||
(factorials, powers) would need explicit bignum workarounds; the language
|
||||
would inherit fixed-width semantics not implied by its arithmetic.
|
||||
2. **`i128` integers.** Same problem, just a bigger ceiling; and the wire
|
||||
precision problem remains for values beyond 2^53.
|
||||
3. **Encode integers as JSON numbers and hope consumers use bignum JSON.**
|
||||
Rejected: the JSON standard/ecosystem does not guarantee it; JavaScript's
|
||||
`JSON.parse` loses digits; this is the exact S-08 failure.
|
||||
4. **Encode every number as a string without type tags.** Rejected: loses the
|
||||
`Int`/`Float` distinction and makes all consumers parse strings; type tags
|
||||
are explicit and cheap.
|
||||
5. **Use `f64` for all numbers.** Rejected: contradicts exact `Int`
|
||||
semantics, the math kernel's exact rational requirement, and the
|
||||
no-silent-precision-loss rule.
|
||||
6. **Different integer crates in value vs math.** Rejected: forces lossy or
|
||||
string conversions at every kernel boundary and duplicates tests.
|
||||
|
||||
## Markdown AST shape
|
||||
|
||||
No AST change. Numeric literals live inside executable payloads only (code
|
||||
spans/plain text of `strong`/`emph`/conditions). A `math` node is inert LaTeX;
|
||||
its numeric interpretation is the math kernel's job. The JSON inspection view
|
||||
of IR/AST contains spans as JSON numbers, not user numeric values.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* `Int` semantics (arbitrary precision, `/` → `Float`, exact comparisons,
|
||||
`//`/`%` sign rules) are frozen language behavior; changing them requires an
|
||||
ADR and conformance fixtures.
|
||||
* `wire_version = 1` with typed tags and decimal-string integers is frozen.
|
||||
Changing the integer representation is a wire-breaking change and needs a
|
||||
new `wire_version` plus plugin ABI considerations.
|
||||
* The dependency pin may move within `0.4` for security fixes, provided
|
||||
`num-bigint` remains a single resolved version shared with `num-rational`.
|
||||
* Package/lock formats are unaffected.
|
||||
|
||||
## Security
|
||||
|
||||
* Untrusted wire data cannot trigger huge allocations through the integer
|
||||
parser without also counting against value depth/string limits: a decimal
|
||||
string is bounded by `max_string_bytes` (16 MiB default) and `BigInt` parsing
|
||||
is linear. The current decoder does not separately cap the *digit count* of
|
||||
an `Int`; the envelope/string caps are the effective bound (open item).
|
||||
* Arithmetic can grow memory; the runtime bounds it with fuel and the
|
||||
`Int` power guards (`MAX_EXACT_EXPONENT`, `MAX_POWER_BITS`) and the math
|
||||
kernel bounds exact numerator/denominator bits (`MAX_RESULT_BITS`).
|
||||
* Exact `Int`/`Float` comparison prevents a class of "looks equal but is not"
|
||||
authorization or verification bugs.
|
||||
* No new capabilities; numeric parsing is pure.
|
||||
|
||||
## Testing
|
||||
|
||||
* `crates/marqdo-value/src/value.rs`: `big_integers_are_arbitrary_precision`,
|
||||
`float_to_bigint_requires_integral_value`, display/format tests.
|
||||
* `crates/marqdo-value/src/wire.rs`: `big_integers_keep_all_digits`,
|
||||
`special_floats_are_tagged`, scalar round trips, untrusted-input limits.
|
||||
* `crates/marqdo-runtime/src/value_ops.rs`: exact integer arithmetic,
|
||||
`/`→Float, floor/mod sign rules, exact mixed comparisons, power guards,
|
||||
division-by-zero.
|
||||
* `crates/marqdo-math/tests/kernel.rs`: exact big integer arithmetic, exact
|
||||
rational division, decimal literals as rationals, `pi`/`sqrt` approximating
|
||||
to `Float`, `Int` vs `Float` result typing.
|
||||
* `crates/marqdo-plugin-abi/src/wire.rs`: request round trips preserving tags
|
||||
and refusing coercible-looking values beyond `i64`.
|
||||
* `crates/marqdo-foreign/tests/kernels.rs`: typed foreign return values decode
|
||||
exactly.
|
||||
|
||||
## Migration & rollout
|
||||
|
||||
No migration: the implementation has used `BigInt` and the typed wire format
|
||||
since M0.5. For future dependency upgrades: run `cargo test --workspace`,
|
||||
verify one resolved `num-bigint` (`cargo tree -i num-bigint`), and re-run wire
|
||||
compatibility tests. For any change to the wire integer encoding, bump
|
||||
`wire_version`, update `spec/value-wire.md`, the plugin ABI documentation and
|
||||
the foreign protocol in one coordinated change.
|
||||
@@ -0,0 +1,286 @@
|
||||
# Marqdo Contributor Guide
|
||||
|
||||
This guide is for people changing the Marqdo CLI workspace. It assumes you have
|
||||
read `AGENTS.md` (the non-negotiable architectural rules) and
|
||||
`docs/DEVELOPMENT_BLUEPRINT.md` §0/§1/§11/§13/§14. Formal contracts live in
|
||||
`spec/`; decisions live in `docs/adr/`.
|
||||
|
||||
## 1. Repository map
|
||||
|
||||
```text
|
||||
marqdo-cli/
|
||||
├── Cargo.toml workspace (15 crates, resolver 2)
|
||||
├── AGENTS.md operator rules for contributors/agents
|
||||
├── README.md user-facing quickstart and limits
|
||||
├── docs/
|
||||
│ ├── DEVELOPMENT_BLUEPRINT.md architecture baseline
|
||||
│ ├── STATUS.md implementation matrix (P0–P7)
|
||||
│ ├── contributor-guide.md this file
|
||||
│ └── adr/0001..0007-*.md accepted decisions
|
||||
├── spec/
|
||||
│ ├── markdown-profile.md frozen parser profile `marqdo.md.v1`
|
||||
│ ├── mapping.md AST → program roles
|
||||
│ ├── expression.ebnf executable payload grammar
|
||||
│ ├── evaluation.md values, numbers, control flow, builtins
|
||||
│ ├── references.md imports/references/calls/ref()
|
||||
│ ├── value-wire.md typed wire_version 1 values
|
||||
│ ├── security.md modes, capabilities, budgets, trust
|
||||
│ ├── package-format.md marqdo.yaml / marqdo.lock / init
|
||||
│ ├── plugin-abi.md C ABI v1, SDK, host lifecycle
|
||||
│ ├── math.md Formula subset and math.* semantics
|
||||
│ └── foreign.md Python/Node kernel protocol
|
||||
├── crates/
|
||||
│ ├── marqdo-span/ Span, LineCol, LineIndex (leaf)
|
||||
│ ├── marqdo-value/ Value, diagnostics, errors, wire encoding
|
||||
│ ├── marqdo-expression/ lexer/parser/AST for executable payloads
|
||||
│ ├── marqdo-markdown/ comrak profile, canonical AST, front matter
|
||||
│ ├── marqdo-ir/ Module/Unit/Function/Stmt IR, builtin table
|
||||
│ ├── marqdo-frontend/ units, map, signature, infer, calls, resources
|
||||
│ ├── marqdo-math/ Formula parser/evaluator (exact rationals)
|
||||
│ ├── marqdo-foreign/ Python/Node one-shot kernel supervisor
|
||||
│ ├── marqdo-sandbox/ modes + platform capability detection
|
||||
│ ├── marqdo-plugin-abi/ C ABI mirrors, manifest, artifact, loader, wire
|
||||
│ ├── marqdo-plugin-sdk/ Rust plugin SDK and `mq_plugin_entry!`
|
||||
│ ├── marqdo-host/ CapabilityHost, budgets, FS, math/foreign/plugin adapters
|
||||
│ ├── marqdo-runtime/ tree-walk interpreter, value ops, builtins
|
||||
│ ├── marqdo-package/ marqdo.yaml / marqdo.lock types + discovery
|
||||
│ └── cli/ composition root: flags, commands, loader, output
|
||||
├── include/marqdo_plugin_abi_v1.h frozen C ABI v1 header
|
||||
├── examples/ hello, factorial, math, references, python-foreign,
|
||||
│ multi-module/, plugin-demo/
|
||||
├── tests/
|
||||
│ ├── conformance/ast/ *.mq.md + *.ast.json AST goldens
|
||||
│ ├── conformance/semantics/ *.mq.md + *.expected.json execution goldens
|
||||
│ ├── negative/ compile-time + runtime hard-error fixtures
|
||||
│ ├── README.md corpus layout and golden workflow
|
||||
│ ├── integration/ properties/ security/ abi/ reserved dirs (see STATUS)
|
||||
└── crates/cli/tests/ Rust harnesses for the corpus and CLI
|
||||
```
|
||||
|
||||
## 2. Dependency direction (must stay acyclic)
|
||||
|
||||
```text
|
||||
span -> serde
|
||||
value -> span
|
||||
expression -> span
|
||||
markdown -> span, comrak (no semantic crate!)
|
||||
ir -> span, expression
|
||||
frontend -> markdown, expression, ir, value
|
||||
host -> value, math, foreign, sandbox
|
||||
runtime -> ir, value, host, expression
|
||||
math -> value
|
||||
foreign -> value
|
||||
plugin-abi -> value
|
||||
plugin-sdk -> plugin-abi
|
||||
package -> serde only
|
||||
sandbox -> serde only
|
||||
cli -> everything (composition root)
|
||||
```
|
||||
|
||||
Rules (from blueprint §11.1, adjusted by ADR-0001):
|
||||
|
||||
* `markdown` owns the only Markdown parser; it must not depend on semantic
|
||||
crates, the host or the runtime. The single exception to "zero Marqdo deps"
|
||||
is `marqdo-span`, which is a dep-free leaf shared by all layers.
|
||||
* `frontend`/`ir`/`runtime` never depend on `cli`.
|
||||
* `runtime` never parses Markdown and never performs I/O directly: all effects
|
||||
go through `marqdo_host::Host`.
|
||||
* `host` never depends on the CLI; kernels/plugins are adapted here.
|
||||
* The CLI is the composition root: flag parsing, project loading, rendering and
|
||||
process exit. It must not become a second interpreter.
|
||||
* Check the graph with `cargo tree` (or `cargo tree -i <crate>`); `runtime ->
|
||||
cli`, `markdown -> frontend`, and dependency cycles are review blockers.
|
||||
|
||||
## 3. Adding a feature or fixture (workflow)
|
||||
|
||||
The project uses a fixture-first workflow (blueprint §13/§14, AGENTS §
|
||||
"Development workflow"). The expected sequence is:
|
||||
|
||||
1. **Find the rule.** Identify the affected spec section and blueprint section.
|
||||
If the change alters observable language semantics, package/lock format,
|
||||
ABI, authorization defaults or public CLI behavior, write/supersede an ADR
|
||||
first (`docs/adr/`, template in blueprint §16.2).
|
||||
2. **Write a failing fixture.** Add a small `.mq.md` positive and/or negative
|
||||
case under the right directory:
|
||||
* AST shape: `tests/conformance/ast/NNN-name.mq.md` + golden
|
||||
`NNN-name.ast.json`;
|
||||
* semantics: `tests/conformance/semantics/NNN-name.mq.md` +
|
||||
`NNN-name.expected.json` (typed wire `args`/`value` + captured `stdout`);
|
||||
* hard error: `tests/negative/NNN-name.mq.md` + `{"codes":[...]}`;
|
||||
* inert-executable warning: `tests/negative/warnings/`;
|
||||
* runtime error: `tests/negative/runtime/` with `{"function","args","codes"}`
|
||||
(optional `grants`, `stack_limit`).
|
||||
Run the relevant harness (see §6) and confirm it fails for the expected
|
||||
reason.
|
||||
3. **Update the spec.** `spec/*.md` is normative. If implementation and spec
|
||||
disagree, fix whichever is wrong and note the tradeoff. Do not silently
|
||||
invent behavior in code and leave the spec stale.
|
||||
4. **Implement the smallest module.** Keep the layers separated: parser
|
||||
(`marqdo-markdown`), semantic mapping (`marqdo-frontend`), IR
|
||||
(`marqdo-ir`), interpreter (`marqdo-runtime`), effects (`marqdo-host`),
|
||||
composition (`cli`). Never add a second Markdown parser, a regex-based
|
||||
syntax decision, or a host bypass.
|
||||
5. **Refresh goldens deliberately** (§4) and run the full gate (§6).
|
||||
6. **Write the tradeoff/limitations** in `docs/STATUS.md` when a P-level claim
|
||||
changes, and in the PR description (paths, commands run, skipped tests).
|
||||
|
||||
If a fixture reveals an implementation bug, fix the code, not the expected
|
||||
behavior. Negative fixtures must assert hard errors — never weaken them to
|
||||
warnings or prose fallbacks.
|
||||
|
||||
## 4. Golden update workflow
|
||||
|
||||
AST goldens (`tests/conformance/ast/*.ast.json`) are the serialized canonical
|
||||
AST from `marqdo_markdown::parse(source).root`. Regenerate **only** for an
|
||||
intentional, ADR-backed profile change:
|
||||
|
||||
```bash
|
||||
UPDATE_GOLDENS=1 cargo test -p marqdo-cli --test conformance_ast
|
||||
```
|
||||
|
||||
The update run rewrites every golden, prints the count and passes. Re-run
|
||||
without the variable and review the diff: **a golden change is a language
|
||||
change, not a bug fix**. A fixture without a golden and a golden without a
|
||||
fixture are both errors.
|
||||
|
||||
Semantic goldens (`*.expected.json`): `UPDATE_SEMANTICS_GOLDENS=1` exists for
|
||||
authoring convenience, but never use it blindly — verify every value against the
|
||||
fixture source first. Invalid, inconsistent or unverified expected values defeat
|
||||
the corpus.
|
||||
|
||||
Negative goldens are hand-written expected code lists (exact sorted multiset for
|
||||
compile-time errors; warning suites; runtime suites). Do not auto-generate them.
|
||||
|
||||
Adding/removing a fixture and its golden must happen in the same change.
|
||||
Fixtures are data: tests must not commit, copy or rewrite files outside a
|
||||
throw-away temp directory. `tests/README.md` is the in-repo reference for corpus
|
||||
counts, harness responsibilities and known observations.
|
||||
|
||||
## 5. Coding conventions
|
||||
|
||||
* Rust edition 2021, `rust-version = 1.82`, workspace version `0.1.0`.
|
||||
* `unsafe` is forbidden in pure crates (`marqdo-math`, `marqdo-expression` use
|
||||
`#![forbid(unsafe_code)]`); the ABI/loader crates contain the only
|
||||
`unsafe` code and every block has a SAFETY comment explaining the contract.
|
||||
* No panics on untrusted input: Markdown, expressions, LaTeX, wire data,
|
||||
manifests and foreign output all produce structured errors. `unwrap()`/
|
||||
`expect()` are acceptable only for genuinely impossible invariants or in
|
||||
tests.
|
||||
* Stable diagnostic/runtime **codes** are part of the contract; human messages
|
||||
are not. Add new codes to `crates/marqdo-value/src/diagnostic.rs` and map
|
||||
categories in `error.rs`.
|
||||
* Values cross layers via `marqdo_value::Value` and the typed wire format
|
||||
(`spec/value-wire.md`); never serialize `Int` as a JSON number.
|
||||
* `check`/`inspect` must remain side-effect free: no execution, no foreign
|
||||
process, no plugin `dlopen`, no file writes.
|
||||
* Prose and executable nodes mix freely; prose never causes an action.
|
||||
* Formatting: `rustfmt.toml` is authoritative; keep `cargo fmt --check` clean.
|
||||
|
||||
## 6. Build, test and quality gate
|
||||
|
||||
The workspace is offline-cached. Use the workspace cache as documented in
|
||||
`tests/README.md` / AGENTS.md:
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
```
|
||||
|
||||
Commands:
|
||||
|
||||
```bash
|
||||
cargo build -p marqdo-cli # binary: target/debug/marqdo
|
||||
cargo test --workspace # all crate unit + integration tests
|
||||
cargo test -p marqdo-cli # conformance corpus + CLI integration tests
|
||||
cargo fmt --check
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
```
|
||||
|
||||
Harness map (`crates/cli/tests/`):
|
||||
|
||||
| Harness | Proves |
|
||||
|---|---|
|
||||
| `conformance_ast.rs` | frozen profile parses fixtures into checked-in AST (kinds, attributes, literals, spans, order) |
|
||||
| `conformance_semantics.rs` | frontend + IR + runtime + host produce expected wire value/stdout |
|
||||
| `negative.rs` | invalid executable shapes are hard errors; warning and runtime suites |
|
||||
| `properties.rs` | prose-invariance (pure narrative edits do not change executable IR modulo spans) |
|
||||
| `integration_cli.rs` | real CLI contract: init/check/run/call/inspect/fmt, multi-module, lock/sync, plugin trust and invocation |
|
||||
|
||||
Crate-level tests worth knowing:
|
||||
|
||||
* `marqdo-expression` — grammar/priority/nesting/limit tests;
|
||||
* `marqdo-math/tests/kernel.rs` — exact arithmetic, bounds, equations, unsupported constructs;
|
||||
* `marqdo-foreign/tests/kernels.rs` — real python3/node (skips if absent), timeout, output caps, env, wire protocol;
|
||||
* `marqdo-plugin-abi/tests/loader_unix.rs` — compiles the C demo with `cc`, loads it, ABI-mismatch/negative cases (skips without `cc`);
|
||||
* `marqdo-runtime/tests/interpreter.rs` — recursion, loops, returns, budgets, ambiguity;
|
||||
* `marqdo-host` unit tests — capabilities, project FS, budgets.
|
||||
|
||||
Tests that need Python/Node/`cc` print a skip notice and pass when unavailable;
|
||||
record such skips in the PR.
|
||||
|
||||
## 7. Running examples
|
||||
|
||||
From the repository root (after `cargo build -p marqdo-cli`):
|
||||
|
||||
```bash
|
||||
B=target/debug/marqdo
|
||||
$B run examples/hello.mq.md
|
||||
$B call examples/factorial.mq.md 数学.阶乘 --arg n=5 # prints 120
|
||||
$B check examples/references.mq.md
|
||||
$B inspect examples/references.mq.md --refs
|
||||
$B inspect examples/hello.mq.md --ast --json
|
||||
$B run examples/math.mq.md # has no main: use call
|
||||
$B call examples/math.mq.md 数学.计算圆面积
|
||||
$B run examples/python-foreign.mq.md --allow-foreign # needs python3
|
||||
$B run examples/multi-module/index.mq.md
|
||||
(cd examples/multi-module && ../../$B lock && ../../$B sync --locked)
|
||||
|
||||
# Plugin demo (build the artifact + refresh its sha256 first)
|
||||
./examples/plugin-demo/build.sh
|
||||
$B plugin inspect examples/plugin-demo
|
||||
$B run examples/plugin-demo/plugin-call.mq.md \
|
||||
--plugin examples/plugin-demo --trust-native-plugin --allow-plugin demo.solver
|
||||
```
|
||||
|
||||
Notes: `run` defaults to `index.mq.md` and executes `main`; `call` requires a
|
||||
file and function; root-only examples (for example `math.mq.md`) define no
|
||||
`main`. `--mode strict` is expected to refuse to run (ADR-0004).
|
||||
|
||||
## 8. Docs, ADRs and commit policy
|
||||
|
||||
* `spec/` = normative contracts; `docs/` = architecture/decisions/status;
|
||||
`README.md` = user quickstart; `AGENTS.md` = contributor rules.
|
||||
* Changes to language semantics, AST mapping, wire format, plugin ABI,
|
||||
package/lock format, authorization defaults or public CLI behavior require an
|
||||
**Accepted ADR** (or superseding one) plus spec updates and tests. Use the
|
||||
blueprint §16.2 template: Context / Decision / Alternatives / Markdown AST
|
||||
shape / Compatibility / Security / Testing / Migration.
|
||||
* ADRs are immutable once Accepted: add a new ADR with
|
||||
`Status: Superseded by ADR-NNNN` rather than rewriting history (typo fixes
|
||||
aside).
|
||||
* Docs must describe what the code does. When implementation and blueprint
|
||||
disagree, document the implementation and flag the divergence explicitly as
|
||||
a limitation/open item; never invent behavior.
|
||||
* Do not edit generated/vendored state, and never modify tests to hide a
|
||||
regression.
|
||||
* Commit in reviewable units with a message stating the affected spec/ADR and
|
||||
the commands run. Do not commit built plugin artifacts (`*.so` is
|
||||
gitignored), the cargo cache, or temp projects.
|
||||
* There is no git-commit automation in this workspace; contributors commit
|
||||
manually after review.
|
||||
|
||||
## 9. Known gotchas
|
||||
|
||||
* `.cache/` is the offline cargo cache/target used in this environment; it is
|
||||
not part of the deliverable.
|
||||
* Foreign availability (`python3`/`node`) and a C compiler (`cc`) determine
|
||||
which tests run; a skipped test is not a passed feature.
|
||||
* The strict sandbox, plugin worker isolation and package registry are not
|
||||
implemented. Do not write docs or tests that pretend they are — see
|
||||
`docs/STATUS.md` and `spec/security.md`.
|
||||
* Runtime error spans in the CLI currently resolve against the entry file only;
|
||||
imported-module error locations are approximate (ADR-0006).
|
||||
* The semantic corpus may pin current quirks (for example the external-link
|
||||
call code, or a float's last digit). A fixture diff for such a quirk is a
|
||||
deliberate change, not a free update.
|
||||
Reference in new issue
Block a user