Complete the documentation set for the M0.5 toolchain, grounded in the code: - spec/references.md: import/reference/call edges, identities, loader rules, anchors, `ref` resolution/ambiguity, `check --refs` validation. - spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal string rule, special floats, limits and exact float round-trip. - spec/security.md: modes (strict refuses without OS isolation), capabilities, path rules, budgets, tracing, foreign process caveats, plugin trust. - spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync, unsupported registry/git/environment work. - spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the implemented CLI plugin-invocation flow with its limitations. - spec/math.md and spec/foreign.md: the implemented math subset and the foreign request/response/timeout/capability contract. - docs/adr/0001..0007: accepted decisions for the span foundation, inert executables outside function bodies, ref ambiguity (S-07), strict refusal, AST normalization, CLI contract and BigInt/typed wire (S-08). - docs/contributor-guide.md: repo map, workflow, fixture/golden updates, quality gate, ADR/commit policy. - docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18 Alpha checklist (conclusion: not yet Alpha-complete), including known divergences rather than overclaims. - README.md rewritten from "design-only starter" to the working implementation with quickstart, implemented phases, deliberate limits and doc index. - AGENTS.md: updated pointers/workflow only; architectural rules unchanged. - Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside functions, unsupported math.simplify/differentiate).
6.9 KiB
ADR-0001: Crate graph and the marqdo-span foundation
Status: Accepted Date: 2026-11-20
Context
docs/DEVELOPMENT_BLUEPRINT.md §11.1 fixes the dependency direction for the
workspace and states: "markdown -> zero other Marqdo crate; responsible for a
stable view of the standard AST." The implemented workspace instead has
fifteen crates, and marqdo-markdown depends on marqdo-span.
Why the divergence was necessary:
- Every layer needs identical byte spans: the Markdown AST
(
crates/marqdo-markdown/src/ast.rs), expression AST (crates/marqdo-expression), IR (crates/marqdo-ir), diagnostics and runtime errors (crates/marqdo-value), effect traces (crates/marqdo-host), plugin errors (crates/marqdo-plugin-abi) and the CLI renderer. - A
Spanmust be the same type across boundaries or every layer needs conversion functions. Conversions are exactly where off-by-one bugs and broken source locations appear, and they would makeExpr/Stmtserde output inconsistent betweeninspect --astandinspect --ir. - The blueprint's "zero Marqdo dependencies" rule exists to keep the parser independent of the semantic layers. A tiny, effect-free span crate preserves that property while satisfying the shared-data-contract requirement.
Decision
- Add
crates/marqdo-spanas a leaf crate with no Marqdo dependencies and onlyserdeas an external dependency. It defines:Span { start: usize, end: usize }— half-open byte range into the original UTF-8 source, withmerge,shift,contains,len;LineCol { line, column }— 1-based, byte columns;LineIndex— precomputed line starts for byte→line/column conversion.
- Treat
Spanas part of the toolchain's shared data contract. Crates that produce or consume source locations depend onmarqdo-spandirectly; they do not wrap it in local span types. marqdo-markdowndepends onmarqdo-span(andcomrak) but on no semantic crate. It still owns only the canonical AST view, front matter extraction and source maps; it performs no mapping, resolution or execution.- Keep the rest of the blueprint's direction:
value -> span,expression -> span,ir -> span + expression,markdown -> span + comrak,frontend -> markdown + expression + ir + value,host -> value + math + foreign + sandbox,runtime -> ir + value + host + expression,cli -> all(composition root). There is noruntime -> cliedge and no cycle;cargo treeis expected to stay acyclic. - Do not introduce the blueprint's separate
semaorenvironmentcrates in M0.5: semantic analysis lives inmarqdo-frontend(analyze,calls,infer,signature,units,map), and package/ environment formatting lives inmarqdo-packageplus the CLI. Splitting them is a refactor, not a semantic change.
Actual dependency graph (workspace Cargo.toml plus crate manifests):
span serde
value span, serde, serde_json, num-bigint, num-traits
expression span, serde, num-bigint, unicode-ident
markdown span, comrak, serde, serde_yaml, unicode-normalization
ir span, expression, serde
frontend span, markdown, expression, ir, value, sha2,
unicode-normalization, unicode-ident
math value, num-bigint, num-rational, num-traits, serde
foreign span, value, serde_json, thiserror
sandbox serde
plugin-abi span, value, serde, serde_json, serde_yaml, thiserror,
libloading, sha2, hex
plugin-sdk plugin-abi, serde, serde_json
host span, value, math, foreign, sandbox, serde, serde_json,
thiserror, glob
runtime span, value, ir, expression, host, num-bigint,
num-rational, num-traits, thiserror
package serde, serde_yaml
cli span, value, expression, markdown, ir, frontend, math,
foreign, plugin-abi, host, runtime, package, sandbox,
clap, serde, serde_json, walkdir, hex, sha2
Alternatives
- Put
Spaninmarqdo-value. Thenmarqdo-markdownwould depend on the runtime value model, which is a semantic layer and pulls innum-bigintandserde_json. Worse layering than a dedicated leaf. - Duplicate
Spanin each crate and convert at boundaries. Avoids a crate but guarantees conversion code, mismatched serde shapes and duplicated tests; rejected. - Define
Spaninmarqdo-markdownand let others depend on it. Makes the parser a de-facto foundation crate and lets non-Markdown layers depend oncomrak's dependencies; rejected. - Use a third-party range type. Adds a dependency and gives no benefit over ~170 lines; rejected.
- Keep the blueprint's literal "markdown has zero Marqdo deps". Would force either duplication or a span type in the parser that cannot be shared with diagnostics; rejected in favor of a documented, narrow exception.
Markdown AST shape
No syntax change. Every MdNode still has span: Span in the original file
(crates/marqdo-markdown/src/ast.rs), and the JSON inspection view is
{"kind":"heading","level":2,"span":{"start":0,"end":7},...}. Spans are
half-open byte offsets, BOM-rebased, and byte columns for diagnostics.
Compatibility
- This is a new repository; there is no legacy compatibility burden.
- Public Rust API: moving
Spanlater is a workspace-wide API change but has no language/ABI/package-format impact. It would require updating everycrates/*/Cargo.toml. - Stable observable contracts (AST JSON
span, diagnostic JSONspan, IRspan) are unchanged by the crate boundary. Changing span semantics (e.g. line/column units) requires a spec change and ADR. - The divergence from blueprint §11.1 is recorded here; future crates should
use
marqdo-spanrather than introducing another location type.
Security
No new capability or effect. marqdo-span is serde-only, performs no I/O and
cannot execute anything. Byte offsets are derived from already-parsed source;
LineIndex::line_col clamps out-of-range offsets instead of panicking.
Testing
crates/marqdo-span/src/lib.rstests multi-byte UTF-8 line/column mapping,merge/shift/contains, and clamping.crates/marqdo-markdown/src/profile.rstests BOM rebasing and that every AST node carries a byte span; frontend tests assert diagnostics point at the expected source line/column.cargo test --workspacemust pass;cargo treemust remain acyclic (no automated CI gate yet — this is a P7 item).
Migration & rollout
Immediate: the crate exists and all layers already use it. Future work may
split the frontend into sema/environment crates; that must preserve the
graph above (runtime must never depend on cli, markdown must never depend
on semantic crates). If a second span representation is ever needed (for
example for generated code), add it as a conversion in marqdo-span, not as a
per-crate duplicate.