Files
marqdo-cli/docs/adr/0001-crate-graph-and-span-foundation.md
cflmy 0116160399 docs: formal specs, ADRs, STATUS matrix, contributor guide and honest README
Complete the documentation set for the M0.5 toolchain, grounded in the code:

- spec/references.md: import/reference/call edges, identities, loader rules,
  anchors, `ref` resolution/ambiguity, `check --refs` validation.
- spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal
  string rule, special floats, limits and exact float round-trip.
- spec/security.md: modes (strict refuses without OS isolation), capabilities,
  path rules, budgets, tracing, foreign process caveats, plugin trust.
- spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync,
  unsupported registry/git/environment work.
- spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact
  rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the
  implemented CLI plugin-invocation flow with its limitations.
- spec/math.md and spec/foreign.md: the implemented math subset and the
  foreign request/response/timeout/capability contract.
- docs/adr/0001..0007: accepted decisions for the span foundation, inert
  executables outside function bodies, ref ambiguity (S-07), strict refusal,
  AST normalization, CLI contract and BigInt/typed wire (S-08).
- docs/contributor-guide.md: repo map, workflow, fixture/golden updates,
  quality gate, ADR/commit policy.
- docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18
  Alpha checklist (conclusion: not yet Alpha-complete), including known
  divergences rather than overclaims.
- README.md rewritten from "design-only starter" to the working implementation
  with quickstart, implemented phases, deliberate limits and doc index.
- AGENTS.md: updated pointers/workflow only; architectural rules unchanged.
- Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside
  functions, unsupported math.simplify/differentiate).
2026-11-20 10:56:26 +08:00

6.9 KiB

ADR-0001: Crate graph and the marqdo-span foundation

Status: Accepted Date: 2026-11-20

Context

docs/DEVELOPMENT_BLUEPRINT.md §11.1 fixes the dependency direction for the workspace and states: "markdown -> zero other Marqdo crate; responsible for a stable view of the standard AST." The implemented workspace instead has fifteen crates, and marqdo-markdown depends on marqdo-span.

Why the divergence was necessary:

  • Every layer needs identical byte spans: the Markdown AST (crates/marqdo-markdown/src/ast.rs), expression AST (crates/marqdo-expression), IR (crates/marqdo-ir), diagnostics and runtime errors (crates/marqdo-value), effect traces (crates/marqdo-host), plugin errors (crates/marqdo-plugin-abi) and the CLI renderer.
  • A Span must be the same type across boundaries or every layer needs conversion functions. Conversions are exactly where off-by-one bugs and broken source locations appear, and they would make Expr/Stmt serde output inconsistent between inspect --ast and inspect --ir.
  • The blueprint's "zero Marqdo dependencies" rule exists to keep the parser independent of the semantic layers. A tiny, effect-free span crate preserves that property while satisfying the shared-data-contract requirement.

Decision

  1. Add crates/marqdo-span as a leaf crate with no Marqdo dependencies and only serde as an external dependency. It defines:
    • Span { start: usize, end: usize } — half-open byte range into the original UTF-8 source, with merge, shift, contains, len;
    • LineCol { line, column } — 1-based, byte columns;
    • LineIndex — precomputed line starts for byte→line/column conversion.
  2. Treat Span as part of the toolchain's shared data contract. Crates that produce or consume source locations depend on marqdo-span directly; they do not wrap it in local span types.
  3. marqdo-markdown depends on marqdo-span (and comrak) but on no semantic crate. It still owns only the canonical AST view, front matter extraction and source maps; it performs no mapping, resolution or execution.
  4. Keep the rest of the blueprint's direction: value -> span, expression -> span, ir -> span + expression, markdown -> span + comrak, frontend -> markdown + expression + ir + value, host -> value + math + foreign + sandbox, runtime -> ir + value + host + expression, cli -> all (composition root). There is no runtime -> cli edge and no cycle; cargo tree is expected to stay acyclic.
  5. Do not introduce the blueprint's separate sema or environment crates in M0.5: semantic analysis lives in marqdo-frontend (analyze, calls, infer, signature, units, map), and package/ environment formatting lives in marqdo-package plus the CLI. Splitting them is a refactor, not a semantic change.

Actual dependency graph (workspace Cargo.toml plus crate manifests):

span            serde
value           span, serde, serde_json, num-bigint, num-traits
expression      span, serde, num-bigint, unicode-ident
markdown        span, comrak, serde, serde_yaml, unicode-normalization
ir              span, expression, serde
frontend        span, markdown, expression, ir, value, sha2,
                unicode-normalization, unicode-ident
math            value, num-bigint, num-rational, num-traits, serde
foreign         span, value, serde_json, thiserror
sandbox         serde
plugin-abi      span, value, serde, serde_json, serde_yaml, thiserror,
                libloading, sha2, hex
plugin-sdk      plugin-abi, serde, serde_json
host            span, value, math, foreign, sandbox, serde, serde_json,
                thiserror, glob
runtime         span, value, ir, expression, host, num-bigint,
                num-rational, num-traits, thiserror
package         serde, serde_yaml
cli             span, value, expression, markdown, ir, frontend, math,
                foreign, plugin-abi, host, runtime, package, sandbox,
                clap, serde, serde_json, walkdir, hex, sha2

Alternatives

  1. Put Span in marqdo-value. Then marqdo-markdown would depend on the runtime value model, which is a semantic layer and pulls in num-bigint and serde_json. Worse layering than a dedicated leaf.
  2. Duplicate Span in each crate and convert at boundaries. Avoids a crate but guarantees conversion code, mismatched serde shapes and duplicated tests; rejected.
  3. Define Span in marqdo-markdown and let others depend on it. Makes the parser a de-facto foundation crate and lets non-Markdown layers depend on comrak's dependencies; rejected.
  4. Use a third-party range type. Adds a dependency and gives no benefit over ~170 lines; rejected.
  5. Keep the blueprint's literal "markdown has zero Marqdo deps". Would force either duplication or a span type in the parser that cannot be shared with diagnostics; rejected in favor of a documented, narrow exception.

Markdown AST shape

No syntax change. Every MdNode still has span: Span in the original file (crates/marqdo-markdown/src/ast.rs), and the JSON inspection view is {"kind":"heading","level":2,"span":{"start":0,"end":7},...}. Spans are half-open byte offsets, BOM-rebased, and byte columns for diagnostics.

Compatibility

  • This is a new repository; there is no legacy compatibility burden.
  • Public Rust API: moving Span later is a workspace-wide API change but has no language/ABI/package-format impact. It would require updating every crates/*/Cargo.toml.
  • Stable observable contracts (AST JSON span, diagnostic JSON span, IR span) are unchanged by the crate boundary. Changing span semantics (e.g. line/column units) requires a spec change and ADR.
  • The divergence from blueprint §11.1 is recorded here; future crates should use marqdo-span rather than introducing another location type.

Security

No new capability or effect. marqdo-span is serde-only, performs no I/O and cannot execute anything. Byte offsets are derived from already-parsed source; LineIndex::line_col clamps out-of-range offsets instead of panicking.

Testing

  • crates/marqdo-span/src/lib.rs tests multi-byte UTF-8 line/column mapping, merge/shift/contains, and clamping.
  • crates/marqdo-markdown/src/profile.rs tests BOM rebasing and that every AST node carries a byte span; frontend tests assert diagnostics point at the expected source line/column.
  • cargo test --workspace must pass; cargo tree must remain acyclic (no automated CI gate yet — this is a P7 item).

Migration & rollout

Immediate: the crate exists and all layers already use it. Future work may split the frontend into sema/environment crates; that must preserve the graph above (runtime must never depend on cli, markdown must never depend on semantic crates). If a second span representation is ever needed (for example for generated code), add it as a conversion in marqdo-span, not as a per-crate duplicate.