Complete the documentation set for the M0.5 toolchain, grounded in the code: - spec/references.md: import/reference/call edges, identities, loader rules, anchors, `ref` resolution/ambiguity, `check --refs` validation. - spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal string rule, special floats, limits and exact float round-trip. - spec/security.md: modes (strict refuses without OS isolation), capabilities, path rules, budgets, tracing, foreign process caveats, plugin trust. - spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync, unsupported registry/git/environment work. - spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the implemented CLI plugin-invocation flow with its limitations. - spec/math.md and spec/foreign.md: the implemented math subset and the foreign request/response/timeout/capability contract. - docs/adr/0001..0007: accepted decisions for the span foundation, inert executables outside function bodies, ref ambiguity (S-07), strict refusal, AST normalization, CLI contract and BigInt/typed wire (S-08). - docs/contributor-guide.md: repo map, workflow, fixture/golden updates, quality gate, ADR/commit policy. - docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18 Alpha checklist (conclusion: not yet Alpha-complete), including known divergences rather than overclaims. - README.md rewritten from "design-only starter" to the working implementation with quickstart, implemented phases, deliberate limits and doc index. - AGENTS.md: updated pointers/workflow only; architectural rules unchanged. - Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside functions, unsupported math.simplify/differentiate).
16 KiB
Marqdo Implementation Status (M0.5)
Snapshot: 2026-11-20, workspace version 0.1.0, plugin ABI 1.0,
wire_version 1, Markdown profile marqdo.md.v1.
This file maps the blueprint's phases (docs/DEVELOPMENT_BLUEPRINT.md §13) to
concrete evidence in the repository and lists honest open items. It is a status
report, not a promise. "Done" means implemented for the documented subset
and covered by tests; it does not mean the blueprint's full ambition is
complete.
Reproduce the evidence:
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test --workspace
cargo test -p marqdo-cli
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
1. Phase matrix
| Phase | Status | Main evidence |
|---|---|---|
| P0 Formal specs + parser goldens | Done | spec/markdown-profile.md, spec/mapping.md, spec/expression.ebnf, spec/evaluation.md; frozen profile test crates/marqdo-markdown/src/profile.rs; 150 AST fixtures + goldens in tests/conformance/ast/; 48 compile-error fixtures in tests/negative/; inspect --ast; conformance_ast.rs/negative.rs harnesses. |
| P1 Interpreter, expressions, control flow | Done | marqdo-expression, marqdo-ir, marqdo-frontend, marqdo-runtime; factorial/countdown examples run without Python (examples/factorial.mq.md, examples/hello.mq.md); 36 execution goldens in tests/conformance/semantics/; runtime tests crates/marqdo-runtime/tests/interpreter.rs; prose invariance crates/cli/tests/properties.rs. |
| P2 Modules, references, CLI experience | Partial | Imports/ModuleId/UnitId/anchor/cycle detection + run/call/check/inspect/fmt (crates/cli), multi-module example and multi_module_imports_and_cycles CLI test; check --refs reference validation (all loaded documents, on-demand static analysis of link targets) and inspect --refs static graph. Missing: cross-module link calls, rename/interface-lock tooling, canonical AST-preserving fmt (see §3). |
| P3 Package environment | Partial | marqdo.yaml/marqdo.lock types+validation (marqdo-package), init, add/remove (manifest editing), deterministic lock, sync --locked; CLI test lock_and_sync_locked_detect_dependency_tampering; examples/multi-module. Missing: git resolution, transitive deps, registry, tree/update/vendor, --offline, environment materialization. |
| P4 Security core and isolation | Partial | CapabilityHost, capabilities at execution points (console and math included), project-rooted FS with traversal rejection, budgets, effect trace, pure/project modes, strict refuses without OS isolation (marqdo-sandbox); CLI test strict_mode_refuses_without_isolation; host unit tests. Missing: OS isolation adapter, plugin worker, network enforcement. |
| P5 Math and Foreign | Partial (documented subset) | marqdo-math exact-rational formula kernel for the grammar in spec/math.md; math.parse/eval/substitute host ops; marqdo-foreign Python/Node one-shot kernels with typed foreign.call; tests crates/marqdo-math/tests/kernel.rs, crates/marqdo-foreign/tests/kernels.rs; examples math.mq.md, python-foreign.mq.md. Missing: simplify/differentiate, matrices/series/symbolic algebra, foreign sessions, Python environment management. |
| P6 Plugin ABI and developer experience | Partial | Frozen include/marqdo_plugin_abi_v1.h; C ABI mirrors/layout assertions, version negotiation, plugin.yaml validation, target triple + sha256 verification, wire envelopes, native loader (marqdo-plugin-abi); Rust SDK macro (marqdo-plugin-sdk); C demo (examples/plugin-demo); CLI plugin inspect and explicitly trusted in-process invocation (--plugin, --trust-native-plugin, --allow-plugin); tests plugin_inspect_and_explicit_trusted_invocation, crates/marqdo-plugin-abi/tests/loader_unix.rs. Missing: isolated worker transport, plugin capability bridge, installation, Rust plugin example crate. |
| P7 Performance, release, CI | Partial | .github/workflows/ci.yml runs fmt/clippy/workspace tests plus CLI example smoke tests (including plugin inspect and invocation) on Linux x86_64. Missing: benchmarks, bytecode backend, cross-platform matrix, signed releases and supply-chain checks. Tree-walk interpreter only. |
2. Feature evidence (selected commands)
| Command | Verified behavior |
|---|---|
marqdo init demo |
creates marqdo.yaml (>=0.1,<1), index.mq.md, .gitignore (.marqdo/); refuses existing manifest (exit 1). |
marqdo check index.mq.md |
static only; 0 when no error diagnostics (warnings allowed); 1 on errors; JSON command/success/errors/diagnostics. |
marqdo run index.mq.md |
executes main through the Rust interpreter/host; program stdout on stdout; exit 1 on runtime/compile/strict errors. |
marqdo call file fn --arg n=5 |
typed constant arguments; prints the non-Unit return value. |
marqdo inspect file --ast/--units/--signature/--ir/--refs/--platform --json |
static dashboards; --platform reports strict unavailable. |
marqdo fmt [--check] |
normalizes CRLF and trailing newlines only; exit 1 if --check detects a change. |
marqdo plugin inspect dir |
validates manifest + verifies sha256; never dlopens (loaded: false). |
marqdo run doc --plugin dir --trust-native-plugin --allow-plugin demo.solver |
calls solver.add(a=20,b=22) and prints 42; without trust/grant it exits 1. |
marqdo lock / sync --locked |
deterministic path-dependency lock; --locked fails on manifest/dependency drift. |
Test corpus at this snapshot (tests/README.md):
- 150 AST fixtures + goldens (
tests/conformance/ast/); - 36 semantic execution fixtures + goldens (
tests/conformance/semantics/); - 48 compile-time hard-error fixtures, 6 inert-executable warning fixtures,
10 runtime-error fixtures (
tests/negative/); crates/cli/tests/harnesses:conformance_ast,conformance_semantics,negative,properties,integration_cli;- crate tests across all 15 crates, including real
python3/nodeforeign tests (skipped if unavailable) and a compiled C plugin test (skipped withoutcc).
At this snapshot cargo test -p marqdo-cli passes all 21 harness tests
(2 CLI unit + AST/semantics/negative/properties/integration) and
cargo test --workspace passes. The fixture counts above match
tests/README.md; empty tests/integration, tests/properties,
tests/security and tests/abi directories are reserved for future
workstreams — the active corpus lives in tests/conformance, tests/negative
and crates/cli/tests.
3. Open items by phase
P0
- Wire floats now round-trip exactly:
serde_json'sfloat_roundtripfeature is enabled and a Pythonmath.pi * 9fixture pins the exact value (spec/value-wire.md§3). - Profile change safety relies on the frozen-options unit test; no CI gate regenerates goldens (intentional).
P1
- The blueprint asks for a draft Turing-completeness argument. The language
supports mutable locals, branch-first-match, per-item
whileloops and recursion with bounded budgets; a formal encoding write-up is still missing. - The default cooperative stack budget is 128 frames so very deep recursion
yields a structured
resource.exhaustedinstead of exhausting the debug-build native stack;--stackraises it deliberately. A true guard-page/iterative interpreter hardening is still open.
P2
fmtis intentionally minimal (line endings/trailing newlines only), not the blueprint's "canonical AST-equivalent formatting"; no semantic formatting, AST-equivalence check or render-preserving rewrite exists.- Reference validation is opt-in:
check --refs(andinspect --refs) resolve links; plaincheckdoes not fail on broken links because links are static knowledge references rather than execution inputs. - Link-form calls cannot cross modules; cross-module calls use imported alias
expressions. The blueprint's
> [f](./other.mq.md#f)(...)example is not supported. ref("#...")is module-local and does not accept a selector for units with multiple resources (it errors withref.ambiguous, ADR-0003).- No rename tooling, interface lock/ABI diffing, or incremental cache.
- Duplicate/nested project boundaries are not diagnosed (nearest
marqdo.yamlwins).
P3
- Git dependencies can be declared (
git+rev, validated) but cannot be resolved or locked:lockerrors "git dependencies are not supported by this build". - No transitive dependency graph, conflict resolution, registry, publishing,
signing,
tree,update,vendor,--offline, orlock --locked. - No environment materialization (
.marqdo/env, caches, kernels, sessions);syncverifies but never installs or rewrites. add/removeare shallow manifest editors; adding a git dependency succeeds at the manifest level even thoughlockwill later refuse it.- JSON errors from
add/remove/lock/syncare plain stderr even with--json.
P4
- Strict mode cannot run at all: no OS isolation adapter (Linux
namespaces/seccomp/cgroups, Windows Job Object/appcontainer, macOS sandbox)
and
worker_processes: false;run --mode strictexits 1 (ADR-0004). consolecapability is always on;printis not capability-checked.- Network capability (
net_hosts,check_net) exists but no host operation performs network I/O and there is no--allow-netflag. - Write-path symlink TOCTOU caveat (
spec/security.md§2.1); symlink/race hardening open. - Budgets do not cover every allocation path; the interpreter's containers are bounded only by fuel and available memory.
- Effect trace does not record argument values or paths (privacy-positive, but
less diagnostic detail); trace is opt-in and implicitly on for
--json. plugin_handlewire values are structurally reconstructable, not unforgeable capability tokens.
P5
- Math is a deliberately limited calculator subset: no
simplify,differentiate, matrices, sums/integrals, trig/log functions, units, assumptions or theorem proving.math.simplify/math.differentiateare host ops that always returnmath.unsupported. - Equation solving is affine-only, single target, target must occur exactly
once; other shapes are
math.unsupported. - Division by zero inside a formula maps to
runtime.type(viaMathErrorKind::Type), notruntime.division_by_zero; documented divergence. - Foreign kernels are one-shot only: no sessions, state bridge, Python/Node environment management, or network policy for child processes.
- Foreign child processes are terminated only when the direct child is killed; grandchildren are not tracked.
- Foreign per-call timeout (10 s) and output cap (1 MiB) are not CLI-configurable.
P6
- No isolated plugin worker (length-prefixed IPC) or Wasm transport; only
explicitly trusted in-process native loading. A native plugin crash is a host
crash (
spec/plugin-abi.md§1/§9). - No plugin capability bridge from the CLI: plugins are loaded with no
capability handler, so every plugin
request_capabilityis denied. - Plugin calls are not statically validated; unknown namespaces are runtime
symbol.unknown. Positional plugin arguments are rejected. - One namespace per plugin (first export prefix); multi-prefix manifests are not reachable beyond the first prefix.
- No plugin installation/package management;
plugins:inmarqdo.yamlis declarative only. No Rust plugin example crate (SDK has doc examples/tests). - No hot unload/reload (ABI v1 design).
P7
- Performance work is untouched: no benchmark suite, no bytecode backend or differential tests against the tree-walker, no parse/compile caching.
- No CI workflows (
.github/is empty of workflow files); no Windows/macOS/Linux matrix, sanitizers/fuzz gates, signing, release automation or supply-chain scanning. - No published version, no compatibility matrix beyond the frozen ABI/format documents.
4. Blueprint §18 "Alpha Definition of Done" mapping
| §18 item | Status | Evidence / gap |
|---|---|---|
| Run/call non-trivial recursive, branch, loop, call, parameter-inference programs without Python | Met | factorial/recursion (examples/factorial.mq.md, runtime + semantic goldens), countdown loop, multi-module alias call; interpreter is native Rust. |
.mq.md readable under the chosen CommonMark/GFM/Math profile; standard AST behavior covered by goldens |
Met | frozen marqdo.md.v1, 150 AST goldens, profile option test, spec/markdown-profile.md. |
| No heuristic downgrade of execution semantics; accurate source locations for invalid execution nodes | Mostly met | hard errors with spans for function-body executable shapes and negative corpus. Gaps: inert executable outside functions is a warning by design (ADR-0002); cross-module runtime error spans resolve against the entry file only; some lazy nested nodes are inert without a warning. |
check/inspect side-effect free; imports, calls and static references verifiable across files |
Mostly met | static load/analysis only; imports + cycle detection; cross-module calls validated; check --refs validates references across loaded documents and statically analyses link targets; inspect --refs prints the entry graph. |
| Minimal manifest, lock file, path/Git dependencies and environment sync | Partial | manifest/lock/path works, sync --locked verifies; git resolution, transitive deps, environment materialization and --offline missing. |
| All host side effects capability-authorized; untrusted code/native plugins cannot masquerade as strict sandbox | Mostly met | capability checks at execution points; strict refuses; native plugins require explicit trust and are documented as unsandboxed. Gaps: network enforcement not implemented (no network builtin is exposed), no worker isolation. |
| Formula computational subset explicit/testable; Foreign I/O, timeout and permissions explicit | Met for the documented subset | spec/math.md, spec/foreign.md; math kernel tests; foreign tests (timeout, protocol, output cap, env); permission gating tests/smoke. Simplification/differentiation explicitly unsupported. |
| Plugin C ABI v1, Rust SDK, C example, isolated worker basic path, ABI compat error tests | Partial | ABI/SDK/C example/loader/ABI-error tests exist; isolated worker path missing. |
| CI has at least one full research pipeline: Markdown function → Math → Foreign → Plugin → controlled output | Partial | CI now runs fmt/clippy/tests and CLI smoke tests; individual math/foreign/plugin paths are covered by examples and tests, but no single end-to-end pipeline document/test exists yet. |
| Examples, README, AGENTS and contributor docs allow a first change without legacy context | Met | README.md, AGENTS.md, this guide, docs/STATUS.md, specs, ADRs, tests/README.md, examples. |
Conclusion: the Alpha definition of done is not fully met. P0/P1 and the documented P5 subset are solid; P2/P3/P4/P6 are functional but partial; P7 is absent. The blocking items for an Alpha claim are the strict sandbox/worker isolation story (P4/P6), git/environment package management (P3), and a CI end-to-end pipeline (P7).
5. What is deliberately limited (do not overclaim)
- Only a single, frozen Markdown profile (
marqdo.md.v1); changing parser options/versions is a language change. - Strict mode is a refusal, not an isolation implementation.
projectmode is cooperative and in-process. - Native plugins are trusted in-process code; there is no isolated plugin worker, and a plugin crash takes down the host.
- Ints are arbitrary precision; finite floats are binary64 and wire round-trip
is exact (
float_roundtripenabled), but decimal-vs-binary formatting still applies when displaying values. - Math is a limited exact-rational calculator, not a CAS or a prover.
- Foreign kernels are optional, one-shot, unsandboxed child processes.
fmtis minimal;checkdoes not yet validate Markdown links; link-form calls cannot cross modules.- Package management is path-only in practice;
marqdo.yamlcapabilities are requests, never grants.