cflmy b3429cbeb3 fix(frontend): emit diagnostics in the specified stable order
spec/mapping.md §8 requires diagnostics ordered by source position, then
code, so IDE/CI diffs are stable. analyze_source appended diagnostics by
pipeline stage (units, mapping, inference, call validation), so a later
heading error could precede an earlier payload error.

Sort the final list by (span.start, code) with span-less diagnostics last,
and pin the order with a test that mixes an indexing-stage duplicate-heading
error with an earlier mapping-stage syntax error.

Verified: clippy -D warnings clean; cargo test --workspace --locked green.
2026-11-21 15:55:23 +08:00

Marqdo CLI

Marqdo is a Markdown-native, independently interpreted programming language. A .mq.md file is ordinary CommonMark/GFM prose plus explicit executable regions: a heading declares @exec or @knowledge, **strong**, *emph* and inline `code` are interchangeable carriers of one statement, and keywords inside that payload — if, else, while, end, return, params — decide control flow. Lists, block quotes and emphasis are document presentation only. The official implementation is a Rust workspace; Python/Node are optional foreign kernels, never the compiler target.

This repository is the new, independent Marqdo toolchain (M0.5). It is not a compatibility rewrite of cflmy/marqdo; the old syntax, package format and plugin ABI v2 are not claimed to work here.

Breaking language revision landed. The single semantic contract is RFC-LANG-0002, with normative summaries in spec/mapping.md and spec/expression.ebnf. The frontend, expression parser, stdlib, examples and test corpus all implement it: cargo test --workspace --locked is green (501 passed, 0 failed, 2 ignored at ffd1cc9). Old syntax (**strong** = execute, *emph* = return, 1. = branch, - = loop, > = call) is not an active language and marqdo_semantics/--legacy selectors do not exist. See docs/STATUS.md and docs/migration/README.md.

Status: stabilization candidate, not released. The language (RFC-LANG-0002), lock v3, Analysis v2, Trace v2, Tool v1, Plugin ABI v1 (native + worker) and pinned Foreign providers are implemented and covered by tests on Linux. The macOS/Windows E0433 platform-cfg failure and the Linux strict probe→launch race are fixed in source at 8af7262, but the native Windows/macOS and GitHub Linux lanes have not been re-run from this checkout. strict OS isolation is available on Linux only when the runtime Bubblewrap probe passes; it was verified against the full escape/resource matrix in a disposable Ubuntu 26.04 VM at 8af7262 (ADR-0021). macOS/Windows fail closed, strict L2 (Worker/Foreign) is not implemented, and the broker/worker are not a sandbox. The release decision is DO NOT RELEASE until the remote gates have same-SHA URLs. See docs/STATUS.md, COMPATIBILITY.md and docs/security-gate.md. The single current CI convergence plan is docs/NEXT_RELEASE_CI_CONVERGENCE.md (historical baseline; refresh before use).

CI: the private remote git.cflmy.cn/cflmy/marqdo-cli uses Woodpecker at ci.cflmy.cn; .woodpecker.yml runs fmt, clippy, workspace tests and a release build. The strict OS-isolation lane still requires the disposable VM described in docs/ci.md.

Build

Requires Rust 1.82+.

cargo build -p marqdo-cli
# binary: target/debug/marqdo  (or $CARGO_TARGET_DIR/debug/marqdo)

Quickstart

marqdo init demo
cd demo
marqdo check index.mq.md
marqdo run index.mq.md            # executes ## @exec main
marqdo inspect index.mq.md --ast
marqdo inspect index.mq.md --signature
marqdo call index.mq.md main

Example project from scratch (index.mq.md):

# @knowledge 数学

## @exec 阶乘

`params(n: Int)`

`if n <= 1`

`return 1`

`else`

`结果 = n * 阶乘(n=n-1)`
`return 结果`

`end`
marqdo check examples/factorial.mq.md
marqdo call examples/factorial.mq.md 数学.阶乘 --arg n=5   # 120

The parser is frozen: marqdo.md.v1 = CommonMark 0.31.2 + GFM tables, strikethrough, task lists + $...$/$$...$$ math (comrak =1.0.0). A .mq.md file still renders normally in any compatible Markdown reader; rendering never executes anything. The Markdown profile is unchanged by the semantic rewrite.

Language in 60 seconds

  • Headings declare the mode. # @knowledge 笔记 is a document namespace; ## @exec main overrides to an executable function. Any H1–H6 may carry @exec/@knowledge; unmarked headings inherit the nearest ancestor. The document root is Knowledge. An Exec H2 is the only callable function; H3–H6 are sections inside it.
  • Closed carriers are interchangeable. In an Exec section, **x = 1**, *x = 1* and `x = 1` are the same assignment. Prefer inline code spans for expressions containing * or **.
  • Statements are explicit. if <Bool>, else, while <Bool>, end, return <expr>, params(n: Int), assignment, or an expression call. Each carrier holds exactly one statement; every if/while is closed by end inside the same heading body.
  • Containers do not compute. 1., -, > and emphasis only organize the document. > print(1) is prose; > **print(1)** contains a call.
  • Knowledge is inert. Nothing in a @knowledge section executes, so it is the place for normal emphasis, explanatory lists, tables and formula resources.
  • No syntax switch. There is no marqdo_semantics field or --legacy flag; all .mq.md files use this syntax.

The full contract is spec/mapping.md and spec/expression.ebnf; the rationale and migration rules are RFC-LANG-0002.

What works (implemented and tested)

  • P0 contracts — frozen marqdo.md.v1 profile, RFC-LANG-0002 AST → program mapping, the v2 statement grammar and evaluation semantics; 160 AST goldens, 73 semantic fixtures, 47 compile-error fixtures, 2 warning fixtures and 21 runtime-error fixtures, all green (tests/).
  • P1 interpreter — native Rust tree-walk interpreter: values, scopes, functions, static parameter inference, assignments, branches, loops, returns, recursion, structured errors, fuel/stack/output budgets.
  • Addressable memory (RFC-CM P1) — opaque run-scoped Memory regions with 64 KiB zero-filled pages, grow-only memory.new/size/grow, bounded byte read/write, m[i] = byte, run/per-region budgets, host-boundary and wire rejection (ADR-0022).
  • Minimal structure primitives (RFC-CM CM-A04) — value-semantics list.append/concat, map.keys/has/get/set/remove, Unicode scalar text.codepoints/from_codepoint, and non-negative table.columns/row/cell; pure builtins with no mutation or new syntax (ADR-0023).
  • Recoverable results (RFC-CM CM-A05) — {ok:true,value} / {ok:false,code,message} as an ordinary Map, with a .mq.md first-fit free-list allocator demonstrating reuse, splitting and alloc.no_space recovery (ADR-0024). No Result type or exception syntax.
  • Bundled stdlib + print prototype (RFC-CM CM-A07/A10) — offline stdlib/ with explicit std:io/std:format/std:text imports, SHA-256 digest verification, Analysis identity and a frozen-CLI closure test; the .mq.md print chain is the only execution path for 打印/print on an official install (ADR-0031), covering all 15 value types; HostOp::Print and the legacy formatter are removed (ADR-0027/0028/0029/0030/0031).
  • HTTP/1.1 framing in .mq.md — stdlib/net/http.mq.md status/header/ chunked parsing and request construction; a loopback TCP test runs a real exchange (ADR-0034; no TLS/host HTTP claim).
  • Blocking TCP endpoint (RFC-CM P5 first slice) — resource.open("tcp", "<ip>:<port>", ["read","write"]) with exact IP grants, project mode and 5-second timeouts; DNS/TLS/UDP/server sockets excluded (ADR-0034).
  • C ABI provider worker (RFC-CM CM-A09) — marqdo-provider-worker loads a trusted C ABI library in a separate framed-JSON helper process (ADR-0038).
  • Provider package manifest v1 (RFC-CM CM-A09) — local marqdo.provider manifest installs a trusted C ABI provider; no registry/signature/CLI install (ADR-0037).
  • C ABI ResourceProvider v1 (RFC-CM CM-A09) — trusted native shared libraries expose resource open/read/write/close through a frozen header; project + name grant; plugin-Worker binding remains open (ADR-0036).
  • Worker ResourceProvider transport v1 (RFC-CM CM-A09) — trusted helper process with framed JSON resource operations; project + name grant; C ABI binding remains open (ADR-0035).
  • In-process ResourceProvider v1 (RFC-CM CM-A09 first slice) — trusted providers register extra resource kinds, project-mode + name-grant gated, using the existing resource.*/Result ABI; no parser/runtime/HostOp change (ADR-0033).
  • Table stdlib algorithms (RFC-CM CM-A08) — stdlib/data/table.mq.md select/filter/sort/append/set/join/group over the logical Table value; value-returning, input-immutable, with GFM AST laziness unchanged (ADR-0032).
  • Resource streams (RFC-CM CM-A06) — opaque host-issued Resource handles; resource.open/read/write/close transfer bytes through Memory; stdout write in pure/project, granted project-file read, and buffered project-file write flushed on close in project; strict refuses all resource opens until verified (ADR-0025/0026).
  • P2 references + CLI — explicit YAML front-matter imports, ModuleId/ UnitId/anchors, import cycle detection, multi-module calls, run/call/check/inspect/fmt, reference validation (check --refs) and the static link graph (inspect --refs).
  • P3 project — lock v3 full transitive package-instance graph for path and pinned Git sources, verified content-addressed Git cache, deterministic lock, strict sync --locked with atomic .marqdo/env materialization. run/call/check never fetch or rewrite the lock.
  • P4 security core — capability-checked host operations, project-relative FS globs with traversal rejection, budgets, effect trace, pure/project modes, and Linux strict mode using Bubblewrap namespaces, no-new-privs, capability dropping, rlimits, a wall-clock supervisor and child self-verification (ADR-0021). It is probe-gated and verified in a disposable Ubuntu VM; unsupported hosts fail closed.
  • P5 Math/Foreign — native Formula value from $$...$$; math.parse/ eval/substitute over a documented exact-rational subset; one-shot Python/ Node kernels via foreign.run/call with typed wire protocol, timeout and output limits; pinned provider/environment digests and ADR-0020 environment binding.
  • P6 plugin ABI — frozen C ABI v1 header (include/marqdo_plugin_abi_v1.h), host bindings, Rust SDK (mq_plugin_entry!), plugin.yaml validation, target-triple + sha256 artifact verification, native and isolated worker transports, brokered fs.read, and explicitly trusted in-process calls (--plugin --trust-native-plugin --allow-plugin). A C ABI is not a sandbox.
  • Public protocols — Analysis v2 and Trace v2 with JSON Schemas and independent Python consumers; Tool Protocol v1 with arbitrary command names, SemVer negotiation and whole-bundle digest verification.

Run everything:

cargo test --workspace
cargo test -p marqdo-cli        # conformance corpus + CLI contract tests
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings

Deliberate limits (do not assume)

  • Strict is Linux-only and L1-only. --mode strict enables the Bubblewrap supervisor only when its runtime probe passes; otherwise it exits
    1. It supports pure Marqdo + Math and rejects Foreign, plugins, kernels, external grants and trace files (exit 2). Worker plugins provide crash/address-space isolation, not an OS sandbox, and are not allowed in strict.
  • Native plugins run in-process and require an explicit unsafe trust flag. A plugin can crash or read the host; a C ABI is not a sandbox.
  • Package management is path + pinned-Git only. There is no registry, floating-branch resolution, conflict solver, --offline, tree, update, vendor, or automatic uv/npm environment creation. Manifest capabilities are requests, never grants.
  • Reference validation is opt-in: check --refs reports broken links; plain check ignores them because links are static knowledge references. Markdown links never call; calls are expressions inside executable payloads, and cross-module calls use imported alias expressions such as geometry.圆面积(r=2).
  • ref("#anchor") is module-local and errors with ref.ambiguous when a heading range contains more than one formula/code block instead of picking the first.
  • fmt is minimal (line endings/trailing newlines), not AST-preserving reformatting.
  • Math is a limited calculator, not a CAS/prover: no simplify/ differentiate/matrices/series; equation solving is affine-only.
  • Foreign kernels are unsandboxed one-shot child processes (no sessions, no state bridge); they run with the host user's filesystem/network access.
  • Finite floats are IEEE-754 binary64; the typed wire round-trip is exact (serde_json/float_roundtrip), tagged integers never lose digits, and display formatting is decimal rather than bit-identical.

Repository layout

crates/marqdo-span            shared byte spans/source maps (leaf)
crates/marqdo-value           Value model, diagnostics, typed wire encoding
crates/marqdo-expression      strict expression/statement parser
crates/marqdo-markdown        frozen comrak profile, canonical AST, front matter
crates/marqdo-ir              module/unit/function/statement IR, builtin table
crates/marqdo-frontend        heading scopes, mapping, inference, static calls
crates/marqdo-math            exact-rational Formula subset
crates/marqdo-foreign         Python/Node one-shot kernel supervisor
crates/marqdo-sandbox         mode parsing + platform isolation detection
crates/marqdo-plugin-abi      C ABI v1 mirrors, manifest, artifacts, loader
crates/marqdo-plugin-sdk      Rust plugin SDK and entry macro
crates/marqdo-host            CapabilityHost, budgets, FS, math/foreign/plugin adapters
crates/marqdo-runtime         tree-walk interpreter
crates/marqdo-package         marqdo.yaml / marqdo.lock / root discovery
crates/cli                    CLI composition root (binary `marqdo`)
include/                      frozen plugin C ABI header
spec/                         normative language/format/ABI specs
docs/                         blueprint, ADRs, status, contributor guide
examples/                     runnable documents + multi-module + plugin demo
tests/                        conformance/negative corpus (see tests/README.md)

Documentation index

Examples

marqdo run examples/hello.mq.md
marqdo call examples/factorial.mq.md 数学.阶乘 --arg n=5
marqdo run examples/multi-module/index.mq.md
marqdo run examples/python-foreign.mq.md --allow-foreign   # needs python3 or node
marqdo run examples/packages-multi-version/index.mq.md
marqdo run examples/kernel-mock/demo.mq.md \
  --kernel examples/kernel-mock/kernel.json --allow-foreign
marqdo inspect examples/references.mq.md --refs
marqdo inspect examples/hello.mq.md --platform             # strict: unavailable

./examples/plugin-demo/build.sh
marqdo plugin inspect examples/plugin-demo/build
marqdo run examples/plugin-demo/plugin-call.mq.md \
  --plugin examples/plugin-demo/build --trust-native-plugin --allow-plugin demo.solver

./examples/plugin-multi-export/build.sh
marqdo run examples/plugin-multi-export/plugin-call.mq.md \
  --plugin examples/plugin-multi-export/build --trust-native-plugin --allow-plugin demo.solver

marqdo tool install examples/tool-unknown-name/tool.json
marqdo benchmark run .
marqdo tool remove benchmark

References

S
Description
No description provided
Readme
3.3 MiB
0 Stars 1 Watchers 0 Forks
Languages
Rust 94.1%
Python 3.5%
C 2%
Shell 0.4%