Compare commits

...
35 Commits
Author SHA1 Message Date
cflmy 328f1a8db3 docs: defer stdlib extraction; add local credential and Woodpecker setup
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was canceled
- Add ADR-0039: keep stdlib in-repo while architecture is small; extraction is
  a future trigger-driven option, not a pending gate. Update ADR-0028,
  STATUS, architecture index, package/P2/completion evidence.
- Add .woodpecker.yml (fmt/clippy/test + release build) for ci.cflmy.cn.
- Add docs/development-credentials.md and ignore .secrets/ plus .env.local;
  real credentials stay in the ignored local file and are never committed.

Gate: fmt, clippy -D warnings, 440 tests passed.
2026-11-20 23:40:38 +08:00
cflmy e3638f5720 docs(evidence): add the RFC-CM completion audit
- Map every objective bullet (P2 closure, Compat-B/C, CM-A08/A09/A10/A11,
  P4-P6) to commits, tests, ADRs and checked-in evidence.
- Record the final 440-test gate and strict VM lane result.
- Explicitly list external/unavailable items (GitHub CI URL, Windows/macOS,
  external stdlib repo publication, plugin-Worker ABI binding, P6 optimizer)
  so none is claimed as completed or CI-verified.

Gate: fmt, clippy -D warnings, 440 tests passed.
2026-11-20 23:31:10 +08:00
cflmy 757c5ed15e feat(provider): add C ABI provider worker binding v1
- Add the separate marqdo-provider-worker binary: loads one trusted c-abi-1
  shared library and serves the ADR-0035 framed-JSON resource protocol, with
  fail-closed startup and no in-process fallback (ADR-0038).
- Reuse StdHost::register_worker_resource_provider; language still sees only
  resource.* and project-mode/name grants.
- Compile a C test provider and run open/read/write/close through the helper;
  cover missing-library fail-closed. Trusted process boundary, not a sandbox;
  existing plugin-Worker ABI binding remains open.
- Add ADR/spec/status/README/changelog/evidence; workspace member and tests.

Gate: fmt, clippy -D warnings, 440 tests passed.
2026-11-20 23:30:22 +08:00
cflmy 313ac035e8 test(p2): prove foreign/plugin shortcuts fail in the pure lane
- Add negative tests: foreign.call -> permission.denied and plugin.invoke ->
  structured failure under --mode pure, complementing the 21-fixture pure lane.
- Update P2 evidence/tests README/changelog.

Gate: fmt, clippy -D warnings, 438 tests passed.
2026-11-20 23:29:14 +08:00
cflmy 974102dfc3 feat(host): add provider package manifest v1
- Add a marqdo.provider schema v1 loader (schema/version/ABI/platform library)
  and StdHost::register_provider_package, reusing the C ABI adapter and shared
  kind/name checks (ADR-0037).
- Add a compiled-provider + provider.json installation test; language use still
  requires project mode and allow_resource_provider.
- Update ADR/spec/status/README/changelog and evidence; no registry, signature
  or CLI install is claimed.

Gate: fmt, clippy -D warnings, 437 tests passed.
2026-11-20 23:28:30 +08:00
cflmy 5e5c3a84d9 test(host): cover C ABI version/mode fail-closed paths
- Build a second C provider reporting ABI v2 and assert registration fails
  with resource.unsupported.
- Assert pure/strict refuse a registered C provider kind with permission.denied.
- Refresh the C ABI evidence/test README.

Gate: fmt, clippy -D warnings, 436 tests passed.
2026-11-20 23:27:37 +08:00
cflmy d909de5c68 docs(status): clarify C ABI provider status and remaining CM-A09 gap 2026-11-20 23:26:51 +08:00
cflmy b22a89e7b4 feat(host): add trusted C ABI ResourceProvider v1
- Add the frozen marqdo_resource_provider.h ABI (version + open/read/write/
  close/last_error) and a libloading-based host adapter with version/symbol
  checks and callsite-attached host errors (ADR-0036).
- Add StdHost::register_c_resource_provider using the same kind/name validation
  and project-mode name grant as the in-process/worker providers.
- Compile a C shared-library test provider; cover round-trip and fail-closed
  missing-library behavior. Trusted native code, not a sandbox; plugin-Worker
  binding remains open.
- Update ADRs/spec/status/README/changelog and evidence.

Gate: fmt, clippy -D warnings, 435 tests passed.
2026-11-20 23:26:25 +08:00
cflmy 01f6fe46a7 feat(host): add worker resource-provider transport v1
- Add a host-managed helper process speaking framed JSON (4-byte BE length)
  with ping/open/read/write/close/shutdown, max 1 MiB frames and structured
  error mapping (ADR-0035).
- Add StdHost::register_worker_resource_provider using the same protocol,
  kind-conflict and name-grant checks as the in-process provider; pure/strict
  refuse provider kinds.
- Tests spawn a Python helper and cover round-trip, grant/mode denial and
  missing-program failure; no parser/Value/runtime/HostOp change.
- Update ADR/spec/status/README/changelog and evidence; C ABI binding remains
  open.

Gate: fmt, clippy -D warnings, 433 tests passed.
2026-11-20 23:25:09 +08:00
cflmy 12be02e24b docs(status): mark P4/P5 local gates PASS with explicit remaining work 2026-11-20 23:23:13 +08:00
cflmy 0946220f76 docs(security): record strict TCP refusal at 70d916d 2026-11-20 23:22:32 +08:00
cflmy 70d916d698 test(security): assert strict refuses TCP resources and --allow-net
- G-C lane now runs a .mq.md TCP resource open under strict and requires
  permission.denied, and rejects the --allow-net grant at the strict flag
  boundary.
- Complements the existing file-resource refusal and host TCP strict test.
2026-11-20 23:21:09 +08:00
cflmy 5ad2068038 docs(p2): include the sparse-matrix fixture in closure counts 2026-11-20 23:20:13 +08:00
cflmy 8bbed34bd3 test(c12): add sparse-matrix triples and multiply fixture
- Fixture 73 implements triple-list lookup, dense conversion and sparse
  multiply with Int arithmetic and exact indexing.
- Include it in the P2 pure closure lane and update golden count to 73,
  C12 evidence mapping, status/README/changelog.

Gate: fmt, clippy -D warnings, 430 tests passed.
2026-11-20 23:19:45 +08:00
cflmy 3484ab944c docs(security): refresh CM-A11 lane evidence at b241d1d 2026-11-20 23:18:53 +08:00
cflmy b241d1d978 feat(stdlib): add deterministic offline stdlib packaging
- tools/pack-stdlib.py builds a byte-deterministic
  marqdo-stdlib-0.1.0.tar.gz (fixed metadata, gzip mtime 0) plus a schema-tagged
  descriptor with tree digest, archive SHA-256 and file list.
- Verified repeated builds are identical and the extracted tree digest matches
  stdlib.json; archive is for the future external repo, not the CLI loader.
- Update ADR-0028, status/changelog/tests README and evidence.

Gate: fmt, clippy -D warnings, 430 tests passed.
2026-11-20 23:17:09 +08:00
cflmy e985da28a1 docs(status): refresh the current 430-test gate count 2026-11-20 23:16:35 +08:00
cflmy 4c8e7fd8ab perf(p6): add a reproducible baseline profile harness
- tools/perf/bench.py builds the release CLI and measures 7 .mq.md workloads
  three times with wall time and peak RSS, validating --json success.
- Check in the schema-tagged JSON report and interpretation; no bytecode/JIT/
  native backend was added, so P6 remains baseline-only, not complete.
- Update STATUS/changelog/tests README.

Gate: fmt, clippy -D warnings, 430 tests passed.
2026-11-20 23:16:02 +08:00
cflmy b020904199 test(p4): prove row-store/columnar table equivalence
- Add semantic fixture 72: a .mq.md columnar List/Map table store, converted,
  sorted and filtered; compare headers/cells with the T0 row Table via a
  cell-wise helper (Table == is not structural in this language version).
- Teach the direct-runtime semantic harness to resolve entry std: imports.
- Update ADR-0032, P4 status, docs/changelog/evidence; semantic goldens 72.

Gate: fmt, clippy -D warnings, 430 tests passed.
2026-11-20 23:14:38 +08:00
cflmy e981168d92 feat(stdlib): add HTTP framing over TCP and close P5 C10 locally
- Add stdlib/net/http.mq.md: status/header/chunked response parsing and request
  construction in .mq.md; no host HTTP/TLS implementation.
- Add stdlib/tests/http.mq.md and an end-to-end loopback test that sends a GET
  over resource.open("tcp"), reads bytes, parses the response and returns
  status/content-type/body.
- Update ADR-0034, status/spec/README/changelog and evidence; refresh the
  bundle digest.

Gate: fmt, clippy -D warnings, 430 tests passed.
2026-11-20 23:13:06 +08:00
cflmy 0d4dc1da03 test(p2): close C01-C06/C11/C12 in a pure no-Foreign/no-Plugin lane
- Add crates/cli/tests/p2_closure.rs: 20 semantic fixtures run through the real
  CLI in --mode pure (Capabilities::pure), source scan for foreign./plugin./
  import:, and golden value/stdout comparison.
- Run the stdlib table self-test in pure mode; no host shortcut or recompilation
  is needed for the selected algorithms.
- Mark P2 gate PASS locally (external stdlib repo publication and remote CI
  remain outside the C01-C12 gate); add evidence and docs.

Gate: fmt, clippy -D warnings, 428 tests passed.
2026-11-20 23:12:03 +08:00
cflmy d6cd785bdb docs(status): align print/table status with Compat-C and CM-A08 2026-11-20 23:10:56 +08:00
cflmy 69a28ed1e1 feat(stdlib): add table GFM rendering and an HTTP/1.1 parser fixture
- stdlib/data/table.mq.md adds 连接片段/连接行/转义单元格/转GFM; logical Table
  renders as a GFM pipe table with pipe/newline escaping (CM-A08 P4 gate).
- Add semantic fixture 71: HTTP/1.1 status/header/chunked response parsing and
  request construction in .mq.md; no host HTTP/TLS implementation (C10).
- Update table CLI test, golden count 71, specs/status/README/changelog and
  evidence.

Gate: fmt, clippy -D warnings, 426 tests passed; bundle digest refreshed.
2026-11-20 23:10:19 +08:00
cflmy 67a6d35daf docs(security): record CM-A11 strict resource verification
- Checked-in raw G-C lane log for 00bdcad: 7/7 adversarial helper matrix,
  positive CLI (Compat-C print via bundled stdlib), strict file-resource
  refusal, external-surface rejection, forged-marker rejection, lane PASS.
- Evidence documents the read-only /run/marqdo-stdlib bind, strict stdio-only
  policy, refused file/TCP/provider resources, and the remaining no-CI-URL /
  Windows/macOS limits.
- STATUS updated to the current 426-test gate and 00bdcad strict result.

Gate: fmt, clippy -D warnings, 426 tests passed; VM lane PASS.
2026-11-20 23:08:23 +08:00
cflmy 00bdcad1ff feat(host): add IP-literal blocking TCP resource endpoint (P5)
- resource.open("tcp", ip:port, ["read","write"]) with exact IP grants,
  project mode only, 5s connect/read/write timeouts; no DNS/TLS/UDP/server
  sockets (ADR-0034).
- Add TCP stream storage/dispatch in the resource table; CLI --allow-net IP;
  pure/strict preflight rejects the grant and the language endpoint.
- Host loopback echo/grant/DNS/strict tests and a .mq.md CLI echo test.
- Update specs, status, architecture, README, changelog and evidence.

Gate: fmt, clippy -D warnings, 426 tests passed.
2026-11-20 23:05:06 +08:00
cflmy 1d87f98539 fix(strict): bind the bundled stdlib and verify strict resources
- Strict child now gets the verified stdlib tree read-only at
  /run/marqdo-stdlib with MARQDO_STDLIB_DIR set, so Compat-C 打印 works under
  strict without exposing a host path (ADR-0021 update).
- Add a strict-lane negative: resource.open(file) is refused with
  permission.denied; hello/factorial/multi-module and --json still pass.
- StrictHostCommand gains stdlib_dir; CLI discovers the bundle and sandbox
  binds it; sandbox tests constructors updated.

Evidence: G-C lane PASS on the disposable Ubuntu 26.04 VM (7/7 adversarial
matrix, positive CLI, strict language-resource refusal, forged marker), plus
fmt/clippy/423 local tests.
2026-11-20 22:57:49 +08:00
cflmy 1be4662049 feat(host): add in-process ResourceProvider v1 (CM-A09 slice)
- Add ResourceProvider v1 trait with name/protocol/kinds/open/read/write/close;
  host still issues the opaque language Resource handle, provider tokens stay
  private.
- Add StdHost::register_resource_provider with protocol/version, reserved-name
  and duplicate-kind checks; add Capabilities.allow_resource_provider grants.
- Provider kinds are project-mode only; pure/strict refuse them; in-process
  providers are trusted and not a sandbox (ADR-0033).
- Reuse resource.* and Result shapes; no parser/Value/runtime/HostOp changes.
- Add mock read/write provider and rejection tests, docs/spec/status/evidence.

Gate: fmt, clippy -D warnings, 423 tests passed.
2026-11-20 22:38:16 +08:00
cflmy a8e62a5ba9 feat(stdlib): add table logical-value algorithms (CM-A08)
- Add stdlib/data/table.mq.md with select/filter/sort/append/set-cell/join/
  group as value-returning .mq.md functions over table.columns/row/cell.
- Keep Value::Table logical semantics, GFM AST laziness and T0 physical
  storage unchanged; no new table builtin or HostOp (ADR-0032).
- Add stdlib/tests/table.mq.md and a CLI test asserting shapes/order and
  input immutability; refresh the bundle digest.
- Update architecture/status/spec/README/changelog and evidence.

Gate: fmt, clippy -D warnings, 420 tests passed.
2026-11-20 22:28:52 +08:00
cflmy 49910ece43 refactor(print): remove HostOp::Print and legacy formatting (Compat-C)
- Delete HostOp::Print and StdHost::op_print; the print builtin only routes to
  bundled io.写值 when Program.print_override is set, otherwise returns a
  structured kernel.unavailable error (ADR-0031).
- Keep the 打印/print builtin name for arity/static validation; Value::display
  remains only for CLI result rendering, not print execution.
- Teach the direct-runtime semantic harness to mirror loader stdlib injection
  and import edges; replace runtime tests that used print with non-print
  effects.
- Update spec/security/status/changelog/README, add ADR-0031 and evidence.

Gate: fmt, clippy -D warnings, 419 tests passed.
2026-11-20 22:22:51 +08:00
cflmy e26971f489 feat(cli): route print through the bundled stdlib (Compat-B)
- Add Program.print_override and lazy stdlib injection: only programs that
  call 打印/print load std:io and route to io.写值 (ADR-0030).
- Record resource.open/read/write/close as trace effect events so the stdlib
  print path is auditable; HostOp::Print no longer runs on that path.
- Permit only stdio/stdout (existing console) in strict; file/network
  resource opens remain permission.denied.
- Exclude std: modules from the human-readable checked-module count; update
  the Python analysis consumer to accept std: instance ids.
- Add print-routing and strict-console tests, ADR-0030 and evidence.

Gate: fmt, clippy -D warnings, 418 tests passed.
2026-11-20 22:10:59 +08:00
cflmy e119727c5d feat(stdlib): format all value types and expose handle identity members
- Add read-only FunctionRef.module/name and PluginHandle.plugin/export
  members; PluginHandle.id stays private (ADR-0029).
- Extend stdlib/format.mq.md to Formula/Code/FunctionRef/PluginHandle/
  Memory/Resource/Unit, with UTF-8 byte length for Code.
- Fix two-digit ordered-list body indentation so arms 10+ stay inside their
  branch (they were previously top-level statements).
- Extend stdlib demo to Formula/Code/FunctionRef and assert all three
  io.写值/打印 pairs match byte-for-byte.
- Add runtime member observation test, ADR-0029, spec/status/changelog/evidence.

Gate: fmt, clippy -D warnings, 416 tests passed.
2026-11-20 21:57:04 +08:00
cflmy 1377af1601 feat(stdlib): ship a verified bundled stdlib with std: imports
- Move the Compat-A print prototype into stdlib/ as the official offline
  bundle with stdlib.json (version 0.1.0 + sha256 content digest).
- Add std:<relative> import resolution in the loader with bundle path
  confinement, extension inference and fail-closed digest verification.
- Record the verified version/digest/instance_id in Analysis v2
  project.stdlib; stdlib modules get a std:name@version#digest instance id.
- Add CLI tests: std:io execution, digest-tamper rejection, missing manifest,
  Analysis identity, no foreign/plugin calls, and frozen binary hash equality.
- Update ADR-0028, specs, status, changelog and evidence.

Gate: fmt, clippy -D warnings, 415 tests passed.
2026-11-20 21:43:14 +08:00
cflmy 1a978c7a42 feat(rfc-cm): add JSON floats and compact formatting
- Replace the JSON number scanner with sign/integer/fraction/exponent
  scanning; return Int or Float via 整数/小数 and reject missing digits.
- Add 片段/格式化文本/格式化列表/格式化对象/格式化 recursive formatter.
- Extend fixture 62 with floats ({1,2.5,-3e2,4E-1,{a:1.0},x}) and a compact
  round-trip string.
- Update P2 status: only the separate stdlib package and frozen-CLI closure
  lane remain.

Gate: fmt, clippy -D warnings, 407 tests passed.
2026-11-20 21:26:56 +08:00
cflmy 5d3a5fb13b test(rfc-cm): close P2 evidence gaps for C01/C06/C12
- Add alloc.double_free detection to the .mq.md free-list allocator and a
  fixture assertion for a second free returning a recoverable Result.
- Add table group fixture 69 (table.row -> Map of value lists).
- Add Int matrix-multiply fixture 70 for C12 layout/iteration.
- Update counts and P2 evidence/status docs.

Gate: fmt, clippy -D warnings, 407 tests passed.
2026-11-20 21:22:44 +08:00
cflmy 92bfbf2568 feat(rfc-cm): baseline P0-P3 compute machine, stdlib and resource slices
Implements the first vertical slices of RFC-CM-0001/0002 on top of a113d89:

- P1: opaque Memory regions (64 KiB pages, budgets, grow-only), byte
  read/write, Memory-index place assignment, wire/host-boundary rejection.
- CM-A04/A05: value-semantics List/Map/Text/Table primitives and the
  recoverable Result map convention.
- CM-A06: opaque Resource handles, resource.open/read/write/close, stdout
  write, granted project-file read and buffered write, strict fail-closed.
- CM-A07 Compat-A: explicit .mq.md format/utf8/io chain matching builtin
  print for the covered value types; builtin remains default.
- P2 fixtures: allocators, list/tree/graph/hash, integer codecs, UTF-8
  encode/decode, CSV/JSON subsets, table filter/sort/join, bytecode VM.
- ADRs 0022-0027, spec updates, evidence reports and 68 semantic goldens.

Gate: cargo fmt --check, clippy -D warnings, 407 tests passed.
2026-11-20 21:16:54 +08:00
214 changed files with 20151 additions and 193 deletions

No files matched your search

+4
View File
@@ -20,3 +20,7 @@ dist/
# Local VM/QEMU tooling and disk images
.tools/
# Local-only credentials and environment files
.secrets/
.env.local
+22
View File
@@ -0,0 +1,22 @@
# Woodpecker CI pipeline for the Rust workspace.
# The strict security lane needs a disposable VM/self-hosted runner and is
# intentionally not part of this docker pipeline; see docs/security-vm.md.
when:
event: [push, pull_request]
steps:
test:
image: rust:1.100-bookworm
commands:
- rustup component add clippy
- cargo fmt --all -- --check
- cargo clippy --workspace --all-targets -- -D warnings
- cargo test --workspace --locked
build:
image: rust:1.100-bookworm
depends_on: [test]
commands:
- cargo build --release --locked -p marqdo-cli
artifacts:
- target/release/marqdo
+140
View File
@@ -43,9 +43,149 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
without rewriting `marqdo.lock` or the materialized `.marqdo/env`.
- G-C lane honours `CARGO_TARGET_DIR`, so a run from a clean `git archive` with
an external target directory uses the binary it just built.
- Addressable `Memory` (RFC-CM-0001 P1, ADR-0022): an opaque run-scoped
region value with 64 KiB zero-filled pages, grow-only capacity, bounded
byte read/write, `memory.new/size/grow`, and `m[offset] = byte`. Copies share
the region identity; different regions never alias. Default budgets are
64 MiB committed per run and 16 MiB per region; `--max-memory` and
`--max-region-memory` override the budgets on `run`/`call`.
- Memory-index place assignment in the expression/IR model
(`AssignTarget::Index` / `PlaceTarget::Index`) with the frozen evaluation
order base, index, RHS and no change to `List`/`Map`/`Table` value semantics.
- Stable `runtime.memory` error category for invalid/stale handles and page
counts; `runtime.index` for offsets, `runtime.type` for operand/byte-range
errors, `resource.exhausted` for budgets.
- Analysis v2 projects an index assignment target as an additive
`{"kind":"index",...}` object while local targets keep the v1 string.
- `examples/memory-buffer.mq.md`, semantic goldens 45–47 and runtime negatives
13–18 cover sharing, isolation, growth, zero-fill, offsets, byte range,
non-Memory index assignment, declared-max growth and negative page counts.
- Partial RFC-CM P2 closure evidence (`.mq.md` only, no Rust algorithm
additions): fixtures 48–50 implement a big-endian u32 codec, a bump
allocator, node allocation and a linked-list traversal/sum, and prove that a
failed allocation leaves the allocator header unchanged. This is **not** the
full P2 gate: a separate stdlib package and a frozen-CLI closure lane are
still missing.
- `Memory` handles are never transferable: `to_wire` emits a bare rejected tag,
`from_wire` fails, and every host/plugin/foreign call rejects arguments that
contain a handle.
- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
`list.append/concat`, `map.keys/has/get/set/remove`,
`text.codepoints/from_codepoint`, and `table.columns/row/cell`. All are pure,
value-returning builtins; inputs are never mutated and no new syntax is
added. Semantic fixtures 51–53 and runtime negatives 19–21 cover value
semantics, Unicode scalars, table projection, wrong types and bounds.
- `.mq.md` table filter (`过滤`) built on `table.columns/row/cell`,
`list.append` and `表`, with a semantic golden showing the input table is
unchanged (fixture 54). No `table.filter`/`HostOp` is added.
- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): an ordinary Map
`{ok:true,value}` / `{ok:false,code,message[,details]}`, constructed by
`.mq.md` functions, with expected failures recoverable and language
invariant violations still fatal. No `Result` type, exception or
`try`/`catch` syntax.
- `.mq.md` first-fit free-list allocator (fixture 55): 8-byte block headers,
free-block reuse, block splitting, address-adjacent coalescing (tested in a
second region whose bump space is exhausted, so only a merged block can
satisfy the request), non-overlapping reused payloads and `alloc.no_space`
returned as a `Result` without aborting the run. No allocator Rust builtin or
`HostOp` was added.
- `.mq.md` P2 closures beyond the first slice: binary tree insert/in-order
traversal (56), CSV row splitter (57), open-addressing hash table (58),
UTF-8 encoder/validating decoder (59–60), graph adjacency list (61),
recursive JSON parser for arrays/objects/strings (62), stack bytecode VM
(63), 16/32/64-bit + LEB128 codecs (64), table sort (65), table equality join
(67) and quoted/escaped CSV (68). No Rust algorithm, container or parser was
added for these; algorithms remain `.mq.md` code. JSON float parsing and a
compact formatter are included.
- Resource/stream first slice (RFC-CM CM-A06, ADR-0025): opaque host-issued
`Resource` handles, `resource.open/read/write/close`, bytes transferred
through `Memory`, stdout write in pure/project under console/output budgets,
granted project-file chunked read in project, EOF as success, closed-handle
Result errors, forged-handle fatal errors, wire/host-boundary rejection, and
strict refusing every resource open. Semantic fixture 66 and host/CLI tests.
- Buffered project-file write streams (ADR-0026): `file` + `["write"]` requires
an `fs_write` grant, buffers in the host, flushes only on `close`, never
flushes on drop, has an 8 MiB ceiling, and is refused in pure/strict. A
`.mq.md` chunked copy loop reproduces a file without a Rust `copy_file`.
- Compat-C print completion (ADR-0031): remove `HostOp::Print` and
`StdHost::op_print`; the print builtin only routes to bundled `io.写值` or
returns `kernel.unavailable`; print traces contain only resource effects.
- Compat-B `打印`/`print` routing (ADR-0030): a `Program.print_override` sends
print calls to bundled `io.写值` when the program uses them; stdlib modules
are injected lazily, resource operations appear as trace effects and the
legacy `HostOp::Print` path does not run. Strict permits only stdio/stdout
(existing console), not file/network resources. No Compat-C removal yet.
- Full-value stdlib formatting (ADR-0029): FunctionRef `module/name` and
PluginHandle `plugin/export` read-only members (host id private);
`stdlib/format.mq.md` now reproduces builtin `打印`/`repr` for all 15 value
types, including Formula/Code and UTF-8 byte length for Code.
- CM-A10 bundled stdlib (ADR-0028): a fixed `stdlib/` tree with
`format/text/io`, explicit `std:<path>` imports, SHA-256 digest verification
before module loading, a `std:marqdo-stdlib@<version>#<digest>` package
instance id, `project.stdlib` in Analysis v2, and test coverage for
`std:` resolution, tamper rejection and frozen-binary hash equality.
- P2 closure gate: add a pure no-Foreign/no-Plugin lane for C01–C06/C11/C12
(20 fixtures + stdlib table), source shortcut scan and golden comparison.
- CM-A10 offline packaging: `tools/pack-stdlib.py` emits a deterministic
`marqdo-stdlib-0.1.0.tar.gz` plus tree/archive digests and a file manifest;
the archive is for the future external repository, not the CLI loader.
- P2 anti-evasion: explicit pure-lane negative tests prove `foreign.call` is
`permission.denied` and `plugin.invoke` fails, alongside the 21 fixture lane.
- C12 sparse matrix: fixture 73 adds triple-list sparse multiply and dense
conversion, included in the P2 pure closure lane.
- P6 baseline profile: add `tools/perf/bench.py` and a checked-in 7-workload
JSON report; no bytecode/JIT/native backend is introduced.
- P4 cross-storage table equivalence: fixture 72 compares row-store Table
sort/filter output with a `.mq.md` columnar store cell-by-cell.
- CM-A09 C ABI provider worker binding (ADR-0038): add the separate
`marqdo-provider-worker` helper that loads a trusted C ABI library and serves
the ADR-0035 framed-JSON protocol; crash isolation but not a sandbox.
- CM-A09 provider package manifest v1 (ADR-0037): local manifest resolves a
trusted C ABI library and registers it; no registry/signatures/CLI install.
- CM-A09 C ABI provider v1 (ADR-0036): trusted shared libraries implement
open/read/write/close/version/last_error; host loads them with version and
symbol checks, project-mode + name grant only. Plugin-Worker binding remains
open.
- CM-A09 worker provider transport (ADR-0035): host-managed helper process
speaking framed JSON for resource open/read/write/close; trusted-only,
project-mode + name grant; C ABI binding still open.
- P5 HTTP framing/client: `stdlib/net/http.mq.md` parses status/headers/chunked
responses and builds requests; `http_over_tcp.rs` runs a loopback TCP HTTP
exchange through `resource.open("tcp")`. No host HTTP/TLS implementation.
- CM-A08 GFM rendering: `stdlib/data/table.mq.md::转GFM` renders a logical
Table as a GFM pipe table with cell escaping; CLI test added.
- P5 HTTP parser fixture (C10): semantic fixture 71 parses HTTP/1.1 status,
headers and chunked transfer bodies and builds requests in `.mq.md`; no host
HTTP/HTTPS implementation is claimed.
- P5 blocking TCP endpoint v1 (ADR-0034): `resource.open("tcp", ip:port,
["read","write"])` with exact IP-literal grants, project mode only, 5s
connect/read/write timeouts; no DNS/TLS/UDP/server sockets. CLI `--allow-net
<IP>`; loopback host and `.mq.md` echo tests. Strict/pure refuse.
- CM-A09 in-process ResourceProvider v1 (ADR-0033): `ResourceProvider` trait,
`StdHost::register_resource_provider`, per-kind registration with protocol
version/conflict checks and `Capabilities.allow_resource_provider`; provider
streams reuse `resource.*` and are project-mode only. A mock read/write
provider test runs without parser/runtime/`HostOp` changes. C/Worker provider
transport remains open.
- CM-A08 table logical/stdlib slice (ADR-0032): `stdlib/data/table.mq.md`
adds select/filter/sort/append/set-cell/join/group as value-returning
`.mq.md` functions; `Value::Table` semantics and GFM AST laziness unchanged;
no physical storage migration or new table builtin.
- CM-A07 **Compat-A** prototype (ADR-0027): `stdlib/` provides
`format.显示`, UTF-8 encoding and `io.写值` (`resource.write(stdout)`), and
its output is comparison-tested byte-for-byte against builtin `打印` for
Unit/None/Bool/Int/Float/Text/List/Map/Table. Builtin `打印` remains the
default; no alias routing, bundled stdlib or `HostOp::Print` removal; the
other value types lack public observation.
### Changed
- Assignment targets are no longer restricted to bare identifiers: a single
index place is accepted at statement level. Member assignment stays a parse
error; chained assignment stays a parse error. `spec/expression.ebnf`,
`spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md` and
`spec/security.md` document the new semantics.
- `docs/STABLE_RELEASE_IMPLEMENTATION.md` is the single next-phase
implementation plan. Older plan documents are historical input, not current
status.
+3 -1
View File
@@ -26,7 +26,9 @@ portable/tested locally but the native platform lane has not run here;
| Feature | Ubuntu Linux x86_64 | macOS arm64 | Windows x86_64 |
|---|---|---|---|
| Markdown profile / AST goldens (160) | PASS | UNVERIFIED (cfg fix + CRLF fix) | UNVERIFIED (cfg fix + CRLF fix) |
| Rust interpreter / semantics (44) | PASS | UNVERIFIED | UNVERIFIED |
| Rust interpreter / semantics (70) | PASS | UNVERIFIED | UNVERIFIED |
| Addressable `Memory` P1 (ADR-0022) | PASS | UNVERIFIED | UNVERIFIED |
| Resource streams CM-A06 (ADR-0025/0026: stdout + project file read/write) | PASS | UNVERIFIED | **FAIL-CLOSED** (`strict` refuses resource opens) |
| `run` / `call` / `check` / `inspect` / `fmt` | PASS | UNVERIFIED | UNVERIFIED |
| lock v3 path/Git graph + sync + tamper | PASS | UNVERIFIED | UNVERIFIED |
| Analysis v2 / Trace v2 + Python consumers | PASS | UNVERIFIED | UNVERIFIED |
Generated
+14
View File
@@ -343,6 +343,7 @@ dependencies = [
"serde",
"serde_json",
"sha2",
"tempfile",
"walkdir",
]
@@ -393,6 +394,7 @@ name = "marqdo-host"
version = "0.1.0"
dependencies = [
"glob",
"libloading",
"marqdo-foreign",
"marqdo-math",
"marqdo-plugin-abi",
@@ -474,6 +476,18 @@ dependencies = [
"serde_json",
]
[[package]]
name = "marqdo-provider-worker"
version = "0.1.0"
dependencies = [
"libloading",
"marqdo-host",
"marqdo-sandbox",
"marqdo-value",
"serde_json",
"tempfile",
]
[[package]]
name = "marqdo-runtime"
version = "0.1.0"
+1
View File
@@ -10,6 +10,7 @@ members = [
"crates/marqdo-math",
"crates/marqdo-foreign",
"crates/marqdo-plugin-abi",
"crates/marqdo-provider-worker",
"crates/marqdo-plugin-sdk",
"crates/marqdo-host",
"crates/marqdo-runtime",
+49 -2
View File
@@ -81,12 +81,59 @@ executes anything.
## What works (implemented and tested)
* **P0 contracts** — frozen Markdown profile, AST → program mapping, strict
expression grammar, evaluation semantics; 160 AST goldens, 44 semantic
goldens, 52 compile-error fixtures and 12 runtime-error fixtures
expression grammar, evaluation semantics; 160 AST goldens, 73 semantic
goldens, 52 compile-error fixtures and 21 runtime-error fixtures
(`tests/`).
* **P1 interpreter** — native Rust tree-walk interpreter: values, scopes,
functions, static parameter inference, assignments, branches, loops,
returns, recursion, structured errors, fuel/stack/output budgets.
* **Addressable memory (RFC-CM P1)** — opaque run-scoped `Memory` regions with
64 KiB zero-filled pages, grow-only `memory.new/size/grow`, bounded byte
read/write, `m[i] = byte`, run/per-region budgets, host-boundary and wire
rejection (ADR-0022).
* **Minimal structure primitives (RFC-CM CM-A04)** — value-semantics
`list.append/concat`, `map.keys/has/get/set/remove`, Unicode scalar
`text.codepoints/from_codepoint`, and non-negative `table.columns/row/cell`;
pure builtins with no mutation or new syntax (ADR-0023).
* **Recoverable results (RFC-CM CM-A05)** — `{ok:true,value}` /
`{ok:false,code,message}` as an ordinary Map, with a `.mq.md` first-fit
free-list allocator demonstrating reuse, splitting and `alloc.no_space`
recovery (ADR-0024). No `Result` type or exception syntax.
* **Bundled stdlib + print prototype (RFC-CM CM-A07/A10)** — offline
`stdlib/` with explicit `std:io`/`std:format`/`std:text` imports, SHA-256
digest verification, Analysis identity and a frozen-CLI closure test; the
`.mq.md` print chain is the only execution path for `打印`/`print` on an official install (ADR-0031),
covering all 15 value types; `HostOp::Print` and the legacy formatter are
removed (ADR-0027/0028/0029/0030/0031).
* **HTTP/1.1 framing in `.mq.md`** — `stdlib/net/http.mq.md` status/header/
chunked parsing and request construction; a loopback TCP test runs a real
exchange (ADR-0034; no TLS/host HTTP claim).
* **Blocking TCP endpoint (RFC-CM P5 first slice)** — `resource.open("tcp",
"<ip>:<port>", ["read","write"])` with exact IP grants, project mode and
5-second timeouts; DNS/TLS/UDP/server sockets excluded (ADR-0034).
* **C ABI provider worker (RFC-CM CM-A09)** — `marqdo-provider-worker` loads a
trusted C ABI library in a separate framed-JSON helper process (ADR-0038).
* **Provider package manifest v1 (RFC-CM CM-A09)** — local `marqdo.provider`
manifest installs a trusted C ABI provider; no registry/signature/CLI install
(ADR-0037).
* **C ABI ResourceProvider v1 (RFC-CM CM-A09)** — trusted native shared
libraries expose resource open/read/write/close through a frozen header;
project + name grant; plugin-Worker binding remains open (ADR-0036).
* **Worker ResourceProvider transport v1 (RFC-CM CM-A09)** — trusted helper
process with framed JSON resource operations; project + name grant; C ABI
binding remains open (ADR-0035).
* **In-process ResourceProvider v1 (RFC-CM CM-A09 first slice)** — trusted
providers register extra resource kinds, project-mode + name-grant gated,
using the existing `resource.*`/`Result` ABI; no parser/runtime/`HostOp`
change (ADR-0033).
* **Table stdlib algorithms (RFC-CM CM-A08)** — `stdlib/data/table.mq.md`
select/filter/sort/append/set/join/group over the logical `Table` value;
value-returning, input-immutable, with GFM AST laziness unchanged (ADR-0032).
* **Resource streams (RFC-CM CM-A06)** — opaque host-issued `Resource` handles;
`resource.open/read/write/close` transfer bytes through `Memory`; stdout
write in pure/project, granted project-file read, and buffered project-file
write flushed on `close` in project; strict refuses all resource opens until
verified (ADR-0025/0026).
* **P2 references + CLI** — explicit YAML front-matter imports, `ModuleId`/
`UnitId`/anchors, import cycle detection, multi-module calls,
`run/call/check/inspect/fmt`, reference validation (`check --refs`) and the
+3
View File
@@ -33,3 +33,6 @@ semver = { workspace = true }
[[bin]]
name = "marqdo"
path = "src/main.rs"
[dev-dependencies]
tempfile = { workspace = true }
+35 -5
View File
@@ -8,7 +8,9 @@
use std::path::Path;
use marqdo_expression::{BinaryOp, Expr, UnaryOp};
use marqdo_ir::{CallTarget, Function, Module, ModuleId, Resource, SignatureKind, Stmt, UnitKind};
use marqdo_ir::{
CallTarget, Function, Module, ModuleId, PlaceTarget, Resource, SignatureKind, Stmt, UnitKind,
};
use marqdo_span::Span;
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
@@ -59,6 +61,11 @@ pub fn build(project: &LoadedProject) -> Value {
"root_id": project.identity.root_id(),
"identity_kind": project.identity.kind(),
"lock_digest": lock_digest(project),
"stdlib": project.stdlib.as_ref().map(|stdlib| json!({
"version": stdlib.version,
"digest": stdlib.digest,
"instance_id": stdlib.instance_id,
})),
},
"modules": modules,
"diagnostics": diagnostics_json(project),
@@ -232,13 +239,31 @@ fn function_json(function: &Function) -> Value {
})
}
/// Project an assignment place. Local targets keep the v1 string form; the new
/// index target is an additive object so existing consumers that compare the
/// local `target` string are unaffected (Analysis v2, ADR-0022).
fn place_json(place: &PlaceTarget) -> Value {
match place {
PlaceTarget::Local { name, .. } => Value::String(name.clone()),
PlaceTarget::Index { base, index, span } => json!({
"kind": "index",
"base": expr_json(base),
"index": expr_json(index),
"span": span_json(*span),
}),
}
}
fn stmt_json(stmt: &Stmt) -> Value {
match stmt {
Stmt::Assign {
name, value, span, ..
target,
value,
span,
..
} => json!({
"kind": "assign",
"target": name,
"target": place_json(target),
"value": expr_json(value),
"span": span_json(*span),
}),
@@ -497,9 +522,14 @@ fn collect_calls(
) {
for stmt in body {
match stmt {
Stmt::Assign { value, .. } | Stmt::Eval { expr: value, .. } => {
expr_calls(project, module, function, value, calls)
Stmt::Assign { target, value, .. } => {
if let PlaceTarget::Index { base, index, .. } = target {
expr_calls(project, module, function, base, calls);
expr_calls(project, module, function, index, calls);
}
expr_calls(project, module, function, value, calls);
}
Stmt::Eval { expr: value, .. } => expr_calls(project, module, function, value, calls),
Stmt::Return { value, .. } => expr_calls(project, module, function, value, calls),
Stmt::Call { call, span } => calls.push(json!({
"function": function.id.to_string(),
+34 -3
View File
@@ -4,7 +4,7 @@ use std::path::{Path, PathBuf};
use std::time::{Duration, Instant};
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_runtime::{run, EvaluatedArg, RuntimeError, Value};
use marqdo_runtime::{run_with_limits, EvaluatedArg, MemoryLimits, RuntimeError, Value};
use marqdo_sandbox::{PlatformCapabilities, SandboxMode};
use marqdo_span::Span;
use serde_json::json;
@@ -22,6 +22,8 @@ pub struct SandboxSettings {
pub allow_read: Vec<String>,
/// Write grants.
pub allow_write: Vec<String>,
/// Network grants (IP literals, ADR-0034).
pub allow_net: Vec<String>,
/// Foreign execution grant.
pub allow_foreign: bool,
/// Plugin grants.
@@ -32,6 +34,10 @@ pub struct SandboxSettings {
pub stack: Option<usize>,
/// Output limit override.
pub max_output: Option<usize>,
/// Total committed addressable-memory budget override (bytes).
pub max_memory: Option<usize>,
/// Per-region committed addressable-memory budget override (bytes).
pub max_region_memory: Option<usize>,
/// Wall-clock timeout in seconds.
pub timeout: Option<u64>,
/// Explicitly trusted plugin manifests to load.
@@ -66,6 +72,7 @@ impl SandboxSettings {
pub fn strict_preflight(&self) -> Result<(), String> {
let requested = self.allow_foreign
|| !self.allow_read.is_empty()
|| !self.allow_net.is_empty()
|| !self.allow_write.is_empty()
|| !self.allow_plugin.is_empty()
|| !self.plugins.is_empty()
@@ -90,6 +97,7 @@ impl SandboxSettings {
match mode {
SandboxMode::Pure => {
if !self.allow_read.is_empty()
|| !self.allow_net.is_empty()
|| !self.allow_write.is_empty()
|| self.allow_foreign
|| !self.allow_plugin.is_empty()
@@ -108,6 +116,9 @@ impl SandboxSettings {
for pattern in &self.allow_write {
capabilities = capabilities.allow_write(pattern.clone());
}
for host in &self.allow_net {
capabilities = capabilities.allow_net(host.clone());
}
if self.allow_foreign {
capabilities = capabilities.allow_foreign();
}
@@ -349,7 +360,13 @@ pub fn check(
let text = output::render_text(&project.diagnostics);
eprint!("{text}");
if errors == 0 {
println!("checked {} module(s): ok", project.program.modules.len());
let checked = project
.program
.modules
.keys()
.filter(|id| !id.instance.starts_with("std:"))
.count();
println!("checked {checked} module(s): ok");
} else {
eprintln!(
"check failed: {} error(s), {} warning(s)",
@@ -672,6 +689,10 @@ fn launch_strict_process(
.unwrap_or(1024 * 1024)
.min(64 * 1024 * 1024);
let timeout = Duration::from_secs(settings.timeout.unwrap_or(30));
let stdlib_dir = crate::stdlib::StdlibBundle::discover()
.ok()
.flatten()
.map(|bundle| bundle.root().to_path_buf());
let command = marqdo_sandbox::StrictHostCommand {
program: &program,
args: &args,
@@ -680,6 +701,7 @@ fn launch_strict_process(
original_document: document,
output_limit,
timeout,
stdlib_dir: stdlib_dir.as_deref(),
};
match marqdo_sandbox::launch_strict(&command) {
Ok(code) => code,
@@ -884,12 +906,21 @@ pub fn execute(
.iter()
.map(|(name, value)| EvaluatedArg::named(name.clone(), value.clone(), Span::default()))
.collect();
let result = run(
let memory_limits = MemoryLimits {
max_total_bytes: settings
.max_memory
.unwrap_or(marqdo_runtime::DEFAULT_MAX_TOTAL_BYTES),
max_region_bytes: settings
.max_region_memory
.unwrap_or(marqdo_runtime::DEFAULT_MAX_REGION_BYTES),
};
let result = run_with_limits(
&project.program,
&mut host,
&project.entry,
&function_name,
runtime_args,
memory_limits,
);
if let Some(path) = &settings.trace_file {
let status = match &result {
+107 -2
View File
@@ -13,13 +13,16 @@
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use marqdo_expression::Expr;
use marqdo_frontend::{analyze_source, AnalysisResult};
use marqdo_ir::{ModuleId, Program, ResolvedImport};
use marqdo_ir::{ModuleId, PlaceTarget, Program, ResolvedImport, Stmt};
use marqdo_package::{
discover_root, LockFile, Manifest, PackageResolver, ResolvedPackage, LOCK_FILE, MANIFEST_FILE,
};
use marqdo_value::{codes, Diagnostic};
use crate::stdlib::StdlibBundle;
/// One analysed source file.
pub struct LoadedFile {
/// Canonical module identity (package instance + package-relative path).
@@ -125,6 +128,17 @@ pub struct LocatedSource<'a> {
pub column: usize,
}
/// Identity of the verified bundled stdlib used by a load, if any.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct StdlibInfo {
/// Declared version.
pub version: String,
/// Verified content digest.
pub digest: String,
/// Package instance id.
pub instance_id: String,
}
/// A fully loaded project ready for checking or running.
pub struct LoadedProject {
/// Project root.
@@ -141,6 +155,8 @@ pub struct LoadedProject {
pub packages: BTreeMap<String, PathBuf>,
/// All diagnostics in deterministic order.
pub diagnostics: Vec<FileDiagnostic>,
/// Verified bundled stdlib identity, when one was available.
pub stdlib: Option<StdlibInfo>,
}
impl LoadedProject {
@@ -205,6 +221,7 @@ struct Loader {
program: Program,
diagnostics: Vec<FileDiagnostic>,
stack: Vec<PathBuf>,
stdlib: Option<StdlibBundle>,
}
/// Load the entry file and its explicitly imported modules.
@@ -250,6 +267,7 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
};
let resolver = PackageResolver::new(root.clone(), manifest.clone(), lock);
let entry_module = resolver.resolve_entry(&entry)?;
let stdlib = StdlibBundle::discover().map_err(|error| format!("stdlib: {error}"))?;
let mut loader = Loader {
resolver,
packages: BTreeMap::new(),
@@ -257,9 +275,27 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
program: Program::default(),
diagnostics: Vec::new(),
stack: Vec::new(),
stdlib,
};
let entry_id = loader.load_module(&entry_module.package, &entry_module.relative_path)?;
loader.program.main_module = Some(entry_id.clone());
// Compat-B (ADR-0030): only programs that actually call 打印/print need the
// stdlib print module; injecting it for every program would change the
// module graph, analysis output and startup cost for no benefit.
if program_uses_print(&loader.program) {
if let Some(stdlib) = loader.stdlib.clone() {
let package = stdlib.package();
let io = loader.load_module(&package, "io.mq.md")?;
let Some(module) = loader.program.module(&io) else {
return Err("stdlib print module was not loaded".to_string());
};
let Some(function) = module.function("写值") else {
return Err("stdlib print module has no `写值` function".to_string());
};
loader.program.print_override = Some(function.id.clone());
}
}
for (module_id, diagnostic) in marqdo_frontend::calls::validate_program(&loader.program) {
if let Some(file) = loader.files.get(&module_id) {
loader.diagnostics.push(FileDiagnostic {
@@ -270,6 +306,11 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
});
}
}
let stdlib_info = loader.stdlib.as_ref().map(|bundle| StdlibInfo {
version: bundle.version().to_string(),
digest: bundle.digest().to_string(),
instance_id: bundle.instance_id(),
});
let identity = match &manifest {
Some(_) => ProjectIdentity::Manifest {
instance: entry_id.instance.clone(),
@@ -286,6 +327,7 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
files: loader.files,
packages: loader.packages,
diagnostics: loader.diagnostics,
stdlib: stdlib_info,
})
}
@@ -360,7 +402,16 @@ impl Loader {
.unwrap_or_default();
for (alias, target, span) in imports {
let resolved = if let Some(alias_name) = target.strip_prefix('@') {
let resolved = if let Some(std_target) = target.strip_prefix("std:") {
match &self.stdlib {
Some(stdlib) => stdlib.resolve(std_target),
None => Err(format!(
"no bundled stdlib is available for `{target}`; install the official \
CLI bundle or set {STDLIB_DIR_ENV}",
STDLIB_DIR_ENV = crate::stdlib::STDLIB_DIR_ENV
)),
}
} else if let Some(alias_name) = target.strip_prefix('@') {
self.resolver.resolve_alias_from(package, alias_name)
} else {
self.resolver.resolve_relative(package, &path, &target)
@@ -466,3 +517,57 @@ impl Loader {
Ok(module_id)
}
}
/// Whether the loaded program calls the legacy `打印`/`print` builtin.
fn program_uses_print(program: &Program) -> bool {
program.modules.values().any(|module| {
module
.functions
.iter()
.any(|function| stmts_use_print(&function.body))
})
}
fn stmts_use_print(stmts: &[Stmt]) -> bool {
stmts.iter().any(|stmt| match stmt {
Stmt::Assign { target, value, .. } => {
let target_uses = match target {
PlaceTarget::Local { .. } => false,
PlaceTarget::Index { base, index, .. } => {
expr_uses_print(base) || expr_uses_print(index)
}
};
target_uses || expr_uses_print(value)
}
Stmt::Eval { expr, .. } | Stmt::Return { value: expr, .. } => expr_uses_print(expr),
Stmt::Call { call, .. } => call.args.iter().any(|arg| expr_uses_print(&arg.value)),
Stmt::Branch { arms, .. } => arms
.iter()
.any(|arm| expr_uses_print(&arm.condition) || stmts_use_print(&arm.body)),
Stmt::Loop {
condition, body, ..
} => expr_uses_print(condition) || stmts_use_print(body),
})
}
fn expr_uses_print(expr: &Expr) -> bool {
match expr {
Expr::Call { callee, args, .. } => {
matches!(callee.as_ref(), Expr::Ident(ident) if matches!(ident.name.as_str(), "打印" | "print"))
|| expr_uses_print(callee)
|| args.iter().any(|arg| expr_uses_print(&arg.value))
}
Expr::List(items) => items.iter().any(expr_uses_print),
Expr::Record(entries) => entries.iter().any(|entry| expr_uses_print(&entry.value)),
Expr::Unary { operand, .. } => expr_uses_print(operand),
Expr::Binary { left, right, .. } => expr_uses_print(left) || expr_uses_print(right),
Expr::Member { base, .. } => expr_uses_print(base),
Expr::Index { base, index, .. } => expr_uses_print(base) || expr_uses_print(index),
Expr::Int(_)
| Expr::Float(_)
| Expr::Text(_)
| Expr::Bool(_)
| Expr::None
| Expr::Ident(_) => false,
}
}
+13
View File
@@ -9,6 +9,7 @@ mod args;
mod commands;
mod loader;
mod output;
mod stdlib;
mod tool;
mod trace;
@@ -85,6 +86,9 @@ struct SandboxArgs {
/// Grant read access to a project-relative glob (repeatable).
#[arg(long = "allow-read", value_name = "GLOB")]
allow_read: Vec<String>,
/// Grant network access to an IP literal host (repeatable, ADR-0034).
#[arg(long = "allow-net", value_name = "IP")]
allow_net: Vec<String>,
/// Grant write access to a project-relative glob (repeatable).
#[arg(long = "allow-write", value_name = "GLOB")]
allow_write: Vec<String>,
@@ -103,6 +107,12 @@ struct SandboxArgs {
/// Override the program output byte limit.
#[arg(long = "max-output")]
max_output: Option<usize>,
/// Override the total committed addressable-memory budget in bytes.
#[arg(long = "max-memory", value_name = "BYTES")]
max_memory: Option<usize>,
/// Override the per-region committed addressable-memory budget in bytes.
#[arg(long = "max-region-memory", value_name = "BYTES")]
max_region_memory: Option<usize>,
/// Wall-clock timeout in seconds.
#[arg(long)]
timeout: Option<u64>,
@@ -131,12 +141,15 @@ impl SandboxArgs {
commands::SandboxSettings {
mode: self.mode.clone(),
allow_read: self.allow_read.clone(),
allow_net: self.allow_net.clone(),
allow_write: self.allow_write.clone(),
allow_foreign: self.allow_foreign,
allow_plugin: self.allow_plugin.clone(),
fuel: self.fuel,
stack: self.stack,
max_output: self.max_output,
max_memory: self.max_memory,
max_region_memory: self.max_region_memory,
timeout: self.timeout,
plugins: self.plugin.clone(),
trust_native_plugin: self.trust_native_plugin,
+265
View File
@@ -0,0 +1,265 @@
//! Bundled standard library discovery, digest verification and `std:` imports.
//!
//! The official CLI ships a fixed `marqdo-stdlib` tree (ADR-0026/CM-A10). This
//! module finds that tree, verifies its declared content digest before any
//! module is loaded, and resolves explicit `std:<relative>` imports to it.
//! Nothing is fetched implicitly and there is no second module search path.
use std::path::{Component, Path, PathBuf};
use marqdo_package::{PackageOrigin, ResolvedModule, ResolvedPackage};
use sha2::{Digest, Sha256};
/// Environment override used by tests and relocatable installations.
pub const STDLIB_DIR_ENV: &str = "MARQDO_STDLIB_DIR";
/// Verified bundled standard library.
#[derive(Debug, Clone)]
pub struct StdlibBundle {
root: PathBuf,
name: String,
version: String,
digest: String,
}
impl StdlibBundle {
/// Discover the bundle.
///
/// Order: `MARQDO_STDLIB_DIR` (explicit, fail if invalid), a `stdlib`
/// directory next to the executable, then the development tree.
pub fn discover() -> Result<Option<Self>, String> {
if let Some(dir) = std::env::var_os(STDLIB_DIR_ENV) {
return Self::load(PathBuf::from(dir)).map(Some);
}
if let Ok(exe) = std::env::current_exe() {
if let Some(parent) = exe.parent() {
let candidate = parent.join("stdlib");
if candidate.is_dir() {
return Self::load(candidate).map(Some);
}
}
}
let dev = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("stdlib");
if dev.is_dir() {
return Self::load(dev).map(Some);
}
Ok(None)
}
/// Load and verify one bundle directory.
pub fn load(root: impl Into<PathBuf>) -> Result<Self, String> {
let root = root.into();
let root = root.canonicalize().map_err(|error| {
format!("cannot open stdlib directory `{}`: {error}", root.display())
})?;
let manifest_path = root.join("stdlib.json");
let text = std::fs::read_to_string(&manifest_path)
.map_err(|error| format!("cannot read `{}`: {error}", manifest_path.display()))?;
let manifest: serde_json::Value = serde_json::from_str(&text)
.map_err(|error| format!("invalid `{}`: {error}", manifest_path.display()))?;
let name = manifest
.get("name")
.and_then(serde_json::Value::as_str)
.ok_or_else(|| format!("`{}` has no name", manifest_path.display()))?
.to_string();
let version = manifest
.get("version")
.and_then(serde_json::Value::as_str)
.ok_or_else(|| format!("`{}` has no version", manifest_path.display()))?
.to_string();
let declared = manifest
.get("digest")
.and_then(serde_json::Value::as_str)
.ok_or_else(|| format!("`{}` has no digest", manifest_path.display()))?
.to_string();
let actual = tree_digest(&root)?;
if declared != actual {
return Err(format!(
"stdlib digest mismatch: declared {declared}, actual {actual}"
));
}
Ok(Self {
root,
name,
version,
digest: actual,
})
}
/// Bundle root.
pub fn root(&self) -> &Path {
&self.root
}
/// Declared stdlib version.
pub fn version(&self) -> &str {
&self.version
}
/// Verified content digest (`sha256:<hex>`).
pub fn digest(&self) -> &str {
&self.digest
}
/// Logical package identity used for module ids.
pub fn instance_id(&self) -> String {
format!(
"std:{}@{}#{}",
self.name,
self.version,
self.digest.trim_start_matches("sha256:")
)
}
/// Package view consumed by the loader.
pub fn package(&self) -> ResolvedPackage {
ResolvedPackage {
instance_id: self.instance_id(),
name: self.name.clone(),
version: self.version.clone(),
origin: PackageOrigin::Path {
declared: PathBuf::from("stdlib"),
},
revision: None,
content_digest: Some(self.digest.clone()),
module_root: self.root.clone(),
entry_module: "index.mq.md".to_string(),
manifest: None,
}
}
/// Resolve a `std:<relative>` target inside the bundle.
pub fn resolve(&self, target: &str) -> Result<ResolvedModule, String> {
let mut normalised = PathBuf::new();
for component in Path::new(target).components() {
match component {
Component::Normal(part) => normalised.push(part),
Component::CurDir => {}
Component::ParentDir | Component::RootDir | Component::Prefix(_) => {
return Err(format!(
"stdlib import `{target}` must be a relative path inside the stdlib"
))
}
}
}
if normalised.as_os_str().is_empty() {
return Err("stdlib import must name a module".to_string());
}
if normalised.extension().is_none() {
normalised.set_extension("mq.md");
}
let candidate = self.root.join(&normalised);
let canonical = candidate.canonicalize().map_err(|error| {
format!(
"cannot open stdlib module `{target}` at `{}`: {error}",
candidate.display()
)
})?;
let relative = canonical
.strip_prefix(&self.root)
.map_err(|_| format!("stdlib import `{target}` escapes the stdlib root"))?;
let relative_path = relative
.components()
.map(|component| component.as_os_str().to_string_lossy().to_string())
.collect::<Vec<_>>()
.join("/");
Ok(ResolvedModule {
package: self.package(),
relative_path,
})
}
}
/// Content digest over all `.mq.md` files: sorted path, NUL, bytes, NUL.
pub fn tree_digest(root: &Path) -> Result<String, String> {
let mut files = Vec::new();
collect_markdown(root, root, &mut files)?;
files.sort();
let mut hasher = Sha256::new();
for (relative, path) in files {
let bytes = std::fs::read(&path)
.map_err(|error| format!("cannot read `{}`: {error}", path.display()))?;
hasher.update(relative.as_bytes());
hasher.update([0]);
hasher.update(&bytes);
hasher.update([0]);
}
Ok(format!("sha256:{}", hex(&hasher.finalize())))
}
fn collect_markdown(
root: &Path,
dir: &Path,
out: &mut Vec<(String, PathBuf)>,
) -> Result<(), String> {
let entries = std::fs::read_dir(dir)
.map_err(|error| format!("cannot read `{}`: {error}", dir.display()))?;
for entry in entries {
let entry = entry.map_err(|error| format!("cannot read `{}`: {error}", dir.display()))?;
let path = entry.path();
if path.is_dir() {
collect_markdown(root, &path, out)?;
} else if path
.file_name()
.is_some_and(|name| name.to_string_lossy().ends_with(".mq.md"))
{
let relative = path
.strip_prefix(root)
.map_err(|_| "stdlib file escaped its root".to_string())?
.components()
.map(|component| component.as_os_str().to_string_lossy().to_string())
.collect::<Vec<_>>()
.join("/");
out.push((relative, path));
}
}
Ok(())
}
fn hex(bytes: &[u8]) -> String {
let mut out = String::with_capacity(bytes.len() * 2);
for byte in bytes {
out.push_str(&format!("{byte:02x}"));
}
out
}
#[cfg(test)]
mod tests {
use super::*;
fn dev_bundle() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("stdlib")
}
#[test]
fn dev_bundle_verifies_and_resolves() {
let bundle = StdlibBundle::load(dev_bundle()).unwrap();
assert!(bundle.digest.starts_with("sha256:"));
let io = bundle.resolve("io").unwrap();
assert_eq!(io.relative_path, "io.mq.md");
assert!(io.package.instance_id.starts_with("std:marqdo-stdlib@"));
assert!(bundle.resolve("../escape").is_err());
}
#[test]
fn digest_tamper_is_rejected() {
let dir = tempfile::tempdir().unwrap();
let source = dev_bundle();
for entry in std::fs::read_dir(&source).unwrap() {
let entry = entry.unwrap();
if entry.path().is_file() {
std::fs::copy(entry.path(), dir.path().join(entry.file_name())).unwrap();
}
}
std::fs::write(dir.path().join("io.mq.md"), "## tampered\n").unwrap();
let error = StdlibBundle::load(dir.path()).unwrap_err();
assert!(error.contains("digest mismatch"), "{error}");
}
}
+28
View File
@@ -181,6 +181,34 @@ fn analysis_v2_identity_is_location_independent() {
assert_eq!(analysis["project"]["identity_kind"], "manifest");
}
#[test]
fn analysis_v2_projects_memory_place_targets_additively() {
let dir = TempDir::new("analysis-place");
let source = "## main\n\n**内存 = memory.new(1, 2)**\n\n**`内存[0] = 7`**\n\n*内存[0]*\n";
write(dir.path(), "index.mq.md", source);
let target = dir.path().join("analysis.json");
let output = cli(
dir.path(),
&["check", "index.mq.md", "--emit", target.to_str().unwrap()],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let analysis: serde_json::Value =
serde_json::from_slice(&std::fs::read(&target).unwrap()).unwrap();
let body = analysis["modules"][0]["functions"][0]["body"]
.as_array()
.expect("body is an array");
assert_eq!(body[0]["target"], "内存");
assert_eq!(body[1]["target"]["kind"], "index");
assert_eq!(body[1]["target"]["base"]["kind"], "variable");
assert_eq!(body[1]["target"]["index"]["kind"], "literal");
assert_eq!(body[1]["target"]["index"]["type"], "int");
}
#[test]
fn analysis_v2_projects_addressable_formula_and_code_resources() {
let dir = TempDir::new("analysis-resources");
+109 -1
View File
@@ -24,7 +24,7 @@ use std::path::{Path, PathBuf};
use std::process::Command;
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_ir::ModuleId;
use marqdo_ir::{ModuleId, Program, ResolvedImport};
use marqdo_runtime::{run, EvaluatedArg};
use marqdo_sandbox::SandboxMode;
@@ -168,6 +168,8 @@ fn semantic_goldens_match_end_to_end_execution() {
let mut program = marqdo_ir::Program::default();
program.insert(analysis.module.clone());
program.main_module = Some(module_id.clone());
install_stdlib_print(&mut program);
install_entry_stdlib_imports(&mut program, &module_id, &analysis);
let mut capabilities = Capabilities::pure();
if grant_foreign {
@@ -263,3 +265,109 @@ fn semantic_goldens_match_end_to_end_execution() {
fixtures.len()
);
}
/// Load the bundled stdlib print modules for direct-runtime fixtures.
///
/// The CLI loader normally injects these for Compat-B. The semantic harness
/// builds a Program directly, so it mirrors the same resolution here; without
/// it, fixtures that call `打印` would hit the Compat-C "no bundle" error.
fn install_stdlib_print(program: &mut Program) {
let root = repo_root().join("stdlib");
if !root.join("stdlib.json").is_file() {
return;
}
let mut loaded = std::collections::BTreeSet::new();
let io = match load_stdlib_module(&root, "io.mq.md", program, &mut loaded) {
Ok(id) => id,
Err(error) => panic!("cannot load bundled stdlib: {error}"),
};
let function = program
.module(&io)
.and_then(|module| module.function("写值"))
.unwrap_or_else(|| panic!("stdlib io has no 写值 function"));
program.print_override = Some(function.id.clone());
}
fn load_stdlib_module(
root: &Path,
relative: &str,
program: &mut Program,
loaded: &mut std::collections::BTreeSet<String>,
) -> Result<ModuleId, String> {
if !loaded.insert(relative.to_string()) {
return Err(format!("stdlib import cycle at `{relative}`"));
}
let path = root.join(relative);
let source = fs::read_to_string(&path)
.map_err(|error| format!("cannot read `{}`: {error}", path.display()))?;
let module_id = ModuleId::new("std:test", relative.replace('\\', "/"));
let analysis = marqdo_frontend::analyze_source(&module_id.to_string(), &source);
if analysis.has_errors() {
return Err(format!("stdlib module `{relative}` has diagnostics"));
}
let imports: Vec<(String, String)> = analysis
.document
.frontmatter()
.and_then(|frontmatter| frontmatter.imports().ok())
.map(|mapping| mapping.into_iter().collect())
.unwrap_or_default();
for (alias, target) in imports {
let target = target.strip_prefix("std:").unwrap_or(&target);
let child = if target.starts_with("./") || target.starts_with("../") {
let base = Path::new(relative)
.parent()
.unwrap_or_else(|| Path::new(""));
base.join(target)
.components()
.map(|component| component.as_os_str().to_string_lossy().to_string())
.collect::<Vec<_>>()
.join("/")
} else {
target.to_string()
};
let to = load_stdlib_module(root, &child, program, loaded)?;
program.resolved_imports.push(ResolvedImport {
from: module_id.clone(),
alias,
to,
});
}
program.insert(analysis.module);
Ok(module_id)
}
/// Resolve `std:` imports declared by a direct-runtime fixture entry.
fn install_entry_stdlib_imports(
program: &mut Program,
module_id: &ModuleId,
analysis: &marqdo_frontend::AnalysisResult,
) {
let root = repo_root().join("stdlib");
let Some(imports) = analysis
.document
.frontmatter()
.and_then(|frontmatter| frontmatter.imports().ok())
else {
return;
};
let mut loaded = std::collections::BTreeSet::new();
for (alias, target) in imports {
let Some(relative) = target.strip_prefix("std:") else {
continue;
};
let relative = if relative.ends_with(".mq.md") {
relative.to_string()
} else {
format!("{relative}.mq.md")
};
let to = match load_stdlib_module(&root, &relative, program, &mut loaded) {
Ok(id) => id,
Err(error) => panic!("cannot load stdlib import `{target}`: {error}"),
};
program.resolved_imports.push(ResolvedImport {
from: module_id.clone(),
alias,
to,
});
}
}
+52
View File
@@ -24,6 +24,58 @@ fn stderr(output: &Output) -> String {
String::from_utf8_lossy(&output.stderr).to_string()
}
#[test]
fn mqmd_stdlib_print_output_matches_builtin_print() {
let output = cli(&["run", "stdlib/demo.mq.md"]);
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
let stdout = String::from_utf8_lossy(&output.stdout);
let lines: Vec<&str> = stdout.lines().collect();
assert_eq!(lines.len(), 6, "{stdout:?}");
for pair in [(0, 1), (2, 3), (4, 5)] {
assert_eq!(
lines[pair.0], lines[pair.1],
"stdlib formatting differs from 打印 at line {}",
pair.0
);
}
assert_eq!(
lines[0],
r#"["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]"#
);
assert_eq!(lines[2], "Table[名称 | 分数]{甲 | 87; 乙 | 92}");
assert_eq!(
lines[4],
r#"[Formula("a = 1"), Code("text", 9 bytes), FunctionRef(root:default::demo.mq.md::演示函数)]"#
);
}
#[test]
fn addressable_memory_example_runs_through_the_cli() {
let output = cli(&["run", "examples/memory-buffer.mq.md", "--json"]);
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
let report: serde_json::Value =
serde_json::from_slice(&output.stdout).expect("run --json emits JSON");
assert_eq!(report["success"], true);
assert_eq!(report["value"]["type"], "int");
assert_eq!(report["value"]["decimal"], "141");
}
#[test]
fn cli_memory_budget_override_fails_closed() {
let output = cli(&[
"run",
"examples/memory-buffer.mq.md",
"--max-memory",
"0",
"--json",
]);
assert_eq!(output.status.code(), Some(1), "{}", stderr(&output));
let report: serde_json::Value =
serde_json::from_slice(&output.stdout).expect("run --json emits JSON");
assert_eq!(report["success"], false);
assert_eq!(report["error"]["code"], "resource.exhausted");
}
#[test]
fn multi_version_packages_run_without_an_implicit_override() {
let output = cli(&["run", "examples/packages-multi-version/index.mq.md"]);
+120
View File
@@ -0,0 +1,120 @@
//! C10/P5: a `.mq.md` HTTP client over the TCP resource endpoint.
use std::io::{Read, Write};
use std::net::TcpListener;
use std::path::Path;
use std::process::Command;
struct TempDir(std::path::PathBuf);
impl TempDir {
fn new(tag: &str) -> Self {
let path =
std::env::temp_dir().join(format!("marqdo-http-tcp-{}-{tag}", std::process::id()));
let _ = std::fs::remove_dir_all(&path);
std::fs::create_dir_all(&path).unwrap();
Self(path)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
#[test]
fn mqmd_http_client_parses_a_response_from_a_tcp_stream() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let address = listener.local_addr().unwrap();
let server = std::thread::spawn(move || {
let (mut stream, _) = listener.accept().unwrap();
let mut request = [0u8; 512];
let count = stream.read(&mut request).unwrap();
assert!(String::from_utf8_lossy(&request[..count]).starts_with("GET / HTTP/1.1\r\n"));
stream
.write_all(
b"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 5\r\n\r\nhello",
)
.unwrap();
});
let dir = TempDir::new("client");
let program = format!(
r#"---
import:
http: "std:net/http"
编码: "std:text"
---
## main
**流 = resource.open("tcp", "{address}", ["read", "write"])**
**请求 = http.构建请求("GET", "/", [["Host", "example.test"]], "")**
**请求内存 = memory.new(1, 1)**
**请求长度 = 编码.utf8编码(请求, 请求内存, 0)**
**写入 = resource.write(流, 请求内存, 0, 请求长度)**
**接收 = memory.new(1, 1)**
**读出 = resource.read(流, 接收, 0, 4096)**
**数量 = 读出.value.count**
**响应文本 = ""**
**下标 = 0**
- `下标 < 数量`
**响应文本 = 响应文本 + text.from_codepoint(接收[下标])**
**下标 = 下标 + 1**
**关闭 = resource.close(流)**
**响应 = http.解析响应(响应文本)**
*[写入.ok, 读出.ok, 响应.状态, 响应.头["Content-Type"], 响应.体, 关闭.ok]*
"#
);
std::fs::write(dir.path().join("index.mq.md"), program).unwrap();
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", "index.mq.md", "--allow-net", "127.0.0.1", "--json"])
.current_dir(dir.path())
.env(
"MARQDO_STDLIB_DIR",
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.join("stdlib"),
)
.output()
.expect("run marqdo");
assert_eq!(
output.status.code(),
Some(0),
"stderr={} stdout={}",
String::from_utf8_lossy(&output.stderr),
String::from_utf8_lossy(&output.stdout)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
assert_eq!(report["success"], true);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["value"], true);
assert_eq!(items[2]["decimal"], "200");
assert_eq!(items[3]["value"], "text/plain");
assert_eq!(items[4]["value"], "hello");
assert_eq!(items[5]["value"], true);
server.join().unwrap();
}
+159
View File
@@ -0,0 +1,159 @@
//! RFC-CM P2 closure lane: C01–C06/C11/C12 in a pure, no-Foreign/no-Plugin host.
use std::path::{Path, PathBuf};
use std::process::Command;
fn repo_root() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
fn fixture(name: &str) -> PathBuf {
repo_root()
.join("tests")
.join("conformance")
.join("semantics")
.join(name)
}
fn expected(name: &str) -> serde_json::Value {
let path = fixture(&name.replace(".mq.md", ".expected.json"));
serde_json::from_slice(&std::fs::read(&path).expect("expected golden")).expect("golden json")
}
#[test]
fn p2_algorithms_run_in_a_pure_no_foreign_no_plugin_host() {
// P2 gate (RFC-CM-0002 §8): C01-C06, C11, C12 use only .mq.md plus the
// declared low-level primitives. These fixtures are run through the real
// CLI in `--mode pure`, which has no foreign/plugin grants, and the source
// is checked for host shortcuts that would violate the anti-evasion rule.
let cases: &[(&str, &str)] = &[
("48-memory-bump-allocator.mq.md", "C01 bump allocator"),
(
"50-memory-allocation-failure-atomic.mq.md",
"C01 allocation atomicity",
),
(
"55-first-fit-free-list-allocator.mq.md",
"C01 free-list split/coalesce",
),
("49-memory-linked-list.mq.md", "C02 linked list"),
("56-memory-binary-tree.mq.md", "C02 tree"),
("61-memory-graph-adjacency.mq.md", "C02 graph"),
("58-memory-hash-table.mq.md", "C02 hash table"),
("64-memory-integer-codecs.mq.md", "C03 codecs"),
("59-utf8-encoder.mq.md", "C04 UTF-8 encode"),
("60-utf8-decoder.mq.md", "C04 UTF-8 decode"),
("57-csv-row-splitter.mq.md", "C05 CSV"),
("62-json-array-parser.mq.md", "C05 JSON"),
("68-csv-quoted-row.mq.md", "C05 CSV quoted"),
("53-table-projection.mq.md", "C06 table projection"),
("54-table-filter-stdlib.mq.md", "C06 table filter"),
("65-table-sort-stdlib.mq.md", "C06 table sort"),
("67-table-join-stdlib.mq.md", "C06 table join"),
("69-table-group-stdlib.mq.md", "C06 table group"),
("63-memory-bytecode-vm.mq.md", "C11 bytecode VM"),
("70-matrix-multiply.mq.md", "C12 matrix"),
("73-sparse-matrix.mq.md", "C12 sparse matrix"),
];
for (name, criterion) in cases {
let path = fixture(name);
let source = std::fs::read_to_string(&path).expect("fixture source");
for forbidden in ["foreign.", "plugin.", "import:"] {
assert!(
!source.contains(forbidden),
"{criterion} fixture {name} uses forbidden host shortcut `{forbidden}`"
);
}
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", path.to_str().unwrap(), "--mode", "pure", "--json"])
.current_dir(repo_root())
.output()
.expect("run marqdo");
assert_eq!(
output.status.code(),
Some(0),
"{criterion} fixture {name} failed in pure mode: {}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value =
serde_json::from_slice(&output.stdout).expect("json report");
assert_eq!(report["success"], true, "{name}");
let golden = expected(name);
assert_eq!(
report["value"], golden["value"],
"{criterion} fixture {name}"
);
assert_eq!(
report["stdout"].as_str().unwrap_or_default(),
golden["stdout"].as_str().unwrap_or_default(),
"{criterion} fixture {name} stdout"
);
}
}
#[test]
fn p2_stdlib_table_also_runs_in_pure_mode() {
let path = repo_root().join("stdlib").join("tests").join("table.mq.md");
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", path.to_str().unwrap(), "--mode", "pure", "--json"])
.current_dir(repo_root().join("stdlib"))
.env("MARQDO_STDLIB_DIR", repo_root().join("stdlib"))
.output()
.expect("run table self-test");
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
assert_eq!(report["success"], true);
let items = report["value"]["items"]
.as_array()
.expect("table test list");
assert_eq!(items.len(), 9);
assert_eq!(
items[8]["value"],
"| 名称 | 分数 |\n| --- | --- |\n| 甲 | 87 |\n| 乙 | 92 |\n| 丙 | 71 |"
);
}
#[test]
fn foreign_and_plugin_shortcuts_are_unavailable_in_the_pure_lane() {
let dir = tempfile::tempdir().unwrap();
let foreign = dir.path().join("foreign.mq.md");
std::fs::write(
&foreign,
"## 代码\n\n```python\nprint(1)\n```\n\n## main\n\n*foreign.call(ref(\"#代码\"), None)*\n",
)
.unwrap();
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", foreign.to_str().unwrap(), "--mode", "pure", "--json"])
.current_dir(dir.path())
.output()
.expect("run foreign fixture");
assert_eq!(output.status.code(), Some(1));
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
assert_eq!(report["success"], false);
assert_eq!(report["error"]["code"], "permission.denied");
let plugin = dir.path().join("plugin.mq.md");
std::fs::write(
&plugin,
"## main\n\n> plugin.invoke(\"missing.export\", {})\n\n*None*\n",
)
.unwrap();
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", plugin.to_str().unwrap(), "--mode", "pure", "--json"])
.current_dir(dir.path())
.output()
.expect("run plugin fixture");
assert_eq!(output.status.code(), Some(1));
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
assert_eq!(report["success"], false);
}
+111
View File
@@ -0,0 +1,111 @@
//! Compat-B `打印` routing tests (ADR-0030).
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
struct TempDir(PathBuf);
impl TempDir {
fn new(tag: &str) -> Self {
let path =
std::env::temp_dir().join(format!("marqdo-print-route-{}-{tag}", std::process::id()));
let _ = std::fs::remove_dir_all(&path);
std::fs::create_dir_all(&path).unwrap();
Self(path)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn cli(dir: &Path, args: &[&str]) -> Output {
Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(args)
.current_dir(dir)
.output()
.expect("run marqdo")
}
fn write(dir: &Path, name: &str, source: &str) -> PathBuf {
let path = dir.join(name);
std::fs::write(&path, source).unwrap();
path
}
#[test]
fn print_is_routed_through_the_stdlib_and_not_hostop_print() {
let dir = TempDir::new("routed");
write(
dir.path(),
"index.mq.md",
"## main\n\n> 打印([\"x\", 1])\n\n*None*\n",
);
let trace = dir.path().join("trace.json");
let output = cli(
dir.path(),
&[
"run",
"index.mq.md",
"--trace-file",
trace.to_str().unwrap(),
],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
assert_eq!(String::from_utf8_lossy(&output.stdout), "[\"x\", 1]\n");
let trace: serde_json::Value = serde_json::from_slice(&std::fs::read(&trace).unwrap()).unwrap();
let operations: Vec<&str> = trace["events"]
.as_array()
.unwrap()
.iter()
.filter_map(|event| event["operation"].as_str())
.collect();
assert!(
operations.contains(&"resource.write"),
"stdlib print must write through the resource ABI: {operations:?}"
);
assert!(
!operations.contains(&"print"),
"legacy HostOp::Print must not run in Compat-B: {operations:?}"
);
}
#[test]
fn stdlib_print_modules_are_injected_only_when_print_is_used() {
let dir = TempDir::new("lazy");
write(dir.path(), "plain.mq.md", "## main\n\n*1*\n");
write(
dir.path(),
"prints.mq.md",
"## main\n\n> 打印(1)\n\n*None*\n",
);
for (file, expected_stdlib) in [("plain.mq.md", false), ("prints.mq.md", true)] {
let target = dir.path().join(format!("{file}.json"));
let output = cli(
dir.path(),
&["check", file, "--emit", target.to_str().unwrap()],
);
assert_eq!(output.status.code(), Some(0), "{file}");
let analysis: serde_json::Value =
serde_json::from_slice(&std::fs::read(&target).unwrap()).unwrap();
let modules = analysis["modules"].as_array().unwrap();
let has_stdlib = modules
.iter()
.any(|module| module["id"].as_str().unwrap_or("").starts_with("std:"));
assert_eq!(has_stdlib, expected_stdlib, "{file}: {modules:?}");
// The bundle identity is recorded whenever a bundle is available; it
// does not imply that this program loaded any stdlib module.
assert!(analysis["project"]["stdlib"].is_object(), "{file}");
}
}
+297
View File
@@ -0,0 +1,297 @@
//! End-to-end resource stream tests through the CLI.
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
struct TempDir(PathBuf);
impl TempDir {
fn new(tag: &str) -> Self {
let path =
std::env::temp_dir().join(format!("marqdo-resource-test-{}-{tag}", std::process::id()));
let _ = std::fs::remove_dir_all(&path);
std::fs::create_dir_all(&path).unwrap();
Self(path)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn cli(dir: &Path, args: &[&str]) -> Output {
Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(args)
.current_dir(dir)
.output()
.expect("run marqdo")
}
fn write_file(path: &Path, contents: &str) {
std::fs::write(path, contents).unwrap();
}
const READER: &str = r#"## main
**内存 = memory.new(1, 1)**
**流 = resource.open("file", "data.txt", ["read"])**
**第一次 = resource.read(流, 内存, 0, 2)**
**第二次 = resource.read(流, 内存, 2, 4)**
**第三次 = resource.read(流, 内存, 6, 1)**
**关闭 = resource.close(流)**
*[第一次.value.count, 第一次.value.eof, 第二次.value.count, 第二次.value.eof, 第三次.value.count, 第三次.value.eof, 内存[0], 内存[1], 内存[2], 内存[3], 内存[4], 内存[5], 关闭.ok]*
"#;
#[test]
fn project_file_stream_reads_chunks_into_memory() {
let dir = TempDir::new("file-read");
write_file(&dir.path().join("data.txt"), "abcdef");
write_file(&dir.path().join("index.mq.md"), READER);
let output = cli(
dir.path(),
&["run", "index.mq.md", "--allow-read", "data.txt", "--json"],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
let values: Vec<&serde_json::Value> = report["value"]["items"]
.as_array()
.expect("list result")
.iter()
.collect();
assert_eq!(values[0]["decimal"], "2");
assert_eq!(values[1]["value"], false);
assert_eq!(values[2]["decimal"], "4");
assert_eq!(values[3]["value"], true);
assert_eq!(values[4]["decimal"], "0");
assert_eq!(values[5]["value"], true);
for (index, expected) in ["97", "98", "99", "100", "101", "102"]
.into_iter()
.enumerate()
{
assert_eq!(values[6 + index]["decimal"], expected);
}
assert_eq!(values[12]["value"], true);
}
const WRITER: &str = r#"## main
**内存 = memory.new(1, 1)**
**`内存[0] = 65`**
**`内存[1] = 66`**
**`内存[2] = 67`**
**写 = resource.open("file", "out.bin", ["write"])**
**写入 = resource.write(写, 内存, 0, 3)**
**关闭写 = resource.close(写)**
**读 = resource.open("file", "out.bin", ["read"])**
**读出 = resource.read(读, 内存, 0, 3)**
**关闭读 = resource.close(读)**
*[写入.value.count, 关闭写.ok, 读出.value.count, 读出.value.eof, 内存[0], 内存[1], 内存[2], 关闭读.ok]*
"#;
#[test]
fn project_file_stream_writes_on_close_then_reads_back() {
let dir = TempDir::new("file-write");
write_file(&dir.path().join("index.mq.md"), WRITER);
let output = cli(
dir.path(),
&[
"run",
"index.mq.md",
"--allow-write",
"out.bin",
"--allow-read",
"out.bin",
"--json",
],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
let values = report["value"]["items"].as_array().expect("list result");
assert_eq!(values[0]["decimal"], "3");
assert_eq!(values[1]["value"], true);
assert_eq!(values[2]["decimal"], "3");
assert_eq!(values[3]["value"], true);
assert_eq!(values[4]["decimal"], "65");
assert_eq!(values[5]["decimal"], "66");
assert_eq!(values[6]["decimal"], "67");
assert_eq!(values[7]["value"], true);
assert_eq!(std::fs::read(dir.path().join("out.bin")).unwrap(), b"ABC");
}
const COPY: &str = r#"## main
**输入 = resource.open("file", "data.txt", ["read"])**
**输出 = resource.open("file", "out.bin", ["write"])**
**内存 = memory.new(1, 1)**
**总数 = 0**
**继续 = true**
- `继续`
**块 = resource.read(输入, 内存, 0, 16)**
1. `块.ok`
> resource.write(输出, 内存, 0, 块.value.count)
**总数 = 总数 + 块.value.count**
**继续 = not 块.value.eof**
2. `true`
**继续 = false**
**关闭读 = resource.close(输入)**
**关闭写 = resource.close(输出)**
*[总数, 关闭读.ok, 关闭写.ok]*
"#;
#[test]
fn file_copy_is_written_in_mqmd_through_chunked_streams() {
let dir = TempDir::new("file-copy");
write_file(&dir.path().join("data.txt"), "abcdef");
write_file(&dir.path().join("index.mq.md"), COPY);
let output = cli(
dir.path(),
&[
"run",
"index.mq.md",
"--allow-read",
"data.txt",
"--allow-write",
"out.bin",
"--json",
],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
let values = report["value"]["items"].as_array().expect("list result");
assert_eq!(values[0]["decimal"], "6");
assert_eq!(values[1]["value"], true);
assert_eq!(values[2]["value"], true);
assert_eq!(
std::fs::read(dir.path().join("out.bin")).unwrap(),
b"abcdef"
);
}
#[test]
fn tcp_stream_round_trips_through_mqmd_with_a_network_grant() {
use std::io::{Read, Write};
use std::net::TcpListener;
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let address = listener.local_addr().unwrap();
let server = std::thread::spawn(move || {
let (mut stream, _) = listener.accept().unwrap();
let mut buffer = [0u8; 4];
stream.read_exact(&mut buffer).unwrap();
stream.write_all(&buffer).unwrap();
});
let dir = TempDir::new("tcp");
let program = format!(
r#"## main
**内存 = memory.new(1, 1)**
**`内存[0] = 112`**
**`内存[1] = 105`**
**`内存[2] = 110`**
**`内存[3] = 103`**
**流 = resource.open("tcp", "{address}", ["read", "write"])**
**写入 = resource.write(流, 内存, 0, 4)**
**读出 = resource.read(流, 内存, 0, 4)**
**关闭 = resource.close(流)**
*[写入.value.count, 读出.value.count, 内存[0], 内存[1], 内存[2], 内存[3], 关闭.ok]*
"#
);
write_file(&dir.path().join("index.mq.md"), &program);
let output = cli(
dir.path(),
&["run", "index.mq.md", "--allow-net", "127.0.0.1", "--json"],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
let values = report["value"]["items"].as_array().expect("list");
assert_eq!(values[0]["decimal"], "4");
assert_eq!(values[1]["decimal"], "4");
for (index, expected) in ["112", "105", "110", "103"].into_iter().enumerate() {
assert_eq!(values[2 + index]["decimal"], expected);
}
assert_eq!(values[6]["value"], true);
server.join().unwrap();
}
#[test]
fn file_stream_requires_a_read_grant() {
let dir = TempDir::new("file-denied");
write_file(&dir.path().join("data.txt"), "abcdef");
write_file(&dir.path().join("index.mq.md"), READER);
let output = cli(dir.path(), &["run", "index.mq.md", "--json"]);
assert_eq!(output.status.code(), Some(1));
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], false);
assert_eq!(report["error"]["code"], "permission.denied");
}
+199
View File
@@ -0,0 +1,199 @@
//! Bundled stdlib (`std:`) and frozen-CLI closure tests (CM-A10).
use std::path::{Path, PathBuf};
use std::process::Command;
fn repo_root() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
fn binary() -> &'static str {
env!("CARGO_BIN_EXE_marqdo")
}
fn stdlib_dir() -> PathBuf {
repo_root().join("stdlib")
}
fn run(dir: &Path, stdlib: &Path, args: &[&str]) -> std::process::Output {
Command::new(binary())
.args(args)
.current_dir(dir)
.env("MARQDO_STDLIB_DIR", stdlib)
.output()
.expect("run marqdo")
}
struct TempDir(PathBuf);
impl TempDir {
fn new(tag: &str) -> Self {
let path = std::env::temp_dir().join(format!("marqdo-stdlib-{}-{tag}", std::process::id()));
let _ = std::fs::remove_dir_all(&path);
std::fs::create_dir_all(&path).unwrap();
Self(path)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn copy_stdlib(source: &Path, target: &Path) {
std::fs::create_dir_all(target).unwrap();
for entry in std::fs::read_dir(source).unwrap() {
let entry = entry.unwrap();
let path = entry.path();
if path.is_dir() {
copy_stdlib(&path, &target.join(entry.file_name()));
} else {
std::fs::copy(&path, target.join(entry.file_name())).unwrap();
}
}
}
fn write_project(dir: &Path, program: &str) {
std::fs::write(dir.join("index.mq.md"), program).unwrap();
}
const IMPORT_IO: &str = r#"---
import:
io: "std:io"
---
## main
> io.写值(["bundle", 1, 2.5])
*None*
"#;
#[test]
fn std_import_resolves_the_verified_bundle() {
let dir = TempDir::new("resolve");
write_project(dir.path(), IMPORT_IO);
let output = run(dir.path(), &stdlib_dir(), &["run", "index.mq.md"]);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
assert_eq!(
String::from_utf8_lossy(&output.stdout),
"[\"bundle\", 1, 2.5]\n"
);
}
#[test]
fn analysis_records_the_verified_stdlib_digest() {
let dir = TempDir::new("analysis");
write_project(dir.path(), IMPORT_IO);
let target = dir.path().join("analysis.json");
let output = run(
dir.path(),
&stdlib_dir(),
&["check", "index.mq.md", "--emit", target.to_str().unwrap()],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let analysis: serde_json::Value =
serde_json::from_slice(&std::fs::read(&target).unwrap()).unwrap();
let manifest: serde_json::Value =
serde_json::from_slice(&std::fs::read(stdlib_dir().join("stdlib.json")).unwrap()).unwrap();
assert_eq!(
analysis["project"]["stdlib"]["version"],
manifest["version"]
);
assert_eq!(analysis["project"]["stdlib"]["digest"], manifest["digest"]);
assert!(analysis["project"]["stdlib"]["instance_id"]
.as_str()
.unwrap()
.starts_with("std:marqdo-stdlib@"));
}
#[test]
fn frozen_binary_hash_is_unchanged_after_stdlib_execution() {
let before = std::fs::read(binary()).unwrap();
let before_hash = marqdo_value_sha(&before);
let dir = TempDir::new("frozen");
write_project(dir.path(), IMPORT_IO);
let output = run(dir.path(), &stdlib_dir(), &["run", "index.mq.md"]);
assert_eq!(output.status.code(), Some(0));
let after = std::fs::read(binary()).unwrap();
assert_eq!(before_hash, marqdo_value_sha(&after));
}
fn marqdo_value_sha(bytes: &[u8]) -> String {
use sha2::{Digest, Sha256};
let mut hasher = Sha256::new();
hasher.update(bytes);
hasher
.finalize()
.iter()
.map(|byte| format!("{byte:02x}"))
.collect()
}
#[test]
fn tampered_bundle_is_rejected_before_module_loading() {
let dir = TempDir::new("tamper");
let bundle = dir.path().join("stdlib");
copy_stdlib(&stdlib_dir(), &bundle);
std::fs::write(bundle.join("io.mq.md"), "## tampered\n").unwrap();
write_project(dir.path(), IMPORT_IO);
let output = run(dir.path(), &bundle, &["run", "index.mq.md"]);
assert!(!output.status.success());
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(stderr.contains("stdlib digest mismatch"), "{stderr}");
}
#[test]
fn bundle_uses_no_foreign_or_plugin_calls() {
fn visit(dir: &Path, offenders: &mut Vec<String>) {
for entry in std::fs::read_dir(dir).unwrap() {
let entry = entry.unwrap();
let path = entry.path();
if path.is_dir() {
visit(&path, offenders);
} else if path.to_string_lossy().ends_with(".mq.md") {
let source = std::fs::read_to_string(&path).unwrap();
if source.contains("foreign.") || source.contains("plugin.") {
offenders.push(path.display().to_string());
}
}
}
}
let mut offenders = Vec::new();
visit(&stdlib_dir(), &mut offenders);
assert!(
offenders.is_empty(),
"stdlib bundle must not depend on foreign/plugin calls: {offenders:?}"
);
}
#[test]
fn missing_bundle_manifest_is_a_hard_error() {
let dir = TempDir::new("missing");
let empty = dir.path().join("empty-stdlib");
std::fs::create_dir_all(&empty).unwrap();
write_project(dir.path(), IMPORT_IO);
let output = run(dir.path(), &empty, &["run", "index.mq.md"]);
assert!(!output.status.success());
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(stderr.contains("stdlib"), "{stderr}");
}
+40
View File
@@ -0,0 +1,40 @@
//! CM-A08/P5 stdlib HTTP framing module tests.
use std::path::Path;
use std::process::Command;
fn repo_root() -> std::path::PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
#[test]
fn stdlib_http_parses_chunked_responses_and_builds_requests() {
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", "tests/http.mq.md", "--json"])
.current_dir(repo_root().join("stdlib"))
.env("MARQDO_STDLIB_DIR", repo_root().join("stdlib"))
.output()
.expect("run http self-test");
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
assert_eq!(report["success"], true);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["decimal"], "200");
assert_eq!(items[1]["value"], "OK");
assert_eq!(items[2]["value"], "text/plain");
assert_eq!(items[3]["value"], "hello world");
assert_eq!(
items[4]["value"],
"POST /echo HTTP/1.1\r\nHost: example.test\r\nContent-Length: 2\r\n\r\nok"
);
}
+72
View File
@@ -0,0 +1,72 @@
//! CM-A08 table stdlib module tests.
use std::path::Path;
use std::process::Command;
fn repo_root() -> std::path::PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
fn run_table_self_test() -> serde_json::Value {
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", "tests/table.mq.md", "--json"])
.current_dir(repo_root().join("stdlib"))
.env("MARQDO_STDLIB_DIR", repo_root().join("stdlib"))
.output()
.expect("run table self-test");
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
serde_json::from_slice(&output.stdout).expect("json output")
}
fn rows(value: &serde_json::Value) -> &Vec<serde_json::Value> {
value["rows"].as_array().expect("rows")
}
#[test]
fn stdlib_table_algorithms_rebuild_without_mutating_inputs() {
let report = run_table_self_test();
assert_eq!(report["success"], true);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items.len(), 9);
assert_eq!(items[0]["header"], serde_json::json!(["分数"]));
assert_eq!(items[0]["rows"].as_array().unwrap().len(), 3);
assert_eq!(rows(&items[1]).len(), 2);
assert_eq!(rows(&items[2]).len(), 3);
assert_eq!(rows(&items[2])[0][1]["decimal"], "71");
assert_eq!(rows(&items[3]).len(), 4);
assert_eq!(rows(&items[3])[3][0]["value"], "丁");
assert_eq!(rows(&items[4])[0][1]["decimal"], "99");
assert_eq!(rows(&items[5]).len(), 2);
assert_eq!(items[5]["header"].as_array().unwrap().len(), 4);
assert_eq!(
items[6]["entries"]["a"]["items"].as_array().unwrap().len(),
2
);
assert_eq!(
items[6]["entries"]["b"]["items"].as_array().unwrap().len(),
1
);
// Original input is unchanged.
assert_eq!(rows(&items[7]).len(), 3);
assert_eq!(rows(&items[7])[0][1]["decimal"], "87");
assert_eq!(
items[8]["value"],
"| 名称 | 分数 |\n| --- | --- |\n| 甲 | 87 |\n| 乙 | 92 |\n| 丙 | 71 |"
);
}
+39 -4
View File
@@ -195,15 +195,50 @@ impl Expr {
}
}
/// The payload of a statement.
/// The writable target on the left of `=` (ADR-0022).
///
/// v1 accepted only [`AssignTarget::Local`]. RFC-CM-0001 adds the second
/// variant because addressable `Memory` needs a writable place; no other
/// collection becomes mutable. Both variants use the same assignment syntax
/// and the same statement position.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum StatementKind {
/// `<bare identifier> = <expression>`; the only assignment form in v1.
Assign {
#[serde(tag = "place", rename_all = "snake_case")]
pub enum AssignTarget {
/// `name = value`: rebind a local slot in the current frame.
Local {
/// Assigned identifier.
name: String,
/// Span of the identifier token (not the whole assignment).
name_span: Span,
},
/// `base[index] = value`: write one byte into an addressable `Memory`.
Index {
/// Base expression, evaluated first.
base: Box<Expr>,
/// Index expression, evaluated second.
index: Box<Expr>,
/// Span from the base start through the closing `]`.
span: Span,
},
}
impl AssignTarget {
/// Best known source span of the assignment target.
pub fn span(&self) -> Span {
match self {
AssignTarget::Local { name_span, .. } => *name_span,
AssignTarget::Index { span, .. } => *span,
}
}
}
/// The payload of a statement.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum StatementKind {
/// `target = expression`; see [`AssignTarget`] for the writable places.
Assign {
/// Assigned place.
target: AssignTarget,
/// Right-hand side expression.
value: Expr,
},
+5 -3
View File
@@ -36,8 +36,8 @@
//! The grammar is frozen in `spec/expression.ebnf`. In order of decreasing
//! precedence: prefix `not` / `-` / `+`, `**` (right associative), postfix
//! calls `.name` / `[index]`, `* / // %`, `+ -`, a single non-associative
//! comparison, `and`, `or`. Assignment is only available at statement start
//! and only for a bare identifier target.
//! comparison, `and`, `or`. Assignment is only available at statement start;
//! its target is a bare identifier or an index expression (`m[8] = 42`).
//!
//! Notable lexical rules:
//!
@@ -84,7 +84,9 @@ use std::fmt;
use serde::Serialize;
pub use ast::{Arg, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp};
pub use ast::{
Arg, AssignTarget, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp,
};
pub use marqdo_span::Span;
pub use parser::MAX_NESTING_DEPTH;
+106 -40
View File
@@ -17,7 +17,9 @@
//! recursing into any sub-expression so adversarial inputs fail with a
//! structured [`ParseError`] instead of overflowing the stack.
use crate::ast::{Arg, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp};
use crate::ast::{
Arg, AssignTarget, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp,
};
use crate::lexer::{self, Token, TokenKind};
use crate::ParseError;
use marqdo_span::Span;
@@ -47,21 +49,16 @@ pub(crate) fn parse_statement(source: &str) -> Result<Statement, ParseError> {
return Err(parser.error_here("expected a statement, found end of input"));
}
let start = parser.peek().span.start;
if let TokenKind::Ident(_) = parser.peek().kind {
if matches!(
parser.peek_n(1).map(|token| &token.kind),
Some(TokenKind::Assign)
) {
return parser.assignment(start);
}
let left = parser.expression()?;
if matches!(parser.peek().kind, TokenKind::Assign) {
return parser.assignment(start, left);
}
let expr = parser.expression()?;
if !parser.at_eof() {
return Err(parser.trailing_statement_error(&expr));
return Err(parser.trailing_statement_error());
}
let end = parser.previous_end();
Ok(Statement {
kind: StatementKind::Expr(expr),
kind: StatementKind::Expr(left),
span: Span::new(start, end),
})
}
@@ -201,11 +198,8 @@ impl Parser {
// Statements
// ------------------------------------------------------------------
fn assignment(&mut self, start: usize) -> Result<Statement, ParseError> {
let TokenKind::Ident(name) = self.peek().kind.clone() else {
return Err(self.error_here("expected an identifier on the left of `=`"));
};
let name_token = self.bump();
fn assignment(&mut self, start: usize, left: Expr) -> Result<Statement, ParseError> {
let target = self.assign_target(left)?;
self.bump(); // `=`
if self.at_eof() {
return Err(self.error_here("expected an expression after `=`"));
@@ -226,29 +220,40 @@ impl Parser {
}
let end = self.previous_end();
Ok(Statement {
kind: StatementKind::Assign {
name,
name_span: name_token.span,
value,
},
kind: StatementKind::Assign { target, value },
span: Span::new(start, end),
})
}
fn trailing_statement_error(&self, expr: &Expr) -> ParseError {
if matches!(self.peek().kind, TokenKind::Assign) {
return if matches!(expr, Expr::Member { .. } | Expr::Index { .. }) {
self.error_here(
"invalid assignment target: member and index assignment are not supported \
in v1; assign a bare identifier",
)
} else {
self.error_here(
"invalid assignment target: assignment requires a bare identifier on the \
left of `=`",
)
};
/// Turn the already-parsed left expression into a writable target.
///
/// v1 only writes local slots. Addressable `Memory` adds index targets;
/// member assignment and every other shape stay hard parse errors so an
/// invalid executable shape never silently becomes prose (ADR-0022).
fn assign_target(&self, left: Expr) -> Result<AssignTarget, ParseError> {
match left {
Expr::Ident(Ident { name, span }) => Ok(AssignTarget::Local {
name,
name_span: span,
}),
Expr::Index { base, index, span } => Ok(AssignTarget::Index { base, index, span }),
Expr::Member { span, .. } => Err(ParseError::new(
span,
"member assignment is not supported in v1; only a local name or a \
Memory index is writable",
)),
other => Err(ParseError::new(
other.span(),
format!(
"invalid assignment target: {} is not a writable place; assign a local \
name or a Memory index",
describe_expr(&other)
),
)),
}
}
fn trailing_statement_error(&self) -> ParseError {
self.error_here(format!(
"unexpected {} after expression",
self.describe_current()
@@ -705,6 +710,24 @@ fn is_keyword(kind: &TokenKind) -> bool {
)
}
fn describe_expr(expr: &Expr) -> &'static str {
match expr {
Expr::Int(_) => "an integer literal",
Expr::Float(_) => "a float literal",
Expr::Text(_) => "a text literal",
Expr::Bool(_) => "a boolean literal",
Expr::None => "`None`",
Expr::Ident(_) => "an identifier",
Expr::List(_) => "a list literal",
Expr::Record(_) => "a record literal",
Expr::Unary { .. } => "a unary expression",
Expr::Binary { .. } => "a binary expression",
Expr::Call { .. } => "a call expression",
Expr::Member { .. } => "a member access",
Expr::Index { .. } => "an index expression",
}
}
fn describe_token(kind: &TokenKind) -> String {
match kind {
TokenKind::Int(raw) => format!("integer literal `{raw}`"),
@@ -1154,14 +1177,19 @@ mod tests {
assert_eq!(
stmt.kind,
StatementKind::Assign {
name: "x".to_owned(),
name_span: Span::new(0, 1),
target: AssignTarget::Local {
name: "x".to_owned(),
name_span: Span::new(0, 1),
},
value: Expr::Int(BigInt::from(1)),
}
);
let stmt = st("结果 = 阶乘(n=3)");
match &stmt.kind {
StatementKind::Assign { name, value, .. } => {
StatementKind::Assign {
target: AssignTarget::Local { name, .. },
value,
} => {
assert_eq!(name, "结果");
assert_eq!(sexpr(value), "(call 阶乘 n=3)");
}
@@ -1170,6 +1198,40 @@ mod tests {
assert_eq!(st("x = y + 1").span(), Span::new(0, 9));
}
#[test]
fn memory_index_assignment_parses_as_a_place() {
let stmt = st("m[8] = 42");
let StatementKind::Assign { target, value } = &stmt.kind else {
panic!("expected assignment, got {:?}", stmt.kind);
};
let AssignTarget::Index { base, index, span } = target else {
panic!("expected index target, got {target:?}");
};
assert_eq!(
**base,
Expr::Ident(Ident {
name: "m".to_owned(),
span: Span::new(0, 1),
})
);
assert_eq!(**index, Expr::Int(BigInt::from(8)));
assert_eq!(*span, Span::new(0, 4));
assert_eq!(*value, Expr::Int(BigInt::from(42)));
assert_eq!(stmt.span, Span::new(0, 9));
// Nested targets are syntactically places; the runtime decides whether
// the base value is a writable Memory region.
let stmt = st("outer[0][1] = 2");
let StatementKind::Assign {
target: AssignTarget::Index { base, .. },
..
} = &stmt.kind
else {
panic!("expected nested index target, got {:?}", stmt.kind);
};
assert!(matches!(base.as_ref(), Expr::Index { .. }));
}
#[test]
fn expression_statements_parse() {
assert_eq!(st("打印(1)").kind, StatementKind::Expr(ex("打印(1)")));
@@ -1185,8 +1247,8 @@ mod tests {
"x =",
"x = 1 = 2",
"a.b = 1",
"a[0] = 1",
"1 = 2",
"f() = 1",
"= 1",
"x = 1 2",
] {
@@ -1194,7 +1256,8 @@ mod tests {
assert_eq!(err.code, "expr.parse", "input {bad:?}");
}
assert!(st_err("x = 1 = 2").message.contains("chained assignment"));
assert!(st_err("a.b = 1").message.contains("member and index"));
assert!(st_err("a.b = 1").message.contains("member assignment"));
assert!(st_err("1 = 2").message.contains("writable place"));
assert!(st_err("x =").message.contains("after `=`"));
}
@@ -1228,7 +1291,10 @@ mod tests {
let stmt = st(" x = 1 ");
assert_eq!(stmt.span(), Span::new(2, 7));
match stmt.kind {
StatementKind::Assign { name_span, .. } => assert_eq!(name_span, Span::new(2, 3)),
StatementKind::Assign {
target: AssignTarget::Local { name_span, .. },
..
} => assert_eq!(name_span, Span::new(2, 3)),
other => panic!("expected assignment, got {other:?}"),
}
+13 -5
View File
@@ -8,7 +8,7 @@
use std::collections::{BTreeMap, BTreeSet};
use marqdo_expression::{Arg, Expr, Ident};
use marqdo_ir::{builtins, Module, ModuleId, Param, Program, SignatureKind, Stmt};
use marqdo_ir::{builtins, Module, ModuleId, Param, PlaceTarget, Program, SignatureKind, Stmt};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
@@ -109,7 +109,10 @@ pub fn validate_program(program: &Program) -> Vec<(ModuleId, Diagnostic)> {
fn collect_assigned(stmts: &[Stmt], out: &mut BTreeSet<String>) {
for stmt in stmts {
match stmt {
Stmt::Assign { name, .. } => {
Stmt::Assign {
target: PlaceTarget::Local { name, .. },
..
} => {
out.insert(name.clone());
}
Stmt::Branch { arms, .. } => {
@@ -138,9 +141,14 @@ impl<'a> CallChecker<'a> {
fn stmts(&mut self, stmts: &[Stmt]) {
for stmt in stmts {
match stmt {
Stmt::Assign { value, .. }
| Stmt::Eval { expr: value, .. }
| Stmt::Return { value, .. } => {
Stmt::Assign { target, value, .. } => {
if let PlaceTarget::Index { base, index, .. } = target {
self.expr(base);
self.expr(index);
}
self.expr(value);
}
Stmt::Eval { expr: value, .. } | Stmt::Return { value, .. } => {
self.expr(value);
}
Stmt::Call { call, .. } => {
+41 -6
View File
@@ -17,7 +17,7 @@
use std::collections::{BTreeMap, BTreeSet};
use marqdo_expression::{Arg, Expr, Ident, RecordEntry};
use marqdo_ir::{BranchArm, Stmt};
use marqdo_ir::{BranchArm, PlaceTarget, Stmt};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
@@ -178,9 +178,24 @@ impl<'a> Analyzer<'a> {
fn analyze_stmts(&mut self, stmts: &[Stmt], mut state: State) -> State {
for stmt in stmts {
state = match stmt {
Stmt::Assign { name, value, .. } => {
self.read_expr(value, &state);
self.assign(name, &mut state);
Stmt::Assign { target, value, .. } => {
match target {
PlaceTarget::Local { name, .. } => {
// Preserve v1 order: the RHS is read before the
// local rebinding makes the name definitely
// assigned.
self.read_expr(value, &state);
self.assign(name, &mut state);
}
PlaceTarget::Index { base, index, .. } => {
// Runtime order for an index place is base, index,
// RHS; reads in the place are free reads like any
// other.
self.read_expr(base, &state);
self.read_expr(index, &state);
self.read_expr(value, &state);
}
}
state
}
Stmt::Eval { expr, .. } => {
@@ -312,8 +327,10 @@ mod tests {
fn assign(name: &str, value: &str) -> Stmt {
Stmt::Assign {
name: name.to_string(),
name_span: Span::new(0, 0),
target: PlaceTarget::Local {
name: name.to_string(),
name_span: Span::new(0, 0),
},
value: parse_expression(value).unwrap(),
span: Span::new(0, 0),
}
@@ -334,6 +351,24 @@ mod tests {
.collect()
}
#[test]
fn index_assignment_reads_its_base_and_index_without_binding_a_local() {
let stmts = vec![
Stmt::Assign {
target: PlaceTarget::Index {
base: parse_expression("m").unwrap(),
index: parse_expression("i").unwrap(),
span: Span::new(0, 0),
},
value: parse_expression("9").unwrap(),
span: Span::new(0, 0),
},
ret("m[0]"),
];
// `m` and `i` are free reads; the place never rebinds `m` as a local.
assert_eq!(params(&stmts), vec!["i".to_string(), "m".to_string()]);
}
#[test]
fn infers_free_reads() {
// n * 阶乘(n=n-1)
+17 -9
View File
@@ -6,9 +6,9 @@
//! diagnostics and never fall back to prose.
use marqdo_expression::{
parse_arguments, parse_expression, parse_statement, Arg, Expr, StatementKind,
parse_arguments, parse_expression, parse_statement, Arg, AssignTarget, Expr, StatementKind,
};
use marqdo_ir::{CallSite, CallTarget, ImportDecl, Stmt};
use marqdo_ir::{CallSite, CallTarget, ImportDecl, PlaceTarget, Stmt};
use marqdo_markdown::{normalize_name, MarkdownDocument, MdKind, MdNode};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
@@ -143,13 +143,21 @@ impl<'a> Mapper<'a> {
Ok(mut statement) => {
segments.rewrite_statement(&mut statement);
Some(match statement.kind {
StatementKind::Assign {
name,
name_span,
value,
} => Stmt::Assign {
name,
name_span,
StatementKind::Assign { target, value } => Stmt::Assign {
target: match target {
AssignTarget::Local { name, name_span } => {
PlaceTarget::Local { name, name_span }
}
AssignTarget::Index {
base,
index,
span: target_span,
} => PlaceTarget::Index {
base: *base,
index: *index,
span: target_span,
},
},
value,
span: node.span,
},
+12 -6
View File
@@ -5,7 +5,7 @@
//! string, so its spans are payload-relative; this module maps them back to the
//! original file so diagnostics point at real source.
use marqdo_expression::{Arg, Expr, RecordEntry, Statement, StatementKind};
use marqdo_expression::{Arg, AssignTarget, Expr, RecordEntry, Statement, StatementKind};
use marqdo_markdown::{MdKind, MdNode, PayloadError};
use marqdo_span::Span;
@@ -140,11 +140,17 @@ impl PayloadSegments {
/// Map and rewrite every span in a parsed statement.
pub fn rewrite_statement(&self, statement: &mut Statement) {
statement.span = self.map_span(statement.span);
if let StatementKind::Assign {
name_span, value, ..
} = &mut statement.kind
{
*name_span = self.map_span(*name_span);
if let StatementKind::Assign { target, value } = &mut statement.kind {
match target {
AssignTarget::Local { name_span, .. } => {
*name_span = self.map_span(*name_span);
}
AssignTarget::Index { base, index, span } => {
*span = self.map_span(*span);
self.rewrite_expr(base);
self.rewrite_expr(index);
}
}
self.rewrite_expr(value);
}
if let StatementKind::Expr(expr) = &mut statement.kind {
+1
View File
@@ -9,6 +9,7 @@ repository.workspace = true
authors.workspace = true
[dependencies]
libloading = { workspace = true }
marqdo-plugin-abi = { workspace = true }
glob = { workspace = true }
marqdo-span = { workspace = true }
@@ -0,0 +1,37 @@
/* Marqdo ResourceProvider C ABI v1 (ADR-0036).
*
* This ABI is for trusted native adapters, not a sandbox. The host loads the
* shared library, checks marqdo_resource_provider_abi_version() == 1 and calls
* these symbols for language-level resource.* operations.
*
* Conventions:
* - return 0 on success, nonzero on failure;
* - failure text is available from marqdo_resource_last_error();
* - rights is a comma-separated list (e.g. "read,write");
* - provider tokens are private to the adapter and must not be exposed to
* language Values, wire frames or trace output.
*/
#ifndef MARQDO_RESOURCE_PROVIDER_H
#define MARQDO_RESOURCE_PROVIDER_H
#include <stdint.h>
#define MARQDO_RESOURCE_PROVIDER_ABI_VERSION 1u
uint32_t marqdo_resource_provider_abi_version(void);
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
const char *rights_csv, uint64_t *out_token);
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
uint32_t capacity, uint32_t *out_read,
int32_t *out_eof);
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
uint32_t length, uint32_t *out_written);
int32_t marqdo_resource_close(uint64_t token);
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity);
#endif /* MARQDO_RESOURCE_PROVIDER_H */
+36
View File
@@ -323,6 +323,8 @@ pub struct Capabilities {
pub foreign: bool,
/// Allowed plugin names.
pub plugins: Vec<String>,
/// Allowed resource-provider names (ADR-0033).
pub resource_providers: Vec<String>,
}
impl Capabilities {
@@ -337,6 +339,7 @@ impl Capabilities {
process: false,
foreign: false,
plugins: Vec::new(),
resource_providers: Vec::new(),
}
}
@@ -371,6 +374,39 @@ impl Capabilities {
self
}
/// Grant network access to one IP literal host (ADR-0034).
pub fn allow_net(mut self, host: impl Into<String>) -> Self {
self.net_hosts.push(host.into());
self
}
/// Grant a registered resource provider by its stable name (ADR-0033).
pub fn allow_resource_provider(mut self, name: impl Into<String>) -> Self {
self.resource_providers.push(name.into());
self
}
/// Check a resource-provider grant.
pub fn check_resource_provider(
&self,
name: &str,
span: Option<Span>,
) -> Result<(), RuntimeError> {
if self
.resource_providers
.iter()
.any(|allowed| allowed == name)
{
Ok(())
} else {
Err(denied(
span,
format!("resource provider `{name}` was not granted"),
format!("grant it with Capabilities::allow_resource_provider({name:?})"),
))
}
}
/// Whether a project-relative path matches a granted pattern.
///
/// Patterns and paths are normalised by stripping a leading `./` so that
+30 -9
View File
@@ -83,6 +83,22 @@ impl ProjectFs {
/// Read a UTF-8 file, bounded by [`MAX_READ_BYTES`].
pub fn read(&self, user_path: &str, span: Option<Span>) -> Result<String, RuntimeError> {
let bytes = self.read_bytes(user_path, span)?;
String::from_utf8(bytes).map_err(|_| {
RuntimeError::new(
ErrorCategory::Type,
span,
format!("`{user_path}` is not valid UTF-8"),
)
})
}
/// Read a file as raw bytes, bounded by [`MAX_READ_BYTES`].
///
/// Used by the resource stream provider; path/capability checks are the
/// caller's responsibility for grants, while this method still rejects
/// traversal and symlink escapes.
pub fn read_bytes(&self, user_path: &str, span: Option<Span>) -> Result<Vec<u8>, RuntimeError> {
let (absolute, relative) = self.resolve(user_path, span)?;
let metadata = std::fs::metadata(&absolute).map_err(|error| {
RuntimeError::new(
@@ -118,19 +134,12 @@ impl ProjectFs {
if !canonical.starts_with(&self.root) {
return Err(invalid_path(span, "path escapes the project root"));
}
let bytes = std::fs::read(&canonical).map_err(|error| {
std::fs::read(&canonical).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot read `{relative}`: {error}"),
)
})?;
String::from_utf8(bytes).map_err(|_| {
RuntimeError::new(
ErrorCategory::Type,
span,
format!("`{relative}` is not valid UTF-8"),
)
})
}
@@ -140,6 +149,18 @@ impl ProjectFs {
user_path: &str,
contents: &str,
span: Option<Span>,
) -> Result<(), RuntimeError> {
self.write_bytes(user_path, contents.as_bytes(), span)
}
/// Write raw bytes inside the project root, creating parent directories.
///
/// This is a single validated write, not a transactional replace.
pub fn write_bytes(
&self,
user_path: &str,
bytes: &[u8],
span: Option<Span>,
) -> Result<(), RuntimeError> {
let (absolute, relative) = self.resolve(user_path, span)?;
// Validate the closest existing ancestor stays inside the root.
@@ -169,7 +190,7 @@ impl ProjectFs {
)
})?;
}
std::fs::write(&absolute, contents).map_err(|error| {
std::fs::write(&absolute, bytes).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
+515 -26
View File
@@ -5,6 +5,7 @@
//! execution, not at parse/compile time.
use std::collections::{BTreeMap, BTreeSet};
use std::net::{SocketAddr, TcpStream};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
@@ -12,7 +13,7 @@ use marqdo_plugin_abi::{
ErrorKind as PluginErrorKind, LoadedPlugin, PluginError, PluginKind, PluginManifest,
};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, RuntimeError, Value};
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value};
use crate::budget::Budget;
use crate::capability::{Capabilities, CapabilityAudit, CapabilityBroker};
@@ -20,14 +21,15 @@ use crate::context::ExecutionContext;
use crate::fs::ProjectFs;
use crate::handles::{contains_plugin_handle, PluginHandleTable};
use crate::plugin_worker::WorkerPlugin;
use crate::provider_c::{load_package, CResourceProvider};
use crate::provider_worker::WorkerResourceProvider;
use crate::resource::{ResourceProvider, ResourceRead, ResourceTable, SharedResourceProvider};
use crate::trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState};
use marqdo_sandbox::{PlatformCapabilities, SandboxMode};
/// Host operations reachable from the language surface.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum HostOp {
/// Write a value to program stdout.
Print,
/// Parse a formula.
MathParse,
/// Evaluate a formula.
@@ -54,7 +56,6 @@ impl HostOp {
/// Stable operation name used in traces and diagnostics.
pub const fn name(self) -> &'static str {
match self {
HostOp::Print => "print",
HostOp::MathParse => "math.parse",
HostOp::MathEval => "math.eval",
HostOp::MathSubstitute => "math.substitute",
@@ -79,6 +80,65 @@ pub trait Host {
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError>;
/// Open a resource/stream by kind, descriptor and requested rights.
///
/// Default: unavailable. Only the standard host implements the first
/// resource kinds (ADR-0025).
fn resource_open(
&mut self,
_kind: &str,
_descriptor: &str,
_rights: &[String],
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"this host does not provide resource streams",
))
}
/// Read up to `max_bytes` from a resource into host memory.
fn resource_read(
&mut self,
_handle: ResourceHandle,
_max_bytes: usize,
ctx: &ExecutionContext,
) -> Result<ResourceRead, RuntimeError> {
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"this host does not provide resource streams",
))
}
/// Write bytes to a resource; returns the accepted count.
fn resource_write(
&mut self,
_handle: ResourceHandle,
_bytes: &[u8],
ctx: &ExecutionContext,
) -> Result<usize, RuntimeError> {
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"this host does not provide resource streams",
))
}
/// Close a resource handle.
fn resource_close(
&mut self,
_handle: ResourceHandle,
ctx: &ExecutionContext,
) -> Result<(), RuntimeError> {
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"this host does not provide resource streams",
))
}
/// Consume evaluation fuel.
fn consume_fuel(&mut self, amount: u64, span: Option<Span>) -> Result<(), RuntimeError>;
@@ -143,6 +203,10 @@ pub struct StdHost {
broker_audit: Arc<Mutex<Vec<CapabilityAudit>>>,
/// Host-issued opaque plugin handles.
pub handles: PluginHandleTable,
/// Host-owned resource/stream table (ADR-0025).
resources: ResourceTable,
/// Registered in-process providers keyed by resource kind (ADR-0033).
resource_providers: BTreeMap<String, (String, SharedResourceProvider)>,
semantic_events: Vec<SemanticEvent>,
next_event_seq: u64,
trace_state: TraceState,
@@ -203,6 +267,8 @@ impl StdHost {
vendor_capabilities: BTreeSet::new(),
broker_audit: Arc::new(Mutex::new(Vec::new())),
handles: PluginHandleTable::default(),
resources: ResourceTable::new(),
resource_providers: BTreeMap::new(),
semantic_events: Vec::new(),
next_event_seq: 0,
trace_state: TraceState::default(),
@@ -417,7 +483,6 @@ impl StdHost {
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
match op {
HostOp::Print => self.op_print(args, ctx),
HostOp::FsRead => self.op_fs_read(args, ctx),
HostOp::FsWrite => self.op_fs_write(args, ctx),
HostOp::MathParse
@@ -432,27 +497,6 @@ impl StdHost {
}
}
fn op_print(
&mut self,
args: Vec<Value>,
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
if !self.capabilities.console {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"program output (console) capability was not granted",
));
}
if args.len() != 1 {
return Err(arity(ctx.span, "打印", 1, args.len()));
}
let text = format!("{}\n", args[0].display());
self.budget.consume_output(text.len(), ctx.span)?;
self.output.extend_from_slice(text.as_bytes());
Ok(Value::Unit)
}
fn op_fs_read(
&mut self,
args: Vec<Value>,
@@ -526,6 +570,327 @@ impl StdHost {
self.fs.write(&path, &contents, ctx.span)?;
Ok(Value::Unit)
}
/// Register an explicitly trusted in-process provider (ADR-0033).
///
/// In-process providers share the host address space and are not a
/// sandbox; only trusted adapters may be registered. Registration is not a
/// language operation and does not grant access by itself.
pub fn register_resource_provider(
&mut self,
provider: Box<dyn ResourceProvider>,
) -> Result<(), RuntimeError> {
self.register_provider_checked(provider)
}
/// Register a trusted C ABI resource provider (ADR-0036).
///
/// The shared library runs in the host address space and is **trusted**, not
/// sandboxed. Registration still requires the provider-name grant at
/// `resource.open` and is project-mode only.
pub fn register_c_resource_provider(
&mut self,
name: &str,
kinds: &[&str],
library: &std::path::Path,
) -> Result<(), RuntimeError> {
let kinds = kinds
.iter()
.map(|kind| (*kind).to_string())
.collect::<Vec<_>>();
let provider = CResourceProvider::open(name, &kinds, library)?;
self.register_provider_checked(Box::new(provider))
}
/// Register a trusted C ABI provider from a `marqdo.provider` manifest
/// (ADR-0037). The manifest is host-side installation metadata; language
/// imports cannot install or authorize a provider.
pub fn register_provider_package(
&mut self,
manifest_path: &std::path::Path,
) -> Result<(), RuntimeError> {
let provider = load_package(manifest_path)?;
self.register_provider_checked(Box::new(provider))
}
/// Register a trusted worker-process resource provider (ADR-0035).
///
/// The helper speaks framed JSON over stdin/stdout and is **not** an OS
/// sandbox; only explicitly trusted helpers may be spawned.
pub fn register_worker_resource_provider(
&mut self,
name: &str,
kinds: &[&str],
program: &std::path::Path,
args: &[String],
) -> Result<(), RuntimeError> {
let kinds = kinds
.iter()
.map(|kind| (*kind).to_string())
.collect::<Vec<_>>();
let provider = WorkerResourceProvider::spawn(name, &kinds, program, args)?;
self.register_provider_checked(Box::new(provider))
}
fn register_provider_checked(
&mut self,
provider: Box<dyn ResourceProvider>,
) -> Result<(), RuntimeError> {
if provider.protocol_version() != 1 {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"resource provider `{}` uses unsupported protocol version {}",
provider.name(),
provider.protocol_version()
),
));
}
let name = provider.name().to_string();
if name.is_empty() || name == "stdio" || name == "file" {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider name `{name}` is reserved or empty"),
));
}
let kinds = provider.kind_strings();
if kinds.is_empty() {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider `{name}` declares no kinds"),
));
}
for kind in &kinds {
if kind.is_empty() || kind == "stdio" || kind == "file" {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider `{name}` declares reserved kind `{kind}`"),
));
}
if let Some((existing, _)) = self.resource_providers.get(kind) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"resource kind `{kind}` is already registered by provider `{existing}`"
),
));
}
}
let provider: SharedResourceProvider = std::sync::Arc::new(std::sync::Mutex::new(provider));
for kind in kinds {
self.resource_providers
.insert(kind, (name.clone(), std::sync::Arc::clone(&provider)));
}
Ok(())
}
fn record_resource_effect(
&mut self,
op: &str,
ctx: &ExecutionContext,
started: Instant,
summary: &str,
) {
let estimate = TraceState::estimate(
"effect",
None,
Some(&ctx.module_id),
Some(summary),
Some(op),
);
if !self.trace_state.accept(estimate) {
return;
}
let seq = self.next_seq();
self.trace.record(
seq,
op,
ctx.module_id.clone(),
ctx.span,
started.elapsed(),
summary.to_string(),
);
}
fn open_resource_stream(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
let started = Instant::now();
let requested: BTreeSet<&str> = rights.iter().map(String::as_str).collect();
if self.mode == SandboxMode::Strict {
// stdio/stdout is the existing console channel already permitted by
// strict L1 (CM-A11); file/network/process providers are not.
let stdio_console =
kind == "stdio" && descriptor == "stdout" && requested == BTreeSet::from(["write"]);
if !stdio_console {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"strict mode permits only the existing stdio/stdout console channel; \
other resource streams are refused",
));
}
}
if self.mode == SandboxMode::Pure && kind == "file" {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"pure mode does not provide project file resources",
));
}
match kind {
"stdio" => {
if descriptor != "stdout" || requested != BTreeSet::from(["write"]) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"the first stream slice supports only stdio/stdout with rights [write]",
));
}
if !self.capabilities.console {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"the console capability is required for stdio/stdout",
));
}
let handle = self.resources.open_stdout();
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some("kind=stdio descriptor=stdout"),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
"file" => {
if requested == BTreeSet::from(["read"]) {
self.capabilities.check_read(descriptor, ctx.span)?;
let (_, relative) = self.fs.resolve(descriptor, ctx.span)?;
let bytes = self.fs.read_bytes(descriptor, ctx.span)?;
let handle = self.resources.open_file(relative, bytes);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some("kind=file rights=read"),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
return Ok(Value::Resource(handle));
}
if requested == BTreeSet::from(["write"]) {
self.capabilities.check_write(descriptor, ctx.span)?;
let (_, relative) = self.fs.resolve(descriptor, ctx.span)?;
let handle = self.resources.open_file_writer(relative);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some("kind=file rights=write"),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
return Ok(Value::Resource(handle));
}
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"project file streams support exactly [read] or [write] in this build",
))
}
"tcp" => {
if self.mode != SandboxMode::Project {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"tcp resources are available only in project mode",
));
}
if requested != BTreeSet::from(["read", "write"]) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"tcp resources require exactly [read, write] in this build",
));
}
let address: SocketAddr = descriptor.parse().map_err(|_| {
RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
format!(
"tcp descriptor `{descriptor}` must be an IP literal address:port (DNS names are not resolved in this build)"
),
)
})?;
self.capabilities
.check_net(&address.ip().to_string(), ctx.span)?;
let stream = TcpStream::connect_timeout(&address, Duration::from_secs(5)).map_err(
|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot connect to {address}: {error}"),
)
},
)?;
let _ = stream.set_read_timeout(Some(Duration::from_secs(5)));
let _ = stream.set_write_timeout(Some(Duration::from_secs(5)));
let handle = self.resources.open_tcp(stream);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind=tcp address={address}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
other => {
if self.mode != SandboxMode::Project {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"resource providers are available only in project mode",
));
}
let Some((provider_name, provider)) = self.resource_providers.get(other).cloned()
else {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
format!("resource kind `{other}` is not registered"),
));
};
self.capabilities
.check_resource_provider(&provider_name, ctx.span)?;
let handle = self
.resources
.open_provider(provider, other, descriptor, rights, ctx.span)?;
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind={other} provider={provider_name}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
}
}
}
impl Host for StdHost {
@@ -535,6 +900,31 @@ impl Host for StdHost {
args: Vec<Value>,
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
// Addressable memory is language-local state. Handles must not reach a
// plugin, foreign kernel or file adapter, where they could not be
// interpreted safely (RFC-CM-0001 §8, ADR-0022 §4).
if args.iter().any(Value::contains_memory) {
return Err(RuntimeError::new(
ErrorCategory::Type,
ctx.span,
format!(
"Memory values cannot cross the host boundary (`{}`); copy bytes into \
Text explicitly first",
op.name()
),
));
}
if args.iter().any(Value::contains_resource) {
return Err(RuntimeError::new(
ErrorCategory::Type,
ctx.span,
format!(
"Resource handles cannot cross the generic host boundary (`{}`); use \
resource.* operations",
op.name()
),
));
}
let started = Instant::now();
let result = self.dispatch(op, args, ctx);
let summary = match &result {
@@ -562,6 +952,105 @@ impl Host for StdHost {
result
}
fn resource_open(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
self.open_resource_stream(kind, descriptor, rights, ctx)
}
fn resource_read(
&mut self,
handle: ResourceHandle,
max_bytes: usize,
ctx: &ExecutionContext,
) -> Result<ResourceRead, RuntimeError> {
let started = Instant::now();
let kind = self.resources.kind(handle, ctx.span)?;
let read = self.resources.read(handle, max_bytes, ctx.span)?;
self.record_semantic(
"resource.read",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!(
"kind={kind} count={} eof={}",
read.bytes.len(),
read.eof
)),
);
self.record_resource_effect(
"resource.read",
ctx,
started,
&format!("kind={kind} count={}", read.bytes.len()),
);
Ok(read)
}
fn resource_write(
&mut self,
handle: ResourceHandle,
bytes: &[u8],
ctx: &ExecutionContext,
) -> Result<usize, RuntimeError> {
let started = Instant::now();
let kind = self.resources.kind(handle, ctx.span)?;
let count = self.resources.write(handle, bytes, ctx.span)?;
if kind == "stdio" {
self.budget.consume_output(count, ctx.span)?;
self.output.extend_from_slice(bytes);
}
self.record_semantic(
"resource.write",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind={kind} count={count}")),
);
self.record_resource_effect(
"resource.write",
ctx,
started,
&format!("kind={kind} count={count}"),
);
Ok(count)
}
fn resource_close(
&mut self,
handle: ResourceHandle,
ctx: &ExecutionContext,
) -> Result<(), RuntimeError> {
let started = Instant::now();
let pending = self.resources.pending_file_write(handle, ctx.span)?;
let count = match pending {
Some((relative, bytes)) => {
self.fs.write_bytes(&relative, &bytes, ctx.span)?;
bytes.len()
}
None => 0,
};
self.resources.close(handle, ctx.span)?;
self.record_semantic(
"resource.close",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("flushed_bytes={count}")),
);
self.record_resource_effect(
"resource.close",
ctx,
started,
&format!("flushed_bytes={count}"),
);
Ok(())
}
fn consume_fuel(&mut self, amount: u64, span: Option<Span>) -> Result<(), RuntimeError> {
self.budget.consume_fuel(amount, span)
}
+4
View File
@@ -12,6 +12,9 @@ pub mod handles;
pub mod host;
pub mod kernels;
mod plugin_worker;
mod provider_c;
mod provider_worker;
pub mod resource;
pub mod trace;
pub use budget::{Budget, DEFAULT_FUEL, DEFAULT_OUTPUT_BYTES, DEFAULT_STACK_LIMIT};
@@ -21,4 +24,5 @@ pub use fs::ProjectFs;
pub use handles::{contains_plugin_handle, PluginHandleTable};
pub use host::{arity, expect_text, Host, HostOp, StdHost};
pub use marqdo_sandbox::{PlatformCapabilities, SandboxMode, StrictIsolation};
pub use resource::{ResourceProvider, ResourceRead, ResourceTable, SharedResourceProvider};
pub use trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState, TraceTruncation};
+292
View File
@@ -0,0 +1,292 @@
//! C ABI resource provider v1 (ADR-0036).
//!
//! The library is trusted native code. Loading it shares the host address space
//! and is explicitly not a sandbox; only explicitly trusted adapters may be
//! registered.
use std::collections::BTreeMap;
use std::ffi::{c_char, CString};
use std::path::{Path, PathBuf};
use libloading::Library;
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, RuntimeError};
use crate::resource::{ResourceProvider, ResourceRead};
const ABI_VERSION: u32 = 1;
const ERROR_MAX: usize = 1024;
type AbiVersionFn = unsafe extern "C" fn() -> u32;
type OpenFn = unsafe extern "C" fn(*const c_char, *const c_char, *const c_char, *mut u64) -> i32;
type ReadFn = unsafe extern "C" fn(u64, *mut u8, u32, *mut u32, *mut i32) -> i32;
type WriteFn = unsafe extern "C" fn(u64, *const u8, u32, *mut u32) -> i32;
type CloseFn = unsafe extern "C" fn(u64) -> i32;
type LastErrorFn = unsafe extern "C" fn(*mut c_char, u32) -> i32;
#[derive(Debug, serde::Deserialize)]
struct ProviderPackage {
schema: String,
schema_version: u32,
name: String,
kinds: Vec<String>,
abi: String,
#[serde(default)]
library: BTreeMap<String, String>,
}
/// Load a C ABI provider from its package manifest (ADR-0037).
pub(crate) fn load_package(manifest_path: &Path) -> Result<CResourceProvider, RuntimeError> {
let bytes = std::fs::read(manifest_path).map_err(|error| {
RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"cannot read provider manifest {}: {error}",
manifest_path.display()
),
)
})?;
let package: ProviderPackage = serde_json::from_slice(&bytes).map_err(|error| {
RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"invalid provider manifest {}: {error}",
manifest_path.display()
),
)
})?;
if package.schema != "marqdo.provider" || package.schema_version != 1 {
return Err(provider_error(
&package.name,
"manifest schema must be marqdo.provider version 1",
));
}
if package.abi != "c-abi-1" {
return Err(provider_error(
&package.name,
&format!(
"unsupported provider ABI `{}` (expected c-abi-1)",
package.abi
),
));
}
let platform = if cfg!(target_os = "linux") {
"linux"
} else if cfg!(target_os = "macos") {
"macos"
} else if cfg!(target_os = "windows") {
"windows"
} else {
"default"
};
let library_name = package
.library
.get(platform)
.or_else(|| package.library.get("default"))
.ok_or_else(|| {
provider_error(
&package.name,
&format!("manifest has no `{platform}` or `default` library entry"),
)
})?;
let base = manifest_path.parent().unwrap_or_else(|| Path::new("."));
let library = PathBuf::from(library_name);
let library = if library.is_absolute() {
library
} else {
base.join(library)
};
CResourceProvider::open(&package.name, &package.kinds, &library)
}
pub(crate) struct CResourceProvider {
name: String,
kinds: Vec<String>,
library: Library,
}
impl CResourceProvider {
pub(crate) fn open(name: &str, kinds: &[String], path: &Path) -> Result<Self, RuntimeError> {
let library = unsafe { Library::new(path) }.map_err(|error| {
RuntimeError::new(
ErrorCategory::KernelUnavailable,
None,
format!(
"cannot load C resource provider `{name}` ({}): {error}",
path.display()
),
)
})?;
let version = {
let symbol: libloading::Symbol<AbiVersionFn> = unsafe {
library
.get(b"marqdo_resource_provider_abi_version\0")
.map_err(|error| {
RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("C resource provider `{name}` is missing abi_version: {error}"),
)
})?
};
unsafe { symbol() }
};
if version != ABI_VERSION {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"C resource provider `{name}` uses ABI version {version}; host requires {ABI_VERSION}"
),
));
}
Ok(Self {
name: name.to_string(),
kinds: kinds.to_vec(),
library,
})
}
fn open_fn(&self) -> Result<libloading::Symbol<'_, OpenFn>, RuntimeError> {
unsafe { self.library.get(b"marqdo_resource_open\0") }
.map_err(|error| provider_error(&self.name, &format!("missing open: {error}")))
}
fn read_fn(&self) -> Result<libloading::Symbol<'_, ReadFn>, RuntimeError> {
unsafe { self.library.get(b"marqdo_resource_read\0") }
.map_err(|error| provider_error(&self.name, &format!("missing read: {error}")))
}
fn write_fn(&self) -> Result<libloading::Symbol<'_, WriteFn>, RuntimeError> {
unsafe { self.library.get(b"marqdo_resource_write\0") }
.map_err(|error| provider_error(&self.name, &format!("missing write: {error}")))
}
fn close_fn(&self) -> Result<libloading::Symbol<'_, CloseFn>, RuntimeError> {
unsafe { self.library.get(b"marqdo_resource_close\0") }
.map_err(|error| provider_error(&self.name, &format!("missing close: {error}")))
}
fn last_error(&self) -> String {
let Ok(symbol) = (unsafe {
self.library
.get::<LastErrorFn>(b"marqdo_resource_last_error\0")
}) else {
return format!("C resource provider `{}` failed", self.name);
};
let mut buffer = vec![0i8; ERROR_MAX];
let code = unsafe { symbol(buffer.as_mut_ptr(), ERROR_MAX as u32) };
if code != 0 {
return format!("C resource provider `{}` failed", self.name);
}
let bytes = buffer
.iter()
.map(|value| *value as u8)
.take_while(|value| *value != 0)
.collect::<Vec<_>>();
String::from_utf8_lossy(&bytes).into_owned()
}
fn failure(&self, span: Option<Span>) -> RuntimeError {
let message = self.last_error();
RuntimeError::new(ErrorCategory::HostError, span, message)
}
}
impl ResourceProvider for CResourceProvider {
fn name(&self) -> &str {
&self.name
}
fn kinds(&self) -> &[&str] {
&[]
}
fn kind_strings(&self) -> Vec<String> {
self.kinds.clone()
}
fn open(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
span: Option<Span>,
) -> Result<u64, RuntimeError> {
let kind = CString::new(kind).map_err(|_| provider_error(&self.name, "kind has NUL"))?;
let descriptor = CString::new(descriptor)
.map_err(|_| provider_error(&self.name, "descriptor has NUL"))?;
let rights = CString::new(rights.join(","))
.map_err(|_| provider_error(&self.name, "rights have NUL"))?;
let mut token = 0u64;
let code = unsafe {
(self.open_fn()?)(
kind.as_ptr(),
descriptor.as_ptr(),
rights.as_ptr(),
&mut token,
)
};
if code != 0 {
return Err(self.failure(span));
}
Ok(token)
}
fn read(
&mut self,
token: u64,
max_bytes: usize,
span: Option<Span>,
) -> Result<ResourceRead, RuntimeError> {
let capacity = u32::try_from(max_bytes).unwrap_or(u32::MAX);
let mut buffer = vec![0u8; capacity as usize];
let mut read = 0u32;
let mut eof = 0i32;
let code =
unsafe { (self.read_fn()?)(token, buffer.as_mut_ptr(), capacity, &mut read, &mut eof) };
if code != 0 {
return Err(self.failure(span));
}
buffer.truncate(read as usize);
Ok(ResourceRead {
bytes: buffer,
eof: eof != 0,
})
}
fn write(
&mut self,
token: u64,
bytes: &[u8],
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let length = u32::try_from(bytes.len())
.map_err(|_| provider_error(&self.name, "write size exceeds the C ABI u32 limit"))?;
let mut written = 0u32;
let code = unsafe { (self.write_fn()?)(token, bytes.as_ptr(), length, &mut written) };
if code != 0 {
return Err(self.failure(span));
}
Ok(written as usize)
}
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError> {
let code = unsafe { (self.close_fn()?)(token) };
if code != 0 {
return Err(self.failure(span));
}
Ok(())
}
}
impl std::fmt::Debug for CResourceProvider {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("CResourceProvider")
.field("name", &self.name)
.field("kinds", &self.kinds)
.finish()
}
}
fn provider_error(name: &str, message: &str) -> RuntimeError {
RuntimeError::new(
ErrorCategory::HostError,
None,
format!("C resource provider `{name}`: {message}"),
)
}
+210
View File
@@ -0,0 +1,210 @@
//! Host-managed worker resource provider (ADR-0035).
//!
//! This is a small, versioned framed-JSON transport for **trusted helper
//! processes**. It is not a C ABI binding and not an OS sandbox: the helper
//! runs as the host user. Registration is a host embedding API; language code
//! only sees the normal `resource.*` operations.
use std::io::{BufReader, Read, Write};
use std::path::Path;
use std::process::{Child, ChildStdin, ChildStdout, Command, Stdio};
use marqdo_span::Span;
use serde_json::{json, Value as Json};
use crate::resource::{ResourceProvider, ResourceRead};
use marqdo_value::{ErrorCategory, RuntimeError};
const MAX_FRAME_BYTES: usize = 1024 * 1024;
/// A `ResourceProvider` backed by a child process speaking framed JSON.
pub(crate) struct WorkerResourceProvider {
name: String,
kinds: Vec<String>,
child: Child,
stdin: ChildStdin,
stdout: BufReader<ChildStdout>,
}
impl WorkerResourceProvider {
pub(crate) fn spawn(
name: &str,
kinds: &[String],
program: &Path,
args: &[String],
) -> Result<Self, RuntimeError> {
let mut child = Command::new(program)
.args(args)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::null())
.spawn()
.map_err(|error| {
RuntimeError::new(
ErrorCategory::KernelUnavailable,
None,
format!(
"cannot spawn worker resource provider `{name}` ({}): {error}",
program.display()
),
)
})?;
let stdin = child
.stdin
.take()
.ok_or_else(|| worker_error("missing stdin pipe"))?;
let stdout = child
.stdout
.take()
.ok_or_else(|| worker_error("missing stdout pipe"))?;
let mut provider = Self {
name: name.to_string(),
kinds: kinds.to_vec(),
child,
stdin,
stdout: BufReader::new(stdout),
};
// Handshake: the helper must answer a ping before it is trusted.
provider.request(json!({"op": "ping"}))?;
Ok(provider)
}
fn request(&mut self, request: Json) -> Result<Json, RuntimeError> {
let bytes = serde_json::to_vec(&request)
.map_err(|error| worker_error(&format!("cannot encode request: {error}")))?;
if bytes.len() > MAX_FRAME_BYTES {
return Err(worker_error("request frame is too large"));
}
self.stdin
.write_all(&(bytes.len() as u32).to_be_bytes())
.and_then(|()| self.stdin.write_all(&bytes))
.and_then(|()| self.stdin.flush())
.map_err(|error| worker_error(&format!("cannot write request frame: {error}")))?;
let mut header = [0u8; 4];
self.stdout
.read_exact(&mut header)
.map_err(|error| worker_error(&format!("cannot read response header: {error}")))?;
let length = u32::from_be_bytes(header) as usize;
if length == 0 || length > MAX_FRAME_BYTES {
return Err(worker_error("invalid response frame length"));
}
let mut payload = vec![0u8; length];
self.stdout
.read_exact(&mut payload)
.map_err(|error| worker_error(&format!("cannot read response frame: {error}")))?;
let response: Json = serde_json::from_slice(&payload)
.map_err(|error| worker_error(&format!("invalid response JSON: {error}")))?;
if response.get("ok").and_then(Json::as_bool) != Some(true) {
let code = response
.get("code")
.and_then(Json::as_str)
.unwrap_or("host.error");
let message = response
.get("message")
.and_then(Json::as_str)
.unwrap_or("worker resource provider failed");
let category = match code {
"permission.denied" => ErrorCategory::PermissionDenied,
"resource.closed" => ErrorCategory::ResourceClosed,
"resource.exhausted" => ErrorCategory::ResourceExhausted,
"resource.unsupported" => ErrorCategory::ResourceUnsupported,
_ => ErrorCategory::HostError,
};
return Err(RuntimeError::new(category, None, message.to_string()));
}
Ok(response)
}
}
impl ResourceProvider for WorkerResourceProvider {
fn name(&self) -> &str {
&self.name
}
fn kinds(&self) -> &[&str] {
// Owned kinds are exposed through `kind_strings`; registration never
// calls this for the worker transport.
&[]
}
fn kind_strings(&self) -> Vec<String> {
self.kinds.clone()
}
fn open(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
span: Option<Span>,
) -> Result<u64, RuntimeError> {
let response = self
.request(json!({
"op": "open",
"kind": kind,
"descriptor": descriptor,
"rights": rights,
}))
.map_err(|error| with_span(error, span))?;
response
.get("token")
.and_then(Json::as_u64)
.ok_or_else(|| worker_error("open response has no token"))
}
fn read(
&mut self,
token: u64,
max_bytes: usize,
span: Option<Span>,
) -> Result<ResourceRead, RuntimeError> {
let response = self
.request(json!({"op": "read", "token": token, "max_bytes": max_bytes}))
.map_err(|error| with_span(error, span))?;
let bytes = response
.get("bytes")
.and_then(Json::as_array)
.ok_or_else(|| worker_error("read response has no bytes"))?
.iter()
.map(|value| value.as_u64().and_then(|v| u8::try_from(v).ok()))
.collect::<Option<Vec<u8>>>()
.ok_or_else(|| worker_error("read response contains a non-byte value"))?;
let eof = response.get("eof").and_then(Json::as_bool).unwrap_or(false);
Ok(ResourceRead { bytes, eof })
}
fn write(
&mut self,
token: u64,
bytes: &[u8],
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let response = self
.request(json!({"op": "write", "token": token, "bytes": bytes}))
.map_err(|error| with_span(error, span))?;
response
.get("count")
.and_then(Json::as_u64)
.and_then(|count| usize::try_from(count).ok())
.ok_or_else(|| worker_error("write response has no count"))
}
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError> {
self.request(json!({"op": "close", "token": token}))
.map_err(|error| with_span(error, span))?;
Ok(())
}
}
impl Drop for WorkerResourceProvider {
fn drop(&mut self) {
let _ = self.request(json!({"op": "shutdown"}));
let _ = self.child.kill();
let _ = self.child.wait();
}
}
fn with_span(mut error: RuntimeError, span: Option<Span>) -> RuntimeError {
if error.span.is_none() {
error.span = span;
}
error
}
fn worker_error(message: &str) -> RuntimeError {
RuntimeError::new(ErrorCategory::HostError, None, message.to_string())
}
+539
View File
@@ -0,0 +1,539 @@
//! Host-owned resource/stream table (ADR-0025).
//!
//! The first slice deliberately supports only stdout writes and bounded
//! project-file reads loaded at open. Handles are run-scoped and opaque; the
//! table validates run identity, slot identity and generation on every call.
use std::collections::BTreeMap;
use std::io::{Read, Write};
use std::net::TcpStream;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError};
static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1);
/// A versioned trusted provider of additional resource kinds (ADR-0033).
///
/// In-process providers share the host address space and are **not** a
/// sandbox; only explicitly trusted code may be registered.
pub trait ResourceProvider: Send {
/// Stable grant name.
fn name(&self) -> &str;
/// Protocol version; must be 1 in this build.
fn protocol_version(&self) -> u32 {
1
}
/// Resource kinds served by this provider.
fn kinds(&self) -> &[&str];
/// Owned kind list used by registration. Providers with dynamically owned
/// kind strings override this.
fn kind_strings(&self) -> Vec<String> {
self.kinds()
.iter()
.map(|kind| (*kind).to_string())
.collect()
}
/// Open one descriptor, returning a provider-private token.
fn open(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
span: Option<Span>,
) -> Result<u64, RuntimeError>;
/// Read up to `max_bytes` from a token.
fn read(
&mut self,
token: u64,
max_bytes: usize,
span: Option<Span>,
) -> Result<ResourceRead, RuntimeError>;
/// Write bytes to a token; return the accepted count.
fn write(
&mut self,
token: u64,
bytes: &[u8],
span: Option<Span>,
) -> Result<usize, RuntimeError>;
/// Close a token.
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError>;
}
/// Shared provider handle registered on the host.
pub type SharedResourceProvider = Arc<Mutex<Box<dyn ResourceProvider>>>;
/// Bytes returned by one stream read.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ResourceRead {
/// Bytes actually read (may be empty at EOF).
pub bytes: Vec<u8>,
/// Whether the stream has reached EOF.
pub eof: bool,
}
enum Stream {
Stdout {
closed: bool,
},
File {
relative: String,
bytes: Vec<u8>,
position: usize,
closed: bool,
},
FileWriter {
relative: String,
bytes: Vec<u8>,
closed: bool,
},
Provider {
kind: String,
provider: SharedResourceProvider,
token: u64,
closed: bool,
},
Tcp {
stream: TcpStream,
closed: bool,
},
}
struct Slot {
generation: u64,
stream: Stream,
}
impl Slot {
fn closed(&self) -> bool {
match &self.stream {
Stream::Stdout { closed } => *closed,
Stream::File { closed, .. } => *closed,
Stream::FileWriter { closed, .. } => *closed,
Stream::Provider { closed, .. } => *closed,
Stream::Tcp { closed, .. } => *closed,
}
}
}
/// Host resource table for one run.
pub struct ResourceTable {
run_id: u64,
next_resource: u64,
slots: BTreeMap<u64, Slot>,
}
impl Default for ResourceTable {
fn default() -> Self {
Self::new()
}
}
impl ResourceTable {
/// Create an empty table with a fresh run identity.
pub fn new() -> Self {
Self {
run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed),
next_resource: 1,
slots: BTreeMap::new(),
}
}
/// Number of live (not necessarily open) slots.
pub fn slot_count(&self) -> usize {
self.slots.len()
}
/// Register the pre-opened stdout stream.
pub fn open_stdout(&mut self) -> ResourceHandle {
self.insert(Stream::Stdout { closed: false })
}
/// Register a project-file read stream opened from already-bounded bytes.
pub fn open_file(&mut self, relative: impl Into<String>, bytes: Vec<u8>) -> ResourceHandle {
self.insert(Stream::File {
relative: relative.into(),
bytes,
position: 0,
closed: false,
})
}
/// Register an empty buffered project-file write stream.
pub fn open_file_writer(&mut self, relative: impl Into<String>) -> ResourceHandle {
self.insert(Stream::FileWriter {
relative: relative.into(),
bytes: Vec::new(),
closed: false,
})
}
/// Register a connected TCP stream.
pub fn open_tcp(&mut self, stream: TcpStream) -> ResourceHandle {
self.insert(Stream::Tcp {
stream,
closed: false,
})
}
/// Open a provider token and wrap it in a host-issued handle.
pub fn open_provider(
&mut self,
provider: SharedResourceProvider,
kind: &str,
descriptor: &str,
rights: &[String],
span: Option<Span>,
) -> Result<ResourceHandle, RuntimeError> {
let token = {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.open(kind, descriptor, rights, span)?
};
Ok(self.insert(Stream::Provider {
kind: kind.to_string(),
provider,
token,
closed: false,
}))
}
fn insert(&mut self, stream: Stream) -> ResourceHandle {
let id = self.next_resource;
self.next_resource += 1;
self.slots.insert(
id,
Slot {
generation: 1,
stream,
},
);
ResourceHandle::from_host(self.run_id, id, 1)
}
fn slot(&self, handle: ResourceHandle, span: Option<Span>) -> Result<&Slot, RuntimeError> {
if handle.run_id() != self.run_id {
return Err(RuntimeError::new(
ErrorCategory::ResourceFault,
span,
"Resource handle belongs to a different host run",
));
}
let Some(slot) = self.slots.get(&handle.resource_id()) else {
return Err(RuntimeError::new(
ErrorCategory::ResourceFault,
span,
"unknown Resource handle",
));
};
if slot.generation != handle.generation() {
return Err(RuntimeError::new(
ErrorCategory::ResourceFault,
span,
"stale Resource handle",
));
}
Ok(slot)
}
fn slot_mut(
&mut self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<&mut Slot, RuntimeError> {
self.slot(handle, span)?;
self.slots
.get_mut(&handle.resource_id())
.ok_or_else(|| unknown_handle(span))
}
/// Read up to `max_bytes` from a file stream.
///
/// Reads at EOF return `count=0, eof=true`. A read on a closed stream is
/// `resource.closed`; stdout is not readable.
pub fn read(
&mut self,
handle: ResourceHandle,
max_bytes: usize,
span: Option<Span>,
) -> Result<ResourceRead, RuntimeError> {
let slot = self.slot_mut(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is closed",
));
}
match &mut slot.stream {
Stream::File {
bytes, position, ..
} => {
let start = *position;
let end = start.saturating_add(max_bytes).min(bytes.len());
let chunk = bytes[start..end].to_vec();
*position = end;
Ok(ResourceRead {
bytes: chunk,
eof: end >= bytes.len(),
})
}
Stream::Stdout { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"stdout does not support resource.read",
)),
Stream::FileWriter { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"a file write stream does not support resource.read",
)),
Stream::Provider {
provider, token, ..
} => {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.read(*token, max_bytes, span)
}
Stream::Tcp { stream, .. } => {
let mut buffer = vec![0u8; max_bytes];
let count = stream.read(&mut buffer).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("tcp read failed: {error}"),
)
})?;
buffer.truncate(count);
Ok(ResourceRead {
bytes: buffer,
eof: count == 0,
})
}
}
}
/// Validate a stdout write and return the accepted byte count.
///
/// The caller performs the output-budget accounting and buffer append; this
/// method only checks handle identity, closed state and stream kind.
pub fn write(
&mut self,
handle: ResourceHandle,
bytes: &[u8],
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let slot = self.slot_mut(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is closed",
));
}
match &mut slot.stream {
Stream::Stdout { .. } => Ok(bytes.len()),
Stream::File { relative, .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
format!("project file `{relative}` was opened read-only"),
)),
Stream::FileWriter {
relative,
bytes: buffer,
..
} => {
let next = buffer.len().checked_add(bytes.len());
let Some(next) = next else {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("file write buffer for `{relative}` overflows"),
));
};
if next > crate::fs::MAX_READ_BYTES as usize {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"file write buffer for `{relative}` exceeds the {} byte limit",
crate::fs::MAX_READ_BYTES
),
));
}
buffer.extend_from_slice(bytes);
Ok(bytes.len())
}
Stream::Provider {
provider, token, ..
} => {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.write(*token, bytes, span)
}
Stream::Tcp { stream, .. } => stream.write(bytes).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("tcp write failed: {error}"),
)
}),
}
}
/// Close a stream. A second close is `resource.closed`.
pub fn close(
&mut self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<(), RuntimeError> {
let slot = self.slot_mut(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is already closed",
));
}
match &mut slot.stream {
Stream::Stdout { closed } => *closed = true,
Stream::File { closed, .. } => *closed = true,
Stream::FileWriter { closed, .. } => *closed = true,
Stream::Provider {
provider,
token,
closed,
..
} => {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.close(*token, span)?;
*closed = true;
}
Stream::Tcp { stream, closed } => {
let _ = stream.shutdown(std::net::Shutdown::Both);
*closed = true;
}
}
Ok(())
}
/// Return the buffered bytes of a file write stream without closing it.
///
/// `None` means the handle is not a file writer (stdout or a reader). The
/// caller must call [`Self::close`] only after the flush succeeds, so a
/// failed flush leaves the writer open with its buffer intact.
pub fn pending_file_write(
&self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<Option<(String, Vec<u8>)>, RuntimeError> {
let slot = self.slot(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is closed",
));
}
match &slot.stream {
Stream::FileWriter {
relative, bytes, ..
} => Ok(Some((relative.clone(), bytes.clone()))),
_ => Ok(None),
}
}
/// Descriptor kind for diagnostics/trace, if the handle is valid.
pub fn kind(&self, handle: ResourceHandle, span: Option<Span>) -> Result<String, RuntimeError> {
let slot = self.slot(handle, span)?;
Ok(match &slot.stream {
Stream::Stdout { .. } => "stdio".to_string(),
Stream::File { .. } | Stream::FileWriter { .. } => "file".to_string(),
Stream::Provider { kind, .. } => kind.clone(),
Stream::Tcp { .. } => "tcp".to_string(),
})
}
}
fn provider_lock_error(span: Option<Span>) -> RuntimeError {
RuntimeError::new(
ErrorCategory::HostError,
span,
"resource provider lock is poisoned",
)
}
fn unknown_handle(span: Option<Span>) -> RuntimeError {
RuntimeError::new(
ErrorCategory::ResourceFault,
span,
"unknown Resource handle",
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn file_reads_are_chunked_with_eof() {
let mut table = ResourceTable::new();
let handle = table.open_file("data.txt", b"abcdef".to_vec());
assert_eq!(
table.read(handle, 2, None).unwrap(),
ResourceRead {
bytes: b"ab".to_vec(),
eof: false
}
);
assert_eq!(
table.read(handle, 8, None).unwrap(),
ResourceRead {
bytes: b"cdef".to_vec(),
eof: true
}
);
assert_eq!(
table.read(handle, 1, None).unwrap(),
ResourceRead {
bytes: Vec::new(),
eof: true
}
);
}
#[test]
fn forged_and_closed_handles_are_rejected() {
let mut table = ResourceTable::new();
let handle = table.open_stdout();
let forged = ResourceHandle::from_host(handle.run_id() + 1, handle.resource_id(), 1);
assert_eq!(
table.write(forged, b"x", None).unwrap_err().code(),
"runtime.resource"
);
table.close(handle, None).unwrap();
assert_eq!(
table.write(handle, b"x", None).unwrap_err().code(),
"resource.closed"
);
assert_eq!(
table.close(handle, None).unwrap_err().code(),
"resource.closed"
);
}
#[test]
fn stdout_is_not_readable_and_file_is_not_writable() {
let mut table = ResourceTable::new();
let stdout = table.open_stdout();
assert_eq!(
table.read(stdout, 1, None).unwrap_err().code(),
"resource.unsupported"
);
let file = table.open_file("x.txt", Vec::new());
assert_eq!(
table.write(file, b"x", None).unwrap_err().code(),
"resource.unsupported"
);
}
}
+241
View File
@@ -0,0 +1,241 @@
#![cfg(unix)]
use std::path::{Path, PathBuf};
use std::process::Command;
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_sandbox::SandboxMode;
use marqdo_value::ErrorCategory;
fn context() -> marqdo_host::ExecutionContext {
marqdo_host::ExecutionContext::new("test".to_string(), None)
}
fn handle(value: marqdo_value::Value) -> marqdo_value::ResourceHandle {
match value {
marqdo_value::Value::Resource(handle) => handle,
other => panic!("expected resource, got {other:?}"),
}
}
fn cc_available() -> bool {
Command::new("cc")
.arg("--version")
.output()
.map(|output| output.status.success())
.unwrap_or(false)
}
fn build_provider_with_version(dir: &Path, version: u32) -> PathBuf {
let source = dir.join("provider.c");
std::fs::write(
&source,
r#"
#include <stdint.h>
#include <stddef.h>
#include <string.h>
static const char DATA[] = "c-abi";
static size_t position = 0;
static uint64_t next_token = 1;
#ifndef PROVIDER_VERSION
#define PROVIDER_VERSION 1u
#endif
uint32_t marqdo_resource_provider_abi_version(void) { return PROVIDER_VERSION; }
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
const char *rights_csv, uint64_t *out_token) {
(void)kind; (void)descriptor; (void)rights_csv;
position = 0;
*out_token = next_token++;
return 0;
}
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
uint32_t capacity, uint32_t *out_read,
int32_t *out_eof) {
(void)token;
size_t remaining = sizeof(DATA) - 1 - position;
size_t count = remaining < capacity ? remaining : capacity;
memcpy(buffer, DATA + position, count);
position += count;
*out_read = (uint32_t)count;
*out_eof = position >= sizeof(DATA) - 1 ? 1 : 0;
return 0;
}
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
uint32_t length, uint32_t *out_written) {
(void)token; (void)bytes;
*out_written = length;
return 0;
}
int32_t marqdo_resource_close(uint64_t token) { (void)token; return 0; }
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity) {
if (capacity > 0) buffer[0] = 0;
return 0;
}
"#,
)
.unwrap();
let extension = if cfg!(target_os = "macos") {
"dylib"
} else {
"so"
};
let library = dir.join(format!("libecho_provider.{extension}"));
let status = Command::new("cc")
.args([
"-shared",
"-fPIC",
"-O2",
&format!("-DPROVIDER_VERSION={version}"),
"-o",
])
.arg(&library)
.arg(&source)
.status()
.expect("run cc");
assert!(status.success(), "C provider build failed");
library
}
fn build_provider(dir: &Path) -> PathBuf {
build_provider_with_version(dir, 1)
}
#[test]
fn c_abi_provider_round_trips() {
if !cc_available() {
eprintln!("skipping C provider test: cc is unavailable");
return;
}
let dir = tempfile::tempdir().unwrap();
let library = build_provider(dir.path());
let project = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("cabi"),
project.path(),
);
host.register_c_resource_provider("cabi", &["cabi.sensor"], &library)
.unwrap();
let ctx = context();
let handle = handle(
host.resource_open("cabi.sensor", "sensor-1", &["read".to_string()], &ctx)
.unwrap(),
);
let first = host.resource_read(handle, 2, &ctx).unwrap();
assert_eq!(first.bytes, b"c-");
assert!(!first.eof);
let rest = host.resource_read(handle, 16, &ctx).unwrap();
assert_eq!(rest.bytes, b"abi");
assert!(rest.eof);
assert_eq!(host.resource_write(handle, b"xyz", &ctx).unwrap(), 3);
host.resource_close(handle, &ctx).unwrap();
assert_eq!(
host.resource_close(handle, &ctx).unwrap_err().code(),
"resource.closed"
);
}
#[test]
fn c_abi_provider_loading_fails_closed() {
let project = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("cabi"),
project.path(),
);
let error = host
.register_c_resource_provider(
"cabi",
&["cabi.sensor"],
Path::new("/definitely/missing/marqdo-provider.so"),
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::KernelUnavailable);
}
#[test]
fn c_abi_version_mismatch_and_modes_fail_closed() {
if !cc_available() {
eprintln!("skipping C provider test: cc is unavailable");
return;
}
let dir = tempfile::tempdir().unwrap();
let wrong = build_provider_with_version(dir.path(), 2);
let project = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("cabi"),
project.path(),
);
let error = host
.register_c_resource_provider("cabi", &["cabi.sensor"], &wrong)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceUnsupported);
let good = build_provider(dir.path());
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
let mut host = StdHost::new(
mode,
Capabilities::project().allow_resource_provider("cabi"),
project.path(),
);
host.register_c_resource_provider("cabi", &["cabi.sensor"], &good)
.unwrap();
let error = host
.resource_open("cabi.sensor", "s", &["read".to_string()], &context())
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
}
#[test]
fn provider_package_manifest_installs_a_c_abi_library() {
if !cc_available() {
eprintln!("skipping C provider test: cc is unavailable");
return;
}
let dir = tempfile::tempdir().unwrap();
let library = build_provider(dir.path());
let manifest = dir.path().join("provider.json");
let platform = if cfg!(target_os = "linux") {
"linux"
} else if cfg!(target_os = "macos") {
"macos"
} else {
"default"
};
std::fs::write(
&manifest,
serde_json::json!({
"schema": "marqdo.provider",
"schema_version": 1,
"name": "cabi",
"kinds": ["cabi.sensor"],
"abi": "c-abi-1",
"library": { platform: library.file_name().unwrap().to_string_lossy() }
})
.to_string(),
)
.unwrap();
let project = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("cabi"),
project.path(),
);
host.register_provider_package(&manifest).unwrap();
let ctx = context();
let handle = handle(
host.resource_open("cabi.sensor", "s", &["read".to_string()], &ctx)
.unwrap(),
);
let read = host.resource_read(handle, 2, &ctx).unwrap();
assert_eq!(read.bytes, b"c-");
}
@@ -0,0 +1,456 @@
//! Resource/stream host boundary tests (ADR-0025).
use std::path::Path;
use marqdo_host::{Capabilities, ExecutionContext, Host, HostOp, StdHost};
use marqdo_sandbox::SandboxMode;
use marqdo_value::{ErrorCategory, Value};
fn context() -> ExecutionContext {
ExecutionContext::new("index.mq.md".to_string(), None)
}
fn handle(value: Value) -> marqdo_value::ResourceHandle {
match value {
Value::Resource(handle) => handle,
other => panic!("expected a Resource handle, got {other:?}"),
}
}
fn write_project_file(root: &Path, name: &str, contents: &[u8]) {
std::fs::write(root.join(name), contents).unwrap();
}
#[test]
fn pure_mode_writes_stdout_through_a_resource() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let ctx = context();
let handle = handle(
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"hello", &ctx).unwrap(), 5);
assert_eq!(host.take_output(), b"hello");
host.resource_close(handle, &ctx).unwrap();
let error = host.resource_close(handle, &ctx).unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceClosed);
assert_eq!(error.code(), "resource.closed");
}
#[test]
fn stdout_write_obeys_the_output_budget_atomically() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
host.budget.max_output_bytes = 3;
let ctx = context();
let handle = handle(
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
.unwrap(),
);
let error = host.resource_write(handle, b"hello", &ctx).unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
assert!(host.take_output().is_empty());
}
#[test]
fn strict_mode_allows_only_the_existing_stdio_console_channel() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Strict, Capabilities::pure(), dir.path());
let ctx = context();
let handle = handle(
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"strict", &ctx).unwrap(), 6);
assert_eq!(host.take_output(), b"strict");
let file = host
.resource_open("file", "a.txt", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(file.category, ErrorCategory::PermissionDenied);
let socket = host
.resource_open("socket", "example.test", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(socket.category, ErrorCategory::PermissionDenied);
}
#[test]
fn resource_handles_cannot_cross_the_generic_host_boundary() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let ctx = context();
let handle = handle(
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
.unwrap(),
);
let error = host
.invoke(HostOp::FsRead, vec![Value::Resource(handle)], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::Type);
assert!(error.message.contains("Resource handles"));
}
#[test]
fn pure_mode_refuses_file_resources_even_with_a_grant() {
let dir = tempfile::tempdir().unwrap();
write_project_file(dir.path(), "a.txt", b"abcdef");
let mut host = StdHost::new(
SandboxMode::Pure,
Capabilities::pure().allow_read("a.txt"),
dir.path(),
);
let error = host
.resource_open("file", "a.txt", &["read".to_string()], &context())
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn project_file_writes_buffer_until_close_and_require_a_grant() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_write("out.bin"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open("file", "out.bin", &["write".to_string()], &ctx)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"abc", &ctx).unwrap(), 3);
assert_eq!(host.resource_write(handle, b"def", &ctx).unwrap(), 3);
assert!(
!dir.path().join("out.bin").exists(),
"write must not flush early"
);
host.resource_close(handle, &ctx).unwrap();
assert_eq!(
std::fs::read(dir.path().join("out.bin")).unwrap(),
b"abcdef"
);
let error = host.resource_read(handle, 1, &ctx).unwrap_err();
assert_eq!(error.code(), "resource.closed");
let second = host.resource_close(handle, &ctx).unwrap_err();
assert_eq!(second.code(), "resource.closed");
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = denied
.resource_open("file", "out.bin", &["write".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn file_write_buffer_has_a_hard_ceiling() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_write("big.bin"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open("file", "big.bin", &["write".to_string()], &ctx)
.unwrap(),
);
let oversized = vec![0u8; marqdo_host::fs::MAX_READ_BYTES as usize + 1];
let error = host.resource_write(handle, &oversized, &ctx).unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
assert_eq!(host.resource_write(handle, b"ok", &ctx).unwrap(), 2);
host.resource_close(handle, &ctx).unwrap();
assert_eq!(std::fs::read(dir.path().join("big.bin")).unwrap(), b"ok");
}
#[test]
fn project_file_reads_are_chunked_and_require_a_grant() {
let dir = tempfile::tempdir().unwrap();
write_project_file(dir.path(), "a.txt", b"abcdef");
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_read("a.txt"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open("file", "a.txt", &["read".to_string()], &ctx)
.unwrap(),
);
let first = host.resource_read(handle, 2, &ctx).unwrap();
assert_eq!(first.bytes, b"ab");
assert!(!first.eof);
let second = host.resource_read(handle, 8, &ctx).unwrap();
assert_eq!(second.bytes, b"cdef");
assert!(second.eof);
assert!(host.resource_read(handle, 1, &ctx).unwrap().eof);
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = denied
.resource_open("file", "a.txt", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn traversal_and_unsupported_rights_are_refused() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_read("**"),
dir.path(),
);
let ctx = context();
let traversal = host
.resource_open("file", "../secret", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(traversal.category, ErrorCategory::PermissionDenied);
let write_rights = host
.resource_open("stdio", "stdout", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(write_rights.category, ErrorCategory::ResourceUnsupported);
let unknown_kind = host
.resource_open("socket", "example.test", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(unknown_kind.category, ErrorCategory::ResourceUnsupported);
}
struct MockProvider {
data: Vec<u8>,
position: usize,
written: Vec<u8>,
version: u32,
kinds: Vec<&'static str>,
}
impl MockProvider {
fn new(data: &[u8], version: u32, kinds: Vec<&'static str>) -> Self {
Self {
data: data.to_vec(),
position: 0,
written: Vec::new(),
version,
kinds,
}
}
}
impl marqdo_host::ResourceProvider for MockProvider {
fn name(&self) -> &str {
"mock"
}
fn protocol_version(&self) -> u32 {
self.version
}
fn kinds(&self) -> &[&str] {
&self.kinds
}
fn open(
&mut self,
_kind: &str,
_descriptor: &str,
_rights: &[String],
_span: Option<marqdo_span::Span>,
) -> Result<u64, marqdo_value::RuntimeError> {
self.position = 0;
Ok(7)
}
fn read(
&mut self,
token: u64,
max_bytes: usize,
_span: Option<marqdo_span::Span>,
) -> Result<marqdo_host::ResourceRead, marqdo_value::RuntimeError> {
assert_eq!(token, 7);
let start = self.position;
let end = start.saturating_add(max_bytes).min(self.data.len());
self.position = end;
Ok(marqdo_host::ResourceRead {
bytes: self.data[start..end].to_vec(),
eof: end >= self.data.len(),
})
}
fn write(
&mut self,
token: u64,
bytes: &[u8],
_span: Option<marqdo_span::Span>,
) -> Result<usize, marqdo_value::RuntimeError> {
assert_eq!(token, 7);
self.written.extend_from_slice(bytes);
Ok(bytes.len())
}
fn close(
&mut self,
token: u64,
_span: Option<marqdo_span::Span>,
) -> Result<(), marqdo_value::RuntimeError> {
assert_eq!(token, 7);
Ok(())
}
}
#[test]
fn registered_provider_streams_work_with_a_grant_in_project_mode() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("mock"),
dir.path(),
);
host.register_resource_provider(Box::new(MockProvider::new(
b"abcdef",
1,
vec!["mock.sensor"],
)))
.unwrap();
let ctx = context();
let handle = handle(
host.resource_open("mock.sensor", "sensor-1", &["read".to_string()], &ctx)
.unwrap(),
);
let first = host.resource_read(handle, 2, &ctx).unwrap();
assert_eq!(first.bytes, b"ab");
assert!(!first.eof);
let rest = host.resource_read(handle, 16, &ctx).unwrap();
assert_eq!(rest.bytes, b"cdef");
assert!(rest.eof);
assert_eq!(host.resource_write(handle, b"x", &ctx).unwrap(), 1);
host.resource_close(handle, &ctx).unwrap();
assert_eq!(
host.resource_close(handle, &ctx).unwrap_err().code(),
"resource.closed"
);
}
#[test]
fn provider_requires_a_name_grant_and_project_mode() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
denied
.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
.unwrap();
let error = denied
.resource_open("mock.sensor", "s", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
let mut host = StdHost::new(
mode,
Capabilities::pure().allow_resource_provider("mock"),
dir.path(),
);
host.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
.unwrap();
let error = host
.resource_open("mock.sensor", "s", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
}
#[test]
fn provider_registration_rejects_versions_reserved_names_and_conflicts() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("mock"),
dir.path(),
);
let bad_version = host
.register_resource_provider(Box::new(MockProvider::new(b"x", 2, vec!["mock.sensor"])))
.unwrap_err();
assert_eq!(bad_version.category, ErrorCategory::ResourceUnsupported);
let reserved = host
.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["stdio"])))
.unwrap_err();
assert_eq!(reserved.category, ErrorCategory::ResourceUnsupported);
host.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
.unwrap();
let conflict = host
.register_resource_provider(Box::new(MockProvider::new(b"y", 1, vec!["mock.sensor"])))
.unwrap_err();
assert_eq!(conflict.category, ErrorCategory::ResourceUnsupported);
}
#[test]
fn tcp_resource_round_trips_with_a_network_grant() {
use std::io::{Read, Write};
use std::net::TcpListener;
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let address = listener.local_addr().unwrap();
let server = std::thread::spawn(move || {
let (mut stream, _) = listener.accept().unwrap();
let mut buffer = [0u8; 4];
stream.read_exact(&mut buffer).unwrap();
stream.write_all(&buffer).unwrap();
});
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_net("127.0.0.1"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open(
"tcp",
&address.to_string(),
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"ping", &ctx).unwrap(), 4);
let read = host.resource_read(handle, 4, &ctx).unwrap();
assert_eq!(read.bytes, b"ping");
host.resource_close(handle, &ctx).unwrap();
server.join().unwrap();
}
#[test]
fn tcp_requires_network_grant_and_rejects_dns_names() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let ctx = context();
let denied = host
.resource_open(
"tcp",
"127.0.0.1:9",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(denied.category, ErrorCategory::PermissionDenied);
let dns = host
.resource_open(
"tcp",
"example.test:80",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(dns.category, ErrorCategory::ResourceUnsupported);
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
let mut host = StdHost::new(
mode,
Capabilities::project().allow_net("127.0.0.1"),
dir.path(),
);
let error = host
.resource_open(
"tcp",
"127.0.0.1:9",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
}
+162
View File
@@ -0,0 +1,162 @@
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_sandbox::SandboxMode;
use marqdo_value::ErrorCategory;
fn context() -> marqdo_host::ExecutionContext {
marqdo_host::ExecutionContext::new("test".to_string(), None)
}
fn handle(value: marqdo_value::Value) -> marqdo_value::ResourceHandle {
match value {
marqdo_value::Value::Resource(handle) => handle,
other => panic!("expected resource, got {other:?}"),
}
}
fn helper() -> (tempfile::TempDir, std::path::PathBuf) {
let dir = tempfile::tempdir().unwrap();
let script = dir.path().join("provider.py");
std::fs::write(
&script,
r#"
import sys, json, struct
def read_frame():
header = sys.stdin.buffer.read(4)
if len(header) < 4:
return None
size = struct.unpack(">I", header)[0]
return json.loads(sys.stdin.buffer.read(size))
def write_frame(value):
payload = json.dumps(value).encode()
sys.stdout.buffer.write(struct.pack(">I", len(payload)) + payload)
sys.stdout.buffer.flush()
data = b"hello world"
position = 0
while True:
request = read_frame()
if request is None:
break
op = request["op"]
if op == "ping":
write_frame({"ok": True})
elif op == "open":
write_frame({"ok": True, "token": 7})
elif op == "read":
count = request["max_bytes"]
chunk = data[position:position + count]
position += len(chunk)
write_frame({"ok": True, "bytes": list(chunk), "eof": position >= len(data)})
elif op == "write":
write_frame({"ok": True, "count": len(request["bytes"])})
elif op == "close":
write_frame({"ok": True})
elif op == "shutdown":
break
else:
write_frame({"ok": False, "code": "resource.unsupported", "message": op})
"#,
)
.unwrap();
(dir, script)
}
#[test]
fn worker_provider_round_trips_through_framed_json() {
let (dir, script) = helper();
let project = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("worker"),
project.path(),
);
host.register_worker_resource_provider(
"worker",
&["worker.sensor"],
&std::path::PathBuf::from("python3"),
&[script.to_string_lossy().to_string()],
)
.unwrap();
let ctx = context();
let handle = handle(
host.resource_open("worker.sensor", "sensor-1", &["read".to_string()], &ctx)
.unwrap(),
);
let first = host.resource_read(handle, 5, &ctx).unwrap();
assert_eq!(first.bytes, b"hello");
assert!(!first.eof);
let rest = host.resource_read(handle, 64, &ctx).unwrap();
assert_eq!(rest.bytes, b" world");
assert!(rest.eof);
assert_eq!(host.resource_write(handle, b"abc", &ctx).unwrap(), 3);
host.resource_close(handle, &ctx).unwrap();
assert_eq!(
host.resource_close(handle, &ctx).unwrap_err().code(),
"resource.closed"
);
drop(dir);
}
#[test]
fn worker_provider_registration_requires_a_grant_and_project_mode() {
let (_dir, script) = helper();
let project = tempfile::tempdir().unwrap();
let ctx = context();
let mut denied = StdHost::new(
SandboxMode::Project,
Capabilities::project(),
project.path(),
);
denied
.register_worker_resource_provider(
"worker",
&["worker.sensor"],
&std::path::PathBuf::from("python3"),
&[script.to_string_lossy().to_string()],
)
.unwrap();
let error = denied
.resource_open("worker.sensor", "s", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
let mut host = StdHost::new(
mode,
Capabilities::project().allow_resource_provider("worker"),
project.path(),
);
host.register_worker_resource_provider(
"worker",
&["worker.sensor"],
&std::path::PathBuf::from("python3"),
&[script.to_string_lossy().to_string()],
)
.unwrap();
let error = host
.resource_open("worker.sensor", "s", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
}
#[test]
fn missing_worker_program_is_a_structured_error() {
let project = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("worker"),
project.path(),
);
let error = host
.register_worker_resource_provider(
"worker",
&["worker.sensor"],
&std::path::PathBuf::from("/definitely/missing/marqdo-provider"),
&[],
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::KernelUnavailable);
}
+157
View File
@@ -167,6 +167,144 @@ const FS_WRITE: Builtin = Builtin {
named_only: false,
effectful: true,
};
const MEMORY_NEW: Builtin = Builtin {
name: "memory.new",
aliases: &[],
params: &[req("initial_pages"), req("max_pages")],
named_only: false,
effectful: false,
};
const MEMORY_SIZE: Builtin = Builtin {
name: "memory.size",
aliases: &[],
params: &[req("memory")],
named_only: false,
effectful: false,
};
const MEMORY_GROW: Builtin = Builtin {
name: "memory.grow",
aliases: &[],
params: &[req("memory"), req("pages")],
named_only: false,
effectful: false,
};
const LIST_APPEND: Builtin = Builtin {
name: "list.append",
aliases: &[],
params: &[req("list"), req("value")],
named_only: false,
effectful: false,
};
const LIST_CONCAT: Builtin = Builtin {
name: "list.concat",
aliases: &[],
params: &[req("left"), req("right")],
named_only: false,
effectful: false,
};
const MAP_KEYS: Builtin = Builtin {
name: "map.keys",
aliases: &[],
params: &[req("map")],
named_only: false,
effectful: false,
};
const MAP_HAS: Builtin = Builtin {
name: "map.has",
aliases: &[],
params: &[req("map"), req("key")],
named_only: false,
effectful: false,
};
const MAP_GET: Builtin = Builtin {
name: "map.get",
aliases: &[],
params: &[req("map"), req("key")],
named_only: false,
effectful: false,
};
const MAP_SET: Builtin = Builtin {
name: "map.set",
aliases: &[],
params: &[req("map"), req("key"), req("value")],
named_only: false,
effectful: false,
};
const MAP_REMOVE: Builtin = Builtin {
name: "map.remove",
aliases: &[],
params: &[req("map"), req("key")],
named_only: false,
effectful: false,
};
const TEXT_CODEPOINTS: Builtin = Builtin {
name: "text.codepoints",
aliases: &[],
params: &[req("text")],
named_only: false,
effectful: false,
};
const TEXT_FROM_CODEPOINT: Builtin = Builtin {
name: "text.from_codepoint",
aliases: &[],
params: &[req("codepoint")],
named_only: false,
effectful: false,
};
const TABLE_COLUMNS: Builtin = Builtin {
name: "table.columns",
aliases: &[],
params: &[req("table")],
named_only: false,
effectful: false,
};
const TABLE_ROW: Builtin = Builtin {
name: "table.row",
aliases: &[],
params: &[req("table"), req("row")],
named_only: false,
effectful: false,
};
const RESOURCE_OPEN: Builtin = Builtin {
name: "resource.open",
aliases: &[],
params: &[req("kind"), req("descriptor"), req("rights")],
named_only: false,
effectful: true,
};
const RESOURCE_READ: Builtin = Builtin {
name: "resource.read",
aliases: &[],
params: &[
req("handle"),
req("memory"),
req("offset"),
req("max_bytes"),
],
named_only: false,
effectful: true,
};
const RESOURCE_WRITE: Builtin = Builtin {
name: "resource.write",
aliases: &[],
params: &[req("handle"), req("memory"), req("offset"), req("length")],
named_only: false,
effectful: true,
};
const RESOURCE_CLOSE: Builtin = Builtin {
name: "resource.close",
aliases: &[],
params: &[req("handle")],
named_only: false,
effectful: true,
};
const TABLE_CELL: Builtin = Builtin {
name: "table.cell",
aliases: &[],
params: &[req("table"), req("row"), req("column")],
named_only: false,
effectful: false,
};
/// All builtins known to the language. Host availability is decided at call
/// time by the capability layer, not here.
@@ -188,6 +326,25 @@ pub const BUILTINS: &[Builtin] = &[
FOREIGN_CALL,
FS_READ,
FS_WRITE,
MEMORY_NEW,
MEMORY_SIZE,
MEMORY_GROW,
LIST_APPEND,
LIST_CONCAT,
MAP_KEYS,
MAP_HAS,
MAP_GET,
MAP_SET,
MAP_REMOVE,
TEXT_CODEPOINTS,
TEXT_FROM_CODEPOINT,
TABLE_COLUMNS,
TABLE_ROW,
TABLE_CELL,
RESOURCE_OPEN,
RESOURCE_READ,
RESOURCE_WRITE,
RESOURCE_CLOSE,
];
/// Find a builtin by canonical name or alias.
+44 -5
View File
@@ -224,16 +224,51 @@ pub struct Function {
pub body: Vec<Stmt>,
}
/// Writable place of an assignment (ADR-0022).
///
/// The Markdown surface is unchanged: both variants are `strong` nodes whose
/// payload contains `target = expression`. `Local` rebinds a frame slot;
/// `Index` writes one byte into an addressable runtime `Memory` region and is
/// rejected for every other base value.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(tag = "place", rename_all = "snake_case")]
pub enum PlaceTarget {
/// Local slot rebinding.
Local {
/// Target name.
name: String,
/// Span of the target name.
name_span: Span,
},
/// `base[index] = value`; only a `Memory` base is writable.
Index {
/// Base expression, evaluated first.
base: Expr,
/// Index expression, evaluated second.
index: Expr,
/// Span from the base start through the closing `]`.
span: Span,
},
}
impl PlaceTarget {
/// Best known source span of the place.
pub fn span(&self) -> Span {
match self {
PlaceTarget::Local { name_span, .. } => *name_span,
PlaceTarget::Index { span, .. } => *span,
}
}
}
/// An executable statement.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(tag = "stmt", rename_all = "snake_case")]
pub enum Stmt {
/// Local assignment (`strong` node with `name = expr`).
/// Assignment to a local slot or a `Memory` index.
Assign {
/// Target name.
name: String,
/// Span of the target name (approximated by the payload span).
name_span: Span,
/// Writable place.
target: PlaceTarget,
/// Value expression.
value: Expr,
/// Span of the whole statement.
@@ -482,6 +517,10 @@ pub struct Program {
/// metadata only: `check` never loads a library.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub external_callables: Vec<callable::CallableDescriptor>,
/// Compat-B/C `打印`/`print` destination (ADR-0030/0031). When set, the
/// runtime calls this stdlib function; there is no legacy HostOp path.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub print_override: Option<FunctionId>,
}
impl Program {
+20
View File
@@ -0,0 +1,20 @@
[package]
name = "marqdo-provider-worker"
version.workspace = true
edition.workspace = true
license.workspace = true
publish = false
[[bin]]
name = "marqdo-provider-worker"
path = "src/main.rs"
[dependencies]
libloading = { workspace = true }
serde_json = { workspace = true }
[dev-dependencies]
marqdo-host = { path = "../marqdo-host" }
marqdo-sandbox = { path = "../marqdo-sandbox" }
marqdo-value = { path = "../marqdo-value" }
tempfile = { workspace = true }
+261
View File
@@ -0,0 +1,261 @@
//! Trusted C ABI provider worker (ADR-0038).
//!
//! This helper loads one trusted `c-abi-1` shared library and serves the
//! ADR-0035 framed-JSON resource protocol over stdin/stdout. It is a transport
//! boundary, not a sandbox: the library runs with the worker's privileges.
use std::ffi::{c_char, CString};
use std::io::{Read, Write};
use std::path::PathBuf;
use libloading::Library;
use serde_json::{json, Value as Json};
const ABI_VERSION: u32 = 1;
const MAX_FRAME_BYTES: usize = 1024 * 1024;
const ERROR_MAX: usize = 1024;
type AbiVersionFn = unsafe extern "C" fn() -> u32;
type OpenFn = unsafe extern "C" fn(*const c_char, *const c_char, *const c_char, *mut u64) -> i32;
type ReadFn = unsafe extern "C" fn(u64, *mut u8, u32, *mut u32, *mut i32) -> i32;
type WriteFn = unsafe extern "C" fn(u64, *const u8, u32, *mut u32) -> i32;
type CloseFn = unsafe extern "C" fn(u64) -> i32;
type LastErrorFn = unsafe extern "C" fn(*mut c_char, u32) -> i32;
struct Provider {
library: Library,
}
impl Provider {
fn load(path: &PathBuf) -> Result<Self, String> {
let library = unsafe { Library::new(path) }
.map_err(|error| format!("cannot load `{}`: {error}", path.display()))?;
let version: libloading::Symbol<AbiVersionFn> = unsafe {
library
.get(b"marqdo_resource_provider_abi_version\0")
.map_err(|error| format!("missing abi_version: {error}"))?
};
if unsafe { version() } != ABI_VERSION {
return Err(format!("provider ABI is not {ABI_VERSION}"));
}
Ok(Self { library })
}
fn last_error(&self) -> String {
let Ok(symbol) = (unsafe {
self.library
.get::<LastErrorFn>(b"marqdo_resource_last_error\0")
}) else {
return "C provider failed".to_string();
};
let mut buffer = vec![0i8; ERROR_MAX];
if unsafe { symbol(buffer.as_mut_ptr(), ERROR_MAX as u32) } != 0 {
return "C provider failed".to_string();
}
let bytes = buffer
.iter()
.map(|value| *value as u8)
.take_while(|value| *value != 0)
.collect::<Vec<_>>();
String::from_utf8_lossy(&bytes).into_owned()
}
fn open(&self, kind: &str, descriptor: &str, rights: &[String]) -> Result<u64, String> {
let kind = CString::new(kind).map_err(|_| "kind has NUL".to_string())?;
let descriptor = CString::new(descriptor).map_err(|_| "descriptor has NUL".to_string())?;
let rights = CString::new(rights.join(",")).map_err(|_| "rights have NUL".to_string())?;
let symbol: libloading::Symbol<OpenFn> = unsafe {
self.library
.get(b"marqdo_resource_open\0")
.map_err(|error| format!("missing open: {error}"))?
};
let mut token = 0u64;
if unsafe {
symbol(
kind.as_ptr(),
descriptor.as_ptr(),
rights.as_ptr(),
&mut token,
)
} != 0
{
return Err(self.last_error());
}
Ok(token)
}
fn read(&self, token: u64, max_bytes: usize) -> Result<(Vec<u8>, bool), String> {
let capacity = u32::try_from(max_bytes).unwrap_or(u32::MAX);
let mut buffer = vec![0u8; capacity as usize];
let mut read = 0u32;
let mut eof = 0i32;
let symbol: libloading::Symbol<ReadFn> = unsafe {
self.library
.get(b"marqdo_resource_read\0")
.map_err(|error| format!("missing read: {error}"))?
};
if unsafe { symbol(token, buffer.as_mut_ptr(), capacity, &mut read, &mut eof) } != 0 {
return Err(self.last_error());
}
buffer.truncate(read as usize);
Ok((buffer, eof != 0))
}
fn write(&self, token: u64, bytes: &[u8]) -> Result<usize, String> {
let length = u32::try_from(bytes.len()).map_err(|_| "write too large".to_string())?;
let mut written = 0u32;
let symbol: libloading::Symbol<WriteFn> = unsafe {
self.library
.get(b"marqdo_resource_write\0")
.map_err(|error| format!("missing write: {error}"))?
};
if unsafe { symbol(token, bytes.as_ptr(), length, &mut written) } != 0 {
return Err(self.last_error());
}
Ok(written as usize)
}
fn close(&self, token: u64) -> Result<(), String> {
let symbol: libloading::Symbol<CloseFn> = unsafe {
self.library
.get(b"marqdo_resource_close\0")
.map_err(|error| format!("missing close: {error}"))?
};
if unsafe { symbol(token) } != 0 {
return Err(self.last_error());
}
Ok(())
}
}
fn read_frame() -> Result<Option<Json>, String> {
let mut header = [0u8; 4];
let mut read = 0usize;
while read < header.len() {
match std::io::stdin().read(&mut header[read..]) {
Ok(0) if read == 0 => return Ok(None),
Ok(0) => return Err("stdin ended inside a frame header".to_string()),
Ok(count) => read += count,
Err(error) if error.kind() == std::io::ErrorKind::Interrupted => continue,
Err(error) => return Err(format!("stdin read failed: {error}")),
}
}
let length = u32::from_be_bytes(header) as usize;
if length == 0 || length > MAX_FRAME_BYTES {
return Err(format!("invalid frame length {length}"));
}
let mut payload = vec![0u8; length];
std::io::stdin()
.read_exact(&mut payload)
.map_err(|error| format!("stdin read failed: {error}"))?;
serde_json::from_slice(&payload)
.map(Some)
.map_err(|error| format!("invalid JSON frame: {error}"))
}
fn write_frame(value: &Json) -> Result<(), String> {
let bytes = serde_json::to_vec(value).map_err(|error| format!("encode failed: {error}"))?;
if bytes.len() > MAX_FRAME_BYTES {
return Err("response frame is too large".to_string());
}
let mut stdout = std::io::stdout();
stdout
.write_all(&(bytes.len() as u32).to_be_bytes())
.and_then(|()| stdout.write_all(&bytes))
.and_then(|()| stdout.flush())
.map_err(|error| format!("stdout write failed: {error}"))
}
fn fail(message: &str) -> Json {
json!({"ok": false, "code": "host.error", "message": message})
}
fn main() {
let Some(path) = std::env::args_os().nth(1) else {
eprintln!("usage: marqdo-provider-worker <c-abi-library>");
std::process::exit(2);
};
let provider = match Provider::load(&PathBuf::from(path)) {
Ok(provider) => provider,
Err(error) => {
eprintln!("provider worker: {error}");
std::process::exit(2);
}
};
loop {
let request = match read_frame() {
Ok(Some(request)) => request,
Ok(None) => break,
Err(error) => {
eprintln!("provider worker: {error}");
std::process::exit(1);
}
};
let op = request.get("op").and_then(Json::as_str).unwrap_or("");
let response = match op {
"ping" => json!({"ok": true}),
"open" => {
let kind = request.get("kind").and_then(Json::as_str).unwrap_or("");
let descriptor = request
.get("descriptor")
.and_then(Json::as_str)
.unwrap_or("");
let rights = request
.get("rights")
.and_then(Json::as_array)
.map(|values| {
values
.iter()
.filter_map(Json::as_str)
.map(str::to_string)
.collect::<Vec<_>>()
})
.unwrap_or_default();
match provider.open(kind, descriptor, &rights) {
Ok(token) => json!({"ok": true, "token": token}),
Err(error) => fail(&error),
}
}
"read" => {
let token = request.get("token").and_then(Json::as_u64).unwrap_or(0);
let max_bytes =
request.get("max_bytes").and_then(Json::as_u64).unwrap_or(0) as usize;
match provider.read(token, max_bytes) {
Ok((bytes, eof)) => json!({"ok": true, "bytes": bytes, "eof": eof}),
Err(error) => fail(&error),
}
}
"write" => {
let token = request.get("token").and_then(Json::as_u64).unwrap_or(0);
let bytes = request
.get("bytes")
.and_then(Json::as_array)
.map(|values| {
values
.iter()
.filter_map(Json::as_u64)
.filter_map(|value| u8::try_from(value).ok())
.collect::<Vec<_>>()
})
.unwrap_or_default();
match provider.write(token, &bytes) {
Ok(count) => json!({"ok": true, "count": count}),
Err(error) => fail(&error),
}
}
"close" => {
let token = request.get("token").and_then(Json::as_u64).unwrap_or(0);
match provider.close(token) {
Ok(()) => json!({"ok": true}),
Err(error) => fail(&error),
}
}
"shutdown" => break,
_ => fail("unsupported worker operation"),
};
if let Err(error) = write_frame(&response) {
eprintln!("provider worker: {error}");
std::process::exit(1);
}
}
}
@@ -0,0 +1,136 @@
#![cfg(unix)]
use std::path::Path;
use std::process::Command;
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_sandbox::SandboxMode;
use marqdo_value::{ErrorCategory, Value};
fn context() -> marqdo_host::ExecutionContext {
marqdo_host::ExecutionContext::new("test".to_string(), None)
}
fn handle(value: Value) -> marqdo_value::ResourceHandle {
match value {
Value::Resource(handle) => handle,
other => panic!("expected resource, got {other:?}"),
}
}
fn cc_available() -> bool {
Command::new("cc")
.arg("--version")
.output()
.map(|output| output.status.success())
.unwrap_or(false)
}
fn build_provider(dir: &Path) -> std::path::PathBuf {
let source = dir.join("provider.c");
std::fs::write(
&source,
r#"
#include <stdint.h>
#include <stddef.h>
#include <string.h>
static const char DATA[] = "worker-abi";
static size_t position = 0;
uint32_t marqdo_resource_provider_abi_version(void) { return 1u; }
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
const char *rights_csv, uint64_t *out_token) {
(void)kind; (void)descriptor; (void)rights_csv;
position = 0; *out_token = 42; return 0;
}
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
uint32_t capacity, uint32_t *out_read,
int32_t *out_eof) {
(void)token;
size_t remaining = sizeof(DATA) - 1 - position;
size_t count = remaining < capacity ? remaining : capacity;
memcpy(buffer, DATA + position, count);
position += count;
*out_read = (uint32_t)count;
*out_eof = position >= sizeof(DATA) - 1 ? 1 : 0;
return 0;
}
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
uint32_t length, uint32_t *out_written) {
(void)token; (void)bytes; *out_written = length; return 0;
}
int32_t marqdo_resource_close(uint64_t token) { (void)token; return 0; }
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity) {
(void)capacity; buffer[0] = 0; return 0;
}
"#,
)
.unwrap();
let extension = if cfg!(target_os = "macos") {
"dylib"
} else {
"so"
};
let library = dir.join(format!("libworker_provider.{extension}"));
let status = Command::new("cc")
.args(["-shared", "-fPIC", "-O2", "-o"])
.arg(&library)
.arg(&source)
.status()
.expect("run cc");
assert!(status.success());
library
}
#[test]
fn c_abi_provider_runs_inside_the_worker_helper() {
if !cc_available() {
eprintln!("skipping worker binding test: cc is unavailable");
return;
}
let dir = tempfile::tempdir().unwrap();
let library = build_provider(dir.path());
let project = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("worker-cabi"),
project.path(),
);
host.register_worker_resource_provider(
"worker-cabi",
&["worker.sensor"],
&std::path::PathBuf::from(env!("CARGO_BIN_EXE_marqdo-provider-worker")),
&[library.to_string_lossy().to_string()],
)
.unwrap();
let ctx = context();
let handle = handle(
host.resource_open("worker.sensor", "s", &["read".to_string()], &ctx)
.unwrap(),
);
let first = host.resource_read(handle, 6, &ctx).unwrap();
assert_eq!(first.bytes, b"worker");
let rest = host.resource_read(handle, 64, &ctx).unwrap();
assert_eq!(rest.bytes, b"-abi");
assert!(rest.eof);
assert_eq!(host.resource_write(handle, b"x", &ctx).unwrap(), 1);
host.resource_close(handle, &ctx).unwrap();
}
#[test]
fn worker_helper_without_c_provider_fails_closed() {
let project = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("worker-cabi"),
project.path(),
);
let error = host
.register_worker_resource_provider(
"worker-cabi",
&["worker.sensor"],
&std::path::PathBuf::from(env!("CARGO_BIN_EXE_marqdo-provider-worker")),
&["/definitely/missing/provider.so".to_string()],
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::HostError);
}
+520 -7
View File
@@ -6,13 +6,13 @@
use std::collections::BTreeMap;
use marqdo_host::HostOp;
use marqdo_host::{HostOp, ResourceRead};
use marqdo_ir::builtins::Builtin;
use marqdo_ir::Resource;
use marqdo_span::Span;
use marqdo_value::{
float_to_bigint, CodeValue, ErrorCategory, FormulaValue, FunctionRef, RuntimeError, Table,
Value,
float_to_bigint, CodeValue, ErrorCategory, FormulaValue, FunctionRef, ResourceHandle,
RuntimeError, Table, Value,
};
use crate::interpreter::Interpreter;
@@ -27,10 +27,15 @@ pub(crate) fn dispatch(
let module = frame.module().clone();
match builtin.name {
"打印" => {
let value = take(&bound, "值", builtin, span)?;
interpreter
.host_invoke(HostOp::Print, vec![value], span, &module)
.map(|_| Value::Unit)
// Compat-C (ADR-0031): the legacy HostOp::Print path is gone. On
// the official CLI the loader routes 打印/print to stdlib io.写值;
// reaching this builtin means the verified bundle was unavailable.
let _ = take(&bound, "值", builtin, span)?;
Err(RuntimeError::new(
ErrorCategory::KernelUnavailable,
span,
"打印 requires the bundled stdlib print module; install the official CLI bundle",
))
}
"长度" => {
let value = take(&bound, "值", builtin, span)?;
@@ -187,6 +192,292 @@ pub(crate) fn dispatch(
&module,
)
}
"memory.new" => {
let initial_pages = page_count(take(&bound, "initial_pages", builtin, span)?, span)?;
let max_pages = page_count(take(&bound, "max_pages", builtin, span)?, span)?;
let handle = interpreter
.memory_mut()
.create(initial_pages, max_pages, span)?;
interpreter.record_memory_event(
"memory.new",
&module,
span,
&format!("initial_pages={initial_pages} max_pages={max_pages}"),
);
Ok(Value::Memory(handle))
}
"memory.size" => {
let memory = take(&bound, "memory", builtin, span)?;
let memory_type = memory.type_name();
let Value::Memory(handle) = memory else {
return Err(RuntimeError::type_error(
span,
format!("memory.size expects a Memory value, got {memory_type}"),
));
};
let pages = interpreter.memory().size(handle, span)?;
Ok(pages_value(pages, span)?)
}
"memory.grow" => {
let memory = take(&bound, "memory", builtin, span)?;
let memory_type = memory.type_name();
let Value::Memory(handle) = memory else {
return Err(RuntimeError::type_error(
span,
format!("memory.grow expects a Memory value, got {memory_type}"),
));
};
let pages = page_count(take(&bound, "pages", builtin, span)?, span)?;
let old_pages = interpreter.memory_mut().grow(handle, pages, span)?;
interpreter.record_memory_event(
"memory.grow",
&module,
span,
&format!("pages={pages} old_pages={old_pages}"),
);
Ok(pages_value(old_pages, span)?)
}
"list.append" => {
let list = take(&bound, "list", builtin, span)?;
let value = take(&bound, "value", builtin, span)?;
let list_type = list.type_name();
let Value::List(mut items) = list else {
return Err(RuntimeError::type_error(
span,
format!("list.append expects a List, got {list_type}"),
));
};
items.push(value);
Ok(Value::List(items))
}
"list.concat" => {
let left = take(&bound, "left", builtin, span)?;
let right = take(&bound, "right", builtin, span)?;
let left_type = left.type_name();
let Value::List(mut items) = left else {
return Err(RuntimeError::type_error(
span,
format!("list.concat left operand must be a List, got {left_type}"),
));
};
let right_type = right.type_name();
let Value::List(other) = right else {
return Err(RuntimeError::type_error(
span,
format!("list.concat right operand must be a List, got {right_type}"),
));
};
items.extend(other);
Ok(Value::List(items))
}
"map.keys" => {
let value = take(&bound, "map", builtin, span)?;
let value_type = value.type_name();
let Value::Map(map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.keys expects a Map, got {value_type}"),
));
};
Ok(Value::List(
map.keys().cloned().map(Value::Text).collect::<Vec<_>>(),
))
}
"map.has" => {
let value = take(&bound, "map", builtin, span)?;
let key = map_key(take(&bound, "key", builtin, span)?, "map.has", span)?;
let value_type = value.type_name();
let Value::Map(map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.has expects a Map, got {value_type}"),
));
};
Ok(Value::Bool(map.contains_key(&key)))
}
"map.get" => {
let value = take(&bound, "map", builtin, span)?;
let key = map_key(take(&bound, "key", builtin, span)?, "map.get", span)?;
let value_type = value.type_name();
let Value::Map(map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.get expects a Map, got {value_type}"),
));
};
Ok(map.get(&key).cloned().unwrap_or(Value::None))
}
"map.set" => {
let value = take(&bound, "map", builtin, span)?;
let key = map_key(take(&bound, "key", builtin, span)?, "map.set", span)?;
let new_value = take(&bound, "value", builtin, span)?;
let value_type = value.type_name();
let Value::Map(mut map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.set expects a Map, got {value_type}"),
));
};
map.insert(key, new_value);
Ok(Value::Map(map))
}
"map.remove" => {
let value = take(&bound, "map", builtin, span)?;
let key = map_key(take(&bound, "key", builtin, span)?, "map.remove", span)?;
let value_type = value.type_name();
let Value::Map(mut map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.remove expects a Map, got {value_type}"),
));
};
map.remove(&key);
Ok(Value::Map(map))
}
"text.codepoints" => {
let value = take(&bound, "text", builtin, span)?;
let value_type = value.type_name();
let Value::Text(text) = value else {
return Err(RuntimeError::type_error(
span,
format!("text.codepoints expects Text, got {value_type}"),
));
};
Ok(Value::List(
text.chars()
.map(|character| Value::int(i64::from(character as u32)))
.collect(),
))
}
"text.from_codepoint" => {
let value = take(&bound, "codepoint", builtin, span)?;
let value_type = value.type_name();
let Value::Int(codepoint) = value else {
return Err(RuntimeError::type_error(
span,
format!("text.from_codepoint expects an Int, got {value_type}"),
));
};
let Some(codepoint) = num_traits::ToPrimitive::to_u32(&codepoint) else {
return Ok(Value::None);
};
Ok(match char::from_u32(codepoint) {
Some(character) => Value::Text(character.to_string()),
None => Value::None,
})
}
"table.columns" => {
let value = take(&bound, "table", builtin, span)?;
let value_type = value.type_name();
let Value::Table(table) = value else {
return Err(RuntimeError::type_error(
span,
format!("table.columns expects a Table, got {value_type}"),
));
};
Ok(Value::List(
table.header.into_iter().map(Value::Text).collect(),
))
}
"table.row" => {
let value = take(&bound, "table", builtin, span)?;
let row = take(&bound, "row", builtin, span)?;
let value_type = value.type_name();
let Value::Table(table) = value else {
return Err(RuntimeError::type_error(
span,
format!("table.row expects a Table, got {value_type}"),
));
};
let position = table_index(row, table.rows.len(), "row", span)?;
Ok(Value::List(table.rows[position].clone()))
}
"table.cell" => {
let value = take(&bound, "table", builtin, span)?;
let row = take(&bound, "row", builtin, span)?;
let column = take(&bound, "column", builtin, span)?;
let value_type = value.type_name();
let Value::Table(table) = value else {
return Err(RuntimeError::type_error(
span,
format!("table.cell expects a Table, got {value_type}"),
));
};
let row = table_index(row, table.rows.len(), "row", span)?;
let column = table_index(column, table.header.len(), "column", span)?;
Ok(table.rows[row][column].clone())
}
"resource.open" => {
let kind = expect_text_value(take(&bound, "kind", builtin, span)?, span)?;
let descriptor = expect_text_value(take(&bound, "descriptor", builtin, span)?, span)?;
let rights = rights_list(take(&bound, "rights", builtin, span)?, span)?;
interpreter.resource_open(&kind, &descriptor, &rights, &module, span)
}
"resource.read" => {
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
let memory = memory_handle(
take(&bound, "memory", builtin, span)?,
"resource.read",
span,
)?;
let offset = take(&bound, "offset", builtin, span)?;
let offset_int = int_value(offset, "resource.read offset", span)?;
let offset_value = Value::Int(offset_int.clone());
let max_bytes = positive_count(
take(&bound, "max_bytes", builtin, span)?,
"resource.read max_bytes",
span,
)?;
let available = interpreter.memory().byte_len(memory, span)?;
let start = non_negative_usize(&offset_value, "resource.read offset", span)?;
if start >= available {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("resource.read offset {start} is at or beyond the Memory length {available}"),
));
}
let limit = max_bytes.min(available - start);
let result = interpreter.resource_read(handle, limit, &module, span);
match result {
Ok(read) => {
interpreter
.memory_mut()
.write_bytes(memory, &offset_int, &read.bytes, span)?;
Ok(resource_ok(resource_read_payload(&read)))
}
Err(error) => resource_recoverable(error),
}
}
"resource.write" => {
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
let memory = memory_handle(
take(&bound, "memory", builtin, span)?,
"resource.write",
span,
)?;
let offset = take(&bound, "offset", builtin, span)?;
let offset_int = int_value(offset, "resource.write offset", span)?;
let length = non_negative_usize(
&take(&bound, "length", builtin, span)?,
"resource.write length",
span,
)?;
let bytes = interpreter
.memory()
.read_bytes(memory, &offset_int, length, span)?;
match interpreter.resource_write(handle, &bytes, &module, span) {
Ok(count) => Ok(resource_ok(resource_write_payload(count))),
Err(error) => resource_recoverable(error),
}
}
"resource.close" => {
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
match interpreter.resource_close(handle, &module, span) {
Ok(()) => Ok(resource_ok(Value::None)),
Err(error) => resource_recoverable(error),
}
}
other => Err(RuntimeError::new(
ErrorCategory::KernelUnavailable,
span,
@@ -210,6 +501,228 @@ fn take(
})
}
/// Validate a page-count argument: an `Int` in the non-negative `u64` range.
fn page_count(value: Value, span: Option<Span>) -> Result<u64, RuntimeError> {
let value_type = value.type_name();
let Value::Int(count) = value else {
return Err(RuntimeError::type_error(
span,
format!("memory page count must be an Int, got {value_type}"),
));
};
num_traits::ToPrimitive::to_u64(&count).ok_or_else(|| {
RuntimeError::new(
ErrorCategory::MemoryFault,
span,
format!("memory page count {count} must be a non-negative integer"),
)
})
}
/// Convert a page count back to a language `Int`.
fn pages_value(pages: u64, span: Option<Span>) -> Result<Value, RuntimeError> {
i64::try_from(pages).map(Value::int).map_err(|_| {
RuntimeError::new(
ErrorCategory::MemoryFault,
span,
format!("page count {pages} does not fit an Int"),
)
})
}
/// Require a `Text` map key and return it.
fn map_key(value: Value, builtin: &str, span: Option<Span>) -> Result<String, RuntimeError> {
let value_type = value.type_name();
match value {
Value::Text(key) => Ok(key),
_ => Err(RuntimeError::type_error(
span,
format!("{builtin} key must be Text, got {value_type}"),
)),
}
}
/// Require a Text value.
fn expect_text_value(value: Value, span: Option<Span>) -> Result<String, RuntimeError> {
let value_type = value.type_name();
let Value::Text(text) = value else {
return Err(RuntimeError::type_error(
span,
format!("expected Text, got {value_type}"),
));
};
Ok(text)
}
/// Require a List of Text rights.
fn rights_list(value: Value, span: Option<Span>) -> Result<Vec<String>, RuntimeError> {
let value_type = value.type_name();
let Value::List(items) = value else {
return Err(RuntimeError::type_error(
span,
format!("resource rights must be a List, got {value_type}"),
));
};
let mut rights = Vec::with_capacity(items.len());
for item in items {
let item_type = item.type_name();
let Value::Text(right) = item else {
return Err(RuntimeError::type_error(
span,
format!("resource rights must contain Text, got {item_type}"),
));
};
rights.push(right);
}
Ok(rights)
}
/// Require an opaque resource handle.
fn resource_handle(value: Value, span: Option<Span>) -> Result<ResourceHandle, RuntimeError> {
let value_type = value.type_name();
let Value::Resource(handle) = value else {
return Err(RuntimeError::type_error(
span,
format!("expected a Resource handle, got {value_type}"),
));
};
Ok(handle)
}
/// Require an opaque Memory handle.
fn memory_handle(
value: Value,
builtin: &str,
span: Option<Span>,
) -> Result<marqdo_value::MemoryHandle, RuntimeError> {
let value_type = value.type_name();
let Value::Memory(handle) = value else {
return Err(RuntimeError::type_error(
span,
format!("{builtin} memory must be a Memory value, got {value_type}"),
));
};
Ok(handle)
}
/// Require an Int value.
fn int_value(
value: Value,
what: &str,
span: Option<Span>,
) -> Result<num_bigint::BigInt, RuntimeError> {
let value_type = value.type_name();
let Value::Int(value) = value else {
return Err(RuntimeError::type_error(
span,
format!("{what} must be an Int, got {value_type}"),
));
};
Ok(value)
}
/// Require a non-negative usize.
fn non_negative_usize(
value: &Value,
what: &str,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let value_type = value.type_name();
let Value::Int(value) = value else {
return Err(RuntimeError::type_error(
span,
format!("{what} must be an Int, got {value_type}"),
));
};
num_traits::ToPrimitive::to_usize(value).ok_or_else(|| {
RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("{what} {value} must be a non-negative integer in range"),
)
})
}
/// Require a strictly positive byte count.
fn positive_count(value: Value, what: &str, span: Option<Span>) -> Result<usize, RuntimeError> {
let count = non_negative_usize(&value, what, span)?;
if count == 0 {
return Err(RuntimeError::type_error(
span,
format!("{what} must be greater than zero"),
));
}
Ok(count)
}
fn resource_ok(value: Value) -> Value {
let mut map = BTreeMap::new();
map.insert("ok".to_string(), Value::Bool(true));
map.insert("value".to_string(), value);
Value::Map(map)
}
fn resource_err(code: &str, message: &str) -> Value {
let mut map = BTreeMap::new();
map.insert("ok".to_string(), Value::Bool(false));
map.insert("code".to_string(), Value::Text(code.to_string()));
map.insert("message".to_string(), Value::Text(message.to_string()));
Value::Map(map)
}
fn resource_read_payload(read: &ResourceRead) -> Value {
let mut map = BTreeMap::new();
map.insert("count".to_string(), Value::int(read.bytes.len() as i64));
map.insert("eof".to_string(), Value::Bool(read.eof));
Value::Map(map)
}
fn resource_write_payload(count: usize) -> Value {
let mut map = BTreeMap::new();
map.insert("count".to_string(), Value::int(count as i64));
Value::Map(map)
}
/// Convert a closed-handle failure into a recoverable Result; other errors are
/// fatal (ADR-0025).
fn resource_recoverable(error: RuntimeError) -> Result<Value, RuntimeError> {
if error.category == ErrorCategory::ResourceClosed {
return Ok(resource_err("resource.closed", &error.message));
}
Err(error)
}
/// Validate a non-negative table row/column coordinate.
fn table_index(
value: Value,
len: usize,
what: &str,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let value_type = value.type_name();
let Value::Int(index) = value else {
return Err(RuntimeError::type_error(
span,
format!("table {what} must be an Int, got {value_type}"),
));
};
let Some(position) = num_traits::ToPrimitive::to_usize(&index) else {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("table {what} {index} is out of bounds for length {len}"),
));
};
if position >= len {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("table {what} {position} is out of bounds for length {len}"),
));
}
Ok(position)
}
fn formula_text(value: Value, span: Option<Span>) -> Result<String, RuntimeError> {
match value {
Value::Formula(formula) => Ok(formula.latex),
+244 -8
View File
@@ -3,12 +3,13 @@
use std::collections::BTreeMap;
use marqdo_expression::{BinaryOp, Expr, RecordEntry};
use marqdo_host::{ExecutionContext, Host, HostOp};
use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, Program, Stmt};
use marqdo_host::{ExecutionContext, Host, HostOp, ResourceRead};
use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, PlaceTarget, Program, Stmt};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, RuntimeError, Value};
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value};
use crate::builtins_impl::FrameView;
use crate::memory::{MemoryLimits, MemoryStore};
use crate::value_ops;
@@ -68,6 +69,7 @@ pub struct Interpreter<'a> {
program: &'a Program,
host: &'a mut dyn Host,
depth: usize,
memory: MemoryStore,
}
/// Run a named function with already-evaluated arguments.
@@ -77,11 +79,31 @@ pub fn run(
module: &ModuleId,
function: &str,
args: Vec<EvaluatedArg>,
) -> Result<RunOutcome, RuntimeError> {
run_with_limits(
program,
host,
module,
function,
args,
MemoryLimits::default(),
)
}
/// Run with explicit addressable-memory limits (ADR-0022).
pub fn run_with_limits(
program: &Program,
host: &mut dyn Host,
module: &ModuleId,
function: &str,
args: Vec<EvaluatedArg>,
limits: MemoryLimits,
) -> Result<RunOutcome, RuntimeError> {
let mut interpreter = Interpreter {
program,
host,
depth: 0,
memory: MemoryStore::new(limits),
};
let value = interpreter.call_by_name(module, function, args, None)?;
Ok(RunOutcome { value })
@@ -300,10 +322,25 @@ impl<'a> Interpreter<'a> {
self.host.consume_fuel(1, Some(stmt.span()))?;
self.host.check_deadline(Some(stmt.span()))?;
match stmt {
Stmt::Assign { name, value, .. } => {
let value = self.eval_expr(value, frame)?;
frame.locals.insert(name.clone(), value);
}
Stmt::Assign { target, value, .. } => match target {
PlaceTarget::Local { name, .. } => {
let value = self.eval_expr(value, frame)?;
frame.locals.insert(name.clone(), value);
}
PlaceTarget::Index { base, index, span } => {
// Frozen order (ADR-0022): base, index, then RHS.
let base_value = self.eval_expr(base, frame)?;
let index_value = self.eval_expr(index, frame)?;
let byte_value = self.eval_expr(value, frame)?;
self.assign_memory_index(
base_value,
index_value,
byte_value,
Some(*span),
frame,
)?;
}
},
Stmt::Eval { expr, .. } => {
self.eval_expr(expr, frame)?;
}
@@ -417,6 +454,62 @@ impl<'a> Interpreter<'a> {
self.call_by_name_from(&module, &name, args, Some(span), Some(&caller))
}
/// Commit one byte into an addressable region after all operand checks.
///
/// The three operands are already evaluated, so a rejected value never
/// produces a partial write (ADR-0022 §3).
fn assign_memory_index(
&mut self,
base: Value,
index: Value,
byte: Value,
span: Option<Span>,
frame: &Frame,
) -> Result<(), RuntimeError> {
let base_type = base.type_name();
let Value::Memory(handle) = base else {
return Err(RuntimeError::new(
ErrorCategory::Type,
span,
format!(
"index assignment requires a Memory base in v1, got {base_type}; List, Map \
and Table values keep their existing value semantics"
),
));
};
let index_type = index.type_name();
let Value::Int(offset) = index else {
return Err(RuntimeError::type_error(
span,
format!("Memory index must be an Int, got {index_type}"),
));
};
let byte_type = byte.type_name();
let byte_repr = byte.display();
let Value::Int(byte) = byte else {
return Err(RuntimeError::type_error(
span,
format!("Memory byte value must be an Int in 0..255, got {byte_type}"),
));
};
let Some(byte) = num_traits::ToPrimitive::to_u8(&byte) else {
return Err(RuntimeError::type_error(
span,
format!("Memory byte value {byte_repr} is outside 0..255"),
));
};
self.memory.write(handle, &offset, byte, span)?;
let module = frame.function.module.canonical();
self.host.record_semantic(
"memory.write",
Some(&frame.function.to_string()),
Some(&module),
span,
Some("byte"),
);
Ok(())
}
fn eval_args(
&mut self,
args: &[marqdo_expression::Arg],
@@ -528,7 +621,22 @@ impl<'a> Interpreter<'a> {
Expr::Index { base, index, span } => {
let base_value = self.eval_expr(base, frame)?;
let index_value = self.eval_expr(index, frame)?;
index_value_of(base_value, index_value, Some(*span))
match base_value {
Value::Memory(handle) => {
let Value::Int(offset) = index_value else {
return Err(RuntimeError::type_error(
Some(*span),
format!(
"Memory index must be an Int, got {}",
index_value.type_name()
),
));
};
let byte = self.memory.read(handle, &offset, Some(*span))?;
Ok(Value::int(i64::from(byte)))
}
other => index_value_of(other, index_value, Some(*span)),
}
}
}
}
@@ -548,6 +656,25 @@ impl<'a> Interpreter<'a> {
let args = self.eval_args(args, frame)?;
return self.call_value(program, value, args, Some(span), Some(&caller_module));
}
if matches!(ident.name.as_str(), "打印" | "print") {
if let Some(target) = program.print_override.clone() {
let args = self.eval_args(args, frame)?;
let Some(function) = program.function(&target) else {
return Err(RuntimeError::new(
ErrorCategory::UnknownSymbol,
Some(span),
format!("print override `{target}` does not exist"),
));
};
return self.call_function(
program,
function,
args,
Some(span),
Some(&caller_module),
);
}
}
if let Some(builtin) = builtins::find(&ident.name) {
let args = self.eval_args(args, frame)?;
return self.call_builtin(builtin, args, Some(span), frame);
@@ -701,6 +828,97 @@ impl<'a> Interpreter<'a> {
self.program
}
/// Addressable-memory store for builtins and place writes.
pub(crate) fn memory(&self) -> &MemoryStore {
&self.memory
}
/// Mutable addressable-memory store for `memory.new`/`memory.grow`.
pub(crate) fn memory_mut(&mut self) -> &mut MemoryStore {
&mut self.memory
}
/// Record a semantic event for a builtin that is not a host operation.
pub(crate) fn record_memory_event(
&mut self,
kind: &str,
module: &ModuleId,
span: Option<Span>,
detail: &str,
) {
let module = module.canonical();
self.host
.record_semantic(kind, None, Some(&module), span, Some(detail));
}
/// Open a resource/stream through the host boundary.
pub(crate) fn resource_open(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
module: &ModuleId,
span: Option<Span>,
) -> Result<Value, RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_open(kind, descriptor, rights, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Read from a resource stream.
pub(crate) fn resource_read(
&mut self,
handle: ResourceHandle,
max_bytes: usize,
module: &ModuleId,
span: Option<Span>,
) -> Result<ResourceRead, RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_read(handle, max_bytes, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Write bytes to a resource stream.
pub(crate) fn resource_write(
&mut self,
handle: ResourceHandle,
bytes: &[u8],
module: &ModuleId,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_write(handle, bytes, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Close a resource stream.
pub(crate) fn resource_close(
&mut self,
handle: ResourceHandle,
module: &ModuleId,
span: Option<Span>,
) -> Result<(), RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_close(handle, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Call a builtin-read host operation on behalf of a frame's module.
pub(crate) fn host_invoke(
&mut self,
@@ -820,6 +1038,24 @@ fn member_of(value: Value, name: &str, span: Option<Span>) -> Result<Value, Runt
format!("Code has no field `{name}`"),
)),
},
Value::FunctionRef(reference) => match name {
"module" => Ok(Value::Text(reference.module)),
"name" => Ok(Value::Text(reference.name)),
_ => Err(RuntimeError::new(
ErrorCategory::UnknownField,
span,
format!("FunctionRef has no field `{name}`"),
)),
},
Value::PluginHandle(handle) => match name {
"plugin" => Ok(Value::Text(handle.plugin)),
"export" => Ok(Value::Text(handle.export)),
_ => Err(RuntimeError::new(
ErrorCategory::UnknownField,
span,
format!("PluginHandle has no field `{name}`"),
)),
},
other => Err(RuntimeError::new(
ErrorCategory::UnknownField,
span,
+5 -1
View File
@@ -8,8 +8,12 @@
mod builtins_impl;
mod interpreter;
mod memory;
mod value_ops;
pub use interpreter::{run, EvaluatedArg, Interpreter, RunOutcome};
pub use interpreter::{run, run_with_limits, EvaluatedArg, Interpreter, RunOutcome};
pub use memory::{
MemoryLimits, MemoryStore, DEFAULT_MAX_REGION_BYTES, DEFAULT_MAX_TOTAL_BYTES, PAGE_SIZE,
};
pub use marqdo_value::{RuntimeError, Value};
+600
View File
@@ -0,0 +1,600 @@
//! Runtime-owned addressable byte regions (RFC-CM-0001 / ADR-0022).
//!
//! A [`MemoryStore`] owns every region created during one run. Language values
//! carry only an opaque [`MemoryHandle`]; offsets are region-relative integers
//! and are never host pointers. Region identity is issued here and cannot be
//! forged from a document, a number or wire data.
//!
//! Semantics frozen by ADR-0022:
//!
//! * a region is a byte array with a 64 KiB logical page size; new bytes are
//! zero-filled;
//! * regions grow only, never shrink; a failed grow leaves the region and the
//! committed budget unchanged;
//! * copies of a `Memory` value share one region, while different regions
//! never alias;
//! * every access is bounds-checked against the region and the run budget, and
//! overflow in offset/page arithmetic is a structured error, never a panic;
//! * regions live for the whole run and are dropped with the store. There is
//! no user-visible `free(region)`.
use std::collections::BTreeMap;
use std::sync::atomic::{AtomicU64, Ordering};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, MemoryHandle, RuntimeError};
use num_bigint::BigInt;
use num_traits::ToPrimitive;
/// Logical page size for every region.
pub const PAGE_SIZE: usize = 64 * 1024;
/// Default maximum committed bytes across all regions in one run (64 MiB).
pub const DEFAULT_MAX_TOTAL_BYTES: usize = 64 * 1024 * 1024;
/// Default maximum committed bytes for one region (16 MiB).
pub const DEFAULT_MAX_REGION_BYTES: usize = 16 * 1024 * 1024;
static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1);
/// Bounds for addressable memory in one run.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct MemoryLimits {
/// Maximum bytes committed across all live regions.
pub max_total_bytes: usize,
/// Maximum bytes committed by a single region (`max_pages` is further
/// capped by this value).
pub max_region_bytes: usize,
}
impl Default for MemoryLimits {
fn default() -> Self {
Self {
max_total_bytes: DEFAULT_MAX_TOTAL_BYTES,
max_region_bytes: DEFAULT_MAX_REGION_BYTES,
}
}
}
impl MemoryLimits {
/// Maximum page count a single region may declare or reach.
pub fn max_region_pages(&self) -> u64 {
(self.max_region_bytes / PAGE_SIZE) as u64
}
}
struct Region {
bytes: Vec<u8>,
/// Declared maximum size for this region.
max_bytes: usize,
generation: u64,
}
/// All addressable regions created by one run.
pub struct MemoryStore {
run_id: u64,
next_region: u64,
regions: BTreeMap<u64, Region>,
committed_bytes: usize,
limits: MemoryLimits,
}
impl MemoryStore {
/// Create an empty store with a fresh, unguessable run identity.
pub fn new(limits: MemoryLimits) -> Self {
Self {
run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed),
next_region: 1,
regions: BTreeMap::new(),
committed_bytes: 0,
limits,
}
}
/// Limits in force for this store.
pub fn limits(&self) -> MemoryLimits {
self.limits
}
/// Bytes currently committed across all regions.
pub fn committed_bytes(&self) -> usize {
self.committed_bytes
}
/// Number of live regions.
pub fn region_count(&self) -> usize {
self.regions.len()
}
/// Create a zero-filled region.
///
/// `initial_pages` is committed immediately; `max_pages` is the declared
/// capacity that `grow` may not exceed. Both are checked against the
/// region and total budgets before anything is allocated, so a rejected
/// create leaves the store unchanged.
pub fn create(
&mut self,
initial_pages: u64,
max_pages: u64,
span: Option<Span>,
) -> Result<MemoryHandle, RuntimeError> {
if initial_pages > max_pages {
return Err(memory_error(
span,
format!(
"memory.new initial_pages ({initial_pages}) must not exceed max_pages \
({max_pages})"
),
));
}
if max_pages > self.limits.max_region_pages() {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory.new max_pages ({max_pages}) exceeds the per-region limit of {} \
pages ({PAGE_SIZE} bytes each)",
self.limits.max_region_pages()
),
));
}
let max_bytes = checked_bytes(max_pages, span, "region capacity")?;
if max_bytes > self.limits.max_region_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory.new max_pages ({max_pages}) exceeds the per-region byte budget of {}",
self.limits.max_region_bytes
),
));
}
let initial_bytes = checked_bytes(initial_pages, span, "initial region size")?;
let committed = self
.committed_bytes
.checked_add(initial_bytes)
.ok_or_else(|| budget_error(span, "memory budget overflow"))?;
if committed > self.limits.max_total_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory budget exceeded: committing {initial_bytes} bytes would reach \
{committed} of {}",
self.limits.max_total_bytes
),
));
}
let mut bytes = Vec::new();
bytes.try_reserve_exact(initial_bytes).map_err(|_| {
RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("cannot allocate {initial_bytes} bytes of addressable memory"),
)
})?;
bytes.resize(initial_bytes, 0);
let region_id = self.next_region;
self.next_region = self
.next_region
.checked_add(1)
.ok_or_else(|| memory_error(span, "memory region identity space exhausted"))?;
self.regions.insert(
region_id,
Region {
bytes,
max_bytes,
generation: 1,
},
);
self.committed_bytes = committed;
Ok(MemoryHandle::from_runtime(self.run_id, region_id, 1))
}
/// Current length of a region in pages.
pub fn size(&self, handle: MemoryHandle, span: Option<Span>) -> Result<u64, RuntimeError> {
let region = self.region(handle, span)?;
Ok((region.bytes.len() / PAGE_SIZE) as u64)
}
/// Grow a region by whole pages and return its previous page count.
///
/// A failed grow is atomic: the region keeps its old length and bytes and
/// the committed budget is unchanged.
pub fn grow(
&mut self,
handle: MemoryHandle,
pages: u64,
span: Option<Span>,
) -> Result<u64, RuntimeError> {
if pages == 0 {
return self.size(handle, span);
}
let region = self.region(handle, span)?;
let old_bytes = region.bytes.len();
let old_pages = (old_bytes / PAGE_SIZE) as u64;
let additional = checked_bytes(pages, span, "memory.grow pages")?;
let new_bytes = old_bytes
.checked_add(additional)
.ok_or_else(|| memory_error(span, "memory.grow size overflows the address space"))?;
if new_bytes > region.max_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory.grow would grow the region to {new_bytes} bytes, beyond its \
declared maximum of {} bytes",
region.max_bytes
),
));
}
let committed = self
.committed_bytes
.checked_add(additional)
.ok_or_else(|| budget_error(span, "memory budget overflow"))?;
if committed > self.limits.max_total_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory budget exceeded: growing by {additional} bytes would reach \
{committed} of {}",
self.limits.max_total_bytes
),
));
}
let region = self
.regions
.get_mut(&handle.region_id())
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
region.bytes.try_reserve(additional).map_err(|_| {
RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("cannot grow addressable memory by {additional} bytes"),
)
})?;
region.bytes.resize(new_bytes, 0);
self.committed_bytes = committed;
Ok(old_pages)
}
/// Total byte length of a region.
pub fn byte_len(
&self,
handle: MemoryHandle,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
Ok(self.region(handle, span)?.bytes.len())
}
/// Read a contiguous byte range. Used by resource I/O adapters.
pub fn read_bytes(
&self,
handle: MemoryHandle,
offset: &BigInt,
length: usize,
span: Option<Span>,
) -> Result<Vec<u8>, RuntimeError> {
let region = self.region(handle, span)?;
let start = range_offset(offset, region.bytes.len(), length, span)?;
Ok(region.bytes[start..start + length].to_vec())
}
/// Write a contiguous byte range atomically after a full bounds check.
pub fn write_bytes(
&mut self,
handle: MemoryHandle,
offset: &BigInt,
bytes: &[u8],
span: Option<Span>,
) -> Result<(), RuntimeError> {
let len = self.region(handle, span)?.bytes.len();
let start = range_offset(offset, len, bytes.len(), span)?;
let region = self
.regions
.get_mut(&handle.region_id())
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
region.bytes[start..start + bytes.len()].copy_from_slice(bytes);
Ok(())
}
/// Read one byte. Offsets are non-negative and region-relative.
pub fn read(
&self,
handle: MemoryHandle,
offset: &BigInt,
span: Option<Span>,
) -> Result<u8, RuntimeError> {
let region = self.region(handle, span)?;
let position = checked_offset(offset, region.bytes.len(), span)?;
Ok(region.bytes[position])
}
/// Write one byte. The caller must have validated the byte range.
pub fn write(
&mut self,
handle: MemoryHandle,
offset: &BigInt,
byte: u8,
span: Option<Span>,
) -> Result<(), RuntimeError> {
let len = self.region(handle, span)?.bytes.len();
let position = checked_offset(offset, len, span)?;
let region = self
.regions
.get_mut(&handle.region_id())
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
region.bytes[position] = byte;
Ok(())
}
fn region(&self, handle: MemoryHandle, span: Option<Span>) -> Result<&Region, RuntimeError> {
if handle.run_id() != self.run_id {
return Err(memory_error(
span,
"Memory handle belongs to a different execution run",
));
}
let Some(region) = self.regions.get(&handle.region_id()) else {
return Err(memory_error(span, "unknown Memory region handle"));
};
if region.generation != handle.generation() {
return Err(memory_error(span, "stale Memory region handle"));
}
Ok(region)
}
}
fn checked_bytes(pages: u64, span: Option<Span>, what: &str) -> Result<usize, RuntimeError> {
let bytes = pages
.checked_mul(PAGE_SIZE as u64)
.ok_or_else(|| memory_error(span, format!("{what} overflows the address space")))?;
usize::try_from(bytes).map_err(|_| {
RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("{what} ({pages} pages) does not fit this platform's address space"),
)
})
}
/// Validate an offset plus a byte count against the region length.
fn range_offset(
offset: &BigInt,
len: usize,
length: usize,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
if offset.sign() == num_bigint::Sign::Minus {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
"Memory offset must be a non-negative integer",
));
}
let Some(start) = offset.to_usize() else {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("Memory offset {offset} is out of bounds for region length {len}"),
));
};
let end = start.checked_add(length).ok_or_else(|| {
memory_error(
span,
"Memory offset plus length overflows the address space",
)
})?;
if end > len {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("Memory range {start}..{end} is out of bounds for region length {len}"),
));
}
Ok(start)
}
fn checked_offset(offset: &BigInt, len: usize, span: Option<Span>) -> Result<usize, RuntimeError> {
if offset.sign() == num_bigint::Sign::Minus {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
"Memory offset must be a non-negative integer",
));
}
let Some(position) = offset.to_usize() else {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("Memory offset {offset} is out of bounds for region length {len}"),
));
};
if position >= len {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("Memory offset {offset} is out of bounds for region length {len}"),
));
}
Ok(position)
}
fn memory_error(span: Option<Span>, message: impl Into<String>) -> RuntimeError {
RuntimeError::new(ErrorCategory::MemoryFault, span, message)
}
fn budget_error(span: Option<Span>, message: impl Into<String>) -> RuntimeError {
RuntimeError::new(ErrorCategory::ResourceExhausted, span, message)
}
#[cfg(test)]
mod tests {
use super::*;
fn int(value: impl Into<BigInt>) -> BigInt {
value.into()
}
fn store() -> MemoryStore {
MemoryStore::new(MemoryLimits::default())
}
#[test]
fn regions_are_isolated_and_handles_share_identity() {
let mut store = store();
let first = store.create(1, 2, None).unwrap();
let second = store.create(1, 2, None).unwrap();
let alias = first;
store.write(first, &int(0), 7, None).unwrap();
store.write(second, &int(0), 9, None).unwrap();
assert_eq!(store.read(first, &int(0), None).unwrap(), 7);
assert_eq!(store.read(alias, &int(0), None).unwrap(), 7);
assert_eq!(store.read(second, &int(0), None).unwrap(), 9);
assert_eq!(first, alias);
assert_ne!(first, second);
}
#[test]
fn create_is_zero_filled_and_reports_pages() {
let mut store = store();
let memory = store.create(1, 4, None).unwrap();
assert_eq!(store.size(memory, None).unwrap(), 1);
assert_eq!(PAGE_SIZE, 64 * 1024);
for offset in [0usize, 1, PAGE_SIZE - 1] {
assert_eq!(store.read(memory, &int(offset as u64), None).unwrap(), 0);
}
// Zero-page creation is legal but every access is out of bounds.
let empty = store.create(0, 1, None).unwrap();
assert_eq!(store.size(empty, None).unwrap(), 0);
assert_eq!(
store.read(empty, &int(0), None).unwrap_err().category,
ErrorCategory::IndexOutOfBounds
);
}
#[test]
fn grow_keeps_old_bytes_and_zero_fills_new_pages() {
let mut store = store();
let memory = store.create(1, 2, None).unwrap();
store.write(memory, &int(3), 200, None).unwrap();
assert_eq!(store.grow(memory, 1, None).unwrap(), 1);
assert_eq!(store.size(memory, None).unwrap(), 2);
assert_eq!(store.read(memory, &int(3), None).unwrap(), 200);
assert_eq!(store.read(memory, &int(PAGE_SIZE as u64), None).unwrap(), 0);
store
.write(memory, &int(PAGE_SIZE as u64 + 5), 42, None)
.unwrap();
assert_eq!(
store
.read(memory, &int(PAGE_SIZE as u64 + 5), None)
.unwrap(),
42
);
// Zero-page growth is a no-op returning the old count.
assert_eq!(store.grow(memory, 0, None).unwrap(), 2);
}
#[test]
fn failed_grow_does_not_change_region_or_budget() {
let mut store = MemoryStore::new(MemoryLimits {
max_total_bytes: PAGE_SIZE * 2,
max_region_bytes: PAGE_SIZE * 4,
});
let memory = store.create(1, 2, None).unwrap();
store.write(memory, &int(0), 1, None).unwrap();
let before = store.committed_bytes();
let error = store.grow(memory, 2, None).unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
assert_eq!(store.size(memory, None).unwrap(), 1);
assert_eq!(store.read(memory, &int(0), None).unwrap(), 1);
assert_eq!(store.committed_bytes(), before);
}
#[test]
fn invalid_offsets_are_structured_errors() {
let mut store = store();
let memory = store.create(1, 1, None).unwrap();
for offset in [int(-1), int(PAGE_SIZE as u64)] {
let error = store.read(memory, &offset, None).unwrap_err();
assert_eq!(error.category, ErrorCategory::IndexOutOfBounds);
}
let huge: BigInt = "999999999999999999999999999999999999999".parse().unwrap();
assert_eq!(
store.read(memory, &huge, None).unwrap_err().category,
ErrorCategory::IndexOutOfBounds
);
assert_eq!(
store.write(memory, &huge, 1, None).unwrap_err().category,
ErrorCategory::IndexOutOfBounds
);
assert_eq!(
store
.write(memory, &int(PAGE_SIZE as u64), 1, None)
.unwrap_err()
.category,
ErrorCategory::IndexOutOfBounds
);
}
#[test]
fn create_validates_page_counts_and_budget() {
let mut store = MemoryStore::new(MemoryLimits {
max_total_bytes: PAGE_SIZE * 3,
max_region_bytes: PAGE_SIZE * 2,
});
assert_eq!(
store.create(2, 1, None).unwrap_err().category,
ErrorCategory::MemoryFault
);
assert_eq!(
store.create(1, 3, None).unwrap_err().category,
ErrorCategory::ResourceExhausted
);
let _first = store.create(2, 2, None).unwrap();
let _second = store.create(1, 1, None).unwrap();
// Total budget is now exactly three pages.
assert_eq!(
store.create(1, 1, None).unwrap_err().category,
ErrorCategory::ResourceExhausted
);
assert_eq!(store.committed_bytes(), PAGE_SIZE * 3);
}
#[test]
fn forged_and_stale_handles_are_rejected() {
let mut store = store();
let memory = store.create(1, 1, None).unwrap();
let forged = MemoryHandle::from_runtime(memory.run_id() + 1, memory.region_id(), 1);
assert_eq!(
store.read(forged, &int(0), None).unwrap_err().category,
ErrorCategory::MemoryFault
);
let stale = MemoryHandle::from_runtime(memory.run_id(), memory.region_id(), 2);
assert_eq!(
store.read(stale, &int(0), None).unwrap_err().category,
ErrorCategory::MemoryFault
);
let unknown = MemoryHandle::from_runtime(memory.run_id(), 999, 1);
assert_eq!(
store.write(unknown, &int(0), 1, None).unwrap_err().category,
ErrorCategory::MemoryFault
);
}
#[test]
fn page_arithmetic_never_overflows() {
let mut store = MemoryStore::new(MemoryLimits {
max_total_bytes: usize::MAX,
max_region_bytes: usize::MAX,
});
assert_eq!(
store.create(u64::MAX, u64::MAX, None).unwrap_err().category,
ErrorCategory::ResourceExhausted
);
}
}
+26
View File
@@ -107,6 +107,32 @@ fn compare(
}
}))
}
(Value::Resource(a), Value::Resource(b)) => {
return Ok(Value::Bool(match op {
BinaryOp::Eq => a == b,
BinaryOp::Ne => a != b,
_ => {
return Err(type_error(
span,
"Resource ordering comparisons are not supported".to_string(),
))
}
}));
}
(Value::Memory(a), Value::Memory(b)) => {
// Identity, not byte content: two handles for the same region are
// equal, regions with identical bytes are not (ADR-0022 §1).
return Ok(Value::Bool(match op {
BinaryOp::Eq => a == b,
BinaryOp::Ne => a != b,
_ => {
return Err(type_error(
span,
"Memory ordering comparisons are not supported".to_string(),
))
}
}));
}
(a, b) => {
if op == BinaryOp::Eq {
return Ok(Value::Bool(false));
+76 -7
View File
@@ -2,7 +2,7 @@
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_ir::{ModuleId, Program};
use marqdo_runtime::{run, EvaluatedArg, RuntimeError, Value};
use marqdo_runtime::{run, run_with_limits, EvaluatedArg, MemoryLimits, RuntimeError, Value};
use marqdo_sandbox::SandboxMode;
use marqdo_span::Span;
@@ -70,11 +70,10 @@ fn recursion_branch_and_return() {
}
#[test]
fn loop_prints_and_returns_unit_for_fallthrough() {
let source = "## 倒计时\n\n**n = 3**\n\n- `n > 0`\n\n > 打印(n)\n\n **n = n - 1**\n";
let (result, output) = run_and_output(source, "倒计时");
fn loop_mutates_and_returns_unit_for_fallthrough() {
let source = "## 倒计时\n\n**n = 3**\n\n- `n > 0`\n\n **n = n - 1**\n";
let (result, _) = run_and_output(source, "倒计时");
assert_eq!(result.unwrap(), Value::Unit);
assert_eq!(output, "3\n2\n1\n");
}
#[test]
@@ -117,14 +116,14 @@ fn function_reference_can_be_called() {
#[test]
fn non_bool_condition_is_a_runtime_error() {
let source = "## f\n\n**x = 1**\n\n- `x`\n\n > 打印(1)\n\n*x*\n";
let source = "## f\n\n**x = 1**\n\n- `x`\n\n **x = x + 1**\n\n*x*\n";
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "runtime.type");
}
#[test]
fn fuel_budget_is_enforced() {
let program = build("## f\n\n- `true`\n\n > 打印(1)\n\n*0*\n");
let program = build("## f\n\n- `true`\n\n **x = 1**\n\n*0*\n");
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::pure(),
@@ -170,3 +169,73 @@ fn ambiguous_ref_resource_is_an_error() {
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "ref.ambiguous");
}
#[test]
fn print_without_the_bundled_stdlib_is_a_structured_error() {
let source = "## f\n\n> 打印(1)\n\n*None*\n";
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "kernel.unavailable");
assert!(
error.message.contains("bundled stdlib"),
"unexpected message: {}",
error.message
);
}
#[test]
fn function_ref_and_resource_members_are_readable() {
let source = "## 目标\n\n*1*\n\n## main\n\n### 公式\n\n$$a = 1$$\n\n### 代码\n\n```text\nx\n```\n\n**f = ref(\"#目标\")**\n\n**g = ref(\"#公式\")**\n\n**c = ref(\"#代码\")**\n\n*[f.module, f.name, g.latex, g.display, c.language, c.source]*\n";
let value = run_named(source, "main", &[]).unwrap();
let Value::List(items) = value else {
panic!("expected list");
};
assert!(matches!(&items[0], Value::Text(module) if module.ends_with("index.mq.md")));
assert_eq!(items[1], Value::text("目标"));
assert_eq!(items[2], Value::text("a = 1"));
assert_eq!(items[3], Value::Bool(true));
assert_eq!(items[4], Value::text("text"));
assert_eq!(items[5], Value::text("x\n"));
}
#[test]
fn memory_region_is_shared_by_handle_identity() {
let source = "## f\n\n**m = memory.new(1, 2)**\n\n**n = m**\n\n**`m[3] = 7`**\n\n*[n[3], m == n, memory.size(n)]*\n";
let value = run_named(source, "f", &[]).unwrap();
assert_eq!(
value,
Value::List(vec![Value::int(7), Value::Bool(true), Value::int(1)])
);
}
#[test]
fn memory_handle_cannot_reach_the_host_boundary() {
let source = "## f\n\n**m = memory.new(1, 1)**\n\n> fs.read(m)\n\n*0*\n";
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "runtime.type");
assert!(
error
.message
.contains("Memory values cannot cross the host boundary"),
"unexpected message: {}",
error.message
);
}
#[test]
fn memory_budget_is_enforced_without_changing_the_region() {
let program = build("## f\n\n**m = memory.new(1, 1)**\n\n*[m[0], memory.grow(m, 1)]*\n");
let mut host = host();
let error = run_with_limits(
&program,
&mut host,
&ModuleId::from("index.mq.md"),
"f",
Vec::new(),
MemoryLimits {
max_total_bytes: 64 * 1024,
max_region_bytes: 64 * 1024 * 2,
},
)
.unwrap_err();
assert_eq!(error.code(), "resource.exhausted");
}
+2
View File
@@ -232,6 +232,8 @@ pub struct StrictHostCommand<'a> {
pub output_limit: usize,
/// Wall-clock budget for the supervised process tree.
pub timeout: Duration,
/// Verified bundled stdlib directory to bind read-only when present.
pub stdlib_dir: Option<&'a Path>,
}
/// Launch a command inside the strict sandbox (Linux only).
+12
View File
@@ -317,6 +317,16 @@ fn run_with(
})?;
let mut args = common_args();
if let Some(stdlib) = command.stdlib_dir {
args.extend([
OsString::from("--ro-bind"),
stdlib.as_os_str().to_os_string(),
OsString::from("/run/marqdo-stdlib"),
OsString::from("--setenv"),
OsString::from("MARQDO_STDLIB_DIR"),
OsString::from("/run/marqdo-stdlib"),
]);
}
args.extend([
OsString::from("--ro-bind"),
project_root.as_os_str().to_os_string(),
@@ -548,6 +558,7 @@ mod tests {
original_document: None,
output_limit: 4096,
timeout: Duration::from_secs(1),
stdlib_dir: None,
};
let error = run_with(
&command,
@@ -585,6 +596,7 @@ mod tests {
original_document: None,
output_limit: 4096,
timeout: Duration::from_secs(1),
stdlib_dir: None,
};
let error = run_with(
&command,
@@ -111,6 +111,7 @@ fn run(helper: &Path, project: &Path, outside: &Path, mode: &str) -> StrictOutco
original_document: None,
output_limit: 64 * 1024,
timeout: Duration::from_secs(3),
stdlib_dir: None,
};
launch_strict_capture(&command).unwrap_or_else(|error| panic!("{mode}: {error}"))
}
@@ -267,6 +268,7 @@ fn strict_supervisor_enforces_memory_cpu_and_output_limits() {
original_document: None,
output_limit: 64 * 1024,
timeout: Duration::from_secs(2),
stdlib_dir: None,
};
let cpu = launch_strict_capture(&command).expect_err("CPU loop must be stopped");
let message = cpu.to_string();
@@ -283,6 +285,7 @@ fn strict_supervisor_enforces_memory_cpu_and_output_limits() {
original_document: None,
output_limit: 64 * 1024,
timeout: Duration::from_secs(3),
stdlib_dir: None,
})
.expect("output helper should be captured, not spawn-failed");
assert!(
+16 -1
View File
@@ -23,8 +23,19 @@ pub enum ErrorCategory {
NotCallable,
/// A map key or record field does not exist.
UnknownField,
/// A list/map index is out of bounds.
/// A list/map/memory index is out of bounds.
IndexOutOfBounds,
/// An addressable-memory operation failed for a reason other than a plain
/// out-of-bounds read: invalid page counts, an unknown/stale handle, or an
/// arithmetic overflow in an address.
MemoryFault,
/// A resource handle is forged, cross-run or stale.
ResourceFault,
/// A resource operation used an already-closed handle (recoverable as a
/// `Result` by the builtin).
ResourceClosed,
/// The requested resource kind/descriptor/rights are not implemented.
ResourceUnsupported,
/// A resource budget was exhausted (fuel, stack, output size).
ResourceExhausted,
/// A capability check denied the operation.
@@ -60,6 +71,10 @@ impl ErrorCategory {
ErrorCategory::NotCallable => "call.not_callable",
ErrorCategory::UnknownField => "runtime.unknown_field",
ErrorCategory::IndexOutOfBounds => codes::RUNTIME_INDEX,
ErrorCategory::MemoryFault => "runtime.memory",
ErrorCategory::ResourceFault => "runtime.resource",
ErrorCategory::ResourceClosed => "resource.closed",
ErrorCategory::ResourceUnsupported => "resource.unsupported",
ErrorCategory::ResourceExhausted => codes::RESOURCE_EXHAUSTED,
ErrorCategory::PermissionDenied => codes::PERMISSION_DENIED,
ErrorCategory::KernelUnavailable => codes::KERNEL_UNAVAILABLE,
+1 -1
View File
@@ -15,6 +15,6 @@ pub use error::{CallFrame, ErrorCategory, RuntimeError};
pub use marqdo_span::{LineCol, Span};
pub use value::{
bigint_to_exact_f64, float_to_bigint, format_float, CodeValue, FormulaValue, FunctionRef,
PluginHandle, Table, Value,
MemoryHandle, PluginHandle, ResourceHandle, Table, Value,
};
pub use wire::{from_wire, to_wire, WireError, WIRE_VERSION};
+139
View File
@@ -75,6 +75,88 @@ pub struct PluginHandle {
pub id: u64,
}
/// Opaque identity of one runtime-managed byte region (ADR-0022).
///
/// A handle is issued only by the runtime's `MemoryStore`. Copying a `Memory`
/// value or passing it as an argument preserves this identity, so two names
/// can observe each other's writes. The fields are deliberately not part of
/// any wire format: [`crate::wire`] rejects the `memory` tag on decode, and no
/// public constructor accepts untrusted data. There is no `usize` address in
/// the language value; offsets are region-relative integers.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct MemoryHandle {
run: u64,
region: u64,
generation: u64,
}
impl MemoryHandle {
/// Create a handle. Only the runtime store may call this; users cannot
/// construct one from a number, a document or wire data.
#[doc(hidden)]
pub const fn from_runtime(run: u64, region: u64, generation: u64) -> Self {
Self {
run,
region,
generation,
}
}
/// Run identity this region belongs to.
pub const fn run_id(&self) -> u64 {
self.run
}
/// Region identity within the run.
pub const fn region_id(&self) -> u64 {
self.region
}
/// Generation of the region slot (guards a future reuse policy).
pub const fn generation(&self) -> u64 {
self.generation
}
}
/// Opaque host-issued handle to an external resource/stream (ADR-0025).
///
/// Fields are private and the constructor is host-only. A handle is never
/// built from wire data or a user integer; it is a different domain from
/// `Memory` and cannot be indexed.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct ResourceHandle {
run: u64,
resource: u64,
generation: u64,
}
impl ResourceHandle {
/// Host-only constructor.
#[doc(hidden)]
pub const fn from_host(run: u64, resource: u64, generation: u64) -> Self {
Self {
run,
resource,
generation,
}
}
/// Host run identity.
pub const fn run_id(&self) -> u64 {
self.run
}
/// Resource slot identity.
pub const fn resource_id(&self) -> u64 {
self.resource
}
/// Slot generation.
pub const fn generation(&self) -> u64 {
self.generation
}
}
/// A Marqdo runtime value.
#[derive(Debug, Clone, PartialEq)]
pub enum Value {
@@ -104,6 +186,11 @@ pub enum Value {
FunctionRef(FunctionRef),
/// Host-owned plugin handle.
PluginHandle(PluginHandle),
/// Runtime-owned addressable byte region (RFC-CM-0001). Copies share the
/// same region identity; they are not byte snapshots.
Memory(MemoryHandle),
/// Host-owned external resource/stream handle (ADR-0025).
Resource(ResourceHandle),
}
impl Value {
@@ -123,6 +210,8 @@ impl Value {
Value::Code(_) => "Code",
Value::FunctionRef(_) => "FunctionRef",
Value::PluginHandle(_) => "PluginHandle",
Value::Memory(_) => "Memory",
Value::Resource(_) => "Resource",
}
}
@@ -200,6 +289,44 @@ impl Value {
Value::Code(c) => format!("Code({:?}, {} bytes)", c.language, c.source.len()),
Value::FunctionRef(r) => format!("FunctionRef({}::{})", r.module, r.name),
Value::PluginHandle(h) => format!("PluginHandle({}::{})", h.plugin, h.export),
// Never render the runtime handle: internal region identity is not
// observable program data (RFC-CM-0001 §8).
Value::Memory(_) => "Memory".to_string(),
// Resource identity is host-private and never program data.
Value::Resource(_) => "Resource".to_string(),
}
}
/// Whether this value contains a live `Memory` handle anywhere.
///
/// Used at the host/plugin/foreign boundary, which must reject handles
/// instead of serializing them.
pub fn contains_memory(&self) -> bool {
match self {
Value::Memory(_) => true,
Value::List(items) => items.iter().any(Value::contains_memory),
Value::Map(entries) => entries.values().any(Value::contains_memory),
Value::Table(table) => table
.rows
.iter()
.flat_map(|row| row.iter())
.any(Value::contains_memory),
_ => false,
}
}
/// Whether this value contains a live `Resource` handle anywhere.
pub fn contains_resource(&self) -> bool {
match self {
Value::Resource(_) => true,
Value::List(items) => items.iter().any(Value::contains_resource),
Value::Map(entries) => entries.values().any(Value::contains_resource),
Value::Table(table) => table
.rows
.iter()
.flat_map(|row| row.iter())
.any(Value::contains_resource),
_ => false,
}
}
@@ -325,6 +452,18 @@ mod tests {
);
}
#[test]
fn memory_display_never_leaks_the_runtime_handle() {
let memory = Value::Memory(MemoryHandle::from_runtime(41, 7, 1));
assert_eq!(memory.type_name(), "Memory");
assert_eq!(memory.display(), "Memory");
assert_eq!(memory.repr(), "Memory");
assert!(!memory.display().contains("41"));
assert!(!memory.repr().contains('7'));
assert!(memory.contains_memory());
assert!(Value::List(vec![memory]).contains_memory());
}
#[test]
fn big_integers_are_arbitrary_precision() {
let big: BigInt = "123456789012345678901234567890".parse().unwrap();
+40
View File
@@ -145,6 +145,12 @@ fn encode(value: &Value) -> Json {
"export": handle.export,
"id": handle.id,
}),
// Active memory handles never cross the wire (RFC-CM-0001 §8).
// The tag is emitted so an accidental transfer is visible instead of
// silently becoming a number; decoding it is a hard error.
Value::Memory(_) => json!({"type": "memory"}),
// Resource handles are host-private; same non-transferable rule.
Value::Resource(_) => json!({"type": "resource"}),
}
}
@@ -321,6 +327,12 @@ fn decode(json: &Json, depth: usize, limits: &WireLimits) -> Result<Value, WireE
.as_u64()
.ok_or_else(|| WireError::Invalid("plugin_handle.id must be an integer".into()))?,
})),
"memory" => Err(WireError::Invalid(
"Memory handles cannot cross the wire; copy bytes explicitly".into(),
)),
"resource" => Err(WireError::Invalid(
"Resource handles cannot cross the wire".into(),
)),
other => Err(WireError::UnknownType(other.to_string())),
}
}
@@ -393,6 +405,34 @@ mod tests {
assert!(matches!(decoded, Value::Float(f) if f == f64::NEG_INFINITY));
}
#[test]
fn memory_handles_are_not_transferable() {
let value = Value::Memory(crate::MemoryHandle::from_runtime(1, 2, 1));
let wire = to_wire(&value);
assert_eq!(wire, json!({"type": "memory"}));
assert_eq!(
from_wire(&wire),
Err(WireError::Invalid(
"Memory handles cannot cross the wire; copy bytes explicitly".into()
))
);
// A hand-written fake handle is rejected by the same tag rule.
assert!(from_wire(&json!({"type": "memory", "region": 2})).is_err());
}
#[test]
fn resource_handles_are_not_transferable() {
let value = Value::Resource(crate::ResourceHandle::from_host(1, 2, 1));
let wire = to_wire(&value);
assert_eq!(wire, json!({"type": "resource"}));
assert_eq!(
from_wire(&wire),
Err(WireError::Invalid(
"Resource handles cannot cross the wire".into()
))
);
}
#[test]
fn untrusted_wire_is_validated() {
assert_eq!(
File diff suppressed because it is too large. Load diff
+498
View File
@@ -0,0 +1,498 @@
# RFC-CM-0001:Marqdo 自主计算机器、可寻址内存与系统交互边界
> **状态**:设计提案(Draft,未实施,未批准)
> **日期**:2026-11-20
> **建议归档路径**:`docs/architecture/RFC-CM-0001-compute-machine.md`
> **验证基线**:`cflmy/marqdo-cli`,Git 提交 `a113d894962d8c00eb02efb42e0e73f8b084a1fa`(检索于 2026-11-20)
> **兼容性**:涉及 `Value`、表达式赋值、运行时、主机能力和线协议的变更,必须通过 ADR、规范版本与相应测试;本 RFC 不声明任何新接口目前可用。
---
## 摘要
Marqdo 的优势不应只是用 Markdown 排版代码,而应是:在不更换 Markdown 语法角色的前提下,以**自身定义的计算语义**构造算法、数据结构、运行时工具和知识制品。现有解释器已提供整数、浮点、文本、列表、映射、控制流和函数调用,但尚未提供**由 Marqdo 程序控制的可寻址字节存储**;其赋值左侧仅支持标识符,系统交互集中在一组固定 `HostOp` 与外部插件能力中。这妨碍了用 `.mq.md` 自己编写内存分配器、二进制协议、共享节点结构、虚拟机器等低层算法。
**提议:以“托管值 + 多个可增长的线性内存区域 + 不透明系统资源”为计算机器的三个基础领域。**
1. 保留现有 `Value` 计算体系,不强迫一切数据都变成字节地址。
2. 引入一个拥有独立身份、按字节索引、可增长、具有边界的 `Memory` 值;每个区域内部线性寻址,物理实现可以是 Vec、分页或其他表示。
3. 扩展现有“位置赋值”:允许对适当的索引目标写入,如 `m[8] = 42`,而不是另创 Markdown 语法或几十个关键字。
4. 为创建、测量和增长内存定义**极少量计算原语**;让 `malloc/free`、内存池、链表、编码、缓冲区等**首先在 `.mq.md` 标准库**中实现。
5. 将终端、文件、网络、进程、设备定义为**外部资源**,经稳定、可授权、可替换的底层适配接口交互;Marqdo 自身可以实现高级系统库与协议,但系统调用或硬件访问的**最后一步不能凭空消失**。
6. CPU 寄存器由解释器/编译后端管理;裸机器码、GPU/QPU、共享内存原子操作暂不进入本 RFC 的基础语义。
这是一种**可编程的抽象机器**,而不是“在每种新需求前追加 Rust Host 函数”的框架,也不是把 x86/RISC-V 指令直接变成 Markdown 语法。
## 1. 设计动机:究竟希望解决什么
### 1.1 真实诉求
Marqdo 必须能在不借助 Python、JavaScript 或宿主语言提供专门算法函数的前提下,**通过自己的控制流和基础存储机制**构造原先不存在的高级能力。例如:实现内存分配器、二叉树、哈希表、字节码解释器、文件格式解析器、数值算法。对于 OS 和硬件,要求的是**高级功能主要由 Marqdo 实现,底层权限与系统调用接口保持稳定**,而不是“所有跨越内核边界的指令都神奇地由 Markdown 自己完成”。
### 1.2 非目标
- 不追求复刻 CPU 的真实寄存器、指令流水线、页表与异常处理体系。
- 不把所有 Markdown `[]()` 链接重新定义为动态指针。链接继续表示文档/符号寻址,不伪装运行时对象地址。
- 不把 GFM 表格当成实际运行内存。表格可以用于数据/状态的可读表达与快照,但不是存储管理器。
- 不要求标准库全部使用 `.mq.md`;原生加速可以有,但**不是实现一般算法的必要条件**。
- 不在第一期承诺用户代码内存安全、数据竞争自由、GPU 加速或跨平台无差异系统调用。
- 不支持在无权限环境中直接执行任意机器码或任意 syscall。
### 1.3 设计原则
**P1 语义最小而充分**:首先增加无法由现有表达式自然实现的基础机制,不先增加高级 API 或新关键词。
**P2 高级能力在语言中生长**:新增算法/数据结构不修改解释器、不修改 `HostOp`、不需要重新编译 CLI。
**P3 读写位置统一**:索引读取和索引赋值遵循同一类“位置”语义;`Memory` 的读取/写入是基础例子,普通列表的值语义不得无声改变。
**P4 内外边界诚实**:内存操作是语言计算;操作系统 I/O 依赖权限、平台接口和底层实现。
**P5 可读性**:普通 Markdown 文档仍然可读;`##`/`**`/`*`/列表/blockquote 的现有角色不因本 RFC 改变。
**P6 安全与可验证性**:将可验证的保证限定在区域边界和权限边界;不对自定义分配器的块级错误作虚假保证。
**P7 零隐式权限**:能够导入模块、持有字符串或解析文档,不代表拥有文件、网络或设备访问权。
## 2. 现有仓库事实及需要保留的设计
依据提交 `a113d894...` 的源代码和规范:
| 项目 | 现状 | 对本 RFC 的约束 |
|---|---|---|
| `marqdo-value` | `Unit/None/Bool/Int/Float/Text/List/Map/Table/Formula/Code/FunctionRef/PluginHandle`,没有普通可寻址 `Memory` 类型 | 应新增专门的存储引用值,而非把现有 List 改成默认共享对象 |
| `marqdo-expression` | 索引 `a[i]`/成员 `a.x` 可读取;`assignment = identifier = expression` | 最小语法修改是赋值目标扩展,不是新 Markdown 节点 |
| `marqdo-runtime` | `Frame.locals: BTreeMap<String, Value>`;`Assign` 以 `locals.insert` 重绑定 | 局部变量重新赋值应保持既有含义;共享可变性仅属于显式 `Memory` |
| `marqdo-frontend` | Markdown AST 是语法角色唯一权威;GFM Table 惰性 | 不更改表格、链接的执行语义;表达式子语言可增补语义 |
| `marqdo-ir` | 内置函数元数据固定且需更新规范 | 新增基础内存原语需要小型且审慎的语言变更;高层库不应无限扩展 Builtin 表 |
| `marqdo-host` | 固定 `HostOp`,包括 Print、Math、Foreign、FsRead/FsWrite、PluginInvoke | 不应将每一个新的标准库函数继续塞入固定 HostOp |
| 插件 ABI v1 | C ABI、worker、能力请求已有雏形 | 可复用函数注册与 Worker,不能错误地把 C ABI 当沙箱 |
| `pure/project/strict` | `pure` 禁止外部 I/O grant;`strict` Linux 探测成功后只支持 L1 范围,其他平台 fail closed | 新系统资源功能首先在项目模式开发;strict 须经新的隔离验证,不得默许开启 |
| 包管理 | path/Git 依赖与 lock v3 已实现;版本范围部分尚未强制 | 独立标准库可利用现有依赖体系,但兼容版本需要明确验证 |
**保留的价值**:现有整数运算、函数调用、条件和循环不必推翻;现有 AST 映射、资源边界和可追踪执行事件是 Marqdo 与脚本转译器的根本区别之一。
源码(固定版本):
- [`spec/expression.ebnf`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/expression.ebnf)
- [`spec/mapping.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/mapping.md)
- [`spec/evaluation.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/evaluation.md)
- [`crates/marqdo-value/src/value.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-value/src/value.rs)
- [`crates/marqdo-runtime/src/interpreter.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-runtime/src/interpreter.rs)
- [`spec/security.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)
## 3. 总体架构与唯一的语义边界
```text
.mq.md 文档 GFM Table/Link/标题/正文仍按 Markdown 正常渲染
│
▼
Canonical Markdown AST ──▶ 已有 Frontend ──▶ Marqdo IR
│
┌───────────────────┤
▼ ▼
值与控制 存储与位置
Int/List/Map MemoryRegion
函数/赋值/循环 字节读写/增长
└─────────┬─────────┘
│
Marqdo 计算核心
│
需要访问外部世界时才穿越边界
▼
Resource capability gate
│ │ │
console file network/device
│ │ │
Platform Adapter / OS ABI
```
逻辑上的执行状态可以表述为:
`State = (Control, Frames, ManagedValues, RegionStore, ResourceTable, Budget)`。
- `Control`:当前 IR 节点和执行顺序;保留已有控制流。
- `Frames/ManagedValues`:现有 `Value`、局部变量和调用语义。
- `RegionStore`:一组可增长字节区域,每个区域按 `(region identity, offset)` 寻址。
- `ResourceTable`:操作系统资源的不透明句柄与授权信息,**不得与内存地址混用**。
- `Budget`:fuel、调用深度、内存限额、I/O 限额、截止时间等。
此设计**不规定**必须先创建自定义字节码 VM。现有树遍历解释器可直接执行新内存语义;将来引入字节码、JIT、LLVM/Cranelift 等属于可选执行方式,必须保持可观察语义。
## 4. 三种基础领域:值、内存、资源
### 4.1 托管值:维持已有语义
`Int/Text/List/Map/Table/Formula/Code` 等继续是运行时管理的值。普通 `b = a` 不得偷偷将普通列表变成共享可变指针。现有对象可以由 Rust 分配内存——**实现语言不等于语言语义**。这个领域负责易用性、文档处理、不可变数据等。
### 4.2 Memory:拥有身份的字节区域
新增 `Value::Memory`(名称暂定),含不可伪造的运行时区域引用。逻辑区域内部是一串字节,下标为非负整数;不同区域即使下标相同也不是同一个位置。
- **区域身份**:由本次运行中的内存实例签发;`Memory` 复制/传参时保留指向相同区域的身份(共享,不复制所有字节)。
- **地址**:`(MemoryHandle, offset)`;`offset` 只在所属区域内有意义;整数本身不是获得其他区域权限的凭据。
- **内容**:每个元素是 `0..255` 的一个字节;初始字节为 0。
- **大小**:区域长度是页大小的整数倍;建议初期采用 `64 KiB` 页,以简化语义与工具链对照,不要求底层真的按页分配。
- **增长**:只增长,不缩小;旧地址偏移不变;新加入的字节全部为零;不得突破区域/运行全局限额。
- **读取**:`m[i] -> Int(0..255)`;越界、非法整数或已失效区域均是类型/存储错误。
- **写入**:`m[i] = b`,`b` 必须是 0..255 的整数,不隐式截断;越界拒绝。
- **相等性**:两个 Memory 引用的相等比较区域身份;不比较全部字节;Memory 不定义大小排序。
- **序列化**:不允许将活动 `Memory` 句柄伪装成数值或跨进程永久有效地址。允许显式**复制字节内容**为快照,需限额;跨进程传输需要显式复制或另立经过授权的共享内存协议。
- **生命周期**:第一版只在单次执行内存在;由 Runtime 保持活动句柄可达性,引用消失后可回收,至少在运行结束时回收全部区域。**不增加用户可见的 `free(region)`**,以免引入悬空区域句柄。用户的 `malloc/free` 管理的是**区域内部的块**。
- **内存安全范围**:Runtime 保证单次操作不访问该 Memory 区域之外;不保证用户自编分配器不会越块、重复 free、释放后复用错误。
此设计不要求新增独立的 `Pointer` 原始类型。**首版指针可由 `Memory` 引用和 `offset` 组合表达**;是否将这一对优化为一等 `Pointer`,由实际性能与调用需求决定,而非现在先引入语法。
### 4.3 Resource:与外部世界交互的独立对象
`ResourceHandle` 是由可信平台/Provider 签发的不透明句柄,代表文件、控制台流、网络连接或专门设备;含授权访问模式与生命周期。**它不是 `Memory`,不能索引成任意 RAM,也不能通过伪造整数创建。**
其常见操作是 `open/read/write/close`,但具体合法操作取决于资源接口。操作系统错误、EOF、部分读写、超时、取消都需要保留独立结果和错误语义。不得假定一次 `read` 总能填满缓冲区、一次 `write` 总能输出所有字节。
### 4.4 为什么要划分,而不是单一“万能地址空间”
`Memory` 代表运行时可管理的存储;`Resource` 代表外部效果。普通内存读写是局部状态改变,终端输入/网络读取可能随外部世界变化。把二者统一为可以随意解引用的指针,会隐藏权限和不可重复性,也会使知识证明与回放语义失真。
## 5. 可编程动态内存:到底哪些必须内置
### 5.1 最小计算内核操作
这里定义**能力**,不冻结最终函数名:
| 基本能力 | 候选用户接口 | 运行时职责 |
|---|---|---|
| 创建区域 | `memory.new(initial_pages, max_pages)` | 签发不可伪造引用,检查预算并分配零初始化区域 |
| 查询大小 | `memory.size(m)` | 返回页数,不泄露真实宿主指针 |
| 增长区域 | `memory.grow(m, pages)` | 原有字节与偏移不变,新字节为零;成功返回原页数,失败返回可区分错误 |
| 读取一字节 | `m[i]` | 边界检查,读取 `0..255` |
| 写入一字节 | `m[i] = b` | 边界和值域检查,更新同一 Memory 区域 |
`memory.new/size/grow` 是操作而非关键词;`m[i]` 利用当前已有的读取表达式,`m[i] = b` 只扩展现有赋值目标语义。这是此 RFC 要求的**唯一用户可见表达式语法增量**。
`len(m)` 可在另一个 ADR 中统一成字节长度而非另设 `memory.size`;这里暂不混淆“页数”和“字节数”。是否以页或字节为 `grow` 单位也应通过 ADR 固定,本文的原型推荐页。
### 5.2 位置赋值(Place)如何实现
对左值 `x` 与 `m[i]`,**赋值含义不同但遵循一个语法规则**:
- `x = value`:仍重新绑定当前函数帧局部变量,现有语义不变。
- `m[i] = byte`:先按既有从左到右顺序求值 `m` 与 `i`,检查目标为 `Memory` 并检查偏移和值,然后更新同一内存实例。
- `list[i] = value`:**不在第一个阶段承诺**。是否将列表修改为值更新/可变对象更新,要另行定义,不能顺便改变现有列表复制与索引语义。
- `obj.field = value`:首版也不必实现。只有在明确记录值/对象语义后再拓展。
表达式编译结构的最小增量是 `Assign { target: PlaceExpr, value: Expr }`,其中 `PlaceExpr` 初期只有 `Local(name)`、`Index(base,index)`;索引目标仅接受 Memory,其他索引仍只读。无需新 Markdown 节点。
### 5.3 自定义指针与分配器
`malloc(n)` 在 `.mq.md` 标准库里持有一个区域 `m` 和元数据指针 `next`,向调用方返回区域内偏移(或 `{region:m, offset:p}` 组合值)。`free` 仅将已分配块放回空闲链表;若空间不足,调用 `memory.grow`。对齐、大小、空闲块链表、合并策略都由 `.mq.md` 编写,**不要求 Rust 为每一种分配算法做特殊 HostOp**。
例:块格式 `[size:u32][next:u32][flags:u8][payload...]`。首版只支持按字节读取,`u32` 的编码/解码可以使用四次索引、`//` 和 `%` 在 `.mq.md` 中实现。若该路径确实过慢,后来可增加 `load_u32` 等**语义等价、可验证的加速实现**,不能因此把完整分配器写到 Host。
分配器可建立:Arena、first-fit、buddy、slab、链表/图、字符串字节编码、编译器符号表、虚拟 CPU 的寄存器文件等。**内存块的逻辑安全由标准库分配器定义**;调用者拿到任意偏移后仍有越块写的能力,这属于明确的低层风险。
### 5.4 内存模型形式定义(草案)
令 `R` 是区域身份集合,`M[r]` 是 r 区域的字节序列,`len(M[r])` 是其有效长度。
- `read(r,i) = M[r][i]`,前提 `0 ≤ i < len(M[r])`。
- `write(r,i,b)` 将 `M[r][i]` 更新为 `b`,前提 `0 ≤ b < 256` 且 `i` 合法;其余字节不变。
- `grow(r,k)` 要求 `k ≥ 0`,若预算允许,尾部追加 `k × PAGE` 个零字节,旧位置的字节值不变;否则返回显式失败且区域不变。
- 不同区域互不别名:`r1 != r2` 意味着修改 `M[r1]` 不影响 `M[r2]`。
- `m2 = m1` 若 m1 为 Memory,则二者指向同一 `r`;若为普通 `List`,原有值语义不变。
实现必须防止巨大 BigInt 偏移到 `usize` 转换时溢出、`offset + length` 整数溢出、预算计算溢出。任何错误不得 panic、越界读写或产生非确定性内存破坏。
## 6. 非线性存储、寄存器与加速的恰当位置
### 6.1 非线性内存不需要独立关键字
- **逻辑非线性结构**(链表、树、图):在一个或多个字节区域中存储偏移/引用即可;不要求新类型。
- **物理非连续内存**:运行时可以用分页、稀疏映射或 Vec 实现同一线性语义;用户无需知道物理页布局。
- **多设备内存**:未来可能带有设备特定权限与同步约束;不要将 CPU 内存随意解释成 GPU/QPU 地址。优先经专用受控资源/后端接入。
### 6.2 不加入真实 CPU 寄存器接口
当前局部变量与表达式临时值已经构成“虚拟寄存器”意义上的计算槽。编译到机器码时,可以由后端分配硬件寄存器。用户要模拟寄存器堆,可以用 Memory 自己实现;特殊硬件寄存器则是设备 I/O,而不是普通变量。
### 6.3 并行和原子操作
本 RFC 的内存规则初始定义为**单线程或按明确顺序执行**。绝不能声称 `m[i] = m[i]+1` 在多线程之间自动原子。未来若允许多个执行任务共享 Memory,必须另行设计同步/原子操作、数据竞争行为和访问序;也可以先采用无共享可变内存的任务隔离。新增原语应依据不可由当前操作正确组合而来的能力,而非先堆锁/屏障关键字。
### 6.4 GPU/QPU 不由普通 Memory 自动获得
Memory 支持字节缓冲区与数据布局,但 GPU 计算仍需要执行后端与数据传输,真实量子资源则不是可随意读取的字节地址。Marqdo 可用独立后端/服务处理硬件特有语义,内核应保持硬件无关。
## 7. 系统交互:能否不依赖 Rust/Python 实现高级功能?
### 7.1 必须给出的诚实答案
**可以**:大量系统功能由 Marqdo 标准库直接编写,包括文本与字节编码、目录遍历算法、协议解析、HTTP 报文、部分网络协议逻辑、缓存、日志格式、CLI 交互逻辑、文件格式读写、数据库协议客户端中的高层逻辑。程序运行时无需执行 Python 或 JavaScript,也不需要为每个高级库函数增加 Rust 分支。
**不能无条件做到**:如果当前运行方式是一个 Rust 编写的解释器,那么跨越用户态与操作系统的**最后一步**必须由解释器内建的系统适配器、已授权插件/worker、或其他可信底层机器代码执行。Markdown 源码不能在无相应运行环境、系统调用 ABI 和权限的情况下直接控制硬件。即使将来 Marqdo 编译成独立原生二进制,它也仍通过操作系统 ABI 或设备驱动与外界交互,只是**不再需要 Rust 解释器常驻运行**。
**不可混淆的四种独立性:**
| 层面 | 能否达到 | 含义 |
|---|---|---|
| A. 算法独立 | **可以,近期目标** | 排序、内存分配、解析器等算法均在 `.mq.md` 中 |
| B. 标准库独立 | **可以,近期目标** | 新标准库函数不修改 Rust Runtime/HostOp,单独版本发布 |
| C. 系统接口独立 | **有条件可以** | 已有的 OS 原语/权限可组合构造新功能;新增设备/OS 原语需相应适配器,但不改语言核心 |
| D. 运行时/宿主实现独立 | **中长期目标** | 独立编译成机器码/Wasm、或自举解释器,不依赖 Rust 解释器运行;仍依赖 OS ABI/硬件 |
**这里的“无需修改宿主”不是“无底层实现”。** 应将“高级逻辑用 Marqdo 实现”与“最后一跳由可信 OS adapter 执行”区分清楚。
### 7.2 最小系统资源操作模型
建议**不增加任何 I/O 关键字**,而以可注册、版本化的 `Resource` 接口提供 OS 能力。候选调用形式(未实施):
- `resource.open(kind, descriptor, rights) -> ResourceHandle`
- `resource.read(handle, memory, offset, max_bytes) -> count | EOF/error`
- `resource.write(handle, memory, offset, bytes) -> count | error`
- `resource.close(handle) -> Result`
其中:
1. `kind` 例如 `stdio`、`file`、`socket`,必须由可信平台适配器或已安装的 Provider 明确注册,**不能因为传入任意字符串就自动获得能力**。
2. `descriptor` 是声明请求的结构化参数,例如项目内文件名、网络目标,必须验证并授权;绝非任意可解引用真实指针。
3. `rights` 是请求的权能;实际授权为“请求 ∩ 操作者 grant ∩ Provider 支持”,拒绝必须显式报错。
4. `read/write` 通过 Memory 区域交换原始字节,边界检查和缓冲区借用由 Runtime/Adapter 保证;不得允许 I/O Adapter 长期保存未固定的宿主 Rust 指针。
5. 区分 EOF/0 字节暂时无进展/取消/系统错误;支持部分读写,标准库负责重试与缓冲。
6. `close` 撤销对应资源句柄;已关闭句柄、重用 ID 必须借助 generation/注册表检查,不能错误访问新资源。
7. 对设备特殊操作,不强行滥用 `read/write`:必要时可由具体 Provider 暴露专门的**版本化、授权操作**;不在通用内核中新增一个无条件允许任意 syscall 的 `control` 后门。
`resource.*` 属于**平台接口层**而不是纯计算内核;即使调用看起来是普通函数,执行前必须检查权限。现有 Host 能力检查可成为迁移入口,但不应为每个新高层标准库函数创建新的 `HostOp` 变体。
### 7.3 应该怎样真正避免“每扩展一次就修改 Host”?
将目前的固定分派进一步分成:
```text
普通 Marqdo 标准库函数
├── 调用 Marqdo 语言计算原语(无需 Host)
└── 仅在外部交互时使用稳定的 Resource ABI
│
Capability Gate
│
Provider/Platform Registry
│
┌────────────────┴───────────────┐
▼ ▼
标准平台适配器 独立第三方 Provider
文件/终端/网络 新设备/专用系统能力
```
关键是**一次性稳定可扩展的外部操作契约**,而不是一张不断增长的 Host 功能目录。
现有 C ABI v1、worker 与 `CallableRegistry` 应先尽量复用;新接口可在不破坏 v1 的条件下增补版本化注册信息。如需升级,必须有兼容矩阵与迁移机制。**Provider 负责它新增的原始能力;`.mq.md` 标准库负责高级功能。**
### 7.4 两种系统访问路线及取舍
| 路线 | 优点 | 风险/限制 | 建议 |
|---|---|---|---|
| 能力化 `Resource` ABI | 统一权限与跨平台行为;适合安全分层 | 新的真正 OS 原语仍需要 Adapter/Provider | **默认且必须支持** |
| 直接的原生 syscall/FFI 边界 | 能较快覆盖大量平台特性 | 平台/ABI 依赖、地址搬运、权限绕过、维护和可验证性问题 | 仅可信、本机、显式授权的可选高级接口;默认 pure/strict 禁止 |
在 Linux 等平台,即使“直接 syscall”,也要处理目标架构调用约定、数字差异、结构体布局、指针与长度、errno 以及 OS 策略;Windows/macOS 并不提供与 Linux 一模一样的 syscall ABI。不得将其伪装成普适的语言核心。
**无悖论地满足用户目标的办法**:向 Marqdo 暴露一个小而稳定的**操作系统能力边界**,然后让标准库以 Marqdo 自己的程序逻辑组合出系统高级功能。这是可实现的强独立性;完全消灭系统边界不是现实目标。
### 7.5 端到端例子 A:完全用 Marqdo 构建交互式 CLI
**先决条件**:可信平台适配器提供 stdin/stdout 的已授权资源,Marqdo 运行时支持 Memory 操作。
1. 标准库 `io.read_line` 在 `.mq.md` 中申请/增长缓冲区,循环调用资源 `read`,查找换行字节。
2. 标准库 `text.utf8_decode` 用 `.mq.md` 处理 UTF-8(正确处理非法序列;编码策略可配置)。
3. 用户程序将文本解析为数字,用现有 `Int` 算术和条件循环处理。
4. 标准库 `io.write_text` 使用 `.mq.md` 的 UTF-8 编码,再经 `write` 输出字节。
**全流程无需 Python/Node,也不需要 Host 特地实现 `input()`。** 然而最底层 stdin/stdout 仍由 OS 适配器调用操作系统接口。
### 7.6 端到端例子 B:新增 HTTP 客户端能力
**先决条件**:系统适配器已经提供经授权的网络连接资源、DNS/地址解析的可用机制,以及必要的加密能力或可插拔 TLS Provider。
1. 标准库构造 HTTP 请求行、首部和消息体的字节序列。
2. 使用 `resource.write` 处理部分发送;使用 `resource.read` 实现缓冲接收。
3. `.mq.md` 解析状态行、首部、chunked 编码等;实现重试、缓存、超时逻辑。
4. 测试 HTTPS 时,必须使用可靠 TLS 实现(可由已审计 Provider 提供;“不依赖外部语言写协议逻辑”不意味着应该自己手写未经验证的 TLS)。
网络的“最后一跳”必须存在,且不同平台的 TCP/TLS 能力可能需要不同 Provider;不能只靠 Memory 推导出 socket。
### 7.7 端到端例子 C:科研加速与 QPU
- 经典数值模拟:`.mq.md` 可以在 Memory 中构造矩阵、压缩存储、数据布局算法,后续由独立计算后端加速。
- QPU:量子设备是具有权限、调度与门集约束的资源,不是普通 Memory;通过专用 Provider/Backend 提交计算、接收结果。
- 这种接口可保持标准库和上层代理/知识文档不绑定 Rust 宿主,但并不能保证算法自动获得 GPU/QPU 性能或精确的量子物理语义。
## 8. 安全、权限与错误模型
| 保护目标 | 首版保证 | 明确不能保证 |
|---|---|---|
| 虚拟 Memory | 区域越界、非法索引、增长限额、无效句柄失败 | 区域内 malloc/free 的块级内存安全 |
| 不透明 Resource | 授权身份、访问模式、资源关闭和 generation 检查 | 平台适配器/不可信原生插件本身绝对安全 |
| pure | 内部算术、内存与受预算控制的逻辑计算;不获得外部 I/O | 允许直接 syscall 或任意外部访问 |
| project | 对批准的文件/终端/网络资源进行操作,使用能力化 API | 等价于 OS 级进程沙箱 |
| strict | 维持现有 fail-closed 策略;新 I/O Provider 必须另行经过 OS 级安全验收 | 当前版本自动支持所有外部能力 |
进一步要求:
- 全局/单区域内存预算、增长次数与 fuel 限额、操作可取消性;内存失败不得半成功。
- 路径资源应使用受控根路径和可验证的 OS 打开策略,避免字符串检查后再打开造成 TOCTOU;网络权限需要处理名称解析/重绑定等问题。
- 插件/worker 不因拥有 API 名字自动获得权限;原生进程内插件不能被当作不可信沙盒。
- Memory/Resource 活动句柄不应直接跨线协议序列化;跨边界传输使用字节复制、快照或显式授权的资源传递协议。
- 诊断保持来源 Markdown 节点位置;可选追踪读取/写入的逻辑区域、偏移和长度,不能将外部文件秘密或原始敏感字节无条件写日志。
## 9. 与当前 Marqdo 语法和知识编译的结合
这一方案故意**只修改最少的表达式语义**。
- `##` 函数、`**` 执行、`*` 返回、Markdown 列表与引用规则继续维持规范。
- `[]()` 是**文档/知识的静态寻址**;动态 Memory 的字节偏移属于执行期位置,两者不强行共享 URI 语法。
- 普通 GFM 表格是文档值;可用于初始化/展示内存快照,但不会自动变成内存分配器。
- `memory.new(...)` 与 `resource.open(...)` 都可按普通函数调用表达;不新增 `memory`、`pointer`、`asm`、`register` 这样的语法关键字。
- 内存状态变化可选输出可读 GFM Table 供 View/Proof 使用;证明必须以抽象执行语义为依据,而不是假设表格文本本身就是执行存储。
示例(**候选语法,当前尚不可运行**):
````markdown
## 研究一个字节缓冲区
此函数演示“可引用的存储区域”与普通 Markdown 叙述并存。
**区域 = memory.new(1, 4)**
**`区域[8] = 42`**
**另一个名字 = 区域**
**`另一个名字[9] = 99`**
*区域[8] + 区域[9]*
````
预期结果 141。注意这里 `另一个名字 = 区域` 分享 Memory 身份;一般列表/表格并不因此被改造成共享字节缓冲区。
## 10. 可行性和备选方案评定
| 设计 | 语言自主性 | 高层开发便利性 | 性能潜力 | 安全/复杂度 | 结论 |
|---|---|---|---|---|---|
| 继续只扩充 HostOp | 低 | 表面高 | 取决于 Rust 实现 | Host 随功能膨胀 | 否决为主要路线 |
| 只有托管对象堆/GC | 中 | 高 | 对一般数据不错 | 需要身份/生命周期语义 | 保留现有托管值,但不足以满足低层目标 |
| **多区域字节内存 + 托管值 + Resource** | **高** | **高** | 后端可优化,字节操作起初较慢 | 明确的区域边界;块级风险可见 | **推荐** |
| 任意裸指针 + 直接机器码 | 极低平台独立性 | 低 | 最高上限 | 高风险、难验证 | 只可选的可信原生扩展 |
| 纯 GFM 表格作堆内存 | 低至中 | 对展示很高 | 非常低 | 动态状态/身份混乱 | 否决 |
该推荐不是宣称“最优”已经被数学证明,而是在现有语法、实现成本、扩展性与安全边界下的**工程选择**。
## 11. CLI/Runtime/标准库仓库的具体职责
### 11.1 `marqdo-cli`:保留小而稳定的内核
- `marqdo-value`:新增不可伪造的 Memory 值表示;必要时添加 ResourceHandle,注意现有 PluginHandle 不能随意直接复用为 OS 资源句柄。
- `marqdo-expression`:`AssignTarget` 的 Local / Index 变体;保留原有表达式读取。
- `marqdo-frontend`:更新表达式 AST 对应 IR 和定位信息;不改变 Markdown 语义映射。
- `marqdo-runtime`:Memory 实例生命周期/增长/边界检查/预算;执行位置写入;参考解释器。
- `marqdo-host`:逐步提供可注册资源操作的能力边界;必须保持旧 `HostOp` 兼容窗口,不能一次砍断所有旧调用。
- `marqdo-package` / CLI:发布标准库兼容信息,继续利用已存在的 path/Git pinned lock;资源授权仍由真实操作者提供。
### 11.2 独立 `marqdo-stdlib`:以 Marqdo 实现通用功能
建议独立仓库 `cflmy/marqdo-stdlib`,首期目录:
```text
marqdo-stdlib/
core/ # 字节编码、整数工具、可恢复错误封装
memory/ # arena, allocator, free-list, binary codecs
collections/ # list/tree/graph/hashmap 等数据结构算法
text/ # UTF-8, parsing
io/ # buffered readers/writers, read_line
data/ # csv, json, 自定义格式
tests/ # 以 marqdo run/call 执行的标准库回归程序
```
高层函数应在 `.mq.md` 里,按普通模块导入。发行版携带一个经测试的 stdlib 版本,支持离线运行;单独发布更新时,要求明确的 Runtime 兼容范围与锁定。标准库可有原生加速 Provider,但不能把它作为常规功能的唯一实现路径。
### 11.3 独立 Platform Providers
系统 Adapter 可以由 Rust、C、汇编或将来的 Marqdo 原生编译代码实现。当前 CLI 不应为普通库函数增加 HostOp;对真正的新 OS 原语,若标准 Provider 不支持,用户必须安装/编写扩展 Provider,并获得相关权限。**不是每个新设备都能只靠修改 `.mq.md` 获得访问能力。**
## 12. 分阶段实施(范围严格受控)
### R0:冻结设计与当前基线(无新功能)
- 此 RFC 和对应 ADR 经审阅,确认 `Memory` 值语义、`index assignment`、错误类别、线协议边界及安全模式。
- 修复已有跨平台 CI/安全收敛问题;固定解释器行为的回归样例。
- 建立一页对照表:新增语义是否需要 `HostOp`、新关键字、新 Markdown 节点。首版目标均为“不需要”。
### R1:最小可寻址存储(核心里程碑)
- `Memory` 区域:创建、大小、增长、读取一字节、写入一字节;实现总/单区域预算、错误恢复与 trace。
- 支持 `**\`m[8] = 42\`**` 这类现有 Markdown 负载中的索引赋值。
- 不开放裸指针、不引入新寄存器、不改旧 List/Map/Table 的赋值与复制行为。
- 运行跨区域隔离与别名测试;报错不能导致原有区域被污染。
### R2:Marqdo 自举的内存与算法库
- `.mq.md` 顺序分配器;随后 free-list 可复用分配器。
- 不修改 Runtime/Host 编写并执行:可增长链表、二叉树、简单虚拟 CPU、UTF-8/二进制解析器。
- 提供内存利用率、分配吞吐、碎片率、执行时间及解释器 fuel 的测量;避免只用“功能跑通”证明成熟度。
### R3:最小系统资源闭环
- 在项目模式先实现 `stdio` 与受控项目文件的 `open/read/write/close`;**功能可用性不代表 strict 安全已通过**。
- `.mq.md` 标准库实现 `read_line`、`write_text`、文件二进制复制与 UTF-8 编码。
- 资源注册层允许 Provider 扩展;验证一个第三方资源端点不修改解释器编译即可接入(是否需要实现新的系统适配器需明示)。
- 对当前 `HostOp` 逐步适配,保留兼容期,不能强行把所有第三方都迁移到新的不成熟 ABI。
### R4:可选增强(不阻断基础版本)
- 索引/字段赋值的一般化、字节多宽度原语及位运算、快照、增量编译、优化执行器。
- 需要共享内存并行时,再设计原子/同步与数据竞争语义。
- 独立网络/TLS、设备、GPU、QPU 后端与原生编译目标。
**必须遵循的顺序**:R1 完整正确 → R2 验证计算自主性 → R3 证明外部交互可组合;不先把 R4 的设备能力和并发类型系统塞入 CLI。
## 13. 端到端验收清单
| 编号 | 验收项目 | 判定标准 |
|---|---|---|
| T1 | 语义一致性 | 旧的局部赋值与 List/Map/Table 行为完全不变,旧 Markdown AST golden 通过 |
| T2 | Memory 隔离 | `m1[0]=x` 不影响 `m2[0]`;`m2=m1` 后写入可相互观察 |
| T3 | 增长与错误 | 成功增长保留旧字节且新字节为 0;负索引、越界、超限增长、巨大 BigInt 均明确失败且不破坏状态 |
| T4 | 自建分配器 | `.mq.md` 实现 malloc/free,释放块重复利用;高低地址和碎片测试通过 |
| T5 | 高级数据结构 | 不改 Rust 核心,仅新增 `.mq.md` 实现可增长链表、树、二进制协议解析器 |
| T6 | 标准库独立升级 | 旧 CLI 二进制保持不变,安装一个与之兼容的新 `.mq.md` 库并成功使用 |
| T7 | 终端闭环 | `marqdo run` 仅使用标准库加基础 OS Adapter,完成多轮输入/输出,无 Python/Node 调用 |
| T8 | 系统资源授权 | 无 grant 无法 open;已关闭/伪造 handle 无法访问;项目路径无法越权;所有拒绝有诊断和必要审计 |
| T9 | 安全模式 | `pure` 不允许外部 I/O;当前 `strict` 不自动放开新 Provider;真实隔离与失败路径有测试 |
| T10 | 扩展接口 | 新高层 I/O/协议函数不增加任何 HostOp;新 Provider 通过约定的独立注册/版本机制接入 |
| T11 | 兼容/性能 | Windows/macOS/Linux 的支持矩阵真实披露;可定量报告字节访问吞吐、内存增长和 Python/Rust baseline 差距 |
| T12 | 文档与证明 | Memory 状态可导出可读快照;执行状态引用源 Markdown span;不将数据表格误认成内存本体 |
## 14. 需要 ADR 决断而不是隐含假设的开放问题
本 RFC 采用一条**可实施的推荐路线**,但下列决策会改变公开语义,须单独冻结:
1. **页语义**:推荐 64 KiB 页、只增长;是否与外部目标标准保持一致,预算如何计入已保留与已提交物理内存。
2. **Memory 的值/调用传递**:推荐复制引用保持身份,区域可达则存活;禁止隐式跨线协议序列化。
3. **Memory 引用寿命**:推荐首版只在一次执行期间有效,采用自动回收或至少结束时回收;内部块分配是 stdlib 职责。
4. **Place AST**:推荐首版只增 Local + Memory Index;其他集合的索引赋值须另行评议。
5. **OS Resource 最小 ABI**:是否先使用现有插件注册机制包装,而不立即新增第二套并行插件协议。
6. **跨平台支持范围**:纯计算 API 保持统一;系统资源能力按平台明示 availability,不能假定所有设备/系统调用均存在。
7. **实验数据**:R2 执行效率可能较低,是否加入 32/64 位宽读写及优化编译由实测结果决定,而非预设几十种关键字。
## 15. 最终建议与决策摘要
**建议通过的架构选择:**
- **语言核心**:保留现有托管值与控制流,新增**显式、多区域、可增长的字节 Memory**;用现有索引表达式扩展可写位置,不创建新的 Markdown 语法层。
- **内存管理能力**:`Memory` 区域由 Runtime 提供并保护区域边界;`malloc/free`、块布局、链表、树、二进制编码等由 `.mq.md` 标准库自己构造。
- **系统能力**:独立、授权、可替换的 `Resource` 边界;标准库在 Marqdo 中组合 I/O,OS 的最后一跳仍由可信本地 Adapter/Provider/编译后端承担。
- **扩展性**:不为每个库函数增加 Rust `HostOp`;只有新增*真正不可由现有原语表达*的硬件/OS 能力,才需要扩展适配器。
- **安全性**:区分区域边界安全、块级内存安全、进程隔离和 OS 权限;不能用“沙盒”一词掩盖缺少的保证。
- **长期演进**:以此作为实现自主数据结构、字节码 VM、优化执行器与量子/科学计算后端的基础,但不许声称三个内存原语就能自动产生 GPU/QPU 加速。
**核心结论:** 如果“摆脱宿主语言依赖”指的是**程序逻辑、标准库算法和高级系统功能不依赖宿主语言来实现**,这条路线可以达成;如果指的是**无需任何底层可信机器代码和操作系统接口即可访问硬件**,则不可能。对真正的独立编程语言而言,恰当目标是**自己定义计算、自己构建高层库、清楚且可扩展地跨越不可消除的系统边界**。
---
## 参考资料(外部设计依据,不是 Marqdo 已实现功能的证据)
1. [WebAssembly Core Specification 3.0(2026-09-11)](https://www.w3.org/TR/wasm-core-2/):线性内存、内存增长、抽象机器、核心与环境 API 的分离;注意其保证为线性内存边界安全,不保证客户程序自己设计的内部分配器没有逻辑错误。
2. [Rust `core` / `alloc` / `std` 文档](https://doc.rust-lang.org/std/):计算基础、分配与平台库的分层思路;并不意味着必须用 Rust 编写 Marqdo 的标准库算法。
3. [Linux Kernel ABI documentation](https://docs.kernel.org/admin-guide/abi.html):OS 用户态接口的稳定性、适用范围和版本边界。
4. [Linux Kernel ioctl interfaces](https://docs.kernel.org/driver-api/ioctl.html):设备特定能力并不都能归约为普通 RAM 读写,ABI 应明确数据布局与操作语义。
5. [Marqdo 当前安全模型](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)及[插件 ABI v1](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/plugin-abi.md)。
+173 -12
View File
@@ -40,12 +40,12 @@ byte-identical).
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
(47 test binaries, 0 failed; includes 7/7 Linux security tests)
430 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
```
### G-C security lane (disposable Ubuntu 26.04 VM, ordinary user `dev`)
Source: `git archive 8af7262` extracted in the VM (not the dev container).
Source: `git archive 70d916d` extracted in the VM (not the dev container).
Host: QEMU/TCG, kernel `7.0.0-43-generic`, `systemd-detect-virt --container`
= `none`, bubblewrap 0.11.1 with `bwrap-userns-restrict`, uid 1000.
@@ -59,7 +59,8 @@ Host: QEMU/TCG, kernel `7.0.0-43-generic`, `systemd-detect-virt --container`
cargo test --workspace --locked (same archive): 47 ok, 0 failed
```
Logs: [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md).
Logs: [`evidence/2026-11-20-strict-resource-cm-a11.log`](evidence/2026-11-20-strict-resource-cm-a11.log)
and [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md).
This is still a **local VM lane**, not a self-hosted GitHub Actions security URL.
## 2. Work package status
@@ -71,12 +72,12 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
| 2 | Cross-platform strict `cfg` | **Fixed locally; native CI UNVERIFIED** | `#[cfg]` dispatch + non-Linux regression test |
| 3 | Strict probe→launch race | **Fixed locally; GitHub Linux CI UNVERIFIED** | fail-closed CLI contract + probe-OK/launch-EAGAIN seam test |
| 4 | Parallel security matrix isolation | **Fixed** | unique paths + helper `OnceLock`; 7/7 in parallel workspace run |
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 44 semantic, 52 compile-negative, 12 runtime-negative fixtures |
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 73 semantic, 52 compile-negative, 21 runtime-negative fixtures |
| 6 | Condition header + source spans | **Fixed** | executable tails rejected; literal-inside-backtick mapping tests |
| 7 | Independent Tool consumers | **Done locally** | `sample-view`/`sample-knowledge`/`sample-proof` E2E test |
| 8 | lock v3 reproducibility | **Implemented; tamper test added** | `package_env` PASS |
| 9 | Analysis/Trace/Tool/ABI/Foreign | **Implemented; supported-matrix bounded** | protocol tests, plugin/worker/foreign tests |
| 10 | Linux strict L1 | **VM PASS at 8af7262; no CI URL** | 7/7 + positive CLI |
| 10 | Linux strict L1 | **VM PASS at 70d916d; no CI URL** | 7/7 + positive CLI + strict resource refusal |
| 11 | Linux strict L2 | **NOT IMPLEMENTED / UNSUPPORTED** | strict rejects Worker/Foreign/native ABI |
| 12 | Windows Native/Worker ABI | **UNSUPPORTED** | no Windows ABI artifact/claim |
| 13 | RC artifact | **Linux candidate only; no release** | clean-install + checksums; macOS not built here |
@@ -86,7 +87,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
- Frozen Markdown profile `marqdo.md.v1` with 160 AST goldens and explicit
CRLF byte semantics.
- Native Rust interpreter with 44 semantic goldens, 52 compile-error fixtures,
- Native Rust interpreter with 73 semantic goldens, 52 compile-error fixtures,
and runtime error source attribution.
- Native `**` power is pinned end-to-end: right associativity, unary-minus
precedence, parentheses, float negative exponents, branch/loop conditions,
@@ -109,6 +110,86 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
version verification.
- Linux strict L1: Bubblewrap namespace supervisor, runtime probe, fail-closed
launch, child self-verification, 7/7 adversarial matrix.
- Addressable memory (RFC-CM P1, ADR-0022): opaque `Memory` values shared by
handle identity, region isolation, zero-filled 64 KiB page growth, bounded
byte read/write, `memory.new/size/grow`, `m[i] = byte`, run and per-region
budgets, host/wire rejection, Analysis v2 target projection.
- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): success is
`{ok:true,value:...}`, failure is `{ok:false,code,message[,details]}` as an
ordinary Map; expected conditions return it, invariant violations stay
fatal. A `.mq.md` first-fit/free-list allocator reuses blocks, splits free
blocks, coalesces address-adjacent blocks and returns `alloc.no_space`
without aborting.
- CM-A07 Compat-A (ADR-0027): explicit `.mq.md` `io.写值` chain
(`format.显示` → UTF-8 encode → `Memory` → `resource.write(stdout)`) matches
builtin `打印` for the covered value set; builtin remains default and no
alias is routed.
- Compat-C print completion (ADR-0031): `HostOp::Print`/`op_print` removed;
the print builtin only routes to bundled `io.写值` or fails with
`kernel.unavailable`; trace has resource effects only.
- Compat-B print routing (ADR-0030): `Program.print_override` routes
`打印/print` to the bundled `io.写值` when the program uses it; stdlib
modules are injected lazily; resource operations are trace `effect` events
and `HostOp::Print` no longer runs on that path. Strict permits only
stdio/stdout; file/network resources remain refused.
- Full-value stdlib formatting (ADR-0029): FunctionRef `module/name` and
PluginHandle `plugin/export` are read-only observation members (host id
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
- C ABI provider worker binding (CM-A09, ADR-0038): the separate
`marqdo-provider-worker` helper loads a trusted C ABI library and serves the
framed-JSON transport; process isolation but not a sandbox.
- Provider package manifest v1 (CM-A09, ADR-0037): a local `marqdo.provider`
manifest resolves a trusted C ABI library and registers it; no registry,
signatures or CLI install yet.
- C ABI ResourceProvider v1 (CM-A09, ADR-0036): trusted shared libraries expose
open/read/write/close through the frozen header; version/load failures are
fail-closed; project-mode + name grant only. Plugin-Worker binding remains open.
- Worker resource provider transport (CM-A09, ADR-0035): host-managed helper
process with framed JSON, ping/open/read/write/close; project-mode + name
grant; trusted, not a sandbox; C ABI binding still open.
- HTTP framing and client (P5/C10): `stdlib/net/http.mq.md` parses status,
headers and `Transfer-Encoding: chunked`, builds requests, and an end-to-end
test parses a loopback TCP response; fixture 71 pins the same logic. No host
HTTP/TLS implementation.
- Blocking TCP endpoint (P5 first slice, ADR-0034): project-mode
`resource.open("tcp", ip:port, ["read","write"])` with exact IP grant and
timeouts; no DNS/TLS/UDP/server sockets; strict/pure refuse. CLI `--allow-net`.
- In-process ResourceProvider v1 (CM-A09 first slice, ADR-0033): trusted
providers register additional resource kinds without parser/runtime/HostOp
changes; project-mode + name grant required; pure/strict refuse provider
kinds; C/Worker transport remains open.
- P6 baseline profile: `tools/perf/bench.py` measures 7 `.mq.md` workloads with
wall time and peak RSS; no optimizer/backend was added and P6 is not claimed
complete.
- Cross-storage table equivalence (P4, ADR-0032): fixture 72 compares T0 row
Table results with a `.mq.md` columnar List/Map store for sort/filter.
- Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md`
provides select/filter/sort/append/set/join/group/`转GFM` over the logical
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
storage are unchanged.
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
`format`/`text`/`io`; explicit `std:` imports resolve only inside the bundle;
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
`project.stdlib`; a frozen-binary test asserts the CLI hash is unchanged after
running a `std:io` project and that tampering is rejected. `打印/print` is
routed to the bundled stdlib (Compat-C, ADR-0030/0031); there is no legacy
host formatter. `tools/pack-stdlib.py` additionally emits a byte-deterministic
offline archive + descriptor as an extraction-ready artifact
([evidence](evidence/2026-11-20-stdlib-package.md)). The stdlib intentionally
stays in this repository for now (ADR-0039); extraction is a future option
when size/release cadence requires it, not a pending gate.
- Resource streams (RFC-CM CM-A06, ADR-0025): opaque host-issued `Resource`
handles with `resource.open/read/write/close`; bytes move only through
`Memory`; stdout write is capability/output-budget checked and project-file
read/write is path/grant checked; file writes buffer until `close` and
never flush on drop; a `.mq.md` chunked copy loop reproduces a file without a
Rust `copy_file`; strict permits only the stdio console channel and refuses
file/TCP/provider resources (CM-A11 lane PASS). Semantic fixture 66 and CLI/host tests.
- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
value-semantics `list.append/concat`, `map.keys/has/get/set/remove`,
Unicode scalar `text.codepoints`/`text.from_codepoint`, and non-negative
`table.columns/row/cell`. Pure builtins; no mutation, no new syntax.
## 4. Open blockers and limits
@@ -116,30 +197,110 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
the Linux EAGAIN contract are UNVERIFIED on GitHub. G1/G2/G3 remain FAIL
until a real run exists.
2. **G-C is VM-verified but not CI-signed.** No self-hosted security-lane URL
for 8af7262 exists.
for 70d916d exists.
3. Windows Native/Worker ABI is explicitly UNSUPPORTED (no Windows `cc`
fixture lane, Unix-gated worker tests).
4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed
broker services; unsupported requests must return `capability.unavailable`.
5. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
5. Resource streams cover project-file read/write, stdout, compatible print,
trusted in-process providers and IP-literal TCP; directories, process,
device, DNS/TLS/HTTP, UDP and provider packaging remain open. Strict permits
only stdio and refuses the rest (CM-A11 lane PASS).
6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
digest-pinned directory with a regular-file executable is supported.
6. No Windows/macOS RC archive exists from this host. The release-candidate
7. No Windows/macOS RC archive exists from this host. The release-candidate
workflow can build macOS on GitHub, but it has not run for this SHA.
7. No release tag/binary publication. The release decision is **DO NOT
8. No release tag/binary publication. The release decision is **DO NOT
RELEASE** until the native CI matrix and a security-lane CI URL exist.
## 5. Evidence files
## 5. RFC-CM compute-machine status
The two 2026-11-20 architecture RFCs are design input, not implemented
behavior. This table records only what is in this tree now.
| RFC phase | State | Evidence / limit |
|---|---|---|
| P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged |
| P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) |
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PASS (local pure lane); stdlib remains in-repo by design** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), an Int matrix multiply (70), and a sparse-matrix multiply (73); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Local gate PASS: `crates/cli/tests/p2_closure.rs` runs 21 C01–C06/C11/C12 fixtures plus the stdlib table self-test in `--mode pure` (no foreign/plugin/import grants), scans sources for foreign/plugin/import shortcuts and compares value/stdout to goldens; evidence [`p2-closure-gate`](evidence/2026-11-20-p2-closure-gate.md). The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). The stdlib intentionally remains in-repo (ADR-0039); a separate repository is a future extraction option, not a gate. Remote CI URL is still pending (see §1). |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Providers are registered through ADR-0033 (in-process) and ADR-0035 (worker helper). Still missing: directories, C ABI provider binding, provider packages, strict provider support. |
| P4 Table logical/physical migration | **PASS (local gate); T1/T2/T3 future** | ADR-0032/CM-A08: `Value::Table` semantics and GFM AST laziness unchanged; `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group/`转GFM` as value-returning `.mq.md` functions with input immutability; zero Rust table algorithms were added. Cross-storage equivalence is pinned by fixture 72 (row Table vs `.mq.md` columnar store, identical headers/rows/cells). The optional T1/T2/T3 physical-storage/column-store migration was not required by the gate and is not implemented. |
| P5 third-party Provider / TCP / HTTP | **PASS (local gate); external network/CI and plugin-Worker resource binding open** | C10: `stdlib/net/http.mq.md` + fixture 71 parse HTTP/1.1 status/headers/chunked bodies and build requests; `http_over_tcp.rs` runs a loopback HTTP exchange wholly in `.mq.md` over `resource.open("tcp",...)`. ADR-0034 IP-literal blocking TCP endpoint (exact IP grant, project-only, 5s timeouts) and ADR-0033 in-process `ResourceProvider` v1 (registration, protocol/kind checks, name grant, mock read/write provider) ADR-0035 host-managed worker provider transport (framed JSON helper, ping/open/read/write/close, trusted-only) and ADR-0036 trusted C ABI provider (shared-library symbols, version/load checks) ADR-0037 local provider-package manifest installation and ADR-0038 C ABI provider worker binding (separate trusted helper process) are tested. External DNS/TLS/UDP, host HTTP, provider packages/signatures and the plugin-Worker resource binding remain out of scope. |
| P6 performance / native backends | **BASELINE ONLY** | reproducible profile harness `tools/perf/bench.py` and [`p6-profile`](evidence/2026-11-20-p6-profile.md) for 7 workloads; no bytecode/JIT/columnar numeric backend or equivalence/optimization gate yet |
The P1 memory domain is pure computation: it needs no grant and is available in
`pure`, `project` and `strict` (bounded by the memory budgets). It adds no
`HostOp`, no Markdown role and no CLI command. `Memory` handles cannot cross
the wire or a host/plugin/foreign boundary. Block-level safety of a
user-written allocator is explicitly **not** claimed.
## 6. Evidence files
- [`evidence/2026-11-20-local-gate.md`](evidence/2026-11-20-local-gate.md) —
earlier local gate and Windows autocrlf reproduction.
- [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md)
— current VM lane, environment, full log and workspace test log.
- [`evidence/2026-11-20-resource-stream-p3.md`](evidence/2026-11-20-resource-stream-p3.md)
— CM-A06 first slice: stdout write, project-file chunked read, strict refusal,
wire/host boundary rejection and test commands.
- [`evidence/2026-11-20-resource-file-write-p3.md`](evidence/2026-11-20-resource-file-write-p3.md)
— buffered project-file write, flush-on-close, buffer ceiling, grants and
CLI write-then-read evidence.
- [`evidence/2026-11-20-stdlib-print-compat-a.md`](evidence/2026-11-20-stdlib-print-compat-a.md)
— CM-A07 Compat-A `.mq.md` print prototype, side-by-side output comparison,
covered/uncovered value types and the Compat-B blocker.
- [`evidence/2026-11-20-bundled-stdlib-cm-a10.md`](evidence/2026-11-20-bundled-stdlib-cm-a10.md)
— CM-A10 bundled stdlib, `std:` imports, digest verification, Analysis
identity and frozen-binary closure test.
- [`evidence/2026-11-20-full-value-formatting.md`](evidence/2026-11-20-full-value-formatting.md)
— ADR-0029 observation members and all-15-type stdlib formatting comparison.
- [`evidence/2026-11-20-print-compat-b.md`](evidence/2026-11-20-print-compat-b.md)
— ADR-0030 Compat-B print routing, trace resource effects, lazy stdlib
injection and strict stdio console permission.
- [`evidence/2026-11-20-print-compat-c.md`](evidence/2026-11-20-print-compat-c.md)
— ADR-0031 removal of `HostOp::Print`, no-bundle fail-closed behavior and
embedder compatibility note.
- [`evidence/2026-11-20-table-cm-a08.md`](evidence/2026-11-20-table-cm-a08.md)
— ADR-0032 table logical value, stdlib select/filter/sort/append/set/join/
group, input immutability and T0 storage boundary.
- [`evidence/2026-11-20-provider-abi-cm-a09.md`](evidence/2026-11-20-provider-abi-cm-a09.md)
— ADR-0033 in-process ResourceProvider v1, grants/mode checks, mock provider
read/write and registration rejection cases.
- [`evidence/2026-11-20-tcp-endpoint.md`](evidence/2026-11-20-tcp-endpoint.md)
— ADR-0034 blocking TCP endpoint, IP grants, DNS refusal, loopback host/CLI
echo tests.
- [`evidence/2026-11-20-table-cross-storage.md`](evidence/2026-11-20-table-cross-storage.md)
— P4 row-store Table vs `.mq.md` columnar store equivalence for sort/filter.
- [`evidence/2026-11-20-c-worker-binding.md`](evidence/2026-11-20-c-worker-binding.md)
— ADR-0038 C ABI provider worker binding, compiled C adapter in the helper
process and fail-closed missing-library case.
- [`evidence/2026-11-20-provider-package.md`](evidence/2026-11-20-provider-package.md)
— ADR-0037 local provider package manifest, C library resolution and
registration test.
- [`evidence/2026-11-20-c-provider.md`](evidence/2026-11-20-c-provider.md)
— ADR-0036 C ABI ResourceProvider v1, compiled C adapter round-trip and
fail-closed load/version checks.
- [`evidence/2026-11-20-worker-provider.md`](evidence/2026-11-20-worker-provider.md)
— ADR-0035 trusted worker resource-provider transport, framed-JSON helper
round-trip and grant/mode/error tests.
- [`evidence/2026-11-20-http-over-tcp.md`](evidence/2026-11-20-http-over-tcp.md)
— P5/C10 `.mq.md` HTTP framing over a loopback TCP stream, stdlib module and
end-to-end client test.
- [`evidence/2026-11-20-p2-closure-gate.md`](evidence/2026-11-20-p2-closure-gate.md)
— P2 C01–C06/C11/C12 pure no-Foreign/no-Plugin closure lane and golden
comparison.
- [`evidence/2026-11-20-http-parser.md`](evidence/2026-11-20-http-parser.md)
— P5/C10 `.mq.md` HTTP/1.1 status/header/chunked response parser and request
builder fixture.
- [`evidence/2026-11-20-strict-resource-cm-a11.md`](evidence/2026-11-20-strict-resource-cm-a11.md)
— CM-A11 strict lane at `00bdcad`: 7/7 adversarial matrix, positive Compat-C
print, strict file-resource refusal, forged-marker rejection and raw log.
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
— G0..G12 matrix and RC decision.
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
L1/L2 limits.
## 6. Reproduce
## 7. Reproduce
```bash
export PATH="$HOME/.cargo/bin:$PATH"
+11 -2
View File
@@ -54,14 +54,23 @@ The trusted supervisor is this process; the restricted child is the same
The document argument is rewritten to `/project/...`; only the document token
changes, all other arguments are preserved.
### 4. Child self-verification
### 4. Bundled stdlib mount
After CM-A07 Compat-C, `打印` is implemented by the bundled `.mq.md` stdlib.
Strict therefore bind-mounts the verified `stdlib/` tree read-only at
`/run/marqdo-stdlib` and sets `MARQDO_STDLIB_DIR` for the child, just like the
program itself. The tree contains only Markdown/read-only data; it grants no
new OS capability, and the loader still verifies its digest in the child. File,
network, process and provider resources remain refused.
### 5. Child self-verification
The child accepts `MARQDO_STRICT_ACTIVE=1` only if it can verify all of:
`getpid() == 1`, `NoNewPrivs: 1`, empty `CapEff`, a non-initial uid map, and no
non-loopback network interface. A forged marker outside the sandbox fails
before project code runs.
### 5. Threat model and non-goals
### 6. Threat model and non-goals
This protects against untrusted Marqdo documents that use the language,
interpreter, Math, console, or attempt direct syscalls. It does not sandbox
@@ -0,0 +1,183 @@
# ADR-0022: Addressable `Memory` and Memory-index place assignment (RFC-CM P1)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md` (CM-A01/CM-A02/CM-A03),
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (P0/P1), ADR-0007,
ADR-0011, ADR-0017, ADR-0021
Supersedes: the v1 rule that assignment targets are bare identifiers only
Specs: `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`,
`spec/analysis-v2.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-memory-p1.md`
## Context
RFC-CM-0001/0002 define three domains: managed values, addressable regions and
capability resources. The first two are a language change; the third is an OS
boundary and is deliberately **not** part of this ADR.
Before this decision the interpreter had no writable place other than a local
slot. `Value` had no byte region, `a[i]` was read-only, and assignment was
parsed only as `identifier = expression`. A `.mq.md` program therefore could
not express an allocator, an encoded record, a free list or a byte protocol
without a new Rust builtin for every algorithm. That is the closure gap
RFC-CM-0002 §2 identifies.
The decision must not:
* turn managed `List`/`Map`/`Table` values into shared mutable objects;
* expose a host pointer, `usize` address or procedural `free(region)`;
* let an active handle cross the value wire or an ABI boundary;
* add a Markdown node, keyword or `HostOp` for memory;
* claim block-level memory safety for user allocators;
* weaken `strict` or grant any external I/O.
## Decision
### 1. `Value::Memory` is an opaque, run-scoped identity
`MemoryHandle` (`crates/marqdo-value/src/value.rs`) is
`(run_id, region_id, generation)` with private fields and a runtime-only
constructor. It is not `Serialize`/`Deserialize`, is never built from user
input, and carries no host address.
Copying the value, passing it as an argument and storing it in a list or map
all copy the handle and preserve region identity. `==`/`!=` compare identity
only; `Memory` has no ordering comparison. `display()`/`repr()` render the
literal `Memory`, never the internal identity.
### 2. Regions are grow-only, bounded byte arrays
Frozen semantics (CM-A01/CM-A02):
| Property | Decision |
|---|---|
| Page size | 64 KiB logical page (`PAGE_SIZE`) |
| Initial content | Every byte is `0` |
| Growth | `memory.grow` only increases; old offsets/bytes unchanged; new bytes zero |
| Failure atomicity | A rejected create/grow allocates, commits and mutates nothing |
| Shrink/free | No shrink and no user-visible region free; regions are dropped with the run |
| Address | `(region identity, non-negative offset)`; an integer alone is never a credential |
| Read | `m[i] -> Int(0..=255)` |
| Write | `m[i] = byte`, `byte` must be `Int(0..=255)`, no truncation |
| Isolation | Different regions never alias; handle copies of one region do |
| Visibility | `memory.size` returns committed pages, not host capacity or pointers |
Limits in `MemoryLimits`:
* `DEFAULT_MAX_TOTAL_BYTES = 64 MiB` committed across the run;
* `DEFAULT_MAX_REGION_BYTES = 16 MiB` per region, which also caps declared
`max_pages` at 256;
* budget arithmetic uses checked `u64`/`usize` operations; allocation uses
`try_reserve` so a host OOM is a structured `resource.exhausted` instead of
a panic.
Limits are a runtime API (`marqdo_runtime::MemoryLimits`, `run_with_limits`)
and are overridable on `run`/`call` with `--max-memory <BYTES>` (total) and
`--max-region-memory <BYTES>` (per region). The defaults are stated in
`spec/evaluation.md`. Both flags apply in `pure`, `project` and `strict`; they
grant no external capability.
### 3. The only new writable place is a `Memory` index
`AssignTarget` gains `Index { base, index, span }` in the expression AST and
`PlaceTarget` mirrors it in the IR. The surface syntax remains a `strong`
node containing `target = expression`; no Markdown role, heading or list
meaning changes.
* `x = value` still rebinds the current frame's local slot.
* `m[i] = byte` evaluates **base, then index, then RHS**, exactly once each.
All three checks (base is `Memory`, offset in range, byte in `0..=255`)
happen before the single byte write, so a failed statement does not write.
* `list[i] = value`, `map[key] = value` and `obj.field = value` are not
defined. Member targets are a parse error; a non-`Memory` index base is
`runtime.type`. No managed collection changes value semantics.
* Nested index targets are accepted syntactically; the innermost base must
still evaluate to a `Memory` value at run time.
* Assignment remains statement-only; `x = y = 1` is still a parse error.
This is CM-A03.
### 4. Handles cannot cross the wire or the host boundary
* `to_wire(Memory)` emits only `{"type":"memory"}` so an accidental transfer
is visible. `from_wire` rejects that tag with `WireError::Invalid`; a
round-trip is intentionally impossible (no `wire_version` bump: the value
shape is rejected, not newly accepted).
* `Host::invoke` rejects any argument tree containing `Memory` before dispatch
(`runtime.type`), so plugins, foreign kernels, math, file and print adapters
never see a handle. A future explicit byte-snapshot operation must copy bytes
into a managed value and is a separate ADR.
* `HostOp` is unchanged. `memory.new/size/grow` and index read/write are
interpreter operations, not capability-gated host effects.
### 5. Error categories are stable and distinct
| Situation | Code |
|---|---|
| Negative/unrepresentable offset, offset >= region length | `runtime.index` |
| Unknown, stale or cross-run handle; invalid page count (negative, > budget declaration arithmetic) | `runtime.memory` |
| Wrong operand type; byte outside `0..=255`; non-Memory index-assignment base | `runtime.type` |
| Declared max, region budget or total budget exceeded | `resource.exhausted` |
No operation panics on malformed input or forged handles. `expect` is used only
after the same handle has been validated in the same call frame.
### 6. Builtins are part of the language surface, not `HostOp`s
`memory.new(initial_pages, max_pages) -> Memory`,
`memory.size(memory) -> Int`, `memory.grow(memory, pages) -> Int` are added to
`marqdo-ir::builtins::BUILTINS` and `spec/evaluation.md`. They are pure
(`effectful: false`): `pure` mode may use them, `strict` needs no new provider,
and no capability grant is consulted.
### 7. Compatibility
* Local assignment IR keeps its `target` string in Analysis v2; an index
target is an additive object (`{"kind":"index",...}`). Consumers that only
handle locals continue to work.
* Expression AST/statement serde gains an enum tag; this is a pre-1.0 internal
contract documented in `spec/expression.ebnf`.
* Existing Markdown AST goldens, value semantics, `List`/`Map`/`Table`
behaviour and all old builtins are unchanged.
* `Memory` is additive to the value type set. Old programs never produce it.
ABI wire consumers see a new rejected tag, not a renumbered existing tag.
## Non-goals
* No `Pointer` value, raw pointer, `free(region)`, registers or machine code.
* No index/field assignment for managed collections; no implicit mutation.
* No multi-threaded sharing, atomics or data-race guarantees.
* No serialization of live regions; no cross-process shared memory.
* No `ResourceHandle`, `Stream`, files, network or OS syscalls. Those are
CM-A05/CM-A06/CM-A09/CM-A11 and later RFC phases.
* No block-level safety claim for a user-written allocator; the runtime only
bounds a single access to its region.
* No allocator or stdlib API in P1; `--max-memory` only bounds the runtime.
## Consequences
* A `.mq.md` program can now build byte encoders, arenas, free lists and
virtual CPU register files without new Rust code. P2 (the Marqdo-written
stdlib and closure fixtures) becomes testable.
* The interpreter owns one `MemoryStore` per run; `run_with_limits` exposes
the budget. This is runtime state, not a second `HostOp` table.
* Trace gains `memory.new`, `memory.grow` and `memory.write` semantic events
without exposing handles or byte secrets.
* The remaining RFC decisions (CM-A04 and later) stay separate. In particular,
generic structural observation/construction, recoverable `Result`, table
algorithm migration and the resource/Stream ABI are **not** implemented by
this ADR.
## Verification
`docs/evidence/2026-11-20-memory-p1.md` records:
* parser/AST tests for index and rejected member targets;
* runtime store tests for isolation, aliasing, zero-fill, atomic failed grow,
negative/huge offsets, forged/stale handles, byte range and budgets;
* semantic goldens `45`–`47` and runtime negatives `13`–`18`;
* wire rejection test and host-boundary rejection test;
* CLI example `examples/memory-buffer.mq.md` returning `141`;
* Analysis v2 index-target projection;
* `cargo fmt --check`, `cargo clippy -- -D warnings`, `cargo test --workspace`.
@@ -0,0 +1,147 @@
# ADR-0023: Minimal structural observation and construction (RFC-CM CM-A04)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A04, P2),
ADR-0007, ADR-0022
Specs: `spec/evaluation.md`, `spec/expression.ebnf`, `spec/analysis-v2.md`
Evidence: `docs/evidence/2026-11-20-structure-api-p2.md`
## Context
RFC-CM-0002 §3.5 identifies a closure gap: a `.mq.md` program can read a
`List`/`Map`/`Table` and print it through the host, but it cannot generally
**construct** the same structures dynamically or **observe** all of their
logical parts. That makes `format`, JSON/CSV parsers and table algorithms
impossible to write in `.mq.md` without adding a task-specific Rust function
for each algorithm.
The current core already has:
* `List` literals, index reads (including negative offsets), `长度`;
* `Map` record literals, `.field` reads, `长度`;
* `Text` concatenation, character indexing and `长度`;
* `Table` construction through `表(header, rows)`, `长度` (rows), display and
wire round-trip.
It lacks: append/concat, dynamic map update/enumeration, Unicode scalar
conversion, and table column/row/cell projection. These are observation and
construction primitives, not algorithms. Sorting, filtering, joining,
serialising and parsing remain standard-library work.
## Decision
Add a small, namespaced, **pure** builtin set. No new Markdown node, keyword,
`HostOp`, capability or mutation semantics is introduced. All constructors
return **new** values; an input collection is never modified in place. This
keeps INV-02 and preserves existing `List`/`Map`/`Table` copy semantics.
### 1. `List`
| Signature | Semantics |
|---|---|
| `list.append(list: List, value) -> List` | New list with `value` at the end; the input list is unchanged. |
| `list.concat(left: List, right: List) -> List` | New list containing `left` then `right`; both inputs unchanged. |
Neither aliases the other list's element storage: values are copied according
to their normal value semantics (`Memory` handles keep region identity).
### 2. `Map`
| Signature | Semantics |
|---|---|
| `map.keys(map: Map) -> List` | `Text` keys in canonical sorted order (`BTreeMap` order, the order used by `display`). |
| `map.has(map: Map, key: Text) -> Bool` | Whether the key exists. |
| `map.get(map: Map, key: Text) -> Value` | Stored value, or `None` when absent. Use `map.has` when `None` is a legal stored value. |
| `map.set(map: Map, key: Text, value) -> Map` | New map with the key set; the input map is unchanged. |
| `map.remove(map: Map, key: Text) -> Map` | New map without the key (missing key is a no-op); the input map is unchanged. |
Keys are always `Text`; arbitrary-value keys are not introduced. `map.get`
returning `None` is a *lookup result*, not the language's recoverable-error
contract; a full `Result` convention remains CM-A05 and is not frozen here.
### 3. `Text`
`Text` remains a Unicode scalar-value sequence, not a byte array. These
primitives make the existing representation observable without introducing a
second text type or implicit UTF-8 conversion.
| Signature | Semantics |
|---|---|
| `text.codepoints(text: Text) -> List` | Unicode scalar values (`Int`) in code-point order. |
| `text.from_codepoint(codepoint: Int) -> Text or None` | One-scalar `Text`; `None` for a surrogate or a value outside `0..=0x10FFFF`. |
UTF-8 encoding/decoding is **not** a core primitive. It can be written in
`.mq.md` using bytes, `text.codepoints` and `text.from_codepoint`; this keeps
the byte/code-point distinction explicit.
### 4. `Table`
`Value::Table` stays the logical value; its physical `Vec<Vec<Value>>` layout
is not part of this contract. Indices are non-negative row/column coordinates
(unlike `List` negative indexing) because a table cell is a document
coordinate. Bounds failures are `runtime.index`.
| Signature | Semantics |
|---|---|
| `table.columns(table: Table) -> List` | Header names (`Text`) in column order. |
| `table.row(table: Table, row: Int) -> List` | A new list of the row's cells in column order. |
| `table.cell(table: Table, row: Int, column: Int) -> Value` | The cell value. |
No `table.set_cell`/`append_row` builtin is added. Rebuilding a table is a
value-level operation that can be written with `表(header, rows)` once list
construction exists; adding a Rust table algorithm would violate the closure
goal.
### 5. Error and compatibility rules
* Wrong collection/text/table type or key type → `runtime.type`.
* Table row/column outside the logical shape → `runtime.index`.
* `text.from_codepoint` never throws for an invalid scalar; it returns `None`.
* `map.get` never throws for a missing key; it returns `None`.
* Existing `List`/`Map`/`Table` behavior, display, wire encoding and Markdown
laziness are unchanged.
* The builtins are `effectful: false`, available in `pure`, `project` and
`strict`, and add no capability. They do not consume output budget.
* Names are frozen as written; aliases are not added. Namespaced calls use the
existing `Expr::Member` resolution path.
## Non-goals
* No `set_cell`, `append_row`, `sort`, `filter`, `join`, `group`, JSON/CSV or
other algorithm builtins; those belong in `.mq.md`.
* No mutable/aliased List or Map, no in-place mutation, no shared structural
sharing protocol.
* No arbitrary-value map keys, no ordered-map insertion semantics, no
duplicate-key policy change.
* No UTF-8/UTF-16 conversion primitive, no byte-array text type.
* No `Result`/`try`/exception syntax; CM-A05 stays open.
* No change to Table storage, GFM parsing, source spans or `ref` semantics.
## Consequences
* A `.mq.md` program can build maps and lists dynamically, enumerate keys,
project table cells and encode Unicode scalars. This unblocks a Marqdo-
written formatter, table filter/projection and a JSON/CSV parser without
per-algorithm Rust.
* The builtin table grows by 12 pure entries. The migration rule remains:
only observation/construction primitives may enter core; algorithms do not.
* Because constructors are value-returning, repeated `map.set` in a loop is
O(n²) for a BTreeMap clone. That is acceptable for P2 semantics; a
persistent/structural-sharing representation is a P6 performance decision,
not a semantic one.
* `map.get` returning `None` for absence is deliberate and documented; a
future CM-A05 `Result` convention must not silently reinterpret existing
programs.
## Verification
`docs/evidence/2026-11-20-structure-api-p2.md` records:
* ADR/spec deltas and builtin metadata;
* semantic fixtures for value-semantics list/map operations, code-point
round-trips, table projection and invalid code points;
* negative fixtures for wrong types and table bounds;
* unit tests, `cargo fmt --check`, `cargo clippy -- -D warnings` and
`cargo test --workspace`.
@@ -0,0 +1,162 @@
# ADR-0024: Recoverable result convention (RFC-CM CM-A05)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A05, P2/P3),
ADR-0023
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-20-result-and-free-list-p2.md`
## Context
A standard library cannot be written in `.mq.md` if every expected condition
(no free block, invalid input, missing field, later end-of-file or a short
read) must abort the whole run. The core has structured `RuntimeError`s with
stable codes, but no way for an ordinary function to return a failure that the
caller can inspect and handle in a normal branch. RFC-CM-0002 §3.6 recommends a
`Result` data contract using existing `Map` values and control flow, without
`try`/`catch` or exception syntax.
This ADR freezes the **data convention**, not a new runtime type. A separate
core `Result` value, algebraic types or automatic propagation would be a
larger change and is deliberately not introduced.
## Decision
### 1. Shape
A recoverable result is an ordinary `Map`:
```text
success: {ok: true, value: <any Value>}
failure: {ok: false, code: <Text>, message: <Text> [, details: <Map>]}
```
* `ok` is always present and always `Bool`.
* On success, `value` is present. It may itself be `None`, a `List`, a `Map`,
a `Table`, a `Memory` handle or any other value.
* On failure, `code` and `message` are present. `code` is a lowercase
dot-separated stable identifier; `message` is human-readable text.
* `details` is optional and, when present, is a `Map` of additional structured
facts. It must not contain host pointers, resource handles or secret bytes.
* No other fields are required. Consumers must tolerate additive fields.
The constructors are ordinary `.mq.md` functions, for example:
```text
## 成功
`(值)`
*{ok: true, value: 值}*
## 失败
`(代码, 消息)`
*{ok: false, code: 代码, message: 消息}*
```
They are **not** builtins. A library may choose different local function names;
the map shape is the contract.
### 2. What is recoverable, what stays fatal
Recoverable (`Result`):
* a user allocator has no fitting block (`alloc.no_space`);
* a parser rejects malformed input or hits an unexpected end
(`parse.invalid_syntax`, `parse.unexpected_eof`);
* a codec rejects an invalid sequence (`codec.invalid_sequence`);
* a lookup that uses an explicit result because `None` is a legal value;
* later, a resource reports EOF/partial/pending as a **success payload** with
its own byte counts, not as a fatal error (P3 contract).
Fatal `RuntimeError` (unchanged):
* violating a language invariant: wrong operand type, out-of-bounds managed
collection access, unknown/stale `Memory` handle, memory budget exhaustion,
division by zero, call arity, unknown symbol;
* a capability denial (authorization is not a recoverable business result);
* an internal invariant failure.
A library must not convert a fatal invariant into an `Err` by catching it;
there is no catch mechanism and none is proposed here. It must validate inputs
before calling an operation whose failure is fatal, or rely on its own
pre-checks.
### 3. Codes and namespaces
`code` is stable within the library that defines it. Recommended prefixes:
| Prefix | Owner |
|---|---|
| `alloc.*` | allocator libraries |
| `parse.*` | parsers |
| `codec.*` | byte/text codecs |
| `table.*` | table algorithms |
| `io.*` | stream wrappers (P3, when the resource contract exists) |
Codes are compared as `Text`; a caller must not depend on the `message` text.
Libraries may add codes; removing or renaming one is a breaking change for that
library.
### 4. EOF and partial I/O (forward contract)
When the Stream/Resource API arrives (CM-A06), a successful read must
distinguish:
```text
{ok: true, value: {count: Int, eof: Bool, pending: Bool}}
```
EOF is not an `Err`; a short read is not an `Err`; a connection reset or a
permission failure is. The exact stream payload is frozen by CM-A06, not by
this ADR. This ADR only reserves the principle.
### 5. Why not a new type or syntax
* A dedicated `Value::Result` would change the value model, wire format,
display, equality and every exhaustive `Value` match for a convention that
existing `Map` plus `ok` already expresses.
* `try`/`catch` would add a control-flow construct that bypasses ordinary
branches, contradicts the RFC's preference for `Result` values and needs new
grammar, scope and error rules.
* Ordinary `Map` results are visible to `type`, `length`, `map.get` and wire
encoding today, and can be built by `.mq.md` libraries without a new ABI.
## Compatibility
* Additive: existing builtins and programs are unchanged.
* `map.get` returning `None` for a missing key remains a lookup convention,
not a `Result`; callers that need to distinguish a stored `None` use
`map.has` (ADR-0023).
* Runtime error categories and codes are unchanged.
* No wire version bump: a `Result` is an ordinary `Map` already covered by the
wire format.
## Non-goals
* No automatic `?`/propagation, exceptions, panic recovery or stack unwinding.
* No `Result` type, generics, pattern matching or exhaustive `Ok`/`Err`
typing.
* No change to `memory.grow` or other core operations from fatal errors to
`Result`; a library may pre-check and return `Err` instead.
* No resource/stream implementation; CM-A06 remains open.
* No promise that every existing library will adopt the convention.
## Consequences
* `.mq.md` algorithms can return and branch on failure without aborting the
run. This is a prerequisite for a Marqdo-written allocator, parser and
eventually `io.read_line`.
* Because the convention is untyped, a caller can inspect the wrong shape;
`类型(result)`, `map.has(result, "ok")` and the frozen field names are the
checks. A future typed-value ADR may improve this without breaking the data
shape.
* The allocator fixture in `54`/`55` demonstrates success and failure maps and
ordinary branching, with no Rust algorithm.
## Verification
`docs/evidence/2026-11-20-result-and-free-list-p2.md` records a `.mq.md`
first-fit/free-list allocator that returns `Result` maps, reuses freed blocks,
splits free blocks, coalesces address-adjacent free blocks and returns
`alloc.no_space` without aborting the run.
@@ -0,0 +1,176 @@
# ADR-0025: Minimal Resource/Stream ABI and byte transfer through `Memory` (RFC-CM CM-A06)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A06, CM-A09,
CM-A11, P3), ADR-0022, ADR-0024
Specs: `spec/evaluation.md`, `spec/security.md`, `spec/value-wire.md`
Evidence: `docs/evidence/2026-11-20-resource-stream-p3.md`
## Context
P1/P2 give `.mq.md` managed values, addressable `Memory`, structure primitives,
a recoverable `Result` convention and a substantial set of in-language
algorithms. Nothing yet lets a standard-library function perform a real byte
transfer to the outside world without a dedicated `HostOp` per high-level
function.
CM-A06 asks for a stable, authorized, versioned resource boundary: obtain a
granted resource, call the operations it promises, release it. CM-A09 says to
extend the existing host/ABI rather than build a second plugin system. CM-A11
says strict must stay fail-closed until a provider is actually verified.
This ADR freezes a **first, deliberately small slice**:
* one opaque host-issued `ResourceHandle` value;
* `resource.open/read/write/close` as pure language builtins, not new `HostOp`
variants;
* byte transfer through `Memory`, never through host pointers;
* stdio `stdout` write and project-file **read** only;
* `Result` maps for EOF/closed/write counts, fatal errors for invalid handle
or capability denial.
File write, directories, network, processes, TLS, asynchronous I/O and a
generic provider registry remain open.
## Decision
### 1. Opaque handle
`Value::ResourceHandle(ResourceHandle)` is a host-issued opaque identity with
private `(run_id, resource_id, generation)` fields and a hidden constructor.
It is not `Serialize`/`Deserialize`, is rejected by the wire, is never
synthesized from a number or a JSON value, and renders as `Resource` in
`display`/`repr` without exposing the id.
It is a **separate domain** from `Memory`: a handle is not indexable, not
byte-addressable and not a capability token the user can manufacture.
### 2. First resource kinds
| Kind | Descriptor | Rights | Availability |
|---|---|---|---|
| `stdio` | `stdout` | `write` | `pure` and `project` when the console grant is present; `strict` rejects until verified |
| `file` | project-relative path | `read` | `project` with an `fs_read` grant |
`file` streams are opened by reading a bounded regular file (existing
`ProjectFs::MAX_READ_BYTES = 8 MiB` limit) into host-owned bytes; `read`
returns successive chunks and `eof` when the position reaches the end. This is
honest about being a first provider: it does not expose a host file descriptor,
does not support seeking, writing, directories, network, processes, devices or
paths outside the project root.
Any other kind/descriptor/right combination is an explicit
`resource.unsupported`/`permission.denied` diagnostic, never a silent fallback.
### 3. Builtins
```text
resource.open(kind: Text, descriptor: Text, rights: List) -> Resource
resource.read(handle: Resource, memory: Memory, offset: Int, max_bytes: Int)
-> {ok:true, value:{count:Int, eof:Bool}} | {ok:false, code, message}
resource.write(handle: Resource, memory: Memory, offset: Int, length: Int)
-> {ok:true, value:{count:Int}} | {ok:false, code, message}
resource.close(handle: Resource)
-> {ok:true, value:None} | {ok:false, code, message}
```
* `resource.open` is effectful and capability-checked at execution time.
* `resource.read` validates `offset` and the remaining buffer **before** asking
the provider, so a failed range check cannot consume stream data. It caps the
request to the remaining region, writes the returned bytes into `Memory` and
returns the actual count and EOF flag. EOF is a **success** with `eof:true`.
* `resource.write` copies bytes out of `Memory` only for the duration of the
call. A short write is reported as a count; the caller loops. It never
stores a host pointer into a region that could later grow.
* `resource.close` closes the host slot. Reading/writing/closing an already
closed handle returns `{ok:false, code:"resource.closed", ...}` (recoverable).
A forged, cross-run or stale handle is `runtime.resource` (fatal).
* All four are pure interpreter operations over host trait methods; they add no
`HostOp` variant. Invalid capacity/bounds remains `runtime.index`; wrong
operand types remain `runtime.type`.
### 4. Authorization and modes
* Authorization is the intersection of the requested operation, the effective
`Capabilities` grant and the mode:
* `stdio/stdout/write` requires `console` (the same grant `打印` uses);
* `file/.../read` requires an `fs_read` pattern match and project-root path
validation (`..`, absolute paths and symlink escapes rejected);
* `pure`: stdio stdout is allowed because console is already a pure-mode
console grant; **file resources are refused by the mode check**, even if a
caller constructs a `Capabilities` set with a matching `fs_read` pattern.
Network/process resources do not exist in this slice.
* `project`: stdio stdout and explicitly granted project-file reads.
* `strict`: only the existing **stdio/stdout** console channel is permitted
(`resource.open("stdio","stdout",["write"])`, console grant); every file,
network, process or device resource open is refused with
`permission.denied`. This is the existing console capability reached through
the resource ABI, not a new OS capability (ADR-0030). Strict L1 is otherwise
unchanged.
* Manifest capability requests are not grants; only the operator-provided
effective set is consulted.
### 5. Errors, budgets and tracing
New stable categories:
| Code | Meaning |
|---|---|
| `runtime.resource` | forged/cross-run/stale handle, invalid resource state |
| `resource.closed` | read/write/close on an already closed handle (recoverable) |
| `resource.unsupported` | kind/descriptor/rights not implemented in this slice |
`resource.write` to stdout charges the existing output budget and appends to
the host output buffer exactly like `打印`; capability denial is
`permission.denied`. Trace records `resource.open`, `resource.read`,
`resource.write` and `resource.close` with kind/count/eof metadata, never raw
file bytes or handles.
### 6. Wire and boundary
* `to_wire(Resource)` emits `{"type":"resource"}` only; decoding it is a hard
error (same rule as `Memory`).
* The generic `Host::invoke` boundary rejects arguments containing a resource
handle, so plugins/foreign kernels/file adapters never receive one.
* Plugin output cannot mint a handle: the constructor is host-only and the
value is not decodable from plugin wire.
### 7. Non-goals
* No `print` migration in this ADR; that is CM-A07 and a later increment.
* File **write** is added by [ADR-0026](0026-buffered-project-file-write-streams.md),
not this decision; this ADR remains read-only for files.
* No file write, directory listing, rename, metadata, network, DNS, TLS,
process, clock, random or device operations.
* No generic third-party provider registration; CM-A09 remains open.
* No zero-copy, async, deadlines per stream, seek or shared memory.
* No strict-mode file/network/process resource support and no claim that
host-in-process I/O is a sandbox; strict stdio is the existing console.
## Compatibility
* Additive value variant; old programs never produce `Resource`.
* New builtins only; existing `fs.read/write`, `打印`, plugin and foreign
behavior is unchanged.
* No wire version bump: the new tag is rejected, not accepted.
* New error categories are additive; existing categories/codes are unchanged.
## Consequences
* `.mq.md` can implement `write_text`-style output by encoding text into
`Memory` (fixture 59) and calling `resource.write`, and can implement
chunked file readers with `resource.read`/`Result`/EOF.
* The remaining high-level I/O surface can grow in the standard library without
a new `HostOp` per function.
* The resource table is host state per run; closed slots are not reused in this
slice, so generation checks are straightforward.
* Strict remains conservative; `.mq.md` resource code runs in `pure` (stdout)
or `project` (file read) only.
## Verification
`docs/evidence/2026-11-20-resource-stream-p3.md` records unit tests for grants,
mode refusal, traversal rejection, forged/closed handles, output budget,
file chunking/EOF, wire rejection and a `.mq.md` stdout fixture.
@@ -0,0 +1,116 @@
# ADR-0026: Buffered project-file write streams (CM-A06 extension)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A06, CM-A09, CM-A11), ADR-0024, ADR-0025
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-resource-file-write-p3.md`
## Context
ADR-0025 froze the first resource ABI slice with `stdio/stdout` write and
project-file **read**. A standard library cannot implement file copy, file
format writers or "write then verify" without a write-capable file resource.
The RFC's P3 target is explicit `open/read/write/close` for project files in
`project` mode; strict must remain fail-closed.
Direct per-write file I/O would expose partial writes and OS error timing to
every caller and would make it easy to leave a half-written project file. This
ADR adds the smallest honest write path: an in-memory buffer per file writer,
flushed through the existing project path policy on `close`.
## Decision
### 1. Provider shape
Add a third stream shape to the host resource table (ADR-0025):
```text
resource.open("file", "<project-relative path>", ["write"]) -> Resource
```
* `project` mode only, with an `fs_write` glob grant matching the normalised
relative path.
* `pure` still refuses file resources by mode check; `strict` still refuses
file resources (only the existing stdio console channel is permitted,
ADR-0030).
* Absolute paths, `..`, symlink escapes and non-regular targets are rejected by
`ProjectFs` exactly like the existing `fs.write`.
* A writer owns an in-memory byte buffer. `resource.write` appends bytes and
returns `{ok:true, value:{count}}`; it does **not** touch the filesystem.
* `resource.close` flushes the buffer through `ProjectFs::write_bytes` and then
marks the handle closed. If the flush fails, the handle stays open with its
buffer intact and the error is reported; the caller may retry or close again.
* If the program exits without closing, no file is written. This is a
deliberate "no flush on drop" rule: a crash or a missing `close` must not
silently publish a partial file.
* `resource.write` is rejected on a file reader; `resource.read` is rejected on
a file writer; stdout remains write-only. Wrong-direction use is
`resource.unsupported`.
* A writer buffer is capped at the existing 8 MiB project read/write ceiling
(`MAX_READ_BYTES`). Exceeding it is `resource.exhausted` and the append is
atomic: the buffer and file are unchanged.
### 2. Rights and modes
| Kind | Descriptor | Rights | Mode | Grant |
|---|---|---|---|---|
| `file` | project-relative path | `["read"]` | project | `fs_read` glob |
| `file` | project-relative path | `["write"]` | project | `fs_write` glob |
| `stdio` | `stdout` | `["write"]` | pure/project | `console` |
Mixed rights (`["read","write"]`) and other kinds/rights are
`resource.unsupported` in this slice. There is no truncate/append/seek flag:
every writer starts empty and replaces the file on flush.
### 3. Errors, budgets and tracing
* `resource.closed` for read/write/close on a closed handle (recoverable).
* `runtime.resource` for forged/cross-run/stale handles (fatal).
* `resource.exhausted` for a buffer over the byte ceiling.
* `permission.denied` for a missing mode/grant or a path policy failure.
* `resource.unsupported` for a direction/rights combination not implemented.
* `resource.write` on a file writer does not charge the program output budget;
the buffer is bounded by the file ceiling. `resource.close` performs one
`ProjectFs::write_bytes` call and records `resource.close` with the byte
count, never the contents.
* No partial file is visible on a failed close because the write is a single
`std::fs::write` to the validated project path (the same atomicity level as
the existing `fs.write`; no temp-file rename is claimed).
### 4. Explicit non-goals
* No append/truncate/seek flags, no random access, no directory or metadata
operations.
* No file write in pure or strict mode.
* No network/process/device/clock/random resources.
* No generic provider registry (CM-A09), no async/nonblocking I/O.
* No claim that `ProjectFs::write_bytes` is crash-atomic; it is a single
validated write, not a transactional replace.
* No CM-A07 `print` migration in this ADR.
## Compatibility
* Additive to ADR-0025: existing `file` read and `stdio` write behavior is
unchanged; `file` write was previously `resource.unsupported`.
* No new `Value` variant, `HostOp`, Markdown node or wire shape.
* Error categories/codes are unchanged; the new `resource.exhausted` diagnostic
reuses the existing stable `resource.exhausted` code family.
## Consequences
* `.mq.md` can now implement file copy and bounded format writers by opening a
read stream and a write stream, transferring bytes through `Memory`, and
closing both. This is C09's core shape without a `copy_file` Rust builtin.
* Buffered close gives deterministic "all or nothing within one write call"
behavior and keeps OS write errors at a point where the caller still has a
Result path.
* Large files require chunked writes; the 8 MiB ceiling is the first-slice
limit, not a permanent design.
## Verification
`docs/evidence/2026-11-20-resource-file-write-p3.md` records host unit tests
(grants, pure/strict refusal, traversal, buffer ceiling, retry-after-failure),
CLI tests for write-then-read and missing grants, and the full workspace gate.
+103
View File
@@ -0,0 +1,103 @@
# ADR-0027: CM-A07 print-migration staging and the Compat-A `.mq.md` stdlib prototype
Status: **Accepted (Compat-A only; Compat-B/C pending)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A07, CM-A10, P3), ADR-0023, ADR-0024, ADR-0025, ADR-0026
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-20-stdlib-print-compat-a.md`
## Context
RFC-CM-0002 §5.4 and CM-A07 require `打印` to stop being a hidden Rust
`Value::display()` path. The migration has three compatibility stages:
* **Compat-A**: a real `.mq.md` print implementation exists and is callable
through explicit imports; the legacy builtin remains the default.
* **Compat-B**: the `print/打印` name routes to the stdlib implementation
(or an alias table maps it), while old behavior is comparison-tested.
* **Compat-C**: the legacy `HostOp::Print` formatting path is removed from the
public builtin implementation.
This ADR freezes the staging policy and implements **Compat-A only**. It does
not change `打印` resolution, does not remove `HostOp::Print`, and does not
claim CM-A07 complete.
## Decision
### 1. Stage policy
* A stage is only complete when its comparison evidence exists:
* A: explicit stdlib functions exist and their output is compared with the
builtin for the covered value set;
* B: name resolution tests prove a single implementation is selected and old
callers still behave;
* C: all old goldens pass with no `HostOp::Print` formatting path.
* Stages are additive; no stage may silently make a second implementation
reachable for the same name.
* `strict` behavior is unchanged by stdlib-only code; a stdlib print still
needs the console/resource grant.
### 2. Compat-A implementation
Add a normal `.mq.md` module package (later moved to the bundled `stdlib/` tree by ADR-0028):
```text
stdlib/format.mq.md # 显示 / 列表显示 / 映射显示 / 表格显示 / 转义文本
stdlib/text.mq.md # utf8编码 (Memory bytes)
stdlib/io.mq.md # 写文本 / 写值 (resource.open stdout + write/close)
stdlib/demo.mq.md # side-by-side comparison with 打印
```
`io.写值(值)` = `format.显示(值) + "\n"` → `text.utf8编码` → `Memory` →
`resource.open("stdio","stdout",["write"])` → `resource.write` → `close`.
This uses only the public core/resource contracts; it is ordinary imported
`.mq.md` code, not a new builtin.
Covered value types: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, `List`,
`Map`, `Table`. Formatting matches the builtin `Value::display()` for the
checked corpus, including Text quoting/escaping inside collections, canonical
map key order and table layout.
**Not covered yet**: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The
language currently has no public observation for the module/name/handle
identity needed to reproduce `Value::display()`/`repr()` for those types.
Adding such observation would be a CM-A04 extension and is out of scope.
### 3. Compatibility rules
* `打印` remains the default builtin in this slice. `io.写值` is only reachable
through an explicit import alias; there is no implicit module, bundled stdlib
resolution or global name shadowing.
* No `HostOp` is added or removed. Both paths write through the console/output
budget; the stdlib path additionally goes through the resource ABI.
* Stdlib modules are ordinary Markdown documents: no special parser, no
embedded Rust formatter, no direct formatting syscall.
* A future Compat-B ADR must define name precedence, bundled module lookup,
stdlib version/digest (CM-A10) and failure behavior; this ADR does not.
### 4. Non-goals
* No `print` alias routing, no removal of `HostOp::Print`.
* No bundled stdlib install/version/digest or offline distribution.
* No formatting of Formula/Code/FunctionRef/PluginHandle.
* No locale-aware, width-aware or float-format changes.
* No change to Markdown rendering, `strict`, or the wire/ABI.
* No claim that the examples directory is the shipped standard library.
## Consequences
* The target call chain (`format.display` → `utf8_encode` → `write_all` →
`resource.write`) is exercised end to end in `.mq.md` and can be compared to
the builtin.
* The missing observation for `Formula/Code/FunctionRef/PluginHandle` is now an
explicit, documented blocker for Compat-B rather than an implicit assumption.
* Because the stdlib is not bundled, ordinary users still get the builtin; the
prototype does not yet reduce the runtime's formatting surface.
## Verification
`docs/evidence/2026-11-20-stdlib-print-compat-a.md` records the side-by-side
demo (`io.写值` vs `打印` for a nested list and a table), the CLI test asserting
byte-identical lines, and the full workspace gate. It also records the covered
and uncovered value types.
@@ -0,0 +1,151 @@
# ADR-0028: Bundled standard library, `std:` imports and digest verification (CM-A10)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A10, CM-A07, P2/P3), ADR-0027
Specs: `spec/analysis-v2.md`, `spec/references.md`
Evidence: `docs/evidence/2026-11-20-bundled-stdlib-cm-a10.md`
## Context
CM-A07 Compat-A put a real `.mq.md` print chain under `examples/stdlib/`, but it
was reachable only by relative path from a demo in the same directory. A
standard library that ordinary projects can use offline needs:
* a fixed bundle shipped with the CLI;
* an explicit import spelling that cannot be shadowed by a project file;
* a content identity so a different bundle with the same version cannot
masquerade;
* no network, Git or package installation required at run time.
The RFC requires the official distribution to carry a tested stdlib version
offline, while allowing independent releases. This ADR freezes the **bundled
first slice**. A separate `marqdo-stdlib` repository is intentionally deferred
by ADR-0039: the bundle stays in this repository while the architecture is
small and clear; the deterministic package remains extraction-ready.
## Decision
### 1. Bundle location
The official bundle is the `stdlib/` tree shipped next to the CLI (or vendored
at the repository root during development). Discovery order:
1. `MARQDO_STDLIB_DIR` — explicit override; if set and invalid, loading fails
hard (no silent fallback);
2. `<executable-dir>/stdlib` if it exists;
3. the development tree `<repo>/stdlib` for source builds.
If no bundle is found, a project that uses `std:` fails with a clear error; a
project that does not use `std:` is unaffected. No network, Git or package
manager is consulted for a bundled stdlib.
### 2. Explicit `std:` imports only
Front-matter imports may name a stdlib module as:
```yaml
import:
io: "std:io"
fmt: "std:format"
parser: "std:data/json"
```
The target after `std:` is a **package-relative** path inside the bundle.
`..`, absolute paths and `std:` targets that escape the bundle are rejected;
`.mq.md` is appended when the target has no extension. `std:` is resolved
before relative/alias imports and never searches the project or dependencies.
There is no implicit `std` namespace, no global name, and no auto-import.
Module ids use a stable package instance:
```text
std:marqdo-stdlib@<version>#<sha256-hex>
```
so the same logical module from a different bundle is a different instance.
### 3. Content digest
`stdlib/stdlib.json` carries the name, version and digest. The digest is:
```text
sha256( for each sorted *.mq.md relative path:
path_utf8, NUL, file_bytes, NUL )
```
It excludes `stdlib.json`, README files and non-`.mq.md` files. The loader
recomputes it before loading any stdlib module and fails closed on mismatch or
a missing manifest. A bundle is therefore identified by **name + version +
exact markdown bytes**, not by a path or a version string alone.
### 4. Visibility and analysis
Analysis v2 records the verified bundle identity under `project.stdlib`:
```json
{"version":"0.1.0","digest":"sha256:<hex>",
"instance_id":"std:marqdo-stdlib@0.1.0#<hex>"}
```
It is `null` when no bundle was available. A project that imports `std:` always
has a non-null identity. Trace/diagnostics keep the real stdlib source spans;
the bundle is normal Markdown, not a special-cased module type.
### 5. Frozen-CLI closure lane
The first closure test builds the CLI once, hashes the executable, runs a
project importing `std:io`, then re-hashes the executable and asserts equality.
It also asserts:
* the project uses only `.mq.md` and standard imports, no `foreign`/`plugin`
calls;
* a tampered bundle fails the digest check before any module executes;
* Analysis records the verified digest.
This is a local gate, not a replacement for a remote CI lane.
### 6. Versioning
The bundle version is `0.1.0` for this slice. A CLI may load a bundle whose
version is declared compatible by this ADR; the RFC's full SemVer/digest
contract and independent release workflow remain CM-A10 follow-up. A future
core feature that the bundle requires must be reflected in both the bundle
version and the CLI's compatibility table before release.
## Non-goals
* Creating/publishing a separate `cflmy/marqdo-stdlib` repository now; see
ADR-0039 for the deliberate deferral and future extraction triggers.
* Automatic download/update, Git fetching, or manifest-driven stdlib
installation.
* A second plugin/provider system or a second module search path.
* Compat-B/C name routing of `打印` (ADR-0027) — the bundle does not change
builtin resolution.
* Windows/macOS packaging layout; only the source-tree/executable-adjacent
layout is exercised locally.
## Offline packaging
`tools/pack-stdlib.py` produces a deterministic `marqdo-stdlib-<version>.tar.gz`
plus a schema-tagged descriptor (`tree_digest`, `archive_sha256`, file list) for
the future separate repository/release workflow. The archive is not loaded by
the CLI; the in-repo `stdlib/` tree remains the verified development/bundled
source. Publishing/signing the external repository is still an external step.
## Consequences
* Any project can `import: {io: "std:io"}` and use the print prototype
offline; `打印` remains the default builtin.
* A changed stdlib file without a digest update is rejected before execution.
* The bundle is still an in-repo vendored package. Independent repository
release and CI signing are explicitly tracked as remaining CM-A10 work.
* Analysis artifacts and a frozen binary can be tied to the exact stdlib
digest, which is the prerequisite for Compat-B and reproducible releases.
## Verification
`docs/evidence/2026-11-20-bundled-stdlib-cm-a10.md` records the `std:` import
test, digest-tamper rejection, Analysis identity assertion, no-foreign policy
scan and frozen binary hash equality.
@@ -0,0 +1,86 @@
# ADR-0029: Full-value stdlib formatting and handle observation (CM-A04/A07)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A04, CM-A07, P2/P3), ADR-0023, ADR-0027, ADR-0028
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-20-full-value-formatting.md`
## Context
Compat-A (ADR-0027) formats Unit/None/Bool/Int/Float/Text/List/Map/Table, but
builtin `打印` also formats Formula, Code, FunctionRef and PluginHandle. A
Compat-B name route cannot be honest while the stdlib silently differs on
those types. The missing pieces are the **identity/observation** fields needed
to reproduce `Value::repr()` for function references and plugin handles;
Formula and Code already expose `latex/display/unit` and
`language/source/content_hash/unit`.
## Decision
### 1. Read-only identity members
Add member reads (no new value types, no mutation, no handles):
| Value | Members |
|---|---|
| `FunctionRef` | `module: Text`, `name: Text` |
| `PluginHandle` | `plugin: Text`, `export: Text` |
The host-assigned numeric `PluginHandle.id` remains private; only the manifest
plugin name and export name are exposed, which are already visible in
`Value::repr()` and are not a capability token. `Memory` and `Resource` remain
opaque and expose no fields. Unknown members are `runtime.unknown_field`.
### 2. Stdlib formatting covers all 15 value types
`stdlib/format.mq.md` gains branches for:
* `Formula`: `Formula(<debug-quoted latex>)`;
* `Code`: `Code(<debug-quoted language>, <utf8-byte-length> bytes)`
(the byte length is computed with `stdlib/text.mq.md`, not `长度`, which is
Unicode scalar count);
* `FunctionRef`: `FunctionRef(module::name)`;
* `PluginHandle`: `PluginHandle(plugin::export)`;
* `Memory`/`Resource`: the literal `Memory`/`Resource`;
* `Unit`: `Unit`.
The existing `Text`/collection escaping (quotes, backslash, newline, tab, CR)
is reused. Control characters outside those escapes are not emitted as
`\u00XX` in this slice; that is a formatting limitation, not a security
boundary, and does not affect the current corpus.
### 3. Compatibility
* The new members are additive observations; no existing expression changes.
* `PluginHandle.id` is not observable and cannot be used to forge a handle.
* `打印` remains the builtin default; this ADR only makes the stdlib formatter
capable of matching it for all value types. Compat-B routing is a separate
decision (ADR-0030).
* Formula/Code provenance spans are not exposed as members; only the fields
already present in the value wire/repr are.
## Non-goals
* No mutable handle inspection, no resource/plugin capability escalation.
* No `\u00XX` escaping of arbitrary control characters.
* No change to `Value::display()`/`repr()` themselves; the stdlib must match
the existing behavior, not redefine it.
* No print name routing in this ADR.
## Consequences
* The stdlib formatter can reproduce `打印` output for all 15 current value
types, removing the Compat-B blocker documented in ADR-0027.
* Function/plugin identity becomes readable in `.mq.md`; this is already public
in display/repr and does not reveal the host-only numeric handle id.
* Code byte length performs one UTF-8 encode into a temporary `Memory`; large
code values cost an extra allocation during formatting. A future core
observation could avoid it without changing output.
## Verification
`docs/evidence/2026-11-20-full-value-formatting.md` records the member-read
tests, a Formula/Code/FunctionRef formatting comparison against builtin
`打印`, and the full workspace gate.
+101
View File
@@ -0,0 +1,101 @@
# ADR-0030: Compat-B `打印` routing through the bundled stdlib
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A07, CM-A11, P3), ADR-0027, ADR-0028, ADR-0029
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-print-compat-b.md`
## Context
ADR-0027 implemented Compat-A: the stdlib print chain exists behind explicit
imports, but builtin `打印` still runs `Value::display()` in Rust. ADR-0029
removed the formatting gap: the stdlib can reproduce `打印` for all 15 value
types. ADR-0028 made a verified stdlib bundle available offline. The remaining
Compat-B requirement is to route the **same name** to one implementation and
prove the old callers behave, before Compat-C removes the legacy formatting
path.
## Decision
### 1. One program-level override
`Program` gains an optional `print_override: FunctionId`. The loader sets it to
the bundled `io.写值` function when:
* the verified bundle is available, and
* at least one loaded module calls `打印` or `print` as a bare builtin call.
The stdlib `io` module is injected only in that case. A program without print
calls keeps the same module graph and Analysis output as before. If no bundle
is available, the loader leaves the override unset and the legacy builtin is the
fallback; an official release always ships the bundle.
### 2. Resolution order
At a call site, name resolution is:
1. a local variable holding a callable (`f = ...; f(x)`) — local shadow wins;
2. `打印`/`print` with `Program.print_override` set — call the stdlib function;
3. the existing builtin table;
4. a same-module/imported function declaration.
This is a controlled compatibility route for exactly two names. It is not a
general name-shadowing mechanism and does not add a new namespace.
### 3. Runtime behavior
The stdlib function is a normal Marqdo function: its source spans, fuel,
frames and trace are the stdlib file's. It formats through
`stdlib/format.mq.md`, encodes UTF-8 through `stdlib/text.mq.md`, and writes
through `resource.open("stdio","stdout",["write"])` plus `resource.write`.
`打印` still returns `Unit` and appends exactly one newline.
The console/output-budget checks remain in the host: the stdlib path uses the
same console grant and output budget as the legacy path. Resource operations
are recorded as Trace `effect` events (`resource.open/read/write/close`); the
legacy `HostOp::Print` effect does not appear in Compat-B runs.
### 4. Strict mode and CM-A11
Strict L1 already permits the console channel. Compat-B needs stdout writes in
strict so the same print implementation works there. Therefore strict mode
permits **only** `stdio/stdout` write with the console grant; project-file,
network, process and device resource opens remain refused, and no new provider
is enabled. This is the existing console capability reached through the
resource ABI, not a new OS capability. File resources stay fail-closed.
### 5. Compatibility and fallback
* Output for all 15 value types is the ADR-0029 formatter and is
comparison-tested against the legacy path.
* Trace shape changes from one `print` effect to resource effects. Consumers
that hard-code `operation == "print"` must accept `resource.*`; this is a
documented P3 migration.
* `check`/`analysis` include stdlib modules for print-using programs only.
The human-readable `checked N module(s)` count excludes stdlib modules so
bundle internals do not change the project count.
* Compat-C (removing `HostOp::Print` and `Value::display()` formatting from the
public builtin) is **not** part of this ADR; it is implemented by ADR-0031,
which supersedes the "legacy fallback" sentence above.
## Non-goals
* No routing for `长度`, `类型`, `文本`, table or any other builtin.
* No bundled stdlib upgrade workflow; the bundle version stays `0.1.0`.
* No removal of `HostOp::Print`, `Value::display()` or the legacy builtin.
* No new OS isolation claim; strict still refuses file/network/process
resources.
* No guarantee for a CLI installed without `stdlib/`; that configuration keeps
legacy print and is not the official layout.
## Verification
`docs/evidence/2026-11-20-print-compat-b.md` records:
* a print program whose trace contains `resource.write` and no `print` effect;
* all semantic goldens and CLI print tests passing byte-identically;
* lazy stdlib injection: no print → no stdlib modules; print → stdlib modules
and `project.stdlib` identity;
* host tests for strict stdio-only permission and file refusal.
@@ -0,0 +1,95 @@
# ADR-0031: Compat-C — remove `HostOp::Print` and the legacy formatting path
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A07, P3), ADR-0027, ADR-0029, ADR-0030
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-print-compat-c.md`
## Context
Compat-A produced a stdlib print chain; Compat-B (ADR-0030) routed
`打印/print` to the bundled `io.写值` and proved output parity for all 15 value
types. The legacy path (`HostOp::Print` → `Value::display()` → output buffer)
remained as a no-bundle fallback and a comparison reference. Compat-C removes
the **public builtin formatting path** so a print call can no longer silently
fall back to a Rust formatter.
## Decision
### 1. Remove the host operation
* `HostOp::Print` and `StdHost::op_print` are deleted. There is no host
operation that formats a value.
* `打印/print` remains a **builtin name** for static arity validation and
compatibility diagnostics. Its implementation no longer writes anything; if
the runtime reaches it (no `Program.print_override`), it returns the
structured `kernel.unavailable` error:
"打印 requires the bundled stdlib print module; install the official CLI
bundle."
* The official CLI ships the verified bundle, so a print-using program always
loads `stdlib/io.mq.md`, sets `print_override`, and executes `io.写值`.
A CLI installed without `stdlib/` cannot print; that layout is not official.
### 2. Formatting ownership
All print formatting, UTF-8 encoding and byte output now live in `.mq.md`:
```text
打印(value)
-> stdlib/io.写值
-> stdlib/format.显示
-> stdlib/text.utf8编码
-> resource.open("stdio","stdout",["write"])
-> resource.write
-> resource.close
```
`Value::display()` still exists for CLI result rendering (`call --json`,
diagnostics), but it is **not** reachable from the print builtin. No `HostOp`
accepts a value for display.
### 3. Trace and budgets
Print produces only resource effects (`resource.open/write/close`) and stdlib
semantic events. There is no `print` effect. Output budget and console grant
are enforced at the resource/stdio boundary exactly as before.
### 4. Compatibility
* All semantic goldens, CLI print tests and the full-value formatting demo pass
unchanged.
* The runtime embedding API (`marqdo_runtime::run`) no longer prints unless the
caller installs a `Program.print_override`; a direct runtime program that
calls `打印` gets the structured no-bundle error. This is a **breaking change
for embedders** and is recorded here; the CLI loader maintains the official
behavior.
* `callable.rs` still advertises the builtin name for diagnostics and arity;
this is not a second implementation.
* Compat-A/B evidence remains as history; its "legacy fallback" wording is
superseded by this ADR.
### 5. Non-goals
* Removing the `打印/print` builtin name, alias, or arity validation.
* Removing `Value::display()` for CLI result output, diagnostics or format
helpers.
* Supporting print without the bundle; no implicit formatting fallback.
* Changing Resource ABI, strict mode or stdlib version.
## Consequences
* It is now impossible for `打印` to execute a hidden Rust formatter; the only
print implementation on an official installation is `.mq.md`.
* Embedders that build a `Program` directly must load the stdlib print module
and set `print_override` (the semantic test harness does this).
* Trace consumers see resource effects, not a print effect.
* A missing/corrupt bundle fails closed for print instead of silently using a
different implementation.
## Verification
`docs/evidence/2026-11-20-print-compat-c.md` records: no `HostOp::Print` or
`op_print` references; the no-bundle structured error; trace still shows
resource effects; all 419 tests and the full-value demo pass.
@@ -0,0 +1,101 @@
# ADR-0032: GFM Table logical value, stdlib algorithms and T0 storage (CM-A08)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A08, P2/P4), ADR-0023
Specs: `spec/evaluation.md`, `spec/mapping.md`
Evidence: `docs/evidence/2026-11-20-table-cm-a08.md`
## Context
RFC-CM §6 separates three layers: the GFM source document (canonical Markdown
AST), the logical `Table` value, and the physical storage. The current runtime
uses `Value::Table { header: Vec<String>, rows: Vec<Vec<Value>> }`, which is
correct but concrete. Table algorithms already exist as `.mq.md` conformance
fixtures (filter/sort/join/group), but they are not shipped as a reusable
stdlib module. CM-A08 requires that high-level table algorithms live in the
standard library and that physical storage can change without changing
observable semantics.
## Decision
### 1. Three layers stay separate
* **GFM source**: unchanged. The Markdown parser owns `table/row/cell`,
spans, laziness and rendering. This ADR adds no table execution to Markdown.
* **Logical value**: `Value::Table` keeps ordered headers, ordered rows, cell
values, width validation, equality/display/wire semantics and value-copy
behavior (`b = a` does not alias).
* **Physical storage**: T0 remains `Vec<Vec<Value>>`. This is an implementation
detail; a future T1 controlled interface or T2 column store must preserve the
logical contract. No column store, view or Memory-backed table is introduced
here.
### 2. Standard-library table module
Add `stdlib/data/table.mq.md`, importable as `std:data/table`:
| Function | Semantics |
|---|---|
| `选择列(数据, 列)` | new table with one column, original unchanged |
| `过滤数值大于(数据, 列, 阈值)` | rows whose numeric cell is `>` threshold |
| `排序(数据, 列)` | insertion sort by a numeric column; new table |
| `插入有序(已排序行, 行, 列)` | helper; value-returning |
| `追加行(数据, 行)` | new table with one appended row (width validated by `表`) |
| `设置单元格(数据, 行, 列, 值)` | rebuilds rows; original unchanged |
| `连接(左, 右, 左列, 右列)` | equality join; headers concatenated |
| `分组(数据, 键列, 值列)` | Text key -> List of values |
| `转GFM(数据)` | render the logical table as a GFM pipe table; `\|`/newline cells escaped |
All functions use only `table.columns/row/cell`, `list.*`, `map.*` and `表`;
none is a new builtin or `HostOp`. The module is ordinary `.mq.md`, loaded
through the verified `std:` bundle and its digest.
### 3. Semantics fixed by this ADR
* Every operation returns a **new** table; inputs are never mutated.
* Row/column coordinates are non-negative; out-of-range is `runtime.index`.
* Numeric algorithms use `Int`/`Float` comparison of cell values; passing a
Text cell where a number is required is a runtime type error, not a silent
coercion.
* GFM cell content such as `**不执行**` or backticked code remains inert data;
no table function parses a cell as executable source.
* Empty tables, zero rows/columns, duplicate headers and Unicode headers keep
their existing behavior; `表` still validates row width.
* Table source spans and document addresses are unchanged; a rebuilt Table is
a logical value, not a rewrite of the source document.
* Cross-storage compatibility: a `.mq.md` columnar representation (List/Map of
columns) must produce the same headers/rows/cells as the T0 row-store Table
for the supported operations (fixture 72). This does not migrate the Rust
storage; it proves the logical contract is storage-independent.
### 4. What is deliberately not done
* No physical storage migration (T1/T2/T3), column store, shared view or
Memory-backed table.
* No `DataFrame`/`TableView` parallel types; `Value::Table` remains the only
table value.
* No per-operation Rust builtin (`table.filter`, `table.join`, ...).
* No mutable table update; `设置单元格` rebuilds.
* No sorting/grouping of heterogeneous mixed-type columns beyond the explicit
numeric/Text contracts.
* No change to GFM parsing or rendering.
## Consequences
* A project can `import: {table: "std:data/table"}` and filter/sort/join/group
tables without new Rust code; this is the P4 migration's first concrete
stdlib artifact.
* The physical representation remains replaceable because algorithms depend
only on the logical observation/construction API.
* Large-table performance is O(n²) for insertion sort/join and copies rows;
P6 performance work (column store, persistent vectors) can change storage
without changing these outputs.
* The bundle digest must be regenerated when this module changes.
## Verification
`docs/evidence/2026-11-20-table-cm-a08.md` records the module self-test and a
CLI project importing `std:data/table` that checks filter/select/append/set/
join/group results and input immutability, plus the full workspace gate.
@@ -0,0 +1,123 @@
# ADR-0033: In-process ResourceProvider v1 (CM-A09 first slice)
Status: **Accepted (in-process slice only)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A11, P3/P5), ADR-0025, ADR-0026, ADR-0031
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-provider-abi-cm-a09.md`
## Context
ADR-0025/0026 hard-code two resource kinds (`stdio/stdout`, project `file`) in
the host. CM-A09 requires a versioned, discoverable provider contract so a new
device/OS endpoint can be added **without** changing the expression parser,
`Value` model, runtime dispatch or `HostOp` enum. The RFC warns against a
second shadow plugin ecosystem: providers should reuse the same host/ABI
infrastructure where possible.
The full C ABI/Worker resource protocol is a larger artifact (C structs, wire
buffers, ownership, process isolation). This ADR freezes the **in-process
trusted provider** slice first; a Worker/C ABI binding can implement the same
contract later. In-process providers are explicitly **not** a sandbox.
## Decision
### 1. Provider contract v1
```rust
pub trait ResourceProvider: Send {
fn name(&self) -> &str; // stable grant name
fn protocol_version(&self) -> u32 { 1 } // must be 1 in this slice
fn kinds(&self) -> &[&str]; // resource kinds it serves
fn open(&mut self, kind: &str, descriptor: &str, rights: &[String],
span: Option<Span>) -> Result<u64, RuntimeError>; // provider token
fn read(&mut self, token: u64, max_bytes: usize,
span: Option<Span>) -> Result<ResourceRead, RuntimeError>;
fn write(&mut self, token: u64, bytes: &[u8],
span: Option<Span>) -> Result<usize, RuntimeError>;
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError>;
}
```
* `ResourceRead { bytes, eof }` is the same host type used by built-in streams.
* The **host** still issues the opaque language `ResourceHandle`; the provider
token is private to the provider and never enters a `Value`, wire message or
trace.
* Bytes cross the provider boundary as copied slices for the duration of a
call; no provider may retain a `Memory` pointer across calls or after a
`grow`.
* `open` returns a provider token; `read/write/close` must reject unknown,
stale or closed tokens with a structured error. The host checks the language
handle before dispatching.
### 2. Registration
`StdHost::register_resource_provider(Box<dyn ResourceProvider>)`:
* rejects `protocol_version != 1`;
* rejects an empty `kinds()` list;
* rejects a provider name or kind that is empty, `stdio`, `file`, or already
registered (no silent shadowing);
* stores the provider under each declared kind and its stable name for grants.
Registration is a host/embedding API, not a language import. Importing a
`.mq.md` package never registers or authorizes a provider. Installation and
authorization remain separate (INV-06).
### 3. Authorization and modes
* `Capabilities` gains `resource_providers: Vec<String>` and
`allow_resource_provider(name)`; a matching name is required at
`resource.open` time.
* `project` only. `pure` and `strict` refuse provider kinds even when
registered; strict remains fail-closed (CM-A11). Existing stdio/file rules
are unchanged.
* A provider's own rights/descriptor validation is not a substitute for the
grant; denial is `permission.denied` with a diagnostic.
* In-process provider code shares the host address space and is **not**
sandboxed. Only explicitly trusted providers may be registered; the API
documentation and ADR state this.
### 4. Runtime surface
No new `Value`, `HostOp`, Markdown node or grammar. A language program still
writes:
```text
resource.open("<kind>", "<descriptor>", ["read"])
```
The existing `resource.read/write/close` and `Result` shapes are reused. The
provider name is used only for the grant check and trace metadata; the language
does not need to know it beyond the grant.
### 5. What remains for CM-A09
* C ABI / Worker resource transport (buffer handoff, ownership, crash isolation,
version negotiation through the existing plugin ABI).
* Provider discovery/installation from packages, signatures and platform
support matrices.
* Network/socket/device providers themselves.
* Strict-mode provider verification.
This ADR is a contract for trusted in-process providers, not a second plugin
ecosystem.
## Consequences
* A new endpoint can be implemented as a `ResourceProvider` and registered
without touching parser/runtime/`HostOp`; language code uses the same
`resource.*` calls.
* The provider token cannot be forged into a language handle; only the host
creates handles.
* In-process providers can crash/block the host; they are trusted. Worker C ABI
is still needed for untrusted extensions.
* `Capabilities` grows one grant list; CLI exposure is deferred because there
is no language-level provider installer yet.
## Verification
`docs/evidence/2026-11-20-provider-abi-cm-a09.md` records a mock read/write
provider registered without parser/runtime changes, grant/pure/strict denial,
protocol/kind conflict registration errors and the full workspace gate.
+87
View File
@@ -0,0 +1,87 @@
# ADR-0034: Blocking TCP resource endpoint v1 (CM-A09/P5 first slice)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A11, P5), ADR-0025, ADR-0033
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-tcp-endpoint.md`
## Context
CM-A09/P5 require a real external endpoint reached through the stable resource
ABI, not a new `HostOp` or Rust builtin per application. The in-process
provider slice (ADR-0033) covers trusted adapters; it does not exercise a real
OS byte stream. This ADR adds the first OS endpoint: a blocking TCP stream.
The scope is deliberately narrow: HTTP/1.1, TLS, DNS, UDP, sockets as a broad
API and nonblocking/async I/O remain standard-library or later work.
## Decision
### 1. Resource kind and descriptor
```text
resource.open("tcp", "<ip-literal>:<port>", ["read", "write"]) -> Resource
```
* descriptor must parse as a `SocketAddr` containing an **IP literal**. DNS
names are rejected with `resource.unsupported`; no resolver is called.
* rights must be exactly `["read","write"]` in this build. A read-only or
write-only TCP handle is future work.
* `resource.read`/`write`/`close` behave like other streams:
* read returns `{count, eof}`; `count=0` is transport EOF;
* write returns a possibly short count;
* close calls `shutdown(Both)` and marks the handle closed;
* closed/forged/stale handles use the existing `resource.closed` /
`runtime.resource` rules.
* connect, read and write have 5-second timeouts. This is blocking I/O on the
interpreter thread; no async runtime is introduced.
### 2. Authorization
* `project` mode only. `pure` and `strict` refuse TCP (`permission.denied`);
strict remains fail-closed (CM-A11), and the G-C lane verifies the refusal.
* `Capabilities.check_net(ip)` requires an exact IP-literal grant. The CLI adds
`--allow-net <IP>` (repeatable); `--allow-net` is refused alongside other
external grants in pure/strict preflight.
* Granting a hostname or a network range does not authorize a different IP;
there is no DNS-based allowlist bypass because DNS is not used.
* No TLS, no proxy, no HTTP parsing in the host. `.mq.md` code composes
protocols over the byte stream.
### 3. Trace
`resource.open/read/write/close` effect events are recorded. The trace records
the IP:port for open but never payload bytes or credentials. A TCP endpoint is
an external effect and is not replayed as pure computation.
### 4. Explicit non-goals
* DNS, TLS, HTTP/1.1 or HTTP/2 host implementations, proxies, redirects.
* UDP, ICMP, raw sockets, socket options, nonblocking/async/select.
* File descriptors exposed to the language, `setsockopt`, `bind`/`listen` server
sockets.
* Strict-mode TCP and any claim that in-process `std::net` is OS isolation.
* Network provider package installation or a generic socket registry.
## Consequences
* `.mq.md` can implement a line protocol/HTTP client parser on top of the TCP
byte stream using the existing Memory/Text/Result primitives. The bundled
`stdlib/net/http.mq.md` provides HTTP/1.1 response parsing (status, headers,
chunked transfer) and request construction; `http_over_tcp.rs` runs a local
HTTP exchange through `resource.open("tcp", ...)` and the module.
* The runtime still has no `net.request` builtin or new `HostOp`; a CLI test
opens a local echo server, writes/reads `ping` through `resource.*`, and
closes.
* Blocking timeouts bound a hung peer but do not preempt a blocking syscall
deeper than the OS timeout; this is documented.
* Network tests use loopback only; external network access is not required for
the gate.
## Verification
`docs/evidence/2026-11-20-tcp-endpoint.md` records host tests (loopback echo,
grant/pure/strict/DNS refusal) and a CLI `.mq.md` echo test with
`--allow-net 127.0.0.1`, plus the full workspace gate.
@@ -0,0 +1,92 @@
# ADR-0035: Worker resource-provider transport v1
Status: **Accepted (trusted helper transport; C ABI binding still open)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A11, P5), ADR-0033, ADR-0034
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-worker-provider.md`
## Context
ADR-0033 provides an in-process `ResourceProvider` v1. CM-A09 also asks for a
process boundary so an adapter can crash or hang without taking down the
interpreter, and for a transport that can later be implemented by the C/Worker
plugin ecosystem. This ADR adds the first process-boundary transport: a
host-managed helper process speaking framed JSON.
It deliberately does **not** claim the C ABI or OS sandbox: the helper runs as
the host user and is trusted. The existing plugin Worker transport remains the
untrusted-plugin path; this is a resource-provider transport for explicitly
trusted adapters.
## Decision
### 1. Framing and protocol v1
* 4-byte big-endian length + JSON, maximum 1 MiB per frame.
* A `ping` handshake must succeed before the provider is registered.
* Operations:
```text
{"op":"open","kind","descriptor","rights"} -> {"ok":true,"token":u64}
{"op":"read","token","max_bytes"} -> {"ok":true,"bytes":[u8...],"eof":bool}
{"op":"write","token","bytes":[u8...]} -> {"ok":true,"count":u64}
{"op":"close","token"} -> {"ok":true}
{"op":"shutdown"} -> process exit
{"ok":false,"code","message"} -> structured failure
```
* Provider tokens stay in the helper; the host still issues the opaque language
`ResourceHandle`. Bytes are copied per call.
* The host maps helper error codes to the existing categories
(`permission.denied`, `resource.closed`, `resource.exhausted`,
`resource.unsupported`, otherwise `host.error`) and attaches the callsite
span.
* The provider is killed on drop; a protocol/IO failure is `host.error`, never a
fallback to another transport.
### 2. Registration and authorization
* `StdHost::register_worker_resource_provider(name, kinds, program, args)` is a
host embedding API. It validates name/kinds with the same rules as ADR-0033
(non-empty, not `stdio`/`file`, no duplicate kind), then registers the
provider under its kinds.
* `resource.open` still requires the provider name grant and project mode.
Pure/strict refuse provider kinds; strict remains fail-closed.
* Language code cannot spawn a helper; registration is not reachable from
`.mq.md` or a package import.
### 3. Security boundary
* The helper runs as the host user with full host privileges. It is **trusted**;
this transport is not a sandbox and does not make arbitrary native code safe.
* The project-mode + name-grant checks are the same as ADR-0033. The helper's
own descriptor/rights validation is additional, not a substitute.
* No network, filesystem or process rights are delegated implicitly by
registration; the helper is responsible for its own behavior, and the host
grant only authorizes the resource kind.
### 4. What remains for CM-A09
* A C ABI / plugin-Worker binding for `ResourceProvider` (buffer handoff,
ownership, version negotiation through the plugin ABI).
* Provider package installation, signatures and discovery.
* Untrusted-provider isolation and strict-mode provider support.
* The transport is versioned (`protocol_version() == 1`); future versions need a
new ADR and negotiation.
## Consequences
* A trusted adapter can live in its own process and be reached through the same
`resource.*`/`Result` language surface without touching the parser, `Value`,
runtime dispatch or `HostOp`.
* A helper crash/exit is a structured host error; the interpreter never falls
back to a different provider.
* The helper protocol is small enough to reimplement in C/Worker adapters later.
## Verification
`docs/evidence/2026-11-20-worker-provider.md` records a Python helper exercising
ping/open/read/write/close, grant/pure/strict denial, missing-program error and
the full workspace gate.
+79
View File
@@ -0,0 +1,79 @@
# ADR-0036: C ABI ResourceProvider v1
Status: **Accepted (trusted native adapter; plugin-Worker binding still open)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A11, P5), ADR-0033, ADR-0035
Header: `crates/marqdo-host/include/marqdo_resource_provider.h`
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-c-provider.md`
## Context
ADR-0033 provides an in-process provider trait and ADR-0035 a host-managed
helper process with framed JSON. CM-A09 also needs a **C ABI** so native
adapters can expose resource endpoints without linking Rust or changing the
runtime. This ADR freezes the first C ABI version for trusted native adapters.
## Decision
### 1. Version and symbols
```c
#define MARQDO_RESOURCE_PROVIDER_ABI_VERSION 1u
uint32_t marqdo_resource_provider_abi_version(void);
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
const char *rights_csv, uint64_t *out_token);
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
uint32_t capacity, uint32_t *out_read,
int32_t *out_eof);
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
uint32_t length, uint32_t *out_written);
int32_t marqdo_resource_close(uint64_t token);
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity);
```
* The host loads the library, requires ABI version 1 and resolves the symbols
at registration; missing symbols/version mismatch fail closed with
`resource.unsupported`/`kernel.unavailable`.
* Return `0` on success, nonzero on failure; failure text comes from
`last_error` and is surfaced as a structured host error with the callsite
span.
* `rights_csv` is a comma-separated list. Provider tokens are private to the
adapter; the host still issues the language-visible `ResourceHandle`.
* Buffers are borrowed for the call only; the adapter must not retain host
pointers. The host caps frame/buffer sizes by the existing resource limits.
### 2. Registration and authorization
* `StdHost::register_c_resource_provider(name, kinds, library_path)` is a host
embedding API; it uses the same name/kind validation as ADR-0033 and the same
provider-name grant at `resource.open`.
* Project mode only; pure/strict refuse provider kinds. Strict stays
fail-closed (CM-A11).
* The C library runs in the host address space with host privileges. It is
**trusted**, not sandboxed; the untrusted-plugin path remains the separate
plugin Worker transport. Registration is not reachable from `.mq.md`.
### 3. Remaining CM-A09 work
* Plugin-Worker binding for `ResourceProvider` through the existing plugin ABI
(buffer handoff/ownership/version negotiation).
* Provider package installation, signatures and discovery.
* Untrusted-provider isolation and strict-mode provider support.
* ABI version negotiation beyond rejecting non-v1 libraries (future ADR).
## Consequences
* Native adapters can expose new resource kinds through `resource.*` without
parser/`Value`/runtime/`HostOp` changes.
* ABI mistakes are fail-closed at registration or surfaced as host errors; no
fallback transport is attempted.
* The header is small and stable enough for a future SDK/package workflow.
## Verification
`docs/evidence/2026-11-20-c-provider.md` records a compiled C test provider
(`cc -shared`), version/load checks, open/read/write/close round-trip, missing
library fail-closed and the full workspace gate.
@@ -0,0 +1,77 @@
# ADR-0037: Provider package manifest v1
Status: **Accepted (host-side installation metadata)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A10, P5), ADR-0036
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-provider-package.md`
## Context
ADR-0036 lets a trusted C shared library implement resource operations, but the
host still needed a code-level path to the library. CM-A09 asks for provider
installation/discovery without changing the parser/runtime. This ADR defines a
small, versioned **package manifest** so a provider distribution can be
installed by a host embedding and reused by future external provider
repositories.
## Decision
### 1. Manifest format
```json
{
"schema": "marqdo.provider",
"schema_version": 1,
"name": "cabi",
"kinds": ["cabi.sensor"],
"abi": "c-abi-1",
"library": {
"linux": "libcabi_provider.so",
"macos": "libcabi_provider.dylib",
"default": "libcabi_provider.so"
}
}
```
* file name: `provider.json` by convention; the host API takes an explicit
manifest path.
* `schema`/`schema_version` must be exactly `marqdo.provider`/`1`.
* `abi` must be `c-abi-1` (ADR-0036); future ABIs need a new ADR.
* library paths are relative to the manifest directory (absolute paths are
accepted only for explicit local tests); platform key is `linux`, `macos`,
`windows`, or `default`.
* unknown fields are tolerated by serde, but unknown schema/ABI are rejected.
### 2. Host API and authorization
* `StdHost::register_provider_package(manifest_path)` reads/validates the
manifest, loads the C ABI library and registers it through the same
kind/name validation as ADR-0033/0036.
* Registration does not authorize language use: `resource.open` still requires
project mode and `Capabilities.allow_resource_provider(name)`. Pure/strict
refuse provider kinds.
* Language imports cannot read or install a provider manifest. Installation is
a host/package-manager action (INV-06).
### 3. Trust boundary
* The manifest points at trusted native code loaded in-process; it is not a
sandbox and does not add signatures. A future provider repository must add
signatures/verification and a Worker binding.
* No network/package registry fetch is implemented here; the host passes a
local path.
## Non-goals
* Provider registry download, signatures, dependency resolution or semver.
* CLI install/uninstall commands (host embedding only).
* Untrusted-provider isolation or strict-mode provider support.
* Automatic platform selection beyond the `library` map.
## Verification
`docs/evidence/2026-11-20-provider-package.md` records the manifest round-trip
test: compile a C provider, write `provider.json`, `register_provider_package`,
then `resource.open/read`.
@@ -0,0 +1,68 @@
# ADR-0038: C ABI provider worker binding v1
Status: **Accepted (trusted helper; existing plugin-Worker ABI binding still open)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A11, P5), ADR-0035, ADR-0036
Bin: `crates/marqdo-provider-worker`
Evidence: `docs/evidence/2026-11-20-c-worker-binding.md`
## Context
ADR-0036 loads a trusted C ABI provider in the host process; ADR-0035 provides a
framed-JSON worker transport. CM-A09 also asks for the Worker side of the C ABI:
the provider library should be loadable in a separate process so a crash does
not take down the interpreter, without inventing a new language surface. This
ADR adds the concrete binding.
## Decision
### 1. Worker binary
`marqdo-provider-worker <c-abi-library>`:
* loads exactly one `c-abi-1` shared library (ADR-0036 symbols);
* verifies `marqdo_resource_provider_abi_version() == 1` at startup;
* serves the ADR-0035 framed-JSON protocol (`ping/open/read/write/close/
shutdown`) over stdin/stdout;
* maps C provider failures to `{"ok":false,"code":"host.error","message":...}`
and never falls back to in-process loading.
The binary is a separate artifact, not a new top-level `marqdo` CLI command.
Registration is still host-side:
`StdHost::register_worker_resource_provider(name, kinds, worker_bin, [library])`
from ADR-0035.
### 2. Process boundary
* The C library runs in the worker process; a crash/exit is a structured host
error and the host does not retry in-process.
* The worker has the host user's OS privileges; this is a **trusted transport**,
not a sandbox and not proof of untrusted-provider safety.
* The language still sees only `resource.*`; provider tokens stay inside the
worker and the host `ResourceHandle` remains the language value.
* Project mode + `allow_resource_provider(name)` are still required; pure/strict
refuse provider kinds (CM-A11).
### 3. What remains for CM-A09
* Binding the provider contract through the **existing plugin-Worker ABI**
(manifest/artifact ownership and version negotiation) rather than the
dedicated helper.
* Provider registry/download/signatures and package installation commands.
* Untrusted-provider isolation and strict-mode providers.
* ABI version negotiation beyond v1.
## Consequences
* A trusted C adapter can be crash-isolated without changing the runtime,
parser, `Value` or `HostOp`.
* The worker helper is small and reusable by a future provider package manager.
* The trust model is explicit: worker isolation ≠ sandbox; the provider library
remains trusted code.
## Verification
`docs/evidence/2026-11-20-c-worker-binding.md` records a compiled C provider,
the worker helper, a full open/read/write/close round-trip through the host
worker transport, and fail-closed startup with a missing library.
+65
View File
@@ -0,0 +1,65 @@
# ADR-0039: Stdlib extraction is intentionally deferred
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A10, P2/P3), ADR-0028
Supersedes: the assumption in ADR-0028/CM-A10 evidence that a separate
`marqdo-stdlib` repository must be created as the next step.
## Context
The RFC and the first CM-A10 slice anticipated a separate `marqdo-stdlib`
repository so the standard library could be versioned and released independently
from the CLI. After implementing the bundled tree, `std:` imports, digest
identity, frozen-binary closure, table/HTTP modules and the deterministic
offline package, the current architecture is clear and the bundle is still
small. A separate repository would add release, version-mapping, signing and
cross-repo CI overhead without a present consumer that needs an independent
release cadence.
This decision **does not delete** the extraction work or the packaging artifact;
it changes the sequencing/intent.
## Decision
1. **Current policy: one repository.** `stdlib/` stays in `marqdo-cli` as the
official, verified bundle. `std:` imports, the SHA-256 tree digest,
module-instance ids and `project.stdlib` Analysis identity are unchanged.
2. **Extraction remains an explicit future option.** The deterministic package
(`tools/pack-stdlib.py`), `marqdo.stdlib-package` descriptor and version
mapping remain maintained as the extraction-ready artifact.
3. **No current release work is blocked on a separate repo.** Documentation,
STATUS and evidence must not describe "publish the external stdlib repo" as
a pending P2/CM-A10 gate; it is an intentionally deferred follow-up.
4. **Extraction triggers** (any one requires a new ADR before splitting):
* the bundle grows enough that CLI release cadence and stdlib cadence
conflict;
* a non-CLI consumer needs an independent stdlib release/SemVer line;
* multiple implementations or a provider/package ecosystem need version
skew management;
* the stdlib needs a different review/ownership boundary or signing policy.
5. **Compatibility if extraction happens:** keep `std:` import spelling,
module ids, digest algorithm and Analysis `project.stdlib` shape; publish a
version mapping and enforce the same frozen-binary/tamper gates.
## Non-goals
* Deleting the packaging script or historical CM-A10 evidence.
* Changing `std:` semantics, bundle digest or module ids now.
* Creating a package registry, signing infrastructure or cross-repo CI while
there is no independent consumer.
* Declaring the stdlib frozen forever; the trigger list above is intentional.
## Consequences
* The repository remains the single source of truth and the dev loop stays
simple.
* "Not external" is no longer reported as a missing gate or blocker.
* A future split can reuse the deterministic archive and digest contract
without redoing the bundling design.
## Verification
Docs/STATUS/evidence reference this ADR; `cargo test --workspace --locked`
continues to exercise the bundled `std:` imports, digest and frozen-CLI lane.
+50
View File
@@ -0,0 +1,50 @@
# Architecture RFCs: compute machine and stdlib migration
Status: **Draft RFCs, partially implemented**. This page is an index; the
normative RFC text stays at its reviewable path under `docs/`.
| Document | Status | Implemented part |
|---|---|---|
| [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). |
| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict permits only the stdio console channel; ADR-0027/0030/0031 Compat-C print via bundled stdlib, no legacy host formatter). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser/formatter for arrays, objects, strings with escapes, ints/floats/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). The stdlib intentionally remains in this repository for now (ADR-0039); no separate external repo, network/generic providers, P4 physical storage migration, P5 providers or P6 performance work. |
The RFCs are architecture constraints, not evidence that a feature exists. The
implementation status lives in `docs/STATUS.md`; frozen behavior is in
`spec/`; decisions are in `docs/adr/`.
## Decisions already frozen
| RFC item | ADR | State |
|---|---|---|
| CM-A01 Memory identity/lifetime | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
| CM-A02 page size, growth, budgets | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
| CM-A03 place assignment and evaluation order | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
| CM-A04 minimal structure observation/construction | [0023](../adr/0023-minimal-structure-observation-and-construction.md) | Accepted |
| CM-A05 recoverable `Result` convention | [0024](../adr/0024-recoverable-result-convention.md) | Accepted |
| CM-A06 minimal Resource/Stream ABI | [0025](../adr/0025-minimal-resource-stream-abi.md) | Accepted (first slice) |
| CM-A06 buffered project-file write | [0026](../adr/0026-buffered-project-file-write-streams.md) | Accepted |
| CM-A07 print migration staging / Compat-A | [0027](../adr/0027-print-migration-compat-a.md) | Accepted (Compat-A only) |
| CM-A10 bundled stdlib / `std:` imports / digest | [0028](../adr/0028-bundled-stdlib-and-std-imports.md) | Accepted (first slice) |
| Full-value stdlib formatting / FunctionRef+PluginHandle observation | [0029](../adr/0029-full-value-formatting-and-handle-observation.md) | Accepted |
| CM-A07 Compat-B print routing | [0030](../adr/0030-print-compat-b-routing.md) | Accepted |
| CM-A07 Compat-C remove legacy print formatting | [0031](../adr/0031-print-compat-c-remove-host-formatting.md) | Accepted |
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
| CM-A09 in-process ResourceProvider v1 | [0033](../adr/0033-in-process-resource-provider-v1.md) | Accepted (trusted in-process slice) |
| P5 blocking TCP resource endpoint v1 | [0034](../adr/0034-tcp-resource-endpoint.md) | Accepted (IP-literal, no DNS/TLS) |
| CM-A09 worker resource-provider transport v1 | [0035](../adr/0035-worker-resource-provider-transport.md) | Accepted (trusted helper) |
| CM-A09 C ABI ResourceProvider v1 | [0036](../adr/0036-c-abi-resource-provider.md) | Accepted (trusted native adapter) |
| CM-A09 provider package manifest v1 | [0037](../adr/0037-provider-package-manifest.md) | Accepted (host-side install) |
| CM-A09 C ABI provider worker binding v1 | [0038](../adr/0038-c-abi-provider-worker-binding.md) | Accepted (trusted helper) |
| Stdlib extraction deferral | [0039](../adr/0039-stdlib-extraction-intent.md) | Accepted (in-repo for now) |
| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication |
## What a future phase must not do
* add a top-level CLI command or Markdown keyword for memory, allocation or I/O;
* add a `HostOp` for every new stdlib function;
* turn managed `List`/`Map`/`Table` into aliased mutable storage;
* expose a host pointer, let a handle cross the wire, or claim block-level
safety for user allocators;
* make `strict` accept an unverified provider by default;
* claim an RFC phase is complete without the closure and negative evidence its
phase gate requires.
+52
View File
@@ -0,0 +1,52 @@
# Local development credentials
Credentials are **not committed**. They live in the git-ignored `.secrets/`
directory (or the untracked `.env.local`) so local development can use the
private Git and Woodpecker instances without rotating secrets through the
repository.
## Where to put them
`.secrets/dev.env` (ignored, mode 600) should define:
```dotenv
MARQDO_GIT_REMOTE=https://git.cflmy.cn/cflmy/marqdo-cli.git
MARQDO_GIT_USERNAME=<username>
MARQDO_GIT_TOKEN=<git access token>
MARQDO_WOODPECKER_URL=https://ci.cflmy.cn
MARQDO_WOODPECKER_TOKEN=<Woodpecker API token>
MARQDO_CI_SECRET=<optional CI secret>
```
Do not put real values in this document, tests, CI YAML or commit messages.
The repository's `.gitignore` excludes `.secrets/` and `.env.local`.
## Git push
` .secrets/git-askpass.sh ` returns the username/token for HTTPS prompts:
```bash
set -a; . ./.secrets/dev.env; set +a
GIT_ASKPASS="$PWD/.secrets/git-askpass.sh" git push origin master
```
Never embed the token in a remote URL stored in `.git/config`, shell history or
a CI file.
## Woodpecker API
```bash
set -a; . ./.secrets/dev.env; set +a
curl -fsS -H "Authorization: Bearer $MARQDO_WOODPECKER_TOKEN" \
"$MARQDO_WOODPECKER_URL/api/user/repos"
```
CI itself is defined in `.woodpecker.yml`; the token is only for local API
queries/triggering, not for the pipeline. Configure any privileged secrets in
the Woodpecker repository settings, never in this repository.
## What is tracked
- `.woodpecker.yml` — public CI pipeline definition.
- this document — names/usage only, no values.
- `.secrets/` and `.env.local` — ignored local files.
@@ -0,0 +1,114 @@
# RFC-CM CM-A10 evidence: bundled stdlib, `std:` imports and digest
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `92bfbf2` plus this step's changes.
Decision record: [ADR-0028](../adr/0028-bundled-stdlib-and-std-imports.md)
Related: [Compat-A](2026-11-20-stdlib-print-compat-a.md), [STATUS §5](../STATUS.md).
Scope: bundled first slice; **no external repository publication**.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
415 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Bundle
`stdlib/` is the official offline bundle:
```text
stdlib/
stdlib.json # name, version 0.1.0, sha256 digest
README.md
format.mq.md # 显示 / 列表显示 / 映射显示 / 表格显示 / 转义文本
text.mq.md # utf8编码 into Memory
io.mq.md # 写文本 / 写值 via resource.write(stdout)
demo.mq.md # builtin 打印 comparison
tests/io.mq.md # std:io self-test
```
Digest scheme: SHA-256 over sorted `*.mq.md` relative paths, NUL, file bytes,
NUL. `stdlib.json` and non-`.mq.md` files are excluded. The loader recomputes
the digest before loading any stdlib module and fails closed on mismatch or a
missing manifest.
`std:<relative>` is an explicit front-matter import target; it resolves only
inside the bundle, rejects `..`/absolute paths, appends `.mq.md` when absent,
and uses package instance
`std:marqdo-stdlib@0.1.0#<sha256-hex>`.
## Tests
`crates/cli/tests/stdlib_bundle.rs` (6):
```text
std_import_resolves_the_verified_bundle
index.mq.md imports std:io; io.写值(["bundle",1,2.5])
-> ["bundle", 1, 2.5]
analysis_records_the_verified_stdlib_digest
check --emit contains project.stdlib.version/digest/instance_id
matching stdlib.json
tampered_bundle_is_rejected_before_module_loading
copy bundle, modify io.mq.md
-> non-zero, "stdlib digest mismatch"
missing_bundle_manifest_is_a_hard_error
empty MARQDO_STDLIB_DIR -> non-zero stdlib error
bundle_uses_no_foreign_or_plugin_calls
scans all stdlib *.mq.md for foreign./plugin. -> none
frozen_binary_hash_is_unchanged_after_stdlib_execution
sha256(marqdo binary) before == after running the std:io project
```
CLI unit tests in `crates/cli/src/stdlib.rs` (2): dev bundle resolves and
verifies; a copied/tampered bundle is rejected.
Existing `mqmd_stdlib_print_output_matches_builtin_print` still passes with the
demo moved to `stdlib/demo.mq.md`.
## Analysis identity
Analysis v2 project object now carries:
```json
{"stdlib":{"version":"0.1.0",
"digest":"sha256:<64 hex>",
"instance_id":"std:marqdo-stdlib@0.1.0#<64 hex>"}}
```
It is `null` when no bundle is available. The digest is verified before load,
so an Analysis artifact cannot name mismatched stdlib bytes.
## What is not done
* No separate `cflmy/marqdo-stdlib` repository, release tag, package registry
or CI-signed digest. The bundle is vendored in this repository and copied
next to the executable in a release layout by later packaging work.
* No CM-A07 Compat-B/C routing; `打印` is still the builtin default.
* No Windows/macOS bundle layout evidence and no remote CI URL.
* No independent stdlib upgrade workflow; changing a module requires updating
`stdlib.json.digest`, and an incompatible core requirement is not yet
enforced by a version table.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-cli --test stdlib_bundle
cargo run -q -p marqdo-cli -- run examples/stdlib-bundle.mq.md
cargo test --workspace --locked
```
+56
View File
@@ -0,0 +1,56 @@
# RFC-CM CM-A09 evidence: C ABI ResourceProvider v1
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `12be02e` plus this step's changes.
Decision record: [ADR-0036](../adr/0036-c-abi-resource-provider.md)
Header: [`marqdo_resource_provider.h`](../../crates/marqdo-host/include/marqdo_resource_provider.h)
Related: [in-process provider](2026-11-20-provider-abi-cm-a09.md),
[worker provider](2026-11-20-worker-provider.md), [STATUS §5](../STATUS.md).
Scope: trusted native C ABI adapter; plugin-Worker binding still open.
## ABI
```c
marqdo_resource_provider_abi_version() -> uint32_t (must be 1)
marqdo_resource_open(kind, descriptor, rights_csv, *token) -> i32
marqdo_resource_read(token, buffer, capacity, *read, *eof) -> i32
marqdo_resource_write(token, bytes, length, *written) -> i32
marqdo_resource_close(token) -> i32
marqdo_resource_last_error(buffer, capacity) -> i32
```
The host resolves these symbols at registration, rejects non-v1 libraries and
missing symbols, attaches callsite spans to errors, and never lets a provider
token enter a language `Value`, wire frame or trace.
## Test
`crates/marqdo-host/tests/c_provider.rs` (`#[cfg(unix)]`):
1. builds a C shared library with `cc -shared -fPIC` (skips if `cc` is absent);
2. registers it as provider `cabi` for kind `cabi.sensor` with a project-mode
grant;
3. `resource.open/read/write/close` round-trips `c-`/`abi`, `write=3`, second
close `resource.closed`;
4. a missing library path fails with `kernel.unavailable`;
5. a library reporting ABI version 2 fails with `resource.unsupported`;
6. pure/strict refuse the C provider kind with `permission.denied`.
No Rust runtime/`HostOp`/parser change was needed; language code still sees the
same `resource.*` surface.
## Security boundary
The C library is loaded in-process with host privileges. It is trusted native
code and explicitly **not** a sandbox. Pure/strict refuse provider kinds; strict
remains fail-closed. Untrusted plugin loading remains the separate plugin Worker
transport.
## Not done
* Plugin-Worker resource-provider binding through the existing plugin ABI.
* Provider package installation/signatures/discovery.
* ABI version negotiation beyond v1 rejection.
* Untrusted-provider isolation or strict-mode providers.
* Windows shared-library build/lane.
@@ -0,0 +1,45 @@
# RFC-CM CM-A09 evidence: C ABI provider worker binding v1
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `313ac03` plus this step's changes.
Decision record: [ADR-0038](../adr/0038-c-abi-provider-worker-binding.md)
Related: [C ABI](2026-11-20-c-provider.md), [worker transport](2026-11-20-worker-provider.md).
Scope: trusted helper binding; not a sandbox and not the plugin-Worker ABI.
## Artifact
`marqdo-provider-worker <library>` loads a `c-abi-1` shared library and serves
the ADR-0035 framed-JSON protocol (ping/open/read/write/close/shutdown). The
host registers it through the existing
`StdHost::register_worker_resource_provider` API; no runtime/`HostOp`/parser
change was required.
## Tests
`crates/marqdo-provider-worker/tests/worker_binding.rs`:
```text
c_abi_provider_runs_inside_the_worker_helper
cc -shared C provider; worker binary loaded;
resource.open/read "worker"/"-abi" eof/write/close through the host.
worker_helper_without_c_provider_fails_closed
missing library -> structured host error, no in-process fallback.
```
The C provider state lives in the child; the host only sees the opaque
`ResourceHandle` and `ResourceRead` bytes.
## Boundary
The worker is a trusted helper with host-user privileges. It isolates crashes
and keeps the C library out of the interpreter process, but it is **not** an OS
sandbox. Pure/strict still refuse provider kinds; strict remains fail-closed.
## Not done
* Binding through the existing plugin-Worker manifest/artifact ABI.
* Provider registry/signatures/install commands.
* Untrusted-provider isolation, strict providers, Windows worker packaging.
* ABI version negotiation beyond rejecting non-v1 libraries.
@@ -0,0 +1,91 @@
# RFC-CM ADR-0029 evidence: full-value stdlib formatting
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `1377af1` plus this step's changes.
Decision record: [ADR-0029](../adr/0029-full-value-formatting-and-handle-observation.md)
Related: [Compat-A](2026-11-20-stdlib-print-compat-a.md),
[bundled stdlib](2026-11-20-bundled-stdlib-cm-a10.md), [STATUS §5](../STATUS.md).
Scope: observation members + stdlib formatter. No print name routing.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
416 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Observation members
`Value::FunctionRef` now exposes `module` and `name`; `Value::PluginHandle`
exposes `plugin` and `export`. The numeric host `PluginHandle.id` remains
private. Formula/Code members were already public. Memory/Resource remain
opaque with no fields.
Runtime test `function_ref_and_resource_members_are_readable` checks:
```text
ref("#目标").module -> root:default::index.mq.md
ref("#目标").name -> 目标
ref("#公式").latex -> a = 1
ref("#公式").display -> true
ref("#代码").language-> text
ref("#代码").source -> "x\n"
```
## Formatter coverage
`stdlib/format.mq.md` now has branches for all 15 value types:
`Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, `List`, `Map`, `Table`,
`Formula`, `Code`, `FunctionRef`, `PluginHandle`, `Memory`, `Resource`.
* Formula/Code/FunctionRef/PluginHandle reproduce the existing `Value::repr()`
spellings (not a new format).
* Code byte length is computed from `text.codepoints` by summing UTF-8 widths,
not from `长度` (scalar count) and without a temporary `Memory`.
* Text inside collections is quoted/escaped; top-level Text is raw.
## Comparison
`stdlib/demo.mq.md` compares `io.写值` with builtin `打印`:
```text
["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]
["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]
Table[名称 | 分数]{甲 | 87; 乙 | 92}
Table[名称 | 分数]{甲 | 87; 乙 | 92}
[Formula("a = 1"), Code("text", 9 bytes), FunctionRef(root:default::demo.mq.md::演示函数)]
[Formula("a = 1"), Code("text", 9 bytes), FunctionRef(root:default::demo.mq.md::演示函数)]
```
`examples_r6::mqmd_stdlib_print_output_matches_builtin_print` asserts all three
pairs byte-identically, including the Formula/Code/FunctionRef line.
## What is not done
* No `打印/print` name routing (Compat-B). Builtin remains the default.
* PluginHandle formatting is covered by code branches and observation, but no
fixture constructs a live PluginHandle without a plugin invocation.
* Control characters outside `"`, `\`, newline, tab and CR are not emitted as
`\u00XX`.
* Memory/Resource formatting is the literal type name, matching
`Value::display()`; no internal identity leaks.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo run -q -p marqdo-cli -- run stdlib/demo.mq.md
cargo test -p marqdo-cli --test examples_r6
cargo test -p marqdo-runtime --test interpreter function_ref_and_resource_members
cargo test --workspace --locked
```
+52
View File
@@ -0,0 +1,52 @@
# RFC-CM P5/C10 evidence: HTTP/1.1 over the TCP resource endpoint
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `0d4dc1d` plus this step's changes.
Related: [ADR-0034](../adr/0034-tcp-resource-endpoint.md),
[HTTP parser fixture](2026-11-20-http-parser.md), [STATUS §5](../STATUS.md).
Scope: `.mq.md` HTTP framing over a loopback TCP stream; no host HTTP/TLS.
## Module
`stdlib/net/http.mq.md` (import `std:net/http`) provides:
```text
解析响应(文本) -> {状态, 原因, 头, 体}
解码分块(体) -> Text
构建请求(方法, 路径, 头列表, 体) -> Text
切片/找子串/找字符/十六进制值
```
It uses only `text.codepoints`/`text.from_codepoint` and List/Map/Text values;
there is no `net.request` or host HTTP parser.
## End-to-end test
`crates/cli/tests/http_over_tcp.rs`:
1. starts a loopback `TcpListener` that reads the request and sends
`HTTP/1.1 200 OK ... Content-Length: 5 ... hello`;
2. runs a generated `.mq.md` program with `--allow-net 127.0.0.1`;
3. the program builds a GET request, UTF-8-encodes it through `std:text` into
`Memory`, writes it to `resource.open("tcp", addr, ["read","write"])`,
reads the response bytes, decodes them to Text with
`text.from_codepoint`, parses with `stdlib/net/http`, and returns
`[write.ok, read.ok, status, content-type, body, close.ok]`;
4. asserts `[true, true, 200, "text/plain", "hello", true]`.
`stdlib/tests/http.mq.md` independently pins the parser/builder without a
socket. Semantic fixture 71 pins the same logic in the conformance corpus.
## Gate
Local container: `cargo test -p marqdo-cli --test http_over_tcp` PASS and
`cargo test -p marqdo-cli --test stdlib_http` PASS; full workspace gate PASS.
No external network access is required.
## Not done
* HTTPS/TLS; no TLS security claim.
* External DNS, redirects, connection reuse, compression, cookies, HTTP/2.
* A host HTTP client/server; this is `.mq.md` framing over the byte stream.
* Strict-mode TCP (strict refuses TCP; project mode only).
+57
View File
@@ -0,0 +1,57 @@
# RFC-CM P5/C10 evidence: `.mq.md` HTTP/1.1 parser
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `67a6d35` plus this step's changes.
Related: [ADR-0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md),
[ADR-0034](../adr/0034-tcp-resource-endpoint.md), [STATUS §5](../STATUS.md).
Scope: Text-level HTTP framing in `.mq.md`; no host HTTP/TLS implementation.
## Fixture
`tests/conformance/semantics/71-http-parser.mq.md` (+ expected golden):
* `找子串`/`找字符`/`切片` over `text.codepoints` and `text.from_codepoint`;
* `解析响应` splits the head/body at `\r\n\r\n`, parses the status line,
headers and `Transfer-Encoding: chunked`;
* `解码分块` parses hex chunk sizes and reassembles the body;
* `构建请求(方法, 路径, 头列表, 体)` emits an HTTP/1.1 request with CRLF
framing.
Input:
```text
HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked
5
hello
6
world
0
```
Result:
```json
[200, "OK", "text/plain", "hello world",
"GET /health HTTP/1.1\r\nAccept: text/plain\r\nHost: example.test\r\n\r\n"]
```
The fixture is run end-to-end by `conformance_semantics` and is part of the
71 semantic goldens.
## Gate
Local container: fmt/clippy/426 tests PASS (one fixture test; the golden count
is 71). No host `net.request`, HTTP client/server or TLS code was added.
## Not done
* HTTP client over TCP, connection reuse, redirects, compression, cookies.
* HTTPS/TLS; this ADR/evidence makes **no** TLS security claim.
* Request-body streaming, trailers, chunked requests, HTTP/2.
* A reusable `stdlib/net/http.mq.md` module (the parser is a conformance
fixture, not a bundled module yet).
* C/Worker provider transport and external stdlib repository publication.
+130
View File
@@ -0,0 +1,130 @@
# RFC-CM P1 evidence: addressable Memory and place assignment
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline commit: `a113d894962d8c00eb02efb42e0e73f8b084a1fa` (`docs: bind
README/COMPATIBILITY/CHANGELOG to the convergence SHA`)
Working tree: RFC-CM P1 changes, **not committed** at the time of this report.
Decision record: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md)
Scope: RFC-CM-0001 R1 / RFC-CM-0002 P1 only. P2–P6 are **not** implemented.
## Environment
* Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
* `PATH=/root/.cargo/bin:$PATH`
* `CARGO_HOME=/root/work/marqdo-cli/.cache/cargo`
* `CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target`
* No GitHub credentials in this checkout; this is **local** evidence, not a
GitHub Actions result. Windows/macOS lanes were not run.
## Static and workspace gate
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests)
```
The 2 ignored tests are the existing `marqdo-plugin-sdk` doc-tests, not new
memory tests.
## Focused evidence
```text
cargo test -p marqdo-runtime --lib memory
8 passed; 0 failed
regions_are_isolated_and_handles_share_identity
create_is_zero_filled_and_reports_pages
grow_keeps_old_bytes_and_zero_fills_new_pages
failed_grow_does_not_change_region_or_budget
invalid_offsets_are_structured_errors
create_validates_page_counts_and_budget
forged_and_stale_handles_are_rejected
page_arithmetic_never_overflows
cargo test -p marqdo-runtime --test interpreter memory
3 passed; 0 failed
memory_region_is_shared_by_handle_identity
memory_handle_cannot_reach_the_host_boundary
memory_budget_is_enforced_without_changing_the_region
cargo test -p marqdo-value --lib memory
2 passed; 0 failed
memory_display_never_leaks_the_runtime_handle
memory_handles_are_not_transferable
cargo test -p marqdo-expression --lib memory_index
1 passed; 0 failed
memory_index_assignment_parses_as_a_place
cargo test -p marqdo-cli \
--test conformance_semantics --test negative \
--test analysis_trace --test examples_r6
semantic_goldens_match_end_to_end_execution ok
runtime_negative_goldens ok
analysis_v2_projects_memory_place_targets_additively ok
addressable_memory_example_runs_through_the_cli ok
cli_memory_budget_override_fails_closed ok
(20 tests total in these four binaries, 0 failed)
```
## End-to-end CLI
```text
$ marqdo run examples/memory-buffer.mq.md --json
{"success": true, "value": {"type": "int", "decimal": "141"}, ...}
$ marqdo run examples/memory-buffer.mq.md --max-memory 0 --json
{"success": false, "error": {"code": "resource.exhausted", ...}, ...}
```
The example creates one 64 KiB page with a declared maximum of four pages,
writes two bytes through two names that share one region, and returns
`42 + 99 = 141`.
## Fixtures
Semantic goldens:
* `tests/conformance/semantics/45-memory-buffer.mq.md` — shared identity
(`另一个名字 = 区域`) and two byte writes.
* `tests/conformance/semantics/46-memory-grow-zero-fill.mq.md` — zero-page
create, `grow` returning the old count, old bytes preserved, new byte zero,
`memory.size`.
* `tests/conformance/semantics/47-memory-isolation-and-alias.mq.md` — two
regions do not alias at the same offset; handle equality is identity.
Runtime negatives:
| Fixture | Expected code |
|---|---|
| `13-memory-out-of-bounds` | `runtime.index` |
| `14-memory-byte-range` | `runtime.type` |
| `15-index-assign-non-memory` | `runtime.type` |
| `16-memory-huge-offset` | `runtime.index` |
| `17-memory-grow-beyond-max` | `resource.exhausted` |
| `18-memory-negative-pages` | `runtime.memory` |
## What this does not prove
* No `.mq.md` stdlib repository or allocator/closure fixture yet (RFC-CM P2).
* No `Resource`, `Stream`, byte-transfer ABI, stdlib `print` migration or
`Table` storage abstraction (P3–P5).
* No Windows/macOS evidence and no remote CI URL.
* No claim of block-level memory safety for a user-written allocator; the
runtime only bounds each access to its region.
* No claim that `strict` accepts new I/O: memory is pure computation and no
`HostOp` or provider was added.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace --locked
cargo run -p marqdo-cli -- run examples/memory-buffer.mq.md --json
```
@@ -0,0 +1,74 @@
# RFC-CM P2 partial evidence: `.mq.md`-written codec, allocator and list
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes.
Related: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md),
[P1 evidence](2026-11-20-memory-p1.md),
[RFC-CM-0002 §7.2](../../docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md).
Scope: **partial P2 only**. The RFC-CM-0002 P2 gate is **not** passed.
> Historical slice: this report predates ADR-0023 (CM-A04 structure API),
> ADR-0024 (CM-A05 Result), the coalescing allocator (fixture 55) and the
> binary tree/CSV fixtures (56–57). Its "what remains" list is superseded by
> [`2026-11-20-structure-api-p2.md`](2026-11-20-structure-api-p2.md) and
> [`2026-11-20-result-and-free-list-p2.md`](2026-11-20-result-and-free-list-p2.md);
> the P2 gate itself is still not passed.
## What was added without Rust algorithm code
Three semantic fixtures are ordinary `.mq.md` documents. They call only
language builtins (`memory.new/size/grow`), indexing, arithmetic, functions,
branches and loops. No Rust crate was changed for the algorithms themselves.
| Fixture | `.mq.md` subject | Result |
|---|---|---|
| `tests/conformance/semantics/48-memory-bump-allocator.mq.md` | big-endian u32 encode/decode and a 4-byte-header bump allocator; allocates 8, 16, 8 bytes and writes a byte | `[4, 12, 28, 36, 255]` |
| `tests/conformance/semantics/49-memory-linked-list.mq.md` | allocates three 8-byte nodes (`value:u8`, `next:u32`), links them and traverses the list | `[4, 12, 20, 60]` |
| `tests/conformance/semantics/50-memory-allocation-failure-atomic.mq.md` | a too-large allocation returns `None` without moving the header; the next allocation still starts at 4 | `[None, 4, 4, 65536]` |
Observed CLI results (all `success: true`):
```text
48: [4, 12, 28, 36, 255]
49: [4, 12, 20, 60]
50: [None, 4, 4, 65536]
```
`cargo test -p marqdo-cli --test conformance_semantics` passes with these three
fixtures included.
## Why this is not the P2 gate
RFC-CM-0002 §9/P2 requires C01–C06 and C11 to pass in an environment with
Foreign/native plugins disabled, and requires a minimal API for dynamic
List/Map/Table construction and observation plus recoverable errors. This
slice does **not** provide:
* duplicate-free/allocator adversarial tests (C01 now has a first-fit
free-list allocator with reuse, splitting and coalescing, fixture 55);
* graph/hash-table structures (C02 now has a singly linked list and a binary
search tree, fixtures 49 and 56, but no graph/hash table);
* general List/Map/Table construction, mutation-free rebuilding or key
enumeration (CM-A04 is undecided);
* a `Result`/recoverable-error contract for parser/IO/EOF/partial conditions
(CM-A05 is undecided);
* an independent `marqdo-stdlib` repository, version/digest contract or
frozen-CLI-binary closure lane (the RFC §10.5 experiment).
It also does not add a stdlib API to the CLI; the fixtures are conformance
documents, not a shipped standard library.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
for f in 48-memory-bump-allocator 49-memory-linked-list \
50-memory-allocation-failure-atomic; do
cargo run -q -p marqdo-cli -- run \
tests/conformance/semantics/$f.mq.md --json
done
cargo test -p marqdo-cli --test conformance_semantics
```
@@ -0,0 +1,69 @@
# RFC-CM P2 closure gate evidence (C01–C06, C11, C12)
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `d6cd785` plus this step's changes.
Related: [memory P2](2026-11-20-memory-p2-closure.md),
[structure API](2026-11-20-structure-api-p2.md),
[result/free-list](2026-11-20-result-and-free-list-p2.md),
[ADRs 0022–0024](../adr/).
Scope: pure no-Foreign/no-Plugin execution lane; external stdlib repository
publication remains separate.
## Lane
`crates/cli/tests/p2_closure.rs` runs 21 checked-in semantic fixtures through
the real CLI with `run --mode pure --json`:
* `--mode pure` constructs `Capabilities::pure()`; no foreign, plugin, read,
write, network or provider grant exists in that host;
* each fixture source is scanned for `foreign.`, `plugin.` and `import:`; all
three must be absent (anti-evasion rule for C01–C06/C11/C12);
* the CLI value and stdout are compared byte-for-byte with the fixture's
`.expected.json` golden;
* `stdlib/tests/table.mq.md` also runs in pure mode with `std:` imports and
checks the nine table results including `转GFM`.
The independently-run conformance harness (`conformance_semantics.rs`) also
builds `StdHost` with `SandboxMode::Pure` and `Capabilities::pure()` for every
fixture.
Negative shortcuts:
```text
foreign.call in pure -> permission.denied
plugin.invoke in pure -> structured failure
```
so the lane proves both that the algorithms run without host shortcuts and that
the shortcuts are actually unavailable.
## Criterion mapping
| Criterion | Fixtures | What is proven |
|---|---|---|
| C01 allocators | 48, 50, 55 | bump allocator, failed-allocation atomicity, first-fit split/coalesce/double-free |
| C02 structures | 49, 56, 58, 61 | linked list, binary tree, open-addressing hash table, graph adjacency |
| C03 codecs | 48, 64 | u32, big-endian 16/32/64, LEB128 |
| C04 Unicode/text | 59, 60 | UTF-8 encode and validating decoder errors |
| C05 JSON/CSV | 57, 62, 68 | CSV split/quoting, recursive JSON parse/format/errors |
| C06 GFM Table | 53, 54, 65, 67, 69 + stdlib table | projection/filter/sort/join/group/GFM render, no Rust table algorithm |
| C11 VM | 63 | Memory-backed stack bytecode VM |
| C12 scientific prep | 70, 73 | Int matrix multiply and sparse-triple matrix multiply with dense conversion |
No fixture compiles Rust, imports a package, invokes a foreign kernel or calls a
plugin. Dynamic arrays are built at runtime; no fixed-length source rewrite is
required.
## Gate
Local container: `cargo test -p marqdo-cli --test p2_closure` PASS; full
`cargo test --workspace --locked` PASS. The existing frozen-binary test covers
the fixed CLI hash and stdlib tamper rejection (ADR-0028).
## Remaining (not a C01–C12 gap)
* The stdlib is intentionally in-repo for now (ADR-0039); a separate
repository with a signed version mapping is a future extraction option, not
a P2 gate. The bundled tree, digest and offline layout are tested in-repo.
* No GitHub CI URL / Windows/macOS lanes from this host.
+57
View File
@@ -0,0 +1,57 @@
{
"schema": "marqdo.perf-baseline",
"schema_version": 1,
"binary": ".cache/target/release/marqdo",
"iterations": 3,
"workloads": [
{
"name": "allocator-free-list",
"source": "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md",
"seconds_median": 0.0173,
"seconds_min": 0.017026568995788693,
"max_rss_kb_median": 8428
},
{
"name": "json-parser",
"source": "tests/conformance/semantics/62-json-array-parser.mq.md",
"seconds_median": 0.0188,
"seconds_min": 0.01880078506655991,
"max_rss_kb_median": 8960
},
{
"name": "table-sort",
"source": "tests/conformance/semantics/65-table-sort-stdlib.mq.md",
"seconds_median": 0.0152,
"seconds_min": 0.014747997978702188,
"max_rss_kb_median": 7476
},
{
"name": "bytecode-vm",
"source": "tests/conformance/semantics/63-memory-bytecode-vm.mq.md",
"seconds_median": 0.0155,
"seconds_min": 0.015298803802579641,
"max_rss_kb_median": 7692
},
{
"name": "matrix-multiply",
"source": "tests/conformance/semantics/70-matrix-multiply.mq.md",
"seconds_median": 0.0151,
"seconds_min": 0.014752828981727362,
"max_rss_kb_median": 7368
},
{
"name": "http-parser",
"source": "tests/conformance/semantics/71-http-parser.mq.md",
"seconds_median": 0.018,
"seconds_min": 0.017968795029446483,
"max_rss_kb_median": 8196
},
{
"name": "stdlib-print-demo",
"source": "stdlib/demo.mq.md",
"seconds_median": 0.0205,
"seconds_min": 0.02037098305299878,
"max_rss_kb_median": 9636
}
]
}
+53
View File
@@ -0,0 +1,53 @@
# RFC-CM P6 baseline profile
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `b020904` plus this step's changes.
Scope: **baseline measurement only**; no bytecode/JIT/native backend was added.
## Harness
`tools/perf/bench.py` builds `marqdo-cli --release`, runs each workload three
times through `/usr/bin/time -f "%e %M"`, measures wall time with
`time.perf_counter`, verifies `--json` success and writes a schema-tagged JSON
report. The checked-in report is
[`2026-11-20-p6-profile.json`](2026-11-20-p6-profile.json).
Environment: Ubuntu 26.04 container, Linux x86_64, Rust 1.100.0, release
profile, 7 selected `.mq.md` workloads. These are small conformance workloads,
so most times are dominated by process/startup and Markdown parsing; they are
not throughput benchmarks of large scientific problems.
## Results (median of 3)
| Workload | Wall s | Peak RSS KiB |
|---|---:|---:|
| allocator-free-list | 0.0173 | 8428 |
| json-parser | 0.0188 | 8960 |
| table-sort | 0.0152 | 7476 |
| bytecode-vm | 0.0155 | 7692 |
| matrix-multiply | 0.0151 | 7368 |
| http-parser | 0.0180 | 8196 |
| stdlib-print-demo | 0.0205 | 9636 |
## Interpretation
* Startup + parse + load dominate these small workloads; the interpreter's
per-byte/per-call cost is visible mainly in the VM/matrix/table loops.
* `stdlib-print-demo` is highest RSS/wall because it loads the bundled stdlib
modules and formats all value types through `.mq.md`.
* The current release binary is a tree-walking Rust interpreter; no bytecode
compiler, JIT, columnar numeric backend or copy-elision pass exists.
* No claim of P6 pass: the RFC requires an optimized implementation that is
equivalent to the reference interpreter and reproducible performance/budget
evidence. This report only establishes the baseline and a reproducible
harness for future work.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
python3 tools/perf/bench.py > /tmp/marqdo-perf.json
```
+108
View File
@@ -0,0 +1,108 @@
# RFC-CM CM-A07 Compat-B evidence: `打印` routing through the stdlib
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `e119727` plus this step's changes.
Decision record: [ADR-0030](../adr/0030-print-compat-b-routing.md)
Related: [Compat-A](2026-11-20-stdlib-print-compat-a.md),
[full-value formatting](2026-11-20-full-value-formatting.md),
[bundled stdlib](2026-11-20-bundled-stdlib-cm-a10.md).
Scope: Compat-B routing; **Compat-C removal is not done**.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
416 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Routing
* `Program.print_override: Option<FunctionId>` (IR, serde-optional).
* Loader scans loaded IR for bare `打印`/`print` calls. Only if found, it loads
bundled `std:io` and sets the override to `io.写值`.
* Runtime resolution: local callable > print override > builtin table >
same-module/imported function. Local shadowing is unchanged.
* The stdlib path formats all 15 value types (ADR-0029), UTF-8-encodes into
`Memory`, opens `stdio/stdout`, writes and closes.
* Programs without print calls keep their original module graph.
## Trace behavior
`crates/cli/tests/print_routing.rs::print_is_routed_through_the_stdlib_and_not_hostop_print`:
```text
run index.mq.md --trace-file trace.json
stdout = ["x", 1]
trace operations include resource.write
trace operations do NOT include print
```
Resource open/read/write/close are now recorded as `effect` events by
`StdHost`, so trace consumers see the real byte path. No values or secrets are
recorded.
## Lazy injection
`stdlib_print_modules_are_injected_only_when_print_is_used`:
```text
plain.mq.md (no print) -> no std: modules
prints.mq.md (打印) -> std: modules present
```
Both loads record `project.stdlib` identity because a verified bundle is
available; only the print-using program loads its modules. `check`'s
human-readable module count excludes `std:` modules so bundle internals do not
change the project count.
## Strict mode
Strict L1 already permits the console. Compat-B needs the same channel, so
strict now permits **only** `resource.open("stdio","stdout",["write"])` with the
console grant. File/network/process/device resource opens remain
`permission.denied`. Host test:
```text
strict_mode_allows_only_the_existing_stdio_console_channel
stdio write -> "strict"
file read -> permission.denied
socket read -> permission.denied
```
This is the existing console capability through the resource ABI, not a new OS
capability; CM-A11 fail-closed remains for all other providers.
## Compatibility
* All semantic goldens and CLI print tests pass byte-identically, including
the full-value formatting demo.
* `HostOp::Print` remains implemented as the no-bundle fallback and for
Compat-A comparison tests; Compat-C removal is a separate decision.
* Trace consumers that hard-code `operation == "print"` must accept
`resource.*`; this is documented as a P3 migration.
## What is not done
* Compat-C: `HostOp::Print` and `Value::display()` formatting are still in the
public builtin path for no-bundle installs.
* No stdlib version upgrade workflow and no external repository publication.
* No Windows/macOS evidence and no remote CI URL.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-cli --test print_routing
cargo test -p marqdo-cli --test analysis_trace
cargo test -p marqdo-host --test resource_streams
cargo test --workspace --locked
```
@@ -0,0 +1,75 @@
# RFC-CM CM-A07 Compat-C evidence: legacy print formatting removed
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `e26971f` plus this step's changes.
Decision record: [ADR-0031](../adr/0031-print-compat-c-remove-host-formatting.md)
Related: [Compat-A](2026-11-20-stdlib-print-compat-a.md),
[Compat-B](2026-11-20-print-compat-b.md),
[full-value formatting](2026-11-20-full-value-formatting.md).
Scope: Compat-C removal of the public host formatting path.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
419 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Removed
* `HostOp::Print` variant, its stable op name and `StdHost::op_print`.
* The print builtin's host dispatch. `打印/print` remains a builtin **name**
for arity validation, but it no longer formats or writes:
* with `Program.print_override` set (official CLI + bundle), the runtime calls
`stdlib/io.写值`;
* without an override, it returns `kernel.unavailable`:
"打印 requires the bundled stdlib print module; install the official CLI
bundle."
`Value::display()` remains for CLI result rendering/diagnostics but is not
reachable from the print builtin.
## Runtime/embedder behavior
`crates/marqdo-runtime/tests/interpreter.rs::print_without_the_bundled_stdlib_is_a_structured_error`
builds a Program without an override and asserts the structured error.
The semantic conformance harness builds Programs directly, so it now mirrors
the loader's Compat-B injection: it loads `stdlib/io.mq.md` (with
`format`/`text`), adds import edges and sets `print_override`. Existing print
fixtures 22/23/30/42 pass byte-identically.
## Trace and output
`crates/cli/tests/print_routing.rs` still asserts a print program's trace
contains `resource.write` and no `print` effect. The CLI loader routes official
runs through stdlib; `打印` output remains byte-identical, including the
full-value Formula/Code/FunctionRef demo.
## Compatibility notes
* Breaking for embedders that call `打印` without installing the stdlib print
module; the official CLI loader does this automatically.
* `HostOp` loses a variant; any external exhaustive match must be updated.
This is pre-1.0 and recorded in ADR-0031.
* No bundle means no print; this is fail-closed rather than a silent formatter.
* Compat-A/B evidence remains historical; their fallback wording is superseded.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-runtime --test interpreter print_without
cargo test -p marqdo-cli --test print_routing
cargo test -p marqdo-cli --test conformance_semantics
cargo test --workspace --locked
```
@@ -0,0 +1,90 @@
# RFC-CM CM-A09 evidence: in-process ResourceProvider v1
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a8e62a5` plus this step's changes.
Decision record: [ADR-0033](../adr/0033-in-process-resource-provider-v1.md)
Related: [resource streams](2026-11-20-resource-stream-p3.md), [STATUS §5](../STATUS.md).
Scope: trusted in-process provider contract only; C/Worker transport remains open.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
423 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Contract
`ResourceProvider` v1:
```text
name() -> stable grant name
protocol_version() -> 1
kinds() -> resource kinds
open(kind, descriptor, rights) -> provider token
read(token, max_bytes) -> {bytes, eof}
write(token, bytes) -> count
close(token)
```
The host still issues the language `ResourceHandle`; the provider token never
enters a `Value`, wire message or trace. Bytes are copied for the call duration;
providers must not retain Memory pointers.
## Registration and authorization
* `StdHost::register_resource_provider` rejects protocol != 1, empty kinds,
reserved `stdio`/`file` names/kinds and duplicate kind registration.
* `Capabilities.resource_providers` + `allow_resource_provider(name)` add the
grant; `resource.open` requires the provider name to be granted.
* Project mode only. Pure and strict refuse provider kinds; registration grants
nothing by itself.
* In-process providers are trusted and **not** sandboxed; the ADR states this
explicitly.
## Tests
`crates/marqdo-host/tests/resource_streams.rs` (12 total; 3 provider tests):
```text
registered_provider_streams_work_with_a_grant_in_project_mode
mock sensor: read ab/cdef with eof, write 1, close, second close resource.closed
provider_requires_a_name_grant_and_project_mode
no grant -> permission.denied
pure -> permission.denied
strict -> permission.denied
provider_registration_rejects_versions_reserved_names_and_conflicts
version 2, reserved stdio kind, duplicate kind -> resource.unsupported
```
No parser, `Value`, runtime dispatch or `HostOp` change was required; the mock
provider is registered through a host API and the language sees the existing
`resource.*` calls.
## Not done
* C ABI / Worker resource transport: buffer handoff, ownership, crash/timeout
isolation and version negotiation through the plugin ABI.
* Provider package installation, signatures, discovery and platform matrix.
* Network/socket/TCP/HTTP, clock, random, process, directory, device providers.
* Strict-mode provider verification.
* No external `marqdo-stdlib` repository publication.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-host --test resource_streams
cargo test --workspace --locked
```
@@ -0,0 +1,45 @@
# RFC-CM CM-A09 evidence: provider package manifest v1
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `5e5c3a8` plus this step's changes.
Decision record: [ADR-0037](../adr/0037-provider-package-manifest.md)
Related: [C ABI provider](2026-11-20-c-provider.md), [STATUS §5](../STATUS.md).
Scope: local manifest installation; no registry/signature/Worker binding.
## Manifest
```json
{
"schema": "marqdo.provider",
"schema_version": 1,
"name": "cabi",
"kinds": ["cabi.sensor"],
"abi": "c-abi-1",
"library": {"linux": "libecho_provider.so", "default": "libecho_provider.so"}
}
```
`StdHost::register_provider_package` validates schema/version/ABI, resolves the
platform library relative to the manifest, loads it through the ADR-0036 C ABI
adapter and registers it through the shared kind/name checks. Language use still
requires project mode and `allow_resource_provider("cabi")`.
## Test
`crates/marqdo-host/tests/c_provider.rs::provider_package_manifest_installs_a_c_abi_library`:
1. compile the C provider with `cc -shared`;
2. write `provider.json` next to it;
3. `register_provider_package(&manifest)` succeeds;
4. `resource.open("cabi.sensor", ...)` and `resource.read` return `c-`.
Existing C tests cover version mismatch, missing library and pure/strict refusal.
No parser/`Value`/runtime/`HostOp` change was needed.
## Not done
* Provider registry/download, signatures, dependency resolution, semver.
* CLI install/uninstall commands; installation is host embedding API only.
* Untrusted-provider isolation, strict-mode providers, plugin-Worker binding.
* Windows library build lane.
@@ -0,0 +1,104 @@
# RFC-CM CM-A06 extension evidence: buffered project-file writes
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes.
Decision record: [ADR-0026](../adr/0026-buffered-project-file-write-streams.md)
Related: [ADR-0025](2026-11-20-resource-stream-p3.md),
[STATUS §5](../STATUS.md).
Scope: project-file **write** extension only; no network/process/directory.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
406 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Implemented behavior
`resource.open("file", "<project-relative path>", ["write"])`:
* project mode only; requires an `fs_write` glob grant and the same
`ProjectFs` path policy (reject absolute, `..`, symlink escapes);
* pure mode refuses file resources by mode; strict refuses every resource open;
* the writer owns an in-memory buffer; `resource.write` appends and returns
`{ok:true, value:{count}}` without touching the filesystem;
* `resource.close` flushes through `ProjectFs::write_bytes` and then marks the
handle closed; a failed flush leaves the handle open with its buffer;
* an unclosed writer never creates the file (no flush on drop);
* buffer ceiling is the existing 8 MiB file limit; an oversized append is
`resource.exhausted` and leaves both buffer and file unchanged;
* `resource.read` on a writer, and `resource.write` on a reader, are
`resource.unsupported`;
* no new `Value`, `HostOp`, Markdown node or wire shape.
## Tests
### Host integration (`crates/marqdo-host/tests/resource_streams.rs`, 9)
```text
pure_mode_writes_stdout_through_a_resource
stdout_write_obeys_the_output_budget_atomically
strict_mode_refuses_every_resource_open
resource_handles_cannot_cross_the_generic_host_boundary
pure_mode_refuses_file_resources_even_with_a_grant
project_file_writes_buffer_until_close_and_require_a_grant
file_write_buffer_has_a_hard_ceiling
project_file_reads_are_chunked_and_require_a_grant
traversal_and_unsupported_rights_are_refused
```
The write test asserts the file does not exist before `close`, appears with
`abcdef` after `close`, closed read/close return `resource.closed`, and the
same open without `fs_write` is `permission.denied`.
### CLI integration (`crates/cli/tests/resource_streams.rs`, 4)
```text
project_file_stream_writes_on_close_then_reads_back
run index.mq.md --allow-write out.bin --allow-read out.bin --json
write count 3, close ok, read count 3 eof true, bytes 65/66/67
out.bin on disk = "ABC"
file_copy_is_written_in_mqmd_through_chunked_streams
.mq.md opens data.txt for read and out.bin for write,
loops resource.read(16) / resource.write(count) until eof,
closes both; total 6; out.bin on disk = "abcdef"
(C09 file copy without a Rust copy_file builtin)
project_file_stream_reads_chunks_into_memory
file_stream_requires_a_read_grant
```
### Host table unit tests (`crates/marqdo-host/src/resource.rs`, 3)
Chunked EOF, forged/closed handles, stdout-not-readable and file-not-writable
(read-only direction) remain covered; the writer uses the same handle checks.
## What this does not prove
* No append/truncate/seek, directories, metadata, network, process, clock,
random, device or generic provider registry.
* No crash-atomic replace: `ProjectFs::write_bytes` is one validated write, not
a temp-file rename.
* No file write in `pure` or `strict`.
* No CM-A07 stdlib `print` migration.
* No Windows/macOS evidence and no remote CI URL.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-host --test resource_streams
cargo test -p marqdo-cli --test resource_streams
cargo test --workspace --locked
```
Loaded 100 of 214 files, more files were not shown because too many files have changed in this diff. Show more