Compare commits
35
Commits
a113d89496
...
328f1a8db3
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
328f1a8db3 | ||
|
|
e3638f5720 | ||
|
|
757c5ed15e | ||
|
|
313ac035e8 | ||
|
|
974102dfc3 | ||
|
|
5e5c3a84d9 | ||
|
|
d909de5c68 | ||
|
|
b22a89e7b4 | ||
|
|
01f6fe46a7 | ||
|
|
12be02e24b | ||
|
|
0946220f76 | ||
|
|
70d916d698 | ||
|
|
5ad2068038 | ||
|
|
8bbed34bd3 | ||
|
|
3484ab944c | ||
|
|
b241d1d978 | ||
|
|
e985da28a1 | ||
|
|
4c8e7fd8ab | ||
|
|
b020904199 | ||
|
|
e981168d92 | ||
|
|
0d4dc1da03 | ||
|
|
d6cd785bdb | ||
|
|
69a28ed1e1 | ||
|
|
67a6d35daf | ||
|
|
00bdcad1ff | ||
|
|
1d87f98539 | ||
|
|
1be4662049 | ||
|
|
a8e62a5ba9 | ||
|
|
49910ece43 | ||
|
|
e26971f489 | ||
|
|
e119727c5d | ||
|
|
1377af1601 | ||
|
|
1a978c7a42 | ||
|
|
5d3a5fb13b | ||
|
|
92bfbf2568 |
No files matched your search
@@ -20,3 +20,7 @@ dist/
|
||||
|
||||
# Local VM/QEMU tooling and disk images
|
||||
.tools/
|
||||
|
||||
# Local-only credentials and environment files
|
||||
.secrets/
|
||||
.env.local
|
||||
@@ -0,0 +1,22 @@
|
||||
# Woodpecker CI pipeline for the Rust workspace.
|
||||
# The strict security lane needs a disposable VM/self-hosted runner and is
|
||||
# intentionally not part of this docker pipeline; see docs/security-vm.md.
|
||||
when:
|
||||
event: [push, pull_request]
|
||||
|
||||
steps:
|
||||
test:
|
||||
image: rust:1.100-bookworm
|
||||
commands:
|
||||
- rustup component add clippy
|
||||
- cargo fmt --all -- --check
|
||||
- cargo clippy --workspace --all-targets -- -D warnings
|
||||
- cargo test --workspace --locked
|
||||
|
||||
build:
|
||||
image: rust:1.100-bookworm
|
||||
depends_on: [test]
|
||||
commands:
|
||||
- cargo build --release --locked -p marqdo-cli
|
||||
artifacts:
|
||||
- target/release/marqdo
|
||||
+140
@@ -43,9 +43,149 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
|
||||
without rewriting `marqdo.lock` or the materialized `.marqdo/env`.
|
||||
- G-C lane honours `CARGO_TARGET_DIR`, so a run from a clean `git archive` with
|
||||
an external target directory uses the binary it just built.
|
||||
- Addressable `Memory` (RFC-CM-0001 P1, ADR-0022): an opaque run-scoped
|
||||
region value with 64 KiB zero-filled pages, grow-only capacity, bounded
|
||||
byte read/write, `memory.new/size/grow`, and `m[offset] = byte`. Copies share
|
||||
the region identity; different regions never alias. Default budgets are
|
||||
64 MiB committed per run and 16 MiB per region; `--max-memory` and
|
||||
`--max-region-memory` override the budgets on `run`/`call`.
|
||||
- Memory-index place assignment in the expression/IR model
|
||||
(`AssignTarget::Index` / `PlaceTarget::Index`) with the frozen evaluation
|
||||
order base, index, RHS and no change to `List`/`Map`/`Table` value semantics.
|
||||
- Stable `runtime.memory` error category for invalid/stale handles and page
|
||||
counts; `runtime.index` for offsets, `runtime.type` for operand/byte-range
|
||||
errors, `resource.exhausted` for budgets.
|
||||
- Analysis v2 projects an index assignment target as an additive
|
||||
`{"kind":"index",...}` object while local targets keep the v1 string.
|
||||
- `examples/memory-buffer.mq.md`, semantic goldens 45–47 and runtime negatives
|
||||
13–18 cover sharing, isolation, growth, zero-fill, offsets, byte range,
|
||||
non-Memory index assignment, declared-max growth and negative page counts.
|
||||
- Partial RFC-CM P2 closure evidence (`.mq.md` only, no Rust algorithm
|
||||
additions): fixtures 48–50 implement a big-endian u32 codec, a bump
|
||||
allocator, node allocation and a linked-list traversal/sum, and prove that a
|
||||
failed allocation leaves the allocator header unchanged. This is **not** the
|
||||
full P2 gate: a separate stdlib package and a frozen-CLI closure lane are
|
||||
still missing.
|
||||
- `Memory` handles are never transferable: `to_wire` emits a bare rejected tag,
|
||||
`from_wire` fails, and every host/plugin/foreign call rejects arguments that
|
||||
contain a handle.
|
||||
- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
|
||||
`list.append/concat`, `map.keys/has/get/set/remove`,
|
||||
`text.codepoints/from_codepoint`, and `table.columns/row/cell`. All are pure,
|
||||
value-returning builtins; inputs are never mutated and no new syntax is
|
||||
added. Semantic fixtures 51–53 and runtime negatives 19–21 cover value
|
||||
semantics, Unicode scalars, table projection, wrong types and bounds.
|
||||
- `.mq.md` table filter (`过滤`) built on `table.columns/row/cell`,
|
||||
`list.append` and `表`, with a semantic golden showing the input table is
|
||||
unchanged (fixture 54). No `table.filter`/`HostOp` is added.
|
||||
- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): an ordinary Map
|
||||
`{ok:true,value}` / `{ok:false,code,message[,details]}`, constructed by
|
||||
`.mq.md` functions, with expected failures recoverable and language
|
||||
invariant violations still fatal. No `Result` type, exception or
|
||||
`try`/`catch` syntax.
|
||||
- `.mq.md` first-fit free-list allocator (fixture 55): 8-byte block headers,
|
||||
free-block reuse, block splitting, address-adjacent coalescing (tested in a
|
||||
second region whose bump space is exhausted, so only a merged block can
|
||||
satisfy the request), non-overlapping reused payloads and `alloc.no_space`
|
||||
returned as a `Result` without aborting the run. No allocator Rust builtin or
|
||||
`HostOp` was added.
|
||||
|
||||
- `.mq.md` P2 closures beyond the first slice: binary tree insert/in-order
|
||||
traversal (56), CSV row splitter (57), open-addressing hash table (58),
|
||||
UTF-8 encoder/validating decoder (59–60), graph adjacency list (61),
|
||||
recursive JSON parser for arrays/objects/strings (62), stack bytecode VM
|
||||
(63), 16/32/64-bit + LEB128 codecs (64), table sort (65), table equality join
|
||||
(67) and quoted/escaped CSV (68). No Rust algorithm, container or parser was
|
||||
added for these; algorithms remain `.mq.md` code. JSON float parsing and a
|
||||
compact formatter are included.
|
||||
- Resource/stream first slice (RFC-CM CM-A06, ADR-0025): opaque host-issued
|
||||
`Resource` handles, `resource.open/read/write/close`, bytes transferred
|
||||
through `Memory`, stdout write in pure/project under console/output budgets,
|
||||
granted project-file chunked read in project, EOF as success, closed-handle
|
||||
Result errors, forged-handle fatal errors, wire/host-boundary rejection, and
|
||||
strict refusing every resource open. Semantic fixture 66 and host/CLI tests.
|
||||
- Buffered project-file write streams (ADR-0026): `file` + `["write"]` requires
|
||||
an `fs_write` grant, buffers in the host, flushes only on `close`, never
|
||||
flushes on drop, has an 8 MiB ceiling, and is refused in pure/strict. A
|
||||
`.mq.md` chunked copy loop reproduces a file without a Rust `copy_file`.
|
||||
- Compat-C print completion (ADR-0031): remove `HostOp::Print` and
|
||||
`StdHost::op_print`; the print builtin only routes to bundled `io.写值` or
|
||||
returns `kernel.unavailable`; print traces contain only resource effects.
|
||||
- Compat-B `打印`/`print` routing (ADR-0030): a `Program.print_override` sends
|
||||
print calls to bundled `io.写值` when the program uses them; stdlib modules
|
||||
are injected lazily, resource operations appear as trace effects and the
|
||||
legacy `HostOp::Print` path does not run. Strict permits only stdio/stdout
|
||||
(existing console), not file/network resources. No Compat-C removal yet.
|
||||
- Full-value stdlib formatting (ADR-0029): FunctionRef `module/name` and
|
||||
PluginHandle `plugin/export` read-only members (host id private);
|
||||
`stdlib/format.mq.md` now reproduces builtin `打印`/`repr` for all 15 value
|
||||
types, including Formula/Code and UTF-8 byte length for Code.
|
||||
- CM-A10 bundled stdlib (ADR-0028): a fixed `stdlib/` tree with
|
||||
`format/text/io`, explicit `std:<path>` imports, SHA-256 digest verification
|
||||
before module loading, a `std:marqdo-stdlib@<version>#<digest>` package
|
||||
instance id, `project.stdlib` in Analysis v2, and test coverage for
|
||||
`std:` resolution, tamper rejection and frozen-binary hash equality.
|
||||
- P2 closure gate: add a pure no-Foreign/no-Plugin lane for C01–C06/C11/C12
|
||||
(20 fixtures + stdlib table), source shortcut scan and golden comparison.
|
||||
- CM-A10 offline packaging: `tools/pack-stdlib.py` emits a deterministic
|
||||
`marqdo-stdlib-0.1.0.tar.gz` plus tree/archive digests and a file manifest;
|
||||
the archive is for the future external repository, not the CLI loader.
|
||||
- P2 anti-evasion: explicit pure-lane negative tests prove `foreign.call` is
|
||||
`permission.denied` and `plugin.invoke` fails, alongside the 21 fixture lane.
|
||||
- C12 sparse matrix: fixture 73 adds triple-list sparse multiply and dense
|
||||
conversion, included in the P2 pure closure lane.
|
||||
- P6 baseline profile: add `tools/perf/bench.py` and a checked-in 7-workload
|
||||
JSON report; no bytecode/JIT/native backend is introduced.
|
||||
- P4 cross-storage table equivalence: fixture 72 compares row-store Table
|
||||
sort/filter output with a `.mq.md` columnar store cell-by-cell.
|
||||
- CM-A09 C ABI provider worker binding (ADR-0038): add the separate
|
||||
`marqdo-provider-worker` helper that loads a trusted C ABI library and serves
|
||||
the ADR-0035 framed-JSON protocol; crash isolation but not a sandbox.
|
||||
- CM-A09 provider package manifest v1 (ADR-0037): local manifest resolves a
|
||||
trusted C ABI library and registers it; no registry/signatures/CLI install.
|
||||
- CM-A09 C ABI provider v1 (ADR-0036): trusted shared libraries implement
|
||||
open/read/write/close/version/last_error; host loads them with version and
|
||||
symbol checks, project-mode + name grant only. Plugin-Worker binding remains
|
||||
open.
|
||||
- CM-A09 worker provider transport (ADR-0035): host-managed helper process
|
||||
speaking framed JSON for resource open/read/write/close; trusted-only,
|
||||
project-mode + name grant; C ABI binding still open.
|
||||
- P5 HTTP framing/client: `stdlib/net/http.mq.md` parses status/headers/chunked
|
||||
responses and builds requests; `http_over_tcp.rs` runs a loopback TCP HTTP
|
||||
exchange through `resource.open("tcp")`. No host HTTP/TLS implementation.
|
||||
- CM-A08 GFM rendering: `stdlib/data/table.mq.md::转GFM` renders a logical
|
||||
Table as a GFM pipe table with cell escaping; CLI test added.
|
||||
- P5 HTTP parser fixture (C10): semantic fixture 71 parses HTTP/1.1 status,
|
||||
headers and chunked transfer bodies and builds requests in `.mq.md`; no host
|
||||
HTTP/HTTPS implementation is claimed.
|
||||
- P5 blocking TCP endpoint v1 (ADR-0034): `resource.open("tcp", ip:port,
|
||||
["read","write"])` with exact IP-literal grants, project mode only, 5s
|
||||
connect/read/write timeouts; no DNS/TLS/UDP/server sockets. CLI `--allow-net
|
||||
<IP>`; loopback host and `.mq.md` echo tests. Strict/pure refuse.
|
||||
- CM-A09 in-process ResourceProvider v1 (ADR-0033): `ResourceProvider` trait,
|
||||
`StdHost::register_resource_provider`, per-kind registration with protocol
|
||||
version/conflict checks and `Capabilities.allow_resource_provider`; provider
|
||||
streams reuse `resource.*` and are project-mode only. A mock read/write
|
||||
provider test runs without parser/runtime/`HostOp` changes. C/Worker provider
|
||||
transport remains open.
|
||||
- CM-A08 table logical/stdlib slice (ADR-0032): `stdlib/data/table.mq.md`
|
||||
adds select/filter/sort/append/set-cell/join/group as value-returning
|
||||
`.mq.md` functions; `Value::Table` semantics and GFM AST laziness unchanged;
|
||||
no physical storage migration or new table builtin.
|
||||
- CM-A07 **Compat-A** prototype (ADR-0027): `stdlib/` provides
|
||||
`format.显示`, UTF-8 encoding and `io.写值` (`resource.write(stdout)`), and
|
||||
its output is comparison-tested byte-for-byte against builtin `打印` for
|
||||
Unit/None/Bool/Int/Float/Text/List/Map/Table. Builtin `打印` remains the
|
||||
default; no alias routing, bundled stdlib or `HostOp::Print` removal; the
|
||||
other value types lack public observation.
|
||||
|
||||
### Changed
|
||||
|
||||
- Assignment targets are no longer restricted to bare identifiers: a single
|
||||
index place is accepted at statement level. Member assignment stays a parse
|
||||
error; chained assignment stays a parse error. `spec/expression.ebnf`,
|
||||
`spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md` and
|
||||
`spec/security.md` document the new semantics.
|
||||
- `docs/STABLE_RELEASE_IMPLEMENTATION.md` is the single next-phase
|
||||
implementation plan. Older plan documents are historical input, not current
|
||||
status.
|
||||
|
||||
+3
-1
@@ -26,7 +26,9 @@ portable/tested locally but the native platform lane has not run here;
|
||||
| Feature | Ubuntu Linux x86_64 | macOS arm64 | Windows x86_64 |
|
||||
|---|---|---|---|
|
||||
| Markdown profile / AST goldens (160) | PASS | UNVERIFIED (cfg fix + CRLF fix) | UNVERIFIED (cfg fix + CRLF fix) |
|
||||
| Rust interpreter / semantics (44) | PASS | UNVERIFIED | UNVERIFIED |
|
||||
| Rust interpreter / semantics (70) | PASS | UNVERIFIED | UNVERIFIED |
|
||||
| Addressable `Memory` P1 (ADR-0022) | PASS | UNVERIFIED | UNVERIFIED |
|
||||
| Resource streams CM-A06 (ADR-0025/0026: stdout + project file read/write) | PASS | UNVERIFIED | **FAIL-CLOSED** (`strict` refuses resource opens) |
|
||||
| `run` / `call` / `check` / `inspect` / `fmt` | PASS | UNVERIFIED | UNVERIFIED |
|
||||
| lock v3 path/Git graph + sync + tamper | PASS | UNVERIFIED | UNVERIFIED |
|
||||
| Analysis v2 / Trace v2 + Python consumers | PASS | UNVERIFIED | UNVERIFIED |
|
||||
|
||||
Generated
+14
@@ -343,6 +343,7 @@ dependencies = [
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"tempfile",
|
||||
"walkdir",
|
||||
]
|
||||
|
||||
@@ -393,6 +394,7 @@ name = "marqdo-host"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"glob",
|
||||
"libloading",
|
||||
"marqdo-foreign",
|
||||
"marqdo-math",
|
||||
"marqdo-plugin-abi",
|
||||
@@ -474,6 +476,18 @@ dependencies = [
|
||||
"serde_json",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "marqdo-provider-worker"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"libloading",
|
||||
"marqdo-host",
|
||||
"marqdo-sandbox",
|
||||
"marqdo-value",
|
||||
"serde_json",
|
||||
"tempfile",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "marqdo-runtime"
|
||||
version = "0.1.0"
|
||||
|
||||
@@ -10,6 +10,7 @@ members = [
|
||||
"crates/marqdo-math",
|
||||
"crates/marqdo-foreign",
|
||||
"crates/marqdo-plugin-abi",
|
||||
"crates/marqdo-provider-worker",
|
||||
"crates/marqdo-plugin-sdk",
|
||||
"crates/marqdo-host",
|
||||
"crates/marqdo-runtime",
|
||||
|
||||
@@ -81,12 +81,59 @@ executes anything.
|
||||
## What works (implemented and tested)
|
||||
|
||||
* **P0 contracts** — frozen Markdown profile, AST → program mapping, strict
|
||||
expression grammar, evaluation semantics; 160 AST goldens, 44 semantic
|
||||
goldens, 52 compile-error fixtures and 12 runtime-error fixtures
|
||||
expression grammar, evaluation semantics; 160 AST goldens, 73 semantic
|
||||
goldens, 52 compile-error fixtures and 21 runtime-error fixtures
|
||||
(`tests/`).
|
||||
* **P1 interpreter** — native Rust tree-walk interpreter: values, scopes,
|
||||
functions, static parameter inference, assignments, branches, loops,
|
||||
returns, recursion, structured errors, fuel/stack/output budgets.
|
||||
* **Addressable memory (RFC-CM P1)** — opaque run-scoped `Memory` regions with
|
||||
64 KiB zero-filled pages, grow-only `memory.new/size/grow`, bounded byte
|
||||
read/write, `m[i] = byte`, run/per-region budgets, host-boundary and wire
|
||||
rejection (ADR-0022).
|
||||
* **Minimal structure primitives (RFC-CM CM-A04)** — value-semantics
|
||||
`list.append/concat`, `map.keys/has/get/set/remove`, Unicode scalar
|
||||
`text.codepoints/from_codepoint`, and non-negative `table.columns/row/cell`;
|
||||
pure builtins with no mutation or new syntax (ADR-0023).
|
||||
* **Recoverable results (RFC-CM CM-A05)** — `{ok:true,value}` /
|
||||
`{ok:false,code,message}` as an ordinary Map, with a `.mq.md` first-fit
|
||||
free-list allocator demonstrating reuse, splitting and `alloc.no_space`
|
||||
recovery (ADR-0024). No `Result` type or exception syntax.
|
||||
* **Bundled stdlib + print prototype (RFC-CM CM-A07/A10)** — offline
|
||||
`stdlib/` with explicit `std:io`/`std:format`/`std:text` imports, SHA-256
|
||||
digest verification, Analysis identity and a frozen-CLI closure test; the
|
||||
`.mq.md` print chain is the only execution path for `打印`/`print` on an official install (ADR-0031),
|
||||
covering all 15 value types; `HostOp::Print` and the legacy formatter are
|
||||
removed (ADR-0027/0028/0029/0030/0031).
|
||||
* **HTTP/1.1 framing in `.mq.md`** — `stdlib/net/http.mq.md` status/header/
|
||||
chunked parsing and request construction; a loopback TCP test runs a real
|
||||
exchange (ADR-0034; no TLS/host HTTP claim).
|
||||
* **Blocking TCP endpoint (RFC-CM P5 first slice)** — `resource.open("tcp",
|
||||
"<ip>:<port>", ["read","write"])` with exact IP grants, project mode and
|
||||
5-second timeouts; DNS/TLS/UDP/server sockets excluded (ADR-0034).
|
||||
* **C ABI provider worker (RFC-CM CM-A09)** — `marqdo-provider-worker` loads a
|
||||
trusted C ABI library in a separate framed-JSON helper process (ADR-0038).
|
||||
* **Provider package manifest v1 (RFC-CM CM-A09)** — local `marqdo.provider`
|
||||
manifest installs a trusted C ABI provider; no registry/signature/CLI install
|
||||
(ADR-0037).
|
||||
* **C ABI ResourceProvider v1 (RFC-CM CM-A09)** — trusted native shared
|
||||
libraries expose resource open/read/write/close through a frozen header;
|
||||
project + name grant; plugin-Worker binding remains open (ADR-0036).
|
||||
* **Worker ResourceProvider transport v1 (RFC-CM CM-A09)** — trusted helper
|
||||
process with framed JSON resource operations; project + name grant; C ABI
|
||||
binding remains open (ADR-0035).
|
||||
* **In-process ResourceProvider v1 (RFC-CM CM-A09 first slice)** — trusted
|
||||
providers register extra resource kinds, project-mode + name-grant gated,
|
||||
using the existing `resource.*`/`Result` ABI; no parser/runtime/`HostOp`
|
||||
change (ADR-0033).
|
||||
* **Table stdlib algorithms (RFC-CM CM-A08)** — `stdlib/data/table.mq.md`
|
||||
select/filter/sort/append/set/join/group over the logical `Table` value;
|
||||
value-returning, input-immutable, with GFM AST laziness unchanged (ADR-0032).
|
||||
* **Resource streams (RFC-CM CM-A06)** — opaque host-issued `Resource` handles;
|
||||
`resource.open/read/write/close` transfer bytes through `Memory`; stdout
|
||||
write in pure/project, granted project-file read, and buffered project-file
|
||||
write flushed on `close` in project; strict refuses all resource opens until
|
||||
verified (ADR-0025/0026).
|
||||
* **P2 references + CLI** — explicit YAML front-matter imports, `ModuleId`/
|
||||
`UnitId`/anchors, import cycle detection, multi-module calls,
|
||||
`run/call/check/inspect/fmt`, reference validation (`check --refs`) and the
|
||||
|
||||
@@ -33,3 +33,6 @@ semver = { workspace = true }
|
||||
[[bin]]
|
||||
name = "marqdo"
|
||||
path = "src/main.rs"
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = { workspace = true }
|
||||
@@ -8,7 +8,9 @@
|
||||
use std::path::Path;
|
||||
|
||||
use marqdo_expression::{BinaryOp, Expr, UnaryOp};
|
||||
use marqdo_ir::{CallTarget, Function, Module, ModuleId, Resource, SignatureKind, Stmt, UnitKind};
|
||||
use marqdo_ir::{
|
||||
CallTarget, Function, Module, ModuleId, PlaceTarget, Resource, SignatureKind, Stmt, UnitKind,
|
||||
};
|
||||
use marqdo_span::Span;
|
||||
use serde_json::{json, Value};
|
||||
use sha2::{Digest, Sha256};
|
||||
@@ -59,6 +61,11 @@ pub fn build(project: &LoadedProject) -> Value {
|
||||
"root_id": project.identity.root_id(),
|
||||
"identity_kind": project.identity.kind(),
|
||||
"lock_digest": lock_digest(project),
|
||||
"stdlib": project.stdlib.as_ref().map(|stdlib| json!({
|
||||
"version": stdlib.version,
|
||||
"digest": stdlib.digest,
|
||||
"instance_id": stdlib.instance_id,
|
||||
})),
|
||||
},
|
||||
"modules": modules,
|
||||
"diagnostics": diagnostics_json(project),
|
||||
@@ -232,13 +239,31 @@ fn function_json(function: &Function) -> Value {
|
||||
})
|
||||
}
|
||||
|
||||
/// Project an assignment place. Local targets keep the v1 string form; the new
|
||||
/// index target is an additive object so existing consumers that compare the
|
||||
/// local `target` string are unaffected (Analysis v2, ADR-0022).
|
||||
fn place_json(place: &PlaceTarget) -> Value {
|
||||
match place {
|
||||
PlaceTarget::Local { name, .. } => Value::String(name.clone()),
|
||||
PlaceTarget::Index { base, index, span } => json!({
|
||||
"kind": "index",
|
||||
"base": expr_json(base),
|
||||
"index": expr_json(index),
|
||||
"span": span_json(*span),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
fn stmt_json(stmt: &Stmt) -> Value {
|
||||
match stmt {
|
||||
Stmt::Assign {
|
||||
name, value, span, ..
|
||||
target,
|
||||
value,
|
||||
span,
|
||||
..
|
||||
} => json!({
|
||||
"kind": "assign",
|
||||
"target": name,
|
||||
"target": place_json(target),
|
||||
"value": expr_json(value),
|
||||
"span": span_json(*span),
|
||||
}),
|
||||
@@ -497,9 +522,14 @@ fn collect_calls(
|
||||
) {
|
||||
for stmt in body {
|
||||
match stmt {
|
||||
Stmt::Assign { value, .. } | Stmt::Eval { expr: value, .. } => {
|
||||
expr_calls(project, module, function, value, calls)
|
||||
Stmt::Assign { target, value, .. } => {
|
||||
if let PlaceTarget::Index { base, index, .. } = target {
|
||||
expr_calls(project, module, function, base, calls);
|
||||
expr_calls(project, module, function, index, calls);
|
||||
}
|
||||
expr_calls(project, module, function, value, calls);
|
||||
}
|
||||
Stmt::Eval { expr: value, .. } => expr_calls(project, module, function, value, calls),
|
||||
Stmt::Return { value, .. } => expr_calls(project, module, function, value, calls),
|
||||
Stmt::Call { call, span } => calls.push(json!({
|
||||
"function": function.id.to_string(),
|
||||
|
||||
@@ -4,7 +4,7 @@ use std::path::{Path, PathBuf};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use marqdo_host::{Capabilities, Host, StdHost};
|
||||
use marqdo_runtime::{run, EvaluatedArg, RuntimeError, Value};
|
||||
use marqdo_runtime::{run_with_limits, EvaluatedArg, MemoryLimits, RuntimeError, Value};
|
||||
use marqdo_sandbox::{PlatformCapabilities, SandboxMode};
|
||||
use marqdo_span::Span;
|
||||
use serde_json::json;
|
||||
@@ -22,6 +22,8 @@ pub struct SandboxSettings {
|
||||
pub allow_read: Vec<String>,
|
||||
/// Write grants.
|
||||
pub allow_write: Vec<String>,
|
||||
/// Network grants (IP literals, ADR-0034).
|
||||
pub allow_net: Vec<String>,
|
||||
/// Foreign execution grant.
|
||||
pub allow_foreign: bool,
|
||||
/// Plugin grants.
|
||||
@@ -32,6 +34,10 @@ pub struct SandboxSettings {
|
||||
pub stack: Option<usize>,
|
||||
/// Output limit override.
|
||||
pub max_output: Option<usize>,
|
||||
/// Total committed addressable-memory budget override (bytes).
|
||||
pub max_memory: Option<usize>,
|
||||
/// Per-region committed addressable-memory budget override (bytes).
|
||||
pub max_region_memory: Option<usize>,
|
||||
/// Wall-clock timeout in seconds.
|
||||
pub timeout: Option<u64>,
|
||||
/// Explicitly trusted plugin manifests to load.
|
||||
@@ -66,6 +72,7 @@ impl SandboxSettings {
|
||||
pub fn strict_preflight(&self) -> Result<(), String> {
|
||||
let requested = self.allow_foreign
|
||||
|| !self.allow_read.is_empty()
|
||||
|| !self.allow_net.is_empty()
|
||||
|| !self.allow_write.is_empty()
|
||||
|| !self.allow_plugin.is_empty()
|
||||
|| !self.plugins.is_empty()
|
||||
@@ -90,6 +97,7 @@ impl SandboxSettings {
|
||||
match mode {
|
||||
SandboxMode::Pure => {
|
||||
if !self.allow_read.is_empty()
|
||||
|| !self.allow_net.is_empty()
|
||||
|| !self.allow_write.is_empty()
|
||||
|| self.allow_foreign
|
||||
|| !self.allow_plugin.is_empty()
|
||||
@@ -108,6 +116,9 @@ impl SandboxSettings {
|
||||
for pattern in &self.allow_write {
|
||||
capabilities = capabilities.allow_write(pattern.clone());
|
||||
}
|
||||
for host in &self.allow_net {
|
||||
capabilities = capabilities.allow_net(host.clone());
|
||||
}
|
||||
if self.allow_foreign {
|
||||
capabilities = capabilities.allow_foreign();
|
||||
}
|
||||
@@ -349,7 +360,13 @@ pub fn check(
|
||||
let text = output::render_text(&project.diagnostics);
|
||||
eprint!("{text}");
|
||||
if errors == 0 {
|
||||
println!("checked {} module(s): ok", project.program.modules.len());
|
||||
let checked = project
|
||||
.program
|
||||
.modules
|
||||
.keys()
|
||||
.filter(|id| !id.instance.starts_with("std:"))
|
||||
.count();
|
||||
println!("checked {checked} module(s): ok");
|
||||
} else {
|
||||
eprintln!(
|
||||
"check failed: {} error(s), {} warning(s)",
|
||||
@@ -672,6 +689,10 @@ fn launch_strict_process(
|
||||
.unwrap_or(1024 * 1024)
|
||||
.min(64 * 1024 * 1024);
|
||||
let timeout = Duration::from_secs(settings.timeout.unwrap_or(30));
|
||||
let stdlib_dir = crate::stdlib::StdlibBundle::discover()
|
||||
.ok()
|
||||
.flatten()
|
||||
.map(|bundle| bundle.root().to_path_buf());
|
||||
let command = marqdo_sandbox::StrictHostCommand {
|
||||
program: &program,
|
||||
args: &args,
|
||||
@@ -680,6 +701,7 @@ fn launch_strict_process(
|
||||
original_document: document,
|
||||
output_limit,
|
||||
timeout,
|
||||
stdlib_dir: stdlib_dir.as_deref(),
|
||||
};
|
||||
match marqdo_sandbox::launch_strict(&command) {
|
||||
Ok(code) => code,
|
||||
@@ -884,12 +906,21 @@ pub fn execute(
|
||||
.iter()
|
||||
.map(|(name, value)| EvaluatedArg::named(name.clone(), value.clone(), Span::default()))
|
||||
.collect();
|
||||
let result = run(
|
||||
let memory_limits = MemoryLimits {
|
||||
max_total_bytes: settings
|
||||
.max_memory
|
||||
.unwrap_or(marqdo_runtime::DEFAULT_MAX_TOTAL_BYTES),
|
||||
max_region_bytes: settings
|
||||
.max_region_memory
|
||||
.unwrap_or(marqdo_runtime::DEFAULT_MAX_REGION_BYTES),
|
||||
};
|
||||
let result = run_with_limits(
|
||||
&project.program,
|
||||
&mut host,
|
||||
&project.entry,
|
||||
&function_name,
|
||||
runtime_args,
|
||||
memory_limits,
|
||||
);
|
||||
if let Some(path) = &settings.trace_file {
|
||||
let status = match &result {
|
||||
|
||||
+107
-2
@@ -13,13 +13,16 @@
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use marqdo_expression::Expr;
|
||||
use marqdo_frontend::{analyze_source, AnalysisResult};
|
||||
use marqdo_ir::{ModuleId, Program, ResolvedImport};
|
||||
use marqdo_ir::{ModuleId, PlaceTarget, Program, ResolvedImport, Stmt};
|
||||
use marqdo_package::{
|
||||
discover_root, LockFile, Manifest, PackageResolver, ResolvedPackage, LOCK_FILE, MANIFEST_FILE,
|
||||
};
|
||||
use marqdo_value::{codes, Diagnostic};
|
||||
|
||||
use crate::stdlib::StdlibBundle;
|
||||
|
||||
/// One analysed source file.
|
||||
pub struct LoadedFile {
|
||||
/// Canonical module identity (package instance + package-relative path).
|
||||
@@ -125,6 +128,17 @@ pub struct LocatedSource<'a> {
|
||||
pub column: usize,
|
||||
}
|
||||
|
||||
/// Identity of the verified bundled stdlib used by a load, if any.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct StdlibInfo {
|
||||
/// Declared version.
|
||||
pub version: String,
|
||||
/// Verified content digest.
|
||||
pub digest: String,
|
||||
/// Package instance id.
|
||||
pub instance_id: String,
|
||||
}
|
||||
|
||||
/// A fully loaded project ready for checking or running.
|
||||
pub struct LoadedProject {
|
||||
/// Project root.
|
||||
@@ -141,6 +155,8 @@ pub struct LoadedProject {
|
||||
pub packages: BTreeMap<String, PathBuf>,
|
||||
/// All diagnostics in deterministic order.
|
||||
pub diagnostics: Vec<FileDiagnostic>,
|
||||
/// Verified bundled stdlib identity, when one was available.
|
||||
pub stdlib: Option<StdlibInfo>,
|
||||
}
|
||||
|
||||
impl LoadedProject {
|
||||
@@ -205,6 +221,7 @@ struct Loader {
|
||||
program: Program,
|
||||
diagnostics: Vec<FileDiagnostic>,
|
||||
stack: Vec<PathBuf>,
|
||||
stdlib: Option<StdlibBundle>,
|
||||
}
|
||||
|
||||
/// Load the entry file and its explicitly imported modules.
|
||||
@@ -250,6 +267,7 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
|
||||
};
|
||||
let resolver = PackageResolver::new(root.clone(), manifest.clone(), lock);
|
||||
let entry_module = resolver.resolve_entry(&entry)?;
|
||||
let stdlib = StdlibBundle::discover().map_err(|error| format!("stdlib: {error}"))?;
|
||||
let mut loader = Loader {
|
||||
resolver,
|
||||
packages: BTreeMap::new(),
|
||||
@@ -257,9 +275,27 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
|
||||
program: Program::default(),
|
||||
diagnostics: Vec::new(),
|
||||
stack: Vec::new(),
|
||||
stdlib,
|
||||
};
|
||||
let entry_id = loader.load_module(&entry_module.package, &entry_module.relative_path)?;
|
||||
loader.program.main_module = Some(entry_id.clone());
|
||||
|
||||
// Compat-B (ADR-0030): only programs that actually call 打印/print need the
|
||||
// stdlib print module; injecting it for every program would change the
|
||||
// module graph, analysis output and startup cost for no benefit.
|
||||
if program_uses_print(&loader.program) {
|
||||
if let Some(stdlib) = loader.stdlib.clone() {
|
||||
let package = stdlib.package();
|
||||
let io = loader.load_module(&package, "io.mq.md")?;
|
||||
let Some(module) = loader.program.module(&io) else {
|
||||
return Err("stdlib print module was not loaded".to_string());
|
||||
};
|
||||
let Some(function) = module.function("写值") else {
|
||||
return Err("stdlib print module has no `写值` function".to_string());
|
||||
};
|
||||
loader.program.print_override = Some(function.id.clone());
|
||||
}
|
||||
}
|
||||
for (module_id, diagnostic) in marqdo_frontend::calls::validate_program(&loader.program) {
|
||||
if let Some(file) = loader.files.get(&module_id) {
|
||||
loader.diagnostics.push(FileDiagnostic {
|
||||
@@ -270,6 +306,11 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
|
||||
});
|
||||
}
|
||||
}
|
||||
let stdlib_info = loader.stdlib.as_ref().map(|bundle| StdlibInfo {
|
||||
version: bundle.version().to_string(),
|
||||
digest: bundle.digest().to_string(),
|
||||
instance_id: bundle.instance_id(),
|
||||
});
|
||||
let identity = match &manifest {
|
||||
Some(_) => ProjectIdentity::Manifest {
|
||||
instance: entry_id.instance.clone(),
|
||||
@@ -286,6 +327,7 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
|
||||
files: loader.files,
|
||||
packages: loader.packages,
|
||||
diagnostics: loader.diagnostics,
|
||||
stdlib: stdlib_info,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -360,7 +402,16 @@ impl Loader {
|
||||
.unwrap_or_default();
|
||||
|
||||
for (alias, target, span) in imports {
|
||||
let resolved = if let Some(alias_name) = target.strip_prefix('@') {
|
||||
let resolved = if let Some(std_target) = target.strip_prefix("std:") {
|
||||
match &self.stdlib {
|
||||
Some(stdlib) => stdlib.resolve(std_target),
|
||||
None => Err(format!(
|
||||
"no bundled stdlib is available for `{target}`; install the official \
|
||||
CLI bundle or set {STDLIB_DIR_ENV}",
|
||||
STDLIB_DIR_ENV = crate::stdlib::STDLIB_DIR_ENV
|
||||
)),
|
||||
}
|
||||
} else if let Some(alias_name) = target.strip_prefix('@') {
|
||||
self.resolver.resolve_alias_from(package, alias_name)
|
||||
} else {
|
||||
self.resolver.resolve_relative(package, &path, &target)
|
||||
@@ -466,3 +517,57 @@ impl Loader {
|
||||
Ok(module_id)
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the loaded program calls the legacy `打印`/`print` builtin.
|
||||
fn program_uses_print(program: &Program) -> bool {
|
||||
program.modules.values().any(|module| {
|
||||
module
|
||||
.functions
|
||||
.iter()
|
||||
.any(|function| stmts_use_print(&function.body))
|
||||
})
|
||||
}
|
||||
|
||||
fn stmts_use_print(stmts: &[Stmt]) -> bool {
|
||||
stmts.iter().any(|stmt| match stmt {
|
||||
Stmt::Assign { target, value, .. } => {
|
||||
let target_uses = match target {
|
||||
PlaceTarget::Local { .. } => false,
|
||||
PlaceTarget::Index { base, index, .. } => {
|
||||
expr_uses_print(base) || expr_uses_print(index)
|
||||
}
|
||||
};
|
||||
target_uses || expr_uses_print(value)
|
||||
}
|
||||
Stmt::Eval { expr, .. } | Stmt::Return { value: expr, .. } => expr_uses_print(expr),
|
||||
Stmt::Call { call, .. } => call.args.iter().any(|arg| expr_uses_print(&arg.value)),
|
||||
Stmt::Branch { arms, .. } => arms
|
||||
.iter()
|
||||
.any(|arm| expr_uses_print(&arm.condition) || stmts_use_print(&arm.body)),
|
||||
Stmt::Loop {
|
||||
condition, body, ..
|
||||
} => expr_uses_print(condition) || stmts_use_print(body),
|
||||
})
|
||||
}
|
||||
|
||||
fn expr_uses_print(expr: &Expr) -> bool {
|
||||
match expr {
|
||||
Expr::Call { callee, args, .. } => {
|
||||
matches!(callee.as_ref(), Expr::Ident(ident) if matches!(ident.name.as_str(), "打印" | "print"))
|
||||
|| expr_uses_print(callee)
|
||||
|| args.iter().any(|arg| expr_uses_print(&arg.value))
|
||||
}
|
||||
Expr::List(items) => items.iter().any(expr_uses_print),
|
||||
Expr::Record(entries) => entries.iter().any(|entry| expr_uses_print(&entry.value)),
|
||||
Expr::Unary { operand, .. } => expr_uses_print(operand),
|
||||
Expr::Binary { left, right, .. } => expr_uses_print(left) || expr_uses_print(right),
|
||||
Expr::Member { base, .. } => expr_uses_print(base),
|
||||
Expr::Index { base, index, .. } => expr_uses_print(base) || expr_uses_print(index),
|
||||
Expr::Int(_)
|
||||
| Expr::Float(_)
|
||||
| Expr::Text(_)
|
||||
| Expr::Bool(_)
|
||||
| Expr::None
|
||||
| Expr::Ident(_) => false,
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,7 @@ mod args;
|
||||
mod commands;
|
||||
mod loader;
|
||||
mod output;
|
||||
mod stdlib;
|
||||
mod tool;
|
||||
mod trace;
|
||||
|
||||
@@ -85,6 +86,9 @@ struct SandboxArgs {
|
||||
/// Grant read access to a project-relative glob (repeatable).
|
||||
#[arg(long = "allow-read", value_name = "GLOB")]
|
||||
allow_read: Vec<String>,
|
||||
/// Grant network access to an IP literal host (repeatable, ADR-0034).
|
||||
#[arg(long = "allow-net", value_name = "IP")]
|
||||
allow_net: Vec<String>,
|
||||
/// Grant write access to a project-relative glob (repeatable).
|
||||
#[arg(long = "allow-write", value_name = "GLOB")]
|
||||
allow_write: Vec<String>,
|
||||
@@ -103,6 +107,12 @@ struct SandboxArgs {
|
||||
/// Override the program output byte limit.
|
||||
#[arg(long = "max-output")]
|
||||
max_output: Option<usize>,
|
||||
/// Override the total committed addressable-memory budget in bytes.
|
||||
#[arg(long = "max-memory", value_name = "BYTES")]
|
||||
max_memory: Option<usize>,
|
||||
/// Override the per-region committed addressable-memory budget in bytes.
|
||||
#[arg(long = "max-region-memory", value_name = "BYTES")]
|
||||
max_region_memory: Option<usize>,
|
||||
/// Wall-clock timeout in seconds.
|
||||
#[arg(long)]
|
||||
timeout: Option<u64>,
|
||||
@@ -131,12 +141,15 @@ impl SandboxArgs {
|
||||
commands::SandboxSettings {
|
||||
mode: self.mode.clone(),
|
||||
allow_read: self.allow_read.clone(),
|
||||
allow_net: self.allow_net.clone(),
|
||||
allow_write: self.allow_write.clone(),
|
||||
allow_foreign: self.allow_foreign,
|
||||
allow_plugin: self.allow_plugin.clone(),
|
||||
fuel: self.fuel,
|
||||
stack: self.stack,
|
||||
max_output: self.max_output,
|
||||
max_memory: self.max_memory,
|
||||
max_region_memory: self.max_region_memory,
|
||||
timeout: self.timeout,
|
||||
plugins: self.plugin.clone(),
|
||||
trust_native_plugin: self.trust_native_plugin,
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
//! Bundled standard library discovery, digest verification and `std:` imports.
|
||||
//!
|
||||
//! The official CLI ships a fixed `marqdo-stdlib` tree (ADR-0026/CM-A10). This
|
||||
//! module finds that tree, verifies its declared content digest before any
|
||||
//! module is loaded, and resolves explicit `std:<relative>` imports to it.
|
||||
//! Nothing is fetched implicitly and there is no second module search path.
|
||||
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
|
||||
use marqdo_package::{PackageOrigin, ResolvedModule, ResolvedPackage};
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
/// Environment override used by tests and relocatable installations.
|
||||
pub const STDLIB_DIR_ENV: &str = "MARQDO_STDLIB_DIR";
|
||||
|
||||
/// Verified bundled standard library.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct StdlibBundle {
|
||||
root: PathBuf,
|
||||
name: String,
|
||||
version: String,
|
||||
digest: String,
|
||||
}
|
||||
|
||||
impl StdlibBundle {
|
||||
/// Discover the bundle.
|
||||
///
|
||||
/// Order: `MARQDO_STDLIB_DIR` (explicit, fail if invalid), a `stdlib`
|
||||
/// directory next to the executable, then the development tree.
|
||||
pub fn discover() -> Result<Option<Self>, String> {
|
||||
if let Some(dir) = std::env::var_os(STDLIB_DIR_ENV) {
|
||||
return Self::load(PathBuf::from(dir)).map(Some);
|
||||
}
|
||||
if let Ok(exe) = std::env::current_exe() {
|
||||
if let Some(parent) = exe.parent() {
|
||||
let candidate = parent.join("stdlib");
|
||||
if candidate.is_dir() {
|
||||
return Self::load(candidate).map(Some);
|
||||
}
|
||||
}
|
||||
}
|
||||
let dev = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||
.join("..")
|
||||
.join("..")
|
||||
.join("stdlib");
|
||||
if dev.is_dir() {
|
||||
return Self::load(dev).map(Some);
|
||||
}
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
/// Load and verify one bundle directory.
|
||||
pub fn load(root: impl Into<PathBuf>) -> Result<Self, String> {
|
||||
let root = root.into();
|
||||
let root = root.canonicalize().map_err(|error| {
|
||||
format!("cannot open stdlib directory `{}`: {error}", root.display())
|
||||
})?;
|
||||
let manifest_path = root.join("stdlib.json");
|
||||
let text = std::fs::read_to_string(&manifest_path)
|
||||
.map_err(|error| format!("cannot read `{}`: {error}", manifest_path.display()))?;
|
||||
let manifest: serde_json::Value = serde_json::from_str(&text)
|
||||
.map_err(|error| format!("invalid `{}`: {error}", manifest_path.display()))?;
|
||||
let name = manifest
|
||||
.get("name")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.ok_or_else(|| format!("`{}` has no name", manifest_path.display()))?
|
||||
.to_string();
|
||||
let version = manifest
|
||||
.get("version")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.ok_or_else(|| format!("`{}` has no version", manifest_path.display()))?
|
||||
.to_string();
|
||||
let declared = manifest
|
||||
.get("digest")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.ok_or_else(|| format!("`{}` has no digest", manifest_path.display()))?
|
||||
.to_string();
|
||||
let actual = tree_digest(&root)?;
|
||||
if declared != actual {
|
||||
return Err(format!(
|
||||
"stdlib digest mismatch: declared {declared}, actual {actual}"
|
||||
));
|
||||
}
|
||||
Ok(Self {
|
||||
root,
|
||||
name,
|
||||
version,
|
||||
digest: actual,
|
||||
})
|
||||
}
|
||||
|
||||
/// Bundle root.
|
||||
pub fn root(&self) -> &Path {
|
||||
&self.root
|
||||
}
|
||||
|
||||
/// Declared stdlib version.
|
||||
pub fn version(&self) -> &str {
|
||||
&self.version
|
||||
}
|
||||
|
||||
/// Verified content digest (`sha256:<hex>`).
|
||||
pub fn digest(&self) -> &str {
|
||||
&self.digest
|
||||
}
|
||||
|
||||
/// Logical package identity used for module ids.
|
||||
pub fn instance_id(&self) -> String {
|
||||
format!(
|
||||
"std:{}@{}#{}",
|
||||
self.name,
|
||||
self.version,
|
||||
self.digest.trim_start_matches("sha256:")
|
||||
)
|
||||
}
|
||||
|
||||
/// Package view consumed by the loader.
|
||||
pub fn package(&self) -> ResolvedPackage {
|
||||
ResolvedPackage {
|
||||
instance_id: self.instance_id(),
|
||||
name: self.name.clone(),
|
||||
version: self.version.clone(),
|
||||
origin: PackageOrigin::Path {
|
||||
declared: PathBuf::from("stdlib"),
|
||||
},
|
||||
revision: None,
|
||||
content_digest: Some(self.digest.clone()),
|
||||
module_root: self.root.clone(),
|
||||
entry_module: "index.mq.md".to_string(),
|
||||
manifest: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolve a `std:<relative>` target inside the bundle.
|
||||
pub fn resolve(&self, target: &str) -> Result<ResolvedModule, String> {
|
||||
let mut normalised = PathBuf::new();
|
||||
for component in Path::new(target).components() {
|
||||
match component {
|
||||
Component::Normal(part) => normalised.push(part),
|
||||
Component::CurDir => {}
|
||||
Component::ParentDir | Component::RootDir | Component::Prefix(_) => {
|
||||
return Err(format!(
|
||||
"stdlib import `{target}` must be a relative path inside the stdlib"
|
||||
))
|
||||
}
|
||||
}
|
||||
}
|
||||
if normalised.as_os_str().is_empty() {
|
||||
return Err("stdlib import must name a module".to_string());
|
||||
}
|
||||
if normalised.extension().is_none() {
|
||||
normalised.set_extension("mq.md");
|
||||
}
|
||||
let candidate = self.root.join(&normalised);
|
||||
let canonical = candidate.canonicalize().map_err(|error| {
|
||||
format!(
|
||||
"cannot open stdlib module `{target}` at `{}`: {error}",
|
||||
candidate.display()
|
||||
)
|
||||
})?;
|
||||
let relative = canonical
|
||||
.strip_prefix(&self.root)
|
||||
.map_err(|_| format!("stdlib import `{target}` escapes the stdlib root"))?;
|
||||
let relative_path = relative
|
||||
.components()
|
||||
.map(|component| component.as_os_str().to_string_lossy().to_string())
|
||||
.collect::<Vec<_>>()
|
||||
.join("/");
|
||||
Ok(ResolvedModule {
|
||||
package: self.package(),
|
||||
relative_path,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Content digest over all `.mq.md` files: sorted path, NUL, bytes, NUL.
|
||||
pub fn tree_digest(root: &Path) -> Result<String, String> {
|
||||
let mut files = Vec::new();
|
||||
collect_markdown(root, root, &mut files)?;
|
||||
files.sort();
|
||||
let mut hasher = Sha256::new();
|
||||
for (relative, path) in files {
|
||||
let bytes = std::fs::read(&path)
|
||||
.map_err(|error| format!("cannot read `{}`: {error}", path.display()))?;
|
||||
hasher.update(relative.as_bytes());
|
||||
hasher.update([0]);
|
||||
hasher.update(&bytes);
|
||||
hasher.update([0]);
|
||||
}
|
||||
Ok(format!("sha256:{}", hex(&hasher.finalize())))
|
||||
}
|
||||
|
||||
fn collect_markdown(
|
||||
root: &Path,
|
||||
dir: &Path,
|
||||
out: &mut Vec<(String, PathBuf)>,
|
||||
) -> Result<(), String> {
|
||||
let entries = std::fs::read_dir(dir)
|
||||
.map_err(|error| format!("cannot read `{}`: {error}", dir.display()))?;
|
||||
for entry in entries {
|
||||
let entry = entry.map_err(|error| format!("cannot read `{}`: {error}", dir.display()))?;
|
||||
let path = entry.path();
|
||||
if path.is_dir() {
|
||||
collect_markdown(root, &path, out)?;
|
||||
} else if path
|
||||
.file_name()
|
||||
.is_some_and(|name| name.to_string_lossy().ends_with(".mq.md"))
|
||||
{
|
||||
let relative = path
|
||||
.strip_prefix(root)
|
||||
.map_err(|_| "stdlib file escaped its root".to_string())?
|
||||
.components()
|
||||
.map(|component| component.as_os_str().to_string_lossy().to_string())
|
||||
.collect::<Vec<_>>()
|
||||
.join("/");
|
||||
out.push((relative, path));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn hex(bytes: &[u8]) -> String {
|
||||
let mut out = String::with_capacity(bytes.len() * 2);
|
||||
for byte in bytes {
|
||||
out.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn dev_bundle() -> PathBuf {
|
||||
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||
.join("..")
|
||||
.join("..")
|
||||
.join("stdlib")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dev_bundle_verifies_and_resolves() {
|
||||
let bundle = StdlibBundle::load(dev_bundle()).unwrap();
|
||||
assert!(bundle.digest.starts_with("sha256:"));
|
||||
let io = bundle.resolve("io").unwrap();
|
||||
assert_eq!(io.relative_path, "io.mq.md");
|
||||
assert!(io.package.instance_id.starts_with("std:marqdo-stdlib@"));
|
||||
assert!(bundle.resolve("../escape").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn digest_tamper_is_rejected() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let source = dev_bundle();
|
||||
for entry in std::fs::read_dir(&source).unwrap() {
|
||||
let entry = entry.unwrap();
|
||||
if entry.path().is_file() {
|
||||
std::fs::copy(entry.path(), dir.path().join(entry.file_name())).unwrap();
|
||||
}
|
||||
}
|
||||
std::fs::write(dir.path().join("io.mq.md"), "## tampered\n").unwrap();
|
||||
let error = StdlibBundle::load(dir.path()).unwrap_err();
|
||||
assert!(error.contains("digest mismatch"), "{error}");
|
||||
}
|
||||
}
|
||||
@@ -181,6 +181,34 @@ fn analysis_v2_identity_is_location_independent() {
|
||||
assert_eq!(analysis["project"]["identity_kind"], "manifest");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn analysis_v2_projects_memory_place_targets_additively() {
|
||||
let dir = TempDir::new("analysis-place");
|
||||
let source = "## main\n\n**内存 = memory.new(1, 2)**\n\n**`内存[0] = 7`**\n\n*内存[0]*\n";
|
||||
write(dir.path(), "index.mq.md", source);
|
||||
let target = dir.path().join("analysis.json");
|
||||
let output = cli(
|
||||
dir.path(),
|
||||
&["check", "index.mq.md", "--emit", target.to_str().unwrap()],
|
||||
);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let analysis: serde_json::Value =
|
||||
serde_json::from_slice(&std::fs::read(&target).unwrap()).unwrap();
|
||||
let body = analysis["modules"][0]["functions"][0]["body"]
|
||||
.as_array()
|
||||
.expect("body is an array");
|
||||
assert_eq!(body[0]["target"], "内存");
|
||||
assert_eq!(body[1]["target"]["kind"], "index");
|
||||
assert_eq!(body[1]["target"]["base"]["kind"], "variable");
|
||||
assert_eq!(body[1]["target"]["index"]["kind"], "literal");
|
||||
assert_eq!(body[1]["target"]["index"]["type"], "int");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn analysis_v2_projects_addressable_formula_and_code_resources() {
|
||||
let dir = TempDir::new("analysis-resources");
|
||||
|
||||
@@ -24,7 +24,7 @@ use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
use marqdo_host::{Capabilities, Host, StdHost};
|
||||
use marqdo_ir::ModuleId;
|
||||
use marqdo_ir::{ModuleId, Program, ResolvedImport};
|
||||
use marqdo_runtime::{run, EvaluatedArg};
|
||||
use marqdo_sandbox::SandboxMode;
|
||||
|
||||
@@ -168,6 +168,8 @@ fn semantic_goldens_match_end_to_end_execution() {
|
||||
let mut program = marqdo_ir::Program::default();
|
||||
program.insert(analysis.module.clone());
|
||||
program.main_module = Some(module_id.clone());
|
||||
install_stdlib_print(&mut program);
|
||||
install_entry_stdlib_imports(&mut program, &module_id, &analysis);
|
||||
|
||||
let mut capabilities = Capabilities::pure();
|
||||
if grant_foreign {
|
||||
@@ -263,3 +265,109 @@ fn semantic_goldens_match_end_to_end_execution() {
|
||||
fixtures.len()
|
||||
);
|
||||
}
|
||||
|
||||
/// Load the bundled stdlib print modules for direct-runtime fixtures.
|
||||
///
|
||||
/// The CLI loader normally injects these for Compat-B. The semantic harness
|
||||
/// builds a Program directly, so it mirrors the same resolution here; without
|
||||
/// it, fixtures that call `打印` would hit the Compat-C "no bundle" error.
|
||||
fn install_stdlib_print(program: &mut Program) {
|
||||
let root = repo_root().join("stdlib");
|
||||
if !root.join("stdlib.json").is_file() {
|
||||
return;
|
||||
}
|
||||
let mut loaded = std::collections::BTreeSet::new();
|
||||
let io = match load_stdlib_module(&root, "io.mq.md", program, &mut loaded) {
|
||||
Ok(id) => id,
|
||||
Err(error) => panic!("cannot load bundled stdlib: {error}"),
|
||||
};
|
||||
let function = program
|
||||
.module(&io)
|
||||
.and_then(|module| module.function("写值"))
|
||||
.unwrap_or_else(|| panic!("stdlib io has no 写值 function"));
|
||||
program.print_override = Some(function.id.clone());
|
||||
}
|
||||
|
||||
fn load_stdlib_module(
|
||||
root: &Path,
|
||||
relative: &str,
|
||||
program: &mut Program,
|
||||
loaded: &mut std::collections::BTreeSet<String>,
|
||||
) -> Result<ModuleId, String> {
|
||||
if !loaded.insert(relative.to_string()) {
|
||||
return Err(format!("stdlib import cycle at `{relative}`"));
|
||||
}
|
||||
let path = root.join(relative);
|
||||
let source = fs::read_to_string(&path)
|
||||
.map_err(|error| format!("cannot read `{}`: {error}", path.display()))?;
|
||||
let module_id = ModuleId::new("std:test", relative.replace('\\', "/"));
|
||||
let analysis = marqdo_frontend::analyze_source(&module_id.to_string(), &source);
|
||||
if analysis.has_errors() {
|
||||
return Err(format!("stdlib module `{relative}` has diagnostics"));
|
||||
}
|
||||
let imports: Vec<(String, String)> = analysis
|
||||
.document
|
||||
.frontmatter()
|
||||
.and_then(|frontmatter| frontmatter.imports().ok())
|
||||
.map(|mapping| mapping.into_iter().collect())
|
||||
.unwrap_or_default();
|
||||
for (alias, target) in imports {
|
||||
let target = target.strip_prefix("std:").unwrap_or(&target);
|
||||
let child = if target.starts_with("./") || target.starts_with("../") {
|
||||
let base = Path::new(relative)
|
||||
.parent()
|
||||
.unwrap_or_else(|| Path::new(""));
|
||||
base.join(target)
|
||||
.components()
|
||||
.map(|component| component.as_os_str().to_string_lossy().to_string())
|
||||
.collect::<Vec<_>>()
|
||||
.join("/")
|
||||
} else {
|
||||
target.to_string()
|
||||
};
|
||||
let to = load_stdlib_module(root, &child, program, loaded)?;
|
||||
program.resolved_imports.push(ResolvedImport {
|
||||
from: module_id.clone(),
|
||||
alias,
|
||||
to,
|
||||
});
|
||||
}
|
||||
program.insert(analysis.module);
|
||||
Ok(module_id)
|
||||
}
|
||||
|
||||
/// Resolve `std:` imports declared by a direct-runtime fixture entry.
|
||||
fn install_entry_stdlib_imports(
|
||||
program: &mut Program,
|
||||
module_id: &ModuleId,
|
||||
analysis: &marqdo_frontend::AnalysisResult,
|
||||
) {
|
||||
let root = repo_root().join("stdlib");
|
||||
let Some(imports) = analysis
|
||||
.document
|
||||
.frontmatter()
|
||||
.and_then(|frontmatter| frontmatter.imports().ok())
|
||||
else {
|
||||
return;
|
||||
};
|
||||
let mut loaded = std::collections::BTreeSet::new();
|
||||
for (alias, target) in imports {
|
||||
let Some(relative) = target.strip_prefix("std:") else {
|
||||
continue;
|
||||
};
|
||||
let relative = if relative.ends_with(".mq.md") {
|
||||
relative.to_string()
|
||||
} else {
|
||||
format!("{relative}.mq.md")
|
||||
};
|
||||
let to = match load_stdlib_module(&root, &relative, program, &mut loaded) {
|
||||
Ok(id) => id,
|
||||
Err(error) => panic!("cannot load stdlib import `{target}`: {error}"),
|
||||
};
|
||||
program.resolved_imports.push(ResolvedImport {
|
||||
from: module_id.clone(),
|
||||
alias,
|
||||
to,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -24,6 +24,58 @@ fn stderr(output: &Output) -> String {
|
||||
String::from_utf8_lossy(&output.stderr).to_string()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mqmd_stdlib_print_output_matches_builtin_print() {
|
||||
let output = cli(&["run", "stdlib/demo.mq.md"]);
|
||||
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
let lines: Vec<&str> = stdout.lines().collect();
|
||||
assert_eq!(lines.len(), 6, "{stdout:?}");
|
||||
for pair in [(0, 1), (2, 3), (4, 5)] {
|
||||
assert_eq!(
|
||||
lines[pair.0], lines[pair.1],
|
||||
"stdlib formatting differs from 打印 at line {}",
|
||||
pair.0
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
lines[0],
|
||||
r#"["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]"#
|
||||
);
|
||||
assert_eq!(lines[2], "Table[名称 | 分数]{甲 | 87; 乙 | 92}");
|
||||
assert_eq!(
|
||||
lines[4],
|
||||
r#"[Formula("a = 1"), Code("text", 9 bytes), FunctionRef(root:default::demo.mq.md::演示函数)]"#
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn addressable_memory_example_runs_through_the_cli() {
|
||||
let output = cli(&["run", "examples/memory-buffer.mq.md", "--json"]);
|
||||
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
|
||||
let report: serde_json::Value =
|
||||
serde_json::from_slice(&output.stdout).expect("run --json emits JSON");
|
||||
assert_eq!(report["success"], true);
|
||||
assert_eq!(report["value"]["type"], "int");
|
||||
assert_eq!(report["value"]["decimal"], "141");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cli_memory_budget_override_fails_closed() {
|
||||
let output = cli(&[
|
||||
"run",
|
||||
"examples/memory-buffer.mq.md",
|
||||
"--max-memory",
|
||||
"0",
|
||||
"--json",
|
||||
]);
|
||||
assert_eq!(output.status.code(), Some(1), "{}", stderr(&output));
|
||||
let report: serde_json::Value =
|
||||
serde_json::from_slice(&output.stdout).expect("run --json emits JSON");
|
||||
assert_eq!(report["success"], false);
|
||||
assert_eq!(report["error"]["code"], "resource.exhausted");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn multi_version_packages_run_without_an_implicit_override() {
|
||||
let output = cli(&["run", "examples/packages-multi-version/index.mq.md"]);
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
//! C10/P5: a `.mq.md` HTTP client over the TCP resource endpoint.
|
||||
|
||||
use std::io::{Read, Write};
|
||||
use std::net::TcpListener;
|
||||
use std::path::Path;
|
||||
use std::process::Command;
|
||||
|
||||
struct TempDir(std::path::PathBuf);
|
||||
|
||||
impl TempDir {
|
||||
fn new(tag: &str) -> Self {
|
||||
let path =
|
||||
std::env::temp_dir().join(format!("marqdo-http-tcp-{}-{tag}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&path);
|
||||
std::fs::create_dir_all(&path).unwrap();
|
||||
Self(path)
|
||||
}
|
||||
fn path(&self) -> &Path {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TempDir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mqmd_http_client_parses_a_response_from_a_tcp_stream() {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
let address = listener.local_addr().unwrap();
|
||||
let server = std::thread::spawn(move || {
|
||||
let (mut stream, _) = listener.accept().unwrap();
|
||||
let mut request = [0u8; 512];
|
||||
let count = stream.read(&mut request).unwrap();
|
||||
assert!(String::from_utf8_lossy(&request[..count]).starts_with("GET / HTTP/1.1\r\n"));
|
||||
stream
|
||||
.write_all(
|
||||
b"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 5\r\n\r\nhello",
|
||||
)
|
||||
.unwrap();
|
||||
});
|
||||
|
||||
let dir = TempDir::new("client");
|
||||
let program = format!(
|
||||
r#"---
|
||||
import:
|
||||
http: "std:net/http"
|
||||
编码: "std:text"
|
||||
---
|
||||
|
||||
## main
|
||||
|
||||
**流 = resource.open("tcp", "{address}", ["read", "write"])**
|
||||
|
||||
**请求 = http.构建请求("GET", "/", [["Host", "example.test"]], "")**
|
||||
|
||||
**请求内存 = memory.new(1, 1)**
|
||||
|
||||
**请求长度 = 编码.utf8编码(请求, 请求内存, 0)**
|
||||
|
||||
**写入 = resource.write(流, 请求内存, 0, 请求长度)**
|
||||
|
||||
**接收 = memory.new(1, 1)**
|
||||
|
||||
**读出 = resource.read(流, 接收, 0, 4096)**
|
||||
|
||||
**数量 = 读出.value.count**
|
||||
|
||||
**响应文本 = ""**
|
||||
|
||||
**下标 = 0**
|
||||
|
||||
- `下标 < 数量`
|
||||
|
||||
**响应文本 = 响应文本 + text.from_codepoint(接收[下标])**
|
||||
|
||||
**下标 = 下标 + 1**
|
||||
|
||||
**关闭 = resource.close(流)**
|
||||
|
||||
**响应 = http.解析响应(响应文本)**
|
||||
|
||||
*[写入.ok, 读出.ok, 响应.状态, 响应.头["Content-Type"], 响应.体, 关闭.ok]*
|
||||
"#
|
||||
);
|
||||
std::fs::write(dir.path().join("index.mq.md"), program).unwrap();
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", "index.mq.md", "--allow-net", "127.0.0.1", "--json"])
|
||||
.current_dir(dir.path())
|
||||
.env(
|
||||
"MARQDO_STDLIB_DIR",
|
||||
Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.unwrap()
|
||||
.parent()
|
||||
.unwrap()
|
||||
.join("stdlib"),
|
||||
)
|
||||
.output()
|
||||
.expect("run marqdo");
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"stderr={} stdout={}",
|
||||
String::from_utf8_lossy(&output.stderr),
|
||||
String::from_utf8_lossy(&output.stdout)
|
||||
);
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
|
||||
assert_eq!(report["success"], true);
|
||||
let items = report["value"]["items"].as_array().expect("list");
|
||||
assert_eq!(items[0]["value"], true);
|
||||
assert_eq!(items[1]["value"], true);
|
||||
assert_eq!(items[2]["decimal"], "200");
|
||||
assert_eq!(items[3]["value"], "text/plain");
|
||||
assert_eq!(items[4]["value"], "hello");
|
||||
assert_eq!(items[5]["value"], true);
|
||||
server.join().unwrap();
|
||||
}
|
||||
@@ -0,0 +1,159 @@
|
||||
//! RFC-CM P2 closure lane: C01–C06/C11/C12 in a pure, no-Foreign/no-Plugin host.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
fn repo_root() -> PathBuf {
|
||||
Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.unwrap()
|
||||
.parent()
|
||||
.unwrap()
|
||||
.to_path_buf()
|
||||
}
|
||||
|
||||
fn fixture(name: &str) -> PathBuf {
|
||||
repo_root()
|
||||
.join("tests")
|
||||
.join("conformance")
|
||||
.join("semantics")
|
||||
.join(name)
|
||||
}
|
||||
|
||||
fn expected(name: &str) -> serde_json::Value {
|
||||
let path = fixture(&name.replace(".mq.md", ".expected.json"));
|
||||
serde_json::from_slice(&std::fs::read(&path).expect("expected golden")).expect("golden json")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn p2_algorithms_run_in_a_pure_no_foreign_no_plugin_host() {
|
||||
// P2 gate (RFC-CM-0002 §8): C01-C06, C11, C12 use only .mq.md plus the
|
||||
// declared low-level primitives. These fixtures are run through the real
|
||||
// CLI in `--mode pure`, which has no foreign/plugin grants, and the source
|
||||
// is checked for host shortcuts that would violate the anti-evasion rule.
|
||||
let cases: &[(&str, &str)] = &[
|
||||
("48-memory-bump-allocator.mq.md", "C01 bump allocator"),
|
||||
(
|
||||
"50-memory-allocation-failure-atomic.mq.md",
|
||||
"C01 allocation atomicity",
|
||||
),
|
||||
(
|
||||
"55-first-fit-free-list-allocator.mq.md",
|
||||
"C01 free-list split/coalesce",
|
||||
),
|
||||
("49-memory-linked-list.mq.md", "C02 linked list"),
|
||||
("56-memory-binary-tree.mq.md", "C02 tree"),
|
||||
("61-memory-graph-adjacency.mq.md", "C02 graph"),
|
||||
("58-memory-hash-table.mq.md", "C02 hash table"),
|
||||
("64-memory-integer-codecs.mq.md", "C03 codecs"),
|
||||
("59-utf8-encoder.mq.md", "C04 UTF-8 encode"),
|
||||
("60-utf8-decoder.mq.md", "C04 UTF-8 decode"),
|
||||
("57-csv-row-splitter.mq.md", "C05 CSV"),
|
||||
("62-json-array-parser.mq.md", "C05 JSON"),
|
||||
("68-csv-quoted-row.mq.md", "C05 CSV quoted"),
|
||||
("53-table-projection.mq.md", "C06 table projection"),
|
||||
("54-table-filter-stdlib.mq.md", "C06 table filter"),
|
||||
("65-table-sort-stdlib.mq.md", "C06 table sort"),
|
||||
("67-table-join-stdlib.mq.md", "C06 table join"),
|
||||
("69-table-group-stdlib.mq.md", "C06 table group"),
|
||||
("63-memory-bytecode-vm.mq.md", "C11 bytecode VM"),
|
||||
("70-matrix-multiply.mq.md", "C12 matrix"),
|
||||
("73-sparse-matrix.mq.md", "C12 sparse matrix"),
|
||||
];
|
||||
for (name, criterion) in cases {
|
||||
let path = fixture(name);
|
||||
let source = std::fs::read_to_string(&path).expect("fixture source");
|
||||
for forbidden in ["foreign.", "plugin.", "import:"] {
|
||||
assert!(
|
||||
!source.contains(forbidden),
|
||||
"{criterion} fixture {name} uses forbidden host shortcut `{forbidden}`"
|
||||
);
|
||||
}
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", path.to_str().unwrap(), "--mode", "pure", "--json"])
|
||||
.current_dir(repo_root())
|
||||
.output()
|
||||
.expect("run marqdo");
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{criterion} fixture {name} failed in pure mode: {}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let report: serde_json::Value =
|
||||
serde_json::from_slice(&output.stdout).expect("json report");
|
||||
assert_eq!(report["success"], true, "{name}");
|
||||
let golden = expected(name);
|
||||
assert_eq!(
|
||||
report["value"], golden["value"],
|
||||
"{criterion} fixture {name}"
|
||||
);
|
||||
assert_eq!(
|
||||
report["stdout"].as_str().unwrap_or_default(),
|
||||
golden["stdout"].as_str().unwrap_or_default(),
|
||||
"{criterion} fixture {name} stdout"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn p2_stdlib_table_also_runs_in_pure_mode() {
|
||||
let path = repo_root().join("stdlib").join("tests").join("table.mq.md");
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", path.to_str().unwrap(), "--mode", "pure", "--json"])
|
||||
.current_dir(repo_root().join("stdlib"))
|
||||
.env("MARQDO_STDLIB_DIR", repo_root().join("stdlib"))
|
||||
.output()
|
||||
.expect("run table self-test");
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
|
||||
assert_eq!(report["success"], true);
|
||||
let items = report["value"]["items"]
|
||||
.as_array()
|
||||
.expect("table test list");
|
||||
assert_eq!(items.len(), 9);
|
||||
assert_eq!(
|
||||
items[8]["value"],
|
||||
"| 名称 | 分数 |\n| --- | --- |\n| 甲 | 87 |\n| 乙 | 92 |\n| 丙 | 71 |"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn foreign_and_plugin_shortcuts_are_unavailable_in_the_pure_lane() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let foreign = dir.path().join("foreign.mq.md");
|
||||
std::fs::write(
|
||||
&foreign,
|
||||
"## 代码\n\n```python\nprint(1)\n```\n\n## main\n\n*foreign.call(ref(\"#代码\"), None)*\n",
|
||||
)
|
||||
.unwrap();
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", foreign.to_str().unwrap(), "--mode", "pure", "--json"])
|
||||
.current_dir(dir.path())
|
||||
.output()
|
||||
.expect("run foreign fixture");
|
||||
assert_eq!(output.status.code(), Some(1));
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
|
||||
assert_eq!(report["success"], false);
|
||||
assert_eq!(report["error"]["code"], "permission.denied");
|
||||
|
||||
let plugin = dir.path().join("plugin.mq.md");
|
||||
std::fs::write(
|
||||
&plugin,
|
||||
"## main\n\n> plugin.invoke(\"missing.export\", {})\n\n*None*\n",
|
||||
)
|
||||
.unwrap();
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", plugin.to_str().unwrap(), "--mode", "pure", "--json"])
|
||||
.current_dir(dir.path())
|
||||
.output()
|
||||
.expect("run plugin fixture");
|
||||
assert_eq!(output.status.code(), Some(1));
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
|
||||
assert_eq!(report["success"], false);
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
//! Compat-B `打印` routing tests (ADR-0030).
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Output};
|
||||
|
||||
struct TempDir(PathBuf);
|
||||
|
||||
impl TempDir {
|
||||
fn new(tag: &str) -> Self {
|
||||
let path =
|
||||
std::env::temp_dir().join(format!("marqdo-print-route-{}-{tag}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&path);
|
||||
std::fs::create_dir_all(&path).unwrap();
|
||||
Self(path)
|
||||
}
|
||||
fn path(&self) -> &Path {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TempDir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn cli(dir: &Path, args: &[&str]) -> Output {
|
||||
Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(args)
|
||||
.current_dir(dir)
|
||||
.output()
|
||||
.expect("run marqdo")
|
||||
}
|
||||
|
||||
fn write(dir: &Path, name: &str, source: &str) -> PathBuf {
|
||||
let path = dir.join(name);
|
||||
std::fs::write(&path, source).unwrap();
|
||||
path
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn print_is_routed_through_the_stdlib_and_not_hostop_print() {
|
||||
let dir = TempDir::new("routed");
|
||||
write(
|
||||
dir.path(),
|
||||
"index.mq.md",
|
||||
"## main\n\n> 打印([\"x\", 1])\n\n*None*\n",
|
||||
);
|
||||
let trace = dir.path().join("trace.json");
|
||||
let output = cli(
|
||||
dir.path(),
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--trace-file",
|
||||
trace.to_str().unwrap(),
|
||||
],
|
||||
);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
assert_eq!(String::from_utf8_lossy(&output.stdout), "[\"x\", 1]\n");
|
||||
|
||||
let trace: serde_json::Value = serde_json::from_slice(&std::fs::read(&trace).unwrap()).unwrap();
|
||||
let operations: Vec<&str> = trace["events"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter_map(|event| event["operation"].as_str())
|
||||
.collect();
|
||||
assert!(
|
||||
operations.contains(&"resource.write"),
|
||||
"stdlib print must write through the resource ABI: {operations:?}"
|
||||
);
|
||||
assert!(
|
||||
!operations.contains(&"print"),
|
||||
"legacy HostOp::Print must not run in Compat-B: {operations:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stdlib_print_modules_are_injected_only_when_print_is_used() {
|
||||
let dir = TempDir::new("lazy");
|
||||
write(dir.path(), "plain.mq.md", "## main\n\n*1*\n");
|
||||
write(
|
||||
dir.path(),
|
||||
"prints.mq.md",
|
||||
"## main\n\n> 打印(1)\n\n*None*\n",
|
||||
);
|
||||
for (file, expected_stdlib) in [("plain.mq.md", false), ("prints.mq.md", true)] {
|
||||
let target = dir.path().join(format!("{file}.json"));
|
||||
let output = cli(
|
||||
dir.path(),
|
||||
&["check", file, "--emit", target.to_str().unwrap()],
|
||||
);
|
||||
assert_eq!(output.status.code(), Some(0), "{file}");
|
||||
let analysis: serde_json::Value =
|
||||
serde_json::from_slice(&std::fs::read(&target).unwrap()).unwrap();
|
||||
let modules = analysis["modules"].as_array().unwrap();
|
||||
let has_stdlib = modules
|
||||
.iter()
|
||||
.any(|module| module["id"].as_str().unwrap_or("").starts_with("std:"));
|
||||
assert_eq!(has_stdlib, expected_stdlib, "{file}: {modules:?}");
|
||||
// The bundle identity is recorded whenever a bundle is available; it
|
||||
// does not imply that this program loaded any stdlib module.
|
||||
assert!(analysis["project"]["stdlib"].is_object(), "{file}");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,297 @@
|
||||
//! End-to-end resource stream tests through the CLI.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Output};
|
||||
|
||||
struct TempDir(PathBuf);
|
||||
|
||||
impl TempDir {
|
||||
fn new(tag: &str) -> Self {
|
||||
let path =
|
||||
std::env::temp_dir().join(format!("marqdo-resource-test-{}-{tag}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&path);
|
||||
std::fs::create_dir_all(&path).unwrap();
|
||||
Self(path)
|
||||
}
|
||||
|
||||
fn path(&self) -> &Path {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TempDir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn cli(dir: &Path, args: &[&str]) -> Output {
|
||||
Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(args)
|
||||
.current_dir(dir)
|
||||
.output()
|
||||
.expect("run marqdo")
|
||||
}
|
||||
|
||||
fn write_file(path: &Path, contents: &str) {
|
||||
std::fs::write(path, contents).unwrap();
|
||||
}
|
||||
|
||||
const READER: &str = r#"## main
|
||||
|
||||
**内存 = memory.new(1, 1)**
|
||||
|
||||
**流 = resource.open("file", "data.txt", ["read"])**
|
||||
|
||||
**第一次 = resource.read(流, 内存, 0, 2)**
|
||||
|
||||
**第二次 = resource.read(流, 内存, 2, 4)**
|
||||
|
||||
**第三次 = resource.read(流, 内存, 6, 1)**
|
||||
|
||||
**关闭 = resource.close(流)**
|
||||
|
||||
*[第一次.value.count, 第一次.value.eof, 第二次.value.count, 第二次.value.eof, 第三次.value.count, 第三次.value.eof, 内存[0], 内存[1], 内存[2], 内存[3], 内存[4], 内存[5], 关闭.ok]*
|
||||
"#;
|
||||
|
||||
#[test]
|
||||
fn project_file_stream_reads_chunks_into_memory() {
|
||||
let dir = TempDir::new("file-read");
|
||||
write_file(&dir.path().join("data.txt"), "abcdef");
|
||||
write_file(&dir.path().join("index.mq.md"), READER);
|
||||
let output = cli(
|
||||
dir.path(),
|
||||
&["run", "index.mq.md", "--allow-read", "data.txt", "--json"],
|
||||
);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
|
||||
assert_eq!(report["success"], true);
|
||||
let values: Vec<&serde_json::Value> = report["value"]["items"]
|
||||
.as_array()
|
||||
.expect("list result")
|
||||
.iter()
|
||||
.collect();
|
||||
assert_eq!(values[0]["decimal"], "2");
|
||||
assert_eq!(values[1]["value"], false);
|
||||
assert_eq!(values[2]["decimal"], "4");
|
||||
assert_eq!(values[3]["value"], true);
|
||||
assert_eq!(values[4]["decimal"], "0");
|
||||
assert_eq!(values[5]["value"], true);
|
||||
for (index, expected) in ["97", "98", "99", "100", "101", "102"]
|
||||
.into_iter()
|
||||
.enumerate()
|
||||
{
|
||||
assert_eq!(values[6 + index]["decimal"], expected);
|
||||
}
|
||||
assert_eq!(values[12]["value"], true);
|
||||
}
|
||||
|
||||
const WRITER: &str = r#"## main
|
||||
|
||||
**内存 = memory.new(1, 1)**
|
||||
|
||||
**`内存[0] = 65`**
|
||||
|
||||
**`内存[1] = 66`**
|
||||
|
||||
**`内存[2] = 67`**
|
||||
|
||||
**写 = resource.open("file", "out.bin", ["write"])**
|
||||
|
||||
**写入 = resource.write(写, 内存, 0, 3)**
|
||||
|
||||
**关闭写 = resource.close(写)**
|
||||
|
||||
**读 = resource.open("file", "out.bin", ["read"])**
|
||||
|
||||
**读出 = resource.read(读, 内存, 0, 3)**
|
||||
|
||||
**关闭读 = resource.close(读)**
|
||||
|
||||
*[写入.value.count, 关闭写.ok, 读出.value.count, 读出.value.eof, 内存[0], 内存[1], 内存[2], 关闭读.ok]*
|
||||
"#;
|
||||
|
||||
#[test]
|
||||
fn project_file_stream_writes_on_close_then_reads_back() {
|
||||
let dir = TempDir::new("file-write");
|
||||
write_file(&dir.path().join("index.mq.md"), WRITER);
|
||||
let output = cli(
|
||||
dir.path(),
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--allow-write",
|
||||
"out.bin",
|
||||
"--allow-read",
|
||||
"out.bin",
|
||||
"--json",
|
||||
],
|
||||
);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
|
||||
assert_eq!(report["success"], true);
|
||||
let values = report["value"]["items"].as_array().expect("list result");
|
||||
assert_eq!(values[0]["decimal"], "3");
|
||||
assert_eq!(values[1]["value"], true);
|
||||
assert_eq!(values[2]["decimal"], "3");
|
||||
assert_eq!(values[3]["value"], true);
|
||||
assert_eq!(values[4]["decimal"], "65");
|
||||
assert_eq!(values[5]["decimal"], "66");
|
||||
assert_eq!(values[6]["decimal"], "67");
|
||||
assert_eq!(values[7]["value"], true);
|
||||
assert_eq!(std::fs::read(dir.path().join("out.bin")).unwrap(), b"ABC");
|
||||
}
|
||||
|
||||
const COPY: &str = r#"## main
|
||||
|
||||
**输入 = resource.open("file", "data.txt", ["read"])**
|
||||
|
||||
**输出 = resource.open("file", "out.bin", ["write"])**
|
||||
|
||||
**内存 = memory.new(1, 1)**
|
||||
|
||||
**总数 = 0**
|
||||
|
||||
**继续 = true**
|
||||
|
||||
- `继续`
|
||||
|
||||
**块 = resource.read(输入, 内存, 0, 16)**
|
||||
|
||||
1. `块.ok`
|
||||
|
||||
> resource.write(输出, 内存, 0, 块.value.count)
|
||||
|
||||
**总数 = 总数 + 块.value.count**
|
||||
|
||||
**继续 = not 块.value.eof**
|
||||
|
||||
2. `true`
|
||||
|
||||
**继续 = false**
|
||||
|
||||
**关闭读 = resource.close(输入)**
|
||||
|
||||
**关闭写 = resource.close(输出)**
|
||||
|
||||
*[总数, 关闭读.ok, 关闭写.ok]*
|
||||
"#;
|
||||
|
||||
#[test]
|
||||
fn file_copy_is_written_in_mqmd_through_chunked_streams() {
|
||||
let dir = TempDir::new("file-copy");
|
||||
write_file(&dir.path().join("data.txt"), "abcdef");
|
||||
write_file(&dir.path().join("index.mq.md"), COPY);
|
||||
let output = cli(
|
||||
dir.path(),
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--allow-read",
|
||||
"data.txt",
|
||||
"--allow-write",
|
||||
"out.bin",
|
||||
"--json",
|
||||
],
|
||||
);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
|
||||
assert_eq!(report["success"], true);
|
||||
let values = report["value"]["items"].as_array().expect("list result");
|
||||
assert_eq!(values[0]["decimal"], "6");
|
||||
assert_eq!(values[1]["value"], true);
|
||||
assert_eq!(values[2]["value"], true);
|
||||
assert_eq!(
|
||||
std::fs::read(dir.path().join("out.bin")).unwrap(),
|
||||
b"abcdef"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tcp_stream_round_trips_through_mqmd_with_a_network_grant() {
|
||||
use std::io::{Read, Write};
|
||||
use std::net::TcpListener;
|
||||
|
||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
let address = listener.local_addr().unwrap();
|
||||
let server = std::thread::spawn(move || {
|
||||
let (mut stream, _) = listener.accept().unwrap();
|
||||
let mut buffer = [0u8; 4];
|
||||
stream.read_exact(&mut buffer).unwrap();
|
||||
stream.write_all(&buffer).unwrap();
|
||||
});
|
||||
|
||||
let dir = TempDir::new("tcp");
|
||||
let program = format!(
|
||||
r#"## main
|
||||
|
||||
**内存 = memory.new(1, 1)**
|
||||
|
||||
**`内存[0] = 112`**
|
||||
|
||||
**`内存[1] = 105`**
|
||||
|
||||
**`内存[2] = 110`**
|
||||
|
||||
**`内存[3] = 103`**
|
||||
|
||||
**流 = resource.open("tcp", "{address}", ["read", "write"])**
|
||||
|
||||
**写入 = resource.write(流, 内存, 0, 4)**
|
||||
|
||||
**读出 = resource.read(流, 内存, 0, 4)**
|
||||
|
||||
**关闭 = resource.close(流)**
|
||||
|
||||
*[写入.value.count, 读出.value.count, 内存[0], 内存[1], 内存[2], 内存[3], 关闭.ok]*
|
||||
"#
|
||||
);
|
||||
write_file(&dir.path().join("index.mq.md"), &program);
|
||||
let output = cli(
|
||||
dir.path(),
|
||||
&["run", "index.mq.md", "--allow-net", "127.0.0.1", "--json"],
|
||||
);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
|
||||
assert_eq!(report["success"], true);
|
||||
let values = report["value"]["items"].as_array().expect("list");
|
||||
assert_eq!(values[0]["decimal"], "4");
|
||||
assert_eq!(values[1]["decimal"], "4");
|
||||
for (index, expected) in ["112", "105", "110", "103"].into_iter().enumerate() {
|
||||
assert_eq!(values[2 + index]["decimal"], expected);
|
||||
}
|
||||
assert_eq!(values[6]["value"], true);
|
||||
server.join().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn file_stream_requires_a_read_grant() {
|
||||
let dir = TempDir::new("file-denied");
|
||||
write_file(&dir.path().join("data.txt"), "abcdef");
|
||||
write_file(&dir.path().join("index.mq.md"), READER);
|
||||
let output = cli(dir.path(), &["run", "index.mq.md", "--json"]);
|
||||
assert_eq!(output.status.code(), Some(1));
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
|
||||
assert_eq!(report["success"], false);
|
||||
assert_eq!(report["error"]["code"], "permission.denied");
|
||||
}
|
||||
@@ -0,0 +1,199 @@
|
||||
//! Bundled stdlib (`std:`) and frozen-CLI closure tests (CM-A10).
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
fn repo_root() -> PathBuf {
|
||||
Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.unwrap()
|
||||
.parent()
|
||||
.unwrap()
|
||||
.to_path_buf()
|
||||
}
|
||||
|
||||
fn binary() -> &'static str {
|
||||
env!("CARGO_BIN_EXE_marqdo")
|
||||
}
|
||||
|
||||
fn stdlib_dir() -> PathBuf {
|
||||
repo_root().join("stdlib")
|
||||
}
|
||||
|
||||
fn run(dir: &Path, stdlib: &Path, args: &[&str]) -> std::process::Output {
|
||||
Command::new(binary())
|
||||
.args(args)
|
||||
.current_dir(dir)
|
||||
.env("MARQDO_STDLIB_DIR", stdlib)
|
||||
.output()
|
||||
.expect("run marqdo")
|
||||
}
|
||||
|
||||
struct TempDir(PathBuf);
|
||||
|
||||
impl TempDir {
|
||||
fn new(tag: &str) -> Self {
|
||||
let path = std::env::temp_dir().join(format!("marqdo-stdlib-{}-{tag}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&path);
|
||||
std::fs::create_dir_all(&path).unwrap();
|
||||
Self(path)
|
||||
}
|
||||
fn path(&self) -> &Path {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TempDir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn copy_stdlib(source: &Path, target: &Path) {
|
||||
std::fs::create_dir_all(target).unwrap();
|
||||
for entry in std::fs::read_dir(source).unwrap() {
|
||||
let entry = entry.unwrap();
|
||||
let path = entry.path();
|
||||
if path.is_dir() {
|
||||
copy_stdlib(&path, &target.join(entry.file_name()));
|
||||
} else {
|
||||
std::fs::copy(&path, target.join(entry.file_name())).unwrap();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn write_project(dir: &Path, program: &str) {
|
||||
std::fs::write(dir.join("index.mq.md"), program).unwrap();
|
||||
}
|
||||
|
||||
const IMPORT_IO: &str = r#"---
|
||||
import:
|
||||
io: "std:io"
|
||||
---
|
||||
|
||||
## main
|
||||
|
||||
> io.写值(["bundle", 1, 2.5])
|
||||
|
||||
*None*
|
||||
"#;
|
||||
|
||||
#[test]
|
||||
fn std_import_resolves_the_verified_bundle() {
|
||||
let dir = TempDir::new("resolve");
|
||||
write_project(dir.path(), IMPORT_IO);
|
||||
let output = run(dir.path(), &stdlib_dir(), &["run", "index.mq.md"]);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
assert_eq!(
|
||||
String::from_utf8_lossy(&output.stdout),
|
||||
"[\"bundle\", 1, 2.5]\n"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn analysis_records_the_verified_stdlib_digest() {
|
||||
let dir = TempDir::new("analysis");
|
||||
write_project(dir.path(), IMPORT_IO);
|
||||
let target = dir.path().join("analysis.json");
|
||||
let output = run(
|
||||
dir.path(),
|
||||
&stdlib_dir(),
|
||||
&["check", "index.mq.md", "--emit", target.to_str().unwrap()],
|
||||
);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let analysis: serde_json::Value =
|
||||
serde_json::from_slice(&std::fs::read(&target).unwrap()).unwrap();
|
||||
let manifest: serde_json::Value =
|
||||
serde_json::from_slice(&std::fs::read(stdlib_dir().join("stdlib.json")).unwrap()).unwrap();
|
||||
assert_eq!(
|
||||
analysis["project"]["stdlib"]["version"],
|
||||
manifest["version"]
|
||||
);
|
||||
assert_eq!(analysis["project"]["stdlib"]["digest"], manifest["digest"]);
|
||||
assert!(analysis["project"]["stdlib"]["instance_id"]
|
||||
.as_str()
|
||||
.unwrap()
|
||||
.starts_with("std:marqdo-stdlib@"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn frozen_binary_hash_is_unchanged_after_stdlib_execution() {
|
||||
let before = std::fs::read(binary()).unwrap();
|
||||
let before_hash = marqdo_value_sha(&before);
|
||||
let dir = TempDir::new("frozen");
|
||||
write_project(dir.path(), IMPORT_IO);
|
||||
let output = run(dir.path(), &stdlib_dir(), &["run", "index.mq.md"]);
|
||||
assert_eq!(output.status.code(), Some(0));
|
||||
let after = std::fs::read(binary()).unwrap();
|
||||
assert_eq!(before_hash, marqdo_value_sha(&after));
|
||||
}
|
||||
|
||||
fn marqdo_value_sha(bytes: &[u8]) -> String {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(bytes);
|
||||
hasher
|
||||
.finalize()
|
||||
.iter()
|
||||
.map(|byte| format!("{byte:02x}"))
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tampered_bundle_is_rejected_before_module_loading() {
|
||||
let dir = TempDir::new("tamper");
|
||||
let bundle = dir.path().join("stdlib");
|
||||
copy_stdlib(&stdlib_dir(), &bundle);
|
||||
std::fs::write(bundle.join("io.mq.md"), "## tampered\n").unwrap();
|
||||
write_project(dir.path(), IMPORT_IO);
|
||||
let output = run(dir.path(), &bundle, &["run", "index.mq.md"]);
|
||||
assert!(!output.status.success());
|
||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
||||
assert!(stderr.contains("stdlib digest mismatch"), "{stderr}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bundle_uses_no_foreign_or_plugin_calls() {
|
||||
fn visit(dir: &Path, offenders: &mut Vec<String>) {
|
||||
for entry in std::fs::read_dir(dir).unwrap() {
|
||||
let entry = entry.unwrap();
|
||||
let path = entry.path();
|
||||
if path.is_dir() {
|
||||
visit(&path, offenders);
|
||||
} else if path.to_string_lossy().ends_with(".mq.md") {
|
||||
let source = std::fs::read_to_string(&path).unwrap();
|
||||
if source.contains("foreign.") || source.contains("plugin.") {
|
||||
offenders.push(path.display().to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut offenders = Vec::new();
|
||||
visit(&stdlib_dir(), &mut offenders);
|
||||
assert!(
|
||||
offenders.is_empty(),
|
||||
"stdlib bundle must not depend on foreign/plugin calls: {offenders:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_bundle_manifest_is_a_hard_error() {
|
||||
let dir = TempDir::new("missing");
|
||||
let empty = dir.path().join("empty-stdlib");
|
||||
std::fs::create_dir_all(&empty).unwrap();
|
||||
write_project(dir.path(), IMPORT_IO);
|
||||
let output = run(dir.path(), &empty, &["run", "index.mq.md"]);
|
||||
assert!(!output.status.success());
|
||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
||||
assert!(stderr.contains("stdlib"), "{stderr}");
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
//! CM-A08/P5 stdlib HTTP framing module tests.
|
||||
|
||||
use std::path::Path;
|
||||
use std::process::Command;
|
||||
|
||||
fn repo_root() -> std::path::PathBuf {
|
||||
Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.unwrap()
|
||||
.parent()
|
||||
.unwrap()
|
||||
.to_path_buf()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stdlib_http_parses_chunked_responses_and_builds_requests() {
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", "tests/http.mq.md", "--json"])
|
||||
.current_dir(repo_root().join("stdlib"))
|
||||
.env("MARQDO_STDLIB_DIR", repo_root().join("stdlib"))
|
||||
.output()
|
||||
.expect("run http self-test");
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json");
|
||||
assert_eq!(report["success"], true);
|
||||
let items = report["value"]["items"].as_array().expect("list");
|
||||
assert_eq!(items[0]["decimal"], "200");
|
||||
assert_eq!(items[1]["value"], "OK");
|
||||
assert_eq!(items[2]["value"], "text/plain");
|
||||
assert_eq!(items[3]["value"], "hello world");
|
||||
assert_eq!(
|
||||
items[4]["value"],
|
||||
"POST /echo HTTP/1.1\r\nHost: example.test\r\nContent-Length: 2\r\n\r\nok"
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
//! CM-A08 table stdlib module tests.
|
||||
|
||||
use std::path::Path;
|
||||
use std::process::Command;
|
||||
|
||||
fn repo_root() -> std::path::PathBuf {
|
||||
Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.unwrap()
|
||||
.parent()
|
||||
.unwrap()
|
||||
.to_path_buf()
|
||||
}
|
||||
|
||||
fn run_table_self_test() -> serde_json::Value {
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", "tests/table.mq.md", "--json"])
|
||||
.current_dir(repo_root().join("stdlib"))
|
||||
.env("MARQDO_STDLIB_DIR", repo_root().join("stdlib"))
|
||||
.output()
|
||||
.expect("run table self-test");
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
serde_json::from_slice(&output.stdout).expect("json output")
|
||||
}
|
||||
|
||||
fn rows(value: &serde_json::Value) -> &Vec<serde_json::Value> {
|
||||
value["rows"].as_array().expect("rows")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stdlib_table_algorithms_rebuild_without_mutating_inputs() {
|
||||
let report = run_table_self_test();
|
||||
assert_eq!(report["success"], true);
|
||||
let items = report["value"]["items"].as_array().expect("list");
|
||||
assert_eq!(items.len(), 9);
|
||||
|
||||
assert_eq!(items[0]["header"], serde_json::json!(["分数"]));
|
||||
assert_eq!(items[0]["rows"].as_array().unwrap().len(), 3);
|
||||
|
||||
assert_eq!(rows(&items[1]).len(), 2);
|
||||
assert_eq!(rows(&items[2]).len(), 3);
|
||||
assert_eq!(rows(&items[2])[0][1]["decimal"], "71");
|
||||
assert_eq!(rows(&items[3]).len(), 4);
|
||||
assert_eq!(rows(&items[3])[3][0]["value"], "丁");
|
||||
assert_eq!(rows(&items[4])[0][1]["decimal"], "99");
|
||||
|
||||
assert_eq!(rows(&items[5]).len(), 2);
|
||||
assert_eq!(items[5]["header"].as_array().unwrap().len(), 4);
|
||||
|
||||
assert_eq!(
|
||||
items[6]["entries"]["a"]["items"].as_array().unwrap().len(),
|
||||
2
|
||||
);
|
||||
assert_eq!(
|
||||
items[6]["entries"]["b"]["items"].as_array().unwrap().len(),
|
||||
1
|
||||
);
|
||||
|
||||
// Original input is unchanged.
|
||||
assert_eq!(rows(&items[7]).len(), 3);
|
||||
assert_eq!(rows(&items[7])[0][1]["decimal"], "87");
|
||||
|
||||
assert_eq!(
|
||||
items[8]["value"],
|
||||
"| 名称 | 分数 |\n| --- | --- |\n| 甲 | 87 |\n| 乙 | 92 |\n| 丙 | 71 |"
|
||||
);
|
||||
}
|
||||
@@ -195,15 +195,50 @@ impl Expr {
|
||||
}
|
||||
}
|
||||
|
||||
/// The payload of a statement.
|
||||
/// The writable target on the left of `=` (ADR-0022).
|
||||
///
|
||||
/// v1 accepted only [`AssignTarget::Local`]. RFC-CM-0001 adds the second
|
||||
/// variant because addressable `Memory` needs a writable place; no other
|
||||
/// collection becomes mutable. Both variants use the same assignment syntax
|
||||
/// and the same statement position.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub enum StatementKind {
|
||||
/// `<bare identifier> = <expression>`; the only assignment form in v1.
|
||||
Assign {
|
||||
#[serde(tag = "place", rename_all = "snake_case")]
|
||||
pub enum AssignTarget {
|
||||
/// `name = value`: rebind a local slot in the current frame.
|
||||
Local {
|
||||
/// Assigned identifier.
|
||||
name: String,
|
||||
/// Span of the identifier token (not the whole assignment).
|
||||
name_span: Span,
|
||||
},
|
||||
/// `base[index] = value`: write one byte into an addressable `Memory`.
|
||||
Index {
|
||||
/// Base expression, evaluated first.
|
||||
base: Box<Expr>,
|
||||
/// Index expression, evaluated second.
|
||||
index: Box<Expr>,
|
||||
/// Span from the base start through the closing `]`.
|
||||
span: Span,
|
||||
},
|
||||
}
|
||||
|
||||
impl AssignTarget {
|
||||
/// Best known source span of the assignment target.
|
||||
pub fn span(&self) -> Span {
|
||||
match self {
|
||||
AssignTarget::Local { name_span, .. } => *name_span,
|
||||
AssignTarget::Index { span, .. } => *span,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The payload of a statement.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub enum StatementKind {
|
||||
/// `target = expression`; see [`AssignTarget`] for the writable places.
|
||||
Assign {
|
||||
/// Assigned place.
|
||||
target: AssignTarget,
|
||||
/// Right-hand side expression.
|
||||
value: Expr,
|
||||
},
|
||||
|
||||
@@ -36,8 +36,8 @@
|
||||
//! The grammar is frozen in `spec/expression.ebnf`. In order of decreasing
|
||||
//! precedence: prefix `not` / `-` / `+`, `**` (right associative), postfix
|
||||
//! calls `.name` / `[index]`, `* / // %`, `+ -`, a single non-associative
|
||||
//! comparison, `and`, `or`. Assignment is only available at statement start
|
||||
//! and only for a bare identifier target.
|
||||
//! comparison, `and`, `or`. Assignment is only available at statement start;
|
||||
//! its target is a bare identifier or an index expression (`m[8] = 42`).
|
||||
//!
|
||||
//! Notable lexical rules:
|
||||
//!
|
||||
@@ -84,7 +84,9 @@ use std::fmt;
|
||||
|
||||
use serde::Serialize;
|
||||
|
||||
pub use ast::{Arg, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp};
|
||||
pub use ast::{
|
||||
Arg, AssignTarget, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp,
|
||||
};
|
||||
pub use marqdo_span::Span;
|
||||
pub use parser::MAX_NESTING_DEPTH;
|
||||
|
||||
|
||||
@@ -17,7 +17,9 @@
|
||||
//! recursing into any sub-expression so adversarial inputs fail with a
|
||||
//! structured [`ParseError`] instead of overflowing the stack.
|
||||
|
||||
use crate::ast::{Arg, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp};
|
||||
use crate::ast::{
|
||||
Arg, AssignTarget, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp,
|
||||
};
|
||||
use crate::lexer::{self, Token, TokenKind};
|
||||
use crate::ParseError;
|
||||
use marqdo_span::Span;
|
||||
@@ -47,21 +49,16 @@ pub(crate) fn parse_statement(source: &str) -> Result<Statement, ParseError> {
|
||||
return Err(parser.error_here("expected a statement, found end of input"));
|
||||
}
|
||||
let start = parser.peek().span.start;
|
||||
if let TokenKind::Ident(_) = parser.peek().kind {
|
||||
if matches!(
|
||||
parser.peek_n(1).map(|token| &token.kind),
|
||||
Some(TokenKind::Assign)
|
||||
) {
|
||||
return parser.assignment(start);
|
||||
}
|
||||
let left = parser.expression()?;
|
||||
if matches!(parser.peek().kind, TokenKind::Assign) {
|
||||
return parser.assignment(start, left);
|
||||
}
|
||||
let expr = parser.expression()?;
|
||||
if !parser.at_eof() {
|
||||
return Err(parser.trailing_statement_error(&expr));
|
||||
return Err(parser.trailing_statement_error());
|
||||
}
|
||||
let end = parser.previous_end();
|
||||
Ok(Statement {
|
||||
kind: StatementKind::Expr(expr),
|
||||
kind: StatementKind::Expr(left),
|
||||
span: Span::new(start, end),
|
||||
})
|
||||
}
|
||||
@@ -201,11 +198,8 @@ impl Parser {
|
||||
// Statements
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
fn assignment(&mut self, start: usize) -> Result<Statement, ParseError> {
|
||||
let TokenKind::Ident(name) = self.peek().kind.clone() else {
|
||||
return Err(self.error_here("expected an identifier on the left of `=`"));
|
||||
};
|
||||
let name_token = self.bump();
|
||||
fn assignment(&mut self, start: usize, left: Expr) -> Result<Statement, ParseError> {
|
||||
let target = self.assign_target(left)?;
|
||||
self.bump(); // `=`
|
||||
if self.at_eof() {
|
||||
return Err(self.error_here("expected an expression after `=`"));
|
||||
@@ -226,29 +220,40 @@ impl Parser {
|
||||
}
|
||||
let end = self.previous_end();
|
||||
Ok(Statement {
|
||||
kind: StatementKind::Assign {
|
||||
name,
|
||||
name_span: name_token.span,
|
||||
value,
|
||||
},
|
||||
kind: StatementKind::Assign { target, value },
|
||||
span: Span::new(start, end),
|
||||
})
|
||||
}
|
||||
|
||||
fn trailing_statement_error(&self, expr: &Expr) -> ParseError {
|
||||
if matches!(self.peek().kind, TokenKind::Assign) {
|
||||
return if matches!(expr, Expr::Member { .. } | Expr::Index { .. }) {
|
||||
self.error_here(
|
||||
"invalid assignment target: member and index assignment are not supported \
|
||||
in v1; assign a bare identifier",
|
||||
)
|
||||
} else {
|
||||
self.error_here(
|
||||
"invalid assignment target: assignment requires a bare identifier on the \
|
||||
left of `=`",
|
||||
)
|
||||
};
|
||||
/// Turn the already-parsed left expression into a writable target.
|
||||
///
|
||||
/// v1 only writes local slots. Addressable `Memory` adds index targets;
|
||||
/// member assignment and every other shape stay hard parse errors so an
|
||||
/// invalid executable shape never silently becomes prose (ADR-0022).
|
||||
fn assign_target(&self, left: Expr) -> Result<AssignTarget, ParseError> {
|
||||
match left {
|
||||
Expr::Ident(Ident { name, span }) => Ok(AssignTarget::Local {
|
||||
name,
|
||||
name_span: span,
|
||||
}),
|
||||
Expr::Index { base, index, span } => Ok(AssignTarget::Index { base, index, span }),
|
||||
Expr::Member { span, .. } => Err(ParseError::new(
|
||||
span,
|
||||
"member assignment is not supported in v1; only a local name or a \
|
||||
Memory index is writable",
|
||||
)),
|
||||
other => Err(ParseError::new(
|
||||
other.span(),
|
||||
format!(
|
||||
"invalid assignment target: {} is not a writable place; assign a local \
|
||||
name or a Memory index",
|
||||
describe_expr(&other)
|
||||
),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn trailing_statement_error(&self) -> ParseError {
|
||||
self.error_here(format!(
|
||||
"unexpected {} after expression",
|
||||
self.describe_current()
|
||||
@@ -705,6 +710,24 @@ fn is_keyword(kind: &TokenKind) -> bool {
|
||||
)
|
||||
}
|
||||
|
||||
fn describe_expr(expr: &Expr) -> &'static str {
|
||||
match expr {
|
||||
Expr::Int(_) => "an integer literal",
|
||||
Expr::Float(_) => "a float literal",
|
||||
Expr::Text(_) => "a text literal",
|
||||
Expr::Bool(_) => "a boolean literal",
|
||||
Expr::None => "`None`",
|
||||
Expr::Ident(_) => "an identifier",
|
||||
Expr::List(_) => "a list literal",
|
||||
Expr::Record(_) => "a record literal",
|
||||
Expr::Unary { .. } => "a unary expression",
|
||||
Expr::Binary { .. } => "a binary expression",
|
||||
Expr::Call { .. } => "a call expression",
|
||||
Expr::Member { .. } => "a member access",
|
||||
Expr::Index { .. } => "an index expression",
|
||||
}
|
||||
}
|
||||
|
||||
fn describe_token(kind: &TokenKind) -> String {
|
||||
match kind {
|
||||
TokenKind::Int(raw) => format!("integer literal `{raw}`"),
|
||||
@@ -1154,14 +1177,19 @@ mod tests {
|
||||
assert_eq!(
|
||||
stmt.kind,
|
||||
StatementKind::Assign {
|
||||
name: "x".to_owned(),
|
||||
name_span: Span::new(0, 1),
|
||||
target: AssignTarget::Local {
|
||||
name: "x".to_owned(),
|
||||
name_span: Span::new(0, 1),
|
||||
},
|
||||
value: Expr::Int(BigInt::from(1)),
|
||||
}
|
||||
);
|
||||
let stmt = st("结果 = 阶乘(n=3)");
|
||||
match &stmt.kind {
|
||||
StatementKind::Assign { name, value, .. } => {
|
||||
StatementKind::Assign {
|
||||
target: AssignTarget::Local { name, .. },
|
||||
value,
|
||||
} => {
|
||||
assert_eq!(name, "结果");
|
||||
assert_eq!(sexpr(value), "(call 阶乘 n=3)");
|
||||
}
|
||||
@@ -1170,6 +1198,40 @@ mod tests {
|
||||
assert_eq!(st("x = y + 1").span(), Span::new(0, 9));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn memory_index_assignment_parses_as_a_place() {
|
||||
let stmt = st("m[8] = 42");
|
||||
let StatementKind::Assign { target, value } = &stmt.kind else {
|
||||
panic!("expected assignment, got {:?}", stmt.kind);
|
||||
};
|
||||
let AssignTarget::Index { base, index, span } = target else {
|
||||
panic!("expected index target, got {target:?}");
|
||||
};
|
||||
assert_eq!(
|
||||
**base,
|
||||
Expr::Ident(Ident {
|
||||
name: "m".to_owned(),
|
||||
span: Span::new(0, 1),
|
||||
})
|
||||
);
|
||||
assert_eq!(**index, Expr::Int(BigInt::from(8)));
|
||||
assert_eq!(*span, Span::new(0, 4));
|
||||
assert_eq!(*value, Expr::Int(BigInt::from(42)));
|
||||
assert_eq!(stmt.span, Span::new(0, 9));
|
||||
|
||||
// Nested targets are syntactically places; the runtime decides whether
|
||||
// the base value is a writable Memory region.
|
||||
let stmt = st("outer[0][1] = 2");
|
||||
let StatementKind::Assign {
|
||||
target: AssignTarget::Index { base, .. },
|
||||
..
|
||||
} = &stmt.kind
|
||||
else {
|
||||
panic!("expected nested index target, got {:?}", stmt.kind);
|
||||
};
|
||||
assert!(matches!(base.as_ref(), Expr::Index { .. }));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn expression_statements_parse() {
|
||||
assert_eq!(st("打印(1)").kind, StatementKind::Expr(ex("打印(1)")));
|
||||
@@ -1185,8 +1247,8 @@ mod tests {
|
||||
"x =",
|
||||
"x = 1 = 2",
|
||||
"a.b = 1",
|
||||
"a[0] = 1",
|
||||
"1 = 2",
|
||||
"f() = 1",
|
||||
"= 1",
|
||||
"x = 1 2",
|
||||
] {
|
||||
@@ -1194,7 +1256,8 @@ mod tests {
|
||||
assert_eq!(err.code, "expr.parse", "input {bad:?}");
|
||||
}
|
||||
assert!(st_err("x = 1 = 2").message.contains("chained assignment"));
|
||||
assert!(st_err("a.b = 1").message.contains("member and index"));
|
||||
assert!(st_err("a.b = 1").message.contains("member assignment"));
|
||||
assert!(st_err("1 = 2").message.contains("writable place"));
|
||||
assert!(st_err("x =").message.contains("after `=`"));
|
||||
}
|
||||
|
||||
@@ -1228,7 +1291,10 @@ mod tests {
|
||||
let stmt = st(" x = 1 ");
|
||||
assert_eq!(stmt.span(), Span::new(2, 7));
|
||||
match stmt.kind {
|
||||
StatementKind::Assign { name_span, .. } => assert_eq!(name_span, Span::new(2, 3)),
|
||||
StatementKind::Assign {
|
||||
target: AssignTarget::Local { name_span, .. },
|
||||
..
|
||||
} => assert_eq!(name_span, Span::new(2, 3)),
|
||||
other => panic!("expected assignment, got {other:?}"),
|
||||
}
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use marqdo_expression::{Arg, Expr, Ident};
|
||||
use marqdo_ir::{builtins, Module, ModuleId, Param, Program, SignatureKind, Stmt};
|
||||
use marqdo_ir::{builtins, Module, ModuleId, Param, PlaceTarget, Program, SignatureKind, Stmt};
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{codes, Diagnostic};
|
||||
|
||||
@@ -109,7 +109,10 @@ pub fn validate_program(program: &Program) -> Vec<(ModuleId, Diagnostic)> {
|
||||
fn collect_assigned(stmts: &[Stmt], out: &mut BTreeSet<String>) {
|
||||
for stmt in stmts {
|
||||
match stmt {
|
||||
Stmt::Assign { name, .. } => {
|
||||
Stmt::Assign {
|
||||
target: PlaceTarget::Local { name, .. },
|
||||
..
|
||||
} => {
|
||||
out.insert(name.clone());
|
||||
}
|
||||
Stmt::Branch { arms, .. } => {
|
||||
@@ -138,9 +141,14 @@ impl<'a> CallChecker<'a> {
|
||||
fn stmts(&mut self, stmts: &[Stmt]) {
|
||||
for stmt in stmts {
|
||||
match stmt {
|
||||
Stmt::Assign { value, .. }
|
||||
| Stmt::Eval { expr: value, .. }
|
||||
| Stmt::Return { value, .. } => {
|
||||
Stmt::Assign { target, value, .. } => {
|
||||
if let PlaceTarget::Index { base, index, .. } = target {
|
||||
self.expr(base);
|
||||
self.expr(index);
|
||||
}
|
||||
self.expr(value);
|
||||
}
|
||||
Stmt::Eval { expr: value, .. } | Stmt::Return { value, .. } => {
|
||||
self.expr(value);
|
||||
}
|
||||
Stmt::Call { call, .. } => {
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use marqdo_expression::{Arg, Expr, Ident, RecordEntry};
|
||||
use marqdo_ir::{BranchArm, Stmt};
|
||||
use marqdo_ir::{BranchArm, PlaceTarget, Stmt};
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{codes, Diagnostic};
|
||||
|
||||
@@ -178,9 +178,24 @@ impl<'a> Analyzer<'a> {
|
||||
fn analyze_stmts(&mut self, stmts: &[Stmt], mut state: State) -> State {
|
||||
for stmt in stmts {
|
||||
state = match stmt {
|
||||
Stmt::Assign { name, value, .. } => {
|
||||
self.read_expr(value, &state);
|
||||
self.assign(name, &mut state);
|
||||
Stmt::Assign { target, value, .. } => {
|
||||
match target {
|
||||
PlaceTarget::Local { name, .. } => {
|
||||
// Preserve v1 order: the RHS is read before the
|
||||
// local rebinding makes the name definitely
|
||||
// assigned.
|
||||
self.read_expr(value, &state);
|
||||
self.assign(name, &mut state);
|
||||
}
|
||||
PlaceTarget::Index { base, index, .. } => {
|
||||
// Runtime order for an index place is base, index,
|
||||
// RHS; reads in the place are free reads like any
|
||||
// other.
|
||||
self.read_expr(base, &state);
|
||||
self.read_expr(index, &state);
|
||||
self.read_expr(value, &state);
|
||||
}
|
||||
}
|
||||
state
|
||||
}
|
||||
Stmt::Eval { expr, .. } => {
|
||||
@@ -312,8 +327,10 @@ mod tests {
|
||||
|
||||
fn assign(name: &str, value: &str) -> Stmt {
|
||||
Stmt::Assign {
|
||||
name: name.to_string(),
|
||||
name_span: Span::new(0, 0),
|
||||
target: PlaceTarget::Local {
|
||||
name: name.to_string(),
|
||||
name_span: Span::new(0, 0),
|
||||
},
|
||||
value: parse_expression(value).unwrap(),
|
||||
span: Span::new(0, 0),
|
||||
}
|
||||
@@ -334,6 +351,24 @@ mod tests {
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn index_assignment_reads_its_base_and_index_without_binding_a_local() {
|
||||
let stmts = vec![
|
||||
Stmt::Assign {
|
||||
target: PlaceTarget::Index {
|
||||
base: parse_expression("m").unwrap(),
|
||||
index: parse_expression("i").unwrap(),
|
||||
span: Span::new(0, 0),
|
||||
},
|
||||
value: parse_expression("9").unwrap(),
|
||||
span: Span::new(0, 0),
|
||||
},
|
||||
ret("m[0]"),
|
||||
];
|
||||
// `m` and `i` are free reads; the place never rebinds `m` as a local.
|
||||
assert_eq!(params(&stmts), vec!["i".to_string(), "m".to_string()]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn infers_free_reads() {
|
||||
// n * 阶乘(n=n-1)
|
||||
|
||||
@@ -6,9 +6,9 @@
|
||||
//! diagnostics and never fall back to prose.
|
||||
|
||||
use marqdo_expression::{
|
||||
parse_arguments, parse_expression, parse_statement, Arg, Expr, StatementKind,
|
||||
parse_arguments, parse_expression, parse_statement, Arg, AssignTarget, Expr, StatementKind,
|
||||
};
|
||||
use marqdo_ir::{CallSite, CallTarget, ImportDecl, Stmt};
|
||||
use marqdo_ir::{CallSite, CallTarget, ImportDecl, PlaceTarget, Stmt};
|
||||
use marqdo_markdown::{normalize_name, MarkdownDocument, MdKind, MdNode};
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{codes, Diagnostic};
|
||||
@@ -143,13 +143,21 @@ impl<'a> Mapper<'a> {
|
||||
Ok(mut statement) => {
|
||||
segments.rewrite_statement(&mut statement);
|
||||
Some(match statement.kind {
|
||||
StatementKind::Assign {
|
||||
name,
|
||||
name_span,
|
||||
value,
|
||||
} => Stmt::Assign {
|
||||
name,
|
||||
name_span,
|
||||
StatementKind::Assign { target, value } => Stmt::Assign {
|
||||
target: match target {
|
||||
AssignTarget::Local { name, name_span } => {
|
||||
PlaceTarget::Local { name, name_span }
|
||||
}
|
||||
AssignTarget::Index {
|
||||
base,
|
||||
index,
|
||||
span: target_span,
|
||||
} => PlaceTarget::Index {
|
||||
base: *base,
|
||||
index: *index,
|
||||
span: target_span,
|
||||
},
|
||||
},
|
||||
value,
|
||||
span: node.span,
|
||||
},
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
//! string, so its spans are payload-relative; this module maps them back to the
|
||||
//! original file so diagnostics point at real source.
|
||||
|
||||
use marqdo_expression::{Arg, Expr, RecordEntry, Statement, StatementKind};
|
||||
use marqdo_expression::{Arg, AssignTarget, Expr, RecordEntry, Statement, StatementKind};
|
||||
use marqdo_markdown::{MdKind, MdNode, PayloadError};
|
||||
use marqdo_span::Span;
|
||||
|
||||
@@ -140,11 +140,17 @@ impl PayloadSegments {
|
||||
/// Map and rewrite every span in a parsed statement.
|
||||
pub fn rewrite_statement(&self, statement: &mut Statement) {
|
||||
statement.span = self.map_span(statement.span);
|
||||
if let StatementKind::Assign {
|
||||
name_span, value, ..
|
||||
} = &mut statement.kind
|
||||
{
|
||||
*name_span = self.map_span(*name_span);
|
||||
if let StatementKind::Assign { target, value } = &mut statement.kind {
|
||||
match target {
|
||||
AssignTarget::Local { name_span, .. } => {
|
||||
*name_span = self.map_span(*name_span);
|
||||
}
|
||||
AssignTarget::Index { base, index, span } => {
|
||||
*span = self.map_span(*span);
|
||||
self.rewrite_expr(base);
|
||||
self.rewrite_expr(index);
|
||||
}
|
||||
}
|
||||
self.rewrite_expr(value);
|
||||
}
|
||||
if let StatementKind::Expr(expr) = &mut statement.kind {
|
||||
|
||||
@@ -9,6 +9,7 @@ repository.workspace = true
|
||||
authors.workspace = true
|
||||
|
||||
[dependencies]
|
||||
libloading = { workspace = true }
|
||||
marqdo-plugin-abi = { workspace = true }
|
||||
glob = { workspace = true }
|
||||
marqdo-span = { workspace = true }
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
/* Marqdo ResourceProvider C ABI v1 (ADR-0036).
|
||||
*
|
||||
* This ABI is for trusted native adapters, not a sandbox. The host loads the
|
||||
* shared library, checks marqdo_resource_provider_abi_version() == 1 and calls
|
||||
* these symbols for language-level resource.* operations.
|
||||
*
|
||||
* Conventions:
|
||||
* - return 0 on success, nonzero on failure;
|
||||
* - failure text is available from marqdo_resource_last_error();
|
||||
* - rights is a comma-separated list (e.g. "read,write");
|
||||
* - provider tokens are private to the adapter and must not be exposed to
|
||||
* language Values, wire frames or trace output.
|
||||
*/
|
||||
#ifndef MARQDO_RESOURCE_PROVIDER_H
|
||||
#define MARQDO_RESOURCE_PROVIDER_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
#define MARQDO_RESOURCE_PROVIDER_ABI_VERSION 1u
|
||||
|
||||
uint32_t marqdo_resource_provider_abi_version(void);
|
||||
|
||||
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
|
||||
const char *rights_csv, uint64_t *out_token);
|
||||
|
||||
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
|
||||
uint32_t capacity, uint32_t *out_read,
|
||||
int32_t *out_eof);
|
||||
|
||||
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
|
||||
uint32_t length, uint32_t *out_written);
|
||||
|
||||
int32_t marqdo_resource_close(uint64_t token);
|
||||
|
||||
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity);
|
||||
|
||||
#endif /* MARQDO_RESOURCE_PROVIDER_H */
|
||||
@@ -323,6 +323,8 @@ pub struct Capabilities {
|
||||
pub foreign: bool,
|
||||
/// Allowed plugin names.
|
||||
pub plugins: Vec<String>,
|
||||
/// Allowed resource-provider names (ADR-0033).
|
||||
pub resource_providers: Vec<String>,
|
||||
}
|
||||
|
||||
impl Capabilities {
|
||||
@@ -337,6 +339,7 @@ impl Capabilities {
|
||||
process: false,
|
||||
foreign: false,
|
||||
plugins: Vec::new(),
|
||||
resource_providers: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -371,6 +374,39 @@ impl Capabilities {
|
||||
self
|
||||
}
|
||||
|
||||
/// Grant network access to one IP literal host (ADR-0034).
|
||||
pub fn allow_net(mut self, host: impl Into<String>) -> Self {
|
||||
self.net_hosts.push(host.into());
|
||||
self
|
||||
}
|
||||
|
||||
/// Grant a registered resource provider by its stable name (ADR-0033).
|
||||
pub fn allow_resource_provider(mut self, name: impl Into<String>) -> Self {
|
||||
self.resource_providers.push(name.into());
|
||||
self
|
||||
}
|
||||
|
||||
/// Check a resource-provider grant.
|
||||
pub fn check_resource_provider(
|
||||
&self,
|
||||
name: &str,
|
||||
span: Option<Span>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
if self
|
||||
.resource_providers
|
||||
.iter()
|
||||
.any(|allowed| allowed == name)
|
||||
{
|
||||
Ok(())
|
||||
} else {
|
||||
Err(denied(
|
||||
span,
|
||||
format!("resource provider `{name}` was not granted"),
|
||||
format!("grant it with Capabilities::allow_resource_provider({name:?})"),
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a project-relative path matches a granted pattern.
|
||||
///
|
||||
/// Patterns and paths are normalised by stripping a leading `./` so that
|
||||
|
||||
@@ -83,6 +83,22 @@ impl ProjectFs {
|
||||
|
||||
/// Read a UTF-8 file, bounded by [`MAX_READ_BYTES`].
|
||||
pub fn read(&self, user_path: &str, span: Option<Span>) -> Result<String, RuntimeError> {
|
||||
let bytes = self.read_bytes(user_path, span)?;
|
||||
String::from_utf8(bytes).map_err(|_| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::Type,
|
||||
span,
|
||||
format!("`{user_path}` is not valid UTF-8"),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/// Read a file as raw bytes, bounded by [`MAX_READ_BYTES`].
|
||||
///
|
||||
/// Used by the resource stream provider; path/capability checks are the
|
||||
/// caller's responsibility for grants, while this method still rejects
|
||||
/// traversal and symlink escapes.
|
||||
pub fn read_bytes(&self, user_path: &str, span: Option<Span>) -> Result<Vec<u8>, RuntimeError> {
|
||||
let (absolute, relative) = self.resolve(user_path, span)?;
|
||||
let metadata = std::fs::metadata(&absolute).map_err(|error| {
|
||||
RuntimeError::new(
|
||||
@@ -118,19 +134,12 @@ impl ProjectFs {
|
||||
if !canonical.starts_with(&self.root) {
|
||||
return Err(invalid_path(span, "path escapes the project root"));
|
||||
}
|
||||
let bytes = std::fs::read(&canonical).map_err(|error| {
|
||||
std::fs::read(&canonical).map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::HostError,
|
||||
span,
|
||||
format!("cannot read `{relative}`: {error}"),
|
||||
)
|
||||
})?;
|
||||
String::from_utf8(bytes).map_err(|_| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::Type,
|
||||
span,
|
||||
format!("`{relative}` is not valid UTF-8"),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -140,6 +149,18 @@ impl ProjectFs {
|
||||
user_path: &str,
|
||||
contents: &str,
|
||||
span: Option<Span>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
self.write_bytes(user_path, contents.as_bytes(), span)
|
||||
}
|
||||
|
||||
/// Write raw bytes inside the project root, creating parent directories.
|
||||
///
|
||||
/// This is a single validated write, not a transactional replace.
|
||||
pub fn write_bytes(
|
||||
&self,
|
||||
user_path: &str,
|
||||
bytes: &[u8],
|
||||
span: Option<Span>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let (absolute, relative) = self.resolve(user_path, span)?;
|
||||
// Validate the closest existing ancestor stays inside the root.
|
||||
@@ -169,7 +190,7 @@ impl ProjectFs {
|
||||
)
|
||||
})?;
|
||||
}
|
||||
std::fs::write(&absolute, contents).map_err(|error| {
|
||||
std::fs::write(&absolute, bytes).map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::HostError,
|
||||
span,
|
||||
|
||||
+515
-26
@@ -5,6 +5,7 @@
|
||||
//! execution, not at parse/compile time.
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::net::{SocketAddr, TcpStream};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
@@ -12,7 +13,7 @@ use marqdo_plugin_abi::{
|
||||
ErrorKind as PluginErrorKind, LoadedPlugin, PluginError, PluginKind, PluginManifest,
|
||||
};
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{ErrorCategory, RuntimeError, Value};
|
||||
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value};
|
||||
|
||||
use crate::budget::Budget;
|
||||
use crate::capability::{Capabilities, CapabilityAudit, CapabilityBroker};
|
||||
@@ -20,14 +21,15 @@ use crate::context::ExecutionContext;
|
||||
use crate::fs::ProjectFs;
|
||||
use crate::handles::{contains_plugin_handle, PluginHandleTable};
|
||||
use crate::plugin_worker::WorkerPlugin;
|
||||
use crate::provider_c::{load_package, CResourceProvider};
|
||||
use crate::provider_worker::WorkerResourceProvider;
|
||||
use crate::resource::{ResourceProvider, ResourceRead, ResourceTable, SharedResourceProvider};
|
||||
use crate::trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState};
|
||||
use marqdo_sandbox::{PlatformCapabilities, SandboxMode};
|
||||
|
||||
/// Host operations reachable from the language surface.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum HostOp {
|
||||
/// Write a value to program stdout.
|
||||
Print,
|
||||
/// Parse a formula.
|
||||
MathParse,
|
||||
/// Evaluate a formula.
|
||||
@@ -54,7 +56,6 @@ impl HostOp {
|
||||
/// Stable operation name used in traces and diagnostics.
|
||||
pub const fn name(self) -> &'static str {
|
||||
match self {
|
||||
HostOp::Print => "print",
|
||||
HostOp::MathParse => "math.parse",
|
||||
HostOp::MathEval => "math.eval",
|
||||
HostOp::MathSubstitute => "math.substitute",
|
||||
@@ -79,6 +80,65 @@ pub trait Host {
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<Value, RuntimeError>;
|
||||
|
||||
/// Open a resource/stream by kind, descriptor and requested rights.
|
||||
///
|
||||
/// Default: unavailable. Only the standard host implements the first
|
||||
/// resource kinds (ADR-0025).
|
||||
fn resource_open(
|
||||
&mut self,
|
||||
_kind: &str,
|
||||
_descriptor: &str,
|
||||
_rights: &[String],
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<Value, RuntimeError> {
|
||||
Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
"this host does not provide resource streams",
|
||||
))
|
||||
}
|
||||
|
||||
/// Read up to `max_bytes` from a resource into host memory.
|
||||
fn resource_read(
|
||||
&mut self,
|
||||
_handle: ResourceHandle,
|
||||
_max_bytes: usize,
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<ResourceRead, RuntimeError> {
|
||||
Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
"this host does not provide resource streams",
|
||||
))
|
||||
}
|
||||
|
||||
/// Write bytes to a resource; returns the accepted count.
|
||||
fn resource_write(
|
||||
&mut self,
|
||||
_handle: ResourceHandle,
|
||||
_bytes: &[u8],
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
"this host does not provide resource streams",
|
||||
))
|
||||
}
|
||||
|
||||
/// Close a resource handle.
|
||||
fn resource_close(
|
||||
&mut self,
|
||||
_handle: ResourceHandle,
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<(), RuntimeError> {
|
||||
Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
"this host does not provide resource streams",
|
||||
))
|
||||
}
|
||||
|
||||
/// Consume evaluation fuel.
|
||||
fn consume_fuel(&mut self, amount: u64, span: Option<Span>) -> Result<(), RuntimeError>;
|
||||
|
||||
@@ -143,6 +203,10 @@ pub struct StdHost {
|
||||
broker_audit: Arc<Mutex<Vec<CapabilityAudit>>>,
|
||||
/// Host-issued opaque plugin handles.
|
||||
pub handles: PluginHandleTable,
|
||||
/// Host-owned resource/stream table (ADR-0025).
|
||||
resources: ResourceTable,
|
||||
/// Registered in-process providers keyed by resource kind (ADR-0033).
|
||||
resource_providers: BTreeMap<String, (String, SharedResourceProvider)>,
|
||||
semantic_events: Vec<SemanticEvent>,
|
||||
next_event_seq: u64,
|
||||
trace_state: TraceState,
|
||||
@@ -203,6 +267,8 @@ impl StdHost {
|
||||
vendor_capabilities: BTreeSet::new(),
|
||||
broker_audit: Arc::new(Mutex::new(Vec::new())),
|
||||
handles: PluginHandleTable::default(),
|
||||
resources: ResourceTable::new(),
|
||||
resource_providers: BTreeMap::new(),
|
||||
semantic_events: Vec::new(),
|
||||
next_event_seq: 0,
|
||||
trace_state: TraceState::default(),
|
||||
@@ -417,7 +483,6 @@ impl StdHost {
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<Value, RuntimeError> {
|
||||
match op {
|
||||
HostOp::Print => self.op_print(args, ctx),
|
||||
HostOp::FsRead => self.op_fs_read(args, ctx),
|
||||
HostOp::FsWrite => self.op_fs_write(args, ctx),
|
||||
HostOp::MathParse
|
||||
@@ -432,27 +497,6 @@ impl StdHost {
|
||||
}
|
||||
}
|
||||
|
||||
fn op_print(
|
||||
&mut self,
|
||||
args: Vec<Value>,
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<Value, RuntimeError> {
|
||||
if !self.capabilities.console {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::PermissionDenied,
|
||||
ctx.span,
|
||||
"program output (console) capability was not granted",
|
||||
));
|
||||
}
|
||||
if args.len() != 1 {
|
||||
return Err(arity(ctx.span, "打印", 1, args.len()));
|
||||
}
|
||||
let text = format!("{}\n", args[0].display());
|
||||
self.budget.consume_output(text.len(), ctx.span)?;
|
||||
self.output.extend_from_slice(text.as_bytes());
|
||||
Ok(Value::Unit)
|
||||
}
|
||||
|
||||
fn op_fs_read(
|
||||
&mut self,
|
||||
args: Vec<Value>,
|
||||
@@ -526,6 +570,327 @@ impl StdHost {
|
||||
self.fs.write(&path, &contents, ctx.span)?;
|
||||
Ok(Value::Unit)
|
||||
}
|
||||
|
||||
/// Register an explicitly trusted in-process provider (ADR-0033).
|
||||
///
|
||||
/// In-process providers share the host address space and are not a
|
||||
/// sandbox; only trusted adapters may be registered. Registration is not a
|
||||
/// language operation and does not grant access by itself.
|
||||
pub fn register_resource_provider(
|
||||
&mut self,
|
||||
provider: Box<dyn ResourceProvider>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
self.register_provider_checked(provider)
|
||||
}
|
||||
|
||||
/// Register a trusted C ABI resource provider (ADR-0036).
|
||||
///
|
||||
/// The shared library runs in the host address space and is **trusted**, not
|
||||
/// sandboxed. Registration still requires the provider-name grant at
|
||||
/// `resource.open` and is project-mode only.
|
||||
pub fn register_c_resource_provider(
|
||||
&mut self,
|
||||
name: &str,
|
||||
kinds: &[&str],
|
||||
library: &std::path::Path,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let kinds = kinds
|
||||
.iter()
|
||||
.map(|kind| (*kind).to_string())
|
||||
.collect::<Vec<_>>();
|
||||
let provider = CResourceProvider::open(name, &kinds, library)?;
|
||||
self.register_provider_checked(Box::new(provider))
|
||||
}
|
||||
|
||||
/// Register a trusted C ABI provider from a `marqdo.provider` manifest
|
||||
/// (ADR-0037). The manifest is host-side installation metadata; language
|
||||
/// imports cannot install or authorize a provider.
|
||||
pub fn register_provider_package(
|
||||
&mut self,
|
||||
manifest_path: &std::path::Path,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let provider = load_package(manifest_path)?;
|
||||
self.register_provider_checked(Box::new(provider))
|
||||
}
|
||||
|
||||
/// Register a trusted worker-process resource provider (ADR-0035).
|
||||
///
|
||||
/// The helper speaks framed JSON over stdin/stdout and is **not** an OS
|
||||
/// sandbox; only explicitly trusted helpers may be spawned.
|
||||
pub fn register_worker_resource_provider(
|
||||
&mut self,
|
||||
name: &str,
|
||||
kinds: &[&str],
|
||||
program: &std::path::Path,
|
||||
args: &[String],
|
||||
) -> Result<(), RuntimeError> {
|
||||
let kinds = kinds
|
||||
.iter()
|
||||
.map(|kind| (*kind).to_string())
|
||||
.collect::<Vec<_>>();
|
||||
let provider = WorkerResourceProvider::spawn(name, &kinds, program, args)?;
|
||||
self.register_provider_checked(Box::new(provider))
|
||||
}
|
||||
|
||||
fn register_provider_checked(
|
||||
&mut self,
|
||||
provider: Box<dyn ResourceProvider>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
if provider.protocol_version() != 1 {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!(
|
||||
"resource provider `{}` uses unsupported protocol version {}",
|
||||
provider.name(),
|
||||
provider.protocol_version()
|
||||
),
|
||||
));
|
||||
}
|
||||
let name = provider.name().to_string();
|
||||
if name.is_empty() || name == "stdio" || name == "file" {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!("resource provider name `{name}` is reserved or empty"),
|
||||
));
|
||||
}
|
||||
let kinds = provider.kind_strings();
|
||||
if kinds.is_empty() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!("resource provider `{name}` declares no kinds"),
|
||||
));
|
||||
}
|
||||
for kind in &kinds {
|
||||
if kind.is_empty() || kind == "stdio" || kind == "file" {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!("resource provider `{name}` declares reserved kind `{kind}`"),
|
||||
));
|
||||
}
|
||||
if let Some((existing, _)) = self.resource_providers.get(kind) {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!(
|
||||
"resource kind `{kind}` is already registered by provider `{existing}`"
|
||||
),
|
||||
));
|
||||
}
|
||||
}
|
||||
let provider: SharedResourceProvider = std::sync::Arc::new(std::sync::Mutex::new(provider));
|
||||
for kind in kinds {
|
||||
self.resource_providers
|
||||
.insert(kind, (name.clone(), std::sync::Arc::clone(&provider)));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn record_resource_effect(
|
||||
&mut self,
|
||||
op: &str,
|
||||
ctx: &ExecutionContext,
|
||||
started: Instant,
|
||||
summary: &str,
|
||||
) {
|
||||
let estimate = TraceState::estimate(
|
||||
"effect",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
Some(summary),
|
||||
Some(op),
|
||||
);
|
||||
if !self.trace_state.accept(estimate) {
|
||||
return;
|
||||
}
|
||||
let seq = self.next_seq();
|
||||
self.trace.record(
|
||||
seq,
|
||||
op,
|
||||
ctx.module_id.clone(),
|
||||
ctx.span,
|
||||
started.elapsed(),
|
||||
summary.to_string(),
|
||||
);
|
||||
}
|
||||
|
||||
fn open_resource_stream(
|
||||
&mut self,
|
||||
kind: &str,
|
||||
descriptor: &str,
|
||||
rights: &[String],
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<Value, RuntimeError> {
|
||||
let started = Instant::now();
|
||||
let requested: BTreeSet<&str> = rights.iter().map(String::as_str).collect();
|
||||
if self.mode == SandboxMode::Strict {
|
||||
// stdio/stdout is the existing console channel already permitted by
|
||||
// strict L1 (CM-A11); file/network/process providers are not.
|
||||
let stdio_console =
|
||||
kind == "stdio" && descriptor == "stdout" && requested == BTreeSet::from(["write"]);
|
||||
if !stdio_console {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::PermissionDenied,
|
||||
ctx.span,
|
||||
"strict mode permits only the existing stdio/stdout console channel; \
|
||||
other resource streams are refused",
|
||||
));
|
||||
}
|
||||
}
|
||||
if self.mode == SandboxMode::Pure && kind == "file" {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::PermissionDenied,
|
||||
ctx.span,
|
||||
"pure mode does not provide project file resources",
|
||||
));
|
||||
}
|
||||
match kind {
|
||||
"stdio" => {
|
||||
if descriptor != "stdout" || requested != BTreeSet::from(["write"]) {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
"the first stream slice supports only stdio/stdout with rights [write]",
|
||||
));
|
||||
}
|
||||
if !self.capabilities.console {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::PermissionDenied,
|
||||
ctx.span,
|
||||
"the console capability is required for stdio/stdout",
|
||||
));
|
||||
}
|
||||
let handle = self.resources.open_stdout();
|
||||
self.record_semantic(
|
||||
"resource.open",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
ctx.span,
|
||||
Some("kind=stdio descriptor=stdout"),
|
||||
);
|
||||
self.record_resource_effect("resource.open", ctx, started, "ok");
|
||||
Ok(Value::Resource(handle))
|
||||
}
|
||||
"file" => {
|
||||
if requested == BTreeSet::from(["read"]) {
|
||||
self.capabilities.check_read(descriptor, ctx.span)?;
|
||||
let (_, relative) = self.fs.resolve(descriptor, ctx.span)?;
|
||||
let bytes = self.fs.read_bytes(descriptor, ctx.span)?;
|
||||
let handle = self.resources.open_file(relative, bytes);
|
||||
self.record_semantic(
|
||||
"resource.open",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
ctx.span,
|
||||
Some("kind=file rights=read"),
|
||||
);
|
||||
self.record_resource_effect("resource.open", ctx, started, "ok");
|
||||
return Ok(Value::Resource(handle));
|
||||
}
|
||||
if requested == BTreeSet::from(["write"]) {
|
||||
self.capabilities.check_write(descriptor, ctx.span)?;
|
||||
let (_, relative) = self.fs.resolve(descriptor, ctx.span)?;
|
||||
let handle = self.resources.open_file_writer(relative);
|
||||
self.record_semantic(
|
||||
"resource.open",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
ctx.span,
|
||||
Some("kind=file rights=write"),
|
||||
);
|
||||
self.record_resource_effect("resource.open", ctx, started, "ok");
|
||||
return Ok(Value::Resource(handle));
|
||||
}
|
||||
Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
"project file streams support exactly [read] or [write] in this build",
|
||||
))
|
||||
}
|
||||
"tcp" => {
|
||||
if self.mode != SandboxMode::Project {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::PermissionDenied,
|
||||
ctx.span,
|
||||
"tcp resources are available only in project mode",
|
||||
));
|
||||
}
|
||||
if requested != BTreeSet::from(["read", "write"]) {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
"tcp resources require exactly [read, write] in this build",
|
||||
));
|
||||
}
|
||||
let address: SocketAddr = descriptor.parse().map_err(|_| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
format!(
|
||||
"tcp descriptor `{descriptor}` must be an IP literal address:port (DNS names are not resolved in this build)"
|
||||
),
|
||||
)
|
||||
})?;
|
||||
self.capabilities
|
||||
.check_net(&address.ip().to_string(), ctx.span)?;
|
||||
let stream = TcpStream::connect_timeout(&address, Duration::from_secs(5)).map_err(
|
||||
|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::HostError,
|
||||
ctx.span,
|
||||
format!("cannot connect to {address}: {error}"),
|
||||
)
|
||||
},
|
||||
)?;
|
||||
let _ = stream.set_read_timeout(Some(Duration::from_secs(5)));
|
||||
let _ = stream.set_write_timeout(Some(Duration::from_secs(5)));
|
||||
let handle = self.resources.open_tcp(stream);
|
||||
self.record_semantic(
|
||||
"resource.open",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
ctx.span,
|
||||
Some(&format!("kind=tcp address={address}")),
|
||||
);
|
||||
self.record_resource_effect("resource.open", ctx, started, "ok");
|
||||
Ok(Value::Resource(handle))
|
||||
}
|
||||
other => {
|
||||
if self.mode != SandboxMode::Project {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::PermissionDenied,
|
||||
ctx.span,
|
||||
"resource providers are available only in project mode",
|
||||
));
|
||||
}
|
||||
let Some((provider_name, provider)) = self.resource_providers.get(other).cloned()
|
||||
else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
ctx.span,
|
||||
format!("resource kind `{other}` is not registered"),
|
||||
));
|
||||
};
|
||||
self.capabilities
|
||||
.check_resource_provider(&provider_name, ctx.span)?;
|
||||
let handle = self
|
||||
.resources
|
||||
.open_provider(provider, other, descriptor, rights, ctx.span)?;
|
||||
self.record_semantic(
|
||||
"resource.open",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
ctx.span,
|
||||
Some(&format!("kind={other} provider={provider_name}")),
|
||||
);
|
||||
self.record_resource_effect("resource.open", ctx, started, "ok");
|
||||
Ok(Value::Resource(handle))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Host for StdHost {
|
||||
@@ -535,6 +900,31 @@ impl Host for StdHost {
|
||||
args: Vec<Value>,
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<Value, RuntimeError> {
|
||||
// Addressable memory is language-local state. Handles must not reach a
|
||||
// plugin, foreign kernel or file adapter, where they could not be
|
||||
// interpreted safely (RFC-CM-0001 §8, ADR-0022 §4).
|
||||
if args.iter().any(Value::contains_memory) {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::Type,
|
||||
ctx.span,
|
||||
format!(
|
||||
"Memory values cannot cross the host boundary (`{}`); copy bytes into \
|
||||
Text explicitly first",
|
||||
op.name()
|
||||
),
|
||||
));
|
||||
}
|
||||
if args.iter().any(Value::contains_resource) {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::Type,
|
||||
ctx.span,
|
||||
format!(
|
||||
"Resource handles cannot cross the generic host boundary (`{}`); use \
|
||||
resource.* operations",
|
||||
op.name()
|
||||
),
|
||||
));
|
||||
}
|
||||
let started = Instant::now();
|
||||
let result = self.dispatch(op, args, ctx);
|
||||
let summary = match &result {
|
||||
@@ -562,6 +952,105 @@ impl Host for StdHost {
|
||||
result
|
||||
}
|
||||
|
||||
fn resource_open(
|
||||
&mut self,
|
||||
kind: &str,
|
||||
descriptor: &str,
|
||||
rights: &[String],
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<Value, RuntimeError> {
|
||||
self.open_resource_stream(kind, descriptor, rights, ctx)
|
||||
}
|
||||
|
||||
fn resource_read(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
max_bytes: usize,
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<ResourceRead, RuntimeError> {
|
||||
let started = Instant::now();
|
||||
let kind = self.resources.kind(handle, ctx.span)?;
|
||||
let read = self.resources.read(handle, max_bytes, ctx.span)?;
|
||||
self.record_semantic(
|
||||
"resource.read",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
ctx.span,
|
||||
Some(&format!(
|
||||
"kind={kind} count={} eof={}",
|
||||
read.bytes.len(),
|
||||
read.eof
|
||||
)),
|
||||
);
|
||||
self.record_resource_effect(
|
||||
"resource.read",
|
||||
ctx,
|
||||
started,
|
||||
&format!("kind={kind} count={}", read.bytes.len()),
|
||||
);
|
||||
Ok(read)
|
||||
}
|
||||
|
||||
fn resource_write(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
bytes: &[u8],
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
let started = Instant::now();
|
||||
let kind = self.resources.kind(handle, ctx.span)?;
|
||||
let count = self.resources.write(handle, bytes, ctx.span)?;
|
||||
if kind == "stdio" {
|
||||
self.budget.consume_output(count, ctx.span)?;
|
||||
self.output.extend_from_slice(bytes);
|
||||
}
|
||||
self.record_semantic(
|
||||
"resource.write",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
ctx.span,
|
||||
Some(&format!("kind={kind} count={count}")),
|
||||
);
|
||||
self.record_resource_effect(
|
||||
"resource.write",
|
||||
ctx,
|
||||
started,
|
||||
&format!("kind={kind} count={count}"),
|
||||
);
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
fn resource_close(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
ctx: &ExecutionContext,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let started = Instant::now();
|
||||
let pending = self.resources.pending_file_write(handle, ctx.span)?;
|
||||
let count = match pending {
|
||||
Some((relative, bytes)) => {
|
||||
self.fs.write_bytes(&relative, &bytes, ctx.span)?;
|
||||
bytes.len()
|
||||
}
|
||||
None => 0,
|
||||
};
|
||||
self.resources.close(handle, ctx.span)?;
|
||||
self.record_semantic(
|
||||
"resource.close",
|
||||
None,
|
||||
Some(&ctx.module_id),
|
||||
ctx.span,
|
||||
Some(&format!("flushed_bytes={count}")),
|
||||
);
|
||||
self.record_resource_effect(
|
||||
"resource.close",
|
||||
ctx,
|
||||
started,
|
||||
&format!("flushed_bytes={count}"),
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn consume_fuel(&mut self, amount: u64, span: Option<Span>) -> Result<(), RuntimeError> {
|
||||
self.budget.consume_fuel(amount, span)
|
||||
}
|
||||
|
||||
@@ -12,6 +12,9 @@ pub mod handles;
|
||||
pub mod host;
|
||||
pub mod kernels;
|
||||
mod plugin_worker;
|
||||
mod provider_c;
|
||||
mod provider_worker;
|
||||
pub mod resource;
|
||||
pub mod trace;
|
||||
|
||||
pub use budget::{Budget, DEFAULT_FUEL, DEFAULT_OUTPUT_BYTES, DEFAULT_STACK_LIMIT};
|
||||
@@ -21,4 +24,5 @@ pub use fs::ProjectFs;
|
||||
pub use handles::{contains_plugin_handle, PluginHandleTable};
|
||||
pub use host::{arity, expect_text, Host, HostOp, StdHost};
|
||||
pub use marqdo_sandbox::{PlatformCapabilities, SandboxMode, StrictIsolation};
|
||||
pub use resource::{ResourceProvider, ResourceRead, ResourceTable, SharedResourceProvider};
|
||||
pub use trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState, TraceTruncation};
|
||||
@@ -0,0 +1,292 @@
|
||||
//! C ABI resource provider v1 (ADR-0036).
|
||||
//!
|
||||
//! The library is trusted native code. Loading it shares the host address space
|
||||
//! and is explicitly not a sandbox; only explicitly trusted adapters may be
|
||||
//! registered.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::ffi::{c_char, CString};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use libloading::Library;
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{ErrorCategory, RuntimeError};
|
||||
|
||||
use crate::resource::{ResourceProvider, ResourceRead};
|
||||
|
||||
const ABI_VERSION: u32 = 1;
|
||||
const ERROR_MAX: usize = 1024;
|
||||
|
||||
type AbiVersionFn = unsafe extern "C" fn() -> u32;
|
||||
type OpenFn = unsafe extern "C" fn(*const c_char, *const c_char, *const c_char, *mut u64) -> i32;
|
||||
type ReadFn = unsafe extern "C" fn(u64, *mut u8, u32, *mut u32, *mut i32) -> i32;
|
||||
type WriteFn = unsafe extern "C" fn(u64, *const u8, u32, *mut u32) -> i32;
|
||||
type CloseFn = unsafe extern "C" fn(u64) -> i32;
|
||||
type LastErrorFn = unsafe extern "C" fn(*mut c_char, u32) -> i32;
|
||||
|
||||
#[derive(Debug, serde::Deserialize)]
|
||||
struct ProviderPackage {
|
||||
schema: String,
|
||||
schema_version: u32,
|
||||
name: String,
|
||||
kinds: Vec<String>,
|
||||
abi: String,
|
||||
#[serde(default)]
|
||||
library: BTreeMap<String, String>,
|
||||
}
|
||||
|
||||
/// Load a C ABI provider from its package manifest (ADR-0037).
|
||||
pub(crate) fn load_package(manifest_path: &Path) -> Result<CResourceProvider, RuntimeError> {
|
||||
let bytes = std::fs::read(manifest_path).map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!(
|
||||
"cannot read provider manifest {}: {error}",
|
||||
manifest_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let package: ProviderPackage = serde_json::from_slice(&bytes).map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!(
|
||||
"invalid provider manifest {}: {error}",
|
||||
manifest_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
if package.schema != "marqdo.provider" || package.schema_version != 1 {
|
||||
return Err(provider_error(
|
||||
&package.name,
|
||||
"manifest schema must be marqdo.provider version 1",
|
||||
));
|
||||
}
|
||||
if package.abi != "c-abi-1" {
|
||||
return Err(provider_error(
|
||||
&package.name,
|
||||
&format!(
|
||||
"unsupported provider ABI `{}` (expected c-abi-1)",
|
||||
package.abi
|
||||
),
|
||||
));
|
||||
}
|
||||
let platform = if cfg!(target_os = "linux") {
|
||||
"linux"
|
||||
} else if cfg!(target_os = "macos") {
|
||||
"macos"
|
||||
} else if cfg!(target_os = "windows") {
|
||||
"windows"
|
||||
} else {
|
||||
"default"
|
||||
};
|
||||
let library_name = package
|
||||
.library
|
||||
.get(platform)
|
||||
.or_else(|| package.library.get("default"))
|
||||
.ok_or_else(|| {
|
||||
provider_error(
|
||||
&package.name,
|
||||
&format!("manifest has no `{platform}` or `default` library entry"),
|
||||
)
|
||||
})?;
|
||||
let base = manifest_path.parent().unwrap_or_else(|| Path::new("."));
|
||||
let library = PathBuf::from(library_name);
|
||||
let library = if library.is_absolute() {
|
||||
library
|
||||
} else {
|
||||
base.join(library)
|
||||
};
|
||||
CResourceProvider::open(&package.name, &package.kinds, &library)
|
||||
}
|
||||
|
||||
pub(crate) struct CResourceProvider {
|
||||
name: String,
|
||||
kinds: Vec<String>,
|
||||
library: Library,
|
||||
}
|
||||
|
||||
impl CResourceProvider {
|
||||
pub(crate) fn open(name: &str, kinds: &[String], path: &Path) -> Result<Self, RuntimeError> {
|
||||
let library = unsafe { Library::new(path) }.map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::KernelUnavailable,
|
||||
None,
|
||||
format!(
|
||||
"cannot load C resource provider `{name}` ({}): {error}",
|
||||
path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let version = {
|
||||
let symbol: libloading::Symbol<AbiVersionFn> = unsafe {
|
||||
library
|
||||
.get(b"marqdo_resource_provider_abi_version\0")
|
||||
.map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!("C resource provider `{name}` is missing abi_version: {error}"),
|
||||
)
|
||||
})?
|
||||
};
|
||||
unsafe { symbol() }
|
||||
};
|
||||
if version != ABI_VERSION {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!(
|
||||
"C resource provider `{name}` uses ABI version {version}; host requires {ABI_VERSION}"
|
||||
),
|
||||
));
|
||||
}
|
||||
Ok(Self {
|
||||
name: name.to_string(),
|
||||
kinds: kinds.to_vec(),
|
||||
library,
|
||||
})
|
||||
}
|
||||
|
||||
fn open_fn(&self) -> Result<libloading::Symbol<'_, OpenFn>, RuntimeError> {
|
||||
unsafe { self.library.get(b"marqdo_resource_open\0") }
|
||||
.map_err(|error| provider_error(&self.name, &format!("missing open: {error}")))
|
||||
}
|
||||
fn read_fn(&self) -> Result<libloading::Symbol<'_, ReadFn>, RuntimeError> {
|
||||
unsafe { self.library.get(b"marqdo_resource_read\0") }
|
||||
.map_err(|error| provider_error(&self.name, &format!("missing read: {error}")))
|
||||
}
|
||||
fn write_fn(&self) -> Result<libloading::Symbol<'_, WriteFn>, RuntimeError> {
|
||||
unsafe { self.library.get(b"marqdo_resource_write\0") }
|
||||
.map_err(|error| provider_error(&self.name, &format!("missing write: {error}")))
|
||||
}
|
||||
fn close_fn(&self) -> Result<libloading::Symbol<'_, CloseFn>, RuntimeError> {
|
||||
unsafe { self.library.get(b"marqdo_resource_close\0") }
|
||||
.map_err(|error| provider_error(&self.name, &format!("missing close: {error}")))
|
||||
}
|
||||
|
||||
fn last_error(&self) -> String {
|
||||
let Ok(symbol) = (unsafe {
|
||||
self.library
|
||||
.get::<LastErrorFn>(b"marqdo_resource_last_error\0")
|
||||
}) else {
|
||||
return format!("C resource provider `{}` failed", self.name);
|
||||
};
|
||||
let mut buffer = vec![0i8; ERROR_MAX];
|
||||
let code = unsafe { symbol(buffer.as_mut_ptr(), ERROR_MAX as u32) };
|
||||
if code != 0 {
|
||||
return format!("C resource provider `{}` failed", self.name);
|
||||
}
|
||||
let bytes = buffer
|
||||
.iter()
|
||||
.map(|value| *value as u8)
|
||||
.take_while(|value| *value != 0)
|
||||
.collect::<Vec<_>>();
|
||||
String::from_utf8_lossy(&bytes).into_owned()
|
||||
}
|
||||
|
||||
fn failure(&self, span: Option<Span>) -> RuntimeError {
|
||||
let message = self.last_error();
|
||||
RuntimeError::new(ErrorCategory::HostError, span, message)
|
||||
}
|
||||
}
|
||||
|
||||
impl ResourceProvider for CResourceProvider {
|
||||
fn name(&self) -> &str {
|
||||
&self.name
|
||||
}
|
||||
fn kinds(&self) -> &[&str] {
|
||||
&[]
|
||||
}
|
||||
fn kind_strings(&self) -> Vec<String> {
|
||||
self.kinds.clone()
|
||||
}
|
||||
fn open(
|
||||
&mut self,
|
||||
kind: &str,
|
||||
descriptor: &str,
|
||||
rights: &[String],
|
||||
span: Option<Span>,
|
||||
) -> Result<u64, RuntimeError> {
|
||||
let kind = CString::new(kind).map_err(|_| provider_error(&self.name, "kind has NUL"))?;
|
||||
let descriptor = CString::new(descriptor)
|
||||
.map_err(|_| provider_error(&self.name, "descriptor has NUL"))?;
|
||||
let rights = CString::new(rights.join(","))
|
||||
.map_err(|_| provider_error(&self.name, "rights have NUL"))?;
|
||||
let mut token = 0u64;
|
||||
let code = unsafe {
|
||||
(self.open_fn()?)(
|
||||
kind.as_ptr(),
|
||||
descriptor.as_ptr(),
|
||||
rights.as_ptr(),
|
||||
&mut token,
|
||||
)
|
||||
};
|
||||
if code != 0 {
|
||||
return Err(self.failure(span));
|
||||
}
|
||||
Ok(token)
|
||||
}
|
||||
fn read(
|
||||
&mut self,
|
||||
token: u64,
|
||||
max_bytes: usize,
|
||||
span: Option<Span>,
|
||||
) -> Result<ResourceRead, RuntimeError> {
|
||||
let capacity = u32::try_from(max_bytes).unwrap_or(u32::MAX);
|
||||
let mut buffer = vec![0u8; capacity as usize];
|
||||
let mut read = 0u32;
|
||||
let mut eof = 0i32;
|
||||
let code =
|
||||
unsafe { (self.read_fn()?)(token, buffer.as_mut_ptr(), capacity, &mut read, &mut eof) };
|
||||
if code != 0 {
|
||||
return Err(self.failure(span));
|
||||
}
|
||||
buffer.truncate(read as usize);
|
||||
Ok(ResourceRead {
|
||||
bytes: buffer,
|
||||
eof: eof != 0,
|
||||
})
|
||||
}
|
||||
fn write(
|
||||
&mut self,
|
||||
token: u64,
|
||||
bytes: &[u8],
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
let length = u32::try_from(bytes.len())
|
||||
.map_err(|_| provider_error(&self.name, "write size exceeds the C ABI u32 limit"))?;
|
||||
let mut written = 0u32;
|
||||
let code = unsafe { (self.write_fn()?)(token, bytes.as_ptr(), length, &mut written) };
|
||||
if code != 0 {
|
||||
return Err(self.failure(span));
|
||||
}
|
||||
Ok(written as usize)
|
||||
}
|
||||
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError> {
|
||||
let code = unsafe { (self.close_fn()?)(token) };
|
||||
if code != 0 {
|
||||
return Err(self.failure(span));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for CResourceProvider {
|
||||
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
formatter
|
||||
.debug_struct("CResourceProvider")
|
||||
.field("name", &self.name)
|
||||
.field("kinds", &self.kinds)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
fn provider_error(name: &str, message: &str) -> RuntimeError {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::HostError,
|
||||
None,
|
||||
format!("C resource provider `{name}`: {message}"),
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,210 @@
|
||||
//! Host-managed worker resource provider (ADR-0035).
|
||||
//!
|
||||
//! This is a small, versioned framed-JSON transport for **trusted helper
|
||||
//! processes**. It is not a C ABI binding and not an OS sandbox: the helper
|
||||
//! runs as the host user. Registration is a host embedding API; language code
|
||||
//! only sees the normal `resource.*` operations.
|
||||
|
||||
use std::io::{BufReader, Read, Write};
|
||||
use std::path::Path;
|
||||
use std::process::{Child, ChildStdin, ChildStdout, Command, Stdio};
|
||||
|
||||
use marqdo_span::Span;
|
||||
use serde_json::{json, Value as Json};
|
||||
|
||||
use crate::resource::{ResourceProvider, ResourceRead};
|
||||
use marqdo_value::{ErrorCategory, RuntimeError};
|
||||
|
||||
const MAX_FRAME_BYTES: usize = 1024 * 1024;
|
||||
|
||||
/// A `ResourceProvider` backed by a child process speaking framed JSON.
|
||||
pub(crate) struct WorkerResourceProvider {
|
||||
name: String,
|
||||
kinds: Vec<String>,
|
||||
child: Child,
|
||||
stdin: ChildStdin,
|
||||
stdout: BufReader<ChildStdout>,
|
||||
}
|
||||
|
||||
impl WorkerResourceProvider {
|
||||
pub(crate) fn spawn(
|
||||
name: &str,
|
||||
kinds: &[String],
|
||||
program: &Path,
|
||||
args: &[String],
|
||||
) -> Result<Self, RuntimeError> {
|
||||
let mut child = Command::new(program)
|
||||
.args(args)
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::null())
|
||||
.spawn()
|
||||
.map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::KernelUnavailable,
|
||||
None,
|
||||
format!(
|
||||
"cannot spawn worker resource provider `{name}` ({}): {error}",
|
||||
program.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let stdin = child
|
||||
.stdin
|
||||
.take()
|
||||
.ok_or_else(|| worker_error("missing stdin pipe"))?;
|
||||
let stdout = child
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| worker_error("missing stdout pipe"))?;
|
||||
let mut provider = Self {
|
||||
name: name.to_string(),
|
||||
kinds: kinds.to_vec(),
|
||||
child,
|
||||
stdin,
|
||||
stdout: BufReader::new(stdout),
|
||||
};
|
||||
// Handshake: the helper must answer a ping before it is trusted.
|
||||
provider.request(json!({"op": "ping"}))?;
|
||||
Ok(provider)
|
||||
}
|
||||
|
||||
fn request(&mut self, request: Json) -> Result<Json, RuntimeError> {
|
||||
let bytes = serde_json::to_vec(&request)
|
||||
.map_err(|error| worker_error(&format!("cannot encode request: {error}")))?;
|
||||
if bytes.len() > MAX_FRAME_BYTES {
|
||||
return Err(worker_error("request frame is too large"));
|
||||
}
|
||||
self.stdin
|
||||
.write_all(&(bytes.len() as u32).to_be_bytes())
|
||||
.and_then(|()| self.stdin.write_all(&bytes))
|
||||
.and_then(|()| self.stdin.flush())
|
||||
.map_err(|error| worker_error(&format!("cannot write request frame: {error}")))?;
|
||||
let mut header = [0u8; 4];
|
||||
self.stdout
|
||||
.read_exact(&mut header)
|
||||
.map_err(|error| worker_error(&format!("cannot read response header: {error}")))?;
|
||||
let length = u32::from_be_bytes(header) as usize;
|
||||
if length == 0 || length > MAX_FRAME_BYTES {
|
||||
return Err(worker_error("invalid response frame length"));
|
||||
}
|
||||
let mut payload = vec![0u8; length];
|
||||
self.stdout
|
||||
.read_exact(&mut payload)
|
||||
.map_err(|error| worker_error(&format!("cannot read response frame: {error}")))?;
|
||||
let response: Json = serde_json::from_slice(&payload)
|
||||
.map_err(|error| worker_error(&format!("invalid response JSON: {error}")))?;
|
||||
if response.get("ok").and_then(Json::as_bool) != Some(true) {
|
||||
let code = response
|
||||
.get("code")
|
||||
.and_then(Json::as_str)
|
||||
.unwrap_or("host.error");
|
||||
let message = response
|
||||
.get("message")
|
||||
.and_then(Json::as_str)
|
||||
.unwrap_or("worker resource provider failed");
|
||||
let category = match code {
|
||||
"permission.denied" => ErrorCategory::PermissionDenied,
|
||||
"resource.closed" => ErrorCategory::ResourceClosed,
|
||||
"resource.exhausted" => ErrorCategory::ResourceExhausted,
|
||||
"resource.unsupported" => ErrorCategory::ResourceUnsupported,
|
||||
_ => ErrorCategory::HostError,
|
||||
};
|
||||
return Err(RuntimeError::new(category, None, message.to_string()));
|
||||
}
|
||||
Ok(response)
|
||||
}
|
||||
}
|
||||
|
||||
impl ResourceProvider for WorkerResourceProvider {
|
||||
fn name(&self) -> &str {
|
||||
&self.name
|
||||
}
|
||||
fn kinds(&self) -> &[&str] {
|
||||
// Owned kinds are exposed through `kind_strings`; registration never
|
||||
// calls this for the worker transport.
|
||||
&[]
|
||||
}
|
||||
fn kind_strings(&self) -> Vec<String> {
|
||||
self.kinds.clone()
|
||||
}
|
||||
fn open(
|
||||
&mut self,
|
||||
kind: &str,
|
||||
descriptor: &str,
|
||||
rights: &[String],
|
||||
span: Option<Span>,
|
||||
) -> Result<u64, RuntimeError> {
|
||||
let response = self
|
||||
.request(json!({
|
||||
"op": "open",
|
||||
"kind": kind,
|
||||
"descriptor": descriptor,
|
||||
"rights": rights,
|
||||
}))
|
||||
.map_err(|error| with_span(error, span))?;
|
||||
response
|
||||
.get("token")
|
||||
.and_then(Json::as_u64)
|
||||
.ok_or_else(|| worker_error("open response has no token"))
|
||||
}
|
||||
fn read(
|
||||
&mut self,
|
||||
token: u64,
|
||||
max_bytes: usize,
|
||||
span: Option<Span>,
|
||||
) -> Result<ResourceRead, RuntimeError> {
|
||||
let response = self
|
||||
.request(json!({"op": "read", "token": token, "max_bytes": max_bytes}))
|
||||
.map_err(|error| with_span(error, span))?;
|
||||
let bytes = response
|
||||
.get("bytes")
|
||||
.and_then(Json::as_array)
|
||||
.ok_or_else(|| worker_error("read response has no bytes"))?
|
||||
.iter()
|
||||
.map(|value| value.as_u64().and_then(|v| u8::try_from(v).ok()))
|
||||
.collect::<Option<Vec<u8>>>()
|
||||
.ok_or_else(|| worker_error("read response contains a non-byte value"))?;
|
||||
let eof = response.get("eof").and_then(Json::as_bool).unwrap_or(false);
|
||||
Ok(ResourceRead { bytes, eof })
|
||||
}
|
||||
fn write(
|
||||
&mut self,
|
||||
token: u64,
|
||||
bytes: &[u8],
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
let response = self
|
||||
.request(json!({"op": "write", "token": token, "bytes": bytes}))
|
||||
.map_err(|error| with_span(error, span))?;
|
||||
response
|
||||
.get("count")
|
||||
.and_then(Json::as_u64)
|
||||
.and_then(|count| usize::try_from(count).ok())
|
||||
.ok_or_else(|| worker_error("write response has no count"))
|
||||
}
|
||||
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError> {
|
||||
self.request(json!({"op": "close", "token": token}))
|
||||
.map_err(|error| with_span(error, span))?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for WorkerResourceProvider {
|
||||
fn drop(&mut self) {
|
||||
let _ = self.request(json!({"op": "shutdown"}));
|
||||
let _ = self.child.kill();
|
||||
let _ = self.child.wait();
|
||||
}
|
||||
}
|
||||
|
||||
fn with_span(mut error: RuntimeError, span: Option<Span>) -> RuntimeError {
|
||||
if error.span.is_none() {
|
||||
error.span = span;
|
||||
}
|
||||
error
|
||||
}
|
||||
|
||||
fn worker_error(message: &str) -> RuntimeError {
|
||||
RuntimeError::new(ErrorCategory::HostError, None, message.to_string())
|
||||
}
|
||||
@@ -0,0 +1,539 @@
|
||||
//! Host-owned resource/stream table (ADR-0025).
|
||||
//!
|
||||
//! The first slice deliberately supports only stdout writes and bounded
|
||||
//! project-file reads loaded at open. Handles are run-scoped and opaque; the
|
||||
//! table validates run identity, slot identity and generation on every call.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::io::{Read, Write};
|
||||
use std::net::TcpStream;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError};
|
||||
|
||||
static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1);
|
||||
|
||||
/// A versioned trusted provider of additional resource kinds (ADR-0033).
|
||||
///
|
||||
/// In-process providers share the host address space and are **not** a
|
||||
/// sandbox; only explicitly trusted code may be registered.
|
||||
pub trait ResourceProvider: Send {
|
||||
/// Stable grant name.
|
||||
fn name(&self) -> &str;
|
||||
/// Protocol version; must be 1 in this build.
|
||||
fn protocol_version(&self) -> u32 {
|
||||
1
|
||||
}
|
||||
/// Resource kinds served by this provider.
|
||||
fn kinds(&self) -> &[&str];
|
||||
|
||||
/// Owned kind list used by registration. Providers with dynamically owned
|
||||
/// kind strings override this.
|
||||
fn kind_strings(&self) -> Vec<String> {
|
||||
self.kinds()
|
||||
.iter()
|
||||
.map(|kind| (*kind).to_string())
|
||||
.collect()
|
||||
}
|
||||
/// Open one descriptor, returning a provider-private token.
|
||||
fn open(
|
||||
&mut self,
|
||||
kind: &str,
|
||||
descriptor: &str,
|
||||
rights: &[String],
|
||||
span: Option<Span>,
|
||||
) -> Result<u64, RuntimeError>;
|
||||
/// Read up to `max_bytes` from a token.
|
||||
fn read(
|
||||
&mut self,
|
||||
token: u64,
|
||||
max_bytes: usize,
|
||||
span: Option<Span>,
|
||||
) -> Result<ResourceRead, RuntimeError>;
|
||||
/// Write bytes to a token; return the accepted count.
|
||||
fn write(
|
||||
&mut self,
|
||||
token: u64,
|
||||
bytes: &[u8],
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError>;
|
||||
/// Close a token.
|
||||
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError>;
|
||||
}
|
||||
|
||||
/// Shared provider handle registered on the host.
|
||||
pub type SharedResourceProvider = Arc<Mutex<Box<dyn ResourceProvider>>>;
|
||||
|
||||
/// Bytes returned by one stream read.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ResourceRead {
|
||||
/// Bytes actually read (may be empty at EOF).
|
||||
pub bytes: Vec<u8>,
|
||||
/// Whether the stream has reached EOF.
|
||||
pub eof: bool,
|
||||
}
|
||||
|
||||
enum Stream {
|
||||
Stdout {
|
||||
closed: bool,
|
||||
},
|
||||
File {
|
||||
relative: String,
|
||||
bytes: Vec<u8>,
|
||||
position: usize,
|
||||
closed: bool,
|
||||
},
|
||||
FileWriter {
|
||||
relative: String,
|
||||
bytes: Vec<u8>,
|
||||
closed: bool,
|
||||
},
|
||||
Provider {
|
||||
kind: String,
|
||||
provider: SharedResourceProvider,
|
||||
token: u64,
|
||||
closed: bool,
|
||||
},
|
||||
Tcp {
|
||||
stream: TcpStream,
|
||||
closed: bool,
|
||||
},
|
||||
}
|
||||
|
||||
struct Slot {
|
||||
generation: u64,
|
||||
stream: Stream,
|
||||
}
|
||||
|
||||
impl Slot {
|
||||
fn closed(&self) -> bool {
|
||||
match &self.stream {
|
||||
Stream::Stdout { closed } => *closed,
|
||||
Stream::File { closed, .. } => *closed,
|
||||
Stream::FileWriter { closed, .. } => *closed,
|
||||
Stream::Provider { closed, .. } => *closed,
|
||||
Stream::Tcp { closed, .. } => *closed,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Host resource table for one run.
|
||||
pub struct ResourceTable {
|
||||
run_id: u64,
|
||||
next_resource: u64,
|
||||
slots: BTreeMap<u64, Slot>,
|
||||
}
|
||||
|
||||
impl Default for ResourceTable {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl ResourceTable {
|
||||
/// Create an empty table with a fresh run identity.
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed),
|
||||
next_resource: 1,
|
||||
slots: BTreeMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Number of live (not necessarily open) slots.
|
||||
pub fn slot_count(&self) -> usize {
|
||||
self.slots.len()
|
||||
}
|
||||
|
||||
/// Register the pre-opened stdout stream.
|
||||
pub fn open_stdout(&mut self) -> ResourceHandle {
|
||||
self.insert(Stream::Stdout { closed: false })
|
||||
}
|
||||
|
||||
/// Register a project-file read stream opened from already-bounded bytes.
|
||||
pub fn open_file(&mut self, relative: impl Into<String>, bytes: Vec<u8>) -> ResourceHandle {
|
||||
self.insert(Stream::File {
|
||||
relative: relative.into(),
|
||||
bytes,
|
||||
position: 0,
|
||||
closed: false,
|
||||
})
|
||||
}
|
||||
|
||||
/// Register an empty buffered project-file write stream.
|
||||
pub fn open_file_writer(&mut self, relative: impl Into<String>) -> ResourceHandle {
|
||||
self.insert(Stream::FileWriter {
|
||||
relative: relative.into(),
|
||||
bytes: Vec::new(),
|
||||
closed: false,
|
||||
})
|
||||
}
|
||||
|
||||
/// Register a connected TCP stream.
|
||||
pub fn open_tcp(&mut self, stream: TcpStream) -> ResourceHandle {
|
||||
self.insert(Stream::Tcp {
|
||||
stream,
|
||||
closed: false,
|
||||
})
|
||||
}
|
||||
|
||||
/// Open a provider token and wrap it in a host-issued handle.
|
||||
pub fn open_provider(
|
||||
&mut self,
|
||||
provider: SharedResourceProvider,
|
||||
kind: &str,
|
||||
descriptor: &str,
|
||||
rights: &[String],
|
||||
span: Option<Span>,
|
||||
) -> Result<ResourceHandle, RuntimeError> {
|
||||
let token = {
|
||||
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
|
||||
provider.open(kind, descriptor, rights, span)?
|
||||
};
|
||||
Ok(self.insert(Stream::Provider {
|
||||
kind: kind.to_string(),
|
||||
provider,
|
||||
token,
|
||||
closed: false,
|
||||
}))
|
||||
}
|
||||
|
||||
fn insert(&mut self, stream: Stream) -> ResourceHandle {
|
||||
let id = self.next_resource;
|
||||
self.next_resource += 1;
|
||||
self.slots.insert(
|
||||
id,
|
||||
Slot {
|
||||
generation: 1,
|
||||
stream,
|
||||
},
|
||||
);
|
||||
ResourceHandle::from_host(self.run_id, id, 1)
|
||||
}
|
||||
|
||||
fn slot(&self, handle: ResourceHandle, span: Option<Span>) -> Result<&Slot, RuntimeError> {
|
||||
if handle.run_id() != self.run_id {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceFault,
|
||||
span,
|
||||
"Resource handle belongs to a different host run",
|
||||
));
|
||||
}
|
||||
let Some(slot) = self.slots.get(&handle.resource_id()) else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceFault,
|
||||
span,
|
||||
"unknown Resource handle",
|
||||
));
|
||||
};
|
||||
if slot.generation != handle.generation() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceFault,
|
||||
span,
|
||||
"stale Resource handle",
|
||||
));
|
||||
}
|
||||
Ok(slot)
|
||||
}
|
||||
|
||||
fn slot_mut(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
span: Option<Span>,
|
||||
) -> Result<&mut Slot, RuntimeError> {
|
||||
self.slot(handle, span)?;
|
||||
self.slots
|
||||
.get_mut(&handle.resource_id())
|
||||
.ok_or_else(|| unknown_handle(span))
|
||||
}
|
||||
|
||||
/// Read up to `max_bytes` from a file stream.
|
||||
///
|
||||
/// Reads at EOF return `count=0, eof=true`. A read on a closed stream is
|
||||
/// `resource.closed`; stdout is not readable.
|
||||
pub fn read(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
max_bytes: usize,
|
||||
span: Option<Span>,
|
||||
) -> Result<ResourceRead, RuntimeError> {
|
||||
let slot = self.slot_mut(handle, span)?;
|
||||
if slot.closed() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceClosed,
|
||||
span,
|
||||
"Resource handle is closed",
|
||||
));
|
||||
}
|
||||
match &mut slot.stream {
|
||||
Stream::File {
|
||||
bytes, position, ..
|
||||
} => {
|
||||
let start = *position;
|
||||
let end = start.saturating_add(max_bytes).min(bytes.len());
|
||||
let chunk = bytes[start..end].to_vec();
|
||||
*position = end;
|
||||
Ok(ResourceRead {
|
||||
bytes: chunk,
|
||||
eof: end >= bytes.len(),
|
||||
})
|
||||
}
|
||||
Stream::Stdout { .. } => Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
span,
|
||||
"stdout does not support resource.read",
|
||||
)),
|
||||
Stream::FileWriter { .. } => Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
span,
|
||||
"a file write stream does not support resource.read",
|
||||
)),
|
||||
Stream::Provider {
|
||||
provider, token, ..
|
||||
} => {
|
||||
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
|
||||
provider.read(*token, max_bytes, span)
|
||||
}
|
||||
Stream::Tcp { stream, .. } => {
|
||||
let mut buffer = vec![0u8; max_bytes];
|
||||
let count = stream.read(&mut buffer).map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::HostError,
|
||||
span,
|
||||
format!("tcp read failed: {error}"),
|
||||
)
|
||||
})?;
|
||||
buffer.truncate(count);
|
||||
Ok(ResourceRead {
|
||||
bytes: buffer,
|
||||
eof: count == 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Validate a stdout write and return the accepted byte count.
|
||||
///
|
||||
/// The caller performs the output-budget accounting and buffer append; this
|
||||
/// method only checks handle identity, closed state and stream kind.
|
||||
pub fn write(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
bytes: &[u8],
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
let slot = self.slot_mut(handle, span)?;
|
||||
if slot.closed() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceClosed,
|
||||
span,
|
||||
"Resource handle is closed",
|
||||
));
|
||||
}
|
||||
match &mut slot.stream {
|
||||
Stream::Stdout { .. } => Ok(bytes.len()),
|
||||
Stream::File { relative, .. } => Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
span,
|
||||
format!("project file `{relative}` was opened read-only"),
|
||||
)),
|
||||
Stream::FileWriter {
|
||||
relative,
|
||||
bytes: buffer,
|
||||
..
|
||||
} => {
|
||||
let next = buffer.len().checked_add(bytes.len());
|
||||
let Some(next) = next else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!("file write buffer for `{relative}` overflows"),
|
||||
));
|
||||
};
|
||||
if next > crate::fs::MAX_READ_BYTES as usize {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!(
|
||||
"file write buffer for `{relative}` exceeds the {} byte limit",
|
||||
crate::fs::MAX_READ_BYTES
|
||||
),
|
||||
));
|
||||
}
|
||||
buffer.extend_from_slice(bytes);
|
||||
Ok(bytes.len())
|
||||
}
|
||||
Stream::Provider {
|
||||
provider, token, ..
|
||||
} => {
|
||||
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
|
||||
provider.write(*token, bytes, span)
|
||||
}
|
||||
Stream::Tcp { stream, .. } => stream.write(bytes).map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::HostError,
|
||||
span,
|
||||
format!("tcp write failed: {error}"),
|
||||
)
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
/// Close a stream. A second close is `resource.closed`.
|
||||
pub fn close(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
span: Option<Span>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let slot = self.slot_mut(handle, span)?;
|
||||
if slot.closed() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceClosed,
|
||||
span,
|
||||
"Resource handle is already closed",
|
||||
));
|
||||
}
|
||||
match &mut slot.stream {
|
||||
Stream::Stdout { closed } => *closed = true,
|
||||
Stream::File { closed, .. } => *closed = true,
|
||||
Stream::FileWriter { closed, .. } => *closed = true,
|
||||
Stream::Provider {
|
||||
provider,
|
||||
token,
|
||||
closed,
|
||||
..
|
||||
} => {
|
||||
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
|
||||
provider.close(*token, span)?;
|
||||
*closed = true;
|
||||
}
|
||||
Stream::Tcp { stream, closed } => {
|
||||
let _ = stream.shutdown(std::net::Shutdown::Both);
|
||||
*closed = true;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Return the buffered bytes of a file write stream without closing it.
|
||||
///
|
||||
/// `None` means the handle is not a file writer (stdout or a reader). The
|
||||
/// caller must call [`Self::close`] only after the flush succeeds, so a
|
||||
/// failed flush leaves the writer open with its buffer intact.
|
||||
pub fn pending_file_write(
|
||||
&self,
|
||||
handle: ResourceHandle,
|
||||
span: Option<Span>,
|
||||
) -> Result<Option<(String, Vec<u8>)>, RuntimeError> {
|
||||
let slot = self.slot(handle, span)?;
|
||||
if slot.closed() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceClosed,
|
||||
span,
|
||||
"Resource handle is closed",
|
||||
));
|
||||
}
|
||||
match &slot.stream {
|
||||
Stream::FileWriter {
|
||||
relative, bytes, ..
|
||||
} => Ok(Some((relative.clone(), bytes.clone()))),
|
||||
_ => Ok(None),
|
||||
}
|
||||
}
|
||||
|
||||
/// Descriptor kind for diagnostics/trace, if the handle is valid.
|
||||
pub fn kind(&self, handle: ResourceHandle, span: Option<Span>) -> Result<String, RuntimeError> {
|
||||
let slot = self.slot(handle, span)?;
|
||||
Ok(match &slot.stream {
|
||||
Stream::Stdout { .. } => "stdio".to_string(),
|
||||
Stream::File { .. } | Stream::FileWriter { .. } => "file".to_string(),
|
||||
Stream::Provider { kind, .. } => kind.clone(),
|
||||
Stream::Tcp { .. } => "tcp".to_string(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn provider_lock_error(span: Option<Span>) -> RuntimeError {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::HostError,
|
||||
span,
|
||||
"resource provider lock is poisoned",
|
||||
)
|
||||
}
|
||||
|
||||
fn unknown_handle(span: Option<Span>) -> RuntimeError {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceFault,
|
||||
span,
|
||||
"unknown Resource handle",
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn file_reads_are_chunked_with_eof() {
|
||||
let mut table = ResourceTable::new();
|
||||
let handle = table.open_file("data.txt", b"abcdef".to_vec());
|
||||
assert_eq!(
|
||||
table.read(handle, 2, None).unwrap(),
|
||||
ResourceRead {
|
||||
bytes: b"ab".to_vec(),
|
||||
eof: false
|
||||
}
|
||||
);
|
||||
assert_eq!(
|
||||
table.read(handle, 8, None).unwrap(),
|
||||
ResourceRead {
|
||||
bytes: b"cdef".to_vec(),
|
||||
eof: true
|
||||
}
|
||||
);
|
||||
assert_eq!(
|
||||
table.read(handle, 1, None).unwrap(),
|
||||
ResourceRead {
|
||||
bytes: Vec::new(),
|
||||
eof: true
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn forged_and_closed_handles_are_rejected() {
|
||||
let mut table = ResourceTable::new();
|
||||
let handle = table.open_stdout();
|
||||
let forged = ResourceHandle::from_host(handle.run_id() + 1, handle.resource_id(), 1);
|
||||
assert_eq!(
|
||||
table.write(forged, b"x", None).unwrap_err().code(),
|
||||
"runtime.resource"
|
||||
);
|
||||
table.close(handle, None).unwrap();
|
||||
assert_eq!(
|
||||
table.write(handle, b"x", None).unwrap_err().code(),
|
||||
"resource.closed"
|
||||
);
|
||||
assert_eq!(
|
||||
table.close(handle, None).unwrap_err().code(),
|
||||
"resource.closed"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stdout_is_not_readable_and_file_is_not_writable() {
|
||||
let mut table = ResourceTable::new();
|
||||
let stdout = table.open_stdout();
|
||||
assert_eq!(
|
||||
table.read(stdout, 1, None).unwrap_err().code(),
|
||||
"resource.unsupported"
|
||||
);
|
||||
let file = table.open_file("x.txt", Vec::new());
|
||||
assert_eq!(
|
||||
table.write(file, b"x", None).unwrap_err().code(),
|
||||
"resource.unsupported"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,241 @@
|
||||
#![cfg(unix)]
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
use marqdo_host::{Capabilities, Host, StdHost};
|
||||
use marqdo_sandbox::SandboxMode;
|
||||
use marqdo_value::ErrorCategory;
|
||||
|
||||
fn context() -> marqdo_host::ExecutionContext {
|
||||
marqdo_host::ExecutionContext::new("test".to_string(), None)
|
||||
}
|
||||
|
||||
fn handle(value: marqdo_value::Value) -> marqdo_value::ResourceHandle {
|
||||
match value {
|
||||
marqdo_value::Value::Resource(handle) => handle,
|
||||
other => panic!("expected resource, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn cc_available() -> bool {
|
||||
Command::new("cc")
|
||||
.arg("--version")
|
||||
.output()
|
||||
.map(|output| output.status.success())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
fn build_provider_with_version(dir: &Path, version: u32) -> PathBuf {
|
||||
let source = dir.join("provider.c");
|
||||
std::fs::write(
|
||||
&source,
|
||||
r#"
|
||||
#include <stdint.h>
|
||||
#include <stddef.h>
|
||||
#include <string.h>
|
||||
|
||||
static const char DATA[] = "c-abi";
|
||||
static size_t position = 0;
|
||||
static uint64_t next_token = 1;
|
||||
|
||||
#ifndef PROVIDER_VERSION
|
||||
#define PROVIDER_VERSION 1u
|
||||
#endif
|
||||
uint32_t marqdo_resource_provider_abi_version(void) { return PROVIDER_VERSION; }
|
||||
|
||||
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
|
||||
const char *rights_csv, uint64_t *out_token) {
|
||||
(void)kind; (void)descriptor; (void)rights_csv;
|
||||
position = 0;
|
||||
*out_token = next_token++;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
|
||||
uint32_t capacity, uint32_t *out_read,
|
||||
int32_t *out_eof) {
|
||||
(void)token;
|
||||
size_t remaining = sizeof(DATA) - 1 - position;
|
||||
size_t count = remaining < capacity ? remaining : capacity;
|
||||
memcpy(buffer, DATA + position, count);
|
||||
position += count;
|
||||
*out_read = (uint32_t)count;
|
||||
*out_eof = position >= sizeof(DATA) - 1 ? 1 : 0;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
|
||||
uint32_t length, uint32_t *out_written) {
|
||||
(void)token; (void)bytes;
|
||||
*out_written = length;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t marqdo_resource_close(uint64_t token) { (void)token; return 0; }
|
||||
|
||||
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity) {
|
||||
if (capacity > 0) buffer[0] = 0;
|
||||
return 0;
|
||||
}
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
let extension = if cfg!(target_os = "macos") {
|
||||
"dylib"
|
||||
} else {
|
||||
"so"
|
||||
};
|
||||
let library = dir.join(format!("libecho_provider.{extension}"));
|
||||
let status = Command::new("cc")
|
||||
.args([
|
||||
"-shared",
|
||||
"-fPIC",
|
||||
"-O2",
|
||||
&format!("-DPROVIDER_VERSION={version}"),
|
||||
"-o",
|
||||
])
|
||||
.arg(&library)
|
||||
.arg(&source)
|
||||
.status()
|
||||
.expect("run cc");
|
||||
assert!(status.success(), "C provider build failed");
|
||||
library
|
||||
}
|
||||
|
||||
fn build_provider(dir: &Path) -> PathBuf {
|
||||
build_provider_with_version(dir, 1)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn c_abi_provider_round_trips() {
|
||||
if !cc_available() {
|
||||
eprintln!("skipping C provider test: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let library = build_provider(dir.path());
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("cabi"),
|
||||
project.path(),
|
||||
);
|
||||
host.register_c_resource_provider("cabi", &["cabi.sensor"], &library)
|
||||
.unwrap();
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("cabi.sensor", "sensor-1", &["read".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let first = host.resource_read(handle, 2, &ctx).unwrap();
|
||||
assert_eq!(first.bytes, b"c-");
|
||||
assert!(!first.eof);
|
||||
let rest = host.resource_read(handle, 16, &ctx).unwrap();
|
||||
assert_eq!(rest.bytes, b"abi");
|
||||
assert!(rest.eof);
|
||||
assert_eq!(host.resource_write(handle, b"xyz", &ctx).unwrap(), 3);
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
assert_eq!(
|
||||
host.resource_close(handle, &ctx).unwrap_err().code(),
|
||||
"resource.closed"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn c_abi_provider_loading_fails_closed() {
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("cabi"),
|
||||
project.path(),
|
||||
);
|
||||
let error = host
|
||||
.register_c_resource_provider(
|
||||
"cabi",
|
||||
&["cabi.sensor"],
|
||||
Path::new("/definitely/missing/marqdo-provider.so"),
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::KernelUnavailable);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn c_abi_version_mismatch_and_modes_fail_closed() {
|
||||
if !cc_available() {
|
||||
eprintln!("skipping C provider test: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let wrong = build_provider_with_version(dir.path(), 2);
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("cabi"),
|
||||
project.path(),
|
||||
);
|
||||
let error = host
|
||||
.register_c_resource_provider("cabi", &["cabi.sensor"], &wrong)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::ResourceUnsupported);
|
||||
|
||||
let good = build_provider(dir.path());
|
||||
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
|
||||
let mut host = StdHost::new(
|
||||
mode,
|
||||
Capabilities::project().allow_resource_provider("cabi"),
|
||||
project.path(),
|
||||
);
|
||||
host.register_c_resource_provider("cabi", &["cabi.sensor"], &good)
|
||||
.unwrap();
|
||||
let error = host
|
||||
.resource_open("cabi.sensor", "s", &["read".to_string()], &context())
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn provider_package_manifest_installs_a_c_abi_library() {
|
||||
if !cc_available() {
|
||||
eprintln!("skipping C provider test: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let library = build_provider(dir.path());
|
||||
let manifest = dir.path().join("provider.json");
|
||||
let platform = if cfg!(target_os = "linux") {
|
||||
"linux"
|
||||
} else if cfg!(target_os = "macos") {
|
||||
"macos"
|
||||
} else {
|
||||
"default"
|
||||
};
|
||||
std::fs::write(
|
||||
&manifest,
|
||||
serde_json::json!({
|
||||
"schema": "marqdo.provider",
|
||||
"schema_version": 1,
|
||||
"name": "cabi",
|
||||
"kinds": ["cabi.sensor"],
|
||||
"abi": "c-abi-1",
|
||||
"library": { platform: library.file_name().unwrap().to_string_lossy() }
|
||||
})
|
||||
.to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("cabi"),
|
||||
project.path(),
|
||||
);
|
||||
host.register_provider_package(&manifest).unwrap();
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("cabi.sensor", "s", &["read".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let read = host.resource_read(handle, 2, &ctx).unwrap();
|
||||
assert_eq!(read.bytes, b"c-");
|
||||
}
|
||||
@@ -0,0 +1,456 @@
|
||||
//! Resource/stream host boundary tests (ADR-0025).
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
use marqdo_host::{Capabilities, ExecutionContext, Host, HostOp, StdHost};
|
||||
use marqdo_sandbox::SandboxMode;
|
||||
use marqdo_value::{ErrorCategory, Value};
|
||||
|
||||
fn context() -> ExecutionContext {
|
||||
ExecutionContext::new("index.mq.md".to_string(), None)
|
||||
}
|
||||
|
||||
fn handle(value: Value) -> marqdo_value::ResourceHandle {
|
||||
match value {
|
||||
Value::Resource(handle) => handle,
|
||||
other => panic!("expected a Resource handle, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn write_project_file(root: &Path, name: &str, contents: &[u8]) {
|
||||
std::fs::write(root.join(name), contents).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pure_mode_writes_stdout_through_a_resource() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
assert_eq!(host.resource_write(handle, b"hello", &ctx).unwrap(), 5);
|
||||
assert_eq!(host.take_output(), b"hello");
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
let error = host.resource_close(handle, &ctx).unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::ResourceClosed);
|
||||
assert_eq!(error.code(), "resource.closed");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stdout_write_obeys_the_output_budget_atomically() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
|
||||
host.budget.max_output_bytes = 3;
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let error = host.resource_write(handle, b"hello", &ctx).unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
|
||||
assert!(host.take_output().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn strict_mode_allows_only_the_existing_stdio_console_channel() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(SandboxMode::Strict, Capabilities::pure(), dir.path());
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
assert_eq!(host.resource_write(handle, b"strict", &ctx).unwrap(), 6);
|
||||
assert_eq!(host.take_output(), b"strict");
|
||||
|
||||
let file = host
|
||||
.resource_open("file", "a.txt", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(file.category, ErrorCategory::PermissionDenied);
|
||||
let socket = host
|
||||
.resource_open("socket", "example.test", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(socket.category, ErrorCategory::PermissionDenied);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resource_handles_cannot_cross_the_generic_host_boundary() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let error = host
|
||||
.invoke(HostOp::FsRead, vec![Value::Resource(handle)], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::Type);
|
||||
assert!(error.message.contains("Resource handles"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pure_mode_refuses_file_resources_even_with_a_grant() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
write_project_file(dir.path(), "a.txt", b"abcdef");
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Pure,
|
||||
Capabilities::pure().allow_read("a.txt"),
|
||||
dir.path(),
|
||||
);
|
||||
let error = host
|
||||
.resource_open("file", "a.txt", &["read".to_string()], &context())
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn project_file_writes_buffer_until_close_and_require_a_grant() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_write("out.bin"),
|
||||
dir.path(),
|
||||
);
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("file", "out.bin", &["write".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
assert_eq!(host.resource_write(handle, b"abc", &ctx).unwrap(), 3);
|
||||
assert_eq!(host.resource_write(handle, b"def", &ctx).unwrap(), 3);
|
||||
assert!(
|
||||
!dir.path().join("out.bin").exists(),
|
||||
"write must not flush early"
|
||||
);
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
assert_eq!(
|
||||
std::fs::read(dir.path().join("out.bin")).unwrap(),
|
||||
b"abcdef"
|
||||
);
|
||||
|
||||
let error = host.resource_read(handle, 1, &ctx).unwrap_err();
|
||||
assert_eq!(error.code(), "resource.closed");
|
||||
let second = host.resource_close(handle, &ctx).unwrap_err();
|
||||
assert_eq!(second.code(), "resource.closed");
|
||||
|
||||
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
|
||||
let error = denied
|
||||
.resource_open("file", "out.bin", &["write".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn file_write_buffer_has_a_hard_ceiling() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_write("big.bin"),
|
||||
dir.path(),
|
||||
);
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("file", "big.bin", &["write".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let oversized = vec![0u8; marqdo_host::fs::MAX_READ_BYTES as usize + 1];
|
||||
let error = host.resource_write(handle, &oversized, &ctx).unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
|
||||
assert_eq!(host.resource_write(handle, b"ok", &ctx).unwrap(), 2);
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
assert_eq!(std::fs::read(dir.path().join("big.bin")).unwrap(), b"ok");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn project_file_reads_are_chunked_and_require_a_grant() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
write_project_file(dir.path(), "a.txt", b"abcdef");
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_read("a.txt"),
|
||||
dir.path(),
|
||||
);
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("file", "a.txt", &["read".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let first = host.resource_read(handle, 2, &ctx).unwrap();
|
||||
assert_eq!(first.bytes, b"ab");
|
||||
assert!(!first.eof);
|
||||
let second = host.resource_read(handle, 8, &ctx).unwrap();
|
||||
assert_eq!(second.bytes, b"cdef");
|
||||
assert!(second.eof);
|
||||
assert!(host.resource_read(handle, 1, &ctx).unwrap().eof);
|
||||
|
||||
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
|
||||
let error = denied
|
||||
.resource_open("file", "a.txt", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn traversal_and_unsupported_rights_are_refused() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_read("**"),
|
||||
dir.path(),
|
||||
);
|
||||
let ctx = context();
|
||||
let traversal = host
|
||||
.resource_open("file", "../secret", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(traversal.category, ErrorCategory::PermissionDenied);
|
||||
let write_rights = host
|
||||
.resource_open("stdio", "stdout", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(write_rights.category, ErrorCategory::ResourceUnsupported);
|
||||
let unknown_kind = host
|
||||
.resource_open("socket", "example.test", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(unknown_kind.category, ErrorCategory::ResourceUnsupported);
|
||||
}
|
||||
|
||||
struct MockProvider {
|
||||
data: Vec<u8>,
|
||||
position: usize,
|
||||
written: Vec<u8>,
|
||||
version: u32,
|
||||
kinds: Vec<&'static str>,
|
||||
}
|
||||
|
||||
impl MockProvider {
|
||||
fn new(data: &[u8], version: u32, kinds: Vec<&'static str>) -> Self {
|
||||
Self {
|
||||
data: data.to_vec(),
|
||||
position: 0,
|
||||
written: Vec::new(),
|
||||
version,
|
||||
kinds,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl marqdo_host::ResourceProvider for MockProvider {
|
||||
fn name(&self) -> &str {
|
||||
"mock"
|
||||
}
|
||||
fn protocol_version(&self) -> u32 {
|
||||
self.version
|
||||
}
|
||||
fn kinds(&self) -> &[&str] {
|
||||
&self.kinds
|
||||
}
|
||||
fn open(
|
||||
&mut self,
|
||||
_kind: &str,
|
||||
_descriptor: &str,
|
||||
_rights: &[String],
|
||||
_span: Option<marqdo_span::Span>,
|
||||
) -> Result<u64, marqdo_value::RuntimeError> {
|
||||
self.position = 0;
|
||||
Ok(7)
|
||||
}
|
||||
fn read(
|
||||
&mut self,
|
||||
token: u64,
|
||||
max_bytes: usize,
|
||||
_span: Option<marqdo_span::Span>,
|
||||
) -> Result<marqdo_host::ResourceRead, marqdo_value::RuntimeError> {
|
||||
assert_eq!(token, 7);
|
||||
let start = self.position;
|
||||
let end = start.saturating_add(max_bytes).min(self.data.len());
|
||||
self.position = end;
|
||||
Ok(marqdo_host::ResourceRead {
|
||||
bytes: self.data[start..end].to_vec(),
|
||||
eof: end >= self.data.len(),
|
||||
})
|
||||
}
|
||||
fn write(
|
||||
&mut self,
|
||||
token: u64,
|
||||
bytes: &[u8],
|
||||
_span: Option<marqdo_span::Span>,
|
||||
) -> Result<usize, marqdo_value::RuntimeError> {
|
||||
assert_eq!(token, 7);
|
||||
self.written.extend_from_slice(bytes);
|
||||
Ok(bytes.len())
|
||||
}
|
||||
fn close(
|
||||
&mut self,
|
||||
token: u64,
|
||||
_span: Option<marqdo_span::Span>,
|
||||
) -> Result<(), marqdo_value::RuntimeError> {
|
||||
assert_eq!(token, 7);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registered_provider_streams_work_with_a_grant_in_project_mode() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("mock"),
|
||||
dir.path(),
|
||||
);
|
||||
host.register_resource_provider(Box::new(MockProvider::new(
|
||||
b"abcdef",
|
||||
1,
|
||||
vec!["mock.sensor"],
|
||||
)))
|
||||
.unwrap();
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("mock.sensor", "sensor-1", &["read".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let first = host.resource_read(handle, 2, &ctx).unwrap();
|
||||
assert_eq!(first.bytes, b"ab");
|
||||
assert!(!first.eof);
|
||||
let rest = host.resource_read(handle, 16, &ctx).unwrap();
|
||||
assert_eq!(rest.bytes, b"cdef");
|
||||
assert!(rest.eof);
|
||||
assert_eq!(host.resource_write(handle, b"x", &ctx).unwrap(), 1);
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
assert_eq!(
|
||||
host.resource_close(handle, &ctx).unwrap_err().code(),
|
||||
"resource.closed"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn provider_requires_a_name_grant_and_project_mode() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let ctx = context();
|
||||
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
|
||||
denied
|
||||
.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
|
||||
.unwrap();
|
||||
let error = denied
|
||||
.resource_open("mock.sensor", "s", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
|
||||
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
|
||||
let mut host = StdHost::new(
|
||||
mode,
|
||||
Capabilities::pure().allow_resource_provider("mock"),
|
||||
dir.path(),
|
||||
);
|
||||
host.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
|
||||
.unwrap();
|
||||
let error = host
|
||||
.resource_open("mock.sensor", "s", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn provider_registration_rejects_versions_reserved_names_and_conflicts() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("mock"),
|
||||
dir.path(),
|
||||
);
|
||||
let bad_version = host
|
||||
.register_resource_provider(Box::new(MockProvider::new(b"x", 2, vec!["mock.sensor"])))
|
||||
.unwrap_err();
|
||||
assert_eq!(bad_version.category, ErrorCategory::ResourceUnsupported);
|
||||
let reserved = host
|
||||
.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["stdio"])))
|
||||
.unwrap_err();
|
||||
assert_eq!(reserved.category, ErrorCategory::ResourceUnsupported);
|
||||
host.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
|
||||
.unwrap();
|
||||
let conflict = host
|
||||
.register_resource_provider(Box::new(MockProvider::new(b"y", 1, vec!["mock.sensor"])))
|
||||
.unwrap_err();
|
||||
assert_eq!(conflict.category, ErrorCategory::ResourceUnsupported);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tcp_resource_round_trips_with_a_network_grant() {
|
||||
use std::io::{Read, Write};
|
||||
use std::net::TcpListener;
|
||||
|
||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
let address = listener.local_addr().unwrap();
|
||||
let server = std::thread::spawn(move || {
|
||||
let (mut stream, _) = listener.accept().unwrap();
|
||||
let mut buffer = [0u8; 4];
|
||||
stream.read_exact(&mut buffer).unwrap();
|
||||
stream.write_all(&buffer).unwrap();
|
||||
});
|
||||
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_net("127.0.0.1"),
|
||||
dir.path(),
|
||||
);
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open(
|
||||
"tcp",
|
||||
&address.to_string(),
|
||||
&["read".to_string(), "write".to_string()],
|
||||
&ctx,
|
||||
)
|
||||
.unwrap(),
|
||||
);
|
||||
assert_eq!(host.resource_write(handle, b"ping", &ctx).unwrap(), 4);
|
||||
let read = host.resource_read(handle, 4, &ctx).unwrap();
|
||||
assert_eq!(read.bytes, b"ping");
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
server.join().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tcp_requires_network_grant_and_rejects_dns_names() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
|
||||
let ctx = context();
|
||||
let denied = host
|
||||
.resource_open(
|
||||
"tcp",
|
||||
"127.0.0.1:9",
|
||||
&["read".to_string(), "write".to_string()],
|
||||
&ctx,
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(denied.category, ErrorCategory::PermissionDenied);
|
||||
let dns = host
|
||||
.resource_open(
|
||||
"tcp",
|
||||
"example.test:80",
|
||||
&["read".to_string(), "write".to_string()],
|
||||
&ctx,
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(dns.category, ErrorCategory::ResourceUnsupported);
|
||||
|
||||
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
|
||||
let mut host = StdHost::new(
|
||||
mode,
|
||||
Capabilities::project().allow_net("127.0.0.1"),
|
||||
dir.path(),
|
||||
);
|
||||
let error = host
|
||||
.resource_open(
|
||||
"tcp",
|
||||
"127.0.0.1:9",
|
||||
&["read".to_string(), "write".to_string()],
|
||||
&ctx,
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,162 @@
|
||||
use marqdo_host::{Capabilities, Host, StdHost};
|
||||
use marqdo_sandbox::SandboxMode;
|
||||
use marqdo_value::ErrorCategory;
|
||||
|
||||
fn context() -> marqdo_host::ExecutionContext {
|
||||
marqdo_host::ExecutionContext::new("test".to_string(), None)
|
||||
}
|
||||
|
||||
fn handle(value: marqdo_value::Value) -> marqdo_value::ResourceHandle {
|
||||
match value {
|
||||
marqdo_value::Value::Resource(handle) => handle,
|
||||
other => panic!("expected resource, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn helper() -> (tempfile::TempDir, std::path::PathBuf) {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let script = dir.path().join("provider.py");
|
||||
std::fs::write(
|
||||
&script,
|
||||
r#"
|
||||
import sys, json, struct
|
||||
|
||||
def read_frame():
|
||||
header = sys.stdin.buffer.read(4)
|
||||
if len(header) < 4:
|
||||
return None
|
||||
size = struct.unpack(">I", header)[0]
|
||||
return json.loads(sys.stdin.buffer.read(size))
|
||||
|
||||
def write_frame(value):
|
||||
payload = json.dumps(value).encode()
|
||||
sys.stdout.buffer.write(struct.pack(">I", len(payload)) + payload)
|
||||
sys.stdout.buffer.flush()
|
||||
|
||||
data = b"hello world"
|
||||
position = 0
|
||||
while True:
|
||||
request = read_frame()
|
||||
if request is None:
|
||||
break
|
||||
op = request["op"]
|
||||
if op == "ping":
|
||||
write_frame({"ok": True})
|
||||
elif op == "open":
|
||||
write_frame({"ok": True, "token": 7})
|
||||
elif op == "read":
|
||||
count = request["max_bytes"]
|
||||
chunk = data[position:position + count]
|
||||
position += len(chunk)
|
||||
write_frame({"ok": True, "bytes": list(chunk), "eof": position >= len(data)})
|
||||
elif op == "write":
|
||||
write_frame({"ok": True, "count": len(request["bytes"])})
|
||||
elif op == "close":
|
||||
write_frame({"ok": True})
|
||||
elif op == "shutdown":
|
||||
break
|
||||
else:
|
||||
write_frame({"ok": False, "code": "resource.unsupported", "message": op})
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
(dir, script)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn worker_provider_round_trips_through_framed_json() {
|
||||
let (dir, script) = helper();
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("worker"),
|
||||
project.path(),
|
||||
);
|
||||
host.register_worker_resource_provider(
|
||||
"worker",
|
||||
&["worker.sensor"],
|
||||
&std::path::PathBuf::from("python3"),
|
||||
&[script.to_string_lossy().to_string()],
|
||||
)
|
||||
.unwrap();
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("worker.sensor", "sensor-1", &["read".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let first = host.resource_read(handle, 5, &ctx).unwrap();
|
||||
assert_eq!(first.bytes, b"hello");
|
||||
assert!(!first.eof);
|
||||
let rest = host.resource_read(handle, 64, &ctx).unwrap();
|
||||
assert_eq!(rest.bytes, b" world");
|
||||
assert!(rest.eof);
|
||||
assert_eq!(host.resource_write(handle, b"abc", &ctx).unwrap(), 3);
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
assert_eq!(
|
||||
host.resource_close(handle, &ctx).unwrap_err().code(),
|
||||
"resource.closed"
|
||||
);
|
||||
drop(dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn worker_provider_registration_requires_a_grant_and_project_mode() {
|
||||
let (_dir, script) = helper();
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let ctx = context();
|
||||
let mut denied = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project(),
|
||||
project.path(),
|
||||
);
|
||||
denied
|
||||
.register_worker_resource_provider(
|
||||
"worker",
|
||||
&["worker.sensor"],
|
||||
&std::path::PathBuf::from("python3"),
|
||||
&[script.to_string_lossy().to_string()],
|
||||
)
|
||||
.unwrap();
|
||||
let error = denied
|
||||
.resource_open("worker.sensor", "s", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
|
||||
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
|
||||
let mut host = StdHost::new(
|
||||
mode,
|
||||
Capabilities::project().allow_resource_provider("worker"),
|
||||
project.path(),
|
||||
);
|
||||
host.register_worker_resource_provider(
|
||||
"worker",
|
||||
&["worker.sensor"],
|
||||
&std::path::PathBuf::from("python3"),
|
||||
&[script.to_string_lossy().to_string()],
|
||||
)
|
||||
.unwrap();
|
||||
let error = host
|
||||
.resource_open("worker.sensor", "s", &["read".to_string()], &ctx)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::PermissionDenied);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_worker_program_is_a_structured_error() {
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("worker"),
|
||||
project.path(),
|
||||
);
|
||||
let error = host
|
||||
.register_worker_resource_provider(
|
||||
"worker",
|
||||
&["worker.sensor"],
|
||||
&std::path::PathBuf::from("/definitely/missing/marqdo-provider"),
|
||||
&[],
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::KernelUnavailable);
|
||||
}
|
||||
@@ -167,6 +167,144 @@ const FS_WRITE: Builtin = Builtin {
|
||||
named_only: false,
|
||||
effectful: true,
|
||||
};
|
||||
const MEMORY_NEW: Builtin = Builtin {
|
||||
name: "memory.new",
|
||||
aliases: &[],
|
||||
params: &[req("initial_pages"), req("max_pages")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const MEMORY_SIZE: Builtin = Builtin {
|
||||
name: "memory.size",
|
||||
aliases: &[],
|
||||
params: &[req("memory")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const MEMORY_GROW: Builtin = Builtin {
|
||||
name: "memory.grow",
|
||||
aliases: &[],
|
||||
params: &[req("memory"), req("pages")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const LIST_APPEND: Builtin = Builtin {
|
||||
name: "list.append",
|
||||
aliases: &[],
|
||||
params: &[req("list"), req("value")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const LIST_CONCAT: Builtin = Builtin {
|
||||
name: "list.concat",
|
||||
aliases: &[],
|
||||
params: &[req("left"), req("right")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const MAP_KEYS: Builtin = Builtin {
|
||||
name: "map.keys",
|
||||
aliases: &[],
|
||||
params: &[req("map")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const MAP_HAS: Builtin = Builtin {
|
||||
name: "map.has",
|
||||
aliases: &[],
|
||||
params: &[req("map"), req("key")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const MAP_GET: Builtin = Builtin {
|
||||
name: "map.get",
|
||||
aliases: &[],
|
||||
params: &[req("map"), req("key")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const MAP_SET: Builtin = Builtin {
|
||||
name: "map.set",
|
||||
aliases: &[],
|
||||
params: &[req("map"), req("key"), req("value")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const MAP_REMOVE: Builtin = Builtin {
|
||||
name: "map.remove",
|
||||
aliases: &[],
|
||||
params: &[req("map"), req("key")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const TEXT_CODEPOINTS: Builtin = Builtin {
|
||||
name: "text.codepoints",
|
||||
aliases: &[],
|
||||
params: &[req("text")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const TEXT_FROM_CODEPOINT: Builtin = Builtin {
|
||||
name: "text.from_codepoint",
|
||||
aliases: &[],
|
||||
params: &[req("codepoint")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const TABLE_COLUMNS: Builtin = Builtin {
|
||||
name: "table.columns",
|
||||
aliases: &[],
|
||||
params: &[req("table")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const TABLE_ROW: Builtin = Builtin {
|
||||
name: "table.row",
|
||||
aliases: &[],
|
||||
params: &[req("table"), req("row")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
const RESOURCE_OPEN: Builtin = Builtin {
|
||||
name: "resource.open",
|
||||
aliases: &[],
|
||||
params: &[req("kind"), req("descriptor"), req("rights")],
|
||||
named_only: false,
|
||||
effectful: true,
|
||||
};
|
||||
const RESOURCE_READ: Builtin = Builtin {
|
||||
name: "resource.read",
|
||||
aliases: &[],
|
||||
params: &[
|
||||
req("handle"),
|
||||
req("memory"),
|
||||
req("offset"),
|
||||
req("max_bytes"),
|
||||
],
|
||||
named_only: false,
|
||||
effectful: true,
|
||||
};
|
||||
const RESOURCE_WRITE: Builtin = Builtin {
|
||||
name: "resource.write",
|
||||
aliases: &[],
|
||||
params: &[req("handle"), req("memory"), req("offset"), req("length")],
|
||||
named_only: false,
|
||||
effectful: true,
|
||||
};
|
||||
const RESOURCE_CLOSE: Builtin = Builtin {
|
||||
name: "resource.close",
|
||||
aliases: &[],
|
||||
params: &[req("handle")],
|
||||
named_only: false,
|
||||
effectful: true,
|
||||
};
|
||||
const TABLE_CELL: Builtin = Builtin {
|
||||
name: "table.cell",
|
||||
aliases: &[],
|
||||
params: &[req("table"), req("row"), req("column")],
|
||||
named_only: false,
|
||||
effectful: false,
|
||||
};
|
||||
|
||||
/// All builtins known to the language. Host availability is decided at call
|
||||
/// time by the capability layer, not here.
|
||||
@@ -188,6 +326,25 @@ pub const BUILTINS: &[Builtin] = &[
|
||||
FOREIGN_CALL,
|
||||
FS_READ,
|
||||
FS_WRITE,
|
||||
MEMORY_NEW,
|
||||
MEMORY_SIZE,
|
||||
MEMORY_GROW,
|
||||
LIST_APPEND,
|
||||
LIST_CONCAT,
|
||||
MAP_KEYS,
|
||||
MAP_HAS,
|
||||
MAP_GET,
|
||||
MAP_SET,
|
||||
MAP_REMOVE,
|
||||
TEXT_CODEPOINTS,
|
||||
TEXT_FROM_CODEPOINT,
|
||||
TABLE_COLUMNS,
|
||||
TABLE_ROW,
|
||||
TABLE_CELL,
|
||||
RESOURCE_OPEN,
|
||||
RESOURCE_READ,
|
||||
RESOURCE_WRITE,
|
||||
RESOURCE_CLOSE,
|
||||
];
|
||||
|
||||
/// Find a builtin by canonical name or alias.
|
||||
|
||||
@@ -224,16 +224,51 @@ pub struct Function {
|
||||
pub body: Vec<Stmt>,
|
||||
}
|
||||
|
||||
/// Writable place of an assignment (ADR-0022).
|
||||
///
|
||||
/// The Markdown surface is unchanged: both variants are `strong` nodes whose
|
||||
/// payload contains `target = expression`. `Local` rebinds a frame slot;
|
||||
/// `Index` writes one byte into an addressable runtime `Memory` region and is
|
||||
/// rejected for every other base value.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(tag = "place", rename_all = "snake_case")]
|
||||
pub enum PlaceTarget {
|
||||
/// Local slot rebinding.
|
||||
Local {
|
||||
/// Target name.
|
||||
name: String,
|
||||
/// Span of the target name.
|
||||
name_span: Span,
|
||||
},
|
||||
/// `base[index] = value`; only a `Memory` base is writable.
|
||||
Index {
|
||||
/// Base expression, evaluated first.
|
||||
base: Expr,
|
||||
/// Index expression, evaluated second.
|
||||
index: Expr,
|
||||
/// Span from the base start through the closing `]`.
|
||||
span: Span,
|
||||
},
|
||||
}
|
||||
|
||||
impl PlaceTarget {
|
||||
/// Best known source span of the place.
|
||||
pub fn span(&self) -> Span {
|
||||
match self {
|
||||
PlaceTarget::Local { name_span, .. } => *name_span,
|
||||
PlaceTarget::Index { span, .. } => *span,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// An executable statement.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(tag = "stmt", rename_all = "snake_case")]
|
||||
pub enum Stmt {
|
||||
/// Local assignment (`strong` node with `name = expr`).
|
||||
/// Assignment to a local slot or a `Memory` index.
|
||||
Assign {
|
||||
/// Target name.
|
||||
name: String,
|
||||
/// Span of the target name (approximated by the payload span).
|
||||
name_span: Span,
|
||||
/// Writable place.
|
||||
target: PlaceTarget,
|
||||
/// Value expression.
|
||||
value: Expr,
|
||||
/// Span of the whole statement.
|
||||
@@ -482,6 +517,10 @@ pub struct Program {
|
||||
/// metadata only: `check` never loads a library.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub external_callables: Vec<callable::CallableDescriptor>,
|
||||
/// Compat-B/C `打印`/`print` destination (ADR-0030/0031). When set, the
|
||||
/// runtime calls this stdlib function; there is no legacy HostOp path.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub print_override: Option<FunctionId>,
|
||||
}
|
||||
|
||||
impl Program {
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
[package]
|
||||
name = "marqdo-provider-worker"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
publish = false
|
||||
|
||||
[[bin]]
|
||||
name = "marqdo-provider-worker"
|
||||
path = "src/main.rs"
|
||||
|
||||
[dependencies]
|
||||
libloading = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
|
||||
[dev-dependencies]
|
||||
marqdo-host = { path = "../marqdo-host" }
|
||||
marqdo-sandbox = { path = "../marqdo-sandbox" }
|
||||
marqdo-value = { path = "../marqdo-value" }
|
||||
tempfile = { workspace = true }
|
||||
@@ -0,0 +1,261 @@
|
||||
//! Trusted C ABI provider worker (ADR-0038).
|
||||
//!
|
||||
//! This helper loads one trusted `c-abi-1` shared library and serves the
|
||||
//! ADR-0035 framed-JSON resource protocol over stdin/stdout. It is a transport
|
||||
//! boundary, not a sandbox: the library runs with the worker's privileges.
|
||||
|
||||
use std::ffi::{c_char, CString};
|
||||
use std::io::{Read, Write};
|
||||
use std::path::PathBuf;
|
||||
|
||||
use libloading::Library;
|
||||
use serde_json::{json, Value as Json};
|
||||
|
||||
const ABI_VERSION: u32 = 1;
|
||||
const MAX_FRAME_BYTES: usize = 1024 * 1024;
|
||||
const ERROR_MAX: usize = 1024;
|
||||
|
||||
type AbiVersionFn = unsafe extern "C" fn() -> u32;
|
||||
type OpenFn = unsafe extern "C" fn(*const c_char, *const c_char, *const c_char, *mut u64) -> i32;
|
||||
type ReadFn = unsafe extern "C" fn(u64, *mut u8, u32, *mut u32, *mut i32) -> i32;
|
||||
type WriteFn = unsafe extern "C" fn(u64, *const u8, u32, *mut u32) -> i32;
|
||||
type CloseFn = unsafe extern "C" fn(u64) -> i32;
|
||||
type LastErrorFn = unsafe extern "C" fn(*mut c_char, u32) -> i32;
|
||||
|
||||
struct Provider {
|
||||
library: Library,
|
||||
}
|
||||
|
||||
impl Provider {
|
||||
fn load(path: &PathBuf) -> Result<Self, String> {
|
||||
let library = unsafe { Library::new(path) }
|
||||
.map_err(|error| format!("cannot load `{}`: {error}", path.display()))?;
|
||||
let version: libloading::Symbol<AbiVersionFn> = unsafe {
|
||||
library
|
||||
.get(b"marqdo_resource_provider_abi_version\0")
|
||||
.map_err(|error| format!("missing abi_version: {error}"))?
|
||||
};
|
||||
if unsafe { version() } != ABI_VERSION {
|
||||
return Err(format!("provider ABI is not {ABI_VERSION}"));
|
||||
}
|
||||
Ok(Self { library })
|
||||
}
|
||||
|
||||
fn last_error(&self) -> String {
|
||||
let Ok(symbol) = (unsafe {
|
||||
self.library
|
||||
.get::<LastErrorFn>(b"marqdo_resource_last_error\0")
|
||||
}) else {
|
||||
return "C provider failed".to_string();
|
||||
};
|
||||
let mut buffer = vec![0i8; ERROR_MAX];
|
||||
if unsafe { symbol(buffer.as_mut_ptr(), ERROR_MAX as u32) } != 0 {
|
||||
return "C provider failed".to_string();
|
||||
}
|
||||
let bytes = buffer
|
||||
.iter()
|
||||
.map(|value| *value as u8)
|
||||
.take_while(|value| *value != 0)
|
||||
.collect::<Vec<_>>();
|
||||
String::from_utf8_lossy(&bytes).into_owned()
|
||||
}
|
||||
|
||||
fn open(&self, kind: &str, descriptor: &str, rights: &[String]) -> Result<u64, String> {
|
||||
let kind = CString::new(kind).map_err(|_| "kind has NUL".to_string())?;
|
||||
let descriptor = CString::new(descriptor).map_err(|_| "descriptor has NUL".to_string())?;
|
||||
let rights = CString::new(rights.join(",")).map_err(|_| "rights have NUL".to_string())?;
|
||||
let symbol: libloading::Symbol<OpenFn> = unsafe {
|
||||
self.library
|
||||
.get(b"marqdo_resource_open\0")
|
||||
.map_err(|error| format!("missing open: {error}"))?
|
||||
};
|
||||
let mut token = 0u64;
|
||||
if unsafe {
|
||||
symbol(
|
||||
kind.as_ptr(),
|
||||
descriptor.as_ptr(),
|
||||
rights.as_ptr(),
|
||||
&mut token,
|
||||
)
|
||||
} != 0
|
||||
{
|
||||
return Err(self.last_error());
|
||||
}
|
||||
Ok(token)
|
||||
}
|
||||
|
||||
fn read(&self, token: u64, max_bytes: usize) -> Result<(Vec<u8>, bool), String> {
|
||||
let capacity = u32::try_from(max_bytes).unwrap_or(u32::MAX);
|
||||
let mut buffer = vec![0u8; capacity as usize];
|
||||
let mut read = 0u32;
|
||||
let mut eof = 0i32;
|
||||
let symbol: libloading::Symbol<ReadFn> = unsafe {
|
||||
self.library
|
||||
.get(b"marqdo_resource_read\0")
|
||||
.map_err(|error| format!("missing read: {error}"))?
|
||||
};
|
||||
if unsafe { symbol(token, buffer.as_mut_ptr(), capacity, &mut read, &mut eof) } != 0 {
|
||||
return Err(self.last_error());
|
||||
}
|
||||
buffer.truncate(read as usize);
|
||||
Ok((buffer, eof != 0))
|
||||
}
|
||||
|
||||
fn write(&self, token: u64, bytes: &[u8]) -> Result<usize, String> {
|
||||
let length = u32::try_from(bytes.len()).map_err(|_| "write too large".to_string())?;
|
||||
let mut written = 0u32;
|
||||
let symbol: libloading::Symbol<WriteFn> = unsafe {
|
||||
self.library
|
||||
.get(b"marqdo_resource_write\0")
|
||||
.map_err(|error| format!("missing write: {error}"))?
|
||||
};
|
||||
if unsafe { symbol(token, bytes.as_ptr(), length, &mut written) } != 0 {
|
||||
return Err(self.last_error());
|
||||
}
|
||||
Ok(written as usize)
|
||||
}
|
||||
|
||||
fn close(&self, token: u64) -> Result<(), String> {
|
||||
let symbol: libloading::Symbol<CloseFn> = unsafe {
|
||||
self.library
|
||||
.get(b"marqdo_resource_close\0")
|
||||
.map_err(|error| format!("missing close: {error}"))?
|
||||
};
|
||||
if unsafe { symbol(token) } != 0 {
|
||||
return Err(self.last_error());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
fn read_frame() -> Result<Option<Json>, String> {
|
||||
let mut header = [0u8; 4];
|
||||
let mut read = 0usize;
|
||||
while read < header.len() {
|
||||
match std::io::stdin().read(&mut header[read..]) {
|
||||
Ok(0) if read == 0 => return Ok(None),
|
||||
Ok(0) => return Err("stdin ended inside a frame header".to_string()),
|
||||
Ok(count) => read += count,
|
||||
Err(error) if error.kind() == std::io::ErrorKind::Interrupted => continue,
|
||||
Err(error) => return Err(format!("stdin read failed: {error}")),
|
||||
}
|
||||
}
|
||||
let length = u32::from_be_bytes(header) as usize;
|
||||
if length == 0 || length > MAX_FRAME_BYTES {
|
||||
return Err(format!("invalid frame length {length}"));
|
||||
}
|
||||
let mut payload = vec![0u8; length];
|
||||
std::io::stdin()
|
||||
.read_exact(&mut payload)
|
||||
.map_err(|error| format!("stdin read failed: {error}"))?;
|
||||
serde_json::from_slice(&payload)
|
||||
.map(Some)
|
||||
.map_err(|error| format!("invalid JSON frame: {error}"))
|
||||
}
|
||||
|
||||
fn write_frame(value: &Json) -> Result<(), String> {
|
||||
let bytes = serde_json::to_vec(value).map_err(|error| format!("encode failed: {error}"))?;
|
||||
if bytes.len() > MAX_FRAME_BYTES {
|
||||
return Err("response frame is too large".to_string());
|
||||
}
|
||||
let mut stdout = std::io::stdout();
|
||||
stdout
|
||||
.write_all(&(bytes.len() as u32).to_be_bytes())
|
||||
.and_then(|()| stdout.write_all(&bytes))
|
||||
.and_then(|()| stdout.flush())
|
||||
.map_err(|error| format!("stdout write failed: {error}"))
|
||||
}
|
||||
|
||||
fn fail(message: &str) -> Json {
|
||||
json!({"ok": false, "code": "host.error", "message": message})
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let Some(path) = std::env::args_os().nth(1) else {
|
||||
eprintln!("usage: marqdo-provider-worker <c-abi-library>");
|
||||
std::process::exit(2);
|
||||
};
|
||||
let provider = match Provider::load(&PathBuf::from(path)) {
|
||||
Ok(provider) => provider,
|
||||
Err(error) => {
|
||||
eprintln!("provider worker: {error}");
|
||||
std::process::exit(2);
|
||||
}
|
||||
};
|
||||
loop {
|
||||
let request = match read_frame() {
|
||||
Ok(Some(request)) => request,
|
||||
Ok(None) => break,
|
||||
Err(error) => {
|
||||
eprintln!("provider worker: {error}");
|
||||
std::process::exit(1);
|
||||
}
|
||||
};
|
||||
let op = request.get("op").and_then(Json::as_str).unwrap_or("");
|
||||
let response = match op {
|
||||
"ping" => json!({"ok": true}),
|
||||
"open" => {
|
||||
let kind = request.get("kind").and_then(Json::as_str).unwrap_or("");
|
||||
let descriptor = request
|
||||
.get("descriptor")
|
||||
.and_then(Json::as_str)
|
||||
.unwrap_or("");
|
||||
let rights = request
|
||||
.get("rights")
|
||||
.and_then(Json::as_array)
|
||||
.map(|values| {
|
||||
values
|
||||
.iter()
|
||||
.filter_map(Json::as_str)
|
||||
.map(str::to_string)
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
match provider.open(kind, descriptor, &rights) {
|
||||
Ok(token) => json!({"ok": true, "token": token}),
|
||||
Err(error) => fail(&error),
|
||||
}
|
||||
}
|
||||
"read" => {
|
||||
let token = request.get("token").and_then(Json::as_u64).unwrap_or(0);
|
||||
let max_bytes =
|
||||
request.get("max_bytes").and_then(Json::as_u64).unwrap_or(0) as usize;
|
||||
match provider.read(token, max_bytes) {
|
||||
Ok((bytes, eof)) => json!({"ok": true, "bytes": bytes, "eof": eof}),
|
||||
Err(error) => fail(&error),
|
||||
}
|
||||
}
|
||||
"write" => {
|
||||
let token = request.get("token").and_then(Json::as_u64).unwrap_or(0);
|
||||
let bytes = request
|
||||
.get("bytes")
|
||||
.and_then(Json::as_array)
|
||||
.map(|values| {
|
||||
values
|
||||
.iter()
|
||||
.filter_map(Json::as_u64)
|
||||
.filter_map(|value| u8::try_from(value).ok())
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
match provider.write(token, &bytes) {
|
||||
Ok(count) => json!({"ok": true, "count": count}),
|
||||
Err(error) => fail(&error),
|
||||
}
|
||||
}
|
||||
"close" => {
|
||||
let token = request.get("token").and_then(Json::as_u64).unwrap_or(0);
|
||||
match provider.close(token) {
|
||||
Ok(()) => json!({"ok": true}),
|
||||
Err(error) => fail(&error),
|
||||
}
|
||||
}
|
||||
"shutdown" => break,
|
||||
_ => fail("unsupported worker operation"),
|
||||
};
|
||||
if let Err(error) = write_frame(&response) {
|
||||
eprintln!("provider worker: {error}");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
#![cfg(unix)]
|
||||
|
||||
use std::path::Path;
|
||||
use std::process::Command;
|
||||
|
||||
use marqdo_host::{Capabilities, Host, StdHost};
|
||||
use marqdo_sandbox::SandboxMode;
|
||||
use marqdo_value::{ErrorCategory, Value};
|
||||
|
||||
fn context() -> marqdo_host::ExecutionContext {
|
||||
marqdo_host::ExecutionContext::new("test".to_string(), None)
|
||||
}
|
||||
|
||||
fn handle(value: Value) -> marqdo_value::ResourceHandle {
|
||||
match value {
|
||||
Value::Resource(handle) => handle,
|
||||
other => panic!("expected resource, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn cc_available() -> bool {
|
||||
Command::new("cc")
|
||||
.arg("--version")
|
||||
.output()
|
||||
.map(|output| output.status.success())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
fn build_provider(dir: &Path) -> std::path::PathBuf {
|
||||
let source = dir.join("provider.c");
|
||||
std::fs::write(
|
||||
&source,
|
||||
r#"
|
||||
#include <stdint.h>
|
||||
#include <stddef.h>
|
||||
#include <string.h>
|
||||
static const char DATA[] = "worker-abi";
|
||||
static size_t position = 0;
|
||||
uint32_t marqdo_resource_provider_abi_version(void) { return 1u; }
|
||||
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
|
||||
const char *rights_csv, uint64_t *out_token) {
|
||||
(void)kind; (void)descriptor; (void)rights_csv;
|
||||
position = 0; *out_token = 42; return 0;
|
||||
}
|
||||
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
|
||||
uint32_t capacity, uint32_t *out_read,
|
||||
int32_t *out_eof) {
|
||||
(void)token;
|
||||
size_t remaining = sizeof(DATA) - 1 - position;
|
||||
size_t count = remaining < capacity ? remaining : capacity;
|
||||
memcpy(buffer, DATA + position, count);
|
||||
position += count;
|
||||
*out_read = (uint32_t)count;
|
||||
*out_eof = position >= sizeof(DATA) - 1 ? 1 : 0;
|
||||
return 0;
|
||||
}
|
||||
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
|
||||
uint32_t length, uint32_t *out_written) {
|
||||
(void)token; (void)bytes; *out_written = length; return 0;
|
||||
}
|
||||
int32_t marqdo_resource_close(uint64_t token) { (void)token; return 0; }
|
||||
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity) {
|
||||
(void)capacity; buffer[0] = 0; return 0;
|
||||
}
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
let extension = if cfg!(target_os = "macos") {
|
||||
"dylib"
|
||||
} else {
|
||||
"so"
|
||||
};
|
||||
let library = dir.join(format!("libworker_provider.{extension}"));
|
||||
let status = Command::new("cc")
|
||||
.args(["-shared", "-fPIC", "-O2", "-o"])
|
||||
.arg(&library)
|
||||
.arg(&source)
|
||||
.status()
|
||||
.expect("run cc");
|
||||
assert!(status.success());
|
||||
library
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn c_abi_provider_runs_inside_the_worker_helper() {
|
||||
if !cc_available() {
|
||||
eprintln!("skipping worker binding test: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let library = build_provider(dir.path());
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("worker-cabi"),
|
||||
project.path(),
|
||||
);
|
||||
host.register_worker_resource_provider(
|
||||
"worker-cabi",
|
||||
&["worker.sensor"],
|
||||
&std::path::PathBuf::from(env!("CARGO_BIN_EXE_marqdo-provider-worker")),
|
||||
&[library.to_string_lossy().to_string()],
|
||||
)
|
||||
.unwrap();
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("worker.sensor", "s", &["read".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let first = host.resource_read(handle, 6, &ctx).unwrap();
|
||||
assert_eq!(first.bytes, b"worker");
|
||||
let rest = host.resource_read(handle, 64, &ctx).unwrap();
|
||||
assert_eq!(rest.bytes, b"-abi");
|
||||
assert!(rest.eof);
|
||||
assert_eq!(host.resource_write(handle, b"x", &ctx).unwrap(), 1);
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn worker_helper_without_c_provider_fails_closed() {
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("worker-cabi"),
|
||||
project.path(),
|
||||
);
|
||||
let error = host
|
||||
.register_worker_resource_provider(
|
||||
"worker-cabi",
|
||||
&["worker.sensor"],
|
||||
&std::path::PathBuf::from(env!("CARGO_BIN_EXE_marqdo-provider-worker")),
|
||||
&["/definitely/missing/provider.so".to_string()],
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::HostError);
|
||||
}
|
||||
@@ -6,13 +6,13 @@
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use marqdo_host::HostOp;
|
||||
use marqdo_host::{HostOp, ResourceRead};
|
||||
use marqdo_ir::builtins::Builtin;
|
||||
use marqdo_ir::Resource;
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{
|
||||
float_to_bigint, CodeValue, ErrorCategory, FormulaValue, FunctionRef, RuntimeError, Table,
|
||||
Value,
|
||||
float_to_bigint, CodeValue, ErrorCategory, FormulaValue, FunctionRef, ResourceHandle,
|
||||
RuntimeError, Table, Value,
|
||||
};
|
||||
|
||||
use crate::interpreter::Interpreter;
|
||||
@@ -27,10 +27,15 @@ pub(crate) fn dispatch(
|
||||
let module = frame.module().clone();
|
||||
match builtin.name {
|
||||
"打印" => {
|
||||
let value = take(&bound, "值", builtin, span)?;
|
||||
interpreter
|
||||
.host_invoke(HostOp::Print, vec![value], span, &module)
|
||||
.map(|_| Value::Unit)
|
||||
// Compat-C (ADR-0031): the legacy HostOp::Print path is gone. On
|
||||
// the official CLI the loader routes 打印/print to stdlib io.写值;
|
||||
// reaching this builtin means the verified bundle was unavailable.
|
||||
let _ = take(&bound, "值", builtin, span)?;
|
||||
Err(RuntimeError::new(
|
||||
ErrorCategory::KernelUnavailable,
|
||||
span,
|
||||
"打印 requires the bundled stdlib print module; install the official CLI bundle",
|
||||
))
|
||||
}
|
||||
"长度" => {
|
||||
let value = take(&bound, "值", builtin, span)?;
|
||||
@@ -187,6 +192,292 @@ pub(crate) fn dispatch(
|
||||
&module,
|
||||
)
|
||||
}
|
||||
"memory.new" => {
|
||||
let initial_pages = page_count(take(&bound, "initial_pages", builtin, span)?, span)?;
|
||||
let max_pages = page_count(take(&bound, "max_pages", builtin, span)?, span)?;
|
||||
let handle = interpreter
|
||||
.memory_mut()
|
||||
.create(initial_pages, max_pages, span)?;
|
||||
interpreter.record_memory_event(
|
||||
"memory.new",
|
||||
&module,
|
||||
span,
|
||||
&format!("initial_pages={initial_pages} max_pages={max_pages}"),
|
||||
);
|
||||
Ok(Value::Memory(handle))
|
||||
}
|
||||
"memory.size" => {
|
||||
let memory = take(&bound, "memory", builtin, span)?;
|
||||
let memory_type = memory.type_name();
|
||||
let Value::Memory(handle) = memory else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("memory.size expects a Memory value, got {memory_type}"),
|
||||
));
|
||||
};
|
||||
let pages = interpreter.memory().size(handle, span)?;
|
||||
Ok(pages_value(pages, span)?)
|
||||
}
|
||||
"memory.grow" => {
|
||||
let memory = take(&bound, "memory", builtin, span)?;
|
||||
let memory_type = memory.type_name();
|
||||
let Value::Memory(handle) = memory else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("memory.grow expects a Memory value, got {memory_type}"),
|
||||
));
|
||||
};
|
||||
let pages = page_count(take(&bound, "pages", builtin, span)?, span)?;
|
||||
let old_pages = interpreter.memory_mut().grow(handle, pages, span)?;
|
||||
interpreter.record_memory_event(
|
||||
"memory.grow",
|
||||
&module,
|
||||
span,
|
||||
&format!("pages={pages} old_pages={old_pages}"),
|
||||
);
|
||||
Ok(pages_value(old_pages, span)?)
|
||||
}
|
||||
"list.append" => {
|
||||
let list = take(&bound, "list", builtin, span)?;
|
||||
let value = take(&bound, "value", builtin, span)?;
|
||||
let list_type = list.type_name();
|
||||
let Value::List(mut items) = list else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("list.append expects a List, got {list_type}"),
|
||||
));
|
||||
};
|
||||
items.push(value);
|
||||
Ok(Value::List(items))
|
||||
}
|
||||
"list.concat" => {
|
||||
let left = take(&bound, "left", builtin, span)?;
|
||||
let right = take(&bound, "right", builtin, span)?;
|
||||
let left_type = left.type_name();
|
||||
let Value::List(mut items) = left else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("list.concat left operand must be a List, got {left_type}"),
|
||||
));
|
||||
};
|
||||
let right_type = right.type_name();
|
||||
let Value::List(other) = right else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("list.concat right operand must be a List, got {right_type}"),
|
||||
));
|
||||
};
|
||||
items.extend(other);
|
||||
Ok(Value::List(items))
|
||||
}
|
||||
"map.keys" => {
|
||||
let value = take(&bound, "map", builtin, span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Map(map) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("map.keys expects a Map, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(Value::List(
|
||||
map.keys().cloned().map(Value::Text).collect::<Vec<_>>(),
|
||||
))
|
||||
}
|
||||
"map.has" => {
|
||||
let value = take(&bound, "map", builtin, span)?;
|
||||
let key = map_key(take(&bound, "key", builtin, span)?, "map.has", span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Map(map) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("map.has expects a Map, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(Value::Bool(map.contains_key(&key)))
|
||||
}
|
||||
"map.get" => {
|
||||
let value = take(&bound, "map", builtin, span)?;
|
||||
let key = map_key(take(&bound, "key", builtin, span)?, "map.get", span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Map(map) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("map.get expects a Map, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(map.get(&key).cloned().unwrap_or(Value::None))
|
||||
}
|
||||
"map.set" => {
|
||||
let value = take(&bound, "map", builtin, span)?;
|
||||
let key = map_key(take(&bound, "key", builtin, span)?, "map.set", span)?;
|
||||
let new_value = take(&bound, "value", builtin, span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Map(mut map) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("map.set expects a Map, got {value_type}"),
|
||||
));
|
||||
};
|
||||
map.insert(key, new_value);
|
||||
Ok(Value::Map(map))
|
||||
}
|
||||
"map.remove" => {
|
||||
let value = take(&bound, "map", builtin, span)?;
|
||||
let key = map_key(take(&bound, "key", builtin, span)?, "map.remove", span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Map(mut map) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("map.remove expects a Map, got {value_type}"),
|
||||
));
|
||||
};
|
||||
map.remove(&key);
|
||||
Ok(Value::Map(map))
|
||||
}
|
||||
"text.codepoints" => {
|
||||
let value = take(&bound, "text", builtin, span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Text(text) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("text.codepoints expects Text, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(Value::List(
|
||||
text.chars()
|
||||
.map(|character| Value::int(i64::from(character as u32)))
|
||||
.collect(),
|
||||
))
|
||||
}
|
||||
"text.from_codepoint" => {
|
||||
let value = take(&bound, "codepoint", builtin, span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Int(codepoint) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("text.from_codepoint expects an Int, got {value_type}"),
|
||||
));
|
||||
};
|
||||
let Some(codepoint) = num_traits::ToPrimitive::to_u32(&codepoint) else {
|
||||
return Ok(Value::None);
|
||||
};
|
||||
Ok(match char::from_u32(codepoint) {
|
||||
Some(character) => Value::Text(character.to_string()),
|
||||
None => Value::None,
|
||||
})
|
||||
}
|
||||
"table.columns" => {
|
||||
let value = take(&bound, "table", builtin, span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Table(table) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("table.columns expects a Table, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(Value::List(
|
||||
table.header.into_iter().map(Value::Text).collect(),
|
||||
))
|
||||
}
|
||||
"table.row" => {
|
||||
let value = take(&bound, "table", builtin, span)?;
|
||||
let row = take(&bound, "row", builtin, span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Table(table) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("table.row expects a Table, got {value_type}"),
|
||||
));
|
||||
};
|
||||
let position = table_index(row, table.rows.len(), "row", span)?;
|
||||
Ok(Value::List(table.rows[position].clone()))
|
||||
}
|
||||
"table.cell" => {
|
||||
let value = take(&bound, "table", builtin, span)?;
|
||||
let row = take(&bound, "row", builtin, span)?;
|
||||
let column = take(&bound, "column", builtin, span)?;
|
||||
let value_type = value.type_name();
|
||||
let Value::Table(table) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("table.cell expects a Table, got {value_type}"),
|
||||
));
|
||||
};
|
||||
let row = table_index(row, table.rows.len(), "row", span)?;
|
||||
let column = table_index(column, table.header.len(), "column", span)?;
|
||||
Ok(table.rows[row][column].clone())
|
||||
}
|
||||
"resource.open" => {
|
||||
let kind = expect_text_value(take(&bound, "kind", builtin, span)?, span)?;
|
||||
let descriptor = expect_text_value(take(&bound, "descriptor", builtin, span)?, span)?;
|
||||
let rights = rights_list(take(&bound, "rights", builtin, span)?, span)?;
|
||||
interpreter.resource_open(&kind, &descriptor, &rights, &module, span)
|
||||
}
|
||||
"resource.read" => {
|
||||
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
|
||||
let memory = memory_handle(
|
||||
take(&bound, "memory", builtin, span)?,
|
||||
"resource.read",
|
||||
span,
|
||||
)?;
|
||||
let offset = take(&bound, "offset", builtin, span)?;
|
||||
let offset_int = int_value(offset, "resource.read offset", span)?;
|
||||
let offset_value = Value::Int(offset_int.clone());
|
||||
let max_bytes = positive_count(
|
||||
take(&bound, "max_bytes", builtin, span)?,
|
||||
"resource.read max_bytes",
|
||||
span,
|
||||
)?;
|
||||
let available = interpreter.memory().byte_len(memory, span)?;
|
||||
let start = non_negative_usize(&offset_value, "resource.read offset", span)?;
|
||||
if start >= available {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
format!("resource.read offset {start} is at or beyond the Memory length {available}"),
|
||||
));
|
||||
}
|
||||
let limit = max_bytes.min(available - start);
|
||||
let result = interpreter.resource_read(handle, limit, &module, span);
|
||||
match result {
|
||||
Ok(read) => {
|
||||
interpreter
|
||||
.memory_mut()
|
||||
.write_bytes(memory, &offset_int, &read.bytes, span)?;
|
||||
Ok(resource_ok(resource_read_payload(&read)))
|
||||
}
|
||||
Err(error) => resource_recoverable(error),
|
||||
}
|
||||
}
|
||||
"resource.write" => {
|
||||
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
|
||||
let memory = memory_handle(
|
||||
take(&bound, "memory", builtin, span)?,
|
||||
"resource.write",
|
||||
span,
|
||||
)?;
|
||||
let offset = take(&bound, "offset", builtin, span)?;
|
||||
let offset_int = int_value(offset, "resource.write offset", span)?;
|
||||
let length = non_negative_usize(
|
||||
&take(&bound, "length", builtin, span)?,
|
||||
"resource.write length",
|
||||
span,
|
||||
)?;
|
||||
let bytes = interpreter
|
||||
.memory()
|
||||
.read_bytes(memory, &offset_int, length, span)?;
|
||||
match interpreter.resource_write(handle, &bytes, &module, span) {
|
||||
Ok(count) => Ok(resource_ok(resource_write_payload(count))),
|
||||
Err(error) => resource_recoverable(error),
|
||||
}
|
||||
}
|
||||
"resource.close" => {
|
||||
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
|
||||
match interpreter.resource_close(handle, &module, span) {
|
||||
Ok(()) => Ok(resource_ok(Value::None)),
|
||||
Err(error) => resource_recoverable(error),
|
||||
}
|
||||
}
|
||||
other => Err(RuntimeError::new(
|
||||
ErrorCategory::KernelUnavailable,
|
||||
span,
|
||||
@@ -210,6 +501,228 @@ fn take(
|
||||
})
|
||||
}
|
||||
|
||||
/// Validate a page-count argument: an `Int` in the non-negative `u64` range.
|
||||
fn page_count(value: Value, span: Option<Span>) -> Result<u64, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
let Value::Int(count) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("memory page count must be an Int, got {value_type}"),
|
||||
));
|
||||
};
|
||||
num_traits::ToPrimitive::to_u64(&count).ok_or_else(|| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::MemoryFault,
|
||||
span,
|
||||
format!("memory page count {count} must be a non-negative integer"),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/// Convert a page count back to a language `Int`.
|
||||
fn pages_value(pages: u64, span: Option<Span>) -> Result<Value, RuntimeError> {
|
||||
i64::try_from(pages).map(Value::int).map_err(|_| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::MemoryFault,
|
||||
span,
|
||||
format!("page count {pages} does not fit an Int"),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/// Require a `Text` map key and return it.
|
||||
fn map_key(value: Value, builtin: &str, span: Option<Span>) -> Result<String, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
match value {
|
||||
Value::Text(key) => Ok(key),
|
||||
_ => Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("{builtin} key must be Text, got {value_type}"),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Require a Text value.
|
||||
fn expect_text_value(value: Value, span: Option<Span>) -> Result<String, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
let Value::Text(text) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("expected Text, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(text)
|
||||
}
|
||||
|
||||
/// Require a List of Text rights.
|
||||
fn rights_list(value: Value, span: Option<Span>) -> Result<Vec<String>, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
let Value::List(items) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("resource rights must be a List, got {value_type}"),
|
||||
));
|
||||
};
|
||||
let mut rights = Vec::with_capacity(items.len());
|
||||
for item in items {
|
||||
let item_type = item.type_name();
|
||||
let Value::Text(right) = item else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("resource rights must contain Text, got {item_type}"),
|
||||
));
|
||||
};
|
||||
rights.push(right);
|
||||
}
|
||||
Ok(rights)
|
||||
}
|
||||
|
||||
/// Require an opaque resource handle.
|
||||
fn resource_handle(value: Value, span: Option<Span>) -> Result<ResourceHandle, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
let Value::Resource(handle) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("expected a Resource handle, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(handle)
|
||||
}
|
||||
|
||||
/// Require an opaque Memory handle.
|
||||
fn memory_handle(
|
||||
value: Value,
|
||||
builtin: &str,
|
||||
span: Option<Span>,
|
||||
) -> Result<marqdo_value::MemoryHandle, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
let Value::Memory(handle) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("{builtin} memory must be a Memory value, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(handle)
|
||||
}
|
||||
|
||||
/// Require an Int value.
|
||||
fn int_value(
|
||||
value: Value,
|
||||
what: &str,
|
||||
span: Option<Span>,
|
||||
) -> Result<num_bigint::BigInt, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
let Value::Int(value) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("{what} must be an Int, got {value_type}"),
|
||||
));
|
||||
};
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
/// Require a non-negative usize.
|
||||
fn non_negative_usize(
|
||||
value: &Value,
|
||||
what: &str,
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
let Value::Int(value) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("{what} must be an Int, got {value_type}"),
|
||||
));
|
||||
};
|
||||
num_traits::ToPrimitive::to_usize(value).ok_or_else(|| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
format!("{what} {value} must be a non-negative integer in range"),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/// Require a strictly positive byte count.
|
||||
fn positive_count(value: Value, what: &str, span: Option<Span>) -> Result<usize, RuntimeError> {
|
||||
let count = non_negative_usize(&value, what, span)?;
|
||||
if count == 0 {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("{what} must be greater than zero"),
|
||||
));
|
||||
}
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
fn resource_ok(value: Value) -> Value {
|
||||
let mut map = BTreeMap::new();
|
||||
map.insert("ok".to_string(), Value::Bool(true));
|
||||
map.insert("value".to_string(), value);
|
||||
Value::Map(map)
|
||||
}
|
||||
|
||||
fn resource_err(code: &str, message: &str) -> Value {
|
||||
let mut map = BTreeMap::new();
|
||||
map.insert("ok".to_string(), Value::Bool(false));
|
||||
map.insert("code".to_string(), Value::Text(code.to_string()));
|
||||
map.insert("message".to_string(), Value::Text(message.to_string()));
|
||||
Value::Map(map)
|
||||
}
|
||||
|
||||
fn resource_read_payload(read: &ResourceRead) -> Value {
|
||||
let mut map = BTreeMap::new();
|
||||
map.insert("count".to_string(), Value::int(read.bytes.len() as i64));
|
||||
map.insert("eof".to_string(), Value::Bool(read.eof));
|
||||
Value::Map(map)
|
||||
}
|
||||
|
||||
fn resource_write_payload(count: usize) -> Value {
|
||||
let mut map = BTreeMap::new();
|
||||
map.insert("count".to_string(), Value::int(count as i64));
|
||||
Value::Map(map)
|
||||
}
|
||||
|
||||
/// Convert a closed-handle failure into a recoverable Result; other errors are
|
||||
/// fatal (ADR-0025).
|
||||
fn resource_recoverable(error: RuntimeError) -> Result<Value, RuntimeError> {
|
||||
if error.category == ErrorCategory::ResourceClosed {
|
||||
return Ok(resource_err("resource.closed", &error.message));
|
||||
}
|
||||
Err(error)
|
||||
}
|
||||
|
||||
/// Validate a non-negative table row/column coordinate.
|
||||
fn table_index(
|
||||
value: Value,
|
||||
len: usize,
|
||||
what: &str,
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
let value_type = value.type_name();
|
||||
let Value::Int(index) = value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("table {what} must be an Int, got {value_type}"),
|
||||
));
|
||||
};
|
||||
let Some(position) = num_traits::ToPrimitive::to_usize(&index) else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
format!("table {what} {index} is out of bounds for length {len}"),
|
||||
));
|
||||
};
|
||||
if position >= len {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
format!("table {what} {position} is out of bounds for length {len}"),
|
||||
));
|
||||
}
|
||||
Ok(position)
|
||||
}
|
||||
|
||||
fn formula_text(value: Value, span: Option<Span>) -> Result<String, RuntimeError> {
|
||||
match value {
|
||||
Value::Formula(formula) => Ok(formula.latex),
|
||||
|
||||
@@ -3,12 +3,13 @@
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use marqdo_expression::{BinaryOp, Expr, RecordEntry};
|
||||
use marqdo_host::{ExecutionContext, Host, HostOp};
|
||||
use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, Program, Stmt};
|
||||
use marqdo_host::{ExecutionContext, Host, HostOp, ResourceRead};
|
||||
use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, PlaceTarget, Program, Stmt};
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{ErrorCategory, RuntimeError, Value};
|
||||
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value};
|
||||
|
||||
use crate::builtins_impl::FrameView;
|
||||
use crate::memory::{MemoryLimits, MemoryStore};
|
||||
|
||||
use crate::value_ops;
|
||||
|
||||
@@ -68,6 +69,7 @@ pub struct Interpreter<'a> {
|
||||
program: &'a Program,
|
||||
host: &'a mut dyn Host,
|
||||
depth: usize,
|
||||
memory: MemoryStore,
|
||||
}
|
||||
|
||||
/// Run a named function with already-evaluated arguments.
|
||||
@@ -77,11 +79,31 @@ pub fn run(
|
||||
module: &ModuleId,
|
||||
function: &str,
|
||||
args: Vec<EvaluatedArg>,
|
||||
) -> Result<RunOutcome, RuntimeError> {
|
||||
run_with_limits(
|
||||
program,
|
||||
host,
|
||||
module,
|
||||
function,
|
||||
args,
|
||||
MemoryLimits::default(),
|
||||
)
|
||||
}
|
||||
|
||||
/// Run with explicit addressable-memory limits (ADR-0022).
|
||||
pub fn run_with_limits(
|
||||
program: &Program,
|
||||
host: &mut dyn Host,
|
||||
module: &ModuleId,
|
||||
function: &str,
|
||||
args: Vec<EvaluatedArg>,
|
||||
limits: MemoryLimits,
|
||||
) -> Result<RunOutcome, RuntimeError> {
|
||||
let mut interpreter = Interpreter {
|
||||
program,
|
||||
host,
|
||||
depth: 0,
|
||||
memory: MemoryStore::new(limits),
|
||||
};
|
||||
let value = interpreter.call_by_name(module, function, args, None)?;
|
||||
Ok(RunOutcome { value })
|
||||
@@ -300,10 +322,25 @@ impl<'a> Interpreter<'a> {
|
||||
self.host.consume_fuel(1, Some(stmt.span()))?;
|
||||
self.host.check_deadline(Some(stmt.span()))?;
|
||||
match stmt {
|
||||
Stmt::Assign { name, value, .. } => {
|
||||
let value = self.eval_expr(value, frame)?;
|
||||
frame.locals.insert(name.clone(), value);
|
||||
}
|
||||
Stmt::Assign { target, value, .. } => match target {
|
||||
PlaceTarget::Local { name, .. } => {
|
||||
let value = self.eval_expr(value, frame)?;
|
||||
frame.locals.insert(name.clone(), value);
|
||||
}
|
||||
PlaceTarget::Index { base, index, span } => {
|
||||
// Frozen order (ADR-0022): base, index, then RHS.
|
||||
let base_value = self.eval_expr(base, frame)?;
|
||||
let index_value = self.eval_expr(index, frame)?;
|
||||
let byte_value = self.eval_expr(value, frame)?;
|
||||
self.assign_memory_index(
|
||||
base_value,
|
||||
index_value,
|
||||
byte_value,
|
||||
Some(*span),
|
||||
frame,
|
||||
)?;
|
||||
}
|
||||
},
|
||||
Stmt::Eval { expr, .. } => {
|
||||
self.eval_expr(expr, frame)?;
|
||||
}
|
||||
@@ -417,6 +454,62 @@ impl<'a> Interpreter<'a> {
|
||||
self.call_by_name_from(&module, &name, args, Some(span), Some(&caller))
|
||||
}
|
||||
|
||||
/// Commit one byte into an addressable region after all operand checks.
|
||||
///
|
||||
/// The three operands are already evaluated, so a rejected value never
|
||||
/// produces a partial write (ADR-0022 §3).
|
||||
fn assign_memory_index(
|
||||
&mut self,
|
||||
base: Value,
|
||||
index: Value,
|
||||
byte: Value,
|
||||
span: Option<Span>,
|
||||
frame: &Frame,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let base_type = base.type_name();
|
||||
let Value::Memory(handle) = base else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::Type,
|
||||
span,
|
||||
format!(
|
||||
"index assignment requires a Memory base in v1, got {base_type}; List, Map \
|
||||
and Table values keep their existing value semantics"
|
||||
),
|
||||
));
|
||||
};
|
||||
let index_type = index.type_name();
|
||||
let Value::Int(offset) = index else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("Memory index must be an Int, got {index_type}"),
|
||||
));
|
||||
};
|
||||
let byte_type = byte.type_name();
|
||||
let byte_repr = byte.display();
|
||||
let Value::Int(byte) = byte else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("Memory byte value must be an Int in 0..255, got {byte_type}"),
|
||||
));
|
||||
};
|
||||
let Some(byte) = num_traits::ToPrimitive::to_u8(&byte) else {
|
||||
return Err(RuntimeError::type_error(
|
||||
span,
|
||||
format!("Memory byte value {byte_repr} is outside 0..255"),
|
||||
));
|
||||
};
|
||||
self.memory.write(handle, &offset, byte, span)?;
|
||||
let module = frame.function.module.canonical();
|
||||
self.host.record_semantic(
|
||||
"memory.write",
|
||||
Some(&frame.function.to_string()),
|
||||
Some(&module),
|
||||
span,
|
||||
Some("byte"),
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn eval_args(
|
||||
&mut self,
|
||||
args: &[marqdo_expression::Arg],
|
||||
@@ -528,7 +621,22 @@ impl<'a> Interpreter<'a> {
|
||||
Expr::Index { base, index, span } => {
|
||||
let base_value = self.eval_expr(base, frame)?;
|
||||
let index_value = self.eval_expr(index, frame)?;
|
||||
index_value_of(base_value, index_value, Some(*span))
|
||||
match base_value {
|
||||
Value::Memory(handle) => {
|
||||
let Value::Int(offset) = index_value else {
|
||||
return Err(RuntimeError::type_error(
|
||||
Some(*span),
|
||||
format!(
|
||||
"Memory index must be an Int, got {}",
|
||||
index_value.type_name()
|
||||
),
|
||||
));
|
||||
};
|
||||
let byte = self.memory.read(handle, &offset, Some(*span))?;
|
||||
Ok(Value::int(i64::from(byte)))
|
||||
}
|
||||
other => index_value_of(other, index_value, Some(*span)),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -548,6 +656,25 @@ impl<'a> Interpreter<'a> {
|
||||
let args = self.eval_args(args, frame)?;
|
||||
return self.call_value(program, value, args, Some(span), Some(&caller_module));
|
||||
}
|
||||
if matches!(ident.name.as_str(), "打印" | "print") {
|
||||
if let Some(target) = program.print_override.clone() {
|
||||
let args = self.eval_args(args, frame)?;
|
||||
let Some(function) = program.function(&target) else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::UnknownSymbol,
|
||||
Some(span),
|
||||
format!("print override `{target}` does not exist"),
|
||||
));
|
||||
};
|
||||
return self.call_function(
|
||||
program,
|
||||
function,
|
||||
args,
|
||||
Some(span),
|
||||
Some(&caller_module),
|
||||
);
|
||||
}
|
||||
}
|
||||
if let Some(builtin) = builtins::find(&ident.name) {
|
||||
let args = self.eval_args(args, frame)?;
|
||||
return self.call_builtin(builtin, args, Some(span), frame);
|
||||
@@ -701,6 +828,97 @@ impl<'a> Interpreter<'a> {
|
||||
self.program
|
||||
}
|
||||
|
||||
/// Addressable-memory store for builtins and place writes.
|
||||
pub(crate) fn memory(&self) -> &MemoryStore {
|
||||
&self.memory
|
||||
}
|
||||
|
||||
/// Mutable addressable-memory store for `memory.new`/`memory.grow`.
|
||||
pub(crate) fn memory_mut(&mut self) -> &mut MemoryStore {
|
||||
&mut self.memory
|
||||
}
|
||||
|
||||
/// Record a semantic event for a builtin that is not a host operation.
|
||||
pub(crate) fn record_memory_event(
|
||||
&mut self,
|
||||
kind: &str,
|
||||
module: &ModuleId,
|
||||
span: Option<Span>,
|
||||
detail: &str,
|
||||
) {
|
||||
let module = module.canonical();
|
||||
self.host
|
||||
.record_semantic(kind, None, Some(&module), span, Some(detail));
|
||||
}
|
||||
|
||||
/// Open a resource/stream through the host boundary.
|
||||
pub(crate) fn resource_open(
|
||||
&mut self,
|
||||
kind: &str,
|
||||
descriptor: &str,
|
||||
rights: &[String],
|
||||
module: &ModuleId,
|
||||
span: Option<Span>,
|
||||
) -> Result<Value, RuntimeError> {
|
||||
let context = ExecutionContext::new(module.to_string(), span);
|
||||
self.host
|
||||
.resource_open(kind, descriptor, rights, &context)
|
||||
.map_err(|mut error| {
|
||||
error.mark_callsite();
|
||||
error
|
||||
})
|
||||
}
|
||||
|
||||
/// Read from a resource stream.
|
||||
pub(crate) fn resource_read(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
max_bytes: usize,
|
||||
module: &ModuleId,
|
||||
span: Option<Span>,
|
||||
) -> Result<ResourceRead, RuntimeError> {
|
||||
let context = ExecutionContext::new(module.to_string(), span);
|
||||
self.host
|
||||
.resource_read(handle, max_bytes, &context)
|
||||
.map_err(|mut error| {
|
||||
error.mark_callsite();
|
||||
error
|
||||
})
|
||||
}
|
||||
|
||||
/// Write bytes to a resource stream.
|
||||
pub(crate) fn resource_write(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
bytes: &[u8],
|
||||
module: &ModuleId,
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
let context = ExecutionContext::new(module.to_string(), span);
|
||||
self.host
|
||||
.resource_write(handle, bytes, &context)
|
||||
.map_err(|mut error| {
|
||||
error.mark_callsite();
|
||||
error
|
||||
})
|
||||
}
|
||||
|
||||
/// Close a resource stream.
|
||||
pub(crate) fn resource_close(
|
||||
&mut self,
|
||||
handle: ResourceHandle,
|
||||
module: &ModuleId,
|
||||
span: Option<Span>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let context = ExecutionContext::new(module.to_string(), span);
|
||||
self.host
|
||||
.resource_close(handle, &context)
|
||||
.map_err(|mut error| {
|
||||
error.mark_callsite();
|
||||
error
|
||||
})
|
||||
}
|
||||
|
||||
/// Call a builtin-read host operation on behalf of a frame's module.
|
||||
pub(crate) fn host_invoke(
|
||||
&mut self,
|
||||
@@ -820,6 +1038,24 @@ fn member_of(value: Value, name: &str, span: Option<Span>) -> Result<Value, Runt
|
||||
format!("Code has no field `{name}`"),
|
||||
)),
|
||||
},
|
||||
Value::FunctionRef(reference) => match name {
|
||||
"module" => Ok(Value::Text(reference.module)),
|
||||
"name" => Ok(Value::Text(reference.name)),
|
||||
_ => Err(RuntimeError::new(
|
||||
ErrorCategory::UnknownField,
|
||||
span,
|
||||
format!("FunctionRef has no field `{name}`"),
|
||||
)),
|
||||
},
|
||||
Value::PluginHandle(handle) => match name {
|
||||
"plugin" => Ok(Value::Text(handle.plugin)),
|
||||
"export" => Ok(Value::Text(handle.export)),
|
||||
_ => Err(RuntimeError::new(
|
||||
ErrorCategory::UnknownField,
|
||||
span,
|
||||
format!("PluginHandle has no field `{name}`"),
|
||||
)),
|
||||
},
|
||||
other => Err(RuntimeError::new(
|
||||
ErrorCategory::UnknownField,
|
||||
span,
|
||||
|
||||
@@ -8,8 +8,12 @@
|
||||
|
||||
mod builtins_impl;
|
||||
mod interpreter;
|
||||
mod memory;
|
||||
mod value_ops;
|
||||
|
||||
pub use interpreter::{run, EvaluatedArg, Interpreter, RunOutcome};
|
||||
pub use interpreter::{run, run_with_limits, EvaluatedArg, Interpreter, RunOutcome};
|
||||
pub use memory::{
|
||||
MemoryLimits, MemoryStore, DEFAULT_MAX_REGION_BYTES, DEFAULT_MAX_TOTAL_BYTES, PAGE_SIZE,
|
||||
};
|
||||
|
||||
pub use marqdo_value::{RuntimeError, Value};
|
||||
@@ -0,0 +1,600 @@
|
||||
//! Runtime-owned addressable byte regions (RFC-CM-0001 / ADR-0022).
|
||||
//!
|
||||
//! A [`MemoryStore`] owns every region created during one run. Language values
|
||||
//! carry only an opaque [`MemoryHandle`]; offsets are region-relative integers
|
||||
//! and are never host pointers. Region identity is issued here and cannot be
|
||||
//! forged from a document, a number or wire data.
|
||||
//!
|
||||
//! Semantics frozen by ADR-0022:
|
||||
//!
|
||||
//! * a region is a byte array with a 64 KiB logical page size; new bytes are
|
||||
//! zero-filled;
|
||||
//! * regions grow only, never shrink; a failed grow leaves the region and the
|
||||
//! committed budget unchanged;
|
||||
//! * copies of a `Memory` value share one region, while different regions
|
||||
//! never alias;
|
||||
//! * every access is bounds-checked against the region and the run budget, and
|
||||
//! overflow in offset/page arithmetic is a structured error, never a panic;
|
||||
//! * regions live for the whole run and are dropped with the store. There is
|
||||
//! no user-visible `free(region)`.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{ErrorCategory, MemoryHandle, RuntimeError};
|
||||
use num_bigint::BigInt;
|
||||
use num_traits::ToPrimitive;
|
||||
|
||||
/// Logical page size for every region.
|
||||
pub const PAGE_SIZE: usize = 64 * 1024;
|
||||
|
||||
/// Default maximum committed bytes across all regions in one run (64 MiB).
|
||||
pub const DEFAULT_MAX_TOTAL_BYTES: usize = 64 * 1024 * 1024;
|
||||
|
||||
/// Default maximum committed bytes for one region (16 MiB).
|
||||
pub const DEFAULT_MAX_REGION_BYTES: usize = 16 * 1024 * 1024;
|
||||
|
||||
static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1);
|
||||
|
||||
/// Bounds for addressable memory in one run.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct MemoryLimits {
|
||||
/// Maximum bytes committed across all live regions.
|
||||
pub max_total_bytes: usize,
|
||||
/// Maximum bytes committed by a single region (`max_pages` is further
|
||||
/// capped by this value).
|
||||
pub max_region_bytes: usize,
|
||||
}
|
||||
|
||||
impl Default for MemoryLimits {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
max_total_bytes: DEFAULT_MAX_TOTAL_BYTES,
|
||||
max_region_bytes: DEFAULT_MAX_REGION_BYTES,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl MemoryLimits {
|
||||
/// Maximum page count a single region may declare or reach.
|
||||
pub fn max_region_pages(&self) -> u64 {
|
||||
(self.max_region_bytes / PAGE_SIZE) as u64
|
||||
}
|
||||
}
|
||||
|
||||
struct Region {
|
||||
bytes: Vec<u8>,
|
||||
/// Declared maximum size for this region.
|
||||
max_bytes: usize,
|
||||
generation: u64,
|
||||
}
|
||||
|
||||
/// All addressable regions created by one run.
|
||||
pub struct MemoryStore {
|
||||
run_id: u64,
|
||||
next_region: u64,
|
||||
regions: BTreeMap<u64, Region>,
|
||||
committed_bytes: usize,
|
||||
limits: MemoryLimits,
|
||||
}
|
||||
|
||||
impl MemoryStore {
|
||||
/// Create an empty store with a fresh, unguessable run identity.
|
||||
pub fn new(limits: MemoryLimits) -> Self {
|
||||
Self {
|
||||
run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed),
|
||||
next_region: 1,
|
||||
regions: BTreeMap::new(),
|
||||
committed_bytes: 0,
|
||||
limits,
|
||||
}
|
||||
}
|
||||
|
||||
/// Limits in force for this store.
|
||||
pub fn limits(&self) -> MemoryLimits {
|
||||
self.limits
|
||||
}
|
||||
|
||||
/// Bytes currently committed across all regions.
|
||||
pub fn committed_bytes(&self) -> usize {
|
||||
self.committed_bytes
|
||||
}
|
||||
|
||||
/// Number of live regions.
|
||||
pub fn region_count(&self) -> usize {
|
||||
self.regions.len()
|
||||
}
|
||||
|
||||
/// Create a zero-filled region.
|
||||
///
|
||||
/// `initial_pages` is committed immediately; `max_pages` is the declared
|
||||
/// capacity that `grow` may not exceed. Both are checked against the
|
||||
/// region and total budgets before anything is allocated, so a rejected
|
||||
/// create leaves the store unchanged.
|
||||
pub fn create(
|
||||
&mut self,
|
||||
initial_pages: u64,
|
||||
max_pages: u64,
|
||||
span: Option<Span>,
|
||||
) -> Result<MemoryHandle, RuntimeError> {
|
||||
if initial_pages > max_pages {
|
||||
return Err(memory_error(
|
||||
span,
|
||||
format!(
|
||||
"memory.new initial_pages ({initial_pages}) must not exceed max_pages \
|
||||
({max_pages})"
|
||||
),
|
||||
));
|
||||
}
|
||||
if max_pages > self.limits.max_region_pages() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!(
|
||||
"memory.new max_pages ({max_pages}) exceeds the per-region limit of {} \
|
||||
pages ({PAGE_SIZE} bytes each)",
|
||||
self.limits.max_region_pages()
|
||||
),
|
||||
));
|
||||
}
|
||||
let max_bytes = checked_bytes(max_pages, span, "region capacity")?;
|
||||
if max_bytes > self.limits.max_region_bytes {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!(
|
||||
"memory.new max_pages ({max_pages}) exceeds the per-region byte budget of {}",
|
||||
self.limits.max_region_bytes
|
||||
),
|
||||
));
|
||||
}
|
||||
let initial_bytes = checked_bytes(initial_pages, span, "initial region size")?;
|
||||
let committed = self
|
||||
.committed_bytes
|
||||
.checked_add(initial_bytes)
|
||||
.ok_or_else(|| budget_error(span, "memory budget overflow"))?;
|
||||
if committed > self.limits.max_total_bytes {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!(
|
||||
"memory budget exceeded: committing {initial_bytes} bytes would reach \
|
||||
{committed} of {}",
|
||||
self.limits.max_total_bytes
|
||||
),
|
||||
));
|
||||
}
|
||||
|
||||
let mut bytes = Vec::new();
|
||||
bytes.try_reserve_exact(initial_bytes).map_err(|_| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!("cannot allocate {initial_bytes} bytes of addressable memory"),
|
||||
)
|
||||
})?;
|
||||
bytes.resize(initial_bytes, 0);
|
||||
|
||||
let region_id = self.next_region;
|
||||
self.next_region = self
|
||||
.next_region
|
||||
.checked_add(1)
|
||||
.ok_or_else(|| memory_error(span, "memory region identity space exhausted"))?;
|
||||
self.regions.insert(
|
||||
region_id,
|
||||
Region {
|
||||
bytes,
|
||||
max_bytes,
|
||||
generation: 1,
|
||||
},
|
||||
);
|
||||
self.committed_bytes = committed;
|
||||
Ok(MemoryHandle::from_runtime(self.run_id, region_id, 1))
|
||||
}
|
||||
|
||||
/// Current length of a region in pages.
|
||||
pub fn size(&self, handle: MemoryHandle, span: Option<Span>) -> Result<u64, RuntimeError> {
|
||||
let region = self.region(handle, span)?;
|
||||
Ok((region.bytes.len() / PAGE_SIZE) as u64)
|
||||
}
|
||||
|
||||
/// Grow a region by whole pages and return its previous page count.
|
||||
///
|
||||
/// A failed grow is atomic: the region keeps its old length and bytes and
|
||||
/// the committed budget is unchanged.
|
||||
pub fn grow(
|
||||
&mut self,
|
||||
handle: MemoryHandle,
|
||||
pages: u64,
|
||||
span: Option<Span>,
|
||||
) -> Result<u64, RuntimeError> {
|
||||
if pages == 0 {
|
||||
return self.size(handle, span);
|
||||
}
|
||||
let region = self.region(handle, span)?;
|
||||
let old_bytes = region.bytes.len();
|
||||
let old_pages = (old_bytes / PAGE_SIZE) as u64;
|
||||
let additional = checked_bytes(pages, span, "memory.grow pages")?;
|
||||
let new_bytes = old_bytes
|
||||
.checked_add(additional)
|
||||
.ok_or_else(|| memory_error(span, "memory.grow size overflows the address space"))?;
|
||||
if new_bytes > region.max_bytes {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!(
|
||||
"memory.grow would grow the region to {new_bytes} bytes, beyond its \
|
||||
declared maximum of {} bytes",
|
||||
region.max_bytes
|
||||
),
|
||||
));
|
||||
}
|
||||
let committed = self
|
||||
.committed_bytes
|
||||
.checked_add(additional)
|
||||
.ok_or_else(|| budget_error(span, "memory budget overflow"))?;
|
||||
if committed > self.limits.max_total_bytes {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!(
|
||||
"memory budget exceeded: growing by {additional} bytes would reach \
|
||||
{committed} of {}",
|
||||
self.limits.max_total_bytes
|
||||
),
|
||||
));
|
||||
}
|
||||
|
||||
let region = self
|
||||
.regions
|
||||
.get_mut(&handle.region_id())
|
||||
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
|
||||
region.bytes.try_reserve(additional).map_err(|_| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!("cannot grow addressable memory by {additional} bytes"),
|
||||
)
|
||||
})?;
|
||||
region.bytes.resize(new_bytes, 0);
|
||||
self.committed_bytes = committed;
|
||||
Ok(old_pages)
|
||||
}
|
||||
|
||||
/// Total byte length of a region.
|
||||
pub fn byte_len(
|
||||
&self,
|
||||
handle: MemoryHandle,
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
Ok(self.region(handle, span)?.bytes.len())
|
||||
}
|
||||
|
||||
/// Read a contiguous byte range. Used by resource I/O adapters.
|
||||
pub fn read_bytes(
|
||||
&self,
|
||||
handle: MemoryHandle,
|
||||
offset: &BigInt,
|
||||
length: usize,
|
||||
span: Option<Span>,
|
||||
) -> Result<Vec<u8>, RuntimeError> {
|
||||
let region = self.region(handle, span)?;
|
||||
let start = range_offset(offset, region.bytes.len(), length, span)?;
|
||||
Ok(region.bytes[start..start + length].to_vec())
|
||||
}
|
||||
|
||||
/// Write a contiguous byte range atomically after a full bounds check.
|
||||
pub fn write_bytes(
|
||||
&mut self,
|
||||
handle: MemoryHandle,
|
||||
offset: &BigInt,
|
||||
bytes: &[u8],
|
||||
span: Option<Span>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let len = self.region(handle, span)?.bytes.len();
|
||||
let start = range_offset(offset, len, bytes.len(), span)?;
|
||||
let region = self
|
||||
.regions
|
||||
.get_mut(&handle.region_id())
|
||||
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
|
||||
region.bytes[start..start + bytes.len()].copy_from_slice(bytes);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Read one byte. Offsets are non-negative and region-relative.
|
||||
pub fn read(
|
||||
&self,
|
||||
handle: MemoryHandle,
|
||||
offset: &BigInt,
|
||||
span: Option<Span>,
|
||||
) -> Result<u8, RuntimeError> {
|
||||
let region = self.region(handle, span)?;
|
||||
let position = checked_offset(offset, region.bytes.len(), span)?;
|
||||
Ok(region.bytes[position])
|
||||
}
|
||||
|
||||
/// Write one byte. The caller must have validated the byte range.
|
||||
pub fn write(
|
||||
&mut self,
|
||||
handle: MemoryHandle,
|
||||
offset: &BigInt,
|
||||
byte: u8,
|
||||
span: Option<Span>,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let len = self.region(handle, span)?.bytes.len();
|
||||
let position = checked_offset(offset, len, span)?;
|
||||
let region = self
|
||||
.regions
|
||||
.get_mut(&handle.region_id())
|
||||
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
|
||||
region.bytes[position] = byte;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn region(&self, handle: MemoryHandle, span: Option<Span>) -> Result<&Region, RuntimeError> {
|
||||
if handle.run_id() != self.run_id {
|
||||
return Err(memory_error(
|
||||
span,
|
||||
"Memory handle belongs to a different execution run",
|
||||
));
|
||||
}
|
||||
let Some(region) = self.regions.get(&handle.region_id()) else {
|
||||
return Err(memory_error(span, "unknown Memory region handle"));
|
||||
};
|
||||
if region.generation != handle.generation() {
|
||||
return Err(memory_error(span, "stale Memory region handle"));
|
||||
}
|
||||
Ok(region)
|
||||
}
|
||||
}
|
||||
|
||||
fn checked_bytes(pages: u64, span: Option<Span>, what: &str) -> Result<usize, RuntimeError> {
|
||||
let bytes = pages
|
||||
.checked_mul(PAGE_SIZE as u64)
|
||||
.ok_or_else(|| memory_error(span, format!("{what} overflows the address space")))?;
|
||||
usize::try_from(bytes).map_err(|_| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceExhausted,
|
||||
span,
|
||||
format!("{what} ({pages} pages) does not fit this platform's address space"),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/// Validate an offset plus a byte count against the region length.
|
||||
fn range_offset(
|
||||
offset: &BigInt,
|
||||
len: usize,
|
||||
length: usize,
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
if offset.sign() == num_bigint::Sign::Minus {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
"Memory offset must be a non-negative integer",
|
||||
));
|
||||
}
|
||||
let Some(start) = offset.to_usize() else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
format!("Memory offset {offset} is out of bounds for region length {len}"),
|
||||
));
|
||||
};
|
||||
let end = start.checked_add(length).ok_or_else(|| {
|
||||
memory_error(
|
||||
span,
|
||||
"Memory offset plus length overflows the address space",
|
||||
)
|
||||
})?;
|
||||
if end > len {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
format!("Memory range {start}..{end} is out of bounds for region length {len}"),
|
||||
));
|
||||
}
|
||||
Ok(start)
|
||||
}
|
||||
|
||||
fn checked_offset(offset: &BigInt, len: usize, span: Option<Span>) -> Result<usize, RuntimeError> {
|
||||
if offset.sign() == num_bigint::Sign::Minus {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
"Memory offset must be a non-negative integer",
|
||||
));
|
||||
}
|
||||
let Some(position) = offset.to_usize() else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
format!("Memory offset {offset} is out of bounds for region length {len}"),
|
||||
));
|
||||
};
|
||||
if position >= len {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::IndexOutOfBounds,
|
||||
span,
|
||||
format!("Memory offset {offset} is out of bounds for region length {len}"),
|
||||
));
|
||||
}
|
||||
Ok(position)
|
||||
}
|
||||
|
||||
fn memory_error(span: Option<Span>, message: impl Into<String>) -> RuntimeError {
|
||||
RuntimeError::new(ErrorCategory::MemoryFault, span, message)
|
||||
}
|
||||
|
||||
fn budget_error(span: Option<Span>, message: impl Into<String>) -> RuntimeError {
|
||||
RuntimeError::new(ErrorCategory::ResourceExhausted, span, message)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn int(value: impl Into<BigInt>) -> BigInt {
|
||||
value.into()
|
||||
}
|
||||
|
||||
fn store() -> MemoryStore {
|
||||
MemoryStore::new(MemoryLimits::default())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn regions_are_isolated_and_handles_share_identity() {
|
||||
let mut store = store();
|
||||
let first = store.create(1, 2, None).unwrap();
|
||||
let second = store.create(1, 2, None).unwrap();
|
||||
let alias = first;
|
||||
store.write(first, &int(0), 7, None).unwrap();
|
||||
store.write(second, &int(0), 9, None).unwrap();
|
||||
assert_eq!(store.read(first, &int(0), None).unwrap(), 7);
|
||||
assert_eq!(store.read(alias, &int(0), None).unwrap(), 7);
|
||||
assert_eq!(store.read(second, &int(0), None).unwrap(), 9);
|
||||
assert_eq!(first, alias);
|
||||
assert_ne!(first, second);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn create_is_zero_filled_and_reports_pages() {
|
||||
let mut store = store();
|
||||
let memory = store.create(1, 4, None).unwrap();
|
||||
assert_eq!(store.size(memory, None).unwrap(), 1);
|
||||
assert_eq!(PAGE_SIZE, 64 * 1024);
|
||||
for offset in [0usize, 1, PAGE_SIZE - 1] {
|
||||
assert_eq!(store.read(memory, &int(offset as u64), None).unwrap(), 0);
|
||||
}
|
||||
// Zero-page creation is legal but every access is out of bounds.
|
||||
let empty = store.create(0, 1, None).unwrap();
|
||||
assert_eq!(store.size(empty, None).unwrap(), 0);
|
||||
assert_eq!(
|
||||
store.read(empty, &int(0), None).unwrap_err().category,
|
||||
ErrorCategory::IndexOutOfBounds
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grow_keeps_old_bytes_and_zero_fills_new_pages() {
|
||||
let mut store = store();
|
||||
let memory = store.create(1, 2, None).unwrap();
|
||||
store.write(memory, &int(3), 200, None).unwrap();
|
||||
assert_eq!(store.grow(memory, 1, None).unwrap(), 1);
|
||||
assert_eq!(store.size(memory, None).unwrap(), 2);
|
||||
assert_eq!(store.read(memory, &int(3), None).unwrap(), 200);
|
||||
assert_eq!(store.read(memory, &int(PAGE_SIZE as u64), None).unwrap(), 0);
|
||||
store
|
||||
.write(memory, &int(PAGE_SIZE as u64 + 5), 42, None)
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
store
|
||||
.read(memory, &int(PAGE_SIZE as u64 + 5), None)
|
||||
.unwrap(),
|
||||
42
|
||||
);
|
||||
// Zero-page growth is a no-op returning the old count.
|
||||
assert_eq!(store.grow(memory, 0, None).unwrap(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn failed_grow_does_not_change_region_or_budget() {
|
||||
let mut store = MemoryStore::new(MemoryLimits {
|
||||
max_total_bytes: PAGE_SIZE * 2,
|
||||
max_region_bytes: PAGE_SIZE * 4,
|
||||
});
|
||||
let memory = store.create(1, 2, None).unwrap();
|
||||
store.write(memory, &int(0), 1, None).unwrap();
|
||||
let before = store.committed_bytes();
|
||||
let error = store.grow(memory, 2, None).unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
|
||||
assert_eq!(store.size(memory, None).unwrap(), 1);
|
||||
assert_eq!(store.read(memory, &int(0), None).unwrap(), 1);
|
||||
assert_eq!(store.committed_bytes(), before);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalid_offsets_are_structured_errors() {
|
||||
let mut store = store();
|
||||
let memory = store.create(1, 1, None).unwrap();
|
||||
for offset in [int(-1), int(PAGE_SIZE as u64)] {
|
||||
let error = store.read(memory, &offset, None).unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::IndexOutOfBounds);
|
||||
}
|
||||
let huge: BigInt = "999999999999999999999999999999999999999".parse().unwrap();
|
||||
assert_eq!(
|
||||
store.read(memory, &huge, None).unwrap_err().category,
|
||||
ErrorCategory::IndexOutOfBounds
|
||||
);
|
||||
assert_eq!(
|
||||
store.write(memory, &huge, 1, None).unwrap_err().category,
|
||||
ErrorCategory::IndexOutOfBounds
|
||||
);
|
||||
assert_eq!(
|
||||
store
|
||||
.write(memory, &int(PAGE_SIZE as u64), 1, None)
|
||||
.unwrap_err()
|
||||
.category,
|
||||
ErrorCategory::IndexOutOfBounds
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn create_validates_page_counts_and_budget() {
|
||||
let mut store = MemoryStore::new(MemoryLimits {
|
||||
max_total_bytes: PAGE_SIZE * 3,
|
||||
max_region_bytes: PAGE_SIZE * 2,
|
||||
});
|
||||
assert_eq!(
|
||||
store.create(2, 1, None).unwrap_err().category,
|
||||
ErrorCategory::MemoryFault
|
||||
);
|
||||
assert_eq!(
|
||||
store.create(1, 3, None).unwrap_err().category,
|
||||
ErrorCategory::ResourceExhausted
|
||||
);
|
||||
let _first = store.create(2, 2, None).unwrap();
|
||||
let _second = store.create(1, 1, None).unwrap();
|
||||
// Total budget is now exactly three pages.
|
||||
assert_eq!(
|
||||
store.create(1, 1, None).unwrap_err().category,
|
||||
ErrorCategory::ResourceExhausted
|
||||
);
|
||||
assert_eq!(store.committed_bytes(), PAGE_SIZE * 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn forged_and_stale_handles_are_rejected() {
|
||||
let mut store = store();
|
||||
let memory = store.create(1, 1, None).unwrap();
|
||||
let forged = MemoryHandle::from_runtime(memory.run_id() + 1, memory.region_id(), 1);
|
||||
assert_eq!(
|
||||
store.read(forged, &int(0), None).unwrap_err().category,
|
||||
ErrorCategory::MemoryFault
|
||||
);
|
||||
let stale = MemoryHandle::from_runtime(memory.run_id(), memory.region_id(), 2);
|
||||
assert_eq!(
|
||||
store.read(stale, &int(0), None).unwrap_err().category,
|
||||
ErrorCategory::MemoryFault
|
||||
);
|
||||
let unknown = MemoryHandle::from_runtime(memory.run_id(), 999, 1);
|
||||
assert_eq!(
|
||||
store.write(unknown, &int(0), 1, None).unwrap_err().category,
|
||||
ErrorCategory::MemoryFault
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn page_arithmetic_never_overflows() {
|
||||
let mut store = MemoryStore::new(MemoryLimits {
|
||||
max_total_bytes: usize::MAX,
|
||||
max_region_bytes: usize::MAX,
|
||||
});
|
||||
assert_eq!(
|
||||
store.create(u64::MAX, u64::MAX, None).unwrap_err().category,
|
||||
ErrorCategory::ResourceExhausted
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -107,6 +107,32 @@ fn compare(
|
||||
}
|
||||
}))
|
||||
}
|
||||
(Value::Resource(a), Value::Resource(b)) => {
|
||||
return Ok(Value::Bool(match op {
|
||||
BinaryOp::Eq => a == b,
|
||||
BinaryOp::Ne => a != b,
|
||||
_ => {
|
||||
return Err(type_error(
|
||||
span,
|
||||
"Resource ordering comparisons are not supported".to_string(),
|
||||
))
|
||||
}
|
||||
}));
|
||||
}
|
||||
(Value::Memory(a), Value::Memory(b)) => {
|
||||
// Identity, not byte content: two handles for the same region are
|
||||
// equal, regions with identical bytes are not (ADR-0022 §1).
|
||||
return Ok(Value::Bool(match op {
|
||||
BinaryOp::Eq => a == b,
|
||||
BinaryOp::Ne => a != b,
|
||||
_ => {
|
||||
return Err(type_error(
|
||||
span,
|
||||
"Memory ordering comparisons are not supported".to_string(),
|
||||
))
|
||||
}
|
||||
}));
|
||||
}
|
||||
(a, b) => {
|
||||
if op == BinaryOp::Eq {
|
||||
return Ok(Value::Bool(false));
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
use marqdo_host::{Capabilities, Host, StdHost};
|
||||
use marqdo_ir::{ModuleId, Program};
|
||||
use marqdo_runtime::{run, EvaluatedArg, RuntimeError, Value};
|
||||
use marqdo_runtime::{run, run_with_limits, EvaluatedArg, MemoryLimits, RuntimeError, Value};
|
||||
use marqdo_sandbox::SandboxMode;
|
||||
use marqdo_span::Span;
|
||||
|
||||
@@ -70,11 +70,10 @@ fn recursion_branch_and_return() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn loop_prints_and_returns_unit_for_fallthrough() {
|
||||
let source = "## 倒计时\n\n**n = 3**\n\n- `n > 0`\n\n > 打印(n)\n\n **n = n - 1**\n";
|
||||
let (result, output) = run_and_output(source, "倒计时");
|
||||
fn loop_mutates_and_returns_unit_for_fallthrough() {
|
||||
let source = "## 倒计时\n\n**n = 3**\n\n- `n > 0`\n\n **n = n - 1**\n";
|
||||
let (result, _) = run_and_output(source, "倒计时");
|
||||
assert_eq!(result.unwrap(), Value::Unit);
|
||||
assert_eq!(output, "3\n2\n1\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -117,14 +116,14 @@ fn function_reference_can_be_called() {
|
||||
|
||||
#[test]
|
||||
fn non_bool_condition_is_a_runtime_error() {
|
||||
let source = "## f\n\n**x = 1**\n\n- `x`\n\n > 打印(1)\n\n*x*\n";
|
||||
let source = "## f\n\n**x = 1**\n\n- `x`\n\n **x = x + 1**\n\n*x*\n";
|
||||
let error = run_named(source, "f", &[]).unwrap_err();
|
||||
assert_eq!(error.code(), "runtime.type");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fuel_budget_is_enforced() {
|
||||
let program = build("## f\n\n- `true`\n\n > 打印(1)\n\n*0*\n");
|
||||
let program = build("## f\n\n- `true`\n\n **x = 1**\n\n*0*\n");
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::pure(),
|
||||
@@ -170,3 +169,73 @@ fn ambiguous_ref_resource_is_an_error() {
|
||||
let error = run_named(source, "f", &[]).unwrap_err();
|
||||
assert_eq!(error.code(), "ref.ambiguous");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn print_without_the_bundled_stdlib_is_a_structured_error() {
|
||||
let source = "## f\n\n> 打印(1)\n\n*None*\n";
|
||||
let error = run_named(source, "f", &[]).unwrap_err();
|
||||
assert_eq!(error.code(), "kernel.unavailable");
|
||||
assert!(
|
||||
error.message.contains("bundled stdlib"),
|
||||
"unexpected message: {}",
|
||||
error.message
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn function_ref_and_resource_members_are_readable() {
|
||||
let source = "## 目标\n\n*1*\n\n## main\n\n### 公式\n\n$$a = 1$$\n\n### 代码\n\n```text\nx\n```\n\n**f = ref(\"#目标\")**\n\n**g = ref(\"#公式\")**\n\n**c = ref(\"#代码\")**\n\n*[f.module, f.name, g.latex, g.display, c.language, c.source]*\n";
|
||||
let value = run_named(source, "main", &[]).unwrap();
|
||||
let Value::List(items) = value else {
|
||||
panic!("expected list");
|
||||
};
|
||||
assert!(matches!(&items[0], Value::Text(module) if module.ends_with("index.mq.md")));
|
||||
assert_eq!(items[1], Value::text("目标"));
|
||||
assert_eq!(items[2], Value::text("a = 1"));
|
||||
assert_eq!(items[3], Value::Bool(true));
|
||||
assert_eq!(items[4], Value::text("text"));
|
||||
assert_eq!(items[5], Value::text("x\n"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn memory_region_is_shared_by_handle_identity() {
|
||||
let source = "## f\n\n**m = memory.new(1, 2)**\n\n**n = m**\n\n**`m[3] = 7`**\n\n*[n[3], m == n, memory.size(n)]*\n";
|
||||
let value = run_named(source, "f", &[]).unwrap();
|
||||
assert_eq!(
|
||||
value,
|
||||
Value::List(vec![Value::int(7), Value::Bool(true), Value::int(1)])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn memory_handle_cannot_reach_the_host_boundary() {
|
||||
let source = "## f\n\n**m = memory.new(1, 1)**\n\n> fs.read(m)\n\n*0*\n";
|
||||
let error = run_named(source, "f", &[]).unwrap_err();
|
||||
assert_eq!(error.code(), "runtime.type");
|
||||
assert!(
|
||||
error
|
||||
.message
|
||||
.contains("Memory values cannot cross the host boundary"),
|
||||
"unexpected message: {}",
|
||||
error.message
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn memory_budget_is_enforced_without_changing_the_region() {
|
||||
let program = build("## f\n\n**m = memory.new(1, 1)**\n\n*[m[0], memory.grow(m, 1)]*\n");
|
||||
let mut host = host();
|
||||
let error = run_with_limits(
|
||||
&program,
|
||||
&mut host,
|
||||
&ModuleId::from("index.mq.md"),
|
||||
"f",
|
||||
Vec::new(),
|
||||
MemoryLimits {
|
||||
max_total_bytes: 64 * 1024,
|
||||
max_region_bytes: 64 * 1024 * 2,
|
||||
},
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.code(), "resource.exhausted");
|
||||
}
|
||||
@@ -232,6 +232,8 @@ pub struct StrictHostCommand<'a> {
|
||||
pub output_limit: usize,
|
||||
/// Wall-clock budget for the supervised process tree.
|
||||
pub timeout: Duration,
|
||||
/// Verified bundled stdlib directory to bind read-only when present.
|
||||
pub stdlib_dir: Option<&'a Path>,
|
||||
}
|
||||
|
||||
/// Launch a command inside the strict sandbox (Linux only).
|
||||
|
||||
@@ -317,6 +317,16 @@ fn run_with(
|
||||
})?;
|
||||
|
||||
let mut args = common_args();
|
||||
if let Some(stdlib) = command.stdlib_dir {
|
||||
args.extend([
|
||||
OsString::from("--ro-bind"),
|
||||
stdlib.as_os_str().to_os_string(),
|
||||
OsString::from("/run/marqdo-stdlib"),
|
||||
OsString::from("--setenv"),
|
||||
OsString::from("MARQDO_STDLIB_DIR"),
|
||||
OsString::from("/run/marqdo-stdlib"),
|
||||
]);
|
||||
}
|
||||
args.extend([
|
||||
OsString::from("--ro-bind"),
|
||||
project_root.as_os_str().to_os_string(),
|
||||
@@ -548,6 +558,7 @@ mod tests {
|
||||
original_document: None,
|
||||
output_limit: 4096,
|
||||
timeout: Duration::from_secs(1),
|
||||
stdlib_dir: None,
|
||||
};
|
||||
let error = run_with(
|
||||
&command,
|
||||
@@ -585,6 +596,7 @@ mod tests {
|
||||
original_document: None,
|
||||
output_limit: 4096,
|
||||
timeout: Duration::from_secs(1),
|
||||
stdlib_dir: None,
|
||||
};
|
||||
let error = run_with(
|
||||
&command,
|
||||
|
||||
@@ -111,6 +111,7 @@ fn run(helper: &Path, project: &Path, outside: &Path, mode: &str) -> StrictOutco
|
||||
original_document: None,
|
||||
output_limit: 64 * 1024,
|
||||
timeout: Duration::from_secs(3),
|
||||
stdlib_dir: None,
|
||||
};
|
||||
launch_strict_capture(&command).unwrap_or_else(|error| panic!("{mode}: {error}"))
|
||||
}
|
||||
@@ -267,6 +268,7 @@ fn strict_supervisor_enforces_memory_cpu_and_output_limits() {
|
||||
original_document: None,
|
||||
output_limit: 64 * 1024,
|
||||
timeout: Duration::from_secs(2),
|
||||
stdlib_dir: None,
|
||||
};
|
||||
let cpu = launch_strict_capture(&command).expect_err("CPU loop must be stopped");
|
||||
let message = cpu.to_string();
|
||||
@@ -283,6 +285,7 @@ fn strict_supervisor_enforces_memory_cpu_and_output_limits() {
|
||||
original_document: None,
|
||||
output_limit: 64 * 1024,
|
||||
timeout: Duration::from_secs(3),
|
||||
stdlib_dir: None,
|
||||
})
|
||||
.expect("output helper should be captured, not spawn-failed");
|
||||
assert!(
|
||||
|
||||
@@ -23,8 +23,19 @@ pub enum ErrorCategory {
|
||||
NotCallable,
|
||||
/// A map key or record field does not exist.
|
||||
UnknownField,
|
||||
/// A list/map index is out of bounds.
|
||||
/// A list/map/memory index is out of bounds.
|
||||
IndexOutOfBounds,
|
||||
/// An addressable-memory operation failed for a reason other than a plain
|
||||
/// out-of-bounds read: invalid page counts, an unknown/stale handle, or an
|
||||
/// arithmetic overflow in an address.
|
||||
MemoryFault,
|
||||
/// A resource handle is forged, cross-run or stale.
|
||||
ResourceFault,
|
||||
/// A resource operation used an already-closed handle (recoverable as a
|
||||
/// `Result` by the builtin).
|
||||
ResourceClosed,
|
||||
/// The requested resource kind/descriptor/rights are not implemented.
|
||||
ResourceUnsupported,
|
||||
/// A resource budget was exhausted (fuel, stack, output size).
|
||||
ResourceExhausted,
|
||||
/// A capability check denied the operation.
|
||||
@@ -60,6 +71,10 @@ impl ErrorCategory {
|
||||
ErrorCategory::NotCallable => "call.not_callable",
|
||||
ErrorCategory::UnknownField => "runtime.unknown_field",
|
||||
ErrorCategory::IndexOutOfBounds => codes::RUNTIME_INDEX,
|
||||
ErrorCategory::MemoryFault => "runtime.memory",
|
||||
ErrorCategory::ResourceFault => "runtime.resource",
|
||||
ErrorCategory::ResourceClosed => "resource.closed",
|
||||
ErrorCategory::ResourceUnsupported => "resource.unsupported",
|
||||
ErrorCategory::ResourceExhausted => codes::RESOURCE_EXHAUSTED,
|
||||
ErrorCategory::PermissionDenied => codes::PERMISSION_DENIED,
|
||||
ErrorCategory::KernelUnavailable => codes::KERNEL_UNAVAILABLE,
|
||||
|
||||
@@ -15,6 +15,6 @@ pub use error::{CallFrame, ErrorCategory, RuntimeError};
|
||||
pub use marqdo_span::{LineCol, Span};
|
||||
pub use value::{
|
||||
bigint_to_exact_f64, float_to_bigint, format_float, CodeValue, FormulaValue, FunctionRef,
|
||||
PluginHandle, Table, Value,
|
||||
MemoryHandle, PluginHandle, ResourceHandle, Table, Value,
|
||||
};
|
||||
pub use wire::{from_wire, to_wire, WireError, WIRE_VERSION};
|
||||
@@ -75,6 +75,88 @@ pub struct PluginHandle {
|
||||
pub id: u64,
|
||||
}
|
||||
|
||||
/// Opaque identity of one runtime-managed byte region (ADR-0022).
|
||||
///
|
||||
/// A handle is issued only by the runtime's `MemoryStore`. Copying a `Memory`
|
||||
/// value or passing it as an argument preserves this identity, so two names
|
||||
/// can observe each other's writes. The fields are deliberately not part of
|
||||
/// any wire format: [`crate::wire`] rejects the `memory` tag on decode, and no
|
||||
/// public constructor accepts untrusted data. There is no `usize` address in
|
||||
/// the language value; offsets are region-relative integers.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
|
||||
pub struct MemoryHandle {
|
||||
run: u64,
|
||||
region: u64,
|
||||
generation: u64,
|
||||
}
|
||||
|
||||
impl MemoryHandle {
|
||||
/// Create a handle. Only the runtime store may call this; users cannot
|
||||
/// construct one from a number, a document or wire data.
|
||||
#[doc(hidden)]
|
||||
pub const fn from_runtime(run: u64, region: u64, generation: u64) -> Self {
|
||||
Self {
|
||||
run,
|
||||
region,
|
||||
generation,
|
||||
}
|
||||
}
|
||||
|
||||
/// Run identity this region belongs to.
|
||||
pub const fn run_id(&self) -> u64 {
|
||||
self.run
|
||||
}
|
||||
|
||||
/// Region identity within the run.
|
||||
pub const fn region_id(&self) -> u64 {
|
||||
self.region
|
||||
}
|
||||
|
||||
/// Generation of the region slot (guards a future reuse policy).
|
||||
pub const fn generation(&self) -> u64 {
|
||||
self.generation
|
||||
}
|
||||
}
|
||||
|
||||
/// Opaque host-issued handle to an external resource/stream (ADR-0025).
|
||||
///
|
||||
/// Fields are private and the constructor is host-only. A handle is never
|
||||
/// built from wire data or a user integer; it is a different domain from
|
||||
/// `Memory` and cannot be indexed.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
|
||||
pub struct ResourceHandle {
|
||||
run: u64,
|
||||
resource: u64,
|
||||
generation: u64,
|
||||
}
|
||||
|
||||
impl ResourceHandle {
|
||||
/// Host-only constructor.
|
||||
#[doc(hidden)]
|
||||
pub const fn from_host(run: u64, resource: u64, generation: u64) -> Self {
|
||||
Self {
|
||||
run,
|
||||
resource,
|
||||
generation,
|
||||
}
|
||||
}
|
||||
|
||||
/// Host run identity.
|
||||
pub const fn run_id(&self) -> u64 {
|
||||
self.run
|
||||
}
|
||||
|
||||
/// Resource slot identity.
|
||||
pub const fn resource_id(&self) -> u64 {
|
||||
self.resource
|
||||
}
|
||||
|
||||
/// Slot generation.
|
||||
pub const fn generation(&self) -> u64 {
|
||||
self.generation
|
||||
}
|
||||
}
|
||||
|
||||
/// A Marqdo runtime value.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum Value {
|
||||
@@ -104,6 +186,11 @@ pub enum Value {
|
||||
FunctionRef(FunctionRef),
|
||||
/// Host-owned plugin handle.
|
||||
PluginHandle(PluginHandle),
|
||||
/// Runtime-owned addressable byte region (RFC-CM-0001). Copies share the
|
||||
/// same region identity; they are not byte snapshots.
|
||||
Memory(MemoryHandle),
|
||||
/// Host-owned external resource/stream handle (ADR-0025).
|
||||
Resource(ResourceHandle),
|
||||
}
|
||||
|
||||
impl Value {
|
||||
@@ -123,6 +210,8 @@ impl Value {
|
||||
Value::Code(_) => "Code",
|
||||
Value::FunctionRef(_) => "FunctionRef",
|
||||
Value::PluginHandle(_) => "PluginHandle",
|
||||
Value::Memory(_) => "Memory",
|
||||
Value::Resource(_) => "Resource",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -200,6 +289,44 @@ impl Value {
|
||||
Value::Code(c) => format!("Code({:?}, {} bytes)", c.language, c.source.len()),
|
||||
Value::FunctionRef(r) => format!("FunctionRef({}::{})", r.module, r.name),
|
||||
Value::PluginHandle(h) => format!("PluginHandle({}::{})", h.plugin, h.export),
|
||||
// Never render the runtime handle: internal region identity is not
|
||||
// observable program data (RFC-CM-0001 §8).
|
||||
Value::Memory(_) => "Memory".to_string(),
|
||||
// Resource identity is host-private and never program data.
|
||||
Value::Resource(_) => "Resource".to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether this value contains a live `Memory` handle anywhere.
|
||||
///
|
||||
/// Used at the host/plugin/foreign boundary, which must reject handles
|
||||
/// instead of serializing them.
|
||||
pub fn contains_memory(&self) -> bool {
|
||||
match self {
|
||||
Value::Memory(_) => true,
|
||||
Value::List(items) => items.iter().any(Value::contains_memory),
|
||||
Value::Map(entries) => entries.values().any(Value::contains_memory),
|
||||
Value::Table(table) => table
|
||||
.rows
|
||||
.iter()
|
||||
.flat_map(|row| row.iter())
|
||||
.any(Value::contains_memory),
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether this value contains a live `Resource` handle anywhere.
|
||||
pub fn contains_resource(&self) -> bool {
|
||||
match self {
|
||||
Value::Resource(_) => true,
|
||||
Value::List(items) => items.iter().any(Value::contains_resource),
|
||||
Value::Map(entries) => entries.values().any(Value::contains_resource),
|
||||
Value::Table(table) => table
|
||||
.rows
|
||||
.iter()
|
||||
.flat_map(|row| row.iter())
|
||||
.any(Value::contains_resource),
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -325,6 +452,18 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn memory_display_never_leaks_the_runtime_handle() {
|
||||
let memory = Value::Memory(MemoryHandle::from_runtime(41, 7, 1));
|
||||
assert_eq!(memory.type_name(), "Memory");
|
||||
assert_eq!(memory.display(), "Memory");
|
||||
assert_eq!(memory.repr(), "Memory");
|
||||
assert!(!memory.display().contains("41"));
|
||||
assert!(!memory.repr().contains('7'));
|
||||
assert!(memory.contains_memory());
|
||||
assert!(Value::List(vec![memory]).contains_memory());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn big_integers_are_arbitrary_precision() {
|
||||
let big: BigInt = "123456789012345678901234567890".parse().unwrap();
|
||||
|
||||
@@ -145,6 +145,12 @@ fn encode(value: &Value) -> Json {
|
||||
"export": handle.export,
|
||||
"id": handle.id,
|
||||
}),
|
||||
// Active memory handles never cross the wire (RFC-CM-0001 §8).
|
||||
// The tag is emitted so an accidental transfer is visible instead of
|
||||
// silently becoming a number; decoding it is a hard error.
|
||||
Value::Memory(_) => json!({"type": "memory"}),
|
||||
// Resource handles are host-private; same non-transferable rule.
|
||||
Value::Resource(_) => json!({"type": "resource"}),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -321,6 +327,12 @@ fn decode(json: &Json, depth: usize, limits: &WireLimits) -> Result<Value, WireE
|
||||
.as_u64()
|
||||
.ok_or_else(|| WireError::Invalid("plugin_handle.id must be an integer".into()))?,
|
||||
})),
|
||||
"memory" => Err(WireError::Invalid(
|
||||
"Memory handles cannot cross the wire; copy bytes explicitly".into(),
|
||||
)),
|
||||
"resource" => Err(WireError::Invalid(
|
||||
"Resource handles cannot cross the wire".into(),
|
||||
)),
|
||||
other => Err(WireError::UnknownType(other.to_string())),
|
||||
}
|
||||
}
|
||||
@@ -393,6 +405,34 @@ mod tests {
|
||||
assert!(matches!(decoded, Value::Float(f) if f == f64::NEG_INFINITY));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn memory_handles_are_not_transferable() {
|
||||
let value = Value::Memory(crate::MemoryHandle::from_runtime(1, 2, 1));
|
||||
let wire = to_wire(&value);
|
||||
assert_eq!(wire, json!({"type": "memory"}));
|
||||
assert_eq!(
|
||||
from_wire(&wire),
|
||||
Err(WireError::Invalid(
|
||||
"Memory handles cannot cross the wire; copy bytes explicitly".into()
|
||||
))
|
||||
);
|
||||
// A hand-written fake handle is rejected by the same tag rule.
|
||||
assert!(from_wire(&json!({"type": "memory", "region": 2})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resource_handles_are_not_transferable() {
|
||||
let value = Value::Resource(crate::ResourceHandle::from_host(1, 2, 1));
|
||||
let wire = to_wire(&value);
|
||||
assert_eq!(wire, json!({"type": "resource"}));
|
||||
assert_eq!(
|
||||
from_wire(&wire),
|
||||
Err(WireError::Invalid(
|
||||
"Resource handles cannot cross the wire".into()
|
||||
))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn untrusted_wire_is_validated() {
|
||||
assert_eq!(
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,498 @@
|
||||
# RFC-CM-0001:Marqdo 自主计算机器、可寻址内存与系统交互边界
|
||||
|
||||
> **状态**:设计提案(Draft,未实施,未批准)
|
||||
> **日期**:2026-11-20
|
||||
> **建议归档路径**:`docs/architecture/RFC-CM-0001-compute-machine.md`
|
||||
> **验证基线**:`cflmy/marqdo-cli`,Git 提交 `a113d894962d8c00eb02efb42e0e73f8b084a1fa`(检索于 2026-11-20)
|
||||
> **兼容性**:涉及 `Value`、表达式赋值、运行时、主机能力和线协议的变更,必须通过 ADR、规范版本与相应测试;本 RFC 不声明任何新接口目前可用。
|
||||
|
||||
---
|
||||
|
||||
## 摘要
|
||||
|
||||
Marqdo 的优势不应只是用 Markdown 排版代码,而应是:在不更换 Markdown 语法角色的前提下,以**自身定义的计算语义**构造算法、数据结构、运行时工具和知识制品。现有解释器已提供整数、浮点、文本、列表、映射、控制流和函数调用,但尚未提供**由 Marqdo 程序控制的可寻址字节存储**;其赋值左侧仅支持标识符,系统交互集中在一组固定 `HostOp` 与外部插件能力中。这妨碍了用 `.mq.md` 自己编写内存分配器、二进制协议、共享节点结构、虚拟机器等低层算法。
|
||||
|
||||
**提议:以“托管值 + 多个可增长的线性内存区域 + 不透明系统资源”为计算机器的三个基础领域。**
|
||||
|
||||
1. 保留现有 `Value` 计算体系,不强迫一切数据都变成字节地址。
|
||||
2. 引入一个拥有独立身份、按字节索引、可增长、具有边界的 `Memory` 值;每个区域内部线性寻址,物理实现可以是 Vec、分页或其他表示。
|
||||
3. 扩展现有“位置赋值”:允许对适当的索引目标写入,如 `m[8] = 42`,而不是另创 Markdown 语法或几十个关键字。
|
||||
4. 为创建、测量和增长内存定义**极少量计算原语**;让 `malloc/free`、内存池、链表、编码、缓冲区等**首先在 `.mq.md` 标准库**中实现。
|
||||
5. 将终端、文件、网络、进程、设备定义为**外部资源**,经稳定、可授权、可替换的底层适配接口交互;Marqdo 自身可以实现高级系统库与协议,但系统调用或硬件访问的**最后一步不能凭空消失**。
|
||||
6. CPU 寄存器由解释器/编译后端管理;裸机器码、GPU/QPU、共享内存原子操作暂不进入本 RFC 的基础语义。
|
||||
|
||||
这是一种**可编程的抽象机器**,而不是“在每种新需求前追加 Rust Host 函数”的框架,也不是把 x86/RISC-V 指令直接变成 Markdown 语法。
|
||||
|
||||
## 1. 设计动机:究竟希望解决什么
|
||||
|
||||
### 1.1 真实诉求
|
||||
|
||||
Marqdo 必须能在不借助 Python、JavaScript 或宿主语言提供专门算法函数的前提下,**通过自己的控制流和基础存储机制**构造原先不存在的高级能力。例如:实现内存分配器、二叉树、哈希表、字节码解释器、文件格式解析器、数值算法。对于 OS 和硬件,要求的是**高级功能主要由 Marqdo 实现,底层权限与系统调用接口保持稳定**,而不是“所有跨越内核边界的指令都神奇地由 Markdown 自己完成”。
|
||||
|
||||
### 1.2 非目标
|
||||
|
||||
- 不追求复刻 CPU 的真实寄存器、指令流水线、页表与异常处理体系。
|
||||
- 不把所有 Markdown `[]()` 链接重新定义为动态指针。链接继续表示文档/符号寻址,不伪装运行时对象地址。
|
||||
- 不把 GFM 表格当成实际运行内存。表格可以用于数据/状态的可读表达与快照,但不是存储管理器。
|
||||
- 不要求标准库全部使用 `.mq.md`;原生加速可以有,但**不是实现一般算法的必要条件**。
|
||||
- 不在第一期承诺用户代码内存安全、数据竞争自由、GPU 加速或跨平台无差异系统调用。
|
||||
- 不支持在无权限环境中直接执行任意机器码或任意 syscall。
|
||||
|
||||
### 1.3 设计原则
|
||||
|
||||
**P1 语义最小而充分**:首先增加无法由现有表达式自然实现的基础机制,不先增加高级 API 或新关键词。
|
||||
**P2 高级能力在语言中生长**:新增算法/数据结构不修改解释器、不修改 `HostOp`、不需要重新编译 CLI。
|
||||
**P3 读写位置统一**:索引读取和索引赋值遵循同一类“位置”语义;`Memory` 的读取/写入是基础例子,普通列表的值语义不得无声改变。
|
||||
**P4 内外边界诚实**:内存操作是语言计算;操作系统 I/O 依赖权限、平台接口和底层实现。
|
||||
**P5 可读性**:普通 Markdown 文档仍然可读;`##`/`**`/`*`/列表/blockquote 的现有角色不因本 RFC 改变。
|
||||
**P6 安全与可验证性**:将可验证的保证限定在区域边界和权限边界;不对自定义分配器的块级错误作虚假保证。
|
||||
**P7 零隐式权限**:能够导入模块、持有字符串或解析文档,不代表拥有文件、网络或设备访问权。
|
||||
|
||||
## 2. 现有仓库事实及需要保留的设计
|
||||
|
||||
依据提交 `a113d894...` 的源代码和规范:
|
||||
|
||||
| 项目 | 现状 | 对本 RFC 的约束 |
|
||||
|---|---|---|
|
||||
| `marqdo-value` | `Unit/None/Bool/Int/Float/Text/List/Map/Table/Formula/Code/FunctionRef/PluginHandle`,没有普通可寻址 `Memory` 类型 | 应新增专门的存储引用值,而非把现有 List 改成默认共享对象 |
|
||||
| `marqdo-expression` | 索引 `a[i]`/成员 `a.x` 可读取;`assignment = identifier = expression` | 最小语法修改是赋值目标扩展,不是新 Markdown 节点 |
|
||||
| `marqdo-runtime` | `Frame.locals: BTreeMap<String, Value>`;`Assign` 以 `locals.insert` 重绑定 | 局部变量重新赋值应保持既有含义;共享可变性仅属于显式 `Memory` |
|
||||
| `marqdo-frontend` | Markdown AST 是语法角色唯一权威;GFM Table 惰性 | 不更改表格、链接的执行语义;表达式子语言可增补语义 |
|
||||
| `marqdo-ir` | 内置函数元数据固定且需更新规范 | 新增基础内存原语需要小型且审慎的语言变更;高层库不应无限扩展 Builtin 表 |
|
||||
| `marqdo-host` | 固定 `HostOp`,包括 Print、Math、Foreign、FsRead/FsWrite、PluginInvoke | 不应将每一个新的标准库函数继续塞入固定 HostOp |
|
||||
| 插件 ABI v1 | C ABI、worker、能力请求已有雏形 | 可复用函数注册与 Worker,不能错误地把 C ABI 当沙箱 |
|
||||
| `pure/project/strict` | `pure` 禁止外部 I/O grant;`strict` Linux 探测成功后只支持 L1 范围,其他平台 fail closed | 新系统资源功能首先在项目模式开发;strict 须经新的隔离验证,不得默许开启 |
|
||||
| 包管理 | path/Git 依赖与 lock v3 已实现;版本范围部分尚未强制 | 独立标准库可利用现有依赖体系,但兼容版本需要明确验证 |
|
||||
|
||||
**保留的价值**:现有整数运算、函数调用、条件和循环不必推翻;现有 AST 映射、资源边界和可追踪执行事件是 Marqdo 与脚本转译器的根本区别之一。
|
||||
|
||||
源码(固定版本):
|
||||
- [`spec/expression.ebnf`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/expression.ebnf)
|
||||
- [`spec/mapping.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/mapping.md)
|
||||
- [`spec/evaluation.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/evaluation.md)
|
||||
- [`crates/marqdo-value/src/value.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-value/src/value.rs)
|
||||
- [`crates/marqdo-runtime/src/interpreter.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-runtime/src/interpreter.rs)
|
||||
- [`spec/security.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)
|
||||
|
||||
## 3. 总体架构与唯一的语义边界
|
||||
|
||||
```text
|
||||
.mq.md 文档 GFM Table/Link/标题/正文仍按 Markdown 正常渲染
|
||||
│
|
||||
▼
|
||||
Canonical Markdown AST ──▶ 已有 Frontend ──▶ Marqdo IR
|
||||
│
|
||||
┌───────────────────┤
|
||||
▼ ▼
|
||||
值与控制 存储与位置
|
||||
Int/List/Map MemoryRegion
|
||||
函数/赋值/循环 字节读写/增长
|
||||
└─────────┬─────────┘
|
||||
│
|
||||
Marqdo 计算核心
|
||||
│
|
||||
需要访问外部世界时才穿越边界
|
||||
▼
|
||||
Resource capability gate
|
||||
│ │ │
|
||||
console file network/device
|
||||
│ │ │
|
||||
Platform Adapter / OS ABI
|
||||
```
|
||||
|
||||
逻辑上的执行状态可以表述为:
|
||||
|
||||
`State = (Control, Frames, ManagedValues, RegionStore, ResourceTable, Budget)`。
|
||||
|
||||
- `Control`:当前 IR 节点和执行顺序;保留已有控制流。
|
||||
- `Frames/ManagedValues`:现有 `Value`、局部变量和调用语义。
|
||||
- `RegionStore`:一组可增长字节区域,每个区域按 `(region identity, offset)` 寻址。
|
||||
- `ResourceTable`:操作系统资源的不透明句柄与授权信息,**不得与内存地址混用**。
|
||||
- `Budget`:fuel、调用深度、内存限额、I/O 限额、截止时间等。
|
||||
|
||||
此设计**不规定**必须先创建自定义字节码 VM。现有树遍历解释器可直接执行新内存语义;将来引入字节码、JIT、LLVM/Cranelift 等属于可选执行方式,必须保持可观察语义。
|
||||
|
||||
## 4. 三种基础领域:值、内存、资源
|
||||
|
||||
### 4.1 托管值:维持已有语义
|
||||
|
||||
`Int/Text/List/Map/Table/Formula/Code` 等继续是运行时管理的值。普通 `b = a` 不得偷偷将普通列表变成共享可变指针。现有对象可以由 Rust 分配内存——**实现语言不等于语言语义**。这个领域负责易用性、文档处理、不可变数据等。
|
||||
|
||||
### 4.2 Memory:拥有身份的字节区域
|
||||
|
||||
新增 `Value::Memory`(名称暂定),含不可伪造的运行时区域引用。逻辑区域内部是一串字节,下标为非负整数;不同区域即使下标相同也不是同一个位置。
|
||||
|
||||
- **区域身份**:由本次运行中的内存实例签发;`Memory` 复制/传参时保留指向相同区域的身份(共享,不复制所有字节)。
|
||||
- **地址**:`(MemoryHandle, offset)`;`offset` 只在所属区域内有意义;整数本身不是获得其他区域权限的凭据。
|
||||
- **内容**:每个元素是 `0..255` 的一个字节;初始字节为 0。
|
||||
- **大小**:区域长度是页大小的整数倍;建议初期采用 `64 KiB` 页,以简化语义与工具链对照,不要求底层真的按页分配。
|
||||
- **增长**:只增长,不缩小;旧地址偏移不变;新加入的字节全部为零;不得突破区域/运行全局限额。
|
||||
- **读取**:`m[i] -> Int(0..255)`;越界、非法整数或已失效区域均是类型/存储错误。
|
||||
- **写入**:`m[i] = b`,`b` 必须是 0..255 的整数,不隐式截断;越界拒绝。
|
||||
- **相等性**:两个 Memory 引用的相等比较区域身份;不比较全部字节;Memory 不定义大小排序。
|
||||
- **序列化**:不允许将活动 `Memory` 句柄伪装成数值或跨进程永久有效地址。允许显式**复制字节内容**为快照,需限额;跨进程传输需要显式复制或另立经过授权的共享内存协议。
|
||||
- **生命周期**:第一版只在单次执行内存在;由 Runtime 保持活动句柄可达性,引用消失后可回收,至少在运行结束时回收全部区域。**不增加用户可见的 `free(region)`**,以免引入悬空区域句柄。用户的 `malloc/free` 管理的是**区域内部的块**。
|
||||
- **内存安全范围**:Runtime 保证单次操作不访问该 Memory 区域之外;不保证用户自编分配器不会越块、重复 free、释放后复用错误。
|
||||
|
||||
此设计不要求新增独立的 `Pointer` 原始类型。**首版指针可由 `Memory` 引用和 `offset` 组合表达**;是否将这一对优化为一等 `Pointer`,由实际性能与调用需求决定,而非现在先引入语法。
|
||||
|
||||
### 4.3 Resource:与外部世界交互的独立对象
|
||||
|
||||
`ResourceHandle` 是由可信平台/Provider 签发的不透明句柄,代表文件、控制台流、网络连接或专门设备;含授权访问模式与生命周期。**它不是 `Memory`,不能索引成任意 RAM,也不能通过伪造整数创建。**
|
||||
|
||||
其常见操作是 `open/read/write/close`,但具体合法操作取决于资源接口。操作系统错误、EOF、部分读写、超时、取消都需要保留独立结果和错误语义。不得假定一次 `read` 总能填满缓冲区、一次 `write` 总能输出所有字节。
|
||||
|
||||
### 4.4 为什么要划分,而不是单一“万能地址空间”
|
||||
|
||||
`Memory` 代表运行时可管理的存储;`Resource` 代表外部效果。普通内存读写是局部状态改变,终端输入/网络读取可能随外部世界变化。把二者统一为可以随意解引用的指针,会隐藏权限和不可重复性,也会使知识证明与回放语义失真。
|
||||
|
||||
## 5. 可编程动态内存:到底哪些必须内置
|
||||
|
||||
### 5.1 最小计算内核操作
|
||||
|
||||
这里定义**能力**,不冻结最终函数名:
|
||||
|
||||
| 基本能力 | 候选用户接口 | 运行时职责 |
|
||||
|---|---|---|
|
||||
| 创建区域 | `memory.new(initial_pages, max_pages)` | 签发不可伪造引用,检查预算并分配零初始化区域 |
|
||||
| 查询大小 | `memory.size(m)` | 返回页数,不泄露真实宿主指针 |
|
||||
| 增长区域 | `memory.grow(m, pages)` | 原有字节与偏移不变,新字节为零;成功返回原页数,失败返回可区分错误 |
|
||||
| 读取一字节 | `m[i]` | 边界检查,读取 `0..255` |
|
||||
| 写入一字节 | `m[i] = b` | 边界和值域检查,更新同一 Memory 区域 |
|
||||
|
||||
`memory.new/size/grow` 是操作而非关键词;`m[i]` 利用当前已有的读取表达式,`m[i] = b` 只扩展现有赋值目标语义。这是此 RFC 要求的**唯一用户可见表达式语法增量**。
|
||||
|
||||
`len(m)` 可在另一个 ADR 中统一成字节长度而非另设 `memory.size`;这里暂不混淆“页数”和“字节数”。是否以页或字节为 `grow` 单位也应通过 ADR 固定,本文的原型推荐页。
|
||||
|
||||
### 5.2 位置赋值(Place)如何实现
|
||||
|
||||
对左值 `x` 与 `m[i]`,**赋值含义不同但遵循一个语法规则**:
|
||||
|
||||
- `x = value`:仍重新绑定当前函数帧局部变量,现有语义不变。
|
||||
- `m[i] = byte`:先按既有从左到右顺序求值 `m` 与 `i`,检查目标为 `Memory` 并检查偏移和值,然后更新同一内存实例。
|
||||
- `list[i] = value`:**不在第一个阶段承诺**。是否将列表修改为值更新/可变对象更新,要另行定义,不能顺便改变现有列表复制与索引语义。
|
||||
- `obj.field = value`:首版也不必实现。只有在明确记录值/对象语义后再拓展。
|
||||
|
||||
表达式编译结构的最小增量是 `Assign { target: PlaceExpr, value: Expr }`,其中 `PlaceExpr` 初期只有 `Local(name)`、`Index(base,index)`;索引目标仅接受 Memory,其他索引仍只读。无需新 Markdown 节点。
|
||||
|
||||
### 5.3 自定义指针与分配器
|
||||
|
||||
`malloc(n)` 在 `.mq.md` 标准库里持有一个区域 `m` 和元数据指针 `next`,向调用方返回区域内偏移(或 `{region:m, offset:p}` 组合值)。`free` 仅将已分配块放回空闲链表;若空间不足,调用 `memory.grow`。对齐、大小、空闲块链表、合并策略都由 `.mq.md` 编写,**不要求 Rust 为每一种分配算法做特殊 HostOp**。
|
||||
|
||||
例:块格式 `[size:u32][next:u32][flags:u8][payload...]`。首版只支持按字节读取,`u32` 的编码/解码可以使用四次索引、`//` 和 `%` 在 `.mq.md` 中实现。若该路径确实过慢,后来可增加 `load_u32` 等**语义等价、可验证的加速实现**,不能因此把完整分配器写到 Host。
|
||||
|
||||
分配器可建立:Arena、first-fit、buddy、slab、链表/图、字符串字节编码、编译器符号表、虚拟 CPU 的寄存器文件等。**内存块的逻辑安全由标准库分配器定义**;调用者拿到任意偏移后仍有越块写的能力,这属于明确的低层风险。
|
||||
|
||||
### 5.4 内存模型形式定义(草案)
|
||||
|
||||
令 `R` 是区域身份集合,`M[r]` 是 r 区域的字节序列,`len(M[r])` 是其有效长度。
|
||||
|
||||
- `read(r,i) = M[r][i]`,前提 `0 ≤ i < len(M[r])`。
|
||||
- `write(r,i,b)` 将 `M[r][i]` 更新为 `b`,前提 `0 ≤ b < 256` 且 `i` 合法;其余字节不变。
|
||||
- `grow(r,k)` 要求 `k ≥ 0`,若预算允许,尾部追加 `k × PAGE` 个零字节,旧位置的字节值不变;否则返回显式失败且区域不变。
|
||||
- 不同区域互不别名:`r1 != r2` 意味着修改 `M[r1]` 不影响 `M[r2]`。
|
||||
- `m2 = m1` 若 m1 为 Memory,则二者指向同一 `r`;若为普通 `List`,原有值语义不变。
|
||||
|
||||
实现必须防止巨大 BigInt 偏移到 `usize` 转换时溢出、`offset + length` 整数溢出、预算计算溢出。任何错误不得 panic、越界读写或产生非确定性内存破坏。
|
||||
|
||||
## 6. 非线性存储、寄存器与加速的恰当位置
|
||||
|
||||
### 6.1 非线性内存不需要独立关键字
|
||||
|
||||
- **逻辑非线性结构**(链表、树、图):在一个或多个字节区域中存储偏移/引用即可;不要求新类型。
|
||||
- **物理非连续内存**:运行时可以用分页、稀疏映射或 Vec 实现同一线性语义;用户无需知道物理页布局。
|
||||
- **多设备内存**:未来可能带有设备特定权限与同步约束;不要将 CPU 内存随意解释成 GPU/QPU 地址。优先经专用受控资源/后端接入。
|
||||
|
||||
### 6.2 不加入真实 CPU 寄存器接口
|
||||
|
||||
当前局部变量与表达式临时值已经构成“虚拟寄存器”意义上的计算槽。编译到机器码时,可以由后端分配硬件寄存器。用户要模拟寄存器堆,可以用 Memory 自己实现;特殊硬件寄存器则是设备 I/O,而不是普通变量。
|
||||
|
||||
### 6.3 并行和原子操作
|
||||
|
||||
本 RFC 的内存规则初始定义为**单线程或按明确顺序执行**。绝不能声称 `m[i] = m[i]+1` 在多线程之间自动原子。未来若允许多个执行任务共享 Memory,必须另行设计同步/原子操作、数据竞争行为和访问序;也可以先采用无共享可变内存的任务隔离。新增原语应依据不可由当前操作正确组合而来的能力,而非先堆锁/屏障关键字。
|
||||
|
||||
### 6.4 GPU/QPU 不由普通 Memory 自动获得
|
||||
|
||||
Memory 支持字节缓冲区与数据布局,但 GPU 计算仍需要执行后端与数据传输,真实量子资源则不是可随意读取的字节地址。Marqdo 可用独立后端/服务处理硬件特有语义,内核应保持硬件无关。
|
||||
|
||||
## 7. 系统交互:能否不依赖 Rust/Python 实现高级功能?
|
||||
|
||||
### 7.1 必须给出的诚实答案
|
||||
|
||||
**可以**:大量系统功能由 Marqdo 标准库直接编写,包括文本与字节编码、目录遍历算法、协议解析、HTTP 报文、部分网络协议逻辑、缓存、日志格式、CLI 交互逻辑、文件格式读写、数据库协议客户端中的高层逻辑。程序运行时无需执行 Python 或 JavaScript,也不需要为每个高级库函数增加 Rust 分支。
|
||||
|
||||
**不能无条件做到**:如果当前运行方式是一个 Rust 编写的解释器,那么跨越用户态与操作系统的**最后一步**必须由解释器内建的系统适配器、已授权插件/worker、或其他可信底层机器代码执行。Markdown 源码不能在无相应运行环境、系统调用 ABI 和权限的情况下直接控制硬件。即使将来 Marqdo 编译成独立原生二进制,它也仍通过操作系统 ABI 或设备驱动与外界交互,只是**不再需要 Rust 解释器常驻运行**。
|
||||
|
||||
**不可混淆的四种独立性:**
|
||||
|
||||
| 层面 | 能否达到 | 含义 |
|
||||
|---|---|---|
|
||||
| A. 算法独立 | **可以,近期目标** | 排序、内存分配、解析器等算法均在 `.mq.md` 中 |
|
||||
| B. 标准库独立 | **可以,近期目标** | 新标准库函数不修改 Rust Runtime/HostOp,单独版本发布 |
|
||||
| C. 系统接口独立 | **有条件可以** | 已有的 OS 原语/权限可组合构造新功能;新增设备/OS 原语需相应适配器,但不改语言核心 |
|
||||
| D. 运行时/宿主实现独立 | **中长期目标** | 独立编译成机器码/Wasm、或自举解释器,不依赖 Rust 解释器运行;仍依赖 OS ABI/硬件 |
|
||||
|
||||
**这里的“无需修改宿主”不是“无底层实现”。** 应将“高级逻辑用 Marqdo 实现”与“最后一跳由可信 OS adapter 执行”区分清楚。
|
||||
|
||||
### 7.2 最小系统资源操作模型
|
||||
|
||||
建议**不增加任何 I/O 关键字**,而以可注册、版本化的 `Resource` 接口提供 OS 能力。候选调用形式(未实施):
|
||||
|
||||
- `resource.open(kind, descriptor, rights) -> ResourceHandle`
|
||||
- `resource.read(handle, memory, offset, max_bytes) -> count | EOF/error`
|
||||
- `resource.write(handle, memory, offset, bytes) -> count | error`
|
||||
- `resource.close(handle) -> Result`
|
||||
|
||||
其中:
|
||||
|
||||
1. `kind` 例如 `stdio`、`file`、`socket`,必须由可信平台适配器或已安装的 Provider 明确注册,**不能因为传入任意字符串就自动获得能力**。
|
||||
2. `descriptor` 是声明请求的结构化参数,例如项目内文件名、网络目标,必须验证并授权;绝非任意可解引用真实指针。
|
||||
3. `rights` 是请求的权能;实际授权为“请求 ∩ 操作者 grant ∩ Provider 支持”,拒绝必须显式报错。
|
||||
4. `read/write` 通过 Memory 区域交换原始字节,边界检查和缓冲区借用由 Runtime/Adapter 保证;不得允许 I/O Adapter 长期保存未固定的宿主 Rust 指针。
|
||||
5. 区分 EOF/0 字节暂时无进展/取消/系统错误;支持部分读写,标准库负责重试与缓冲。
|
||||
6. `close` 撤销对应资源句柄;已关闭句柄、重用 ID 必须借助 generation/注册表检查,不能错误访问新资源。
|
||||
7. 对设备特殊操作,不强行滥用 `read/write`:必要时可由具体 Provider 暴露专门的**版本化、授权操作**;不在通用内核中新增一个无条件允许任意 syscall 的 `control` 后门。
|
||||
|
||||
`resource.*` 属于**平台接口层**而不是纯计算内核;即使调用看起来是普通函数,执行前必须检查权限。现有 Host 能力检查可成为迁移入口,但不应为每个新高层标准库函数创建新的 `HostOp` 变体。
|
||||
|
||||
### 7.3 应该怎样真正避免“每扩展一次就修改 Host”?
|
||||
|
||||
将目前的固定分派进一步分成:
|
||||
|
||||
```text
|
||||
普通 Marqdo 标准库函数
|
||||
├── 调用 Marqdo 语言计算原语(无需 Host)
|
||||
└── 仅在外部交互时使用稳定的 Resource ABI
|
||||
│
|
||||
Capability Gate
|
||||
│
|
||||
Provider/Platform Registry
|
||||
│
|
||||
┌────────────────┴───────────────┐
|
||||
▼ ▼
|
||||
标准平台适配器 独立第三方 Provider
|
||||
文件/终端/网络 新设备/专用系统能力
|
||||
```
|
||||
|
||||
关键是**一次性稳定可扩展的外部操作契约**,而不是一张不断增长的 Host 功能目录。
|
||||
|
||||
现有 C ABI v1、worker 与 `CallableRegistry` 应先尽量复用;新接口可在不破坏 v1 的条件下增补版本化注册信息。如需升级,必须有兼容矩阵与迁移机制。**Provider 负责它新增的原始能力;`.mq.md` 标准库负责高级功能。**
|
||||
|
||||
### 7.4 两种系统访问路线及取舍
|
||||
|
||||
| 路线 | 优点 | 风险/限制 | 建议 |
|
||||
|---|---|---|---|
|
||||
| 能力化 `Resource` ABI | 统一权限与跨平台行为;适合安全分层 | 新的真正 OS 原语仍需要 Adapter/Provider | **默认且必须支持** |
|
||||
| 直接的原生 syscall/FFI 边界 | 能较快覆盖大量平台特性 | 平台/ABI 依赖、地址搬运、权限绕过、维护和可验证性问题 | 仅可信、本机、显式授权的可选高级接口;默认 pure/strict 禁止 |
|
||||
|
||||
在 Linux 等平台,即使“直接 syscall”,也要处理目标架构调用约定、数字差异、结构体布局、指针与长度、errno 以及 OS 策略;Windows/macOS 并不提供与 Linux 一模一样的 syscall ABI。不得将其伪装成普适的语言核心。
|
||||
|
||||
**无悖论地满足用户目标的办法**:向 Marqdo 暴露一个小而稳定的**操作系统能力边界**,然后让标准库以 Marqdo 自己的程序逻辑组合出系统高级功能。这是可实现的强独立性;完全消灭系统边界不是现实目标。
|
||||
|
||||
### 7.5 端到端例子 A:完全用 Marqdo 构建交互式 CLI
|
||||
|
||||
**先决条件**:可信平台适配器提供 stdin/stdout 的已授权资源,Marqdo 运行时支持 Memory 操作。
|
||||
|
||||
1. 标准库 `io.read_line` 在 `.mq.md` 中申请/增长缓冲区,循环调用资源 `read`,查找换行字节。
|
||||
2. 标准库 `text.utf8_decode` 用 `.mq.md` 处理 UTF-8(正确处理非法序列;编码策略可配置)。
|
||||
3. 用户程序将文本解析为数字,用现有 `Int` 算术和条件循环处理。
|
||||
4. 标准库 `io.write_text` 使用 `.mq.md` 的 UTF-8 编码,再经 `write` 输出字节。
|
||||
|
||||
**全流程无需 Python/Node,也不需要 Host 特地实现 `input()`。** 然而最底层 stdin/stdout 仍由 OS 适配器调用操作系统接口。
|
||||
|
||||
### 7.6 端到端例子 B:新增 HTTP 客户端能力
|
||||
|
||||
**先决条件**:系统适配器已经提供经授权的网络连接资源、DNS/地址解析的可用机制,以及必要的加密能力或可插拔 TLS Provider。
|
||||
|
||||
1. 标准库构造 HTTP 请求行、首部和消息体的字节序列。
|
||||
2. 使用 `resource.write` 处理部分发送;使用 `resource.read` 实现缓冲接收。
|
||||
3. `.mq.md` 解析状态行、首部、chunked 编码等;实现重试、缓存、超时逻辑。
|
||||
4. 测试 HTTPS 时,必须使用可靠 TLS 实现(可由已审计 Provider 提供;“不依赖外部语言写协议逻辑”不意味着应该自己手写未经验证的 TLS)。
|
||||
|
||||
网络的“最后一跳”必须存在,且不同平台的 TCP/TLS 能力可能需要不同 Provider;不能只靠 Memory 推导出 socket。
|
||||
|
||||
### 7.7 端到端例子 C:科研加速与 QPU
|
||||
|
||||
- 经典数值模拟:`.mq.md` 可以在 Memory 中构造矩阵、压缩存储、数据布局算法,后续由独立计算后端加速。
|
||||
- QPU:量子设备是具有权限、调度与门集约束的资源,不是普通 Memory;通过专用 Provider/Backend 提交计算、接收结果。
|
||||
- 这种接口可保持标准库和上层代理/知识文档不绑定 Rust 宿主,但并不能保证算法自动获得 GPU/QPU 性能或精确的量子物理语义。
|
||||
|
||||
## 8. 安全、权限与错误模型
|
||||
|
||||
| 保护目标 | 首版保证 | 明确不能保证 |
|
||||
|---|---|---|
|
||||
| 虚拟 Memory | 区域越界、非法索引、增长限额、无效句柄失败 | 区域内 malloc/free 的块级内存安全 |
|
||||
| 不透明 Resource | 授权身份、访问模式、资源关闭和 generation 检查 | 平台适配器/不可信原生插件本身绝对安全 |
|
||||
| pure | 内部算术、内存与受预算控制的逻辑计算;不获得外部 I/O | 允许直接 syscall 或任意外部访问 |
|
||||
| project | 对批准的文件/终端/网络资源进行操作,使用能力化 API | 等价于 OS 级进程沙箱 |
|
||||
| strict | 维持现有 fail-closed 策略;新 I/O Provider 必须另行经过 OS 级安全验收 | 当前版本自动支持所有外部能力 |
|
||||
|
||||
进一步要求:
|
||||
|
||||
- 全局/单区域内存预算、增长次数与 fuel 限额、操作可取消性;内存失败不得半成功。
|
||||
- 路径资源应使用受控根路径和可验证的 OS 打开策略,避免字符串检查后再打开造成 TOCTOU;网络权限需要处理名称解析/重绑定等问题。
|
||||
- 插件/worker 不因拥有 API 名字自动获得权限;原生进程内插件不能被当作不可信沙盒。
|
||||
- Memory/Resource 活动句柄不应直接跨线协议序列化;跨边界传输使用字节复制、快照或显式授权的资源传递协议。
|
||||
- 诊断保持来源 Markdown 节点位置;可选追踪读取/写入的逻辑区域、偏移和长度,不能将外部文件秘密或原始敏感字节无条件写日志。
|
||||
|
||||
## 9. 与当前 Marqdo 语法和知识编译的结合
|
||||
|
||||
这一方案故意**只修改最少的表达式语义**。
|
||||
|
||||
- `##` 函数、`**` 执行、`*` 返回、Markdown 列表与引用规则继续维持规范。
|
||||
- `[]()` 是**文档/知识的静态寻址**;动态 Memory 的字节偏移属于执行期位置,两者不强行共享 URI 语法。
|
||||
- 普通 GFM 表格是文档值;可用于初始化/展示内存快照,但不会自动变成内存分配器。
|
||||
- `memory.new(...)` 与 `resource.open(...)` 都可按普通函数调用表达;不新增 `memory`、`pointer`、`asm`、`register` 这样的语法关键字。
|
||||
- 内存状态变化可选输出可读 GFM Table 供 View/Proof 使用;证明必须以抽象执行语义为依据,而不是假设表格文本本身就是执行存储。
|
||||
|
||||
示例(**候选语法,当前尚不可运行**):
|
||||
|
||||
````markdown
|
||||
## 研究一个字节缓冲区
|
||||
|
||||
此函数演示“可引用的存储区域”与普通 Markdown 叙述并存。
|
||||
|
||||
**区域 = memory.new(1, 4)**
|
||||
|
||||
**`区域[8] = 42`**
|
||||
|
||||
**另一个名字 = 区域**
|
||||
|
||||
**`另一个名字[9] = 99`**
|
||||
|
||||
*区域[8] + 区域[9]*
|
||||
````
|
||||
|
||||
预期结果 141。注意这里 `另一个名字 = 区域` 分享 Memory 身份;一般列表/表格并不因此被改造成共享字节缓冲区。
|
||||
|
||||
## 10. 可行性和备选方案评定
|
||||
|
||||
| 设计 | 语言自主性 | 高层开发便利性 | 性能潜力 | 安全/复杂度 | 结论 |
|
||||
|---|---|---|---|---|---|
|
||||
| 继续只扩充 HostOp | 低 | 表面高 | 取决于 Rust 实现 | Host 随功能膨胀 | 否决为主要路线 |
|
||||
| 只有托管对象堆/GC | 中 | 高 | 对一般数据不错 | 需要身份/生命周期语义 | 保留现有托管值,但不足以满足低层目标 |
|
||||
| **多区域字节内存 + 托管值 + Resource** | **高** | **高** | 后端可优化,字节操作起初较慢 | 明确的区域边界;块级风险可见 | **推荐** |
|
||||
| 任意裸指针 + 直接机器码 | 极低平台独立性 | 低 | 最高上限 | 高风险、难验证 | 只可选的可信原生扩展 |
|
||||
| 纯 GFM 表格作堆内存 | 低至中 | 对展示很高 | 非常低 | 动态状态/身份混乱 | 否决 |
|
||||
|
||||
该推荐不是宣称“最优”已经被数学证明,而是在现有语法、实现成本、扩展性与安全边界下的**工程选择**。
|
||||
|
||||
## 11. CLI/Runtime/标准库仓库的具体职责
|
||||
|
||||
### 11.1 `marqdo-cli`:保留小而稳定的内核
|
||||
|
||||
- `marqdo-value`:新增不可伪造的 Memory 值表示;必要时添加 ResourceHandle,注意现有 PluginHandle 不能随意直接复用为 OS 资源句柄。
|
||||
- `marqdo-expression`:`AssignTarget` 的 Local / Index 变体;保留原有表达式读取。
|
||||
- `marqdo-frontend`:更新表达式 AST 对应 IR 和定位信息;不改变 Markdown 语义映射。
|
||||
- `marqdo-runtime`:Memory 实例生命周期/增长/边界检查/预算;执行位置写入;参考解释器。
|
||||
- `marqdo-host`:逐步提供可注册资源操作的能力边界;必须保持旧 `HostOp` 兼容窗口,不能一次砍断所有旧调用。
|
||||
- `marqdo-package` / CLI:发布标准库兼容信息,继续利用已存在的 path/Git pinned lock;资源授权仍由真实操作者提供。
|
||||
|
||||
### 11.2 独立 `marqdo-stdlib`:以 Marqdo 实现通用功能
|
||||
|
||||
建议独立仓库 `cflmy/marqdo-stdlib`,首期目录:
|
||||
|
||||
```text
|
||||
marqdo-stdlib/
|
||||
core/ # 字节编码、整数工具、可恢复错误封装
|
||||
memory/ # arena, allocator, free-list, binary codecs
|
||||
collections/ # list/tree/graph/hashmap 等数据结构算法
|
||||
text/ # UTF-8, parsing
|
||||
io/ # buffered readers/writers, read_line
|
||||
data/ # csv, json, 自定义格式
|
||||
tests/ # 以 marqdo run/call 执行的标准库回归程序
|
||||
```
|
||||
|
||||
高层函数应在 `.mq.md` 里,按普通模块导入。发行版携带一个经测试的 stdlib 版本,支持离线运行;单独发布更新时,要求明确的 Runtime 兼容范围与锁定。标准库可有原生加速 Provider,但不能把它作为常规功能的唯一实现路径。
|
||||
|
||||
### 11.3 独立 Platform Providers
|
||||
|
||||
系统 Adapter 可以由 Rust、C、汇编或将来的 Marqdo 原生编译代码实现。当前 CLI 不应为普通库函数增加 HostOp;对真正的新 OS 原语,若标准 Provider 不支持,用户必须安装/编写扩展 Provider,并获得相关权限。**不是每个新设备都能只靠修改 `.mq.md` 获得访问能力。**
|
||||
|
||||
## 12. 分阶段实施(范围严格受控)
|
||||
|
||||
### R0:冻结设计与当前基线(无新功能)
|
||||
|
||||
- 此 RFC 和对应 ADR 经审阅,确认 `Memory` 值语义、`index assignment`、错误类别、线协议边界及安全模式。
|
||||
- 修复已有跨平台 CI/安全收敛问题;固定解释器行为的回归样例。
|
||||
- 建立一页对照表:新增语义是否需要 `HostOp`、新关键字、新 Markdown 节点。首版目标均为“不需要”。
|
||||
|
||||
### R1:最小可寻址存储(核心里程碑)
|
||||
|
||||
- `Memory` 区域:创建、大小、增长、读取一字节、写入一字节;实现总/单区域预算、错误恢复与 trace。
|
||||
- 支持 `**\`m[8] = 42\`**` 这类现有 Markdown 负载中的索引赋值。
|
||||
- 不开放裸指针、不引入新寄存器、不改旧 List/Map/Table 的赋值与复制行为。
|
||||
- 运行跨区域隔离与别名测试;报错不能导致原有区域被污染。
|
||||
|
||||
### R2:Marqdo 自举的内存与算法库
|
||||
|
||||
- `.mq.md` 顺序分配器;随后 free-list 可复用分配器。
|
||||
- 不修改 Runtime/Host 编写并执行:可增长链表、二叉树、简单虚拟 CPU、UTF-8/二进制解析器。
|
||||
- 提供内存利用率、分配吞吐、碎片率、执行时间及解释器 fuel 的测量;避免只用“功能跑通”证明成熟度。
|
||||
|
||||
### R3:最小系统资源闭环
|
||||
|
||||
- 在项目模式先实现 `stdio` 与受控项目文件的 `open/read/write/close`;**功能可用性不代表 strict 安全已通过**。
|
||||
- `.mq.md` 标准库实现 `read_line`、`write_text`、文件二进制复制与 UTF-8 编码。
|
||||
- 资源注册层允许 Provider 扩展;验证一个第三方资源端点不修改解释器编译即可接入(是否需要实现新的系统适配器需明示)。
|
||||
- 对当前 `HostOp` 逐步适配,保留兼容期,不能强行把所有第三方都迁移到新的不成熟 ABI。
|
||||
|
||||
### R4:可选增强(不阻断基础版本)
|
||||
|
||||
- 索引/字段赋值的一般化、字节多宽度原语及位运算、快照、增量编译、优化执行器。
|
||||
- 需要共享内存并行时,再设计原子/同步与数据竞争语义。
|
||||
- 独立网络/TLS、设备、GPU、QPU 后端与原生编译目标。
|
||||
|
||||
**必须遵循的顺序**:R1 完整正确 → R2 验证计算自主性 → R3 证明外部交互可组合;不先把 R4 的设备能力和并发类型系统塞入 CLI。
|
||||
|
||||
## 13. 端到端验收清单
|
||||
|
||||
| 编号 | 验收项目 | 判定标准 |
|
||||
|---|---|---|
|
||||
| T1 | 语义一致性 | 旧的局部赋值与 List/Map/Table 行为完全不变,旧 Markdown AST golden 通过 |
|
||||
| T2 | Memory 隔离 | `m1[0]=x` 不影响 `m2[0]`;`m2=m1` 后写入可相互观察 |
|
||||
| T3 | 增长与错误 | 成功增长保留旧字节且新字节为 0;负索引、越界、超限增长、巨大 BigInt 均明确失败且不破坏状态 |
|
||||
| T4 | 自建分配器 | `.mq.md` 实现 malloc/free,释放块重复利用;高低地址和碎片测试通过 |
|
||||
| T5 | 高级数据结构 | 不改 Rust 核心,仅新增 `.mq.md` 实现可增长链表、树、二进制协议解析器 |
|
||||
| T6 | 标准库独立升级 | 旧 CLI 二进制保持不变,安装一个与之兼容的新 `.mq.md` 库并成功使用 |
|
||||
| T7 | 终端闭环 | `marqdo run` 仅使用标准库加基础 OS Adapter,完成多轮输入/输出,无 Python/Node 调用 |
|
||||
| T8 | 系统资源授权 | 无 grant 无法 open;已关闭/伪造 handle 无法访问;项目路径无法越权;所有拒绝有诊断和必要审计 |
|
||||
| T9 | 安全模式 | `pure` 不允许外部 I/O;当前 `strict` 不自动放开新 Provider;真实隔离与失败路径有测试 |
|
||||
| T10 | 扩展接口 | 新高层 I/O/协议函数不增加任何 HostOp;新 Provider 通过约定的独立注册/版本机制接入 |
|
||||
| T11 | 兼容/性能 | Windows/macOS/Linux 的支持矩阵真实披露;可定量报告字节访问吞吐、内存增长和 Python/Rust baseline 差距 |
|
||||
| T12 | 文档与证明 | Memory 状态可导出可读快照;执行状态引用源 Markdown span;不将数据表格误认成内存本体 |
|
||||
|
||||
## 14. 需要 ADR 决断而不是隐含假设的开放问题
|
||||
|
||||
本 RFC 采用一条**可实施的推荐路线**,但下列决策会改变公开语义,须单独冻结:
|
||||
|
||||
1. **页语义**:推荐 64 KiB 页、只增长;是否与外部目标标准保持一致,预算如何计入已保留与已提交物理内存。
|
||||
2. **Memory 的值/调用传递**:推荐复制引用保持身份,区域可达则存活;禁止隐式跨线协议序列化。
|
||||
3. **Memory 引用寿命**:推荐首版只在一次执行期间有效,采用自动回收或至少结束时回收;内部块分配是 stdlib 职责。
|
||||
4. **Place AST**:推荐首版只增 Local + Memory Index;其他集合的索引赋值须另行评议。
|
||||
5. **OS Resource 最小 ABI**:是否先使用现有插件注册机制包装,而不立即新增第二套并行插件协议。
|
||||
6. **跨平台支持范围**:纯计算 API 保持统一;系统资源能力按平台明示 availability,不能假定所有设备/系统调用均存在。
|
||||
7. **实验数据**:R2 执行效率可能较低,是否加入 32/64 位宽读写及优化编译由实测结果决定,而非预设几十种关键字。
|
||||
|
||||
## 15. 最终建议与决策摘要
|
||||
|
||||
**建议通过的架构选择:**
|
||||
|
||||
- **语言核心**:保留现有托管值与控制流,新增**显式、多区域、可增长的字节 Memory**;用现有索引表达式扩展可写位置,不创建新的 Markdown 语法层。
|
||||
- **内存管理能力**:`Memory` 区域由 Runtime 提供并保护区域边界;`malloc/free`、块布局、链表、树、二进制编码等由 `.mq.md` 标准库自己构造。
|
||||
- **系统能力**:独立、授权、可替换的 `Resource` 边界;标准库在 Marqdo 中组合 I/O,OS 的最后一跳仍由可信本地 Adapter/Provider/编译后端承担。
|
||||
- **扩展性**:不为每个库函数增加 Rust `HostOp`;只有新增*真正不可由现有原语表达*的硬件/OS 能力,才需要扩展适配器。
|
||||
- **安全性**:区分区域边界安全、块级内存安全、进程隔离和 OS 权限;不能用“沙盒”一词掩盖缺少的保证。
|
||||
- **长期演进**:以此作为实现自主数据结构、字节码 VM、优化执行器与量子/科学计算后端的基础,但不许声称三个内存原语就能自动产生 GPU/QPU 加速。
|
||||
|
||||
**核心结论:** 如果“摆脱宿主语言依赖”指的是**程序逻辑、标准库算法和高级系统功能不依赖宿主语言来实现**,这条路线可以达成;如果指的是**无需任何底层可信机器代码和操作系统接口即可访问硬件**,则不可能。对真正的独立编程语言而言,恰当目标是**自己定义计算、自己构建高层库、清楚且可扩展地跨越不可消除的系统边界**。
|
||||
|
||||
---
|
||||
|
||||
## 参考资料(外部设计依据,不是 Marqdo 已实现功能的证据)
|
||||
|
||||
1. [WebAssembly Core Specification 3.0(2026-09-11)](https://www.w3.org/TR/wasm-core-2/):线性内存、内存增长、抽象机器、核心与环境 API 的分离;注意其保证为线性内存边界安全,不保证客户程序自己设计的内部分配器没有逻辑错误。
|
||||
2. [Rust `core` / `alloc` / `std` 文档](https://doc.rust-lang.org/std/):计算基础、分配与平台库的分层思路;并不意味着必须用 Rust 编写 Marqdo 的标准库算法。
|
||||
3. [Linux Kernel ABI documentation](https://docs.kernel.org/admin-guide/abi.html):OS 用户态接口的稳定性、适用范围和版本边界。
|
||||
4. [Linux Kernel ioctl interfaces](https://docs.kernel.org/driver-api/ioctl.html):设备特定能力并不都能归约为普通 RAM 读写,ABI 应明确数据布局与操作语义。
|
||||
5. [Marqdo 当前安全模型](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)及[插件 ABI v1](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/plugin-abi.md)。
|
||||
+173
-12
@@ -40,12 +40,12 @@ byte-identical).
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
(47 test binaries, 0 failed; includes 7/7 Linux security tests)
|
||||
430 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
```
|
||||
|
||||
### G-C security lane (disposable Ubuntu 26.04 VM, ordinary user `dev`)
|
||||
|
||||
Source: `git archive 8af7262` extracted in the VM (not the dev container).
|
||||
Source: `git archive 70d916d` extracted in the VM (not the dev container).
|
||||
Host: QEMU/TCG, kernel `7.0.0-43-generic`, `systemd-detect-virt --container`
|
||||
= `none`, bubblewrap 0.11.1 with `bwrap-userns-restrict`, uid 1000.
|
||||
|
||||
@@ -59,7 +59,8 @@ Host: QEMU/TCG, kernel `7.0.0-43-generic`, `systemd-detect-virt --container`
|
||||
cargo test --workspace --locked (same archive): 47 ok, 0 failed
|
||||
```
|
||||
|
||||
Logs: [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md).
|
||||
Logs: [`evidence/2026-11-20-strict-resource-cm-a11.log`](evidence/2026-11-20-strict-resource-cm-a11.log)
|
||||
and [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md).
|
||||
This is still a **local VM lane**, not a self-hosted GitHub Actions security URL.
|
||||
|
||||
## 2. Work package status
|
||||
@@ -71,12 +72,12 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
| 2 | Cross-platform strict `cfg` | **Fixed locally; native CI UNVERIFIED** | `#[cfg]` dispatch + non-Linux regression test |
|
||||
| 3 | Strict probe→launch race | **Fixed locally; GitHub Linux CI UNVERIFIED** | fail-closed CLI contract + probe-OK/launch-EAGAIN seam test |
|
||||
| 4 | Parallel security matrix isolation | **Fixed** | unique paths + helper `OnceLock`; 7/7 in parallel workspace run |
|
||||
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 44 semantic, 52 compile-negative, 12 runtime-negative fixtures |
|
||||
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 73 semantic, 52 compile-negative, 21 runtime-negative fixtures |
|
||||
| 6 | Condition header + source spans | **Fixed** | executable tails rejected; literal-inside-backtick mapping tests |
|
||||
| 7 | Independent Tool consumers | **Done locally** | `sample-view`/`sample-knowledge`/`sample-proof` E2E test |
|
||||
| 8 | lock v3 reproducibility | **Implemented; tamper test added** | `package_env` PASS |
|
||||
| 9 | Analysis/Trace/Tool/ABI/Foreign | **Implemented; supported-matrix bounded** | protocol tests, plugin/worker/foreign tests |
|
||||
| 10 | Linux strict L1 | **VM PASS at 8af7262; no CI URL** | 7/7 + positive CLI |
|
||||
| 10 | Linux strict L1 | **VM PASS at 70d916d; no CI URL** | 7/7 + positive CLI + strict resource refusal |
|
||||
| 11 | Linux strict L2 | **NOT IMPLEMENTED / UNSUPPORTED** | strict rejects Worker/Foreign/native ABI |
|
||||
| 12 | Windows Native/Worker ABI | **UNSUPPORTED** | no Windows ABI artifact/claim |
|
||||
| 13 | RC artifact | **Linux candidate only; no release** | clean-install + checksums; macOS not built here |
|
||||
@@ -86,7 +87,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
|
||||
- Frozen Markdown profile `marqdo.md.v1` with 160 AST goldens and explicit
|
||||
CRLF byte semantics.
|
||||
- Native Rust interpreter with 44 semantic goldens, 52 compile-error fixtures,
|
||||
- Native Rust interpreter with 73 semantic goldens, 52 compile-error fixtures,
|
||||
and runtime error source attribution.
|
||||
- Native `**` power is pinned end-to-end: right associativity, unary-minus
|
||||
precedence, parentheses, float negative exponents, branch/loop conditions,
|
||||
@@ -109,6 +110,86 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
version verification.
|
||||
- Linux strict L1: Bubblewrap namespace supervisor, runtime probe, fail-closed
|
||||
launch, child self-verification, 7/7 adversarial matrix.
|
||||
- Addressable memory (RFC-CM P1, ADR-0022): opaque `Memory` values shared by
|
||||
handle identity, region isolation, zero-filled 64 KiB page growth, bounded
|
||||
byte read/write, `memory.new/size/grow`, `m[i] = byte`, run and per-region
|
||||
budgets, host/wire rejection, Analysis v2 target projection.
|
||||
- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): success is
|
||||
`{ok:true,value:...}`, failure is `{ok:false,code,message[,details]}` as an
|
||||
ordinary Map; expected conditions return it, invariant violations stay
|
||||
fatal. A `.mq.md` first-fit/free-list allocator reuses blocks, splits free
|
||||
blocks, coalesces address-adjacent blocks and returns `alloc.no_space`
|
||||
without aborting.
|
||||
- CM-A07 Compat-A (ADR-0027): explicit `.mq.md` `io.写值` chain
|
||||
(`format.显示` → UTF-8 encode → `Memory` → `resource.write(stdout)`) matches
|
||||
builtin `打印` for the covered value set; builtin remains default and no
|
||||
alias is routed.
|
||||
- Compat-C print completion (ADR-0031): `HostOp::Print`/`op_print` removed;
|
||||
the print builtin only routes to bundled `io.写值` or fails with
|
||||
`kernel.unavailable`; trace has resource effects only.
|
||||
- Compat-B print routing (ADR-0030): `Program.print_override` routes
|
||||
`打印/print` to the bundled `io.写值` when the program uses it; stdlib
|
||||
modules are injected lazily; resource operations are trace `effect` events
|
||||
and `HostOp::Print` no longer runs on that path. Strict permits only
|
||||
stdio/stdout; file/network resources remain refused.
|
||||
- Full-value stdlib formatting (ADR-0029): FunctionRef `module/name` and
|
||||
PluginHandle `plugin/export` are read-only observation members (host id
|
||||
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
|
||||
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
|
||||
- C ABI provider worker binding (CM-A09, ADR-0038): the separate
|
||||
`marqdo-provider-worker` helper loads a trusted C ABI library and serves the
|
||||
framed-JSON transport; process isolation but not a sandbox.
|
||||
- Provider package manifest v1 (CM-A09, ADR-0037): a local `marqdo.provider`
|
||||
manifest resolves a trusted C ABI library and registers it; no registry,
|
||||
signatures or CLI install yet.
|
||||
- C ABI ResourceProvider v1 (CM-A09, ADR-0036): trusted shared libraries expose
|
||||
open/read/write/close through the frozen header; version/load failures are
|
||||
fail-closed; project-mode + name grant only. Plugin-Worker binding remains open.
|
||||
- Worker resource provider transport (CM-A09, ADR-0035): host-managed helper
|
||||
process with framed JSON, ping/open/read/write/close; project-mode + name
|
||||
grant; trusted, not a sandbox; C ABI binding still open.
|
||||
- HTTP framing and client (P5/C10): `stdlib/net/http.mq.md` parses status,
|
||||
headers and `Transfer-Encoding: chunked`, builds requests, and an end-to-end
|
||||
test parses a loopback TCP response; fixture 71 pins the same logic. No host
|
||||
HTTP/TLS implementation.
|
||||
- Blocking TCP endpoint (P5 first slice, ADR-0034): project-mode
|
||||
`resource.open("tcp", ip:port, ["read","write"])` with exact IP grant and
|
||||
timeouts; no DNS/TLS/UDP/server sockets; strict/pure refuse. CLI `--allow-net`.
|
||||
- In-process ResourceProvider v1 (CM-A09 first slice, ADR-0033): trusted
|
||||
providers register additional resource kinds without parser/runtime/HostOp
|
||||
changes; project-mode + name grant required; pure/strict refuse provider
|
||||
kinds; C/Worker transport remains open.
|
||||
- P6 baseline profile: `tools/perf/bench.py` measures 7 `.mq.md` workloads with
|
||||
wall time and peak RSS; no optimizer/backend was added and P6 is not claimed
|
||||
complete.
|
||||
- Cross-storage table equivalence (P4, ADR-0032): fixture 72 compares T0 row
|
||||
Table results with a `.mq.md` columnar List/Map store for sort/filter.
|
||||
- Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md`
|
||||
provides select/filter/sort/append/set/join/group/`转GFM` over the logical
|
||||
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
|
||||
storage are unchanged.
|
||||
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
|
||||
`format`/`text`/`io`; explicit `std:` imports resolve only inside the bundle;
|
||||
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
|
||||
`project.stdlib`; a frozen-binary test asserts the CLI hash is unchanged after
|
||||
running a `std:io` project and that tampering is rejected. `打印/print` is
|
||||
routed to the bundled stdlib (Compat-C, ADR-0030/0031); there is no legacy
|
||||
host formatter. `tools/pack-stdlib.py` additionally emits a byte-deterministic
|
||||
offline archive + descriptor as an extraction-ready artifact
|
||||
([evidence](evidence/2026-11-20-stdlib-package.md)). The stdlib intentionally
|
||||
stays in this repository for now (ADR-0039); extraction is a future option
|
||||
when size/release cadence requires it, not a pending gate.
|
||||
- Resource streams (RFC-CM CM-A06, ADR-0025): opaque host-issued `Resource`
|
||||
handles with `resource.open/read/write/close`; bytes move only through
|
||||
`Memory`; stdout write is capability/output-budget checked and project-file
|
||||
read/write is path/grant checked; file writes buffer until `close` and
|
||||
never flush on drop; a `.mq.md` chunked copy loop reproduces a file without a
|
||||
Rust `copy_file`; strict permits only the stdio console channel and refuses
|
||||
file/TCP/provider resources (CM-A11 lane PASS). Semantic fixture 66 and CLI/host tests.
|
||||
- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
|
||||
value-semantics `list.append/concat`, `map.keys/has/get/set/remove`,
|
||||
Unicode scalar `text.codepoints`/`text.from_codepoint`, and non-negative
|
||||
`table.columns/row/cell`. Pure builtins; no mutation, no new syntax.
|
||||
|
||||
## 4. Open blockers and limits
|
||||
|
||||
@@ -116,30 +197,110 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
the Linux EAGAIN contract are UNVERIFIED on GitHub. G1/G2/G3 remain FAIL
|
||||
until a real run exists.
|
||||
2. **G-C is VM-verified but not CI-signed.** No self-hosted security-lane URL
|
||||
for 8af7262 exists.
|
||||
for 70d916d exists.
|
||||
3. Windows Native/Worker ABI is explicitly UNSUPPORTED (no Windows `cc`
|
||||
fixture lane, Unix-gated worker tests).
|
||||
4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed
|
||||
broker services; unsupported requests must return `capability.unavailable`.
|
||||
5. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
|
||||
5. Resource streams cover project-file read/write, stdout, compatible print,
|
||||
trusted in-process providers and IP-literal TCP; directories, process,
|
||||
device, DNS/TLS/HTTP, UDP and provider packaging remain open. Strict permits
|
||||
only stdio and refuses the rest (CM-A11 lane PASS).
|
||||
6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
|
||||
digest-pinned directory with a regular-file executable is supported.
|
||||
6. No Windows/macOS RC archive exists from this host. The release-candidate
|
||||
7. No Windows/macOS RC archive exists from this host. The release-candidate
|
||||
workflow can build macOS on GitHub, but it has not run for this SHA.
|
||||
7. No release tag/binary publication. The release decision is **DO NOT
|
||||
8. No release tag/binary publication. The release decision is **DO NOT
|
||||
RELEASE** until the native CI matrix and a security-lane CI URL exist.
|
||||
|
||||
## 5. Evidence files
|
||||
## 5. RFC-CM compute-machine status
|
||||
|
||||
The two 2026-11-20 architecture RFCs are design input, not implemented
|
||||
behavior. This table records only what is in this tree now.
|
||||
|
||||
| RFC phase | State | Evidence / limit |
|
||||
|---|---|---|
|
||||
| P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged |
|
||||
| P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) |
|
||||
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PASS (local pure lane); stdlib remains in-repo by design** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), an Int matrix multiply (70), and a sparse-matrix multiply (73); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Local gate PASS: `crates/cli/tests/p2_closure.rs` runs 21 C01–C06/C11/C12 fixtures plus the stdlib table self-test in `--mode pure` (no foreign/plugin/import grants), scans sources for foreign/plugin/import shortcuts and compares value/stdout to goldens; evidence [`p2-closure-gate`](evidence/2026-11-20-p2-closure-gate.md). The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). The stdlib intentionally remains in-repo (ADR-0039); a separate repository is a future extraction option, not a gate. Remote CI URL is still pending (see §1). |
|
||||
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Providers are registered through ADR-0033 (in-process) and ADR-0035 (worker helper). Still missing: directories, C ABI provider binding, provider packages, strict provider support. |
|
||||
| P4 Table logical/physical migration | **PASS (local gate); T1/T2/T3 future** | ADR-0032/CM-A08: `Value::Table` semantics and GFM AST laziness unchanged; `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group/`转GFM` as value-returning `.mq.md` functions with input immutability; zero Rust table algorithms were added. Cross-storage equivalence is pinned by fixture 72 (row Table vs `.mq.md` columnar store, identical headers/rows/cells). The optional T1/T2/T3 physical-storage/column-store migration was not required by the gate and is not implemented. |
|
||||
| P5 third-party Provider / TCP / HTTP | **PASS (local gate); external network/CI and plugin-Worker resource binding open** | C10: `stdlib/net/http.mq.md` + fixture 71 parse HTTP/1.1 status/headers/chunked bodies and build requests; `http_over_tcp.rs` runs a loopback HTTP exchange wholly in `.mq.md` over `resource.open("tcp",...)`. ADR-0034 IP-literal blocking TCP endpoint (exact IP grant, project-only, 5s timeouts) and ADR-0033 in-process `ResourceProvider` v1 (registration, protocol/kind checks, name grant, mock read/write provider) ADR-0035 host-managed worker provider transport (framed JSON helper, ping/open/read/write/close, trusted-only) and ADR-0036 trusted C ABI provider (shared-library symbols, version/load checks) ADR-0037 local provider-package manifest installation and ADR-0038 C ABI provider worker binding (separate trusted helper process) are tested. External DNS/TLS/UDP, host HTTP, provider packages/signatures and the plugin-Worker resource binding remain out of scope. |
|
||||
| P6 performance / native backends | **BASELINE ONLY** | reproducible profile harness `tools/perf/bench.py` and [`p6-profile`](evidence/2026-11-20-p6-profile.md) for 7 workloads; no bytecode/JIT/columnar numeric backend or equivalence/optimization gate yet |
|
||||
|
||||
The P1 memory domain is pure computation: it needs no grant and is available in
|
||||
`pure`, `project` and `strict` (bounded by the memory budgets). It adds no
|
||||
`HostOp`, no Markdown role and no CLI command. `Memory` handles cannot cross
|
||||
the wire or a host/plugin/foreign boundary. Block-level safety of a
|
||||
user-written allocator is explicitly **not** claimed.
|
||||
|
||||
## 6. Evidence files
|
||||
|
||||
- [`evidence/2026-11-20-local-gate.md`](evidence/2026-11-20-local-gate.md) —
|
||||
earlier local gate and Windows autocrlf reproduction.
|
||||
- [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md)
|
||||
— current VM lane, environment, full log and workspace test log.
|
||||
- [`evidence/2026-11-20-resource-stream-p3.md`](evidence/2026-11-20-resource-stream-p3.md)
|
||||
— CM-A06 first slice: stdout write, project-file chunked read, strict refusal,
|
||||
wire/host boundary rejection and test commands.
|
||||
- [`evidence/2026-11-20-resource-file-write-p3.md`](evidence/2026-11-20-resource-file-write-p3.md)
|
||||
— buffered project-file write, flush-on-close, buffer ceiling, grants and
|
||||
CLI write-then-read evidence.
|
||||
- [`evidence/2026-11-20-stdlib-print-compat-a.md`](evidence/2026-11-20-stdlib-print-compat-a.md)
|
||||
— CM-A07 Compat-A `.mq.md` print prototype, side-by-side output comparison,
|
||||
covered/uncovered value types and the Compat-B blocker.
|
||||
- [`evidence/2026-11-20-bundled-stdlib-cm-a10.md`](evidence/2026-11-20-bundled-stdlib-cm-a10.md)
|
||||
— CM-A10 bundled stdlib, `std:` imports, digest verification, Analysis
|
||||
identity and frozen-binary closure test.
|
||||
- [`evidence/2026-11-20-full-value-formatting.md`](evidence/2026-11-20-full-value-formatting.md)
|
||||
— ADR-0029 observation members and all-15-type stdlib formatting comparison.
|
||||
- [`evidence/2026-11-20-print-compat-b.md`](evidence/2026-11-20-print-compat-b.md)
|
||||
— ADR-0030 Compat-B print routing, trace resource effects, lazy stdlib
|
||||
injection and strict stdio console permission.
|
||||
- [`evidence/2026-11-20-print-compat-c.md`](evidence/2026-11-20-print-compat-c.md)
|
||||
— ADR-0031 removal of `HostOp::Print`, no-bundle fail-closed behavior and
|
||||
embedder compatibility note.
|
||||
- [`evidence/2026-11-20-table-cm-a08.md`](evidence/2026-11-20-table-cm-a08.md)
|
||||
— ADR-0032 table logical value, stdlib select/filter/sort/append/set/join/
|
||||
group, input immutability and T0 storage boundary.
|
||||
- [`evidence/2026-11-20-provider-abi-cm-a09.md`](evidence/2026-11-20-provider-abi-cm-a09.md)
|
||||
— ADR-0033 in-process ResourceProvider v1, grants/mode checks, mock provider
|
||||
read/write and registration rejection cases.
|
||||
- [`evidence/2026-11-20-tcp-endpoint.md`](evidence/2026-11-20-tcp-endpoint.md)
|
||||
— ADR-0034 blocking TCP endpoint, IP grants, DNS refusal, loopback host/CLI
|
||||
echo tests.
|
||||
- [`evidence/2026-11-20-table-cross-storage.md`](evidence/2026-11-20-table-cross-storage.md)
|
||||
— P4 row-store Table vs `.mq.md` columnar store equivalence for sort/filter.
|
||||
- [`evidence/2026-11-20-c-worker-binding.md`](evidence/2026-11-20-c-worker-binding.md)
|
||||
— ADR-0038 C ABI provider worker binding, compiled C adapter in the helper
|
||||
process and fail-closed missing-library case.
|
||||
- [`evidence/2026-11-20-provider-package.md`](evidence/2026-11-20-provider-package.md)
|
||||
— ADR-0037 local provider package manifest, C library resolution and
|
||||
registration test.
|
||||
- [`evidence/2026-11-20-c-provider.md`](evidence/2026-11-20-c-provider.md)
|
||||
— ADR-0036 C ABI ResourceProvider v1, compiled C adapter round-trip and
|
||||
fail-closed load/version checks.
|
||||
- [`evidence/2026-11-20-worker-provider.md`](evidence/2026-11-20-worker-provider.md)
|
||||
— ADR-0035 trusted worker resource-provider transport, framed-JSON helper
|
||||
round-trip and grant/mode/error tests.
|
||||
- [`evidence/2026-11-20-http-over-tcp.md`](evidence/2026-11-20-http-over-tcp.md)
|
||||
— P5/C10 `.mq.md` HTTP framing over a loopback TCP stream, stdlib module and
|
||||
end-to-end client test.
|
||||
- [`evidence/2026-11-20-p2-closure-gate.md`](evidence/2026-11-20-p2-closure-gate.md)
|
||||
— P2 C01–C06/C11/C12 pure no-Foreign/no-Plugin closure lane and golden
|
||||
comparison.
|
||||
- [`evidence/2026-11-20-http-parser.md`](evidence/2026-11-20-http-parser.md)
|
||||
— P5/C10 `.mq.md` HTTP/1.1 status/header/chunked response parser and request
|
||||
builder fixture.
|
||||
- [`evidence/2026-11-20-strict-resource-cm-a11.md`](evidence/2026-11-20-strict-resource-cm-a11.md)
|
||||
— CM-A11 strict lane at `00bdcad`: 7/7 adversarial matrix, positive Compat-C
|
||||
print, strict file-resource refusal, forged-marker rejection and raw log.
|
||||
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
|
||||
— G0..G12 matrix and RC decision.
|
||||
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
|
||||
L1/L2 limits.
|
||||
|
||||
## 6. Reproduce
|
||||
## 7. Reproduce
|
||||
|
||||
```bash
|
||||
export PATH="$HOME/.cargo/bin:$PATH"
|
||||
|
||||
@@ -54,14 +54,23 @@ The trusted supervisor is this process; the restricted child is the same
|
||||
The document argument is rewritten to `/project/...`; only the document token
|
||||
changes, all other arguments are preserved.
|
||||
|
||||
### 4. Child self-verification
|
||||
### 4. Bundled stdlib mount
|
||||
|
||||
After CM-A07 Compat-C, `打印` is implemented by the bundled `.mq.md` stdlib.
|
||||
Strict therefore bind-mounts the verified `stdlib/` tree read-only at
|
||||
`/run/marqdo-stdlib` and sets `MARQDO_STDLIB_DIR` for the child, just like the
|
||||
program itself. The tree contains only Markdown/read-only data; it grants no
|
||||
new OS capability, and the loader still verifies its digest in the child. File,
|
||||
network, process and provider resources remain refused.
|
||||
|
||||
### 5. Child self-verification
|
||||
|
||||
The child accepts `MARQDO_STRICT_ACTIVE=1` only if it can verify all of:
|
||||
`getpid() == 1`, `NoNewPrivs: 1`, empty `CapEff`, a non-initial uid map, and no
|
||||
non-loopback network interface. A forged marker outside the sandbox fails
|
||||
before project code runs.
|
||||
|
||||
### 5. Threat model and non-goals
|
||||
### 6. Threat model and non-goals
|
||||
|
||||
This protects against untrusted Marqdo documents that use the language,
|
||||
interpreter, Math, console, or attempt direct syscalls. It does not sandbox
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
# ADR-0022: Addressable `Memory` and Memory-index place assignment (RFC-CM P1)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md` (CM-A01/CM-A02/CM-A03),
|
||||
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (P0/P1), ADR-0007,
|
||||
ADR-0011, ADR-0017, ADR-0021
|
||||
Supersedes: the v1 rule that assignment targets are bare identifiers only
|
||||
Specs: `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`,
|
||||
`spec/analysis-v2.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-memory-p1.md`
|
||||
|
||||
## Context
|
||||
|
||||
RFC-CM-0001/0002 define three domains: managed values, addressable regions and
|
||||
capability resources. The first two are a language change; the third is an OS
|
||||
boundary and is deliberately **not** part of this ADR.
|
||||
|
||||
Before this decision the interpreter had no writable place other than a local
|
||||
slot. `Value` had no byte region, `a[i]` was read-only, and assignment was
|
||||
parsed only as `identifier = expression`. A `.mq.md` program therefore could
|
||||
not express an allocator, an encoded record, a free list or a byte protocol
|
||||
without a new Rust builtin for every algorithm. That is the closure gap
|
||||
RFC-CM-0002 §2 identifies.
|
||||
|
||||
The decision must not:
|
||||
|
||||
* turn managed `List`/`Map`/`Table` values into shared mutable objects;
|
||||
* expose a host pointer, `usize` address or procedural `free(region)`;
|
||||
* let an active handle cross the value wire or an ABI boundary;
|
||||
* add a Markdown node, keyword or `HostOp` for memory;
|
||||
* claim block-level memory safety for user allocators;
|
||||
* weaken `strict` or grant any external I/O.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. `Value::Memory` is an opaque, run-scoped identity
|
||||
|
||||
`MemoryHandle` (`crates/marqdo-value/src/value.rs`) is
|
||||
`(run_id, region_id, generation)` with private fields and a runtime-only
|
||||
constructor. It is not `Serialize`/`Deserialize`, is never built from user
|
||||
input, and carries no host address.
|
||||
|
||||
Copying the value, passing it as an argument and storing it in a list or map
|
||||
all copy the handle and preserve region identity. `==`/`!=` compare identity
|
||||
only; `Memory` has no ordering comparison. `display()`/`repr()` render the
|
||||
literal `Memory`, never the internal identity.
|
||||
|
||||
### 2. Regions are grow-only, bounded byte arrays
|
||||
|
||||
Frozen semantics (CM-A01/CM-A02):
|
||||
|
||||
| Property | Decision |
|
||||
|---|---|
|
||||
| Page size | 64 KiB logical page (`PAGE_SIZE`) |
|
||||
| Initial content | Every byte is `0` |
|
||||
| Growth | `memory.grow` only increases; old offsets/bytes unchanged; new bytes zero |
|
||||
| Failure atomicity | A rejected create/grow allocates, commits and mutates nothing |
|
||||
| Shrink/free | No shrink and no user-visible region free; regions are dropped with the run |
|
||||
| Address | `(region identity, non-negative offset)`; an integer alone is never a credential |
|
||||
| Read | `m[i] -> Int(0..=255)` |
|
||||
| Write | `m[i] = byte`, `byte` must be `Int(0..=255)`, no truncation |
|
||||
| Isolation | Different regions never alias; handle copies of one region do |
|
||||
| Visibility | `memory.size` returns committed pages, not host capacity or pointers |
|
||||
|
||||
Limits in `MemoryLimits`:
|
||||
|
||||
* `DEFAULT_MAX_TOTAL_BYTES = 64 MiB` committed across the run;
|
||||
* `DEFAULT_MAX_REGION_BYTES = 16 MiB` per region, which also caps declared
|
||||
`max_pages` at 256;
|
||||
* budget arithmetic uses checked `u64`/`usize` operations; allocation uses
|
||||
`try_reserve` so a host OOM is a structured `resource.exhausted` instead of
|
||||
a panic.
|
||||
|
||||
Limits are a runtime API (`marqdo_runtime::MemoryLimits`, `run_with_limits`)
|
||||
and are overridable on `run`/`call` with `--max-memory <BYTES>` (total) and
|
||||
`--max-region-memory <BYTES>` (per region). The defaults are stated in
|
||||
`spec/evaluation.md`. Both flags apply in `pure`, `project` and `strict`; they
|
||||
grant no external capability.
|
||||
|
||||
### 3. The only new writable place is a `Memory` index
|
||||
|
||||
`AssignTarget` gains `Index { base, index, span }` in the expression AST and
|
||||
`PlaceTarget` mirrors it in the IR. The surface syntax remains a `strong`
|
||||
node containing `target = expression`; no Markdown role, heading or list
|
||||
meaning changes.
|
||||
|
||||
* `x = value` still rebinds the current frame's local slot.
|
||||
* `m[i] = byte` evaluates **base, then index, then RHS**, exactly once each.
|
||||
All three checks (base is `Memory`, offset in range, byte in `0..=255`)
|
||||
happen before the single byte write, so a failed statement does not write.
|
||||
* `list[i] = value`, `map[key] = value` and `obj.field = value` are not
|
||||
defined. Member targets are a parse error; a non-`Memory` index base is
|
||||
`runtime.type`. No managed collection changes value semantics.
|
||||
* Nested index targets are accepted syntactically; the innermost base must
|
||||
still evaluate to a `Memory` value at run time.
|
||||
* Assignment remains statement-only; `x = y = 1` is still a parse error.
|
||||
|
||||
This is CM-A03.
|
||||
|
||||
### 4. Handles cannot cross the wire or the host boundary
|
||||
|
||||
* `to_wire(Memory)` emits only `{"type":"memory"}` so an accidental transfer
|
||||
is visible. `from_wire` rejects that tag with `WireError::Invalid`; a
|
||||
round-trip is intentionally impossible (no `wire_version` bump: the value
|
||||
shape is rejected, not newly accepted).
|
||||
* `Host::invoke` rejects any argument tree containing `Memory` before dispatch
|
||||
(`runtime.type`), so plugins, foreign kernels, math, file and print adapters
|
||||
never see a handle. A future explicit byte-snapshot operation must copy bytes
|
||||
into a managed value and is a separate ADR.
|
||||
* `HostOp` is unchanged. `memory.new/size/grow` and index read/write are
|
||||
interpreter operations, not capability-gated host effects.
|
||||
|
||||
### 5. Error categories are stable and distinct
|
||||
|
||||
| Situation | Code |
|
||||
|---|---|
|
||||
| Negative/unrepresentable offset, offset >= region length | `runtime.index` |
|
||||
| Unknown, stale or cross-run handle; invalid page count (negative, > budget declaration arithmetic) | `runtime.memory` |
|
||||
| Wrong operand type; byte outside `0..=255`; non-Memory index-assignment base | `runtime.type` |
|
||||
| Declared max, region budget or total budget exceeded | `resource.exhausted` |
|
||||
|
||||
No operation panics on malformed input or forged handles. `expect` is used only
|
||||
after the same handle has been validated in the same call frame.
|
||||
|
||||
### 6. Builtins are part of the language surface, not `HostOp`s
|
||||
|
||||
`memory.new(initial_pages, max_pages) -> Memory`,
|
||||
`memory.size(memory) -> Int`, `memory.grow(memory, pages) -> Int` are added to
|
||||
`marqdo-ir::builtins::BUILTINS` and `spec/evaluation.md`. They are pure
|
||||
(`effectful: false`): `pure` mode may use them, `strict` needs no new provider,
|
||||
and no capability grant is consulted.
|
||||
|
||||
### 7. Compatibility
|
||||
|
||||
* Local assignment IR keeps its `target` string in Analysis v2; an index
|
||||
target is an additive object (`{"kind":"index",...}`). Consumers that only
|
||||
handle locals continue to work.
|
||||
* Expression AST/statement serde gains an enum tag; this is a pre-1.0 internal
|
||||
contract documented in `spec/expression.ebnf`.
|
||||
* Existing Markdown AST goldens, value semantics, `List`/`Map`/`Table`
|
||||
behaviour and all old builtins are unchanged.
|
||||
* `Memory` is additive to the value type set. Old programs never produce it.
|
||||
ABI wire consumers see a new rejected tag, not a renumbered existing tag.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* No `Pointer` value, raw pointer, `free(region)`, registers or machine code.
|
||||
* No index/field assignment for managed collections; no implicit mutation.
|
||||
* No multi-threaded sharing, atomics or data-race guarantees.
|
||||
* No serialization of live regions; no cross-process shared memory.
|
||||
* No `ResourceHandle`, `Stream`, files, network or OS syscalls. Those are
|
||||
CM-A05/CM-A06/CM-A09/CM-A11 and later RFC phases.
|
||||
* No block-level safety claim for a user-written allocator; the runtime only
|
||||
bounds a single access to its region.
|
||||
* No allocator or stdlib API in P1; `--max-memory` only bounds the runtime.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A `.mq.md` program can now build byte encoders, arenas, free lists and
|
||||
virtual CPU register files without new Rust code. P2 (the Marqdo-written
|
||||
stdlib and closure fixtures) becomes testable.
|
||||
* The interpreter owns one `MemoryStore` per run; `run_with_limits` exposes
|
||||
the budget. This is runtime state, not a second `HostOp` table.
|
||||
* Trace gains `memory.new`, `memory.grow` and `memory.write` semantic events
|
||||
without exposing handles or byte secrets.
|
||||
* The remaining RFC decisions (CM-A04 and later) stay separate. In particular,
|
||||
generic structural observation/construction, recoverable `Result`, table
|
||||
algorithm migration and the resource/Stream ABI are **not** implemented by
|
||||
this ADR.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-memory-p1.md` records:
|
||||
|
||||
* parser/AST tests for index and rejected member targets;
|
||||
* runtime store tests for isolation, aliasing, zero-fill, atomic failed grow,
|
||||
negative/huge offsets, forged/stale handles, byte range and budgets;
|
||||
* semantic goldens `45`–`47` and runtime negatives `13`–`18`;
|
||||
* wire rejection test and host-boundary rejection test;
|
||||
* CLI example `examples/memory-buffer.mq.md` returning `141`;
|
||||
* Analysis v2 index-target projection;
|
||||
* `cargo fmt --check`, `cargo clippy -- -D warnings`, `cargo test --workspace`.
|
||||
@@ -0,0 +1,147 @@
|
||||
# ADR-0023: Minimal structural observation and construction (RFC-CM CM-A04)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
|
||||
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A04, P2),
|
||||
ADR-0007, ADR-0022
|
||||
Specs: `spec/evaluation.md`, `spec/expression.ebnf`, `spec/analysis-v2.md`
|
||||
Evidence: `docs/evidence/2026-11-20-structure-api-p2.md`
|
||||
|
||||
## Context
|
||||
|
||||
RFC-CM-0002 §3.5 identifies a closure gap: a `.mq.md` program can read a
|
||||
`List`/`Map`/`Table` and print it through the host, but it cannot generally
|
||||
**construct** the same structures dynamically or **observe** all of their
|
||||
logical parts. That makes `format`, JSON/CSV parsers and table algorithms
|
||||
impossible to write in `.mq.md` without adding a task-specific Rust function
|
||||
for each algorithm.
|
||||
|
||||
The current core already has:
|
||||
|
||||
* `List` literals, index reads (including negative offsets), `长度`;
|
||||
* `Map` record literals, `.field` reads, `长度`;
|
||||
* `Text` concatenation, character indexing and `长度`;
|
||||
* `Table` construction through `表(header, rows)`, `长度` (rows), display and
|
||||
wire round-trip.
|
||||
|
||||
It lacks: append/concat, dynamic map update/enumeration, Unicode scalar
|
||||
conversion, and table column/row/cell projection. These are observation and
|
||||
construction primitives, not algorithms. Sorting, filtering, joining,
|
||||
serialising and parsing remain standard-library work.
|
||||
|
||||
## Decision
|
||||
|
||||
Add a small, namespaced, **pure** builtin set. No new Markdown node, keyword,
|
||||
`HostOp`, capability or mutation semantics is introduced. All constructors
|
||||
return **new** values; an input collection is never modified in place. This
|
||||
keeps INV-02 and preserves existing `List`/`Map`/`Table` copy semantics.
|
||||
|
||||
### 1. `List`
|
||||
|
||||
| Signature | Semantics |
|
||||
|---|---|
|
||||
| `list.append(list: List, value) -> List` | New list with `value` at the end; the input list is unchanged. |
|
||||
| `list.concat(left: List, right: List) -> List` | New list containing `left` then `right`; both inputs unchanged. |
|
||||
|
||||
Neither aliases the other list's element storage: values are copied according
|
||||
to their normal value semantics (`Memory` handles keep region identity).
|
||||
|
||||
### 2. `Map`
|
||||
|
||||
| Signature | Semantics |
|
||||
|---|---|
|
||||
| `map.keys(map: Map) -> List` | `Text` keys in canonical sorted order (`BTreeMap` order, the order used by `display`). |
|
||||
| `map.has(map: Map, key: Text) -> Bool` | Whether the key exists. |
|
||||
| `map.get(map: Map, key: Text) -> Value` | Stored value, or `None` when absent. Use `map.has` when `None` is a legal stored value. |
|
||||
| `map.set(map: Map, key: Text, value) -> Map` | New map with the key set; the input map is unchanged. |
|
||||
| `map.remove(map: Map, key: Text) -> Map` | New map without the key (missing key is a no-op); the input map is unchanged. |
|
||||
|
||||
Keys are always `Text`; arbitrary-value keys are not introduced. `map.get`
|
||||
returning `None` is a *lookup result*, not the language's recoverable-error
|
||||
contract; a full `Result` convention remains CM-A05 and is not frozen here.
|
||||
|
||||
### 3. `Text`
|
||||
|
||||
`Text` remains a Unicode scalar-value sequence, not a byte array. These
|
||||
primitives make the existing representation observable without introducing a
|
||||
second text type or implicit UTF-8 conversion.
|
||||
|
||||
| Signature | Semantics |
|
||||
|---|---|
|
||||
| `text.codepoints(text: Text) -> List` | Unicode scalar values (`Int`) in code-point order. |
|
||||
| `text.from_codepoint(codepoint: Int) -> Text or None` | One-scalar `Text`; `None` for a surrogate or a value outside `0..=0x10FFFF`. |
|
||||
|
||||
UTF-8 encoding/decoding is **not** a core primitive. It can be written in
|
||||
`.mq.md` using bytes, `text.codepoints` and `text.from_codepoint`; this keeps
|
||||
the byte/code-point distinction explicit.
|
||||
|
||||
### 4. `Table`
|
||||
|
||||
`Value::Table` stays the logical value; its physical `Vec<Vec<Value>>` layout
|
||||
is not part of this contract. Indices are non-negative row/column coordinates
|
||||
(unlike `List` negative indexing) because a table cell is a document
|
||||
coordinate. Bounds failures are `runtime.index`.
|
||||
|
||||
| Signature | Semantics |
|
||||
|---|---|
|
||||
| `table.columns(table: Table) -> List` | Header names (`Text`) in column order. |
|
||||
| `table.row(table: Table, row: Int) -> List` | A new list of the row's cells in column order. |
|
||||
| `table.cell(table: Table, row: Int, column: Int) -> Value` | The cell value. |
|
||||
|
||||
No `table.set_cell`/`append_row` builtin is added. Rebuilding a table is a
|
||||
value-level operation that can be written with `表(header, rows)` once list
|
||||
construction exists; adding a Rust table algorithm would violate the closure
|
||||
goal.
|
||||
|
||||
### 5. Error and compatibility rules
|
||||
|
||||
* Wrong collection/text/table type or key type → `runtime.type`.
|
||||
* Table row/column outside the logical shape → `runtime.index`.
|
||||
* `text.from_codepoint` never throws for an invalid scalar; it returns `None`.
|
||||
* `map.get` never throws for a missing key; it returns `None`.
|
||||
* Existing `List`/`Map`/`Table` behavior, display, wire encoding and Markdown
|
||||
laziness are unchanged.
|
||||
* The builtins are `effectful: false`, available in `pure`, `project` and
|
||||
`strict`, and add no capability. They do not consume output budget.
|
||||
* Names are frozen as written; aliases are not added. Namespaced calls use the
|
||||
existing `Expr::Member` resolution path.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* No `set_cell`, `append_row`, `sort`, `filter`, `join`, `group`, JSON/CSV or
|
||||
other algorithm builtins; those belong in `.mq.md`.
|
||||
* No mutable/aliased List or Map, no in-place mutation, no shared structural
|
||||
sharing protocol.
|
||||
* No arbitrary-value map keys, no ordered-map insertion semantics, no
|
||||
duplicate-key policy change.
|
||||
* No UTF-8/UTF-16 conversion primitive, no byte-array text type.
|
||||
* No `Result`/`try`/exception syntax; CM-A05 stays open.
|
||||
* No change to Table storage, GFM parsing, source spans or `ref` semantics.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A `.mq.md` program can build maps and lists dynamically, enumerate keys,
|
||||
project table cells and encode Unicode scalars. This unblocks a Marqdo-
|
||||
written formatter, table filter/projection and a JSON/CSV parser without
|
||||
per-algorithm Rust.
|
||||
* The builtin table grows by 12 pure entries. The migration rule remains:
|
||||
only observation/construction primitives may enter core; algorithms do not.
|
||||
* Because constructors are value-returning, repeated `map.set` in a loop is
|
||||
O(n²) for a BTreeMap clone. That is acceptable for P2 semantics; a
|
||||
persistent/structural-sharing representation is a P6 performance decision,
|
||||
not a semantic one.
|
||||
* `map.get` returning `None` for absence is deliberate and documented; a
|
||||
future CM-A05 `Result` convention must not silently reinterpret existing
|
||||
programs.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-structure-api-p2.md` records:
|
||||
|
||||
* ADR/spec deltas and builtin metadata;
|
||||
* semantic fixtures for value-semantics list/map operations, code-point
|
||||
round-trips, table projection and invalid code points;
|
||||
* negative fixtures for wrong types and table bounds;
|
||||
* unit tests, `cargo fmt --check`, `cargo clippy -- -D warnings` and
|
||||
`cargo test --workspace`.
|
||||
@@ -0,0 +1,162 @@
|
||||
# ADR-0024: Recoverable result convention (RFC-CM CM-A05)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
|
||||
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A05, P2/P3),
|
||||
ADR-0023
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-20-result-and-free-list-p2.md`
|
||||
|
||||
## Context
|
||||
|
||||
A standard library cannot be written in `.mq.md` if every expected condition
|
||||
(no free block, invalid input, missing field, later end-of-file or a short
|
||||
read) must abort the whole run. The core has structured `RuntimeError`s with
|
||||
stable codes, but no way for an ordinary function to return a failure that the
|
||||
caller can inspect and handle in a normal branch. RFC-CM-0002 §3.6 recommends a
|
||||
`Result` data contract using existing `Map` values and control flow, without
|
||||
`try`/`catch` or exception syntax.
|
||||
|
||||
This ADR freezes the **data convention**, not a new runtime type. A separate
|
||||
core `Result` value, algebraic types or automatic propagation would be a
|
||||
larger change and is deliberately not introduced.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Shape
|
||||
|
||||
A recoverable result is an ordinary `Map`:
|
||||
|
||||
```text
|
||||
success: {ok: true, value: <any Value>}
|
||||
failure: {ok: false, code: <Text>, message: <Text> [, details: <Map>]}
|
||||
```
|
||||
|
||||
* `ok` is always present and always `Bool`.
|
||||
* On success, `value` is present. It may itself be `None`, a `List`, a `Map`,
|
||||
a `Table`, a `Memory` handle or any other value.
|
||||
* On failure, `code` and `message` are present. `code` is a lowercase
|
||||
dot-separated stable identifier; `message` is human-readable text.
|
||||
* `details` is optional and, when present, is a `Map` of additional structured
|
||||
facts. It must not contain host pointers, resource handles or secret bytes.
|
||||
* No other fields are required. Consumers must tolerate additive fields.
|
||||
|
||||
The constructors are ordinary `.mq.md` functions, for example:
|
||||
|
||||
```text
|
||||
## 成功
|
||||
`(值)`
|
||||
*{ok: true, value: 值}*
|
||||
|
||||
## 失败
|
||||
`(代码, 消息)`
|
||||
*{ok: false, code: 代码, message: 消息}*
|
||||
```
|
||||
|
||||
They are **not** builtins. A library may choose different local function names;
|
||||
the map shape is the contract.
|
||||
|
||||
### 2. What is recoverable, what stays fatal
|
||||
|
||||
Recoverable (`Result`):
|
||||
|
||||
* a user allocator has no fitting block (`alloc.no_space`);
|
||||
* a parser rejects malformed input or hits an unexpected end
|
||||
(`parse.invalid_syntax`, `parse.unexpected_eof`);
|
||||
* a codec rejects an invalid sequence (`codec.invalid_sequence`);
|
||||
* a lookup that uses an explicit result because `None` is a legal value;
|
||||
* later, a resource reports EOF/partial/pending as a **success payload** with
|
||||
its own byte counts, not as a fatal error (P3 contract).
|
||||
|
||||
Fatal `RuntimeError` (unchanged):
|
||||
|
||||
* violating a language invariant: wrong operand type, out-of-bounds managed
|
||||
collection access, unknown/stale `Memory` handle, memory budget exhaustion,
|
||||
division by zero, call arity, unknown symbol;
|
||||
* a capability denial (authorization is not a recoverable business result);
|
||||
* an internal invariant failure.
|
||||
|
||||
A library must not convert a fatal invariant into an `Err` by catching it;
|
||||
there is no catch mechanism and none is proposed here. It must validate inputs
|
||||
before calling an operation whose failure is fatal, or rely on its own
|
||||
pre-checks.
|
||||
|
||||
### 3. Codes and namespaces
|
||||
|
||||
`code` is stable within the library that defines it. Recommended prefixes:
|
||||
|
||||
| Prefix | Owner |
|
||||
|---|---|
|
||||
| `alloc.*` | allocator libraries |
|
||||
| `parse.*` | parsers |
|
||||
| `codec.*` | byte/text codecs |
|
||||
| `table.*` | table algorithms |
|
||||
| `io.*` | stream wrappers (P3, when the resource contract exists) |
|
||||
|
||||
Codes are compared as `Text`; a caller must not depend on the `message` text.
|
||||
Libraries may add codes; removing or renaming one is a breaking change for that
|
||||
library.
|
||||
|
||||
### 4. EOF and partial I/O (forward contract)
|
||||
|
||||
When the Stream/Resource API arrives (CM-A06), a successful read must
|
||||
distinguish:
|
||||
|
||||
```text
|
||||
{ok: true, value: {count: Int, eof: Bool, pending: Bool}}
|
||||
```
|
||||
|
||||
EOF is not an `Err`; a short read is not an `Err`; a connection reset or a
|
||||
permission failure is. The exact stream payload is frozen by CM-A06, not by
|
||||
this ADR. This ADR only reserves the principle.
|
||||
|
||||
### 5. Why not a new type or syntax
|
||||
|
||||
* A dedicated `Value::Result` would change the value model, wire format,
|
||||
display, equality and every exhaustive `Value` match for a convention that
|
||||
existing `Map` plus `ok` already expresses.
|
||||
* `try`/`catch` would add a control-flow construct that bypasses ordinary
|
||||
branches, contradicts the RFC's preference for `Result` values and needs new
|
||||
grammar, scope and error rules.
|
||||
* Ordinary `Map` results are visible to `type`, `length`, `map.get` and wire
|
||||
encoding today, and can be built by `.mq.md` libraries without a new ABI.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Additive: existing builtins and programs are unchanged.
|
||||
* `map.get` returning `None` for a missing key remains a lookup convention,
|
||||
not a `Result`; callers that need to distinguish a stored `None` use
|
||||
`map.has` (ADR-0023).
|
||||
* Runtime error categories and codes are unchanged.
|
||||
* No wire version bump: a `Result` is an ordinary `Map` already covered by the
|
||||
wire format.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* No automatic `?`/propagation, exceptions, panic recovery or stack unwinding.
|
||||
* No `Result` type, generics, pattern matching or exhaustive `Ok`/`Err`
|
||||
typing.
|
||||
* No change to `memory.grow` or other core operations from fatal errors to
|
||||
`Result`; a library may pre-check and return `Err` instead.
|
||||
* No resource/stream implementation; CM-A06 remains open.
|
||||
* No promise that every existing library will adopt the convention.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `.mq.md` algorithms can return and branch on failure without aborting the
|
||||
run. This is a prerequisite for a Marqdo-written allocator, parser and
|
||||
eventually `io.read_line`.
|
||||
* Because the convention is untyped, a caller can inspect the wrong shape;
|
||||
`类型(result)`, `map.has(result, "ok")` and the frozen field names are the
|
||||
checks. A future typed-value ADR may improve this without breaking the data
|
||||
shape.
|
||||
* The allocator fixture in `54`/`55` demonstrates success and failure maps and
|
||||
ordinary branching, with no Rust algorithm.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-result-and-free-list-p2.md` records a `.mq.md`
|
||||
first-fit/free-list allocator that returns `Result` maps, reuses freed blocks,
|
||||
splits free blocks, coalesces address-adjacent free blocks and returns
|
||||
`alloc.no_space` without aborting the run.
|
||||
@@ -0,0 +1,176 @@
|
||||
# ADR-0025: Minimal Resource/Stream ABI and byte transfer through `Memory` (RFC-CM CM-A06)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
|
||||
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A06, CM-A09,
|
||||
CM-A11, P3), ADR-0022, ADR-0024
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`, `spec/value-wire.md`
|
||||
Evidence: `docs/evidence/2026-11-20-resource-stream-p3.md`
|
||||
|
||||
## Context
|
||||
|
||||
P1/P2 give `.mq.md` managed values, addressable `Memory`, structure primitives,
|
||||
a recoverable `Result` convention and a substantial set of in-language
|
||||
algorithms. Nothing yet lets a standard-library function perform a real byte
|
||||
transfer to the outside world without a dedicated `HostOp` per high-level
|
||||
function.
|
||||
|
||||
CM-A06 asks for a stable, authorized, versioned resource boundary: obtain a
|
||||
granted resource, call the operations it promises, release it. CM-A09 says to
|
||||
extend the existing host/ABI rather than build a second plugin system. CM-A11
|
||||
says strict must stay fail-closed until a provider is actually verified.
|
||||
|
||||
This ADR freezes a **first, deliberately small slice**:
|
||||
|
||||
* one opaque host-issued `ResourceHandle` value;
|
||||
* `resource.open/read/write/close` as pure language builtins, not new `HostOp`
|
||||
variants;
|
||||
* byte transfer through `Memory`, never through host pointers;
|
||||
* stdio `stdout` write and project-file **read** only;
|
||||
* `Result` maps for EOF/closed/write counts, fatal errors for invalid handle
|
||||
or capability denial.
|
||||
|
||||
File write, directories, network, processes, TLS, asynchronous I/O and a
|
||||
generic provider registry remain open.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Opaque handle
|
||||
|
||||
`Value::ResourceHandle(ResourceHandle)` is a host-issued opaque identity with
|
||||
private `(run_id, resource_id, generation)` fields and a hidden constructor.
|
||||
It is not `Serialize`/`Deserialize`, is rejected by the wire, is never
|
||||
synthesized from a number or a JSON value, and renders as `Resource` in
|
||||
`display`/`repr` without exposing the id.
|
||||
|
||||
It is a **separate domain** from `Memory`: a handle is not indexable, not
|
||||
byte-addressable and not a capability token the user can manufacture.
|
||||
|
||||
### 2. First resource kinds
|
||||
|
||||
| Kind | Descriptor | Rights | Availability |
|
||||
|---|---|---|---|
|
||||
| `stdio` | `stdout` | `write` | `pure` and `project` when the console grant is present; `strict` rejects until verified |
|
||||
| `file` | project-relative path | `read` | `project` with an `fs_read` grant |
|
||||
|
||||
`file` streams are opened by reading a bounded regular file (existing
|
||||
`ProjectFs::MAX_READ_BYTES = 8 MiB` limit) into host-owned bytes; `read`
|
||||
returns successive chunks and `eof` when the position reaches the end. This is
|
||||
honest about being a first provider: it does not expose a host file descriptor,
|
||||
does not support seeking, writing, directories, network, processes, devices or
|
||||
paths outside the project root.
|
||||
|
||||
Any other kind/descriptor/right combination is an explicit
|
||||
`resource.unsupported`/`permission.denied` diagnostic, never a silent fallback.
|
||||
|
||||
### 3. Builtins
|
||||
|
||||
```text
|
||||
resource.open(kind: Text, descriptor: Text, rights: List) -> Resource
|
||||
resource.read(handle: Resource, memory: Memory, offset: Int, max_bytes: Int)
|
||||
-> {ok:true, value:{count:Int, eof:Bool}} | {ok:false, code, message}
|
||||
resource.write(handle: Resource, memory: Memory, offset: Int, length: Int)
|
||||
-> {ok:true, value:{count:Int}} | {ok:false, code, message}
|
||||
resource.close(handle: Resource)
|
||||
-> {ok:true, value:None} | {ok:false, code, message}
|
||||
```
|
||||
|
||||
* `resource.open` is effectful and capability-checked at execution time.
|
||||
* `resource.read` validates `offset` and the remaining buffer **before** asking
|
||||
the provider, so a failed range check cannot consume stream data. It caps the
|
||||
request to the remaining region, writes the returned bytes into `Memory` and
|
||||
returns the actual count and EOF flag. EOF is a **success** with `eof:true`.
|
||||
* `resource.write` copies bytes out of `Memory` only for the duration of the
|
||||
call. A short write is reported as a count; the caller loops. It never
|
||||
stores a host pointer into a region that could later grow.
|
||||
* `resource.close` closes the host slot. Reading/writing/closing an already
|
||||
closed handle returns `{ok:false, code:"resource.closed", ...}` (recoverable).
|
||||
A forged, cross-run or stale handle is `runtime.resource` (fatal).
|
||||
* All four are pure interpreter operations over host trait methods; they add no
|
||||
`HostOp` variant. Invalid capacity/bounds remains `runtime.index`; wrong
|
||||
operand types remain `runtime.type`.
|
||||
|
||||
### 4. Authorization and modes
|
||||
|
||||
* Authorization is the intersection of the requested operation, the effective
|
||||
`Capabilities` grant and the mode:
|
||||
* `stdio/stdout/write` requires `console` (the same grant `打印` uses);
|
||||
* `file/.../read` requires an `fs_read` pattern match and project-root path
|
||||
validation (`..`, absolute paths and symlink escapes rejected);
|
||||
* `pure`: stdio stdout is allowed because console is already a pure-mode
|
||||
console grant; **file resources are refused by the mode check**, even if a
|
||||
caller constructs a `Capabilities` set with a matching `fs_read` pattern.
|
||||
Network/process resources do not exist in this slice.
|
||||
* `project`: stdio stdout and explicitly granted project-file reads.
|
||||
* `strict`: only the existing **stdio/stdout** console channel is permitted
|
||||
(`resource.open("stdio","stdout",["write"])`, console grant); every file,
|
||||
network, process or device resource open is refused with
|
||||
`permission.denied`. This is the existing console capability reached through
|
||||
the resource ABI, not a new OS capability (ADR-0030). Strict L1 is otherwise
|
||||
unchanged.
|
||||
* Manifest capability requests are not grants; only the operator-provided
|
||||
effective set is consulted.
|
||||
|
||||
### 5. Errors, budgets and tracing
|
||||
|
||||
New stable categories:
|
||||
|
||||
| Code | Meaning |
|
||||
|---|---|
|
||||
| `runtime.resource` | forged/cross-run/stale handle, invalid resource state |
|
||||
| `resource.closed` | read/write/close on an already closed handle (recoverable) |
|
||||
| `resource.unsupported` | kind/descriptor/rights not implemented in this slice |
|
||||
|
||||
`resource.write` to stdout charges the existing output budget and appends to
|
||||
the host output buffer exactly like `打印`; capability denial is
|
||||
`permission.denied`. Trace records `resource.open`, `resource.read`,
|
||||
`resource.write` and `resource.close` with kind/count/eof metadata, never raw
|
||||
file bytes or handles.
|
||||
|
||||
### 6. Wire and boundary
|
||||
|
||||
* `to_wire(Resource)` emits `{"type":"resource"}` only; decoding it is a hard
|
||||
error (same rule as `Memory`).
|
||||
* The generic `Host::invoke` boundary rejects arguments containing a resource
|
||||
handle, so plugins/foreign kernels/file adapters never receive one.
|
||||
* Plugin output cannot mint a handle: the constructor is host-only and the
|
||||
value is not decodable from plugin wire.
|
||||
|
||||
### 7. Non-goals
|
||||
|
||||
* No `print` migration in this ADR; that is CM-A07 and a later increment.
|
||||
* File **write** is added by [ADR-0026](0026-buffered-project-file-write-streams.md),
|
||||
not this decision; this ADR remains read-only for files.
|
||||
* No file write, directory listing, rename, metadata, network, DNS, TLS,
|
||||
process, clock, random or device operations.
|
||||
* No generic third-party provider registration; CM-A09 remains open.
|
||||
* No zero-copy, async, deadlines per stream, seek or shared memory.
|
||||
* No strict-mode file/network/process resource support and no claim that
|
||||
host-in-process I/O is a sandbox; strict stdio is the existing console.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Additive value variant; old programs never produce `Resource`.
|
||||
* New builtins only; existing `fs.read/write`, `打印`, plugin and foreign
|
||||
behavior is unchanged.
|
||||
* No wire version bump: the new tag is rejected, not accepted.
|
||||
* New error categories are additive; existing categories/codes are unchanged.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `.mq.md` can implement `write_text`-style output by encoding text into
|
||||
`Memory` (fixture 59) and calling `resource.write`, and can implement
|
||||
chunked file readers with `resource.read`/`Result`/EOF.
|
||||
* The remaining high-level I/O surface can grow in the standard library without
|
||||
a new `HostOp` per function.
|
||||
* The resource table is host state per run; closed slots are not reused in this
|
||||
slice, so generation checks are straightforward.
|
||||
* Strict remains conservative; `.mq.md` resource code runs in `pure` (stdout)
|
||||
or `project` (file read) only.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-resource-stream-p3.md` records unit tests for grants,
|
||||
mode refusal, traversal rejection, forged/closed handles, output budget,
|
||||
file chunking/EOF, wire rejection and a `.mq.md` stdout fixture.
|
||||
@@ -0,0 +1,116 @@
|
||||
# ADR-0026: Buffered project-file write streams (CM-A06 extension)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A06, CM-A09, CM-A11), ADR-0024, ADR-0025
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-resource-file-write-p3.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0025 froze the first resource ABI slice with `stdio/stdout` write and
|
||||
project-file **read**. A standard library cannot implement file copy, file
|
||||
format writers or "write then verify" without a write-capable file resource.
|
||||
The RFC's P3 target is explicit `open/read/write/close` for project files in
|
||||
`project` mode; strict must remain fail-closed.
|
||||
|
||||
Direct per-write file I/O would expose partial writes and OS error timing to
|
||||
every caller and would make it easy to leave a half-written project file. This
|
||||
ADR adds the smallest honest write path: an in-memory buffer per file writer,
|
||||
flushed through the existing project path policy on `close`.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Provider shape
|
||||
|
||||
Add a third stream shape to the host resource table (ADR-0025):
|
||||
|
||||
```text
|
||||
resource.open("file", "<project-relative path>", ["write"]) -> Resource
|
||||
```
|
||||
|
||||
* `project` mode only, with an `fs_write` glob grant matching the normalised
|
||||
relative path.
|
||||
* `pure` still refuses file resources by mode check; `strict` still refuses
|
||||
file resources (only the existing stdio console channel is permitted,
|
||||
ADR-0030).
|
||||
* Absolute paths, `..`, symlink escapes and non-regular targets are rejected by
|
||||
`ProjectFs` exactly like the existing `fs.write`.
|
||||
* A writer owns an in-memory byte buffer. `resource.write` appends bytes and
|
||||
returns `{ok:true, value:{count}}`; it does **not** touch the filesystem.
|
||||
* `resource.close` flushes the buffer through `ProjectFs::write_bytes` and then
|
||||
marks the handle closed. If the flush fails, the handle stays open with its
|
||||
buffer intact and the error is reported; the caller may retry or close again.
|
||||
* If the program exits without closing, no file is written. This is a
|
||||
deliberate "no flush on drop" rule: a crash or a missing `close` must not
|
||||
silently publish a partial file.
|
||||
* `resource.write` is rejected on a file reader; `resource.read` is rejected on
|
||||
a file writer; stdout remains write-only. Wrong-direction use is
|
||||
`resource.unsupported`.
|
||||
* A writer buffer is capped at the existing 8 MiB project read/write ceiling
|
||||
(`MAX_READ_BYTES`). Exceeding it is `resource.exhausted` and the append is
|
||||
atomic: the buffer and file are unchanged.
|
||||
|
||||
### 2. Rights and modes
|
||||
|
||||
| Kind | Descriptor | Rights | Mode | Grant |
|
||||
|---|---|---|---|---|
|
||||
| `file` | project-relative path | `["read"]` | project | `fs_read` glob |
|
||||
| `file` | project-relative path | `["write"]` | project | `fs_write` glob |
|
||||
| `stdio` | `stdout` | `["write"]` | pure/project | `console` |
|
||||
|
||||
Mixed rights (`["read","write"]`) and other kinds/rights are
|
||||
`resource.unsupported` in this slice. There is no truncate/append/seek flag:
|
||||
every writer starts empty and replaces the file on flush.
|
||||
|
||||
### 3. Errors, budgets and tracing
|
||||
|
||||
* `resource.closed` for read/write/close on a closed handle (recoverable).
|
||||
* `runtime.resource` for forged/cross-run/stale handles (fatal).
|
||||
* `resource.exhausted` for a buffer over the byte ceiling.
|
||||
* `permission.denied` for a missing mode/grant or a path policy failure.
|
||||
* `resource.unsupported` for a direction/rights combination not implemented.
|
||||
* `resource.write` on a file writer does not charge the program output budget;
|
||||
the buffer is bounded by the file ceiling. `resource.close` performs one
|
||||
`ProjectFs::write_bytes` call and records `resource.close` with the byte
|
||||
count, never the contents.
|
||||
* No partial file is visible on a failed close because the write is a single
|
||||
`std::fs::write` to the validated project path (the same atomicity level as
|
||||
the existing `fs.write`; no temp-file rename is claimed).
|
||||
|
||||
### 4. Explicit non-goals
|
||||
|
||||
* No append/truncate/seek flags, no random access, no directory or metadata
|
||||
operations.
|
||||
* No file write in pure or strict mode.
|
||||
* No network/process/device/clock/random resources.
|
||||
* No generic provider registry (CM-A09), no async/nonblocking I/O.
|
||||
* No claim that `ProjectFs::write_bytes` is crash-atomic; it is a single
|
||||
validated write, not a transactional replace.
|
||||
* No CM-A07 `print` migration in this ADR.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Additive to ADR-0025: existing `file` read and `stdio` write behavior is
|
||||
unchanged; `file` write was previously `resource.unsupported`.
|
||||
* No new `Value` variant, `HostOp`, Markdown node or wire shape.
|
||||
* Error categories/codes are unchanged; the new `resource.exhausted` diagnostic
|
||||
reuses the existing stable `resource.exhausted` code family.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `.mq.md` can now implement file copy and bounded format writers by opening a
|
||||
read stream and a write stream, transferring bytes through `Memory`, and
|
||||
closing both. This is C09's core shape without a `copy_file` Rust builtin.
|
||||
* Buffered close gives deterministic "all or nothing within one write call"
|
||||
behavior and keeps OS write errors at a point where the caller still has a
|
||||
Result path.
|
||||
* Large files require chunked writes; the 8 MiB ceiling is the first-slice
|
||||
limit, not a permanent design.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-resource-file-write-p3.md` records host unit tests
|
||||
(grants, pure/strict refusal, traversal, buffer ceiling, retry-after-failure),
|
||||
CLI tests for write-then-read and missing grants, and the full workspace gate.
|
||||
@@ -0,0 +1,103 @@
|
||||
# ADR-0027: CM-A07 print-migration staging and the Compat-A `.mq.md` stdlib prototype
|
||||
|
||||
Status: **Accepted (Compat-A only; Compat-B/C pending)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A07, CM-A10, P3), ADR-0023, ADR-0024, ADR-0025, ADR-0026
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-20-stdlib-print-compat-a.md`
|
||||
|
||||
## Context
|
||||
|
||||
RFC-CM-0002 §5.4 and CM-A07 require `打印` to stop being a hidden Rust
|
||||
`Value::display()` path. The migration has three compatibility stages:
|
||||
|
||||
* **Compat-A**: a real `.mq.md` print implementation exists and is callable
|
||||
through explicit imports; the legacy builtin remains the default.
|
||||
* **Compat-B**: the `print/打印` name routes to the stdlib implementation
|
||||
(or an alias table maps it), while old behavior is comparison-tested.
|
||||
* **Compat-C**: the legacy `HostOp::Print` formatting path is removed from the
|
||||
public builtin implementation.
|
||||
|
||||
This ADR freezes the staging policy and implements **Compat-A only**. It does
|
||||
not change `打印` resolution, does not remove `HostOp::Print`, and does not
|
||||
claim CM-A07 complete.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Stage policy
|
||||
|
||||
* A stage is only complete when its comparison evidence exists:
|
||||
* A: explicit stdlib functions exist and their output is compared with the
|
||||
builtin for the covered value set;
|
||||
* B: name resolution tests prove a single implementation is selected and old
|
||||
callers still behave;
|
||||
* C: all old goldens pass with no `HostOp::Print` formatting path.
|
||||
* Stages are additive; no stage may silently make a second implementation
|
||||
reachable for the same name.
|
||||
* `strict` behavior is unchanged by stdlib-only code; a stdlib print still
|
||||
needs the console/resource grant.
|
||||
|
||||
### 2. Compat-A implementation
|
||||
|
||||
Add a normal `.mq.md` module package (later moved to the bundled `stdlib/` tree by ADR-0028):
|
||||
|
||||
```text
|
||||
stdlib/format.mq.md # 显示 / 列表显示 / 映射显示 / 表格显示 / 转义文本
|
||||
stdlib/text.mq.md # utf8编码 (Memory bytes)
|
||||
stdlib/io.mq.md # 写文本 / 写值 (resource.open stdout + write/close)
|
||||
stdlib/demo.mq.md # side-by-side comparison with 打印
|
||||
```
|
||||
|
||||
`io.写值(值)` = `format.显示(值) + "\n"` → `text.utf8编码` → `Memory` →
|
||||
`resource.open("stdio","stdout",["write"])` → `resource.write` → `close`.
|
||||
This uses only the public core/resource contracts; it is ordinary imported
|
||||
`.mq.md` code, not a new builtin.
|
||||
|
||||
Covered value types: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, `List`,
|
||||
`Map`, `Table`. Formatting matches the builtin `Value::display()` for the
|
||||
checked corpus, including Text quoting/escaping inside collections, canonical
|
||||
map key order and table layout.
|
||||
|
||||
**Not covered yet**: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The
|
||||
language currently has no public observation for the module/name/handle
|
||||
identity needed to reproduce `Value::display()`/`repr()` for those types.
|
||||
Adding such observation would be a CM-A04 extension and is out of scope.
|
||||
|
||||
### 3. Compatibility rules
|
||||
|
||||
* `打印` remains the default builtin in this slice. `io.写值` is only reachable
|
||||
through an explicit import alias; there is no implicit module, bundled stdlib
|
||||
resolution or global name shadowing.
|
||||
* No `HostOp` is added or removed. Both paths write through the console/output
|
||||
budget; the stdlib path additionally goes through the resource ABI.
|
||||
* Stdlib modules are ordinary Markdown documents: no special parser, no
|
||||
embedded Rust formatter, no direct formatting syscall.
|
||||
* A future Compat-B ADR must define name precedence, bundled module lookup,
|
||||
stdlib version/digest (CM-A10) and failure behavior; this ADR does not.
|
||||
|
||||
### 4. Non-goals
|
||||
|
||||
* No `print` alias routing, no removal of `HostOp::Print`.
|
||||
* No bundled stdlib install/version/digest or offline distribution.
|
||||
* No formatting of Formula/Code/FunctionRef/PluginHandle.
|
||||
* No locale-aware, width-aware or float-format changes.
|
||||
* No change to Markdown rendering, `strict`, or the wire/ABI.
|
||||
* No claim that the examples directory is the shipped standard library.
|
||||
|
||||
## Consequences
|
||||
|
||||
* The target call chain (`format.display` → `utf8_encode` → `write_all` →
|
||||
`resource.write`) is exercised end to end in `.mq.md` and can be compared to
|
||||
the builtin.
|
||||
* The missing observation for `Formula/Code/FunctionRef/PluginHandle` is now an
|
||||
explicit, documented blocker for Compat-B rather than an implicit assumption.
|
||||
* Because the stdlib is not bundled, ordinary users still get the builtin; the
|
||||
prototype does not yet reduce the runtime's formatting surface.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-stdlib-print-compat-a.md` records the side-by-side
|
||||
demo (`io.写值` vs `打印` for a nested list and a table), the CLI test asserting
|
||||
byte-identical lines, and the full workspace gate. It also records the covered
|
||||
and uncovered value types.
|
||||
@@ -0,0 +1,151 @@
|
||||
# ADR-0028: Bundled standard library, `std:` imports and digest verification (CM-A10)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A10, CM-A07, P2/P3), ADR-0027
|
||||
Specs: `spec/analysis-v2.md`, `spec/references.md`
|
||||
Evidence: `docs/evidence/2026-11-20-bundled-stdlib-cm-a10.md`
|
||||
|
||||
## Context
|
||||
|
||||
CM-A07 Compat-A put a real `.mq.md` print chain under `examples/stdlib/`, but it
|
||||
was reachable only by relative path from a demo in the same directory. A
|
||||
standard library that ordinary projects can use offline needs:
|
||||
|
||||
* a fixed bundle shipped with the CLI;
|
||||
* an explicit import spelling that cannot be shadowed by a project file;
|
||||
* a content identity so a different bundle with the same version cannot
|
||||
masquerade;
|
||||
* no network, Git or package installation required at run time.
|
||||
|
||||
The RFC requires the official distribution to carry a tested stdlib version
|
||||
offline, while allowing independent releases. This ADR freezes the **bundled
|
||||
first slice**. A separate `marqdo-stdlib` repository is intentionally deferred
|
||||
by ADR-0039: the bundle stays in this repository while the architecture is
|
||||
small and clear; the deterministic package remains extraction-ready.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Bundle location
|
||||
|
||||
The official bundle is the `stdlib/` tree shipped next to the CLI (or vendored
|
||||
at the repository root during development). Discovery order:
|
||||
|
||||
1. `MARQDO_STDLIB_DIR` — explicit override; if set and invalid, loading fails
|
||||
hard (no silent fallback);
|
||||
2. `<executable-dir>/stdlib` if it exists;
|
||||
3. the development tree `<repo>/stdlib` for source builds.
|
||||
|
||||
If no bundle is found, a project that uses `std:` fails with a clear error; a
|
||||
project that does not use `std:` is unaffected. No network, Git or package
|
||||
manager is consulted for a bundled stdlib.
|
||||
|
||||
### 2. Explicit `std:` imports only
|
||||
|
||||
Front-matter imports may name a stdlib module as:
|
||||
|
||||
```yaml
|
||||
import:
|
||||
io: "std:io"
|
||||
fmt: "std:format"
|
||||
parser: "std:data/json"
|
||||
```
|
||||
|
||||
The target after `std:` is a **package-relative** path inside the bundle.
|
||||
`..`, absolute paths and `std:` targets that escape the bundle are rejected;
|
||||
`.mq.md` is appended when the target has no extension. `std:` is resolved
|
||||
before relative/alias imports and never searches the project or dependencies.
|
||||
There is no implicit `std` namespace, no global name, and no auto-import.
|
||||
|
||||
Module ids use a stable package instance:
|
||||
|
||||
```text
|
||||
std:marqdo-stdlib@<version>#<sha256-hex>
|
||||
```
|
||||
|
||||
so the same logical module from a different bundle is a different instance.
|
||||
|
||||
### 3. Content digest
|
||||
|
||||
`stdlib/stdlib.json` carries the name, version and digest. The digest is:
|
||||
|
||||
```text
|
||||
sha256( for each sorted *.mq.md relative path:
|
||||
path_utf8, NUL, file_bytes, NUL )
|
||||
```
|
||||
|
||||
It excludes `stdlib.json`, README files and non-`.mq.md` files. The loader
|
||||
recomputes it before loading any stdlib module and fails closed on mismatch or
|
||||
a missing manifest. A bundle is therefore identified by **name + version +
|
||||
exact markdown bytes**, not by a path or a version string alone.
|
||||
|
||||
### 4. Visibility and analysis
|
||||
|
||||
Analysis v2 records the verified bundle identity under `project.stdlib`:
|
||||
|
||||
```json
|
||||
{"version":"0.1.0","digest":"sha256:<hex>",
|
||||
"instance_id":"std:marqdo-stdlib@0.1.0#<hex>"}
|
||||
```
|
||||
|
||||
It is `null` when no bundle was available. A project that imports `std:` always
|
||||
has a non-null identity. Trace/diagnostics keep the real stdlib source spans;
|
||||
the bundle is normal Markdown, not a special-cased module type.
|
||||
|
||||
### 5. Frozen-CLI closure lane
|
||||
|
||||
The first closure test builds the CLI once, hashes the executable, runs a
|
||||
project importing `std:io`, then re-hashes the executable and asserts equality.
|
||||
It also asserts:
|
||||
|
||||
* the project uses only `.mq.md` and standard imports, no `foreign`/`plugin`
|
||||
calls;
|
||||
* a tampered bundle fails the digest check before any module executes;
|
||||
* Analysis records the verified digest.
|
||||
|
||||
This is a local gate, not a replacement for a remote CI lane.
|
||||
|
||||
### 6. Versioning
|
||||
|
||||
The bundle version is `0.1.0` for this slice. A CLI may load a bundle whose
|
||||
version is declared compatible by this ADR; the RFC's full SemVer/digest
|
||||
contract and independent release workflow remain CM-A10 follow-up. A future
|
||||
core feature that the bundle requires must be reflected in both the bundle
|
||||
version and the CLI's compatibility table before release.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* Creating/publishing a separate `cflmy/marqdo-stdlib` repository now; see
|
||||
ADR-0039 for the deliberate deferral and future extraction triggers.
|
||||
* Automatic download/update, Git fetching, or manifest-driven stdlib
|
||||
installation.
|
||||
* A second plugin/provider system or a second module search path.
|
||||
* Compat-B/C name routing of `打印` (ADR-0027) — the bundle does not change
|
||||
builtin resolution.
|
||||
* Windows/macOS packaging layout; only the source-tree/executable-adjacent
|
||||
layout is exercised locally.
|
||||
|
||||
## Offline packaging
|
||||
|
||||
`tools/pack-stdlib.py` produces a deterministic `marqdo-stdlib-<version>.tar.gz`
|
||||
plus a schema-tagged descriptor (`tree_digest`, `archive_sha256`, file list) for
|
||||
the future separate repository/release workflow. The archive is not loaded by
|
||||
the CLI; the in-repo `stdlib/` tree remains the verified development/bundled
|
||||
source. Publishing/signing the external repository is still an external step.
|
||||
|
||||
## Consequences
|
||||
|
||||
* Any project can `import: {io: "std:io"}` and use the print prototype
|
||||
offline; `打印` remains the default builtin.
|
||||
* A changed stdlib file without a digest update is rejected before execution.
|
||||
* The bundle is still an in-repo vendored package. Independent repository
|
||||
release and CI signing are explicitly tracked as remaining CM-A10 work.
|
||||
* Analysis artifacts and a frozen binary can be tied to the exact stdlib
|
||||
digest, which is the prerequisite for Compat-B and reproducible releases.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-bundled-stdlib-cm-a10.md` records the `std:` import
|
||||
test, digest-tamper rejection, Analysis identity assertion, no-foreign policy
|
||||
scan and frozen binary hash equality.
|
||||
@@ -0,0 +1,86 @@
|
||||
# ADR-0029: Full-value stdlib formatting and handle observation (CM-A04/A07)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A04, CM-A07, P2/P3), ADR-0023, ADR-0027, ADR-0028
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-20-full-value-formatting.md`
|
||||
|
||||
## Context
|
||||
|
||||
Compat-A (ADR-0027) formats Unit/None/Bool/Int/Float/Text/List/Map/Table, but
|
||||
builtin `打印` also formats Formula, Code, FunctionRef and PluginHandle. A
|
||||
Compat-B name route cannot be honest while the stdlib silently differs on
|
||||
those types. The missing pieces are the **identity/observation** fields needed
|
||||
to reproduce `Value::repr()` for function references and plugin handles;
|
||||
Formula and Code already expose `latex/display/unit` and
|
||||
`language/source/content_hash/unit`.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Read-only identity members
|
||||
|
||||
Add member reads (no new value types, no mutation, no handles):
|
||||
|
||||
| Value | Members |
|
||||
|---|---|
|
||||
| `FunctionRef` | `module: Text`, `name: Text` |
|
||||
| `PluginHandle` | `plugin: Text`, `export: Text` |
|
||||
|
||||
The host-assigned numeric `PluginHandle.id` remains private; only the manifest
|
||||
plugin name and export name are exposed, which are already visible in
|
||||
`Value::repr()` and are not a capability token. `Memory` and `Resource` remain
|
||||
opaque and expose no fields. Unknown members are `runtime.unknown_field`.
|
||||
|
||||
### 2. Stdlib formatting covers all 15 value types
|
||||
|
||||
`stdlib/format.mq.md` gains branches for:
|
||||
|
||||
* `Formula`: `Formula(<debug-quoted latex>)`;
|
||||
* `Code`: `Code(<debug-quoted language>, <utf8-byte-length> bytes)`
|
||||
(the byte length is computed with `stdlib/text.mq.md`, not `长度`, which is
|
||||
Unicode scalar count);
|
||||
* `FunctionRef`: `FunctionRef(module::name)`;
|
||||
* `PluginHandle`: `PluginHandle(plugin::export)`;
|
||||
* `Memory`/`Resource`: the literal `Memory`/`Resource`;
|
||||
* `Unit`: `Unit`.
|
||||
|
||||
The existing `Text`/collection escaping (quotes, backslash, newline, tab, CR)
|
||||
is reused. Control characters outside those escapes are not emitted as
|
||||
`\u00XX` in this slice; that is a formatting limitation, not a security
|
||||
boundary, and does not affect the current corpus.
|
||||
|
||||
### 3. Compatibility
|
||||
|
||||
* The new members are additive observations; no existing expression changes.
|
||||
* `PluginHandle.id` is not observable and cannot be used to forge a handle.
|
||||
* `打印` remains the builtin default; this ADR only makes the stdlib formatter
|
||||
capable of matching it for all value types. Compat-B routing is a separate
|
||||
decision (ADR-0030).
|
||||
* Formula/Code provenance spans are not exposed as members; only the fields
|
||||
already present in the value wire/repr are.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* No mutable handle inspection, no resource/plugin capability escalation.
|
||||
* No `\u00XX` escaping of arbitrary control characters.
|
||||
* No change to `Value::display()`/`repr()` themselves; the stdlib must match
|
||||
the existing behavior, not redefine it.
|
||||
* No print name routing in this ADR.
|
||||
|
||||
## Consequences
|
||||
|
||||
* The stdlib formatter can reproduce `打印` output for all 15 current value
|
||||
types, removing the Compat-B blocker documented in ADR-0027.
|
||||
* Function/plugin identity becomes readable in `.mq.md`; this is already public
|
||||
in display/repr and does not reveal the host-only numeric handle id.
|
||||
* Code byte length performs one UTF-8 encode into a temporary `Memory`; large
|
||||
code values cost an extra allocation during formatting. A future core
|
||||
observation could avoid it without changing output.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-full-value-formatting.md` records the member-read
|
||||
tests, a Formula/Code/FunctionRef formatting comparison against builtin
|
||||
`打印`, and the full workspace gate.
|
||||
@@ -0,0 +1,101 @@
|
||||
# ADR-0030: Compat-B `打印` routing through the bundled stdlib
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A07, CM-A11, P3), ADR-0027, ADR-0028, ADR-0029
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-print-compat-b.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0027 implemented Compat-A: the stdlib print chain exists behind explicit
|
||||
imports, but builtin `打印` still runs `Value::display()` in Rust. ADR-0029
|
||||
removed the formatting gap: the stdlib can reproduce `打印` for all 15 value
|
||||
types. ADR-0028 made a verified stdlib bundle available offline. The remaining
|
||||
Compat-B requirement is to route the **same name** to one implementation and
|
||||
prove the old callers behave, before Compat-C removes the legacy formatting
|
||||
path.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. One program-level override
|
||||
|
||||
`Program` gains an optional `print_override: FunctionId`. The loader sets it to
|
||||
the bundled `io.写值` function when:
|
||||
|
||||
* the verified bundle is available, and
|
||||
* at least one loaded module calls `打印` or `print` as a bare builtin call.
|
||||
|
||||
The stdlib `io` module is injected only in that case. A program without print
|
||||
calls keeps the same module graph and Analysis output as before. If no bundle
|
||||
is available, the loader leaves the override unset and the legacy builtin is the
|
||||
fallback; an official release always ships the bundle.
|
||||
|
||||
### 2. Resolution order
|
||||
|
||||
At a call site, name resolution is:
|
||||
|
||||
1. a local variable holding a callable (`f = ...; f(x)`) — local shadow wins;
|
||||
2. `打印`/`print` with `Program.print_override` set — call the stdlib function;
|
||||
3. the existing builtin table;
|
||||
4. a same-module/imported function declaration.
|
||||
|
||||
This is a controlled compatibility route for exactly two names. It is not a
|
||||
general name-shadowing mechanism and does not add a new namespace.
|
||||
|
||||
### 3. Runtime behavior
|
||||
|
||||
The stdlib function is a normal Marqdo function: its source spans, fuel,
|
||||
frames and trace are the stdlib file's. It formats through
|
||||
`stdlib/format.mq.md`, encodes UTF-8 through `stdlib/text.mq.md`, and writes
|
||||
through `resource.open("stdio","stdout",["write"])` plus `resource.write`.
|
||||
`打印` still returns `Unit` and appends exactly one newline.
|
||||
|
||||
The console/output-budget checks remain in the host: the stdlib path uses the
|
||||
same console grant and output budget as the legacy path. Resource operations
|
||||
are recorded as Trace `effect` events (`resource.open/read/write/close`); the
|
||||
legacy `HostOp::Print` effect does not appear in Compat-B runs.
|
||||
|
||||
### 4. Strict mode and CM-A11
|
||||
|
||||
Strict L1 already permits the console channel. Compat-B needs stdout writes in
|
||||
strict so the same print implementation works there. Therefore strict mode
|
||||
permits **only** `stdio/stdout` write with the console grant; project-file,
|
||||
network, process and device resource opens remain refused, and no new provider
|
||||
is enabled. This is the existing console capability reached through the
|
||||
resource ABI, not a new OS capability. File resources stay fail-closed.
|
||||
|
||||
### 5. Compatibility and fallback
|
||||
|
||||
* Output for all 15 value types is the ADR-0029 formatter and is
|
||||
comparison-tested against the legacy path.
|
||||
* Trace shape changes from one `print` effect to resource effects. Consumers
|
||||
that hard-code `operation == "print"` must accept `resource.*`; this is a
|
||||
documented P3 migration.
|
||||
* `check`/`analysis` include stdlib modules for print-using programs only.
|
||||
The human-readable `checked N module(s)` count excludes stdlib modules so
|
||||
bundle internals do not change the project count.
|
||||
* Compat-C (removing `HostOp::Print` and `Value::display()` formatting from the
|
||||
public builtin) is **not** part of this ADR; it is implemented by ADR-0031,
|
||||
which supersedes the "legacy fallback" sentence above.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* No routing for `长度`, `类型`, `文本`, table or any other builtin.
|
||||
* No bundled stdlib upgrade workflow; the bundle version stays `0.1.0`.
|
||||
* No removal of `HostOp::Print`, `Value::display()` or the legacy builtin.
|
||||
* No new OS isolation claim; strict still refuses file/network/process
|
||||
resources.
|
||||
* No guarantee for a CLI installed without `stdlib/`; that configuration keeps
|
||||
legacy print and is not the official layout.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-print-compat-b.md` records:
|
||||
|
||||
* a print program whose trace contains `resource.write` and no `print` effect;
|
||||
* all semantic goldens and CLI print tests passing byte-identically;
|
||||
* lazy stdlib injection: no print → no stdlib modules; print → stdlib modules
|
||||
and `project.stdlib` identity;
|
||||
* host tests for strict stdio-only permission and file refusal.
|
||||
@@ -0,0 +1,95 @@
|
||||
# ADR-0031: Compat-C — remove `HostOp::Print` and the legacy formatting path
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A07, P3), ADR-0027, ADR-0029, ADR-0030
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-print-compat-c.md`
|
||||
|
||||
## Context
|
||||
|
||||
Compat-A produced a stdlib print chain; Compat-B (ADR-0030) routed
|
||||
`打印/print` to the bundled `io.写值` and proved output parity for all 15 value
|
||||
types. The legacy path (`HostOp::Print` → `Value::display()` → output buffer)
|
||||
remained as a no-bundle fallback and a comparison reference. Compat-C removes
|
||||
the **public builtin formatting path** so a print call can no longer silently
|
||||
fall back to a Rust formatter.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Remove the host operation
|
||||
|
||||
* `HostOp::Print` and `StdHost::op_print` are deleted. There is no host
|
||||
operation that formats a value.
|
||||
* `打印/print` remains a **builtin name** for static arity validation and
|
||||
compatibility diagnostics. Its implementation no longer writes anything; if
|
||||
the runtime reaches it (no `Program.print_override`), it returns the
|
||||
structured `kernel.unavailable` error:
|
||||
"打印 requires the bundled stdlib print module; install the official CLI
|
||||
bundle."
|
||||
* The official CLI ships the verified bundle, so a print-using program always
|
||||
loads `stdlib/io.mq.md`, sets `print_override`, and executes `io.写值`.
|
||||
A CLI installed without `stdlib/` cannot print; that layout is not official.
|
||||
|
||||
### 2. Formatting ownership
|
||||
|
||||
All print formatting, UTF-8 encoding and byte output now live in `.mq.md`:
|
||||
|
||||
```text
|
||||
打印(value)
|
||||
-> stdlib/io.写值
|
||||
-> stdlib/format.显示
|
||||
-> stdlib/text.utf8编码
|
||||
-> resource.open("stdio","stdout",["write"])
|
||||
-> resource.write
|
||||
-> resource.close
|
||||
```
|
||||
|
||||
`Value::display()` still exists for CLI result rendering (`call --json`,
|
||||
diagnostics), but it is **not** reachable from the print builtin. No `HostOp`
|
||||
accepts a value for display.
|
||||
|
||||
### 3. Trace and budgets
|
||||
|
||||
Print produces only resource effects (`resource.open/write/close`) and stdlib
|
||||
semantic events. There is no `print` effect. Output budget and console grant
|
||||
are enforced at the resource/stdio boundary exactly as before.
|
||||
|
||||
### 4. Compatibility
|
||||
|
||||
* All semantic goldens, CLI print tests and the full-value formatting demo pass
|
||||
unchanged.
|
||||
* The runtime embedding API (`marqdo_runtime::run`) no longer prints unless the
|
||||
caller installs a `Program.print_override`; a direct runtime program that
|
||||
calls `打印` gets the structured no-bundle error. This is a **breaking change
|
||||
for embedders** and is recorded here; the CLI loader maintains the official
|
||||
behavior.
|
||||
* `callable.rs` still advertises the builtin name for diagnostics and arity;
|
||||
this is not a second implementation.
|
||||
* Compat-A/B evidence remains as history; its "legacy fallback" wording is
|
||||
superseded by this ADR.
|
||||
|
||||
### 5. Non-goals
|
||||
|
||||
* Removing the `打印/print` builtin name, alias, or arity validation.
|
||||
* Removing `Value::display()` for CLI result output, diagnostics or format
|
||||
helpers.
|
||||
* Supporting print without the bundle; no implicit formatting fallback.
|
||||
* Changing Resource ABI, strict mode or stdlib version.
|
||||
|
||||
## Consequences
|
||||
|
||||
* It is now impossible for `打印` to execute a hidden Rust formatter; the only
|
||||
print implementation on an official installation is `.mq.md`.
|
||||
* Embedders that build a `Program` directly must load the stdlib print module
|
||||
and set `print_override` (the semantic test harness does this).
|
||||
* Trace consumers see resource effects, not a print effect.
|
||||
* A missing/corrupt bundle fails closed for print instead of silently using a
|
||||
different implementation.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-print-compat-c.md` records: no `HostOp::Print` or
|
||||
`op_print` references; the no-bundle structured error; trace still shows
|
||||
resource effects; all 419 tests and the full-value demo pass.
|
||||
@@ -0,0 +1,101 @@
|
||||
# ADR-0032: GFM Table logical value, stdlib algorithms and T0 storage (CM-A08)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A08, P2/P4), ADR-0023
|
||||
Specs: `spec/evaluation.md`, `spec/mapping.md`
|
||||
Evidence: `docs/evidence/2026-11-20-table-cm-a08.md`
|
||||
|
||||
## Context
|
||||
|
||||
RFC-CM §6 separates three layers: the GFM source document (canonical Markdown
|
||||
AST), the logical `Table` value, and the physical storage. The current runtime
|
||||
uses `Value::Table { header: Vec<String>, rows: Vec<Vec<Value>> }`, which is
|
||||
correct but concrete. Table algorithms already exist as `.mq.md` conformance
|
||||
fixtures (filter/sort/join/group), but they are not shipped as a reusable
|
||||
stdlib module. CM-A08 requires that high-level table algorithms live in the
|
||||
standard library and that physical storage can change without changing
|
||||
observable semantics.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Three layers stay separate
|
||||
|
||||
* **GFM source**: unchanged. The Markdown parser owns `table/row/cell`,
|
||||
spans, laziness and rendering. This ADR adds no table execution to Markdown.
|
||||
* **Logical value**: `Value::Table` keeps ordered headers, ordered rows, cell
|
||||
values, width validation, equality/display/wire semantics and value-copy
|
||||
behavior (`b = a` does not alias).
|
||||
* **Physical storage**: T0 remains `Vec<Vec<Value>>`. This is an implementation
|
||||
detail; a future T1 controlled interface or T2 column store must preserve the
|
||||
logical contract. No column store, view or Memory-backed table is introduced
|
||||
here.
|
||||
|
||||
### 2. Standard-library table module
|
||||
|
||||
Add `stdlib/data/table.mq.md`, importable as `std:data/table`:
|
||||
|
||||
| Function | Semantics |
|
||||
|---|---|
|
||||
| `选择列(数据, 列)` | new table with one column, original unchanged |
|
||||
| `过滤数值大于(数据, 列, 阈值)` | rows whose numeric cell is `>` threshold |
|
||||
| `排序(数据, 列)` | insertion sort by a numeric column; new table |
|
||||
| `插入有序(已排序行, 行, 列)` | helper; value-returning |
|
||||
| `追加行(数据, 行)` | new table with one appended row (width validated by `表`) |
|
||||
| `设置单元格(数据, 行, 列, 值)` | rebuilds rows; original unchanged |
|
||||
| `连接(左, 右, 左列, 右列)` | equality join; headers concatenated |
|
||||
| `分组(数据, 键列, 值列)` | Text key -> List of values |
|
||||
| `转GFM(数据)` | render the logical table as a GFM pipe table; `\|`/newline cells escaped |
|
||||
|
||||
All functions use only `table.columns/row/cell`, `list.*`, `map.*` and `表`;
|
||||
none is a new builtin or `HostOp`. The module is ordinary `.mq.md`, loaded
|
||||
through the verified `std:` bundle and its digest.
|
||||
|
||||
### 3. Semantics fixed by this ADR
|
||||
|
||||
* Every operation returns a **new** table; inputs are never mutated.
|
||||
* Row/column coordinates are non-negative; out-of-range is `runtime.index`.
|
||||
* Numeric algorithms use `Int`/`Float` comparison of cell values; passing a
|
||||
Text cell where a number is required is a runtime type error, not a silent
|
||||
coercion.
|
||||
* GFM cell content such as `**不执行**` or backticked code remains inert data;
|
||||
no table function parses a cell as executable source.
|
||||
* Empty tables, zero rows/columns, duplicate headers and Unicode headers keep
|
||||
their existing behavior; `表` still validates row width.
|
||||
* Table source spans and document addresses are unchanged; a rebuilt Table is
|
||||
a logical value, not a rewrite of the source document.
|
||||
* Cross-storage compatibility: a `.mq.md` columnar representation (List/Map of
|
||||
columns) must produce the same headers/rows/cells as the T0 row-store Table
|
||||
for the supported operations (fixture 72). This does not migrate the Rust
|
||||
storage; it proves the logical contract is storage-independent.
|
||||
|
||||
### 4. What is deliberately not done
|
||||
|
||||
* No physical storage migration (T1/T2/T3), column store, shared view or
|
||||
Memory-backed table.
|
||||
* No `DataFrame`/`TableView` parallel types; `Value::Table` remains the only
|
||||
table value.
|
||||
* No per-operation Rust builtin (`table.filter`, `table.join`, ...).
|
||||
* No mutable table update; `设置单元格` rebuilds.
|
||||
* No sorting/grouping of heterogeneous mixed-type columns beyond the explicit
|
||||
numeric/Text contracts.
|
||||
* No change to GFM parsing or rendering.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A project can `import: {table: "std:data/table"}` and filter/sort/join/group
|
||||
tables without new Rust code; this is the P4 migration's first concrete
|
||||
stdlib artifact.
|
||||
* The physical representation remains replaceable because algorithms depend
|
||||
only on the logical observation/construction API.
|
||||
* Large-table performance is O(n²) for insertion sort/join and copies rows;
|
||||
P6 performance work (column store, persistent vectors) can change storage
|
||||
without changing these outputs.
|
||||
* The bundle digest must be regenerated when this module changes.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-table-cm-a08.md` records the module self-test and a
|
||||
CLI project importing `std:data/table` that checks filter/select/append/set/
|
||||
join/group results and input immutability, plus the full workspace gate.
|
||||
@@ -0,0 +1,123 @@
|
||||
# ADR-0033: In-process ResourceProvider v1 (CM-A09 first slice)
|
||||
|
||||
Status: **Accepted (in-process slice only)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A11, P3/P5), ADR-0025, ADR-0026, ADR-0031
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-provider-abi-cm-a09.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0025/0026 hard-code two resource kinds (`stdio/stdout`, project `file`) in
|
||||
the host. CM-A09 requires a versioned, discoverable provider contract so a new
|
||||
device/OS endpoint can be added **without** changing the expression parser,
|
||||
`Value` model, runtime dispatch or `HostOp` enum. The RFC warns against a
|
||||
second shadow plugin ecosystem: providers should reuse the same host/ABI
|
||||
infrastructure where possible.
|
||||
|
||||
The full C ABI/Worker resource protocol is a larger artifact (C structs, wire
|
||||
buffers, ownership, process isolation). This ADR freezes the **in-process
|
||||
trusted provider** slice first; a Worker/C ABI binding can implement the same
|
||||
contract later. In-process providers are explicitly **not** a sandbox.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Provider contract v1
|
||||
|
||||
```rust
|
||||
pub trait ResourceProvider: Send {
|
||||
fn name(&self) -> &str; // stable grant name
|
||||
fn protocol_version(&self) -> u32 { 1 } // must be 1 in this slice
|
||||
fn kinds(&self) -> &[&str]; // resource kinds it serves
|
||||
fn open(&mut self, kind: &str, descriptor: &str, rights: &[String],
|
||||
span: Option<Span>) -> Result<u64, RuntimeError>; // provider token
|
||||
fn read(&mut self, token: u64, max_bytes: usize,
|
||||
span: Option<Span>) -> Result<ResourceRead, RuntimeError>;
|
||||
fn write(&mut self, token: u64, bytes: &[u8],
|
||||
span: Option<Span>) -> Result<usize, RuntimeError>;
|
||||
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError>;
|
||||
}
|
||||
```
|
||||
|
||||
* `ResourceRead { bytes, eof }` is the same host type used by built-in streams.
|
||||
* The **host** still issues the opaque language `ResourceHandle`; the provider
|
||||
token is private to the provider and never enters a `Value`, wire message or
|
||||
trace.
|
||||
* Bytes cross the provider boundary as copied slices for the duration of a
|
||||
call; no provider may retain a `Memory` pointer across calls or after a
|
||||
`grow`.
|
||||
* `open` returns a provider token; `read/write/close` must reject unknown,
|
||||
stale or closed tokens with a structured error. The host checks the language
|
||||
handle before dispatching.
|
||||
|
||||
### 2. Registration
|
||||
|
||||
`StdHost::register_resource_provider(Box<dyn ResourceProvider>)`:
|
||||
|
||||
* rejects `protocol_version != 1`;
|
||||
* rejects an empty `kinds()` list;
|
||||
* rejects a provider name or kind that is empty, `stdio`, `file`, or already
|
||||
registered (no silent shadowing);
|
||||
* stores the provider under each declared kind and its stable name for grants.
|
||||
|
||||
Registration is a host/embedding API, not a language import. Importing a
|
||||
`.mq.md` package never registers or authorizes a provider. Installation and
|
||||
authorization remain separate (INV-06).
|
||||
|
||||
### 3. Authorization and modes
|
||||
|
||||
* `Capabilities` gains `resource_providers: Vec<String>` and
|
||||
`allow_resource_provider(name)`; a matching name is required at
|
||||
`resource.open` time.
|
||||
* `project` only. `pure` and `strict` refuse provider kinds even when
|
||||
registered; strict remains fail-closed (CM-A11). Existing stdio/file rules
|
||||
are unchanged.
|
||||
* A provider's own rights/descriptor validation is not a substitute for the
|
||||
grant; denial is `permission.denied` with a diagnostic.
|
||||
* In-process provider code shares the host address space and is **not**
|
||||
sandboxed. Only explicitly trusted providers may be registered; the API
|
||||
documentation and ADR state this.
|
||||
|
||||
### 4. Runtime surface
|
||||
|
||||
No new `Value`, `HostOp`, Markdown node or grammar. A language program still
|
||||
writes:
|
||||
|
||||
```text
|
||||
resource.open("<kind>", "<descriptor>", ["read"])
|
||||
```
|
||||
|
||||
The existing `resource.read/write/close` and `Result` shapes are reused. The
|
||||
provider name is used only for the grant check and trace metadata; the language
|
||||
does not need to know it beyond the grant.
|
||||
|
||||
### 5. What remains for CM-A09
|
||||
|
||||
* C ABI / Worker resource transport (buffer handoff, ownership, crash isolation,
|
||||
version negotiation through the existing plugin ABI).
|
||||
* Provider discovery/installation from packages, signatures and platform
|
||||
support matrices.
|
||||
* Network/socket/device providers themselves.
|
||||
* Strict-mode provider verification.
|
||||
|
||||
This ADR is a contract for trusted in-process providers, not a second plugin
|
||||
ecosystem.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A new endpoint can be implemented as a `ResourceProvider` and registered
|
||||
without touching parser/runtime/`HostOp`; language code uses the same
|
||||
`resource.*` calls.
|
||||
* The provider token cannot be forged into a language handle; only the host
|
||||
creates handles.
|
||||
* In-process providers can crash/block the host; they are trusted. Worker C ABI
|
||||
is still needed for untrusted extensions.
|
||||
* `Capabilities` grows one grant list; CLI exposure is deferred because there
|
||||
is no language-level provider installer yet.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-provider-abi-cm-a09.md` records a mock read/write
|
||||
provider registered without parser/runtime changes, grant/pure/strict denial,
|
||||
protocol/kind conflict registration errors and the full workspace gate.
|
||||
@@ -0,0 +1,87 @@
|
||||
# ADR-0034: Blocking TCP resource endpoint v1 (CM-A09/P5 first slice)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A11, P5), ADR-0025, ADR-0033
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-tcp-endpoint.md`
|
||||
|
||||
## Context
|
||||
|
||||
CM-A09/P5 require a real external endpoint reached through the stable resource
|
||||
ABI, not a new `HostOp` or Rust builtin per application. The in-process
|
||||
provider slice (ADR-0033) covers trusted adapters; it does not exercise a real
|
||||
OS byte stream. This ADR adds the first OS endpoint: a blocking TCP stream.
|
||||
|
||||
The scope is deliberately narrow: HTTP/1.1, TLS, DNS, UDP, sockets as a broad
|
||||
API and nonblocking/async I/O remain standard-library or later work.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Resource kind and descriptor
|
||||
|
||||
```text
|
||||
resource.open("tcp", "<ip-literal>:<port>", ["read", "write"]) -> Resource
|
||||
```
|
||||
|
||||
* descriptor must parse as a `SocketAddr` containing an **IP literal**. DNS
|
||||
names are rejected with `resource.unsupported`; no resolver is called.
|
||||
* rights must be exactly `["read","write"]` in this build. A read-only or
|
||||
write-only TCP handle is future work.
|
||||
* `resource.read`/`write`/`close` behave like other streams:
|
||||
* read returns `{count, eof}`; `count=0` is transport EOF;
|
||||
* write returns a possibly short count;
|
||||
* close calls `shutdown(Both)` and marks the handle closed;
|
||||
* closed/forged/stale handles use the existing `resource.closed` /
|
||||
`runtime.resource` rules.
|
||||
* connect, read and write have 5-second timeouts. This is blocking I/O on the
|
||||
interpreter thread; no async runtime is introduced.
|
||||
|
||||
### 2. Authorization
|
||||
|
||||
* `project` mode only. `pure` and `strict` refuse TCP (`permission.denied`);
|
||||
strict remains fail-closed (CM-A11), and the G-C lane verifies the refusal.
|
||||
* `Capabilities.check_net(ip)` requires an exact IP-literal grant. The CLI adds
|
||||
`--allow-net <IP>` (repeatable); `--allow-net` is refused alongside other
|
||||
external grants in pure/strict preflight.
|
||||
* Granting a hostname or a network range does not authorize a different IP;
|
||||
there is no DNS-based allowlist bypass because DNS is not used.
|
||||
* No TLS, no proxy, no HTTP parsing in the host. `.mq.md` code composes
|
||||
protocols over the byte stream.
|
||||
|
||||
### 3. Trace
|
||||
|
||||
`resource.open/read/write/close` effect events are recorded. The trace records
|
||||
the IP:port for open but never payload bytes or credentials. A TCP endpoint is
|
||||
an external effect and is not replayed as pure computation.
|
||||
|
||||
### 4. Explicit non-goals
|
||||
|
||||
* DNS, TLS, HTTP/1.1 or HTTP/2 host implementations, proxies, redirects.
|
||||
* UDP, ICMP, raw sockets, socket options, nonblocking/async/select.
|
||||
* File descriptors exposed to the language, `setsockopt`, `bind`/`listen` server
|
||||
sockets.
|
||||
* Strict-mode TCP and any claim that in-process `std::net` is OS isolation.
|
||||
* Network provider package installation or a generic socket registry.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `.mq.md` can implement a line protocol/HTTP client parser on top of the TCP
|
||||
byte stream using the existing Memory/Text/Result primitives. The bundled
|
||||
`stdlib/net/http.mq.md` provides HTTP/1.1 response parsing (status, headers,
|
||||
chunked transfer) and request construction; `http_over_tcp.rs` runs a local
|
||||
HTTP exchange through `resource.open("tcp", ...)` and the module.
|
||||
* The runtime still has no `net.request` builtin or new `HostOp`; a CLI test
|
||||
opens a local echo server, writes/reads `ping` through `resource.*`, and
|
||||
closes.
|
||||
* Blocking timeouts bound a hung peer but do not preempt a blocking syscall
|
||||
deeper than the OS timeout; this is documented.
|
||||
* Network tests use loopback only; external network access is not required for
|
||||
the gate.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-tcp-endpoint.md` records host tests (loopback echo,
|
||||
grant/pure/strict/DNS refusal) and a CLI `.mq.md` echo test with
|
||||
`--allow-net 127.0.0.1`, plus the full workspace gate.
|
||||
@@ -0,0 +1,92 @@
|
||||
# ADR-0035: Worker resource-provider transport v1
|
||||
|
||||
Status: **Accepted (trusted helper transport; C ABI binding still open)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A11, P5), ADR-0033, ADR-0034
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-worker-provider.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0033 provides an in-process `ResourceProvider` v1. CM-A09 also asks for a
|
||||
process boundary so an adapter can crash or hang without taking down the
|
||||
interpreter, and for a transport that can later be implemented by the C/Worker
|
||||
plugin ecosystem. This ADR adds the first process-boundary transport: a
|
||||
host-managed helper process speaking framed JSON.
|
||||
|
||||
It deliberately does **not** claim the C ABI or OS sandbox: the helper runs as
|
||||
the host user and is trusted. The existing plugin Worker transport remains the
|
||||
untrusted-plugin path; this is a resource-provider transport for explicitly
|
||||
trusted adapters.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Framing and protocol v1
|
||||
|
||||
* 4-byte big-endian length + JSON, maximum 1 MiB per frame.
|
||||
* A `ping` handshake must succeed before the provider is registered.
|
||||
* Operations:
|
||||
|
||||
```text
|
||||
{"op":"open","kind","descriptor","rights"} -> {"ok":true,"token":u64}
|
||||
{"op":"read","token","max_bytes"} -> {"ok":true,"bytes":[u8...],"eof":bool}
|
||||
{"op":"write","token","bytes":[u8...]} -> {"ok":true,"count":u64}
|
||||
{"op":"close","token"} -> {"ok":true}
|
||||
{"op":"shutdown"} -> process exit
|
||||
{"ok":false,"code","message"} -> structured failure
|
||||
```
|
||||
|
||||
* Provider tokens stay in the helper; the host still issues the opaque language
|
||||
`ResourceHandle`. Bytes are copied per call.
|
||||
* The host maps helper error codes to the existing categories
|
||||
(`permission.denied`, `resource.closed`, `resource.exhausted`,
|
||||
`resource.unsupported`, otherwise `host.error`) and attaches the callsite
|
||||
span.
|
||||
* The provider is killed on drop; a protocol/IO failure is `host.error`, never a
|
||||
fallback to another transport.
|
||||
|
||||
### 2. Registration and authorization
|
||||
|
||||
* `StdHost::register_worker_resource_provider(name, kinds, program, args)` is a
|
||||
host embedding API. It validates name/kinds with the same rules as ADR-0033
|
||||
(non-empty, not `stdio`/`file`, no duplicate kind), then registers the
|
||||
provider under its kinds.
|
||||
* `resource.open` still requires the provider name grant and project mode.
|
||||
Pure/strict refuse provider kinds; strict remains fail-closed.
|
||||
* Language code cannot spawn a helper; registration is not reachable from
|
||||
`.mq.md` or a package import.
|
||||
|
||||
### 3. Security boundary
|
||||
|
||||
* The helper runs as the host user with full host privileges. It is **trusted**;
|
||||
this transport is not a sandbox and does not make arbitrary native code safe.
|
||||
* The project-mode + name-grant checks are the same as ADR-0033. The helper's
|
||||
own descriptor/rights validation is additional, not a substitute.
|
||||
* No network, filesystem or process rights are delegated implicitly by
|
||||
registration; the helper is responsible for its own behavior, and the host
|
||||
grant only authorizes the resource kind.
|
||||
|
||||
### 4. What remains for CM-A09
|
||||
|
||||
* A C ABI / plugin-Worker binding for `ResourceProvider` (buffer handoff,
|
||||
ownership, version negotiation through the plugin ABI).
|
||||
* Provider package installation, signatures and discovery.
|
||||
* Untrusted-provider isolation and strict-mode provider support.
|
||||
* The transport is versioned (`protocol_version() == 1`); future versions need a
|
||||
new ADR and negotiation.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A trusted adapter can live in its own process and be reached through the same
|
||||
`resource.*`/`Result` language surface without touching the parser, `Value`,
|
||||
runtime dispatch or `HostOp`.
|
||||
* A helper crash/exit is a structured host error; the interpreter never falls
|
||||
back to a different provider.
|
||||
* The helper protocol is small enough to reimplement in C/Worker adapters later.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-worker-provider.md` records a Python helper exercising
|
||||
ping/open/read/write/close, grant/pure/strict denial, missing-program error and
|
||||
the full workspace gate.
|
||||
@@ -0,0 +1,79 @@
|
||||
# ADR-0036: C ABI ResourceProvider v1
|
||||
|
||||
Status: **Accepted (trusted native adapter; plugin-Worker binding still open)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A11, P5), ADR-0033, ADR-0035
|
||||
Header: `crates/marqdo-host/include/marqdo_resource_provider.h`
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-c-provider.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0033 provides an in-process provider trait and ADR-0035 a host-managed
|
||||
helper process with framed JSON. CM-A09 also needs a **C ABI** so native
|
||||
adapters can expose resource endpoints without linking Rust or changing the
|
||||
runtime. This ADR freezes the first C ABI version for trusted native adapters.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Version and symbols
|
||||
|
||||
```c
|
||||
#define MARQDO_RESOURCE_PROVIDER_ABI_VERSION 1u
|
||||
|
||||
uint32_t marqdo_resource_provider_abi_version(void);
|
||||
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
|
||||
const char *rights_csv, uint64_t *out_token);
|
||||
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
|
||||
uint32_t capacity, uint32_t *out_read,
|
||||
int32_t *out_eof);
|
||||
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
|
||||
uint32_t length, uint32_t *out_written);
|
||||
int32_t marqdo_resource_close(uint64_t token);
|
||||
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity);
|
||||
```
|
||||
|
||||
* The host loads the library, requires ABI version 1 and resolves the symbols
|
||||
at registration; missing symbols/version mismatch fail closed with
|
||||
`resource.unsupported`/`kernel.unavailable`.
|
||||
* Return `0` on success, nonzero on failure; failure text comes from
|
||||
`last_error` and is surfaced as a structured host error with the callsite
|
||||
span.
|
||||
* `rights_csv` is a comma-separated list. Provider tokens are private to the
|
||||
adapter; the host still issues the language-visible `ResourceHandle`.
|
||||
* Buffers are borrowed for the call only; the adapter must not retain host
|
||||
pointers. The host caps frame/buffer sizes by the existing resource limits.
|
||||
|
||||
### 2. Registration and authorization
|
||||
|
||||
* `StdHost::register_c_resource_provider(name, kinds, library_path)` is a host
|
||||
embedding API; it uses the same name/kind validation as ADR-0033 and the same
|
||||
provider-name grant at `resource.open`.
|
||||
* Project mode only; pure/strict refuse provider kinds. Strict stays
|
||||
fail-closed (CM-A11).
|
||||
* The C library runs in the host address space with host privileges. It is
|
||||
**trusted**, not sandboxed; the untrusted-plugin path remains the separate
|
||||
plugin Worker transport. Registration is not reachable from `.mq.md`.
|
||||
|
||||
### 3. Remaining CM-A09 work
|
||||
|
||||
* Plugin-Worker binding for `ResourceProvider` through the existing plugin ABI
|
||||
(buffer handoff/ownership/version negotiation).
|
||||
* Provider package installation, signatures and discovery.
|
||||
* Untrusted-provider isolation and strict-mode provider support.
|
||||
* ABI version negotiation beyond rejecting non-v1 libraries (future ADR).
|
||||
|
||||
## Consequences
|
||||
|
||||
* Native adapters can expose new resource kinds through `resource.*` without
|
||||
parser/`Value`/runtime/`HostOp` changes.
|
||||
* ABI mistakes are fail-closed at registration or surfaced as host errors; no
|
||||
fallback transport is attempted.
|
||||
* The header is small and stable enough for a future SDK/package workflow.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-c-provider.md` records a compiled C test provider
|
||||
(`cc -shared`), version/load checks, open/read/write/close round-trip, missing
|
||||
library fail-closed and the full workspace gate.
|
||||
@@ -0,0 +1,77 @@
|
||||
# ADR-0037: Provider package manifest v1
|
||||
|
||||
Status: **Accepted (host-side installation metadata)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A10, P5), ADR-0036
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-provider-package.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0036 lets a trusted C shared library implement resource operations, but the
|
||||
host still needed a code-level path to the library. CM-A09 asks for provider
|
||||
installation/discovery without changing the parser/runtime. This ADR defines a
|
||||
small, versioned **package manifest** so a provider distribution can be
|
||||
installed by a host embedding and reused by future external provider
|
||||
repositories.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Manifest format
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "marqdo.provider",
|
||||
"schema_version": 1,
|
||||
"name": "cabi",
|
||||
"kinds": ["cabi.sensor"],
|
||||
"abi": "c-abi-1",
|
||||
"library": {
|
||||
"linux": "libcabi_provider.so",
|
||||
"macos": "libcabi_provider.dylib",
|
||||
"default": "libcabi_provider.so"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
* file name: `provider.json` by convention; the host API takes an explicit
|
||||
manifest path.
|
||||
* `schema`/`schema_version` must be exactly `marqdo.provider`/`1`.
|
||||
* `abi` must be `c-abi-1` (ADR-0036); future ABIs need a new ADR.
|
||||
* library paths are relative to the manifest directory (absolute paths are
|
||||
accepted only for explicit local tests); platform key is `linux`, `macos`,
|
||||
`windows`, or `default`.
|
||||
* unknown fields are tolerated by serde, but unknown schema/ABI are rejected.
|
||||
|
||||
### 2. Host API and authorization
|
||||
|
||||
* `StdHost::register_provider_package(manifest_path)` reads/validates the
|
||||
manifest, loads the C ABI library and registers it through the same
|
||||
kind/name validation as ADR-0033/0036.
|
||||
* Registration does not authorize language use: `resource.open` still requires
|
||||
project mode and `Capabilities.allow_resource_provider(name)`. Pure/strict
|
||||
refuse provider kinds.
|
||||
* Language imports cannot read or install a provider manifest. Installation is
|
||||
a host/package-manager action (INV-06).
|
||||
|
||||
### 3. Trust boundary
|
||||
|
||||
* The manifest points at trusted native code loaded in-process; it is not a
|
||||
sandbox and does not add signatures. A future provider repository must add
|
||||
signatures/verification and a Worker binding.
|
||||
* No network/package registry fetch is implemented here; the host passes a
|
||||
local path.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* Provider registry download, signatures, dependency resolution or semver.
|
||||
* CLI install/uninstall commands (host embedding only).
|
||||
* Untrusted-provider isolation or strict-mode provider support.
|
||||
* Automatic platform selection beyond the `library` map.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-provider-package.md` records the manifest round-trip
|
||||
test: compile a C provider, write `provider.json`, `register_provider_package`,
|
||||
then `resource.open/read`.
|
||||
@@ -0,0 +1,68 @@
|
||||
# ADR-0038: C ABI provider worker binding v1
|
||||
|
||||
Status: **Accepted (trusted helper; existing plugin-Worker ABI binding still open)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A11, P5), ADR-0035, ADR-0036
|
||||
Bin: `crates/marqdo-provider-worker`
|
||||
Evidence: `docs/evidence/2026-11-20-c-worker-binding.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0036 loads a trusted C ABI provider in the host process; ADR-0035 provides a
|
||||
framed-JSON worker transport. CM-A09 also asks for the Worker side of the C ABI:
|
||||
the provider library should be loadable in a separate process so a crash does
|
||||
not take down the interpreter, without inventing a new language surface. This
|
||||
ADR adds the concrete binding.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Worker binary
|
||||
|
||||
`marqdo-provider-worker <c-abi-library>`:
|
||||
|
||||
* loads exactly one `c-abi-1` shared library (ADR-0036 symbols);
|
||||
* verifies `marqdo_resource_provider_abi_version() == 1` at startup;
|
||||
* serves the ADR-0035 framed-JSON protocol (`ping/open/read/write/close/
|
||||
shutdown`) over stdin/stdout;
|
||||
* maps C provider failures to `{"ok":false,"code":"host.error","message":...}`
|
||||
and never falls back to in-process loading.
|
||||
|
||||
The binary is a separate artifact, not a new top-level `marqdo` CLI command.
|
||||
Registration is still host-side:
|
||||
`StdHost::register_worker_resource_provider(name, kinds, worker_bin, [library])`
|
||||
from ADR-0035.
|
||||
|
||||
### 2. Process boundary
|
||||
|
||||
* The C library runs in the worker process; a crash/exit is a structured host
|
||||
error and the host does not retry in-process.
|
||||
* The worker has the host user's OS privileges; this is a **trusted transport**,
|
||||
not a sandbox and not proof of untrusted-provider safety.
|
||||
* The language still sees only `resource.*`; provider tokens stay inside the
|
||||
worker and the host `ResourceHandle` remains the language value.
|
||||
* Project mode + `allow_resource_provider(name)` are still required; pure/strict
|
||||
refuse provider kinds (CM-A11).
|
||||
|
||||
### 3. What remains for CM-A09
|
||||
|
||||
* Binding the provider contract through the **existing plugin-Worker ABI**
|
||||
(manifest/artifact ownership and version negotiation) rather than the
|
||||
dedicated helper.
|
||||
* Provider registry/download/signatures and package installation commands.
|
||||
* Untrusted-provider isolation and strict-mode providers.
|
||||
* ABI version negotiation beyond v1.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A trusted C adapter can be crash-isolated without changing the runtime,
|
||||
parser, `Value` or `HostOp`.
|
||||
* The worker helper is small and reusable by a future provider package manager.
|
||||
* The trust model is explicit: worker isolation ≠ sandbox; the provider library
|
||||
remains trusted code.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-c-worker-binding.md` records a compiled C provider,
|
||||
the worker helper, a full open/read/write/close round-trip through the host
|
||||
worker transport, and fail-closed startup with a missing library.
|
||||
@@ -0,0 +1,65 @@
|
||||
# ADR-0039: Stdlib extraction is intentionally deferred
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A10, P2/P3), ADR-0028
|
||||
Supersedes: the assumption in ADR-0028/CM-A10 evidence that a separate
|
||||
`marqdo-stdlib` repository must be created as the next step.
|
||||
|
||||
## Context
|
||||
|
||||
The RFC and the first CM-A10 slice anticipated a separate `marqdo-stdlib`
|
||||
repository so the standard library could be versioned and released independently
|
||||
from the CLI. After implementing the bundled tree, `std:` imports, digest
|
||||
identity, frozen-binary closure, table/HTTP modules and the deterministic
|
||||
offline package, the current architecture is clear and the bundle is still
|
||||
small. A separate repository would add release, version-mapping, signing and
|
||||
cross-repo CI overhead without a present consumer that needs an independent
|
||||
release cadence.
|
||||
|
||||
This decision **does not delete** the extraction work or the packaging artifact;
|
||||
it changes the sequencing/intent.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Current policy: one repository.** `stdlib/` stays in `marqdo-cli` as the
|
||||
official, verified bundle. `std:` imports, the SHA-256 tree digest,
|
||||
module-instance ids and `project.stdlib` Analysis identity are unchanged.
|
||||
2. **Extraction remains an explicit future option.** The deterministic package
|
||||
(`tools/pack-stdlib.py`), `marqdo.stdlib-package` descriptor and version
|
||||
mapping remain maintained as the extraction-ready artifact.
|
||||
3. **No current release work is blocked on a separate repo.** Documentation,
|
||||
STATUS and evidence must not describe "publish the external stdlib repo" as
|
||||
a pending P2/CM-A10 gate; it is an intentionally deferred follow-up.
|
||||
4. **Extraction triggers** (any one requires a new ADR before splitting):
|
||||
* the bundle grows enough that CLI release cadence and stdlib cadence
|
||||
conflict;
|
||||
* a non-CLI consumer needs an independent stdlib release/SemVer line;
|
||||
* multiple implementations or a provider/package ecosystem need version
|
||||
skew management;
|
||||
* the stdlib needs a different review/ownership boundary or signing policy.
|
||||
5. **Compatibility if extraction happens:** keep `std:` import spelling,
|
||||
module ids, digest algorithm and Analysis `project.stdlib` shape; publish a
|
||||
version mapping and enforce the same frozen-binary/tamper gates.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* Deleting the packaging script or historical CM-A10 evidence.
|
||||
* Changing `std:` semantics, bundle digest or module ids now.
|
||||
* Creating a package registry, signing infrastructure or cross-repo CI while
|
||||
there is no independent consumer.
|
||||
* Declaring the stdlib frozen forever; the trigger list above is intentional.
|
||||
|
||||
## Consequences
|
||||
|
||||
* The repository remains the single source of truth and the dev loop stays
|
||||
simple.
|
||||
* "Not external" is no longer reported as a missing gate or blocker.
|
||||
* A future split can reuse the deterministic archive and digest contract
|
||||
without redoing the bundling design.
|
||||
|
||||
## Verification
|
||||
|
||||
Docs/STATUS/evidence reference this ADR; `cargo test --workspace --locked`
|
||||
continues to exercise the bundled `std:` imports, digest and frozen-CLI lane.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Architecture RFCs: compute machine and stdlib migration
|
||||
|
||||
Status: **Draft RFCs, partially implemented**. This page is an index; the
|
||||
normative RFC text stays at its reviewable path under `docs/`.
|
||||
|
||||
| Document | Status | Implemented part |
|
||||
|---|---|---|
|
||||
| [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). |
|
||||
| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict permits only the stdio console channel; ADR-0027/0030/0031 Compat-C print via bundled stdlib, no legacy host formatter). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser/formatter for arrays, objects, strings with escapes, ints/floats/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). The stdlib intentionally remains in this repository for now (ADR-0039); no separate external repo, network/generic providers, P4 physical storage migration, P5 providers or P6 performance work. |
|
||||
|
||||
The RFCs are architecture constraints, not evidence that a feature exists. The
|
||||
implementation status lives in `docs/STATUS.md`; frozen behavior is in
|
||||
`spec/`; decisions are in `docs/adr/`.
|
||||
|
||||
## Decisions already frozen
|
||||
|
||||
| RFC item | ADR | State |
|
||||
|---|---|---|
|
||||
| CM-A01 Memory identity/lifetime | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A02 page size, growth, budgets | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A03 place assignment and evaluation order | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A04 minimal structure observation/construction | [0023](../adr/0023-minimal-structure-observation-and-construction.md) | Accepted |
|
||||
| CM-A05 recoverable `Result` convention | [0024](../adr/0024-recoverable-result-convention.md) | Accepted |
|
||||
| CM-A06 minimal Resource/Stream ABI | [0025](../adr/0025-minimal-resource-stream-abi.md) | Accepted (first slice) |
|
||||
| CM-A06 buffered project-file write | [0026](../adr/0026-buffered-project-file-write-streams.md) | Accepted |
|
||||
| CM-A07 print migration staging / Compat-A | [0027](../adr/0027-print-migration-compat-a.md) | Accepted (Compat-A only) |
|
||||
| CM-A10 bundled stdlib / `std:` imports / digest | [0028](../adr/0028-bundled-stdlib-and-std-imports.md) | Accepted (first slice) |
|
||||
| Full-value stdlib formatting / FunctionRef+PluginHandle observation | [0029](../adr/0029-full-value-formatting-and-handle-observation.md) | Accepted |
|
||||
| CM-A07 Compat-B print routing | [0030](../adr/0030-print-compat-b-routing.md) | Accepted |
|
||||
| CM-A07 Compat-C remove legacy print formatting | [0031](../adr/0031-print-compat-c-remove-host-formatting.md) | Accepted |
|
||||
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
|
||||
| CM-A09 in-process ResourceProvider v1 | [0033](../adr/0033-in-process-resource-provider-v1.md) | Accepted (trusted in-process slice) |
|
||||
| P5 blocking TCP resource endpoint v1 | [0034](../adr/0034-tcp-resource-endpoint.md) | Accepted (IP-literal, no DNS/TLS) |
|
||||
| CM-A09 worker resource-provider transport v1 | [0035](../adr/0035-worker-resource-provider-transport.md) | Accepted (trusted helper) |
|
||||
| CM-A09 C ABI ResourceProvider v1 | [0036](../adr/0036-c-abi-resource-provider.md) | Accepted (trusted native adapter) |
|
||||
| CM-A09 provider package manifest v1 | [0037](../adr/0037-provider-package-manifest.md) | Accepted (host-side install) |
|
||||
| CM-A09 C ABI provider worker binding v1 | [0038](../adr/0038-c-abi-provider-worker-binding.md) | Accepted (trusted helper) |
|
||||
| Stdlib extraction deferral | [0039](../adr/0039-stdlib-extraction-intent.md) | Accepted (in-repo for now) |
|
||||
| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication |
|
||||
|
||||
## What a future phase must not do
|
||||
|
||||
* add a top-level CLI command or Markdown keyword for memory, allocation or I/O;
|
||||
* add a `HostOp` for every new stdlib function;
|
||||
* turn managed `List`/`Map`/`Table` into aliased mutable storage;
|
||||
* expose a host pointer, let a handle cross the wire, or claim block-level
|
||||
safety for user allocators;
|
||||
* make `strict` accept an unverified provider by default;
|
||||
* claim an RFC phase is complete without the closure and negative evidence its
|
||||
phase gate requires.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Local development credentials
|
||||
|
||||
Credentials are **not committed**. They live in the git-ignored `.secrets/`
|
||||
directory (or the untracked `.env.local`) so local development can use the
|
||||
private Git and Woodpecker instances without rotating secrets through the
|
||||
repository.
|
||||
|
||||
## Where to put them
|
||||
|
||||
`.secrets/dev.env` (ignored, mode 600) should define:
|
||||
|
||||
```dotenv
|
||||
MARQDO_GIT_REMOTE=https://git.cflmy.cn/cflmy/marqdo-cli.git
|
||||
MARQDO_GIT_USERNAME=<username>
|
||||
MARQDO_GIT_TOKEN=<git access token>
|
||||
MARQDO_WOODPECKER_URL=https://ci.cflmy.cn
|
||||
MARQDO_WOODPECKER_TOKEN=<Woodpecker API token>
|
||||
MARQDO_CI_SECRET=<optional CI secret>
|
||||
```
|
||||
|
||||
Do not put real values in this document, tests, CI YAML or commit messages.
|
||||
The repository's `.gitignore` excludes `.secrets/` and `.env.local`.
|
||||
|
||||
## Git push
|
||||
|
||||
` .secrets/git-askpass.sh ` returns the username/token for HTTPS prompts:
|
||||
|
||||
```bash
|
||||
set -a; . ./.secrets/dev.env; set +a
|
||||
GIT_ASKPASS="$PWD/.secrets/git-askpass.sh" git push origin master
|
||||
```
|
||||
|
||||
Never embed the token in a remote URL stored in `.git/config`, shell history or
|
||||
a CI file.
|
||||
|
||||
## Woodpecker API
|
||||
|
||||
```bash
|
||||
set -a; . ./.secrets/dev.env; set +a
|
||||
curl -fsS -H "Authorization: Bearer $MARQDO_WOODPECKER_TOKEN" \
|
||||
"$MARQDO_WOODPECKER_URL/api/user/repos"
|
||||
```
|
||||
|
||||
CI itself is defined in `.woodpecker.yml`; the token is only for local API
|
||||
queries/triggering, not for the pipeline. Configure any privileged secrets in
|
||||
the Woodpecker repository settings, never in this repository.
|
||||
|
||||
## What is tracked
|
||||
|
||||
- `.woodpecker.yml` — public CI pipeline definition.
|
||||
- this document — names/usage only, no values.
|
||||
- `.secrets/` and `.env.local` — ignored local files.
|
||||
@@ -0,0 +1,114 @@
|
||||
# RFC-CM CM-A10 evidence: bundled stdlib, `std:` imports and digest
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `92bfbf2` plus this step's changes.
|
||||
Decision record: [ADR-0028](../adr/0028-bundled-stdlib-and-std-imports.md)
|
||||
Related: [Compat-A](2026-11-20-stdlib-print-compat-a.md), [STATUS §5](../STATUS.md).
|
||||
Scope: bundled first slice; **no external repository publication**.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
415 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Bundle
|
||||
|
||||
`stdlib/` is the official offline bundle:
|
||||
|
||||
```text
|
||||
stdlib/
|
||||
stdlib.json # name, version 0.1.0, sha256 digest
|
||||
README.md
|
||||
format.mq.md # 显示 / 列表显示 / 映射显示 / 表格显示 / 转义文本
|
||||
text.mq.md # utf8编码 into Memory
|
||||
io.mq.md # 写文本 / 写值 via resource.write(stdout)
|
||||
demo.mq.md # builtin 打印 comparison
|
||||
tests/io.mq.md # std:io self-test
|
||||
```
|
||||
|
||||
Digest scheme: SHA-256 over sorted `*.mq.md` relative paths, NUL, file bytes,
|
||||
NUL. `stdlib.json` and non-`.mq.md` files are excluded. The loader recomputes
|
||||
the digest before loading any stdlib module and fails closed on mismatch or a
|
||||
missing manifest.
|
||||
|
||||
`std:<relative>` is an explicit front-matter import target; it resolves only
|
||||
inside the bundle, rejects `..`/absolute paths, appends `.mq.md` when absent,
|
||||
and uses package instance
|
||||
`std:marqdo-stdlib@0.1.0#<sha256-hex>`.
|
||||
|
||||
## Tests
|
||||
|
||||
`crates/cli/tests/stdlib_bundle.rs` (6):
|
||||
|
||||
```text
|
||||
std_import_resolves_the_verified_bundle
|
||||
index.mq.md imports std:io; io.写值(["bundle",1,2.5])
|
||||
-> ["bundle", 1, 2.5]
|
||||
|
||||
analysis_records_the_verified_stdlib_digest
|
||||
check --emit contains project.stdlib.version/digest/instance_id
|
||||
matching stdlib.json
|
||||
|
||||
tampered_bundle_is_rejected_before_module_loading
|
||||
copy bundle, modify io.mq.md
|
||||
-> non-zero, "stdlib digest mismatch"
|
||||
|
||||
missing_bundle_manifest_is_a_hard_error
|
||||
empty MARQDO_STDLIB_DIR -> non-zero stdlib error
|
||||
|
||||
bundle_uses_no_foreign_or_plugin_calls
|
||||
scans all stdlib *.mq.md for foreign./plugin. -> none
|
||||
|
||||
frozen_binary_hash_is_unchanged_after_stdlib_execution
|
||||
sha256(marqdo binary) before == after running the std:io project
|
||||
```
|
||||
|
||||
CLI unit tests in `crates/cli/src/stdlib.rs` (2): dev bundle resolves and
|
||||
verifies; a copied/tampered bundle is rejected.
|
||||
|
||||
Existing `mqmd_stdlib_print_output_matches_builtin_print` still passes with the
|
||||
demo moved to `stdlib/demo.mq.md`.
|
||||
|
||||
## Analysis identity
|
||||
|
||||
Analysis v2 project object now carries:
|
||||
|
||||
```json
|
||||
{"stdlib":{"version":"0.1.0",
|
||||
"digest":"sha256:<64 hex>",
|
||||
"instance_id":"std:marqdo-stdlib@0.1.0#<64 hex>"}}
|
||||
```
|
||||
|
||||
It is `null` when no bundle is available. The digest is verified before load,
|
||||
so an Analysis artifact cannot name mismatched stdlib bytes.
|
||||
|
||||
## What is not done
|
||||
|
||||
* No separate `cflmy/marqdo-stdlib` repository, release tag, package registry
|
||||
or CI-signed digest. The bundle is vendored in this repository and copied
|
||||
next to the executable in a release layout by later packaging work.
|
||||
* No CM-A07 Compat-B/C routing; `打印` is still the builtin default.
|
||||
* No Windows/macOS bundle layout evidence and no remote CI URL.
|
||||
* No independent stdlib upgrade workflow; changing a module requires updating
|
||||
`stdlib.json.digest`, and an incompatible core requirement is not yet
|
||||
enforced by a version table.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-cli --test stdlib_bundle
|
||||
cargo run -q -p marqdo-cli -- run examples/stdlib-bundle.mq.md
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,56 @@
|
||||
# RFC-CM CM-A09 evidence: C ABI ResourceProvider v1
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `12be02e` plus this step's changes.
|
||||
Decision record: [ADR-0036](../adr/0036-c-abi-resource-provider.md)
|
||||
Header: [`marqdo_resource_provider.h`](../../crates/marqdo-host/include/marqdo_resource_provider.h)
|
||||
Related: [in-process provider](2026-11-20-provider-abi-cm-a09.md),
|
||||
[worker provider](2026-11-20-worker-provider.md), [STATUS §5](../STATUS.md).
|
||||
Scope: trusted native C ABI adapter; plugin-Worker binding still open.
|
||||
|
||||
## ABI
|
||||
|
||||
```c
|
||||
marqdo_resource_provider_abi_version() -> uint32_t (must be 1)
|
||||
marqdo_resource_open(kind, descriptor, rights_csv, *token) -> i32
|
||||
marqdo_resource_read(token, buffer, capacity, *read, *eof) -> i32
|
||||
marqdo_resource_write(token, bytes, length, *written) -> i32
|
||||
marqdo_resource_close(token) -> i32
|
||||
marqdo_resource_last_error(buffer, capacity) -> i32
|
||||
```
|
||||
|
||||
The host resolves these symbols at registration, rejects non-v1 libraries and
|
||||
missing symbols, attaches callsite spans to errors, and never lets a provider
|
||||
token enter a language `Value`, wire frame or trace.
|
||||
|
||||
## Test
|
||||
|
||||
`crates/marqdo-host/tests/c_provider.rs` (`#[cfg(unix)]`):
|
||||
|
||||
1. builds a C shared library with `cc -shared -fPIC` (skips if `cc` is absent);
|
||||
2. registers it as provider `cabi` for kind `cabi.sensor` with a project-mode
|
||||
grant;
|
||||
3. `resource.open/read/write/close` round-trips `c-`/`abi`, `write=3`, second
|
||||
close `resource.closed`;
|
||||
4. a missing library path fails with `kernel.unavailable`;
|
||||
5. a library reporting ABI version 2 fails with `resource.unsupported`;
|
||||
6. pure/strict refuse the C provider kind with `permission.denied`.
|
||||
|
||||
No Rust runtime/`HostOp`/parser change was needed; language code still sees the
|
||||
same `resource.*` surface.
|
||||
|
||||
## Security boundary
|
||||
|
||||
The C library is loaded in-process with host privileges. It is trusted native
|
||||
code and explicitly **not** a sandbox. Pure/strict refuse provider kinds; strict
|
||||
remains fail-closed. Untrusted plugin loading remains the separate plugin Worker
|
||||
transport.
|
||||
|
||||
## Not done
|
||||
|
||||
* Plugin-Worker resource-provider binding through the existing plugin ABI.
|
||||
* Provider package installation/signatures/discovery.
|
||||
* ABI version negotiation beyond v1 rejection.
|
||||
* Untrusted-provider isolation or strict-mode providers.
|
||||
* Windows shared-library build/lane.
|
||||
@@ -0,0 +1,45 @@
|
||||
# RFC-CM CM-A09 evidence: C ABI provider worker binding v1
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `313ac03` plus this step's changes.
|
||||
Decision record: [ADR-0038](../adr/0038-c-abi-provider-worker-binding.md)
|
||||
Related: [C ABI](2026-11-20-c-provider.md), [worker transport](2026-11-20-worker-provider.md).
|
||||
Scope: trusted helper binding; not a sandbox and not the plugin-Worker ABI.
|
||||
|
||||
## Artifact
|
||||
|
||||
`marqdo-provider-worker <library>` loads a `c-abi-1` shared library and serves
|
||||
the ADR-0035 framed-JSON protocol (ping/open/read/write/close/shutdown). The
|
||||
host registers it through the existing
|
||||
`StdHost::register_worker_resource_provider` API; no runtime/`HostOp`/parser
|
||||
change was required.
|
||||
|
||||
## Tests
|
||||
|
||||
`crates/marqdo-provider-worker/tests/worker_binding.rs`:
|
||||
|
||||
```text
|
||||
c_abi_provider_runs_inside_the_worker_helper
|
||||
cc -shared C provider; worker binary loaded;
|
||||
resource.open/read "worker"/"-abi" eof/write/close through the host.
|
||||
|
||||
worker_helper_without_c_provider_fails_closed
|
||||
missing library -> structured host error, no in-process fallback.
|
||||
```
|
||||
|
||||
The C provider state lives in the child; the host only sees the opaque
|
||||
`ResourceHandle` and `ResourceRead` bytes.
|
||||
|
||||
## Boundary
|
||||
|
||||
The worker is a trusted helper with host-user privileges. It isolates crashes
|
||||
and keeps the C library out of the interpreter process, but it is **not** an OS
|
||||
sandbox. Pure/strict still refuse provider kinds; strict remains fail-closed.
|
||||
|
||||
## Not done
|
||||
|
||||
* Binding through the existing plugin-Worker manifest/artifact ABI.
|
||||
* Provider registry/signatures/install commands.
|
||||
* Untrusted-provider isolation, strict providers, Windows worker packaging.
|
||||
* ABI version negotiation beyond rejecting non-v1 libraries.
|
||||
@@ -0,0 +1,91 @@
|
||||
# RFC-CM ADR-0029 evidence: full-value stdlib formatting
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `1377af1` plus this step's changes.
|
||||
Decision record: [ADR-0029](../adr/0029-full-value-formatting-and-handle-observation.md)
|
||||
Related: [Compat-A](2026-11-20-stdlib-print-compat-a.md),
|
||||
[bundled stdlib](2026-11-20-bundled-stdlib-cm-a10.md), [STATUS §5](../STATUS.md).
|
||||
Scope: observation members + stdlib formatter. No print name routing.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
416 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Observation members
|
||||
|
||||
`Value::FunctionRef` now exposes `module` and `name`; `Value::PluginHandle`
|
||||
exposes `plugin` and `export`. The numeric host `PluginHandle.id` remains
|
||||
private. Formula/Code members were already public. Memory/Resource remain
|
||||
opaque with no fields.
|
||||
|
||||
Runtime test `function_ref_and_resource_members_are_readable` checks:
|
||||
|
||||
```text
|
||||
ref("#目标").module -> root:default::index.mq.md
|
||||
ref("#目标").name -> 目标
|
||||
ref("#公式").latex -> a = 1
|
||||
ref("#公式").display -> true
|
||||
ref("#代码").language-> text
|
||||
ref("#代码").source -> "x\n"
|
||||
```
|
||||
|
||||
## Formatter coverage
|
||||
|
||||
`stdlib/format.mq.md` now has branches for all 15 value types:
|
||||
|
||||
`Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, `List`, `Map`, `Table`,
|
||||
`Formula`, `Code`, `FunctionRef`, `PluginHandle`, `Memory`, `Resource`.
|
||||
|
||||
* Formula/Code/FunctionRef/PluginHandle reproduce the existing `Value::repr()`
|
||||
spellings (not a new format).
|
||||
* Code byte length is computed from `text.codepoints` by summing UTF-8 widths,
|
||||
not from `长度` (scalar count) and without a temporary `Memory`.
|
||||
* Text inside collections is quoted/escaped; top-level Text is raw.
|
||||
|
||||
## Comparison
|
||||
|
||||
`stdlib/demo.mq.md` compares `io.写值` with builtin `打印`:
|
||||
|
||||
```text
|
||||
["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]
|
||||
["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]
|
||||
Table[名称 | 分数]{甲 | 87; 乙 | 92}
|
||||
Table[名称 | 分数]{甲 | 87; 乙 | 92}
|
||||
[Formula("a = 1"), Code("text", 9 bytes), FunctionRef(root:default::demo.mq.md::演示函数)]
|
||||
[Formula("a = 1"), Code("text", 9 bytes), FunctionRef(root:default::demo.mq.md::演示函数)]
|
||||
```
|
||||
|
||||
`examples_r6::mqmd_stdlib_print_output_matches_builtin_print` asserts all three
|
||||
pairs byte-identically, including the Formula/Code/FunctionRef line.
|
||||
|
||||
## What is not done
|
||||
|
||||
* No `打印/print` name routing (Compat-B). Builtin remains the default.
|
||||
* PluginHandle formatting is covered by code branches and observation, but no
|
||||
fixture constructs a live PluginHandle without a plugin invocation.
|
||||
* Control characters outside `"`, `\`, newline, tab and CR are not emitted as
|
||||
`\u00XX`.
|
||||
* Memory/Resource formatting is the literal type name, matching
|
||||
`Value::display()`; no internal identity leaks.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo run -q -p marqdo-cli -- run stdlib/demo.mq.md
|
||||
cargo test -p marqdo-cli --test examples_r6
|
||||
cargo test -p marqdo-runtime --test interpreter function_ref_and_resource_members
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,52 @@
|
||||
# RFC-CM P5/C10 evidence: HTTP/1.1 over the TCP resource endpoint
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `0d4dc1d` plus this step's changes.
|
||||
Related: [ADR-0034](../adr/0034-tcp-resource-endpoint.md),
|
||||
[HTTP parser fixture](2026-11-20-http-parser.md), [STATUS §5](../STATUS.md).
|
||||
Scope: `.mq.md` HTTP framing over a loopback TCP stream; no host HTTP/TLS.
|
||||
|
||||
## Module
|
||||
|
||||
`stdlib/net/http.mq.md` (import `std:net/http`) provides:
|
||||
|
||||
```text
|
||||
解析响应(文本) -> {状态, 原因, 头, 体}
|
||||
解码分块(体) -> Text
|
||||
构建请求(方法, 路径, 头列表, 体) -> Text
|
||||
切片/找子串/找字符/十六进制值
|
||||
```
|
||||
|
||||
It uses only `text.codepoints`/`text.from_codepoint` and List/Map/Text values;
|
||||
there is no `net.request` or host HTTP parser.
|
||||
|
||||
## End-to-end test
|
||||
|
||||
`crates/cli/tests/http_over_tcp.rs`:
|
||||
|
||||
1. starts a loopback `TcpListener` that reads the request and sends
|
||||
`HTTP/1.1 200 OK ... Content-Length: 5 ... hello`;
|
||||
2. runs a generated `.mq.md` program with `--allow-net 127.0.0.1`;
|
||||
3. the program builds a GET request, UTF-8-encodes it through `std:text` into
|
||||
`Memory`, writes it to `resource.open("tcp", addr, ["read","write"])`,
|
||||
reads the response bytes, decodes them to Text with
|
||||
`text.from_codepoint`, parses with `stdlib/net/http`, and returns
|
||||
`[write.ok, read.ok, status, content-type, body, close.ok]`;
|
||||
4. asserts `[true, true, 200, "text/plain", "hello", true]`.
|
||||
|
||||
`stdlib/tests/http.mq.md` independently pins the parser/builder without a
|
||||
socket. Semantic fixture 71 pins the same logic in the conformance corpus.
|
||||
|
||||
## Gate
|
||||
|
||||
Local container: `cargo test -p marqdo-cli --test http_over_tcp` PASS and
|
||||
`cargo test -p marqdo-cli --test stdlib_http` PASS; full workspace gate PASS.
|
||||
No external network access is required.
|
||||
|
||||
## Not done
|
||||
|
||||
* HTTPS/TLS; no TLS security claim.
|
||||
* External DNS, redirects, connection reuse, compression, cookies, HTTP/2.
|
||||
* A host HTTP client/server; this is `.mq.md` framing over the byte stream.
|
||||
* Strict-mode TCP (strict refuses TCP; project mode only).
|
||||
@@ -0,0 +1,57 @@
|
||||
# RFC-CM P5/C10 evidence: `.mq.md` HTTP/1.1 parser
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `67a6d35` plus this step's changes.
|
||||
Related: [ADR-0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md),
|
||||
[ADR-0034](../adr/0034-tcp-resource-endpoint.md), [STATUS §5](../STATUS.md).
|
||||
Scope: Text-level HTTP framing in `.mq.md`; no host HTTP/TLS implementation.
|
||||
|
||||
## Fixture
|
||||
|
||||
`tests/conformance/semantics/71-http-parser.mq.md` (+ expected golden):
|
||||
|
||||
* `找子串`/`找字符`/`切片` over `text.codepoints` and `text.from_codepoint`;
|
||||
* `解析响应` splits the head/body at `\r\n\r\n`, parses the status line,
|
||||
headers and `Transfer-Encoding: chunked`;
|
||||
* `解码分块` parses hex chunk sizes and reassembles the body;
|
||||
* `构建请求(方法, 路径, 头列表, 体)` emits an HTTP/1.1 request with CRLF
|
||||
framing.
|
||||
|
||||
Input:
|
||||
|
||||
```text
|
||||
HTTP/1.1 200 OK
|
||||
Content-Type: text/plain
|
||||
Transfer-Encoding: chunked
|
||||
|
||||
5
|
||||
hello
|
||||
6
|
||||
world
|
||||
0
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```json
|
||||
[200, "OK", "text/plain", "hello world",
|
||||
"GET /health HTTP/1.1\r\nAccept: text/plain\r\nHost: example.test\r\n\r\n"]
|
||||
```
|
||||
|
||||
The fixture is run end-to-end by `conformance_semantics` and is part of the
|
||||
71 semantic goldens.
|
||||
|
||||
## Gate
|
||||
|
||||
Local container: fmt/clippy/426 tests PASS (one fixture test; the golden count
|
||||
is 71). No host `net.request`, HTTP client/server or TLS code was added.
|
||||
|
||||
## Not done
|
||||
|
||||
* HTTP client over TCP, connection reuse, redirects, compression, cookies.
|
||||
* HTTPS/TLS; this ADR/evidence makes **no** TLS security claim.
|
||||
* Request-body streaming, trailers, chunked requests, HTTP/2.
|
||||
* A reusable `stdlib/net/http.mq.md` module (the parser is a conformance
|
||||
fixture, not a bundled module yet).
|
||||
* C/Worker provider transport and external stdlib repository publication.
|
||||
@@ -0,0 +1,130 @@
|
||||
# RFC-CM P1 evidence: addressable Memory and place assignment
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline commit: `a113d894962d8c00eb02efb42e0e73f8b084a1fa` (`docs: bind
|
||||
README/COMPATIBILITY/CHANGELOG to the convergence SHA`)
|
||||
Working tree: RFC-CM P1 changes, **not committed** at the time of this report.
|
||||
Decision record: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md)
|
||||
Scope: RFC-CM-0001 R1 / RFC-CM-0002 P1 only. P2–P6 are **not** implemented.
|
||||
|
||||
## Environment
|
||||
|
||||
* Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
* `PATH=/root/.cargo/bin:$PATH`
|
||||
* `CARGO_HOME=/root/work/marqdo-cli/.cache/cargo`
|
||||
* `CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target`
|
||||
* No GitHub credentials in this checkout; this is **local** evidence, not a
|
||||
GitHub Actions result. Windows/macOS lanes were not run.
|
||||
|
||||
## Static and workspace gate
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests)
|
||||
```
|
||||
|
||||
The 2 ignored tests are the existing `marqdo-plugin-sdk` doc-tests, not new
|
||||
memory tests.
|
||||
|
||||
## Focused evidence
|
||||
|
||||
```text
|
||||
cargo test -p marqdo-runtime --lib memory
|
||||
8 passed; 0 failed
|
||||
regions_are_isolated_and_handles_share_identity
|
||||
create_is_zero_filled_and_reports_pages
|
||||
grow_keeps_old_bytes_and_zero_fills_new_pages
|
||||
failed_grow_does_not_change_region_or_budget
|
||||
invalid_offsets_are_structured_errors
|
||||
create_validates_page_counts_and_budget
|
||||
forged_and_stale_handles_are_rejected
|
||||
page_arithmetic_never_overflows
|
||||
|
||||
cargo test -p marqdo-runtime --test interpreter memory
|
||||
3 passed; 0 failed
|
||||
memory_region_is_shared_by_handle_identity
|
||||
memory_handle_cannot_reach_the_host_boundary
|
||||
memory_budget_is_enforced_without_changing_the_region
|
||||
|
||||
cargo test -p marqdo-value --lib memory
|
||||
2 passed; 0 failed
|
||||
memory_display_never_leaks_the_runtime_handle
|
||||
memory_handles_are_not_transferable
|
||||
|
||||
cargo test -p marqdo-expression --lib memory_index
|
||||
1 passed; 0 failed
|
||||
memory_index_assignment_parses_as_a_place
|
||||
|
||||
cargo test -p marqdo-cli \
|
||||
--test conformance_semantics --test negative \
|
||||
--test analysis_trace --test examples_r6
|
||||
semantic_goldens_match_end_to_end_execution ok
|
||||
runtime_negative_goldens ok
|
||||
analysis_v2_projects_memory_place_targets_additively ok
|
||||
addressable_memory_example_runs_through_the_cli ok
|
||||
cli_memory_budget_override_fails_closed ok
|
||||
(20 tests total in these four binaries, 0 failed)
|
||||
```
|
||||
|
||||
## End-to-end CLI
|
||||
|
||||
```text
|
||||
$ marqdo run examples/memory-buffer.mq.md --json
|
||||
{"success": true, "value": {"type": "int", "decimal": "141"}, ...}
|
||||
|
||||
$ marqdo run examples/memory-buffer.mq.md --max-memory 0 --json
|
||||
{"success": false, "error": {"code": "resource.exhausted", ...}, ...}
|
||||
```
|
||||
|
||||
The example creates one 64 KiB page with a declared maximum of four pages,
|
||||
writes two bytes through two names that share one region, and returns
|
||||
`42 + 99 = 141`.
|
||||
|
||||
## Fixtures
|
||||
|
||||
Semantic goldens:
|
||||
|
||||
* `tests/conformance/semantics/45-memory-buffer.mq.md` — shared identity
|
||||
(`另一个名字 = 区域`) and two byte writes.
|
||||
* `tests/conformance/semantics/46-memory-grow-zero-fill.mq.md` — zero-page
|
||||
create, `grow` returning the old count, old bytes preserved, new byte zero,
|
||||
`memory.size`.
|
||||
* `tests/conformance/semantics/47-memory-isolation-and-alias.mq.md` — two
|
||||
regions do not alias at the same offset; handle equality is identity.
|
||||
|
||||
Runtime negatives:
|
||||
|
||||
| Fixture | Expected code |
|
||||
|---|---|
|
||||
| `13-memory-out-of-bounds` | `runtime.index` |
|
||||
| `14-memory-byte-range` | `runtime.type` |
|
||||
| `15-index-assign-non-memory` | `runtime.type` |
|
||||
| `16-memory-huge-offset` | `runtime.index` |
|
||||
| `17-memory-grow-beyond-max` | `resource.exhausted` |
|
||||
| `18-memory-negative-pages` | `runtime.memory` |
|
||||
|
||||
## What this does not prove
|
||||
|
||||
* No `.mq.md` stdlib repository or allocator/closure fixture yet (RFC-CM P2).
|
||||
* No `Resource`, `Stream`, byte-transfer ABI, stdlib `print` migration or
|
||||
`Table` storage abstraction (P3–P5).
|
||||
* No Windows/macOS evidence and no remote CI URL.
|
||||
* No claim of block-level memory safety for a user-written allocator; the
|
||||
runtime only bounds each access to its region.
|
||||
* No claim that `strict` accepts new I/O: memory is pure computation and no
|
||||
`HostOp` or provider was added.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo fmt --all -- --check
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
cargo test --workspace --locked
|
||||
cargo run -p marqdo-cli -- run examples/memory-buffer.mq.md --json
|
||||
```
|
||||
@@ -0,0 +1,74 @@
|
||||
# RFC-CM P2 partial evidence: `.mq.md`-written codec, allocator and list
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes.
|
||||
Related: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md),
|
||||
[P1 evidence](2026-11-20-memory-p1.md),
|
||||
[RFC-CM-0002 §7.2](../../docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md).
|
||||
Scope: **partial P2 only**. The RFC-CM-0002 P2 gate is **not** passed.
|
||||
|
||||
> Historical slice: this report predates ADR-0023 (CM-A04 structure API),
|
||||
> ADR-0024 (CM-A05 Result), the coalescing allocator (fixture 55) and the
|
||||
> binary tree/CSV fixtures (56–57). Its "what remains" list is superseded by
|
||||
> [`2026-11-20-structure-api-p2.md`](2026-11-20-structure-api-p2.md) and
|
||||
> [`2026-11-20-result-and-free-list-p2.md`](2026-11-20-result-and-free-list-p2.md);
|
||||
> the P2 gate itself is still not passed.
|
||||
|
||||
## What was added without Rust algorithm code
|
||||
|
||||
Three semantic fixtures are ordinary `.mq.md` documents. They call only
|
||||
language builtins (`memory.new/size/grow`), indexing, arithmetic, functions,
|
||||
branches and loops. No Rust crate was changed for the algorithms themselves.
|
||||
|
||||
| Fixture | `.mq.md` subject | Result |
|
||||
|---|---|---|
|
||||
| `tests/conformance/semantics/48-memory-bump-allocator.mq.md` | big-endian u32 encode/decode and a 4-byte-header bump allocator; allocates 8, 16, 8 bytes and writes a byte | `[4, 12, 28, 36, 255]` |
|
||||
| `tests/conformance/semantics/49-memory-linked-list.mq.md` | allocates three 8-byte nodes (`value:u8`, `next:u32`), links them and traverses the list | `[4, 12, 20, 60]` |
|
||||
| `tests/conformance/semantics/50-memory-allocation-failure-atomic.mq.md` | a too-large allocation returns `None` without moving the header; the next allocation still starts at 4 | `[None, 4, 4, 65536]` |
|
||||
|
||||
Observed CLI results (all `success: true`):
|
||||
|
||||
```text
|
||||
48: [4, 12, 28, 36, 255]
|
||||
49: [4, 12, 20, 60]
|
||||
50: [None, 4, 4, 65536]
|
||||
```
|
||||
|
||||
`cargo test -p marqdo-cli --test conformance_semantics` passes with these three
|
||||
fixtures included.
|
||||
|
||||
## Why this is not the P2 gate
|
||||
|
||||
RFC-CM-0002 §9/P2 requires C01–C06 and C11 to pass in an environment with
|
||||
Foreign/native plugins disabled, and requires a minimal API for dynamic
|
||||
List/Map/Table construction and observation plus recoverable errors. This
|
||||
slice does **not** provide:
|
||||
|
||||
* duplicate-free/allocator adversarial tests (C01 now has a first-fit
|
||||
free-list allocator with reuse, splitting and coalescing, fixture 55);
|
||||
* graph/hash-table structures (C02 now has a singly linked list and a binary
|
||||
search tree, fixtures 49 and 56, but no graph/hash table);
|
||||
* general List/Map/Table construction, mutation-free rebuilding or key
|
||||
enumeration (CM-A04 is undecided);
|
||||
* a `Result`/recoverable-error contract for parser/IO/EOF/partial conditions
|
||||
(CM-A05 is undecided);
|
||||
* an independent `marqdo-stdlib` repository, version/digest contract or
|
||||
frozen-CLI-binary closure lane (the RFC §10.5 experiment).
|
||||
|
||||
It also does not add a stdlib API to the CLI; the fixtures are conformance
|
||||
documents, not a shipped standard library.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
for f in 48-memory-bump-allocator 49-memory-linked-list \
|
||||
50-memory-allocation-failure-atomic; do
|
||||
cargo run -q -p marqdo-cli -- run \
|
||||
tests/conformance/semantics/$f.mq.md --json
|
||||
done
|
||||
cargo test -p marqdo-cli --test conformance_semantics
|
||||
```
|
||||
@@ -0,0 +1,69 @@
|
||||
# RFC-CM P2 closure gate evidence (C01–C06, C11, C12)
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `d6cd785` plus this step's changes.
|
||||
Related: [memory P2](2026-11-20-memory-p2-closure.md),
|
||||
[structure API](2026-11-20-structure-api-p2.md),
|
||||
[result/free-list](2026-11-20-result-and-free-list-p2.md),
|
||||
[ADRs 0022–0024](../adr/).
|
||||
Scope: pure no-Foreign/no-Plugin execution lane; external stdlib repository
|
||||
publication remains separate.
|
||||
|
||||
## Lane
|
||||
|
||||
`crates/cli/tests/p2_closure.rs` runs 21 checked-in semantic fixtures through
|
||||
the real CLI with `run --mode pure --json`:
|
||||
|
||||
* `--mode pure` constructs `Capabilities::pure()`; no foreign, plugin, read,
|
||||
write, network or provider grant exists in that host;
|
||||
* each fixture source is scanned for `foreign.`, `plugin.` and `import:`; all
|
||||
three must be absent (anti-evasion rule for C01–C06/C11/C12);
|
||||
* the CLI value and stdout are compared byte-for-byte with the fixture's
|
||||
`.expected.json` golden;
|
||||
* `stdlib/tests/table.mq.md` also runs in pure mode with `std:` imports and
|
||||
checks the nine table results including `转GFM`.
|
||||
|
||||
The independently-run conformance harness (`conformance_semantics.rs`) also
|
||||
builds `StdHost` with `SandboxMode::Pure` and `Capabilities::pure()` for every
|
||||
fixture.
|
||||
|
||||
Negative shortcuts:
|
||||
|
||||
```text
|
||||
foreign.call in pure -> permission.denied
|
||||
plugin.invoke in pure -> structured failure
|
||||
```
|
||||
|
||||
so the lane proves both that the algorithms run without host shortcuts and that
|
||||
the shortcuts are actually unavailable.
|
||||
|
||||
## Criterion mapping
|
||||
|
||||
| Criterion | Fixtures | What is proven |
|
||||
|---|---|---|
|
||||
| C01 allocators | 48, 50, 55 | bump allocator, failed-allocation atomicity, first-fit split/coalesce/double-free |
|
||||
| C02 structures | 49, 56, 58, 61 | linked list, binary tree, open-addressing hash table, graph adjacency |
|
||||
| C03 codecs | 48, 64 | u32, big-endian 16/32/64, LEB128 |
|
||||
| C04 Unicode/text | 59, 60 | UTF-8 encode and validating decoder errors |
|
||||
| C05 JSON/CSV | 57, 62, 68 | CSV split/quoting, recursive JSON parse/format/errors |
|
||||
| C06 GFM Table | 53, 54, 65, 67, 69 + stdlib table | projection/filter/sort/join/group/GFM render, no Rust table algorithm |
|
||||
| C11 VM | 63 | Memory-backed stack bytecode VM |
|
||||
| C12 scientific prep | 70, 73 | Int matrix multiply and sparse-triple matrix multiply with dense conversion |
|
||||
|
||||
No fixture compiles Rust, imports a package, invokes a foreign kernel or calls a
|
||||
plugin. Dynamic arrays are built at runtime; no fixed-length source rewrite is
|
||||
required.
|
||||
|
||||
## Gate
|
||||
|
||||
Local container: `cargo test -p marqdo-cli --test p2_closure` PASS; full
|
||||
`cargo test --workspace --locked` PASS. The existing frozen-binary test covers
|
||||
the fixed CLI hash and stdlib tamper rejection (ADR-0028).
|
||||
|
||||
## Remaining (not a C01–C12 gap)
|
||||
|
||||
* The stdlib is intentionally in-repo for now (ADR-0039); a separate
|
||||
repository with a signed version mapping is a future extraction option, not
|
||||
a P2 gate. The bundled tree, digest and offline layout are tested in-repo.
|
||||
* No GitHub CI URL / Windows/macOS lanes from this host.
|
||||
@@ -0,0 +1,57 @@
|
||||
{
|
||||
"schema": "marqdo.perf-baseline",
|
||||
"schema_version": 1,
|
||||
"binary": ".cache/target/release/marqdo",
|
||||
"iterations": 3,
|
||||
"workloads": [
|
||||
{
|
||||
"name": "allocator-free-list",
|
||||
"source": "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md",
|
||||
"seconds_median": 0.0173,
|
||||
"seconds_min": 0.017026568995788693,
|
||||
"max_rss_kb_median": 8428
|
||||
},
|
||||
{
|
||||
"name": "json-parser",
|
||||
"source": "tests/conformance/semantics/62-json-array-parser.mq.md",
|
||||
"seconds_median": 0.0188,
|
||||
"seconds_min": 0.01880078506655991,
|
||||
"max_rss_kb_median": 8960
|
||||
},
|
||||
{
|
||||
"name": "table-sort",
|
||||
"source": "tests/conformance/semantics/65-table-sort-stdlib.mq.md",
|
||||
"seconds_median": 0.0152,
|
||||
"seconds_min": 0.014747997978702188,
|
||||
"max_rss_kb_median": 7476
|
||||
},
|
||||
{
|
||||
"name": "bytecode-vm",
|
||||
"source": "tests/conformance/semantics/63-memory-bytecode-vm.mq.md",
|
||||
"seconds_median": 0.0155,
|
||||
"seconds_min": 0.015298803802579641,
|
||||
"max_rss_kb_median": 7692
|
||||
},
|
||||
{
|
||||
"name": "matrix-multiply",
|
||||
"source": "tests/conformance/semantics/70-matrix-multiply.mq.md",
|
||||
"seconds_median": 0.0151,
|
||||
"seconds_min": 0.014752828981727362,
|
||||
"max_rss_kb_median": 7368
|
||||
},
|
||||
{
|
||||
"name": "http-parser",
|
||||
"source": "tests/conformance/semantics/71-http-parser.mq.md",
|
||||
"seconds_median": 0.018,
|
||||
"seconds_min": 0.017968795029446483,
|
||||
"max_rss_kb_median": 8196
|
||||
},
|
||||
{
|
||||
"name": "stdlib-print-demo",
|
||||
"source": "stdlib/demo.mq.md",
|
||||
"seconds_median": 0.0205,
|
||||
"seconds_min": 0.02037098305299878,
|
||||
"max_rss_kb_median": 9636
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
# RFC-CM P6 baseline profile
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `b020904` plus this step's changes.
|
||||
Scope: **baseline measurement only**; no bytecode/JIT/native backend was added.
|
||||
|
||||
## Harness
|
||||
|
||||
`tools/perf/bench.py` builds `marqdo-cli --release`, runs each workload three
|
||||
times through `/usr/bin/time -f "%e %M"`, measures wall time with
|
||||
`time.perf_counter`, verifies `--json` success and writes a schema-tagged JSON
|
||||
report. The checked-in report is
|
||||
[`2026-11-20-p6-profile.json`](2026-11-20-p6-profile.json).
|
||||
|
||||
Environment: Ubuntu 26.04 container, Linux x86_64, Rust 1.100.0, release
|
||||
profile, 7 selected `.mq.md` workloads. These are small conformance workloads,
|
||||
so most times are dominated by process/startup and Markdown parsing; they are
|
||||
not throughput benchmarks of large scientific problems.
|
||||
|
||||
## Results (median of 3)
|
||||
|
||||
| Workload | Wall s | Peak RSS KiB |
|
||||
|---|---:|---:|
|
||||
| allocator-free-list | 0.0173 | 8428 |
|
||||
| json-parser | 0.0188 | 8960 |
|
||||
| table-sort | 0.0152 | 7476 |
|
||||
| bytecode-vm | 0.0155 | 7692 |
|
||||
| matrix-multiply | 0.0151 | 7368 |
|
||||
| http-parser | 0.0180 | 8196 |
|
||||
| stdlib-print-demo | 0.0205 | 9636 |
|
||||
|
||||
## Interpretation
|
||||
|
||||
* Startup + parse + load dominate these small workloads; the interpreter's
|
||||
per-byte/per-call cost is visible mainly in the VM/matrix/table loops.
|
||||
* `stdlib-print-demo` is highest RSS/wall because it loads the bundled stdlib
|
||||
modules and formats all value types through `.mq.md`.
|
||||
* The current release binary is a tree-walking Rust interpreter; no bytecode
|
||||
compiler, JIT, columnar numeric backend or copy-elision pass exists.
|
||||
* No claim of P6 pass: the RFC requires an optimized implementation that is
|
||||
equivalent to the reference interpreter and reproducible performance/budget
|
||||
evidence. This report only establishes the baseline and a reproducible
|
||||
harness for future work.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
python3 tools/perf/bench.py > /tmp/marqdo-perf.json
|
||||
```
|
||||
@@ -0,0 +1,108 @@
|
||||
# RFC-CM CM-A07 Compat-B evidence: `打印` routing through the stdlib
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `e119727` plus this step's changes.
|
||||
Decision record: [ADR-0030](../adr/0030-print-compat-b-routing.md)
|
||||
Related: [Compat-A](2026-11-20-stdlib-print-compat-a.md),
|
||||
[full-value formatting](2026-11-20-full-value-formatting.md),
|
||||
[bundled stdlib](2026-11-20-bundled-stdlib-cm-a10.md).
|
||||
Scope: Compat-B routing; **Compat-C removal is not done**.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
416 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Routing
|
||||
|
||||
* `Program.print_override: Option<FunctionId>` (IR, serde-optional).
|
||||
* Loader scans loaded IR for bare `打印`/`print` calls. Only if found, it loads
|
||||
bundled `std:io` and sets the override to `io.写值`.
|
||||
* Runtime resolution: local callable > print override > builtin table >
|
||||
same-module/imported function. Local shadowing is unchanged.
|
||||
* The stdlib path formats all 15 value types (ADR-0029), UTF-8-encodes into
|
||||
`Memory`, opens `stdio/stdout`, writes and closes.
|
||||
* Programs without print calls keep their original module graph.
|
||||
|
||||
## Trace behavior
|
||||
|
||||
`crates/cli/tests/print_routing.rs::print_is_routed_through_the_stdlib_and_not_hostop_print`:
|
||||
|
||||
```text
|
||||
run index.mq.md --trace-file trace.json
|
||||
stdout = ["x", 1]
|
||||
trace operations include resource.write
|
||||
trace operations do NOT include print
|
||||
```
|
||||
|
||||
Resource open/read/write/close are now recorded as `effect` events by
|
||||
`StdHost`, so trace consumers see the real byte path. No values or secrets are
|
||||
recorded.
|
||||
|
||||
## Lazy injection
|
||||
|
||||
`stdlib_print_modules_are_injected_only_when_print_is_used`:
|
||||
|
||||
```text
|
||||
plain.mq.md (no print) -> no std: modules
|
||||
prints.mq.md (打印) -> std: modules present
|
||||
```
|
||||
|
||||
Both loads record `project.stdlib` identity because a verified bundle is
|
||||
available; only the print-using program loads its modules. `check`'s
|
||||
human-readable module count excludes `std:` modules so bundle internals do not
|
||||
change the project count.
|
||||
|
||||
## Strict mode
|
||||
|
||||
Strict L1 already permits the console. Compat-B needs the same channel, so
|
||||
strict now permits **only** `resource.open("stdio","stdout",["write"])` with the
|
||||
console grant. File/network/process/device resource opens remain
|
||||
`permission.denied`. Host test:
|
||||
|
||||
```text
|
||||
strict_mode_allows_only_the_existing_stdio_console_channel
|
||||
stdio write -> "strict"
|
||||
file read -> permission.denied
|
||||
socket read -> permission.denied
|
||||
```
|
||||
|
||||
This is the existing console capability through the resource ABI, not a new OS
|
||||
capability; CM-A11 fail-closed remains for all other providers.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* All semantic goldens and CLI print tests pass byte-identically, including
|
||||
the full-value formatting demo.
|
||||
* `HostOp::Print` remains implemented as the no-bundle fallback and for
|
||||
Compat-A comparison tests; Compat-C removal is a separate decision.
|
||||
* Trace consumers that hard-code `operation == "print"` must accept
|
||||
`resource.*`; this is documented as a P3 migration.
|
||||
|
||||
## What is not done
|
||||
|
||||
* Compat-C: `HostOp::Print` and `Value::display()` formatting are still in the
|
||||
public builtin path for no-bundle installs.
|
||||
* No stdlib version upgrade workflow and no external repository publication.
|
||||
* No Windows/macOS evidence and no remote CI URL.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-cli --test print_routing
|
||||
cargo test -p marqdo-cli --test analysis_trace
|
||||
cargo test -p marqdo-host --test resource_streams
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,75 @@
|
||||
# RFC-CM CM-A07 Compat-C evidence: legacy print formatting removed
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `e26971f` plus this step's changes.
|
||||
Decision record: [ADR-0031](../adr/0031-print-compat-c-remove-host-formatting.md)
|
||||
Related: [Compat-A](2026-11-20-stdlib-print-compat-a.md),
|
||||
[Compat-B](2026-11-20-print-compat-b.md),
|
||||
[full-value formatting](2026-11-20-full-value-formatting.md).
|
||||
Scope: Compat-C removal of the public host formatting path.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
419 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Removed
|
||||
|
||||
* `HostOp::Print` variant, its stable op name and `StdHost::op_print`.
|
||||
* The print builtin's host dispatch. `打印/print` remains a builtin **name**
|
||||
for arity validation, but it no longer formats or writes:
|
||||
* with `Program.print_override` set (official CLI + bundle), the runtime calls
|
||||
`stdlib/io.写值`;
|
||||
* without an override, it returns `kernel.unavailable`:
|
||||
"打印 requires the bundled stdlib print module; install the official CLI
|
||||
bundle."
|
||||
|
||||
`Value::display()` remains for CLI result rendering/diagnostics but is not
|
||||
reachable from the print builtin.
|
||||
|
||||
## Runtime/embedder behavior
|
||||
|
||||
`crates/marqdo-runtime/tests/interpreter.rs::print_without_the_bundled_stdlib_is_a_structured_error`
|
||||
builds a Program without an override and asserts the structured error.
|
||||
|
||||
The semantic conformance harness builds Programs directly, so it now mirrors
|
||||
the loader's Compat-B injection: it loads `stdlib/io.mq.md` (with
|
||||
`format`/`text`), adds import edges and sets `print_override`. Existing print
|
||||
fixtures 22/23/30/42 pass byte-identically.
|
||||
|
||||
## Trace and output
|
||||
|
||||
`crates/cli/tests/print_routing.rs` still asserts a print program's trace
|
||||
contains `resource.write` and no `print` effect. The CLI loader routes official
|
||||
runs through stdlib; `打印` output remains byte-identical, including the
|
||||
full-value Formula/Code/FunctionRef demo.
|
||||
|
||||
## Compatibility notes
|
||||
|
||||
* Breaking for embedders that call `打印` without installing the stdlib print
|
||||
module; the official CLI loader does this automatically.
|
||||
* `HostOp` loses a variant; any external exhaustive match must be updated.
|
||||
This is pre-1.0 and recorded in ADR-0031.
|
||||
* No bundle means no print; this is fail-closed rather than a silent formatter.
|
||||
* Compat-A/B evidence remains historical; their fallback wording is superseded.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-runtime --test interpreter print_without
|
||||
cargo test -p marqdo-cli --test print_routing
|
||||
cargo test -p marqdo-cli --test conformance_semantics
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,90 @@
|
||||
# RFC-CM CM-A09 evidence: in-process ResourceProvider v1
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a8e62a5` plus this step's changes.
|
||||
Decision record: [ADR-0033](../adr/0033-in-process-resource-provider-v1.md)
|
||||
Related: [resource streams](2026-11-20-resource-stream-p3.md), [STATUS §5](../STATUS.md).
|
||||
Scope: trusted in-process provider contract only; C/Worker transport remains open.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
423 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Contract
|
||||
|
||||
`ResourceProvider` v1:
|
||||
|
||||
```text
|
||||
name() -> stable grant name
|
||||
protocol_version() -> 1
|
||||
kinds() -> resource kinds
|
||||
open(kind, descriptor, rights) -> provider token
|
||||
read(token, max_bytes) -> {bytes, eof}
|
||||
write(token, bytes) -> count
|
||||
close(token)
|
||||
```
|
||||
|
||||
The host still issues the language `ResourceHandle`; the provider token never
|
||||
enters a `Value`, wire message or trace. Bytes are copied for the call duration;
|
||||
providers must not retain Memory pointers.
|
||||
|
||||
## Registration and authorization
|
||||
|
||||
* `StdHost::register_resource_provider` rejects protocol != 1, empty kinds,
|
||||
reserved `stdio`/`file` names/kinds and duplicate kind registration.
|
||||
* `Capabilities.resource_providers` + `allow_resource_provider(name)` add the
|
||||
grant; `resource.open` requires the provider name to be granted.
|
||||
* Project mode only. Pure and strict refuse provider kinds; registration grants
|
||||
nothing by itself.
|
||||
* In-process providers are trusted and **not** sandboxed; the ADR states this
|
||||
explicitly.
|
||||
|
||||
## Tests
|
||||
|
||||
`crates/marqdo-host/tests/resource_streams.rs` (12 total; 3 provider tests):
|
||||
|
||||
```text
|
||||
registered_provider_streams_work_with_a_grant_in_project_mode
|
||||
mock sensor: read ab/cdef with eof, write 1, close, second close resource.closed
|
||||
|
||||
provider_requires_a_name_grant_and_project_mode
|
||||
no grant -> permission.denied
|
||||
pure -> permission.denied
|
||||
strict -> permission.denied
|
||||
|
||||
provider_registration_rejects_versions_reserved_names_and_conflicts
|
||||
version 2, reserved stdio kind, duplicate kind -> resource.unsupported
|
||||
```
|
||||
|
||||
No parser, `Value`, runtime dispatch or `HostOp` change was required; the mock
|
||||
provider is registered through a host API and the language sees the existing
|
||||
`resource.*` calls.
|
||||
|
||||
## Not done
|
||||
|
||||
* C ABI / Worker resource transport: buffer handoff, ownership, crash/timeout
|
||||
isolation and version negotiation through the plugin ABI.
|
||||
* Provider package installation, signatures, discovery and platform matrix.
|
||||
* Network/socket/TCP/HTTP, clock, random, process, directory, device providers.
|
||||
* Strict-mode provider verification.
|
||||
* No external `marqdo-stdlib` repository publication.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-host --test resource_streams
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,45 @@
|
||||
# RFC-CM CM-A09 evidence: provider package manifest v1
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `5e5c3a8` plus this step's changes.
|
||||
Decision record: [ADR-0037](../adr/0037-provider-package-manifest.md)
|
||||
Related: [C ABI provider](2026-11-20-c-provider.md), [STATUS §5](../STATUS.md).
|
||||
Scope: local manifest installation; no registry/signature/Worker binding.
|
||||
|
||||
## Manifest
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "marqdo.provider",
|
||||
"schema_version": 1,
|
||||
"name": "cabi",
|
||||
"kinds": ["cabi.sensor"],
|
||||
"abi": "c-abi-1",
|
||||
"library": {"linux": "libecho_provider.so", "default": "libecho_provider.so"}
|
||||
}
|
||||
```
|
||||
|
||||
`StdHost::register_provider_package` validates schema/version/ABI, resolves the
|
||||
platform library relative to the manifest, loads it through the ADR-0036 C ABI
|
||||
adapter and registers it through the shared kind/name checks. Language use still
|
||||
requires project mode and `allow_resource_provider("cabi")`.
|
||||
|
||||
## Test
|
||||
|
||||
`crates/marqdo-host/tests/c_provider.rs::provider_package_manifest_installs_a_c_abi_library`:
|
||||
|
||||
1. compile the C provider with `cc -shared`;
|
||||
2. write `provider.json` next to it;
|
||||
3. `register_provider_package(&manifest)` succeeds;
|
||||
4. `resource.open("cabi.sensor", ...)` and `resource.read` return `c-`.
|
||||
|
||||
Existing C tests cover version mismatch, missing library and pure/strict refusal.
|
||||
No parser/`Value`/runtime/`HostOp` change was needed.
|
||||
|
||||
## Not done
|
||||
|
||||
* Provider registry/download, signatures, dependency resolution, semver.
|
||||
* CLI install/uninstall commands; installation is host embedding API only.
|
||||
* Untrusted-provider isolation, strict-mode providers, plugin-Worker binding.
|
||||
* Windows library build lane.
|
||||
@@ -0,0 +1,104 @@
|
||||
# RFC-CM CM-A06 extension evidence: buffered project-file writes
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes.
|
||||
Decision record: [ADR-0026](../adr/0026-buffered-project-file-write-streams.md)
|
||||
Related: [ADR-0025](2026-11-20-resource-stream-p3.md),
|
||||
[STATUS §5](../STATUS.md).
|
||||
Scope: project-file **write** extension only; no network/process/directory.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
406 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Implemented behavior
|
||||
|
||||
`resource.open("file", "<project-relative path>", ["write"])`:
|
||||
|
||||
* project mode only; requires an `fs_write` glob grant and the same
|
||||
`ProjectFs` path policy (reject absolute, `..`, symlink escapes);
|
||||
* pure mode refuses file resources by mode; strict refuses every resource open;
|
||||
* the writer owns an in-memory buffer; `resource.write` appends and returns
|
||||
`{ok:true, value:{count}}` without touching the filesystem;
|
||||
* `resource.close` flushes through `ProjectFs::write_bytes` and then marks the
|
||||
handle closed; a failed flush leaves the handle open with its buffer;
|
||||
* an unclosed writer never creates the file (no flush on drop);
|
||||
* buffer ceiling is the existing 8 MiB file limit; an oversized append is
|
||||
`resource.exhausted` and leaves both buffer and file unchanged;
|
||||
* `resource.read` on a writer, and `resource.write` on a reader, are
|
||||
`resource.unsupported`;
|
||||
* no new `Value`, `HostOp`, Markdown node or wire shape.
|
||||
|
||||
## Tests
|
||||
|
||||
### Host integration (`crates/marqdo-host/tests/resource_streams.rs`, 9)
|
||||
|
||||
```text
|
||||
pure_mode_writes_stdout_through_a_resource
|
||||
stdout_write_obeys_the_output_budget_atomically
|
||||
strict_mode_refuses_every_resource_open
|
||||
resource_handles_cannot_cross_the_generic_host_boundary
|
||||
pure_mode_refuses_file_resources_even_with_a_grant
|
||||
project_file_writes_buffer_until_close_and_require_a_grant
|
||||
file_write_buffer_has_a_hard_ceiling
|
||||
project_file_reads_are_chunked_and_require_a_grant
|
||||
traversal_and_unsupported_rights_are_refused
|
||||
```
|
||||
|
||||
The write test asserts the file does not exist before `close`, appears with
|
||||
`abcdef` after `close`, closed read/close return `resource.closed`, and the
|
||||
same open without `fs_write` is `permission.denied`.
|
||||
|
||||
### CLI integration (`crates/cli/tests/resource_streams.rs`, 4)
|
||||
|
||||
```text
|
||||
project_file_stream_writes_on_close_then_reads_back
|
||||
run index.mq.md --allow-write out.bin --allow-read out.bin --json
|
||||
write count 3, close ok, read count 3 eof true, bytes 65/66/67
|
||||
out.bin on disk = "ABC"
|
||||
|
||||
file_copy_is_written_in_mqmd_through_chunked_streams
|
||||
.mq.md opens data.txt for read and out.bin for write,
|
||||
loops resource.read(16) / resource.write(count) until eof,
|
||||
closes both; total 6; out.bin on disk = "abcdef"
|
||||
(C09 file copy without a Rust copy_file builtin)
|
||||
|
||||
project_file_stream_reads_chunks_into_memory
|
||||
file_stream_requires_a_read_grant
|
||||
```
|
||||
|
||||
### Host table unit tests (`crates/marqdo-host/src/resource.rs`, 3)
|
||||
|
||||
Chunked EOF, forged/closed handles, stdout-not-readable and file-not-writable
|
||||
(read-only direction) remain covered; the writer uses the same handle checks.
|
||||
|
||||
## What this does not prove
|
||||
|
||||
* No append/truncate/seek, directories, metadata, network, process, clock,
|
||||
random, device or generic provider registry.
|
||||
* No crash-atomic replace: `ProjectFs::write_bytes` is one validated write, not
|
||||
a temp-file rename.
|
||||
* No file write in `pure` or `strict`.
|
||||
* No CM-A07 stdlib `print` migration.
|
||||
* No Windows/macOS evidence and no remote CI URL.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-host --test resource_streams
|
||||
cargo test -p marqdo-cli --test resource_streams
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
Loaded 100 of 214 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user