diff --git a/CHANGELOG.md b/CHANGELOG.md index d861940..21f9565 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,9 +43,84 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601. without rewriting `marqdo.lock` or the materialized `.marqdo/env`. - G-C lane honours `CARGO_TARGET_DIR`, so a run from a clean `git archive` with an external target directory uses the binary it just built. +- Addressable `Memory` (RFC-CM-0001 P1, ADR-0022): an opaque run-scoped + region value with 64 KiB zero-filled pages, grow-only capacity, bounded + byte read/write, `memory.new/size/grow`, and `m[offset] = byte`. Copies share + the region identity; different regions never alias. Default budgets are + 64 MiB committed per run and 16 MiB per region; `--max-memory` and + `--max-region-memory` override the budgets on `run`/`call`. +- Memory-index place assignment in the expression/IR model + (`AssignTarget::Index` / `PlaceTarget::Index`) with the frozen evaluation + order base, index, RHS and no change to `List`/`Map`/`Table` value semantics. +- Stable `runtime.memory` error category for invalid/stale handles and page + counts; `runtime.index` for offsets, `runtime.type` for operand/byte-range + errors, `resource.exhausted` for budgets. +- Analysis v2 projects an index assignment target as an additive + `{"kind":"index",...}` object while local targets keep the v1 string. +- `examples/memory-buffer.mq.md`, semantic goldens 45–47 and runtime negatives + 13–18 cover sharing, isolation, growth, zero-fill, offsets, byte range, + non-Memory index assignment, declared-max growth and negative page counts. +- Partial RFC-CM P2 closure evidence (`.mq.md` only, no Rust algorithm + additions): fixtures 48–50 implement a big-endian u32 codec, a bump + allocator, node allocation and a linked-list traversal/sum, and prove that a + failed allocation leaves the allocator header unchanged. This is **not** the + full P2 gate: JSON floats, table group and a frozen-CLI stdlib closure lane + are still missing. +- `Memory` handles are never transferable: `to_wire` emits a bare rejected tag, + `from_wire` fails, and every host/plugin/foreign call rejects arguments that + contain a handle. +- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023): + `list.append/concat`, `map.keys/has/get/set/remove`, + `text.codepoints/from_codepoint`, and `table.columns/row/cell`. All are pure, + value-returning builtins; inputs are never mutated and no new syntax is + added. Semantic fixtures 51–53 and runtime negatives 19–21 cover value + semantics, Unicode scalars, table projection, wrong types and bounds. +- `.mq.md` table filter (`过滤`) built on `table.columns/row/cell`, + `list.append` and `表`, with a semantic golden showing the input table is + unchanged (fixture 54). No `table.filter`/`HostOp` is added. +- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): an ordinary Map + `{ok:true,value}` / `{ok:false,code,message[,details]}`, constructed by + `.mq.md` functions, with expected failures recoverable and language + invariant violations still fatal. No `Result` type, exception or + `try`/`catch` syntax. +- `.mq.md` first-fit free-list allocator (fixture 55): 8-byte block headers, + free-block reuse, block splitting, address-adjacent coalescing (tested in a + second region whose bump space is exhausted, so only a merged block can + satisfy the request), non-overlapping reused payloads and `alloc.no_space` + returned as a `Result` without aborting the run. No allocator Rust builtin or + `HostOp` was added. + +- `.mq.md` P2 closures beyond the first slice: binary tree insert/in-order + traversal (56), CSV row splitter (57), open-addressing hash table (58), + UTF-8 encoder/validating decoder (59–60), graph adjacency list (61), + recursive JSON parser for arrays/objects/strings (62), stack bytecode VM + (63), 16/32/64-bit + LEB128 codecs (64), table sort (65), table equality join + (67) and quoted/escaped CSV (68). No Rust algorithm, container or parser was + added for these; algorithms remain `.mq.md` code. JSON floats remain open. +- Resource/stream first slice (RFC-CM CM-A06, ADR-0025): opaque host-issued + `Resource` handles, `resource.open/read/write/close`, bytes transferred + through `Memory`, stdout write in pure/project under console/output budgets, + granted project-file chunked read in project, EOF as success, closed-handle + Result errors, forged-handle fatal errors, wire/host-boundary rejection, and + strict refusing every resource open. Semantic fixture 66 and host/CLI tests. +- Buffered project-file write streams (ADR-0026): `file` + `["write"]` requires + an `fs_write` grant, buffers in the host, flushes only on `close`, never + flushes on drop, has an 8 MiB ceiling, and is refused in pure/strict. A + `.mq.md` chunked copy loop reproduces a file without a Rust `copy_file`. +- CM-A07 **Compat-A** prototype (ADR-0027): `examples/stdlib/` provides + `format.显示`, UTF-8 encoding and `io.写值` (`resource.write(stdout)`), and + its output is comparison-tested byte-for-byte against builtin `打印` for + Unit/None/Bool/Int/Float/Text/List/Map/Table. Builtin `打印` remains the + default; no alias routing, bundled stdlib or `HostOp::Print` removal; the + other value types lack public observation. ### Changed +- Assignment targets are no longer restricted to bare identifiers: a single + index place is accepted at statement level. Member assignment stays a parse + error; chained assignment stays a parse error. `spec/expression.ebnf`, + `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md` and + `spec/security.md` document the new semantics. - `docs/STABLE_RELEASE_IMPLEMENTATION.md` is the single next-phase implementation plan. Older plan documents are historical input, not current status. diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index f2fdbd7..15276ad 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -26,7 +26,9 @@ portable/tested locally but the native platform lane has not run here; | Feature | Ubuntu Linux x86_64 | macOS arm64 | Windows x86_64 | |---|---|---|---| | Markdown profile / AST goldens (160) | PASS | UNVERIFIED (cfg fix + CRLF fix) | UNVERIFIED (cfg fix + CRLF fix) | -| Rust interpreter / semantics (44) | PASS | UNVERIFIED | UNVERIFIED | +| Rust interpreter / semantics (68) | PASS | UNVERIFIED | UNVERIFIED | +| Addressable `Memory` P1 (ADR-0022) | PASS | UNVERIFIED | UNVERIFIED | +| Resource streams CM-A06 (ADR-0025/0026: stdout + project file read/write) | PASS | UNVERIFIED | **FAIL-CLOSED** (`strict` refuses resource opens) | | `run` / `call` / `check` / `inspect` / `fmt` | PASS | UNVERIFIED | UNVERIFIED | | lock v3 path/Git graph + sync + tamper | PASS | UNVERIFIED | UNVERIFIED | | Analysis v2 / Trace v2 + Python consumers | PASS | UNVERIFIED | UNVERIFIED | diff --git a/README.md b/README.md index 00be4bb..9c6e6b9 100644 --- a/README.md +++ b/README.md @@ -81,12 +81,33 @@ executes anything. ## What works (implemented and tested) * **P0 contracts** — frozen Markdown profile, AST → program mapping, strict - expression grammar, evaluation semantics; 160 AST goldens, 44 semantic - goldens, 52 compile-error fixtures and 12 runtime-error fixtures + expression grammar, evaluation semantics; 160 AST goldens, 68 semantic + goldens, 52 compile-error fixtures and 21 runtime-error fixtures (`tests/`). * **P1 interpreter** — native Rust tree-walk interpreter: values, scopes, functions, static parameter inference, assignments, branches, loops, returns, recursion, structured errors, fuel/stack/output budgets. +* **Addressable memory (RFC-CM P1)** — opaque run-scoped `Memory` regions with + 64 KiB zero-filled pages, grow-only `memory.new/size/grow`, bounded byte + read/write, `m[i] = byte`, run/per-region budgets, host-boundary and wire + rejection (ADR-0022). +* **Minimal structure primitives (RFC-CM CM-A04)** — value-semantics + `list.append/concat`, `map.keys/has/get/set/remove`, Unicode scalar + `text.codepoints/from_codepoint`, and non-negative `table.columns/row/cell`; + pure builtins with no mutation or new syntax (ADR-0023). +* **Recoverable results (RFC-CM CM-A05)** — `{ok:true,value}` / + `{ok:false,code,message}` as an ordinary Map, with a `.mq.md` first-fit + free-list allocator demonstrating reuse, splitting and `alloc.no_space` + recovery (ADR-0024). No `Result` type or exception syntax. +* **Stdlib print prototype (RFC-CM CM-A07 Compat-A)** — ordinary imported + `.mq.md` `format.显示` + UTF-8 + `resource.write(stdout)` matches builtin + `打印` for the covered value set; the builtin is still the default and no + alias is routed (ADR-0027). +* **Resource streams (RFC-CM CM-A06)** — opaque host-issued `Resource` handles; + `resource.open/read/write/close` transfer bytes through `Memory`; stdout + write in pure/project, granted project-file read, and buffered project-file + write flushed on `close` in project; strict refuses all resource opens until + verified (ADR-0025/0026). * **P2 references + CLI** — explicit YAML front-matter imports, `ModuleId`/ `UnitId`/anchors, import cycle detection, multi-module calls, `run/call/check/inspect/fmt`, reference validation (`check --refs`) and the diff --git a/crates/cli/src/analysis.rs b/crates/cli/src/analysis.rs index 55c8c26..f6eba5e 100644 --- a/crates/cli/src/analysis.rs +++ b/crates/cli/src/analysis.rs @@ -8,7 +8,9 @@ use std::path::Path; use marqdo_expression::{BinaryOp, Expr, UnaryOp}; -use marqdo_ir::{CallTarget, Function, Module, ModuleId, Resource, SignatureKind, Stmt, UnitKind}; +use marqdo_ir::{ + CallTarget, Function, Module, ModuleId, PlaceTarget, Resource, SignatureKind, Stmt, UnitKind, +}; use marqdo_span::Span; use serde_json::{json, Value}; use sha2::{Digest, Sha256}; @@ -232,13 +234,31 @@ fn function_json(function: &Function) -> Value { }) } +/// Project an assignment place. Local targets keep the v1 string form; the new +/// index target is an additive object so existing consumers that compare the +/// local `target` string are unaffected (Analysis v2, ADR-0022). +fn place_json(place: &PlaceTarget) -> Value { + match place { + PlaceTarget::Local { name, .. } => Value::String(name.clone()), + PlaceTarget::Index { base, index, span } => json!({ + "kind": "index", + "base": expr_json(base), + "index": expr_json(index), + "span": span_json(*span), + }), + } +} + fn stmt_json(stmt: &Stmt) -> Value { match stmt { Stmt::Assign { - name, value, span, .. + target, + value, + span, + .. } => json!({ "kind": "assign", - "target": name, + "target": place_json(target), "value": expr_json(value), "span": span_json(*span), }), @@ -497,9 +517,14 @@ fn collect_calls( ) { for stmt in body { match stmt { - Stmt::Assign { value, .. } | Stmt::Eval { expr: value, .. } => { - expr_calls(project, module, function, value, calls) + Stmt::Assign { target, value, .. } => { + if let PlaceTarget::Index { base, index, .. } = target { + expr_calls(project, module, function, base, calls); + expr_calls(project, module, function, index, calls); + } + expr_calls(project, module, function, value, calls); } + Stmt::Eval { expr: value, .. } => expr_calls(project, module, function, value, calls), Stmt::Return { value, .. } => expr_calls(project, module, function, value, calls), Stmt::Call { call, span } => calls.push(json!({ "function": function.id.to_string(), diff --git a/crates/cli/src/commands.rs b/crates/cli/src/commands.rs index 3e78f62..70b8546 100644 --- a/crates/cli/src/commands.rs +++ b/crates/cli/src/commands.rs @@ -4,7 +4,7 @@ use std::path::{Path, PathBuf}; use std::time::{Duration, Instant}; use marqdo_host::{Capabilities, Host, StdHost}; -use marqdo_runtime::{run, EvaluatedArg, RuntimeError, Value}; +use marqdo_runtime::{run_with_limits, EvaluatedArg, MemoryLimits, RuntimeError, Value}; use marqdo_sandbox::{PlatformCapabilities, SandboxMode}; use marqdo_span::Span; use serde_json::json; @@ -32,6 +32,10 @@ pub struct SandboxSettings { pub stack: Option, /// Output limit override. pub max_output: Option, + /// Total committed addressable-memory budget override (bytes). + pub max_memory: Option, + /// Per-region committed addressable-memory budget override (bytes). + pub max_region_memory: Option, /// Wall-clock timeout in seconds. pub timeout: Option, /// Explicitly trusted plugin manifests to load. @@ -884,12 +888,21 @@ pub fn execute( .iter() .map(|(name, value)| EvaluatedArg::named(name.clone(), value.clone(), Span::default())) .collect(); - let result = run( + let memory_limits = MemoryLimits { + max_total_bytes: settings + .max_memory + .unwrap_or(marqdo_runtime::DEFAULT_MAX_TOTAL_BYTES), + max_region_bytes: settings + .max_region_memory + .unwrap_or(marqdo_runtime::DEFAULT_MAX_REGION_BYTES), + }; + let result = run_with_limits( &project.program, &mut host, &project.entry, &function_name, runtime_args, + memory_limits, ); if let Some(path) = &settings.trace_file { let status = match &result { diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index b1c1706..c1f264b 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -103,6 +103,12 @@ struct SandboxArgs { /// Override the program output byte limit. #[arg(long = "max-output")] max_output: Option, + /// Override the total committed addressable-memory budget in bytes. + #[arg(long = "max-memory", value_name = "BYTES")] + max_memory: Option, + /// Override the per-region committed addressable-memory budget in bytes. + #[arg(long = "max-region-memory", value_name = "BYTES")] + max_region_memory: Option, /// Wall-clock timeout in seconds. #[arg(long)] timeout: Option, @@ -137,6 +143,8 @@ impl SandboxArgs { fuel: self.fuel, stack: self.stack, max_output: self.max_output, + max_memory: self.max_memory, + max_region_memory: self.max_region_memory, timeout: self.timeout, plugins: self.plugin.clone(), trust_native_plugin: self.trust_native_plugin, diff --git a/crates/cli/tests/analysis_trace.rs b/crates/cli/tests/analysis_trace.rs index d0e4602..5b125bf 100644 --- a/crates/cli/tests/analysis_trace.rs +++ b/crates/cli/tests/analysis_trace.rs @@ -181,6 +181,34 @@ fn analysis_v2_identity_is_location_independent() { assert_eq!(analysis["project"]["identity_kind"], "manifest"); } +#[test] +fn analysis_v2_projects_memory_place_targets_additively() { + let dir = TempDir::new("analysis-place"); + let source = "## main\n\n**内存 = memory.new(1, 2)**\n\n**`内存[0] = 7`**\n\n*内存[0]*\n"; + write(dir.path(), "index.mq.md", source); + let target = dir.path().join("analysis.json"); + let output = cli( + dir.path(), + &["check", "index.mq.md", "--emit", target.to_str().unwrap()], + ); + assert_eq!( + output.status.code(), + Some(0), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let analysis: serde_json::Value = + serde_json::from_slice(&std::fs::read(&target).unwrap()).unwrap(); + let body = analysis["modules"][0]["functions"][0]["body"] + .as_array() + .expect("body is an array"); + assert_eq!(body[0]["target"], "内存"); + assert_eq!(body[1]["target"]["kind"], "index"); + assert_eq!(body[1]["target"]["base"]["kind"], "variable"); + assert_eq!(body[1]["target"]["index"]["kind"], "literal"); + assert_eq!(body[1]["target"]["index"]["type"], "int"); +} + #[test] fn analysis_v2_projects_addressable_formula_and_code_resources() { let dir = TempDir::new("analysis-resources"); diff --git a/crates/cli/tests/examples_r6.rs b/crates/cli/tests/examples_r6.rs index b27f22a..9702219 100644 --- a/crates/cli/tests/examples_r6.rs +++ b/crates/cli/tests/examples_r6.rs @@ -24,6 +24,49 @@ fn stderr(output: &Output) -> String { String::from_utf8_lossy(&output.stderr).to_string() } +#[test] +fn mqmd_stdlib_print_output_matches_builtin_print() { + let output = cli(&["run", "examples/stdlib/demo.mq.md"]); + assert_eq!(output.status.code(), Some(0), "{}", stderr(&output)); + let stdout = String::from_utf8_lossy(&output.stdout); + let lines: Vec<&str> = stdout.lines().collect(); + assert_eq!(lines.len(), 4, "{stdout:?}"); + assert_eq!(lines[0], lines[1], "list formatting differs from 打印"); + assert_eq!(lines[2], lines[3], "table formatting differs from 打印"); + assert_eq!( + lines[0], + r#"["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]"# + ); + assert_eq!(lines[2], "Table[名称 | 分数]{甲 | 87; 乙 | 92}"); +} + +#[test] +fn addressable_memory_example_runs_through_the_cli() { + let output = cli(&["run", "examples/memory-buffer.mq.md", "--json"]); + assert_eq!(output.status.code(), Some(0), "{}", stderr(&output)); + let report: serde_json::Value = + serde_json::from_slice(&output.stdout).expect("run --json emits JSON"); + assert_eq!(report["success"], true); + assert_eq!(report["value"]["type"], "int"); + assert_eq!(report["value"]["decimal"], "141"); +} + +#[test] +fn cli_memory_budget_override_fails_closed() { + let output = cli(&[ + "run", + "examples/memory-buffer.mq.md", + "--max-memory", + "0", + "--json", + ]); + assert_eq!(output.status.code(), Some(1), "{}", stderr(&output)); + let report: serde_json::Value = + serde_json::from_slice(&output.stdout).expect("run --json emits JSON"); + assert_eq!(report["success"], false); + assert_eq!(report["error"]["code"], "resource.exhausted"); +} + #[test] fn multi_version_packages_run_without_an_implicit_override() { let output = cli(&["run", "examples/packages-multi-version/index.mq.md"]); diff --git a/crates/cli/tests/resource_streams.rs b/crates/cli/tests/resource_streams.rs new file mode 100644 index 0000000..944270b --- /dev/null +++ b/crates/cli/tests/resource_streams.rs @@ -0,0 +1,235 @@ +//! End-to-end resource stream tests through the CLI. + +use std::path::{Path, PathBuf}; +use std::process::{Command, Output}; + +struct TempDir(PathBuf); + +impl TempDir { + fn new(tag: &str) -> Self { + let path = + std::env::temp_dir().join(format!("marqdo-resource-test-{}-{tag}", std::process::id())); + let _ = std::fs::remove_dir_all(&path); + std::fs::create_dir_all(&path).unwrap(); + Self(path) + } + + fn path(&self) -> &Path { + &self.0 + } +} + +impl Drop for TempDir { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.0); + } +} + +fn cli(dir: &Path, args: &[&str]) -> Output { + Command::new(env!("CARGO_BIN_EXE_marqdo")) + .args(args) + .current_dir(dir) + .output() + .expect("run marqdo") +} + +fn write_file(path: &Path, contents: &str) { + std::fs::write(path, contents).unwrap(); +} + +const READER: &str = r#"## main + +**内存 = memory.new(1, 1)** + +**流 = resource.open("file", "data.txt", ["read"])** + +**第一次 = resource.read(流, 内存, 0, 2)** + +**第二次 = resource.read(流, 内存, 2, 4)** + +**第三次 = resource.read(流, 内存, 6, 1)** + +**关闭 = resource.close(流)** + +*[第一次.value.count, 第一次.value.eof, 第二次.value.count, 第二次.value.eof, 第三次.value.count, 第三次.value.eof, 内存[0], 内存[1], 内存[2], 内存[3], 内存[4], 内存[5], 关闭.ok]* +"#; + +#[test] +fn project_file_stream_reads_chunks_into_memory() { + let dir = TempDir::new("file-read"); + write_file(&dir.path().join("data.txt"), "abcdef"); + write_file(&dir.path().join("index.mq.md"), READER); + let output = cli( + dir.path(), + &["run", "index.mq.md", "--allow-read", "data.txt", "--json"], + ); + assert_eq!( + output.status.code(), + Some(0), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output"); + assert_eq!(report["success"], true); + let values: Vec<&serde_json::Value> = report["value"]["items"] + .as_array() + .expect("list result") + .iter() + .collect(); + assert_eq!(values[0]["decimal"], "2"); + assert_eq!(values[1]["value"], false); + assert_eq!(values[2]["decimal"], "4"); + assert_eq!(values[3]["value"], true); + assert_eq!(values[4]["decimal"], "0"); + assert_eq!(values[5]["value"], true); + for (index, expected) in ["97", "98", "99", "100", "101", "102"] + .into_iter() + .enumerate() + { + assert_eq!(values[6 + index]["decimal"], expected); + } + assert_eq!(values[12]["value"], true); +} + +const WRITER: &str = r#"## main + +**内存 = memory.new(1, 1)** + +**`内存[0] = 65`** + +**`内存[1] = 66`** + +**`内存[2] = 67`** + +**写 = resource.open("file", "out.bin", ["write"])** + +**写入 = resource.write(写, 内存, 0, 3)** + +**关闭写 = resource.close(写)** + +**读 = resource.open("file", "out.bin", ["read"])** + +**读出 = resource.read(读, 内存, 0, 3)** + +**关闭读 = resource.close(读)** + +*[写入.value.count, 关闭写.ok, 读出.value.count, 读出.value.eof, 内存[0], 内存[1], 内存[2], 关闭读.ok]* +"#; + +#[test] +fn project_file_stream_writes_on_close_then_reads_back() { + let dir = TempDir::new("file-write"); + write_file(&dir.path().join("index.mq.md"), WRITER); + let output = cli( + dir.path(), + &[ + "run", + "index.mq.md", + "--allow-write", + "out.bin", + "--allow-read", + "out.bin", + "--json", + ], + ); + assert_eq!( + output.status.code(), + Some(0), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output"); + assert_eq!(report["success"], true); + let values = report["value"]["items"].as_array().expect("list result"); + assert_eq!(values[0]["decimal"], "3"); + assert_eq!(values[1]["value"], true); + assert_eq!(values[2]["decimal"], "3"); + assert_eq!(values[3]["value"], true); + assert_eq!(values[4]["decimal"], "65"); + assert_eq!(values[5]["decimal"], "66"); + assert_eq!(values[6]["decimal"], "67"); + assert_eq!(values[7]["value"], true); + assert_eq!(std::fs::read(dir.path().join("out.bin")).unwrap(), b"ABC"); +} + +const COPY: &str = r#"## main + +**输入 = resource.open("file", "data.txt", ["read"])** + +**输出 = resource.open("file", "out.bin", ["write"])** + +**内存 = memory.new(1, 1)** + +**总数 = 0** + +**继续 = true** + +- `继续` + + **块 = resource.read(输入, 内存, 0, 16)** + + 1. `块.ok` + + > resource.write(输出, 内存, 0, 块.value.count) + + **总数 = 总数 + 块.value.count** + + **继续 = not 块.value.eof** + + 2. `true` + + **继续 = false** + +**关闭读 = resource.close(输入)** + +**关闭写 = resource.close(输出)** + +*[总数, 关闭读.ok, 关闭写.ok]* +"#; + +#[test] +fn file_copy_is_written_in_mqmd_through_chunked_streams() { + let dir = TempDir::new("file-copy"); + write_file(&dir.path().join("data.txt"), "abcdef"); + write_file(&dir.path().join("index.mq.md"), COPY); + let output = cli( + dir.path(), + &[ + "run", + "index.mq.md", + "--allow-read", + "data.txt", + "--allow-write", + "out.bin", + "--json", + ], + ); + assert_eq!( + output.status.code(), + Some(0), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output"); + assert_eq!(report["success"], true); + let values = report["value"]["items"].as_array().expect("list result"); + assert_eq!(values[0]["decimal"], "6"); + assert_eq!(values[1]["value"], true); + assert_eq!(values[2]["value"], true); + assert_eq!( + std::fs::read(dir.path().join("out.bin")).unwrap(), + b"abcdef" + ); +} + +#[test] +fn file_stream_requires_a_read_grant() { + let dir = TempDir::new("file-denied"); + write_file(&dir.path().join("data.txt"), "abcdef"); + write_file(&dir.path().join("index.mq.md"), READER); + let output = cli(dir.path(), &["run", "index.mq.md", "--json"]); + assert_eq!(output.status.code(), Some(1)); + let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output"); + assert_eq!(report["success"], false); + assert_eq!(report["error"]["code"], "permission.denied"); +} diff --git a/crates/marqdo-expression/src/ast.rs b/crates/marqdo-expression/src/ast.rs index c041b28..bc597cb 100644 --- a/crates/marqdo-expression/src/ast.rs +++ b/crates/marqdo-expression/src/ast.rs @@ -195,15 +195,50 @@ impl Expr { } } -/// The payload of a statement. +/// The writable target on the left of `=` (ADR-0022). +/// +/// v1 accepted only [`AssignTarget::Local`]. RFC-CM-0001 adds the second +/// variant because addressable `Memory` needs a writable place; no other +/// collection becomes mutable. Both variants use the same assignment syntax +/// and the same statement position. #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -pub enum StatementKind { - /// ` = `; the only assignment form in v1. - Assign { +#[serde(tag = "place", rename_all = "snake_case")] +pub enum AssignTarget { + /// `name = value`: rebind a local slot in the current frame. + Local { /// Assigned identifier. name: String, /// Span of the identifier token (not the whole assignment). name_span: Span, + }, + /// `base[index] = value`: write one byte into an addressable `Memory`. + Index { + /// Base expression, evaluated first. + base: Box, + /// Index expression, evaluated second. + index: Box, + /// Span from the base start through the closing `]`. + span: Span, + }, +} + +impl AssignTarget { + /// Best known source span of the assignment target. + pub fn span(&self) -> Span { + match self { + AssignTarget::Local { name_span, .. } => *name_span, + AssignTarget::Index { span, .. } => *span, + } + } +} + +/// The payload of a statement. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub enum StatementKind { + /// `target = expression`; see [`AssignTarget`] for the writable places. + Assign { + /// Assigned place. + target: AssignTarget, /// Right-hand side expression. value: Expr, }, diff --git a/crates/marqdo-expression/src/lib.rs b/crates/marqdo-expression/src/lib.rs index f26748a..69f714d 100644 --- a/crates/marqdo-expression/src/lib.rs +++ b/crates/marqdo-expression/src/lib.rs @@ -36,8 +36,8 @@ //! The grammar is frozen in `spec/expression.ebnf`. In order of decreasing //! precedence: prefix `not` / `-` / `+`, `**` (right associative), postfix //! calls `.name` / `[index]`, `* / // %`, `+ -`, a single non-associative -//! comparison, `and`, `or`. Assignment is only available at statement start -//! and only for a bare identifier target. +//! comparison, `and`, `or`. Assignment is only available at statement start; +//! its target is a bare identifier or an index expression (`m[8] = 42`). //! //! Notable lexical rules: //! @@ -84,7 +84,9 @@ use std::fmt; use serde::Serialize; -pub use ast::{Arg, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp}; +pub use ast::{ + Arg, AssignTarget, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp, +}; pub use marqdo_span::Span; pub use parser::MAX_NESTING_DEPTH; diff --git a/crates/marqdo-expression/src/parser.rs b/crates/marqdo-expression/src/parser.rs index 239bacf..9436e3e 100644 --- a/crates/marqdo-expression/src/parser.rs +++ b/crates/marqdo-expression/src/parser.rs @@ -17,7 +17,9 @@ //! recursing into any sub-expression so adversarial inputs fail with a //! structured [`ParseError`] instead of overflowing the stack. -use crate::ast::{Arg, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp}; +use crate::ast::{ + Arg, AssignTarget, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp, +}; use crate::lexer::{self, Token, TokenKind}; use crate::ParseError; use marqdo_span::Span; @@ -47,21 +49,16 @@ pub(crate) fn parse_statement(source: &str) -> Result { return Err(parser.error_here("expected a statement, found end of input")); } let start = parser.peek().span.start; - if let TokenKind::Ident(_) = parser.peek().kind { - if matches!( - parser.peek_n(1).map(|token| &token.kind), - Some(TokenKind::Assign) - ) { - return parser.assignment(start); - } + let left = parser.expression()?; + if matches!(parser.peek().kind, TokenKind::Assign) { + return parser.assignment(start, left); } - let expr = parser.expression()?; if !parser.at_eof() { - return Err(parser.trailing_statement_error(&expr)); + return Err(parser.trailing_statement_error()); } let end = parser.previous_end(); Ok(Statement { - kind: StatementKind::Expr(expr), + kind: StatementKind::Expr(left), span: Span::new(start, end), }) } @@ -201,11 +198,8 @@ impl Parser { // Statements // ------------------------------------------------------------------ - fn assignment(&mut self, start: usize) -> Result { - let TokenKind::Ident(name) = self.peek().kind.clone() else { - return Err(self.error_here("expected an identifier on the left of `=`")); - }; - let name_token = self.bump(); + fn assignment(&mut self, start: usize, left: Expr) -> Result { + let target = self.assign_target(left)?; self.bump(); // `=` if self.at_eof() { return Err(self.error_here("expected an expression after `=`")); @@ -226,29 +220,40 @@ impl Parser { } let end = self.previous_end(); Ok(Statement { - kind: StatementKind::Assign { - name, - name_span: name_token.span, - value, - }, + kind: StatementKind::Assign { target, value }, span: Span::new(start, end), }) } - fn trailing_statement_error(&self, expr: &Expr) -> ParseError { - if matches!(self.peek().kind, TokenKind::Assign) { - return if matches!(expr, Expr::Member { .. } | Expr::Index { .. }) { - self.error_here( - "invalid assignment target: member and index assignment are not supported \ - in v1; assign a bare identifier", - ) - } else { - self.error_here( - "invalid assignment target: assignment requires a bare identifier on the \ - left of `=`", - ) - }; + /// Turn the already-parsed left expression into a writable target. + /// + /// v1 only writes local slots. Addressable `Memory` adds index targets; + /// member assignment and every other shape stay hard parse errors so an + /// invalid executable shape never silently becomes prose (ADR-0022). + fn assign_target(&self, left: Expr) -> Result { + match left { + Expr::Ident(Ident { name, span }) => Ok(AssignTarget::Local { + name, + name_span: span, + }), + Expr::Index { base, index, span } => Ok(AssignTarget::Index { base, index, span }), + Expr::Member { span, .. } => Err(ParseError::new( + span, + "member assignment is not supported in v1; only a local name or a \ + Memory index is writable", + )), + other => Err(ParseError::new( + other.span(), + format!( + "invalid assignment target: {} is not a writable place; assign a local \ + name or a Memory index", + describe_expr(&other) + ), + )), } + } + + fn trailing_statement_error(&self) -> ParseError { self.error_here(format!( "unexpected {} after expression", self.describe_current() @@ -705,6 +710,24 @@ fn is_keyword(kind: &TokenKind) -> bool { ) } +fn describe_expr(expr: &Expr) -> &'static str { + match expr { + Expr::Int(_) => "an integer literal", + Expr::Float(_) => "a float literal", + Expr::Text(_) => "a text literal", + Expr::Bool(_) => "a boolean literal", + Expr::None => "`None`", + Expr::Ident(_) => "an identifier", + Expr::List(_) => "a list literal", + Expr::Record(_) => "a record literal", + Expr::Unary { .. } => "a unary expression", + Expr::Binary { .. } => "a binary expression", + Expr::Call { .. } => "a call expression", + Expr::Member { .. } => "a member access", + Expr::Index { .. } => "an index expression", + } +} + fn describe_token(kind: &TokenKind) -> String { match kind { TokenKind::Int(raw) => format!("integer literal `{raw}`"), @@ -1154,14 +1177,19 @@ mod tests { assert_eq!( stmt.kind, StatementKind::Assign { - name: "x".to_owned(), - name_span: Span::new(0, 1), + target: AssignTarget::Local { + name: "x".to_owned(), + name_span: Span::new(0, 1), + }, value: Expr::Int(BigInt::from(1)), } ); let stmt = st("结果 = 阶乘(n=3)"); match &stmt.kind { - StatementKind::Assign { name, value, .. } => { + StatementKind::Assign { + target: AssignTarget::Local { name, .. }, + value, + } => { assert_eq!(name, "结果"); assert_eq!(sexpr(value), "(call 阶乘 n=3)"); } @@ -1170,6 +1198,40 @@ mod tests { assert_eq!(st("x = y + 1").span(), Span::new(0, 9)); } + #[test] + fn memory_index_assignment_parses_as_a_place() { + let stmt = st("m[8] = 42"); + let StatementKind::Assign { target, value } = &stmt.kind else { + panic!("expected assignment, got {:?}", stmt.kind); + }; + let AssignTarget::Index { base, index, span } = target else { + panic!("expected index target, got {target:?}"); + }; + assert_eq!( + **base, + Expr::Ident(Ident { + name: "m".to_owned(), + span: Span::new(0, 1), + }) + ); + assert_eq!(**index, Expr::Int(BigInt::from(8))); + assert_eq!(*span, Span::new(0, 4)); + assert_eq!(*value, Expr::Int(BigInt::from(42))); + assert_eq!(stmt.span, Span::new(0, 9)); + + // Nested targets are syntactically places; the runtime decides whether + // the base value is a writable Memory region. + let stmt = st("outer[0][1] = 2"); + let StatementKind::Assign { + target: AssignTarget::Index { base, .. }, + .. + } = &stmt.kind + else { + panic!("expected nested index target, got {:?}", stmt.kind); + }; + assert!(matches!(base.as_ref(), Expr::Index { .. })); + } + #[test] fn expression_statements_parse() { assert_eq!(st("打印(1)").kind, StatementKind::Expr(ex("打印(1)"))); @@ -1185,8 +1247,8 @@ mod tests { "x =", "x = 1 = 2", "a.b = 1", - "a[0] = 1", "1 = 2", + "f() = 1", "= 1", "x = 1 2", ] { @@ -1194,7 +1256,8 @@ mod tests { assert_eq!(err.code, "expr.parse", "input {bad:?}"); } assert!(st_err("x = 1 = 2").message.contains("chained assignment")); - assert!(st_err("a.b = 1").message.contains("member and index")); + assert!(st_err("a.b = 1").message.contains("member assignment")); + assert!(st_err("1 = 2").message.contains("writable place")); assert!(st_err("x =").message.contains("after `=`")); } @@ -1228,7 +1291,10 @@ mod tests { let stmt = st(" x = 1 "); assert_eq!(stmt.span(), Span::new(2, 7)); match stmt.kind { - StatementKind::Assign { name_span, .. } => assert_eq!(name_span, Span::new(2, 3)), + StatementKind::Assign { + target: AssignTarget::Local { name_span, .. }, + .. + } => assert_eq!(name_span, Span::new(2, 3)), other => panic!("expected assignment, got {other:?}"), } diff --git a/crates/marqdo-frontend/src/calls.rs b/crates/marqdo-frontend/src/calls.rs index 223e729..b7defd3 100644 --- a/crates/marqdo-frontend/src/calls.rs +++ b/crates/marqdo-frontend/src/calls.rs @@ -8,7 +8,7 @@ use std::collections::{BTreeMap, BTreeSet}; use marqdo_expression::{Arg, Expr, Ident}; -use marqdo_ir::{builtins, Module, ModuleId, Param, Program, SignatureKind, Stmt}; +use marqdo_ir::{builtins, Module, ModuleId, Param, PlaceTarget, Program, SignatureKind, Stmt}; use marqdo_span::Span; use marqdo_value::{codes, Diagnostic}; @@ -109,7 +109,10 @@ pub fn validate_program(program: &Program) -> Vec<(ModuleId, Diagnostic)> { fn collect_assigned(stmts: &[Stmt], out: &mut BTreeSet) { for stmt in stmts { match stmt { - Stmt::Assign { name, .. } => { + Stmt::Assign { + target: PlaceTarget::Local { name, .. }, + .. + } => { out.insert(name.clone()); } Stmt::Branch { arms, .. } => { @@ -138,9 +141,14 @@ impl<'a> CallChecker<'a> { fn stmts(&mut self, stmts: &[Stmt]) { for stmt in stmts { match stmt { - Stmt::Assign { value, .. } - | Stmt::Eval { expr: value, .. } - | Stmt::Return { value, .. } => { + Stmt::Assign { target, value, .. } => { + if let PlaceTarget::Index { base, index, .. } = target { + self.expr(base); + self.expr(index); + } + self.expr(value); + } + Stmt::Eval { expr: value, .. } | Stmt::Return { value, .. } => { self.expr(value); } Stmt::Call { call, .. } => { diff --git a/crates/marqdo-frontend/src/infer.rs b/crates/marqdo-frontend/src/infer.rs index 7c0e0a5..8785555 100644 --- a/crates/marqdo-frontend/src/infer.rs +++ b/crates/marqdo-frontend/src/infer.rs @@ -17,7 +17,7 @@ use std::collections::{BTreeMap, BTreeSet}; use marqdo_expression::{Arg, Expr, Ident, RecordEntry}; -use marqdo_ir::{BranchArm, Stmt}; +use marqdo_ir::{BranchArm, PlaceTarget, Stmt}; use marqdo_span::Span; use marqdo_value::{codes, Diagnostic}; @@ -178,9 +178,24 @@ impl<'a> Analyzer<'a> { fn analyze_stmts(&mut self, stmts: &[Stmt], mut state: State) -> State { for stmt in stmts { state = match stmt { - Stmt::Assign { name, value, .. } => { - self.read_expr(value, &state); - self.assign(name, &mut state); + Stmt::Assign { target, value, .. } => { + match target { + PlaceTarget::Local { name, .. } => { + // Preserve v1 order: the RHS is read before the + // local rebinding makes the name definitely + // assigned. + self.read_expr(value, &state); + self.assign(name, &mut state); + } + PlaceTarget::Index { base, index, .. } => { + // Runtime order for an index place is base, index, + // RHS; reads in the place are free reads like any + // other. + self.read_expr(base, &state); + self.read_expr(index, &state); + self.read_expr(value, &state); + } + } state } Stmt::Eval { expr, .. } => { @@ -312,8 +327,10 @@ mod tests { fn assign(name: &str, value: &str) -> Stmt { Stmt::Assign { - name: name.to_string(), - name_span: Span::new(0, 0), + target: PlaceTarget::Local { + name: name.to_string(), + name_span: Span::new(0, 0), + }, value: parse_expression(value).unwrap(), span: Span::new(0, 0), } @@ -334,6 +351,24 @@ mod tests { .collect() } + #[test] + fn index_assignment_reads_its_base_and_index_without_binding_a_local() { + let stmts = vec![ + Stmt::Assign { + target: PlaceTarget::Index { + base: parse_expression("m").unwrap(), + index: parse_expression("i").unwrap(), + span: Span::new(0, 0), + }, + value: parse_expression("9").unwrap(), + span: Span::new(0, 0), + }, + ret("m[0]"), + ]; + // `m` and `i` are free reads; the place never rebinds `m` as a local. + assert_eq!(params(&stmts), vec!["i".to_string(), "m".to_string()]); + } + #[test] fn infers_free_reads() { // n * 阶乘(n=n-1) diff --git a/crates/marqdo-frontend/src/map.rs b/crates/marqdo-frontend/src/map.rs index b2aab42..d5ce6f7 100644 --- a/crates/marqdo-frontend/src/map.rs +++ b/crates/marqdo-frontend/src/map.rs @@ -6,9 +6,9 @@ //! diagnostics and never fall back to prose. use marqdo_expression::{ - parse_arguments, parse_expression, parse_statement, Arg, Expr, StatementKind, + parse_arguments, parse_expression, parse_statement, Arg, AssignTarget, Expr, StatementKind, }; -use marqdo_ir::{CallSite, CallTarget, ImportDecl, Stmt}; +use marqdo_ir::{CallSite, CallTarget, ImportDecl, PlaceTarget, Stmt}; use marqdo_markdown::{normalize_name, MarkdownDocument, MdKind, MdNode}; use marqdo_span::Span; use marqdo_value::{codes, Diagnostic}; @@ -143,13 +143,21 @@ impl<'a> Mapper<'a> { Ok(mut statement) => { segments.rewrite_statement(&mut statement); Some(match statement.kind { - StatementKind::Assign { - name, - name_span, - value, - } => Stmt::Assign { - name, - name_span, + StatementKind::Assign { target, value } => Stmt::Assign { + target: match target { + AssignTarget::Local { name, name_span } => { + PlaceTarget::Local { name, name_span } + } + AssignTarget::Index { + base, + index, + span: target_span, + } => PlaceTarget::Index { + base: *base, + index: *index, + span: target_span, + }, + }, value, span: node.span, }, diff --git a/crates/marqdo-frontend/src/payload.rs b/crates/marqdo-frontend/src/payload.rs index a5597b4..660a581 100644 --- a/crates/marqdo-frontend/src/payload.rs +++ b/crates/marqdo-frontend/src/payload.rs @@ -5,7 +5,7 @@ //! string, so its spans are payload-relative; this module maps them back to the //! original file so diagnostics point at real source. -use marqdo_expression::{Arg, Expr, RecordEntry, Statement, StatementKind}; +use marqdo_expression::{Arg, AssignTarget, Expr, RecordEntry, Statement, StatementKind}; use marqdo_markdown::{MdKind, MdNode, PayloadError}; use marqdo_span::Span; @@ -140,11 +140,17 @@ impl PayloadSegments { /// Map and rewrite every span in a parsed statement. pub fn rewrite_statement(&self, statement: &mut Statement) { statement.span = self.map_span(statement.span); - if let StatementKind::Assign { - name_span, value, .. - } = &mut statement.kind - { - *name_span = self.map_span(*name_span); + if let StatementKind::Assign { target, value } = &mut statement.kind { + match target { + AssignTarget::Local { name_span, .. } => { + *name_span = self.map_span(*name_span); + } + AssignTarget::Index { base, index, span } => { + *span = self.map_span(*span); + self.rewrite_expr(base); + self.rewrite_expr(index); + } + } self.rewrite_expr(value); } if let StatementKind::Expr(expr) = &mut statement.kind { diff --git a/crates/marqdo-host/src/fs.rs b/crates/marqdo-host/src/fs.rs index 6b868fa..10c3015 100644 --- a/crates/marqdo-host/src/fs.rs +++ b/crates/marqdo-host/src/fs.rs @@ -83,6 +83,22 @@ impl ProjectFs { /// Read a UTF-8 file, bounded by [`MAX_READ_BYTES`]. pub fn read(&self, user_path: &str, span: Option) -> Result { + let bytes = self.read_bytes(user_path, span)?; + String::from_utf8(bytes).map_err(|_| { + RuntimeError::new( + ErrorCategory::Type, + span, + format!("`{user_path}` is not valid UTF-8"), + ) + }) + } + + /// Read a file as raw bytes, bounded by [`MAX_READ_BYTES`]. + /// + /// Used by the resource stream provider; path/capability checks are the + /// caller's responsibility for grants, while this method still rejects + /// traversal and symlink escapes. + pub fn read_bytes(&self, user_path: &str, span: Option) -> Result, RuntimeError> { let (absolute, relative) = self.resolve(user_path, span)?; let metadata = std::fs::metadata(&absolute).map_err(|error| { RuntimeError::new( @@ -118,19 +134,12 @@ impl ProjectFs { if !canonical.starts_with(&self.root) { return Err(invalid_path(span, "path escapes the project root")); } - let bytes = std::fs::read(&canonical).map_err(|error| { + std::fs::read(&canonical).map_err(|error| { RuntimeError::new( ErrorCategory::HostError, span, format!("cannot read `{relative}`: {error}"), ) - })?; - String::from_utf8(bytes).map_err(|_| { - RuntimeError::new( - ErrorCategory::Type, - span, - format!("`{relative}` is not valid UTF-8"), - ) }) } @@ -140,6 +149,18 @@ impl ProjectFs { user_path: &str, contents: &str, span: Option, + ) -> Result<(), RuntimeError> { + self.write_bytes(user_path, contents.as_bytes(), span) + } + + /// Write raw bytes inside the project root, creating parent directories. + /// + /// This is a single validated write, not a transactional replace. + pub fn write_bytes( + &self, + user_path: &str, + bytes: &[u8], + span: Option, ) -> Result<(), RuntimeError> { let (absolute, relative) = self.resolve(user_path, span)?; // Validate the closest existing ancestor stays inside the root. @@ -169,7 +190,7 @@ impl ProjectFs { ) })?; } - std::fs::write(&absolute, contents).map_err(|error| { + std::fs::write(&absolute, bytes).map_err(|error| { RuntimeError::new( ErrorCategory::HostError, span, diff --git a/crates/marqdo-host/src/host.rs b/crates/marqdo-host/src/host.rs index 16fbce2..3ccd342 100644 --- a/crates/marqdo-host/src/host.rs +++ b/crates/marqdo-host/src/host.rs @@ -12,7 +12,7 @@ use marqdo_plugin_abi::{ ErrorKind as PluginErrorKind, LoadedPlugin, PluginError, PluginKind, PluginManifest, }; use marqdo_span::Span; -use marqdo_value::{ErrorCategory, RuntimeError, Value}; +use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value}; use crate::budget::Budget; use crate::capability::{Capabilities, CapabilityAudit, CapabilityBroker}; @@ -20,6 +20,7 @@ use crate::context::ExecutionContext; use crate::fs::ProjectFs; use crate::handles::{contains_plugin_handle, PluginHandleTable}; use crate::plugin_worker::WorkerPlugin; +use crate::resource::{ResourceRead, ResourceTable}; use crate::trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState}; use marqdo_sandbox::{PlatformCapabilities, SandboxMode}; @@ -79,6 +80,65 @@ pub trait Host { ctx: &ExecutionContext, ) -> Result; + /// Open a resource/stream by kind, descriptor and requested rights. + /// + /// Default: unavailable. Only the standard host implements the first + /// resource kinds (ADR-0025). + fn resource_open( + &mut self, + _kind: &str, + _descriptor: &str, + _rights: &[String], + ctx: &ExecutionContext, + ) -> Result { + Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + ctx.span, + "this host does not provide resource streams", + )) + } + + /// Read up to `max_bytes` from a resource into host memory. + fn resource_read( + &mut self, + _handle: ResourceHandle, + _max_bytes: usize, + ctx: &ExecutionContext, + ) -> Result { + Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + ctx.span, + "this host does not provide resource streams", + )) + } + + /// Write bytes to a resource; returns the accepted count. + fn resource_write( + &mut self, + _handle: ResourceHandle, + _bytes: &[u8], + ctx: &ExecutionContext, + ) -> Result { + Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + ctx.span, + "this host does not provide resource streams", + )) + } + + /// Close a resource handle. + fn resource_close( + &mut self, + _handle: ResourceHandle, + ctx: &ExecutionContext, + ) -> Result<(), RuntimeError> { + Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + ctx.span, + "this host does not provide resource streams", + )) + } + /// Consume evaluation fuel. fn consume_fuel(&mut self, amount: u64, span: Option) -> Result<(), RuntimeError>; @@ -143,6 +203,8 @@ pub struct StdHost { broker_audit: Arc>>, /// Host-issued opaque plugin handles. pub handles: PluginHandleTable, + /// Host-owned resource/stream table (ADR-0025). + resources: ResourceTable, semantic_events: Vec, next_event_seq: u64, trace_state: TraceState, @@ -203,6 +265,7 @@ impl StdHost { vendor_capabilities: BTreeSet::new(), broker_audit: Arc::new(Mutex::new(Vec::new())), handles: PluginHandleTable::default(), + resources: ResourceTable::new(), semantic_events: Vec::new(), next_event_seq: 0, trace_state: TraceState::default(), @@ -526,6 +589,96 @@ impl StdHost { self.fs.write(&path, &contents, ctx.span)?; Ok(Value::Unit) } + + fn open_resource_stream( + &mut self, + kind: &str, + descriptor: &str, + rights: &[String], + ctx: &ExecutionContext, + ) -> Result { + if self.mode == SandboxMode::Strict { + return Err(RuntimeError::new( + ErrorCategory::PermissionDenied, + ctx.span, + "resource streams are not verified in strict mode and are refused", + )); + } + if self.mode == SandboxMode::Pure && kind == "file" { + return Err(RuntimeError::new( + ErrorCategory::PermissionDenied, + ctx.span, + "pure mode does not provide project file resources", + )); + } + let requested: BTreeSet<&str> = rights.iter().map(String::as_str).collect(); + match kind { + "stdio" => { + if descriptor != "stdout" || requested != BTreeSet::from(["write"]) { + return Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + ctx.span, + "the first stream slice supports only stdio/stdout with rights [write]", + )); + } + if !self.capabilities.console { + return Err(RuntimeError::new( + ErrorCategory::PermissionDenied, + ctx.span, + "the console capability is required for stdio/stdout", + )); + } + let handle = self.resources.open_stdout(); + self.record_semantic( + "resource.open", + None, + Some(&ctx.module_id), + ctx.span, + Some("kind=stdio descriptor=stdout"), + ); + Ok(Value::Resource(handle)) + } + "file" => { + if requested == BTreeSet::from(["read"]) { + self.capabilities.check_read(descriptor, ctx.span)?; + let (_, relative) = self.fs.resolve(descriptor, ctx.span)?; + let bytes = self.fs.read_bytes(descriptor, ctx.span)?; + let handle = self.resources.open_file(relative, bytes); + self.record_semantic( + "resource.open", + None, + Some(&ctx.module_id), + ctx.span, + Some("kind=file rights=read"), + ); + return Ok(Value::Resource(handle)); + } + if requested == BTreeSet::from(["write"]) { + self.capabilities.check_write(descriptor, ctx.span)?; + let (_, relative) = self.fs.resolve(descriptor, ctx.span)?; + let handle = self.resources.open_file_writer(relative); + self.record_semantic( + "resource.open", + None, + Some(&ctx.module_id), + ctx.span, + Some("kind=file rights=write"), + ); + return Ok(Value::Resource(handle)); + } + Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + ctx.span, + "project file streams support exactly [read] or [write] in this build", + )) + } + other => Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + ctx.span, + format!("resource kind `{other}` is not implemented in this build"), + )), + } + } } impl Host for StdHost { @@ -535,6 +688,31 @@ impl Host for StdHost { args: Vec, ctx: &ExecutionContext, ) -> Result { + // Addressable memory is language-local state. Handles must not reach a + // plugin, foreign kernel or file adapter, where they could not be + // interpreted safely (RFC-CM-0001 §8, ADR-0022 §4). + if args.iter().any(Value::contains_memory) { + return Err(RuntimeError::new( + ErrorCategory::Type, + ctx.span, + format!( + "Memory values cannot cross the host boundary (`{}`); copy bytes into \ + Text explicitly first", + op.name() + ), + )); + } + if args.iter().any(Value::contains_resource) { + return Err(RuntimeError::new( + ErrorCategory::Type, + ctx.span, + format!( + "Resource handles cannot cross the generic host boundary (`{}`); use \ + resource.* operations", + op.name() + ), + )); + } let started = Instant::now(); let result = self.dispatch(op, args, ctx); let summary = match &result { @@ -562,6 +740,84 @@ impl Host for StdHost { result } + fn resource_open( + &mut self, + kind: &str, + descriptor: &str, + rights: &[String], + ctx: &ExecutionContext, + ) -> Result { + self.open_resource_stream(kind, descriptor, rights, ctx) + } + + fn resource_read( + &mut self, + handle: ResourceHandle, + max_bytes: usize, + ctx: &ExecutionContext, + ) -> Result { + let kind = self.resources.kind(handle, ctx.span)?; + let read = self.resources.read(handle, max_bytes, ctx.span)?; + self.record_semantic( + "resource.read", + None, + Some(&ctx.module_id), + ctx.span, + Some(&format!( + "kind={kind} count={} eof={}", + read.bytes.len(), + read.eof + )), + ); + Ok(read) + } + + fn resource_write( + &mut self, + handle: ResourceHandle, + bytes: &[u8], + ctx: &ExecutionContext, + ) -> Result { + let kind = self.resources.kind(handle, ctx.span)?; + let count = self.resources.write(handle, bytes, ctx.span)?; + if kind == "stdio" { + self.budget.consume_output(count, ctx.span)?; + self.output.extend_from_slice(bytes); + } + self.record_semantic( + "resource.write", + None, + Some(&ctx.module_id), + ctx.span, + Some(&format!("kind={kind} count={count}")), + ); + Ok(count) + } + + fn resource_close( + &mut self, + handle: ResourceHandle, + ctx: &ExecutionContext, + ) -> Result<(), RuntimeError> { + let pending = self.resources.pending_file_write(handle, ctx.span)?; + let count = match pending { + Some((relative, bytes)) => { + self.fs.write_bytes(&relative, &bytes, ctx.span)?; + bytes.len() + } + None => 0, + }; + self.resources.close(handle, ctx.span)?; + self.record_semantic( + "resource.close", + None, + Some(&ctx.module_id), + ctx.span, + Some(&format!("flushed_bytes={count}")), + ); + Ok(()) + } + fn consume_fuel(&mut self, amount: u64, span: Option) -> Result<(), RuntimeError> { self.budget.consume_fuel(amount, span) } diff --git a/crates/marqdo-host/src/lib.rs b/crates/marqdo-host/src/lib.rs index 7269f76..2134093 100644 --- a/crates/marqdo-host/src/lib.rs +++ b/crates/marqdo-host/src/lib.rs @@ -12,6 +12,7 @@ pub mod handles; pub mod host; pub mod kernels; mod plugin_worker; +pub mod resource; pub mod trace; pub use budget::{Budget, DEFAULT_FUEL, DEFAULT_OUTPUT_BYTES, DEFAULT_STACK_LIMIT}; @@ -21,4 +22,5 @@ pub use fs::ProjectFs; pub use handles::{contains_plugin_handle, PluginHandleTable}; pub use host::{arity, expect_text, Host, HostOp, StdHost}; pub use marqdo_sandbox::{PlatformCapabilities, SandboxMode, StrictIsolation}; +pub use resource::{ResourceRead, ResourceTable}; pub use trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState, TraceTruncation}; diff --git a/crates/marqdo-host/src/resource.rs b/crates/marqdo-host/src/resource.rs new file mode 100644 index 0000000..cf461ef --- /dev/null +++ b/crates/marqdo-host/src/resource.rs @@ -0,0 +1,390 @@ +//! Host-owned resource/stream table (ADR-0025). +//! +//! The first slice deliberately supports only stdout writes and bounded +//! project-file reads loaded at open. Handles are run-scoped and opaque; the +//! table validates run identity, slot identity and generation on every call. + +use std::collections::BTreeMap; +use std::sync::atomic::{AtomicU64, Ordering}; + +use marqdo_span::Span; +use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError}; + +static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1); + +/// Bytes returned by one stream read. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ResourceRead { + /// Bytes actually read (may be empty at EOF). + pub bytes: Vec, + /// Whether the stream has reached EOF. + pub eof: bool, +} + +enum Stream { + Stdout { + closed: bool, + }, + File { + relative: String, + bytes: Vec, + position: usize, + closed: bool, + }, + FileWriter { + relative: String, + bytes: Vec, + closed: bool, + }, +} + +struct Slot { + generation: u64, + stream: Stream, +} + +impl Slot { + fn closed(&self) -> bool { + match &self.stream { + Stream::Stdout { closed } => *closed, + Stream::File { closed, .. } => *closed, + Stream::FileWriter { closed, .. } => *closed, + } + } +} + +/// Host resource table for one run. +pub struct ResourceTable { + run_id: u64, + next_resource: u64, + slots: BTreeMap, +} + +impl Default for ResourceTable { + fn default() -> Self { + Self::new() + } +} + +impl ResourceTable { + /// Create an empty table with a fresh run identity. + pub fn new() -> Self { + Self { + run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed), + next_resource: 1, + slots: BTreeMap::new(), + } + } + + /// Number of live (not necessarily open) slots. + pub fn slot_count(&self) -> usize { + self.slots.len() + } + + /// Register the pre-opened stdout stream. + pub fn open_stdout(&mut self) -> ResourceHandle { + self.insert(Stream::Stdout { closed: false }) + } + + /// Register a project-file read stream opened from already-bounded bytes. + pub fn open_file(&mut self, relative: impl Into, bytes: Vec) -> ResourceHandle { + self.insert(Stream::File { + relative: relative.into(), + bytes, + position: 0, + closed: false, + }) + } + + /// Register an empty buffered project-file write stream. + pub fn open_file_writer(&mut self, relative: impl Into) -> ResourceHandle { + self.insert(Stream::FileWriter { + relative: relative.into(), + bytes: Vec::new(), + closed: false, + }) + } + + fn insert(&mut self, stream: Stream) -> ResourceHandle { + let id = self.next_resource; + self.next_resource += 1; + self.slots.insert( + id, + Slot { + generation: 1, + stream, + }, + ); + ResourceHandle::from_host(self.run_id, id, 1) + } + + fn slot(&self, handle: ResourceHandle, span: Option) -> Result<&Slot, RuntimeError> { + if handle.run_id() != self.run_id { + return Err(RuntimeError::new( + ErrorCategory::ResourceFault, + span, + "Resource handle belongs to a different host run", + )); + } + let Some(slot) = self.slots.get(&handle.resource_id()) else { + return Err(RuntimeError::new( + ErrorCategory::ResourceFault, + span, + "unknown Resource handle", + )); + }; + if slot.generation != handle.generation() { + return Err(RuntimeError::new( + ErrorCategory::ResourceFault, + span, + "stale Resource handle", + )); + } + Ok(slot) + } + + fn slot_mut( + &mut self, + handle: ResourceHandle, + span: Option, + ) -> Result<&mut Slot, RuntimeError> { + self.slot(handle, span)?; + self.slots + .get_mut(&handle.resource_id()) + .ok_or_else(|| unknown_handle(span)) + } + + /// Read up to `max_bytes` from a file stream. + /// + /// Reads at EOF return `count=0, eof=true`. A read on a closed stream is + /// `resource.closed`; stdout is not readable. + pub fn read( + &mut self, + handle: ResourceHandle, + max_bytes: usize, + span: Option, + ) -> Result { + let slot = self.slot_mut(handle, span)?; + if slot.closed() { + return Err(RuntimeError::new( + ErrorCategory::ResourceClosed, + span, + "Resource handle is closed", + )); + } + match &mut slot.stream { + Stream::File { + bytes, position, .. + } => { + let start = *position; + let end = start.saturating_add(max_bytes).min(bytes.len()); + let chunk = bytes[start..end].to_vec(); + *position = end; + Ok(ResourceRead { + bytes: chunk, + eof: end >= bytes.len(), + }) + } + Stream::Stdout { .. } => Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + span, + "stdout does not support resource.read", + )), + Stream::FileWriter { .. } => Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + span, + "a file write stream does not support resource.read", + )), + } + } + + /// Validate a stdout write and return the accepted byte count. + /// + /// The caller performs the output-budget accounting and buffer append; this + /// method only checks handle identity, closed state and stream kind. + pub fn write( + &mut self, + handle: ResourceHandle, + bytes: &[u8], + span: Option, + ) -> Result { + let slot = self.slot_mut(handle, span)?; + if slot.closed() { + return Err(RuntimeError::new( + ErrorCategory::ResourceClosed, + span, + "Resource handle is closed", + )); + } + match &mut slot.stream { + Stream::Stdout { .. } => Ok(bytes.len()), + Stream::File { relative, .. } => Err(RuntimeError::new( + ErrorCategory::ResourceUnsupported, + span, + format!("project file `{relative}` was opened read-only"), + )), + Stream::FileWriter { + relative, + bytes: buffer, + .. + } => { + let next = buffer.len().checked_add(bytes.len()); + let Some(next) = next else { + return Err(RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!("file write buffer for `{relative}` overflows"), + )); + }; + if next > crate::fs::MAX_READ_BYTES as usize { + return Err(RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!( + "file write buffer for `{relative}` exceeds the {} byte limit", + crate::fs::MAX_READ_BYTES + ), + )); + } + buffer.extend_from_slice(bytes); + Ok(bytes.len()) + } + } + } + + /// Close a stream. A second close is `resource.closed`. + pub fn close( + &mut self, + handle: ResourceHandle, + span: Option, + ) -> Result<(), RuntimeError> { + let slot = self.slot_mut(handle, span)?; + if slot.closed() { + return Err(RuntimeError::new( + ErrorCategory::ResourceClosed, + span, + "Resource handle is already closed", + )); + } + match &mut slot.stream { + Stream::Stdout { closed } => *closed = true, + Stream::File { closed, .. } => *closed = true, + Stream::FileWriter { closed, .. } => *closed = true, + } + Ok(()) + } + + /// Return the buffered bytes of a file write stream without closing it. + /// + /// `None` means the handle is not a file writer (stdout or a reader). The + /// caller must call [`Self::close`] only after the flush succeeds, so a + /// failed flush leaves the writer open with its buffer intact. + pub fn pending_file_write( + &self, + handle: ResourceHandle, + span: Option, + ) -> Result)>, RuntimeError> { + let slot = self.slot(handle, span)?; + if slot.closed() { + return Err(RuntimeError::new( + ErrorCategory::ResourceClosed, + span, + "Resource handle is closed", + )); + } + match &slot.stream { + Stream::FileWriter { + relative, bytes, .. + } => Ok(Some((relative.clone(), bytes.clone()))), + _ => Ok(None), + } + } + + /// Descriptor kind for diagnostics/trace, if the handle is valid. + pub fn kind( + &self, + handle: ResourceHandle, + span: Option, + ) -> Result<&'static str, RuntimeError> { + let slot = self.slot(handle, span)?; + Ok(match slot.stream { + Stream::Stdout { .. } => "stdio", + Stream::File { .. } | Stream::FileWriter { .. } => "file", + }) + } +} + +fn unknown_handle(span: Option) -> RuntimeError { + RuntimeError::new( + ErrorCategory::ResourceFault, + span, + "unknown Resource handle", + ) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn file_reads_are_chunked_with_eof() { + let mut table = ResourceTable::new(); + let handle = table.open_file("data.txt", b"abcdef".to_vec()); + assert_eq!( + table.read(handle, 2, None).unwrap(), + ResourceRead { + bytes: b"ab".to_vec(), + eof: false + } + ); + assert_eq!( + table.read(handle, 8, None).unwrap(), + ResourceRead { + bytes: b"cdef".to_vec(), + eof: true + } + ); + assert_eq!( + table.read(handle, 1, None).unwrap(), + ResourceRead { + bytes: Vec::new(), + eof: true + } + ); + } + + #[test] + fn forged_and_closed_handles_are_rejected() { + let mut table = ResourceTable::new(); + let handle = table.open_stdout(); + let forged = ResourceHandle::from_host(handle.run_id() + 1, handle.resource_id(), 1); + assert_eq!( + table.write(forged, b"x", None).unwrap_err().code(), + "runtime.resource" + ); + table.close(handle, None).unwrap(); + assert_eq!( + table.write(handle, b"x", None).unwrap_err().code(), + "resource.closed" + ); + assert_eq!( + table.close(handle, None).unwrap_err().code(), + "resource.closed" + ); + } + + #[test] + fn stdout_is_not_readable_and_file_is_not_writable() { + let mut table = ResourceTable::new(); + let stdout = table.open_stdout(); + assert_eq!( + table.read(stdout, 1, None).unwrap_err().code(), + "resource.unsupported" + ); + let file = table.open_file("x.txt", Vec::new()); + assert_eq!( + table.write(file, b"x", None).unwrap_err().code(), + "resource.unsupported" + ); + } +} diff --git a/crates/marqdo-host/tests/resource_streams.rs b/crates/marqdo-host/tests/resource_streams.rs new file mode 100644 index 0000000..3f5d78d --- /dev/null +++ b/crates/marqdo-host/tests/resource_streams.rs @@ -0,0 +1,205 @@ +//! Resource/stream host boundary tests (ADR-0025). + +use std::path::Path; + +use marqdo_host::{Capabilities, ExecutionContext, Host, HostOp, StdHost}; +use marqdo_sandbox::SandboxMode; +use marqdo_value::{ErrorCategory, Value}; + +fn context() -> ExecutionContext { + ExecutionContext::new("index.mq.md".to_string(), None) +} + +fn handle(value: Value) -> marqdo_value::ResourceHandle { + match value { + Value::Resource(handle) => handle, + other => panic!("expected a Resource handle, got {other:?}"), + } +} + +fn write_project_file(root: &Path, name: &str, contents: &[u8]) { + std::fs::write(root.join(name), contents).unwrap(); +} + +#[test] +fn pure_mode_writes_stdout_through_a_resource() { + let dir = tempfile::tempdir().unwrap(); + let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path()); + let ctx = context(); + let handle = handle( + host.resource_open("stdio", "stdout", &["write".to_string()], &ctx) + .unwrap(), + ); + assert_eq!(host.resource_write(handle, b"hello", &ctx).unwrap(), 5); + assert_eq!(host.take_output(), b"hello"); + host.resource_close(handle, &ctx).unwrap(); + let error = host.resource_close(handle, &ctx).unwrap_err(); + assert_eq!(error.category, ErrorCategory::ResourceClosed); + assert_eq!(error.code(), "resource.closed"); +} + +#[test] +fn stdout_write_obeys_the_output_budget_atomically() { + let dir = tempfile::tempdir().unwrap(); + let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path()); + host.budget.max_output_bytes = 3; + let ctx = context(); + let handle = handle( + host.resource_open("stdio", "stdout", &["write".to_string()], &ctx) + .unwrap(), + ); + let error = host.resource_write(handle, b"hello", &ctx).unwrap_err(); + assert_eq!(error.category, ErrorCategory::ResourceExhausted); + assert!(host.take_output().is_empty()); +} + +#[test] +fn strict_mode_refuses_every_resource_open() { + let dir = tempfile::tempdir().unwrap(); + let mut host = StdHost::new(SandboxMode::Strict, Capabilities::pure(), dir.path()); + let error = host + .resource_open("stdio", "stdout", &["write".to_string()], &context()) + .unwrap_err(); + assert_eq!(error.category, ErrorCategory::PermissionDenied); +} + +#[test] +fn resource_handles_cannot_cross_the_generic_host_boundary() { + let dir = tempfile::tempdir().unwrap(); + let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path()); + let ctx = context(); + let handle = handle( + host.resource_open("stdio", "stdout", &["write".to_string()], &ctx) + .unwrap(), + ); + let error = host + .invoke(HostOp::Print, vec![Value::Resource(handle)], &ctx) + .unwrap_err(); + assert_eq!(error.category, ErrorCategory::Type); + assert!(error.message.contains("Resource handles")); +} + +#[test] +fn pure_mode_refuses_file_resources_even_with_a_grant() { + let dir = tempfile::tempdir().unwrap(); + write_project_file(dir.path(), "a.txt", b"abcdef"); + let mut host = StdHost::new( + SandboxMode::Pure, + Capabilities::pure().allow_read("a.txt"), + dir.path(), + ); + let error = host + .resource_open("file", "a.txt", &["read".to_string()], &context()) + .unwrap_err(); + assert_eq!(error.category, ErrorCategory::PermissionDenied); +} + +#[test] +fn project_file_writes_buffer_until_close_and_require_a_grant() { + let dir = tempfile::tempdir().unwrap(); + let mut host = StdHost::new( + SandboxMode::Project, + Capabilities::project().allow_write("out.bin"), + dir.path(), + ); + let ctx = context(); + let handle = handle( + host.resource_open("file", "out.bin", &["write".to_string()], &ctx) + .unwrap(), + ); + assert_eq!(host.resource_write(handle, b"abc", &ctx).unwrap(), 3); + assert_eq!(host.resource_write(handle, b"def", &ctx).unwrap(), 3); + assert!( + !dir.path().join("out.bin").exists(), + "write must not flush early" + ); + host.resource_close(handle, &ctx).unwrap(); + assert_eq!( + std::fs::read(dir.path().join("out.bin")).unwrap(), + b"abcdef" + ); + + let error = host.resource_read(handle, 1, &ctx).unwrap_err(); + assert_eq!(error.code(), "resource.closed"); + let second = host.resource_close(handle, &ctx).unwrap_err(); + assert_eq!(second.code(), "resource.closed"); + + let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path()); + let error = denied + .resource_open("file", "out.bin", &["write".to_string()], &ctx) + .unwrap_err(); + assert_eq!(error.category, ErrorCategory::PermissionDenied); +} + +#[test] +fn file_write_buffer_has_a_hard_ceiling() { + let dir = tempfile::tempdir().unwrap(); + let mut host = StdHost::new( + SandboxMode::Project, + Capabilities::project().allow_write("big.bin"), + dir.path(), + ); + let ctx = context(); + let handle = handle( + host.resource_open("file", "big.bin", &["write".to_string()], &ctx) + .unwrap(), + ); + let oversized = vec![0u8; marqdo_host::fs::MAX_READ_BYTES as usize + 1]; + let error = host.resource_write(handle, &oversized, &ctx).unwrap_err(); + assert_eq!(error.category, ErrorCategory::ResourceExhausted); + assert_eq!(host.resource_write(handle, b"ok", &ctx).unwrap(), 2); + host.resource_close(handle, &ctx).unwrap(); + assert_eq!(std::fs::read(dir.path().join("big.bin")).unwrap(), b"ok"); +} + +#[test] +fn project_file_reads_are_chunked_and_require_a_grant() { + let dir = tempfile::tempdir().unwrap(); + write_project_file(dir.path(), "a.txt", b"abcdef"); + let mut host = StdHost::new( + SandboxMode::Project, + Capabilities::project().allow_read("a.txt"), + dir.path(), + ); + let ctx = context(); + let handle = handle( + host.resource_open("file", "a.txt", &["read".to_string()], &ctx) + .unwrap(), + ); + let first = host.resource_read(handle, 2, &ctx).unwrap(); + assert_eq!(first.bytes, b"ab"); + assert!(!first.eof); + let second = host.resource_read(handle, 8, &ctx).unwrap(); + assert_eq!(second.bytes, b"cdef"); + assert!(second.eof); + assert!(host.resource_read(handle, 1, &ctx).unwrap().eof); + + let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path()); + let error = denied + .resource_open("file", "a.txt", &["read".to_string()], &ctx) + .unwrap_err(); + assert_eq!(error.category, ErrorCategory::PermissionDenied); +} + +#[test] +fn traversal_and_unsupported_rights_are_refused() { + let dir = tempfile::tempdir().unwrap(); + let mut host = StdHost::new( + SandboxMode::Project, + Capabilities::project().allow_read("**"), + dir.path(), + ); + let ctx = context(); + let traversal = host + .resource_open("file", "../secret", &["read".to_string()], &ctx) + .unwrap_err(); + assert_eq!(traversal.category, ErrorCategory::PermissionDenied); + let write_rights = host + .resource_open("stdio", "stdout", &["read".to_string()], &ctx) + .unwrap_err(); + assert_eq!(write_rights.category, ErrorCategory::ResourceUnsupported); + let unknown_kind = host + .resource_open("socket", "example.test", &["read".to_string()], &ctx) + .unwrap_err(); + assert_eq!(unknown_kind.category, ErrorCategory::ResourceUnsupported); +} diff --git a/crates/marqdo-ir/src/builtins.rs b/crates/marqdo-ir/src/builtins.rs index f68035d..a864d66 100644 --- a/crates/marqdo-ir/src/builtins.rs +++ b/crates/marqdo-ir/src/builtins.rs @@ -167,6 +167,144 @@ const FS_WRITE: Builtin = Builtin { named_only: false, effectful: true, }; +const MEMORY_NEW: Builtin = Builtin { + name: "memory.new", + aliases: &[], + params: &[req("initial_pages"), req("max_pages")], + named_only: false, + effectful: false, +}; +const MEMORY_SIZE: Builtin = Builtin { + name: "memory.size", + aliases: &[], + params: &[req("memory")], + named_only: false, + effectful: false, +}; +const MEMORY_GROW: Builtin = Builtin { + name: "memory.grow", + aliases: &[], + params: &[req("memory"), req("pages")], + named_only: false, + effectful: false, +}; +const LIST_APPEND: Builtin = Builtin { + name: "list.append", + aliases: &[], + params: &[req("list"), req("value")], + named_only: false, + effectful: false, +}; +const LIST_CONCAT: Builtin = Builtin { + name: "list.concat", + aliases: &[], + params: &[req("left"), req("right")], + named_only: false, + effectful: false, +}; +const MAP_KEYS: Builtin = Builtin { + name: "map.keys", + aliases: &[], + params: &[req("map")], + named_only: false, + effectful: false, +}; +const MAP_HAS: Builtin = Builtin { + name: "map.has", + aliases: &[], + params: &[req("map"), req("key")], + named_only: false, + effectful: false, +}; +const MAP_GET: Builtin = Builtin { + name: "map.get", + aliases: &[], + params: &[req("map"), req("key")], + named_only: false, + effectful: false, +}; +const MAP_SET: Builtin = Builtin { + name: "map.set", + aliases: &[], + params: &[req("map"), req("key"), req("value")], + named_only: false, + effectful: false, +}; +const MAP_REMOVE: Builtin = Builtin { + name: "map.remove", + aliases: &[], + params: &[req("map"), req("key")], + named_only: false, + effectful: false, +}; +const TEXT_CODEPOINTS: Builtin = Builtin { + name: "text.codepoints", + aliases: &[], + params: &[req("text")], + named_only: false, + effectful: false, +}; +const TEXT_FROM_CODEPOINT: Builtin = Builtin { + name: "text.from_codepoint", + aliases: &[], + params: &[req("codepoint")], + named_only: false, + effectful: false, +}; +const TABLE_COLUMNS: Builtin = Builtin { + name: "table.columns", + aliases: &[], + params: &[req("table")], + named_only: false, + effectful: false, +}; +const TABLE_ROW: Builtin = Builtin { + name: "table.row", + aliases: &[], + params: &[req("table"), req("row")], + named_only: false, + effectful: false, +}; +const RESOURCE_OPEN: Builtin = Builtin { + name: "resource.open", + aliases: &[], + params: &[req("kind"), req("descriptor"), req("rights")], + named_only: false, + effectful: true, +}; +const RESOURCE_READ: Builtin = Builtin { + name: "resource.read", + aliases: &[], + params: &[ + req("handle"), + req("memory"), + req("offset"), + req("max_bytes"), + ], + named_only: false, + effectful: true, +}; +const RESOURCE_WRITE: Builtin = Builtin { + name: "resource.write", + aliases: &[], + params: &[req("handle"), req("memory"), req("offset"), req("length")], + named_only: false, + effectful: true, +}; +const RESOURCE_CLOSE: Builtin = Builtin { + name: "resource.close", + aliases: &[], + params: &[req("handle")], + named_only: false, + effectful: true, +}; +const TABLE_CELL: Builtin = Builtin { + name: "table.cell", + aliases: &[], + params: &[req("table"), req("row"), req("column")], + named_only: false, + effectful: false, +}; /// All builtins known to the language. Host availability is decided at call /// time by the capability layer, not here. @@ -188,6 +326,25 @@ pub const BUILTINS: &[Builtin] = &[ FOREIGN_CALL, FS_READ, FS_WRITE, + MEMORY_NEW, + MEMORY_SIZE, + MEMORY_GROW, + LIST_APPEND, + LIST_CONCAT, + MAP_KEYS, + MAP_HAS, + MAP_GET, + MAP_SET, + MAP_REMOVE, + TEXT_CODEPOINTS, + TEXT_FROM_CODEPOINT, + TABLE_COLUMNS, + TABLE_ROW, + TABLE_CELL, + RESOURCE_OPEN, + RESOURCE_READ, + RESOURCE_WRITE, + RESOURCE_CLOSE, ]; /// Find a builtin by canonical name or alias. diff --git a/crates/marqdo-ir/src/lib.rs b/crates/marqdo-ir/src/lib.rs index aa8ab5f..3ca80b8 100644 --- a/crates/marqdo-ir/src/lib.rs +++ b/crates/marqdo-ir/src/lib.rs @@ -224,16 +224,51 @@ pub struct Function { pub body: Vec, } +/// Writable place of an assignment (ADR-0022). +/// +/// The Markdown surface is unchanged: both variants are `strong` nodes whose +/// payload contains `target = expression`. `Local` rebinds a frame slot; +/// `Index` writes one byte into an addressable runtime `Memory` region and is +/// rejected for every other base value. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(tag = "place", rename_all = "snake_case")] +pub enum PlaceTarget { + /// Local slot rebinding. + Local { + /// Target name. + name: String, + /// Span of the target name. + name_span: Span, + }, + /// `base[index] = value`; only a `Memory` base is writable. + Index { + /// Base expression, evaluated first. + base: Expr, + /// Index expression, evaluated second. + index: Expr, + /// Span from the base start through the closing `]`. + span: Span, + }, +} + +impl PlaceTarget { + /// Best known source span of the place. + pub fn span(&self) -> Span { + match self { + PlaceTarget::Local { name_span, .. } => *name_span, + PlaceTarget::Index { span, .. } => *span, + } + } +} + /// An executable statement. #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[serde(tag = "stmt", rename_all = "snake_case")] pub enum Stmt { - /// Local assignment (`strong` node with `name = expr`). + /// Assignment to a local slot or a `Memory` index. Assign { - /// Target name. - name: String, - /// Span of the target name (approximated by the payload span). - name_span: Span, + /// Writable place. + target: PlaceTarget, /// Value expression. value: Expr, /// Span of the whole statement. diff --git a/crates/marqdo-runtime/src/builtins_impl.rs b/crates/marqdo-runtime/src/builtins_impl.rs index bc74dad..480c1b1 100644 --- a/crates/marqdo-runtime/src/builtins_impl.rs +++ b/crates/marqdo-runtime/src/builtins_impl.rs @@ -6,13 +6,13 @@ use std::collections::BTreeMap; -use marqdo_host::HostOp; +use marqdo_host::{HostOp, ResourceRead}; use marqdo_ir::builtins::Builtin; use marqdo_ir::Resource; use marqdo_span::Span; use marqdo_value::{ - float_to_bigint, CodeValue, ErrorCategory, FormulaValue, FunctionRef, RuntimeError, Table, - Value, + float_to_bigint, CodeValue, ErrorCategory, FormulaValue, FunctionRef, ResourceHandle, + RuntimeError, Table, Value, }; use crate::interpreter::Interpreter; @@ -187,6 +187,292 @@ pub(crate) fn dispatch( &module, ) } + "memory.new" => { + let initial_pages = page_count(take(&bound, "initial_pages", builtin, span)?, span)?; + let max_pages = page_count(take(&bound, "max_pages", builtin, span)?, span)?; + let handle = interpreter + .memory_mut() + .create(initial_pages, max_pages, span)?; + interpreter.record_memory_event( + "memory.new", + &module, + span, + &format!("initial_pages={initial_pages} max_pages={max_pages}"), + ); + Ok(Value::Memory(handle)) + } + "memory.size" => { + let memory = take(&bound, "memory", builtin, span)?; + let memory_type = memory.type_name(); + let Value::Memory(handle) = memory else { + return Err(RuntimeError::type_error( + span, + format!("memory.size expects a Memory value, got {memory_type}"), + )); + }; + let pages = interpreter.memory().size(handle, span)?; + Ok(pages_value(pages, span)?) + } + "memory.grow" => { + let memory = take(&bound, "memory", builtin, span)?; + let memory_type = memory.type_name(); + let Value::Memory(handle) = memory else { + return Err(RuntimeError::type_error( + span, + format!("memory.grow expects a Memory value, got {memory_type}"), + )); + }; + let pages = page_count(take(&bound, "pages", builtin, span)?, span)?; + let old_pages = interpreter.memory_mut().grow(handle, pages, span)?; + interpreter.record_memory_event( + "memory.grow", + &module, + span, + &format!("pages={pages} old_pages={old_pages}"), + ); + Ok(pages_value(old_pages, span)?) + } + "list.append" => { + let list = take(&bound, "list", builtin, span)?; + let value = take(&bound, "value", builtin, span)?; + let list_type = list.type_name(); + let Value::List(mut items) = list else { + return Err(RuntimeError::type_error( + span, + format!("list.append expects a List, got {list_type}"), + )); + }; + items.push(value); + Ok(Value::List(items)) + } + "list.concat" => { + let left = take(&bound, "left", builtin, span)?; + let right = take(&bound, "right", builtin, span)?; + let left_type = left.type_name(); + let Value::List(mut items) = left else { + return Err(RuntimeError::type_error( + span, + format!("list.concat left operand must be a List, got {left_type}"), + )); + }; + let right_type = right.type_name(); + let Value::List(other) = right else { + return Err(RuntimeError::type_error( + span, + format!("list.concat right operand must be a List, got {right_type}"), + )); + }; + items.extend(other); + Ok(Value::List(items)) + } + "map.keys" => { + let value = take(&bound, "map", builtin, span)?; + let value_type = value.type_name(); + let Value::Map(map) = value else { + return Err(RuntimeError::type_error( + span, + format!("map.keys expects a Map, got {value_type}"), + )); + }; + Ok(Value::List( + map.keys().cloned().map(Value::Text).collect::>(), + )) + } + "map.has" => { + let value = take(&bound, "map", builtin, span)?; + let key = map_key(take(&bound, "key", builtin, span)?, "map.has", span)?; + let value_type = value.type_name(); + let Value::Map(map) = value else { + return Err(RuntimeError::type_error( + span, + format!("map.has expects a Map, got {value_type}"), + )); + }; + Ok(Value::Bool(map.contains_key(&key))) + } + "map.get" => { + let value = take(&bound, "map", builtin, span)?; + let key = map_key(take(&bound, "key", builtin, span)?, "map.get", span)?; + let value_type = value.type_name(); + let Value::Map(map) = value else { + return Err(RuntimeError::type_error( + span, + format!("map.get expects a Map, got {value_type}"), + )); + }; + Ok(map.get(&key).cloned().unwrap_or(Value::None)) + } + "map.set" => { + let value = take(&bound, "map", builtin, span)?; + let key = map_key(take(&bound, "key", builtin, span)?, "map.set", span)?; + let new_value = take(&bound, "value", builtin, span)?; + let value_type = value.type_name(); + let Value::Map(mut map) = value else { + return Err(RuntimeError::type_error( + span, + format!("map.set expects a Map, got {value_type}"), + )); + }; + map.insert(key, new_value); + Ok(Value::Map(map)) + } + "map.remove" => { + let value = take(&bound, "map", builtin, span)?; + let key = map_key(take(&bound, "key", builtin, span)?, "map.remove", span)?; + let value_type = value.type_name(); + let Value::Map(mut map) = value else { + return Err(RuntimeError::type_error( + span, + format!("map.remove expects a Map, got {value_type}"), + )); + }; + map.remove(&key); + Ok(Value::Map(map)) + } + "text.codepoints" => { + let value = take(&bound, "text", builtin, span)?; + let value_type = value.type_name(); + let Value::Text(text) = value else { + return Err(RuntimeError::type_error( + span, + format!("text.codepoints expects Text, got {value_type}"), + )); + }; + Ok(Value::List( + text.chars() + .map(|character| Value::int(i64::from(character as u32))) + .collect(), + )) + } + "text.from_codepoint" => { + let value = take(&bound, "codepoint", builtin, span)?; + let value_type = value.type_name(); + let Value::Int(codepoint) = value else { + return Err(RuntimeError::type_error( + span, + format!("text.from_codepoint expects an Int, got {value_type}"), + )); + }; + let Some(codepoint) = num_traits::ToPrimitive::to_u32(&codepoint) else { + return Ok(Value::None); + }; + Ok(match char::from_u32(codepoint) { + Some(character) => Value::Text(character.to_string()), + None => Value::None, + }) + } + "table.columns" => { + let value = take(&bound, "table", builtin, span)?; + let value_type = value.type_name(); + let Value::Table(table) = value else { + return Err(RuntimeError::type_error( + span, + format!("table.columns expects a Table, got {value_type}"), + )); + }; + Ok(Value::List( + table.header.into_iter().map(Value::Text).collect(), + )) + } + "table.row" => { + let value = take(&bound, "table", builtin, span)?; + let row = take(&bound, "row", builtin, span)?; + let value_type = value.type_name(); + let Value::Table(table) = value else { + return Err(RuntimeError::type_error( + span, + format!("table.row expects a Table, got {value_type}"), + )); + }; + let position = table_index(row, table.rows.len(), "row", span)?; + Ok(Value::List(table.rows[position].clone())) + } + "table.cell" => { + let value = take(&bound, "table", builtin, span)?; + let row = take(&bound, "row", builtin, span)?; + let column = take(&bound, "column", builtin, span)?; + let value_type = value.type_name(); + let Value::Table(table) = value else { + return Err(RuntimeError::type_error( + span, + format!("table.cell expects a Table, got {value_type}"), + )); + }; + let row = table_index(row, table.rows.len(), "row", span)?; + let column = table_index(column, table.header.len(), "column", span)?; + Ok(table.rows[row][column].clone()) + } + "resource.open" => { + let kind = expect_text_value(take(&bound, "kind", builtin, span)?, span)?; + let descriptor = expect_text_value(take(&bound, "descriptor", builtin, span)?, span)?; + let rights = rights_list(take(&bound, "rights", builtin, span)?, span)?; + interpreter.resource_open(&kind, &descriptor, &rights, &module, span) + } + "resource.read" => { + let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?; + let memory = memory_handle( + take(&bound, "memory", builtin, span)?, + "resource.read", + span, + )?; + let offset = take(&bound, "offset", builtin, span)?; + let offset_int = int_value(offset, "resource.read offset", span)?; + let offset_value = Value::Int(offset_int.clone()); + let max_bytes = positive_count( + take(&bound, "max_bytes", builtin, span)?, + "resource.read max_bytes", + span, + )?; + let available = interpreter.memory().byte_len(memory, span)?; + let start = non_negative_usize(&offset_value, "resource.read offset", span)?; + if start >= available { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + format!("resource.read offset {start} is at or beyond the Memory length {available}"), + )); + } + let limit = max_bytes.min(available - start); + let result = interpreter.resource_read(handle, limit, &module, span); + match result { + Ok(read) => { + interpreter + .memory_mut() + .write_bytes(memory, &offset_int, &read.bytes, span)?; + Ok(resource_ok(resource_read_payload(&read))) + } + Err(error) => resource_recoverable(error), + } + } + "resource.write" => { + let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?; + let memory = memory_handle( + take(&bound, "memory", builtin, span)?, + "resource.write", + span, + )?; + let offset = take(&bound, "offset", builtin, span)?; + let offset_int = int_value(offset, "resource.write offset", span)?; + let length = non_negative_usize( + &take(&bound, "length", builtin, span)?, + "resource.write length", + span, + )?; + let bytes = interpreter + .memory() + .read_bytes(memory, &offset_int, length, span)?; + match interpreter.resource_write(handle, &bytes, &module, span) { + Ok(count) => Ok(resource_ok(resource_write_payload(count))), + Err(error) => resource_recoverable(error), + } + } + "resource.close" => { + let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?; + match interpreter.resource_close(handle, &module, span) { + Ok(()) => Ok(resource_ok(Value::None)), + Err(error) => resource_recoverable(error), + } + } other => Err(RuntimeError::new( ErrorCategory::KernelUnavailable, span, @@ -210,6 +496,228 @@ fn take( }) } +/// Validate a page-count argument: an `Int` in the non-negative `u64` range. +fn page_count(value: Value, span: Option) -> Result { + let value_type = value.type_name(); + let Value::Int(count) = value else { + return Err(RuntimeError::type_error( + span, + format!("memory page count must be an Int, got {value_type}"), + )); + }; + num_traits::ToPrimitive::to_u64(&count).ok_or_else(|| { + RuntimeError::new( + ErrorCategory::MemoryFault, + span, + format!("memory page count {count} must be a non-negative integer"), + ) + }) +} + +/// Convert a page count back to a language `Int`. +fn pages_value(pages: u64, span: Option) -> Result { + i64::try_from(pages).map(Value::int).map_err(|_| { + RuntimeError::new( + ErrorCategory::MemoryFault, + span, + format!("page count {pages} does not fit an Int"), + ) + }) +} + +/// Require a `Text` map key and return it. +fn map_key(value: Value, builtin: &str, span: Option) -> Result { + let value_type = value.type_name(); + match value { + Value::Text(key) => Ok(key), + _ => Err(RuntimeError::type_error( + span, + format!("{builtin} key must be Text, got {value_type}"), + )), + } +} + +/// Require a Text value. +fn expect_text_value(value: Value, span: Option) -> Result { + let value_type = value.type_name(); + let Value::Text(text) = value else { + return Err(RuntimeError::type_error( + span, + format!("expected Text, got {value_type}"), + )); + }; + Ok(text) +} + +/// Require a List of Text rights. +fn rights_list(value: Value, span: Option) -> Result, RuntimeError> { + let value_type = value.type_name(); + let Value::List(items) = value else { + return Err(RuntimeError::type_error( + span, + format!("resource rights must be a List, got {value_type}"), + )); + }; + let mut rights = Vec::with_capacity(items.len()); + for item in items { + let item_type = item.type_name(); + let Value::Text(right) = item else { + return Err(RuntimeError::type_error( + span, + format!("resource rights must contain Text, got {item_type}"), + )); + }; + rights.push(right); + } + Ok(rights) +} + +/// Require an opaque resource handle. +fn resource_handle(value: Value, span: Option) -> Result { + let value_type = value.type_name(); + let Value::Resource(handle) = value else { + return Err(RuntimeError::type_error( + span, + format!("expected a Resource handle, got {value_type}"), + )); + }; + Ok(handle) +} + +/// Require an opaque Memory handle. +fn memory_handle( + value: Value, + builtin: &str, + span: Option, +) -> Result { + let value_type = value.type_name(); + let Value::Memory(handle) = value else { + return Err(RuntimeError::type_error( + span, + format!("{builtin} memory must be a Memory value, got {value_type}"), + )); + }; + Ok(handle) +} + +/// Require an Int value. +fn int_value( + value: Value, + what: &str, + span: Option, +) -> Result { + let value_type = value.type_name(); + let Value::Int(value) = value else { + return Err(RuntimeError::type_error( + span, + format!("{what} must be an Int, got {value_type}"), + )); + }; + Ok(value) +} + +/// Require a non-negative usize. +fn non_negative_usize( + value: &Value, + what: &str, + span: Option, +) -> Result { + let value_type = value.type_name(); + let Value::Int(value) = value else { + return Err(RuntimeError::type_error( + span, + format!("{what} must be an Int, got {value_type}"), + )); + }; + num_traits::ToPrimitive::to_usize(value).ok_or_else(|| { + RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + format!("{what} {value} must be a non-negative integer in range"), + ) + }) +} + +/// Require a strictly positive byte count. +fn positive_count(value: Value, what: &str, span: Option) -> Result { + let count = non_negative_usize(&value, what, span)?; + if count == 0 { + return Err(RuntimeError::type_error( + span, + format!("{what} must be greater than zero"), + )); + } + Ok(count) +} + +fn resource_ok(value: Value) -> Value { + let mut map = BTreeMap::new(); + map.insert("ok".to_string(), Value::Bool(true)); + map.insert("value".to_string(), value); + Value::Map(map) +} + +fn resource_err(code: &str, message: &str) -> Value { + let mut map = BTreeMap::new(); + map.insert("ok".to_string(), Value::Bool(false)); + map.insert("code".to_string(), Value::Text(code.to_string())); + map.insert("message".to_string(), Value::Text(message.to_string())); + Value::Map(map) +} + +fn resource_read_payload(read: &ResourceRead) -> Value { + let mut map = BTreeMap::new(); + map.insert("count".to_string(), Value::int(read.bytes.len() as i64)); + map.insert("eof".to_string(), Value::Bool(read.eof)); + Value::Map(map) +} + +fn resource_write_payload(count: usize) -> Value { + let mut map = BTreeMap::new(); + map.insert("count".to_string(), Value::int(count as i64)); + Value::Map(map) +} + +/// Convert a closed-handle failure into a recoverable Result; other errors are +/// fatal (ADR-0025). +fn resource_recoverable(error: RuntimeError) -> Result { + if error.category == ErrorCategory::ResourceClosed { + return Ok(resource_err("resource.closed", &error.message)); + } + Err(error) +} + +/// Validate a non-negative table row/column coordinate. +fn table_index( + value: Value, + len: usize, + what: &str, + span: Option, +) -> Result { + let value_type = value.type_name(); + let Value::Int(index) = value else { + return Err(RuntimeError::type_error( + span, + format!("table {what} must be an Int, got {value_type}"), + )); + }; + let Some(position) = num_traits::ToPrimitive::to_usize(&index) else { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + format!("table {what} {index} is out of bounds for length {len}"), + )); + }; + if position >= len { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + format!("table {what} {position} is out of bounds for length {len}"), + )); + } + Ok(position) +} + fn formula_text(value: Value, span: Option) -> Result { match value { Value::Formula(formula) => Ok(formula.latex), diff --git a/crates/marqdo-runtime/src/interpreter.rs b/crates/marqdo-runtime/src/interpreter.rs index ba4fc11..cf5f245 100644 --- a/crates/marqdo-runtime/src/interpreter.rs +++ b/crates/marqdo-runtime/src/interpreter.rs @@ -3,12 +3,13 @@ use std::collections::BTreeMap; use marqdo_expression::{BinaryOp, Expr, RecordEntry}; -use marqdo_host::{ExecutionContext, Host, HostOp}; -use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, Program, Stmt}; +use marqdo_host::{ExecutionContext, Host, HostOp, ResourceRead}; +use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, PlaceTarget, Program, Stmt}; use marqdo_span::Span; -use marqdo_value::{ErrorCategory, RuntimeError, Value}; +use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value}; use crate::builtins_impl::FrameView; +use crate::memory::{MemoryLimits, MemoryStore}; use crate::value_ops; @@ -68,6 +69,7 @@ pub struct Interpreter<'a> { program: &'a Program, host: &'a mut dyn Host, depth: usize, + memory: MemoryStore, } /// Run a named function with already-evaluated arguments. @@ -77,11 +79,31 @@ pub fn run( module: &ModuleId, function: &str, args: Vec, +) -> Result { + run_with_limits( + program, + host, + module, + function, + args, + MemoryLimits::default(), + ) +} + +/// Run with explicit addressable-memory limits (ADR-0022). +pub fn run_with_limits( + program: &Program, + host: &mut dyn Host, + module: &ModuleId, + function: &str, + args: Vec, + limits: MemoryLimits, ) -> Result { let mut interpreter = Interpreter { program, host, depth: 0, + memory: MemoryStore::new(limits), }; let value = interpreter.call_by_name(module, function, args, None)?; Ok(RunOutcome { value }) @@ -300,10 +322,25 @@ impl<'a> Interpreter<'a> { self.host.consume_fuel(1, Some(stmt.span()))?; self.host.check_deadline(Some(stmt.span()))?; match stmt { - Stmt::Assign { name, value, .. } => { - let value = self.eval_expr(value, frame)?; - frame.locals.insert(name.clone(), value); - } + Stmt::Assign { target, value, .. } => match target { + PlaceTarget::Local { name, .. } => { + let value = self.eval_expr(value, frame)?; + frame.locals.insert(name.clone(), value); + } + PlaceTarget::Index { base, index, span } => { + // Frozen order (ADR-0022): base, index, then RHS. + let base_value = self.eval_expr(base, frame)?; + let index_value = self.eval_expr(index, frame)?; + let byte_value = self.eval_expr(value, frame)?; + self.assign_memory_index( + base_value, + index_value, + byte_value, + Some(*span), + frame, + )?; + } + }, Stmt::Eval { expr, .. } => { self.eval_expr(expr, frame)?; } @@ -417,6 +454,62 @@ impl<'a> Interpreter<'a> { self.call_by_name_from(&module, &name, args, Some(span), Some(&caller)) } + /// Commit one byte into an addressable region after all operand checks. + /// + /// The three operands are already evaluated, so a rejected value never + /// produces a partial write (ADR-0022 §3). + fn assign_memory_index( + &mut self, + base: Value, + index: Value, + byte: Value, + span: Option, + frame: &Frame, + ) -> Result<(), RuntimeError> { + let base_type = base.type_name(); + let Value::Memory(handle) = base else { + return Err(RuntimeError::new( + ErrorCategory::Type, + span, + format!( + "index assignment requires a Memory base in v1, got {base_type}; List, Map \ + and Table values keep their existing value semantics" + ), + )); + }; + let index_type = index.type_name(); + let Value::Int(offset) = index else { + return Err(RuntimeError::type_error( + span, + format!("Memory index must be an Int, got {index_type}"), + )); + }; + let byte_type = byte.type_name(); + let byte_repr = byte.display(); + let Value::Int(byte) = byte else { + return Err(RuntimeError::type_error( + span, + format!("Memory byte value must be an Int in 0..255, got {byte_type}"), + )); + }; + let Some(byte) = num_traits::ToPrimitive::to_u8(&byte) else { + return Err(RuntimeError::type_error( + span, + format!("Memory byte value {byte_repr} is outside 0..255"), + )); + }; + self.memory.write(handle, &offset, byte, span)?; + let module = frame.function.module.canonical(); + self.host.record_semantic( + "memory.write", + Some(&frame.function.to_string()), + Some(&module), + span, + Some("byte"), + ); + Ok(()) + } + fn eval_args( &mut self, args: &[marqdo_expression::Arg], @@ -528,7 +621,22 @@ impl<'a> Interpreter<'a> { Expr::Index { base, index, span } => { let base_value = self.eval_expr(base, frame)?; let index_value = self.eval_expr(index, frame)?; - index_value_of(base_value, index_value, Some(*span)) + match base_value { + Value::Memory(handle) => { + let Value::Int(offset) = index_value else { + return Err(RuntimeError::type_error( + Some(*span), + format!( + "Memory index must be an Int, got {}", + index_value.type_name() + ), + )); + }; + let byte = self.memory.read(handle, &offset, Some(*span))?; + Ok(Value::int(i64::from(byte))) + } + other => index_value_of(other, index_value, Some(*span)), + } } } } @@ -701,6 +809,97 @@ impl<'a> Interpreter<'a> { self.program } + /// Addressable-memory store for builtins and place writes. + pub(crate) fn memory(&self) -> &MemoryStore { + &self.memory + } + + /// Mutable addressable-memory store for `memory.new`/`memory.grow`. + pub(crate) fn memory_mut(&mut self) -> &mut MemoryStore { + &mut self.memory + } + + /// Record a semantic event for a builtin that is not a host operation. + pub(crate) fn record_memory_event( + &mut self, + kind: &str, + module: &ModuleId, + span: Option, + detail: &str, + ) { + let module = module.canonical(); + self.host + .record_semantic(kind, None, Some(&module), span, Some(detail)); + } + + /// Open a resource/stream through the host boundary. + pub(crate) fn resource_open( + &mut self, + kind: &str, + descriptor: &str, + rights: &[String], + module: &ModuleId, + span: Option, + ) -> Result { + let context = ExecutionContext::new(module.to_string(), span); + self.host + .resource_open(kind, descriptor, rights, &context) + .map_err(|mut error| { + error.mark_callsite(); + error + }) + } + + /// Read from a resource stream. + pub(crate) fn resource_read( + &mut self, + handle: ResourceHandle, + max_bytes: usize, + module: &ModuleId, + span: Option, + ) -> Result { + let context = ExecutionContext::new(module.to_string(), span); + self.host + .resource_read(handle, max_bytes, &context) + .map_err(|mut error| { + error.mark_callsite(); + error + }) + } + + /// Write bytes to a resource stream. + pub(crate) fn resource_write( + &mut self, + handle: ResourceHandle, + bytes: &[u8], + module: &ModuleId, + span: Option, + ) -> Result { + let context = ExecutionContext::new(module.to_string(), span); + self.host + .resource_write(handle, bytes, &context) + .map_err(|mut error| { + error.mark_callsite(); + error + }) + } + + /// Close a resource stream. + pub(crate) fn resource_close( + &mut self, + handle: ResourceHandle, + module: &ModuleId, + span: Option, + ) -> Result<(), RuntimeError> { + let context = ExecutionContext::new(module.to_string(), span); + self.host + .resource_close(handle, &context) + .map_err(|mut error| { + error.mark_callsite(); + error + }) + } + /// Call a builtin-read host operation on behalf of a frame's module. pub(crate) fn host_invoke( &mut self, diff --git a/crates/marqdo-runtime/src/lib.rs b/crates/marqdo-runtime/src/lib.rs index b551e3a..5001d93 100644 --- a/crates/marqdo-runtime/src/lib.rs +++ b/crates/marqdo-runtime/src/lib.rs @@ -8,8 +8,12 @@ mod builtins_impl; mod interpreter; +mod memory; mod value_ops; -pub use interpreter::{run, EvaluatedArg, Interpreter, RunOutcome}; +pub use interpreter::{run, run_with_limits, EvaluatedArg, Interpreter, RunOutcome}; +pub use memory::{ + MemoryLimits, MemoryStore, DEFAULT_MAX_REGION_BYTES, DEFAULT_MAX_TOTAL_BYTES, PAGE_SIZE, +}; pub use marqdo_value::{RuntimeError, Value}; diff --git a/crates/marqdo-runtime/src/memory.rs b/crates/marqdo-runtime/src/memory.rs new file mode 100644 index 0000000..6463397 --- /dev/null +++ b/crates/marqdo-runtime/src/memory.rs @@ -0,0 +1,600 @@ +//! Runtime-owned addressable byte regions (RFC-CM-0001 / ADR-0022). +//! +//! A [`MemoryStore`] owns every region created during one run. Language values +//! carry only an opaque [`MemoryHandle`]; offsets are region-relative integers +//! and are never host pointers. Region identity is issued here and cannot be +//! forged from a document, a number or wire data. +//! +//! Semantics frozen by ADR-0022: +//! +//! * a region is a byte array with a 64 KiB logical page size; new bytes are +//! zero-filled; +//! * regions grow only, never shrink; a failed grow leaves the region and the +//! committed budget unchanged; +//! * copies of a `Memory` value share one region, while different regions +//! never alias; +//! * every access is bounds-checked against the region and the run budget, and +//! overflow in offset/page arithmetic is a structured error, never a panic; +//! * regions live for the whole run and are dropped with the store. There is +//! no user-visible `free(region)`. + +use std::collections::BTreeMap; +use std::sync::atomic::{AtomicU64, Ordering}; + +use marqdo_span::Span; +use marqdo_value::{ErrorCategory, MemoryHandle, RuntimeError}; +use num_bigint::BigInt; +use num_traits::ToPrimitive; + +/// Logical page size for every region. +pub const PAGE_SIZE: usize = 64 * 1024; + +/// Default maximum committed bytes across all regions in one run (64 MiB). +pub const DEFAULT_MAX_TOTAL_BYTES: usize = 64 * 1024 * 1024; + +/// Default maximum committed bytes for one region (16 MiB). +pub const DEFAULT_MAX_REGION_BYTES: usize = 16 * 1024 * 1024; + +static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1); + +/// Bounds for addressable memory in one run. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct MemoryLimits { + /// Maximum bytes committed across all live regions. + pub max_total_bytes: usize, + /// Maximum bytes committed by a single region (`max_pages` is further + /// capped by this value). + pub max_region_bytes: usize, +} + +impl Default for MemoryLimits { + fn default() -> Self { + Self { + max_total_bytes: DEFAULT_MAX_TOTAL_BYTES, + max_region_bytes: DEFAULT_MAX_REGION_BYTES, + } + } +} + +impl MemoryLimits { + /// Maximum page count a single region may declare or reach. + pub fn max_region_pages(&self) -> u64 { + (self.max_region_bytes / PAGE_SIZE) as u64 + } +} + +struct Region { + bytes: Vec, + /// Declared maximum size for this region. + max_bytes: usize, + generation: u64, +} + +/// All addressable regions created by one run. +pub struct MemoryStore { + run_id: u64, + next_region: u64, + regions: BTreeMap, + committed_bytes: usize, + limits: MemoryLimits, +} + +impl MemoryStore { + /// Create an empty store with a fresh, unguessable run identity. + pub fn new(limits: MemoryLimits) -> Self { + Self { + run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed), + next_region: 1, + regions: BTreeMap::new(), + committed_bytes: 0, + limits, + } + } + + /// Limits in force for this store. + pub fn limits(&self) -> MemoryLimits { + self.limits + } + + /// Bytes currently committed across all regions. + pub fn committed_bytes(&self) -> usize { + self.committed_bytes + } + + /// Number of live regions. + pub fn region_count(&self) -> usize { + self.regions.len() + } + + /// Create a zero-filled region. + /// + /// `initial_pages` is committed immediately; `max_pages` is the declared + /// capacity that `grow` may not exceed. Both are checked against the + /// region and total budgets before anything is allocated, so a rejected + /// create leaves the store unchanged. + pub fn create( + &mut self, + initial_pages: u64, + max_pages: u64, + span: Option, + ) -> Result { + if initial_pages > max_pages { + return Err(memory_error( + span, + format!( + "memory.new initial_pages ({initial_pages}) must not exceed max_pages \ + ({max_pages})" + ), + )); + } + if max_pages > self.limits.max_region_pages() { + return Err(RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!( + "memory.new max_pages ({max_pages}) exceeds the per-region limit of {} \ + pages ({PAGE_SIZE} bytes each)", + self.limits.max_region_pages() + ), + )); + } + let max_bytes = checked_bytes(max_pages, span, "region capacity")?; + if max_bytes > self.limits.max_region_bytes { + return Err(RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!( + "memory.new max_pages ({max_pages}) exceeds the per-region byte budget of {}", + self.limits.max_region_bytes + ), + )); + } + let initial_bytes = checked_bytes(initial_pages, span, "initial region size")?; + let committed = self + .committed_bytes + .checked_add(initial_bytes) + .ok_or_else(|| budget_error(span, "memory budget overflow"))?; + if committed > self.limits.max_total_bytes { + return Err(RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!( + "memory budget exceeded: committing {initial_bytes} bytes would reach \ + {committed} of {}", + self.limits.max_total_bytes + ), + )); + } + + let mut bytes = Vec::new(); + bytes.try_reserve_exact(initial_bytes).map_err(|_| { + RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!("cannot allocate {initial_bytes} bytes of addressable memory"), + ) + })?; + bytes.resize(initial_bytes, 0); + + let region_id = self.next_region; + self.next_region = self + .next_region + .checked_add(1) + .ok_or_else(|| memory_error(span, "memory region identity space exhausted"))?; + self.regions.insert( + region_id, + Region { + bytes, + max_bytes, + generation: 1, + }, + ); + self.committed_bytes = committed; + Ok(MemoryHandle::from_runtime(self.run_id, region_id, 1)) + } + + /// Current length of a region in pages. + pub fn size(&self, handle: MemoryHandle, span: Option) -> Result { + let region = self.region(handle, span)?; + Ok((region.bytes.len() / PAGE_SIZE) as u64) + } + + /// Grow a region by whole pages and return its previous page count. + /// + /// A failed grow is atomic: the region keeps its old length and bytes and + /// the committed budget is unchanged. + pub fn grow( + &mut self, + handle: MemoryHandle, + pages: u64, + span: Option, + ) -> Result { + if pages == 0 { + return self.size(handle, span); + } + let region = self.region(handle, span)?; + let old_bytes = region.bytes.len(); + let old_pages = (old_bytes / PAGE_SIZE) as u64; + let additional = checked_bytes(pages, span, "memory.grow pages")?; + let new_bytes = old_bytes + .checked_add(additional) + .ok_or_else(|| memory_error(span, "memory.grow size overflows the address space"))?; + if new_bytes > region.max_bytes { + return Err(RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!( + "memory.grow would grow the region to {new_bytes} bytes, beyond its \ + declared maximum of {} bytes", + region.max_bytes + ), + )); + } + let committed = self + .committed_bytes + .checked_add(additional) + .ok_or_else(|| budget_error(span, "memory budget overflow"))?; + if committed > self.limits.max_total_bytes { + return Err(RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!( + "memory budget exceeded: growing by {additional} bytes would reach \ + {committed} of {}", + self.limits.max_total_bytes + ), + )); + } + + let region = self + .regions + .get_mut(&handle.region_id()) + .ok_or_else(|| memory_error(span, "unknown Memory region handle"))?; + region.bytes.try_reserve(additional).map_err(|_| { + RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!("cannot grow addressable memory by {additional} bytes"), + ) + })?; + region.bytes.resize(new_bytes, 0); + self.committed_bytes = committed; + Ok(old_pages) + } + + /// Total byte length of a region. + pub fn byte_len( + &self, + handle: MemoryHandle, + span: Option, + ) -> Result { + Ok(self.region(handle, span)?.bytes.len()) + } + + /// Read a contiguous byte range. Used by resource I/O adapters. + pub fn read_bytes( + &self, + handle: MemoryHandle, + offset: &BigInt, + length: usize, + span: Option, + ) -> Result, RuntimeError> { + let region = self.region(handle, span)?; + let start = range_offset(offset, region.bytes.len(), length, span)?; + Ok(region.bytes[start..start + length].to_vec()) + } + + /// Write a contiguous byte range atomically after a full bounds check. + pub fn write_bytes( + &mut self, + handle: MemoryHandle, + offset: &BigInt, + bytes: &[u8], + span: Option, + ) -> Result<(), RuntimeError> { + let len = self.region(handle, span)?.bytes.len(); + let start = range_offset(offset, len, bytes.len(), span)?; + let region = self + .regions + .get_mut(&handle.region_id()) + .ok_or_else(|| memory_error(span, "unknown Memory region handle"))?; + region.bytes[start..start + bytes.len()].copy_from_slice(bytes); + Ok(()) + } + + /// Read one byte. Offsets are non-negative and region-relative. + pub fn read( + &self, + handle: MemoryHandle, + offset: &BigInt, + span: Option, + ) -> Result { + let region = self.region(handle, span)?; + let position = checked_offset(offset, region.bytes.len(), span)?; + Ok(region.bytes[position]) + } + + /// Write one byte. The caller must have validated the byte range. + pub fn write( + &mut self, + handle: MemoryHandle, + offset: &BigInt, + byte: u8, + span: Option, + ) -> Result<(), RuntimeError> { + let len = self.region(handle, span)?.bytes.len(); + let position = checked_offset(offset, len, span)?; + let region = self + .regions + .get_mut(&handle.region_id()) + .ok_or_else(|| memory_error(span, "unknown Memory region handle"))?; + region.bytes[position] = byte; + Ok(()) + } + + fn region(&self, handle: MemoryHandle, span: Option) -> Result<&Region, RuntimeError> { + if handle.run_id() != self.run_id { + return Err(memory_error( + span, + "Memory handle belongs to a different execution run", + )); + } + let Some(region) = self.regions.get(&handle.region_id()) else { + return Err(memory_error(span, "unknown Memory region handle")); + }; + if region.generation != handle.generation() { + return Err(memory_error(span, "stale Memory region handle")); + } + Ok(region) + } +} + +fn checked_bytes(pages: u64, span: Option, what: &str) -> Result { + let bytes = pages + .checked_mul(PAGE_SIZE as u64) + .ok_or_else(|| memory_error(span, format!("{what} overflows the address space")))?; + usize::try_from(bytes).map_err(|_| { + RuntimeError::new( + ErrorCategory::ResourceExhausted, + span, + format!("{what} ({pages} pages) does not fit this platform's address space"), + ) + }) +} + +/// Validate an offset plus a byte count against the region length. +fn range_offset( + offset: &BigInt, + len: usize, + length: usize, + span: Option, +) -> Result { + if offset.sign() == num_bigint::Sign::Minus { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + "Memory offset must be a non-negative integer", + )); + } + let Some(start) = offset.to_usize() else { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + format!("Memory offset {offset} is out of bounds for region length {len}"), + )); + }; + let end = start.checked_add(length).ok_or_else(|| { + memory_error( + span, + "Memory offset plus length overflows the address space", + ) + })?; + if end > len { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + format!("Memory range {start}..{end} is out of bounds for region length {len}"), + )); + } + Ok(start) +} + +fn checked_offset(offset: &BigInt, len: usize, span: Option) -> Result { + if offset.sign() == num_bigint::Sign::Minus { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + "Memory offset must be a non-negative integer", + )); + } + let Some(position) = offset.to_usize() else { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + format!("Memory offset {offset} is out of bounds for region length {len}"), + )); + }; + if position >= len { + return Err(RuntimeError::new( + ErrorCategory::IndexOutOfBounds, + span, + format!("Memory offset {offset} is out of bounds for region length {len}"), + )); + } + Ok(position) +} + +fn memory_error(span: Option, message: impl Into) -> RuntimeError { + RuntimeError::new(ErrorCategory::MemoryFault, span, message) +} + +fn budget_error(span: Option, message: impl Into) -> RuntimeError { + RuntimeError::new(ErrorCategory::ResourceExhausted, span, message) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn int(value: impl Into) -> BigInt { + value.into() + } + + fn store() -> MemoryStore { + MemoryStore::new(MemoryLimits::default()) + } + + #[test] + fn regions_are_isolated_and_handles_share_identity() { + let mut store = store(); + let first = store.create(1, 2, None).unwrap(); + let second = store.create(1, 2, None).unwrap(); + let alias = first; + store.write(first, &int(0), 7, None).unwrap(); + store.write(second, &int(0), 9, None).unwrap(); + assert_eq!(store.read(first, &int(0), None).unwrap(), 7); + assert_eq!(store.read(alias, &int(0), None).unwrap(), 7); + assert_eq!(store.read(second, &int(0), None).unwrap(), 9); + assert_eq!(first, alias); + assert_ne!(first, second); + } + + #[test] + fn create_is_zero_filled_and_reports_pages() { + let mut store = store(); + let memory = store.create(1, 4, None).unwrap(); + assert_eq!(store.size(memory, None).unwrap(), 1); + assert_eq!(PAGE_SIZE, 64 * 1024); + for offset in [0usize, 1, PAGE_SIZE - 1] { + assert_eq!(store.read(memory, &int(offset as u64), None).unwrap(), 0); + } + // Zero-page creation is legal but every access is out of bounds. + let empty = store.create(0, 1, None).unwrap(); + assert_eq!(store.size(empty, None).unwrap(), 0); + assert_eq!( + store.read(empty, &int(0), None).unwrap_err().category, + ErrorCategory::IndexOutOfBounds + ); + } + + #[test] + fn grow_keeps_old_bytes_and_zero_fills_new_pages() { + let mut store = store(); + let memory = store.create(1, 2, None).unwrap(); + store.write(memory, &int(3), 200, None).unwrap(); + assert_eq!(store.grow(memory, 1, None).unwrap(), 1); + assert_eq!(store.size(memory, None).unwrap(), 2); + assert_eq!(store.read(memory, &int(3), None).unwrap(), 200); + assert_eq!(store.read(memory, &int(PAGE_SIZE as u64), None).unwrap(), 0); + store + .write(memory, &int(PAGE_SIZE as u64 + 5), 42, None) + .unwrap(); + assert_eq!( + store + .read(memory, &int(PAGE_SIZE as u64 + 5), None) + .unwrap(), + 42 + ); + // Zero-page growth is a no-op returning the old count. + assert_eq!(store.grow(memory, 0, None).unwrap(), 2); + } + + #[test] + fn failed_grow_does_not_change_region_or_budget() { + let mut store = MemoryStore::new(MemoryLimits { + max_total_bytes: PAGE_SIZE * 2, + max_region_bytes: PAGE_SIZE * 4, + }); + let memory = store.create(1, 2, None).unwrap(); + store.write(memory, &int(0), 1, None).unwrap(); + let before = store.committed_bytes(); + let error = store.grow(memory, 2, None).unwrap_err(); + assert_eq!(error.category, ErrorCategory::ResourceExhausted); + assert_eq!(store.size(memory, None).unwrap(), 1); + assert_eq!(store.read(memory, &int(0), None).unwrap(), 1); + assert_eq!(store.committed_bytes(), before); + } + + #[test] + fn invalid_offsets_are_structured_errors() { + let mut store = store(); + let memory = store.create(1, 1, None).unwrap(); + for offset in [int(-1), int(PAGE_SIZE as u64)] { + let error = store.read(memory, &offset, None).unwrap_err(); + assert_eq!(error.category, ErrorCategory::IndexOutOfBounds); + } + let huge: BigInt = "999999999999999999999999999999999999999".parse().unwrap(); + assert_eq!( + store.read(memory, &huge, None).unwrap_err().category, + ErrorCategory::IndexOutOfBounds + ); + assert_eq!( + store.write(memory, &huge, 1, None).unwrap_err().category, + ErrorCategory::IndexOutOfBounds + ); + assert_eq!( + store + .write(memory, &int(PAGE_SIZE as u64), 1, None) + .unwrap_err() + .category, + ErrorCategory::IndexOutOfBounds + ); + } + + #[test] + fn create_validates_page_counts_and_budget() { + let mut store = MemoryStore::new(MemoryLimits { + max_total_bytes: PAGE_SIZE * 3, + max_region_bytes: PAGE_SIZE * 2, + }); + assert_eq!( + store.create(2, 1, None).unwrap_err().category, + ErrorCategory::MemoryFault + ); + assert_eq!( + store.create(1, 3, None).unwrap_err().category, + ErrorCategory::ResourceExhausted + ); + let _first = store.create(2, 2, None).unwrap(); + let _second = store.create(1, 1, None).unwrap(); + // Total budget is now exactly three pages. + assert_eq!( + store.create(1, 1, None).unwrap_err().category, + ErrorCategory::ResourceExhausted + ); + assert_eq!(store.committed_bytes(), PAGE_SIZE * 3); + } + + #[test] + fn forged_and_stale_handles_are_rejected() { + let mut store = store(); + let memory = store.create(1, 1, None).unwrap(); + let forged = MemoryHandle::from_runtime(memory.run_id() + 1, memory.region_id(), 1); + assert_eq!( + store.read(forged, &int(0), None).unwrap_err().category, + ErrorCategory::MemoryFault + ); + let stale = MemoryHandle::from_runtime(memory.run_id(), memory.region_id(), 2); + assert_eq!( + store.read(stale, &int(0), None).unwrap_err().category, + ErrorCategory::MemoryFault + ); + let unknown = MemoryHandle::from_runtime(memory.run_id(), 999, 1); + assert_eq!( + store.write(unknown, &int(0), 1, None).unwrap_err().category, + ErrorCategory::MemoryFault + ); + } + + #[test] + fn page_arithmetic_never_overflows() { + let mut store = MemoryStore::new(MemoryLimits { + max_total_bytes: usize::MAX, + max_region_bytes: usize::MAX, + }); + assert_eq!( + store.create(u64::MAX, u64::MAX, None).unwrap_err().category, + ErrorCategory::ResourceExhausted + ); + } +} diff --git a/crates/marqdo-runtime/src/value_ops.rs b/crates/marqdo-runtime/src/value_ops.rs index d568119..9feab4d 100644 --- a/crates/marqdo-runtime/src/value_ops.rs +++ b/crates/marqdo-runtime/src/value_ops.rs @@ -107,6 +107,32 @@ fn compare( } })) } + (Value::Resource(a), Value::Resource(b)) => { + return Ok(Value::Bool(match op { + BinaryOp::Eq => a == b, + BinaryOp::Ne => a != b, + _ => { + return Err(type_error( + span, + "Resource ordering comparisons are not supported".to_string(), + )) + } + })); + } + (Value::Memory(a), Value::Memory(b)) => { + // Identity, not byte content: two handles for the same region are + // equal, regions with identical bytes are not (ADR-0022 §1). + return Ok(Value::Bool(match op { + BinaryOp::Eq => a == b, + BinaryOp::Ne => a != b, + _ => { + return Err(type_error( + span, + "Memory ordering comparisons are not supported".to_string(), + )) + } + })); + } (a, b) => { if op == BinaryOp::Eq { return Ok(Value::Bool(false)); diff --git a/crates/marqdo-runtime/tests/interpreter.rs b/crates/marqdo-runtime/tests/interpreter.rs index a054a8b..2601a44 100644 --- a/crates/marqdo-runtime/tests/interpreter.rs +++ b/crates/marqdo-runtime/tests/interpreter.rs @@ -2,7 +2,7 @@ use marqdo_host::{Capabilities, Host, StdHost}; use marqdo_ir::{ModuleId, Program}; -use marqdo_runtime::{run, EvaluatedArg, RuntimeError, Value}; +use marqdo_runtime::{run, run_with_limits, EvaluatedArg, MemoryLimits, RuntimeError, Value}; use marqdo_sandbox::SandboxMode; use marqdo_span::Span; @@ -170,3 +170,46 @@ fn ambiguous_ref_resource_is_an_error() { let error = run_named(source, "f", &[]).unwrap_err(); assert_eq!(error.code(), "ref.ambiguous"); } + +#[test] +fn memory_region_is_shared_by_handle_identity() { + let source = "## f\n\n**m = memory.new(1, 2)**\n\n**n = m**\n\n**`m[3] = 7`**\n\n*[n[3], m == n, memory.size(n)]*\n"; + let value = run_named(source, "f", &[]).unwrap(); + assert_eq!( + value, + Value::List(vec![Value::int(7), Value::Bool(true), Value::int(1)]) + ); +} + +#[test] +fn memory_handle_cannot_reach_the_host_boundary() { + let source = "## f\n\n**m = memory.new(1, 1)**\n\n> 打印(m)\n\n*0*\n"; + let error = run_named(source, "f", &[]).unwrap_err(); + assert_eq!(error.code(), "runtime.type"); + assert!( + error + .message + .contains("Memory values cannot cross the host boundary"), + "unexpected message: {}", + error.message + ); +} + +#[test] +fn memory_budget_is_enforced_without_changing_the_region() { + let program = build("## f\n\n**m = memory.new(1, 1)**\n\n*[m[0], memory.grow(m, 1)]*\n"); + let mut host = host(); + let error = run_with_limits( + &program, + &mut host, + &ModuleId::from("index.mq.md"), + "f", + Vec::new(), + MemoryLimits { + max_total_bytes: 64 * 1024, + max_region_bytes: 64 * 1024 * 2, + }, + ) + .unwrap_err(); + assert_eq!(error.code(), "resource.exhausted"); +} diff --git a/crates/marqdo-value/src/error.rs b/crates/marqdo-value/src/error.rs index 69691e2..16ce085 100644 --- a/crates/marqdo-value/src/error.rs +++ b/crates/marqdo-value/src/error.rs @@ -23,8 +23,19 @@ pub enum ErrorCategory { NotCallable, /// A map key or record field does not exist. UnknownField, - /// A list/map index is out of bounds. + /// A list/map/memory index is out of bounds. IndexOutOfBounds, + /// An addressable-memory operation failed for a reason other than a plain + /// out-of-bounds read: invalid page counts, an unknown/stale handle, or an + /// arithmetic overflow in an address. + MemoryFault, + /// A resource handle is forged, cross-run or stale. + ResourceFault, + /// A resource operation used an already-closed handle (recoverable as a + /// `Result` by the builtin). + ResourceClosed, + /// The requested resource kind/descriptor/rights are not implemented. + ResourceUnsupported, /// A resource budget was exhausted (fuel, stack, output size). ResourceExhausted, /// A capability check denied the operation. @@ -60,6 +71,10 @@ impl ErrorCategory { ErrorCategory::NotCallable => "call.not_callable", ErrorCategory::UnknownField => "runtime.unknown_field", ErrorCategory::IndexOutOfBounds => codes::RUNTIME_INDEX, + ErrorCategory::MemoryFault => "runtime.memory", + ErrorCategory::ResourceFault => "runtime.resource", + ErrorCategory::ResourceClosed => "resource.closed", + ErrorCategory::ResourceUnsupported => "resource.unsupported", ErrorCategory::ResourceExhausted => codes::RESOURCE_EXHAUSTED, ErrorCategory::PermissionDenied => codes::PERMISSION_DENIED, ErrorCategory::KernelUnavailable => codes::KERNEL_UNAVAILABLE, diff --git a/crates/marqdo-value/src/lib.rs b/crates/marqdo-value/src/lib.rs index 308eddb..4dfa75d 100644 --- a/crates/marqdo-value/src/lib.rs +++ b/crates/marqdo-value/src/lib.rs @@ -15,6 +15,6 @@ pub use error::{CallFrame, ErrorCategory, RuntimeError}; pub use marqdo_span::{LineCol, Span}; pub use value::{ bigint_to_exact_f64, float_to_bigint, format_float, CodeValue, FormulaValue, FunctionRef, - PluginHandle, Table, Value, + MemoryHandle, PluginHandle, ResourceHandle, Table, Value, }; pub use wire::{from_wire, to_wire, WireError, WIRE_VERSION}; diff --git a/crates/marqdo-value/src/value.rs b/crates/marqdo-value/src/value.rs index 0f28ef5..f11c7f8 100644 --- a/crates/marqdo-value/src/value.rs +++ b/crates/marqdo-value/src/value.rs @@ -75,6 +75,88 @@ pub struct PluginHandle { pub id: u64, } +/// Opaque identity of one runtime-managed byte region (ADR-0022). +/// +/// A handle is issued only by the runtime's `MemoryStore`. Copying a `Memory` +/// value or passing it as an argument preserves this identity, so two names +/// can observe each other's writes. The fields are deliberately not part of +/// any wire format: [`crate::wire`] rejects the `memory` tag on decode, and no +/// public constructor accepts untrusted data. There is no `usize` address in +/// the language value; offsets are region-relative integers. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct MemoryHandle { + run: u64, + region: u64, + generation: u64, +} + +impl MemoryHandle { + /// Create a handle. Only the runtime store may call this; users cannot + /// construct one from a number, a document or wire data. + #[doc(hidden)] + pub const fn from_runtime(run: u64, region: u64, generation: u64) -> Self { + Self { + run, + region, + generation, + } + } + + /// Run identity this region belongs to. + pub const fn run_id(&self) -> u64 { + self.run + } + + /// Region identity within the run. + pub const fn region_id(&self) -> u64 { + self.region + } + + /// Generation of the region slot (guards a future reuse policy). + pub const fn generation(&self) -> u64 { + self.generation + } +} + +/// Opaque host-issued handle to an external resource/stream (ADR-0025). +/// +/// Fields are private and the constructor is host-only. A handle is never +/// built from wire data or a user integer; it is a different domain from +/// `Memory` and cannot be indexed. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct ResourceHandle { + run: u64, + resource: u64, + generation: u64, +} + +impl ResourceHandle { + /// Host-only constructor. + #[doc(hidden)] + pub const fn from_host(run: u64, resource: u64, generation: u64) -> Self { + Self { + run, + resource, + generation, + } + } + + /// Host run identity. + pub const fn run_id(&self) -> u64 { + self.run + } + + /// Resource slot identity. + pub const fn resource_id(&self) -> u64 { + self.resource + } + + /// Slot generation. + pub const fn generation(&self) -> u64 { + self.generation + } +} + /// A Marqdo runtime value. #[derive(Debug, Clone, PartialEq)] pub enum Value { @@ -104,6 +186,11 @@ pub enum Value { FunctionRef(FunctionRef), /// Host-owned plugin handle. PluginHandle(PluginHandle), + /// Runtime-owned addressable byte region (RFC-CM-0001). Copies share the + /// same region identity; they are not byte snapshots. + Memory(MemoryHandle), + /// Host-owned external resource/stream handle (ADR-0025). + Resource(ResourceHandle), } impl Value { @@ -123,6 +210,8 @@ impl Value { Value::Code(_) => "Code", Value::FunctionRef(_) => "FunctionRef", Value::PluginHandle(_) => "PluginHandle", + Value::Memory(_) => "Memory", + Value::Resource(_) => "Resource", } } @@ -200,6 +289,44 @@ impl Value { Value::Code(c) => format!("Code({:?}, {} bytes)", c.language, c.source.len()), Value::FunctionRef(r) => format!("FunctionRef({}::{})", r.module, r.name), Value::PluginHandle(h) => format!("PluginHandle({}::{})", h.plugin, h.export), + // Never render the runtime handle: internal region identity is not + // observable program data (RFC-CM-0001 §8). + Value::Memory(_) => "Memory".to_string(), + // Resource identity is host-private and never program data. + Value::Resource(_) => "Resource".to_string(), + } + } + + /// Whether this value contains a live `Memory` handle anywhere. + /// + /// Used at the host/plugin/foreign boundary, which must reject handles + /// instead of serializing them. + pub fn contains_memory(&self) -> bool { + match self { + Value::Memory(_) => true, + Value::List(items) => items.iter().any(Value::contains_memory), + Value::Map(entries) => entries.values().any(Value::contains_memory), + Value::Table(table) => table + .rows + .iter() + .flat_map(|row| row.iter()) + .any(Value::contains_memory), + _ => false, + } + } + + /// Whether this value contains a live `Resource` handle anywhere. + pub fn contains_resource(&self) -> bool { + match self { + Value::Resource(_) => true, + Value::List(items) => items.iter().any(Value::contains_resource), + Value::Map(entries) => entries.values().any(Value::contains_resource), + Value::Table(table) => table + .rows + .iter() + .flat_map(|row| row.iter()) + .any(Value::contains_resource), + _ => false, } } @@ -325,6 +452,18 @@ mod tests { ); } + #[test] + fn memory_display_never_leaks_the_runtime_handle() { + let memory = Value::Memory(MemoryHandle::from_runtime(41, 7, 1)); + assert_eq!(memory.type_name(), "Memory"); + assert_eq!(memory.display(), "Memory"); + assert_eq!(memory.repr(), "Memory"); + assert!(!memory.display().contains("41")); + assert!(!memory.repr().contains('7')); + assert!(memory.contains_memory()); + assert!(Value::List(vec![memory]).contains_memory()); + } + #[test] fn big_integers_are_arbitrary_precision() { let big: BigInt = "123456789012345678901234567890".parse().unwrap(); diff --git a/crates/marqdo-value/src/wire.rs b/crates/marqdo-value/src/wire.rs index efe891b..84c1603 100644 --- a/crates/marqdo-value/src/wire.rs +++ b/crates/marqdo-value/src/wire.rs @@ -145,6 +145,12 @@ fn encode(value: &Value) -> Json { "export": handle.export, "id": handle.id, }), + // Active memory handles never cross the wire (RFC-CM-0001 §8). + // The tag is emitted so an accidental transfer is visible instead of + // silently becoming a number; decoding it is a hard error. + Value::Memory(_) => json!({"type": "memory"}), + // Resource handles are host-private; same non-transferable rule. + Value::Resource(_) => json!({"type": "resource"}), } } @@ -321,6 +327,12 @@ fn decode(json: &Json, depth: usize, limits: &WireLimits) -> Result Err(WireError::Invalid( + "Memory handles cannot cross the wire; copy bytes explicitly".into(), + )), + "resource" => Err(WireError::Invalid( + "Resource handles cannot cross the wire".into(), + )), other => Err(WireError::UnknownType(other.to_string())), } } @@ -393,6 +405,34 @@ mod tests { assert!(matches!(decoded, Value::Float(f) if f == f64::NEG_INFINITY)); } + #[test] + fn memory_handles_are_not_transferable() { + let value = Value::Memory(crate::MemoryHandle::from_runtime(1, 2, 1)); + let wire = to_wire(&value); + assert_eq!(wire, json!({"type": "memory"})); + assert_eq!( + from_wire(&wire), + Err(WireError::Invalid( + "Memory handles cannot cross the wire; copy bytes explicitly".into() + )) + ); + // A hand-written fake handle is rejected by the same tag rule. + assert!(from_wire(&json!({"type": "memory", "region": 2})).is_err()); + } + + #[test] + fn resource_handles_are_not_transferable() { + let value = Value::Resource(crate::ResourceHandle::from_host(1, 2, 1)); + let wire = to_wire(&value); + assert_eq!(wire, json!({"type": "resource"})); + assert_eq!( + from_wire(&wire), + Err(WireError::Invalid( + "Resource handles cannot cross the wire".into() + )) + ); + } + #[test] fn untrusted_wire_is_validated() { assert_eq!( diff --git a/docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md b/docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md new file mode 100644 index 0000000..f9daf37 --- /dev/null +++ b/docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md @@ -0,0 +1,1031 @@ +# RFC-CM-0002:Marqdo 最小计算内核、标准库自举与系统能力迁移 + +> **状态**:Draft / 架构与迁移提案,未批准、未实施。所有代码均为目标语义草案,不能当作已运行示例。 +> **日期**:2026-11-20 +> **依赖与基线**:在《RFC-CM-0001:Marqdo 自主计算机器、可寻址内存与系统交互边界》之上细化,不代替其中的多内存区域与资源边界原则。 +> **源码核对基线**:`cflmy/marqdo-cli`,`a113d894962d8c00eb02efb42e0e73f8b084a1fa`。后续实现前必须重新确认 HEAD 和工作分支差异。 +> **建议路径**:`docs/architecture/RFC-CM-0002-computational-closure-and-stdlib-migration.md` +> **约定**:文中的“必须”指**提议批准本 RFC 后**的实现约束,不表明仓库已有此能力;“候选接口”不是已冻结的语法/API。 + +--- + +## 0. 执行摘要与最重要的结论 + +本 RFC 不主张“将 Rust 代码机械地翻译为 Marqdo”,而是要**改变计算功能的所有权**: + +1. Marqdo Runtime 只承担**不能由自身程序构造的计算和存储机制**:基础值运算、结构访问、通用可赋值位置、有身份的内存区域、字节读写/增长、运行时安全检查、函数执行与受限资源调用。 +2. 普通数据结构、缓冲区、分配器、二进制编解码、文本格式化、GFM 表格变换、标准输入输出包装、文件协议与应用算法在 `.mq.md` 标准库中实现。 +3. “打印”等高级操作**不再是不可替换的 Host 特例**;其格式化、UTF-8、换行、部分写循环等属于 Marqdo。最终将字节送出 stdout 的动作仍属于操作系统边界。 +4. GFM Table 不等于一段裸字节:**Markdown 表格的源码结构/可读性保留,Table 的逻辑值语义固定,底层存储可替换或选择列式/区域视图**;表格常规算法归入标准库。 +5. 扩展功能不应要求经常修改 `HostOp`。资源访问使用**版本化、可发现、带授权的低层 Provider 契约**;只能以现有 ABI/CallableRegistry 为基础增量演进,绝不并列制造第二套影子插件系统。 +6. “不依赖宿主语言”的精确定义:开发者仅修改 `.mq.md` 就能扩展**有共同底层原语支持的算法与系统高级功能**;操作系统新原始能力、设备驱动、新硬件指令以及部分性能优化依然需要平台适配器或编译后端。运行时由 Rust 实现,不等同于算法外包给 Rust。 +7. 必须以**计算闭包验证**验收:将标准库高级 Builtin/Foreign Kernel 禁用,不增加 Rust/ABI 实现,纯 Marqdo 仍可完成指定完整应用。否则语言内核尚不充分。 + +**核心决策建议:** `Managed Value + Addressable Region + Capability Resource` 三领域;**只有语义不能自行实现,才有资格进入语言内核**。但“能用低级指令极慢地模拟”不等于拥有实用语言基础;基本的**有界数据结构构造/遍历、错误恢复和高效字节接口**必须被证明可用。 + +### 一页式职责分离 + +```text +普通 .mq.md 应用 / Agent / 科研程序 + │ + ├── Marqdo 标准库 (.mq.md 为主) ─────────────────────────────────┐ + │ print, read_line, format, utf8, table, json, http, allocator ... │ + │ │ + ▼ ▼ +Marqdo 语义核心 资源调用边界 + AST→IR、值、函数、控制流 capability + provider + 位置赋值 / 内存区域 / 有限集合基础 stdio / fs / net / device + 错误恢复 / 有限类型与遍历 │ + │ ▼ + └─────────────── CPU 解释器/未来独立编译器 ───────────────────► OS / Device + +源码 Markdown 链接/GFM Table/标题属于文档层;不是通用堆地址和系统调用指令。 +``` + +--- + +## 1. 设计目标、拒绝的捷径与不变量 + +### 1.1 目标 + +- **G1 自主计算**:不需要 Python/Node/Foreign、为算法专写 Rust Builtin、为每个容器专写 HostOp,就能用 `.mq.md` 完成数据结构和计算算法。 +- **G2 系统构造性**:只要基础 OS 资源端点已存在并获授权,Marqdo 应自行实现 `print/read_line/copy_file/HTTP parser/CLI commands` 等高层 API。 +- **G3 内核稳定**:未来 100 个独立标准库函数,不意味着增加 100 个内置函数或触及 Rust 编译器/解释器。 +- **G4 语义统一**:高级函数运行在与普通程序**同一个**值、内存、调用、错误、来源映射和权限体系下,没有“标准库专用魔法规则”。 +- **G5 兼容现有 Markdown 语义**:`##/**/*`、链接/表格惰性、函数与模块寻址、执行顺序不被重写。 +- **G6 工程品质**:至少能构造可扩展容器、网络字节协议、异构表格、持续交互工具,同时明确性能瓶颈和安全保证。 +- **G7 明确安全边界**:内部虚拟内存权限与资源访问权限分别验证;不能把“运行在 Rust 内”误写为“自动内存安全”,也不能把普通 Worker 等同于 OS 隔离。 + +### 1.2 明确不做 + +- 不新增面向业务的 Markdown 关键字,例如 `PRINT/ALLOC/HTTP/TENSOR/GPU`。 +- 不以“任何 Rust 实现都是语法糖”为判断依据;整数、浮点、GC/容器表示、文件描述符管理当然必须有底层实现。 +- 不将 GFM Table 直接等价于物理 RAM、也不让编辑源码的表格自动修改运行时堆。 +- 不为与 OS 交互提供无限制的 `syscall(number, raw_address)` 默认后门。 +- 不强行让已有 `Value::List`、`Value::Map`、`Value::Table` 全部搬到线性内存,避免将高级常用值退化为只能手工处理的裸字节。 +- 不假定提供读写内存就能自然获得原子同步、时钟、密码学安全随机数、TLS、GPU 或 QPU。 +- 不先开发自己的 JIT、CPU 寄存器、异步事件循环或巨型调度框架。 + +### 1.3 跨整个迁移期都必须成立的不变量 + +| ID | 不变量 | 解释 | +|---|---|---| +| INV-01 | Markdown AST 是结构语义的唯一权威 | 不在文本扫描中偷偷执行表格、链接、代码围栏 | +| INV-02 | 普通数值和 List/Map/Table 原有可观察语义不变 | 别名行为变化需要新的显式抽象/ADR | +| INV-03 | Memory 引用有身份,数字偏移不具有全局权限 | 不能用整数伪造另一区域地址 | +| INV-04 | 任何虚拟内存访问均在有效区域内 | 大整数转换、地址加法、长度计算须检查溢出 | +| INV-05 | 函数与标准库共享同一解释执行机制 | 不为标准库增加隐藏的 Rust 快捷调度 | +| INV-06 | 纯计算不会自动获得 OS 权限 | 安装依赖、调用函数、拥有路径字符串都不构成授权 | +| INV-07 | 操作系统交互必须能被识别为外部效应 | 权限、审计、失败、部分读写均可见 | +| INV-08 | Resource/Memory 句柄不能从不可信 JSON 伪造 | 不可伪造身份、generation、所有权/权限检查 | +| INV-09 | 输入/输出数据不隐式混入诊断通道 | stdout、stderr、JSON/trace 按原规范保持隔离 | +| INV-10 | 不承诺未实现的安全或硬件能力 | strict/worker/设备/并行都按实际支持矩阵陈述 | +| INV-11 | 高层功能可在独立 stdlib 迭代 | 不修改 CLI 源码、无需重新链接 Rust 运行时 | +| INV-12 | 被替换的内置功能必须有兼容性黄金测试 | 不接受“通常一样”或仅观察几条 happy path | + +--- + +## 2. 现有仓库审查:应该保留什么、必须改变什么 + +本节来自固定提交 `a113d894...` 的实际代码,**不是本 RFC 的拟议能力**。 + +| 源码/规范 | 当前事实 | 架构影响 | +|---|---|---| +| `crates/marqdo-value/src/value.rs` | `Value` 含 `Unit/None/Bool/Int/Float/Text/List/Map/Table/Formula/Code/FunctionRef/PluginHandle`,GFM `Table { header: Vec, rows: Vec> }` | 缺通用可寻址字节区域;Table 强绑定具体 Rust 行存储 | +| `crates/marqdo-expression/src/ast.rs` | `StatementKind::Assign` 左侧只有 identifier;成员/索引目前可读 | 内存区域写入需要 `Place` 目标语义 | +| `crates/marqdo-runtime/src/interpreter.rs` | `Frame.locals` 存 `BTreeMap`;赋值重绑定本地槽 | 保持现有值语义;Memory 另有引用身份 | +| `crates/marqdo-runtime/src/builtins_impl.rs` | `打印` 转发 `HostOp::Print`;`table()` 在 Rust 校验 headers/rows;`文本(str)` **仅接受 Text/Int/Float/Bool/None/Unit** | `打印` 迁移前不具备由 Marqdo 泛化格式化全部 Value 的必要观察接口 | +| `crates/marqdo-host/src/host.rs` | `op_print` 经 `Value::display()` 格式化,追加换行、检查 console grant 和 output budget、写入 Rust 输出缓冲区 | 迁移必须把“值格式化”和“真正字节输出”拆开;不能丢掉权限及预算 | +| `crates/marqdo-host/src/capability.rs` | `CapabilityBroker::execute` 只对部分能力提供真实执行,其他可返回 unavailable | 不能把“接口已声明”当作“端点已实现” | +| `spec/evaluation.md` | Builtin 的稳定名字包括 `打印/print`、`长度`、`类型`、`文本`、`整数`、`小数`、`表`、`ref`、math/foreign/fs | 需要明确公开 API 兼容表及迁移阶段 | +| `spec/mapping.md` | GFM Table cells 惰性;Markdown 链接纯引用、不会因为渲染执行 | 不得让表格直接成为低层命令/指针操作 | +| `spec/security.md` | `pure/project/strict` 分层;Linux strict L1 有探测与限定支持,其他情况 fail closed;console 输出特殊允许 | 新 I/O 端点不能自动在 strict 里开放;旧 stdout 行为要保留 | +| `spec/plugin-abi.md` | C ABI v1、worker、capability request、签发的 PluginHandle | 先评估复用而不是另造 ABI;普通子进程不是安全沙箱 | +| `spec/package-format.md` | path/Git+revision、lock v3;Manifest 中部分 runtime 范围约束尚未真正强制 | stdlib 分仓与离线分发要补上可验证版本契约 | + +当前存在一个**容易被忽视的闭包缺口**:`Value::display()` 在 Rust 中能打印列表/表格等复杂值,但现有 Marqdo 的 `文本()` 不能转换这些值,且 Table 没有完整的公开通用遍历/构造 API。**直接删掉 `HostOp::Print` 并在 `.mq.md` 写 `打印` 是做不到无损迁移的。** 必须先补足对所有值类型的受控观察与组合能力。 + +### 2.1 迁移工作的实际代码冲击面 + +```text +语言与解析:crates/marqdo-expression, crates/marqdo-frontend, crates/marqdo-ir +值与执行:crates/marqdo-value, crates/marqdo-runtime +系统边界:crates/marqdo-host, crates/marqdo-sandbox, 当前 Plugin ABI +包分发: crates/marqdo-package, crates/cli +文档规范:spec/evaluation.md, spec/expression.ebnf, + spec/mapping.md, spec/references.md, spec/security.md, spec/value-wire.md +回归: AST goldens, conformance, CLI, platform, security, lockfile, workload +``` + +除非某个需求确实突破了现有抽象,不能因为“准备标准库迁移”就在上述每个 crate 中引入一套独立规则。 + +--- + +## 3. 不可再分解的计算内核:**不是最少函数数量,而是最小充分语义** + +### 3.1 三个语义领域 + +**Managed Value**:`Int/Float/Bool/Text/List/Map/Table/...` 的可观察值语义,由 Runtime 管理存储,用户不能据此直接获得宿主进程地址。`Text` 是 Unicode 字符序列语义,不与 UTF-8 字节数组混淆。 + +**Addressable Region**:多个不同身份、独立限额、可增长、按字节索引的存储区域;句柄复制表示共享同一块区域,偏移是区域内部的整数。用户可以在其中构造分配器、数据结构和自定义二进制布局。 + +**Capability Resource**:由可信环境授予的不透明外部对象,例如 stdin/stdout、文件目录权限、连接、设备。其 read/write 可能涉及不可重放的外部效应,不等同于普通 Memory。 + +此三分法刻意不包括“真实 CPU 寄存器”。局部变量/临时值已经是抽象执行槽;硬件寄存器分配属于编译器后端。GPU/QPU 硬件内存属于今后的有约束资源或后端,不作为普通可任意读写的 Region 伪装。 + +### 3.2 计算闭包:内核应提供的不可省略能力 + +| 分类 | 最小语言机制 | 为什么不可简单只靠库补齐 | 归属 | +|---|---|---|---| +| 控制 | 函数、参数、调用、条件、循环、递归、返回 | 语言自身执行与调用语义 | 维持 Core | +| 数值 | Bool、精确 Int、Float、基本算术/比较 | 构成一般计算和地址运算 | Core;位操作优先在库实现/测成本再定原语 | +| 变量/位置 | 普通绑定重置、索引读写、必要时字段路径写入 | 无位置写入则共享存储不可编程 | Core,少量 grammar 扩展 | +| 动态存储 | 区域创建、大小、增长、有界字节读写 | 程序不能从自身获得新内存;必须由 Runtime 执行 | Core,**非 HostOp** | +| 通用值构造 | 可动态构造、访问、增补 List/Map/Text/表格结构 | 若能读不能构造,`format/table/json` 无法独立实现 | Core 的结构原语 + Stdlib 算法 | +| 可观察类型 | 类型判定/长度/枚举结构成员/安全读取元数据 | 没有足够反射就无法由 Marqdo 格式化全部 Value | 极小结构契约 | +| 错误控制 | 可识别、可传播、可恢复的错误结果 | I/O 和解析器正常失败不能都触发进程终止 | Core 可恢复结果语义(形态需 ADR) | +| 外部资源 | 资源获取、带缓冲区读写、关闭、结构化错误 | 用户态内存不能凭空实现 OS 系统调用 | 受权系统边界 | + +**特别说明:** “无位置赋值也可以用不可变列表的递归重建模拟”在理论上成立,但现有 Marqdo 缺乏动态 List 构造的普遍闭环。不能用这种理论可能性掩盖实际标准库无法写出的事实。计算内核是否充分,应以用户语言的**有效构造能力**为判定标准。 + +### 3.3 内存区域规范(继承 RFC-0001,补充细则) + +`MemoryHandle = (run_id, region_id, generation)` 为运行时创建、不可被用户伪造的句柄;句柄的字节表示不属于用户可观察 API。`Memory` 对外可按索引读写非负偏移的字节。不同区域互不别名;同一区域的句柄复制与传参保留身份。区域内容和长度由 Runtime 内部管理,逻辑偏移不是 Rust 指针。 + +**拟议接口(不是新语法关键字):** + +```text +memory.new(initial_pages, max_pages) -> Memory +memory.size(memory) -> page_count +memory.grow(memory, pages) -> old_page_count or recoverable error +memory[offset] -> Int(0..255) +memory[offset] = Int(0..255) +``` + +- 初始页为零填充;推荐先采用 64 KiB 逻辑页,但是否冻结为规范常量须 ADR;`memory.size` 和 `grow` 不能出现页/字节混用。 +- `grow` 只能增加;失败时原区域状态不变;旧偏移继续有效;请求超过单区域/总内存预算必须失败。 +- 巨大 BigInt、负偏移、非法字节、无效/失效句柄都必须是稳定、可定位的错误,不得 panic。 +- 分配块 `malloc/free` 属于标准库,管理的是**区域内部子区间**;不等于调用方可以释放 Runtime 的整个区域。区域生存期首版由运行/引用可达性管理,至少执行结束必定释放。 +- `m[i] = x` 必须检查赋值左侧,不能让 `list[i] = x` 在未定义 List 语义时静默地变为共享可变对象。 +- 所有资源限额应可测量:区域总数、已提交字节、逻辑最大容量、复制/增长预算、运行时间和 fuel;内存分配失败不得误报为成功。 + +### 3.4 Place/索引写入:最小 AST 变化 + +现有 `assignment = identifier '=' expression` 扩展为候选形式: + +```ebnf +assignment = assign_target, "=", expression ; +assign_target = identifier, { "[", expression, "]" | ".", identifier } ; +``` + +上式是**允许将来统一的语法形态**;首期语义仅接受 `Local(name)` 和 `Index(memory,index)`。对于尚未定义的目标(如 Map 成员更新、嵌套 List 更新),必须给出明确诊断,不能悄悄套用 Rust 的可变引用规则。 + +赋值左侧先求 base,再求 index,再求 RHS(精确求值顺序须写入 `spec/evaluation.md`);若参数检查不通过,操作不提交。`Memory` 区域更新是共享的;普通 `List/Map/Table` 仍按既定值语义处理。 + +### 3.5 一项必须补齐的核心能力:动态复合值构造与结构观察 + +只有按字节的虚拟内存,的确可以**从零编码任何数据结构**;但如果普通 `.mq.md` 程序无法动态构造新的 List、Map 或遍历 Table,就会迫使所有高级库把 Unicode 字符串和异构数据都退化为自定义字节内存布局,工程上反而不可接受。 + +因此建议在**同一套值访问语义**下补齐最少的结构操作,而非给每种业务对象加新的类型: + +- `List`:访问长度、按序读取、动态构造/追加、明确的值更新(可用返回新列表的操作实现); +- `Map`:查询、枚举键、创建/关联/删除键,不暴露 Rust `BTreeMap` 接口作为唯一调用方式; +- `Text`:长度、Unicode 标量遍历、拼接;UTF-8 编码/解码可在 `.mq.md` 实现,但应允许通过 Memory 生成字节; +- `Table`:公开列名、行数、行/单元格读取的逻辑接口;如有变化操作,遵守值语义; +- 特殊文档值 `Formula/Code/FunctionRef`:只暴露合法的**可观察元信息**,不能未经授权展开私密资源内容; +- `Memory/Resource`:只暴露类别、有限状态及受权操作,**不提供完整原始地址或可伪造 ID**。 + +这些可以作为**少量统一的结构原语**,其对外名称由 ADR 设计;`collections/` 标准库仍负责排序、过滤、连接、分组、图遍历、表格变换等真正的算法。新增一个标准库的 `table.join` 不应触发新的内置操作。 + +**判据**:构造 JSON parser/formatter、GFM Table 变换、任意尺寸 List、泛型 `print` 时,不能发现“在 Rust 里可以观察/创建,Marqdo 里完全做不到”。若做不到,就说明计算核心缺乏**通用结构观察或构造机制**。 + +### 3.6 可恢复错误:别让标准库只能让进程中止 + +传统错误如 `unknown symbol`、程序越界可当语言诊断;但文件不存在、网络被关闭、UTF-8 非法序列、解析失败、存储不足等往往是**可预期的运行时条件**,用户必须能够在 Marqdo 中判断和恢复。 + +推荐在语言中支持标准的 `Result` 数据契约(可以用现有 Map/Record 值编码;不必增加 `try/catch` 关键字),区分: + +- `Ok(value)` / `Err(code, message, metadata)`:业务可恢复错误,函数直接返回并让上层用普通分支处理; +- 违反语言不变量/用户错误或预算超限:保留稳定诊断,必要时停止当前执行; +- 外部资源可能的 `EOF / partial / pending / canceled`:须显式区分,不把 EOF 当成临时无数据或错误。 + +错误格式、传播约定及能否从 `memory.grow` 失败恢复须用 ADR 冻结。优先用现有 `Map`、函数和条件组合,不要默认增加新异常语法。 + +--- +## 4. 系统交互不是“宿主函数集合”,而是稳定的资源操作边界 + +### 4.1 一次性澄清“彻底不依赖 Rust”的目标 + +**可以达到**:`print`、`input`、`read_line`、UTF-8、CSV/JSON、排序、哈希表、文件复制、HTTP/1.1 解析等能力由 `.mq.md` 组合实现,并能用**未重新编译**的 CLI 执行。其核心算法不得依赖 Rust 提供专门的对应 API。 + +**有前提地达到**:网络客户端、进程管理、目录遍历、定时操作等:当**已授权且已有的资源提供者**暴露了相应的不可再分操作,Marqdo 可自行组合高级功能。若目标环境没有 `socket connect`、`directory enumerate` 或时钟/熵源,任意次数内存 load/store 都不能凭空获得这些能力。 + +**不可能按字面实现**:从虚拟内存中的字节计算“推导”出所有操作系统/硬件功能,或在所有平台上不经机器指令、系统 ABI 与设备驱动就完成实际 I/O。即使未来自举成本机二进制,系统边界仍然存在,只是不一定经过 Rust 解释器。 + +**本 RFC 的真实目标不是“没有 ABI”,而是“**`ABI` 的扩张速率与标准库功能数量脱钩**”**。普通函数增长只增加 `.mq.md`;新增真正的底层设备能力最多增加独立、可注册的端点实现,而不污染语言语法、值计算和普通模块解析。 + +### 4.2 资源不是 Memory,设备寄存器也不是普通 byte[] + +执行状态中保留 `ResourceTable`,其句柄由运行时/可信 Provider 签发,带实例、generation、实际权限和关闭状态。资源表条目必须覆盖:资源类别、Provider 身份、协议版本、已授予操作、生命周期、预算绑定、审计上下文。 + +逻辑上仅需要三种动作:**取得已授权资源 → 调用其承诺的操作 → 释放资源**。实际可支持的操作由该资源实现的**版本化契约**列举;不能把 `resource.call(handle, "anything", bytes)` 当成权限绕过的万能后门。 + +通用流类资源的契约可描述为: + +```text +Capability.lookup(name, request) -> GrantedResource | Denied | Unavailable +Stream.read(handle, memory, offset, length) -> {count, eof} | Error +Stream.write(handle, memory, offset, length) -> {count} | Error +Resource.close(handle) -> Ok | Error +``` + +**这些是接口契约,不要求 `Stream`、`Resource` 成为语言关键字。** 按兼容策略,可以用普通模块函数包装有权限的系统端点。Provider 通过已存在的 callable 解析和受控注册进入程序;具体底层调用不通过用户伪造字符串/句柄获得新权限。 + +- 读:`count` 可为 `0..length`,`eof` 与 `count=0` 不必等价;非阻塞资源可报告 `would_block/pending`。 +- 写:可能仅写入前缀,调用者循环处理余量;中途错误可能发生在**已经完成部分外部效应**之后,不能假装事务回滚。 +- Memory 是传输缓冲区,但 Provider 只能在**本次调用期间**借用受验证的缓冲区,不能持有可随时被 `grow` 搬迁的宿主地址;首版可采用安全复制以降低复杂度。 +- 关闭后 generation 必须失效;活动资源的复制/传参规则明确为共享句柄身份,但“谁负责 close”应定义为资源所有权规则,不能被运行时 `Value::clone` 意外复制底层 OS fd。 +- 设备专有接口可注册独立的命名协议(如 `device.control.v1`),但不应伪装为可随意解引用的内存。MMIO、DMA、QPU 需要专门的权限和排序/一致性约束。 +- 基本 stdout/stdin/stderr 不是任意可构造路径,须由运行环境作为授予的预打开资源显式提供;防止应用拿整数 1 就伪造 stdout 权能。 + +### 4.3 避免“只有 open/read/write 就宣称一切系统能力具备”的错误 + +资源类的基本接口是**模式**,不是所有系统能力的完整清单: + +| 外部原始能力 | 可否仅从已授权字节流组合? | 语言/平台的明确边界 | +|---|---|---| +| 行输入、UTF-8、缓冲、文本打印 | **可** | 需要 stdin/stdout 流端点;解析由 Marqdo 实现 | +| 文件内二进制格式 | **可** | 需要文件流读取/写入 | +| 任意文件打开、目录列举、重命名、元数据、文件映射 | **不可完全由单一流推导** | 需要有权限的文件系统端点与对应操作 | +| TCP 报文构建、HTTP/1.1 解析 | **可** | 需要已连接 TCP 字节流;建立连接本身需网络端点 | +| DNS 解析、网络连接、监听、TLS | **不由普通文件流自然产生** | DNS/Socket/安全熵/加密、证书信任等由服务/库/端点组合;安全 TLS 不宜未经审计自行实现 | +| 时间、计时器、安全随机源 | **不可** | 必须由可信平台端点提供可信时钟/熵源 | +| 进程创建、信号、环境变量、终端属性 | **不可** | 需要明确平台能力,权限逐项授权 | +| 共享内存原子操作、内存顺序 | **普通字节读写不充分** | 并行计算语义另行 ADR,不允许在无同步下假设原子性 | +| GPU 计算、QPU 任务 | **不可** | 设备执行后端/驱动契约;Qubit 不等于普通内存指针 | + +**审查规则**:开发者申请增加高级函数时,先展示它调用的既有低层端点和 `.mq.md` 算法;只有**缺失真正不可由现有原语获得的外部效应**,才允许提出平台 Provider 扩展。一个新的算法不得绕过此规则要求新的 Rust Builtin。 + +### 4.4 Provider 扩展:复用现有 ABI/调用解析,不再造第二个生态 + +当前存在 `CallableRegistry`、外部可调用描述符、C ABI v1、Worker、`PluginHandle`、CapabilityBroker。它们应被利用为**可注册端点的技术基础**。本 RFC 提议增补其资源语义和能力化调用约束,而非新建平行的 `SystemProviderRegistryV2`、`SyscallHost`、`NativeToolKernel` 三套分派。 + +关键设计要求: + +1. Provider 发布一个**可枚举的、版本化的能力协议**,包括资源类型、参数/返回形态、需要的 grant、最大传输大小、错误码、关闭语义。 +2. 导入 `.mq.md` 包不会隐式加载/授权设备 Provider;加载与授权分开。 +3. 普通 Marqdo 模块调用在解析时先解析模块函数,再到受控外部描述符;对兼容保留的旧 Builtin 单独处理名称优先级。 +4. 端点 ABI 不得直接跨边界传递 Rust `Vec` 的内存地址,也不能通过可伪造 JSON 生成活句柄;字节传输与 handle generation 需有明确编码与所有权。 +5. 若当前 ABI 的 JSON Wire 无法高效承载 Memory→Stream 数据,应允许**向现有 ABI 版本化扩展数据缓冲区传输协议**或使用复制路径;不要求第一天零拷贝,也不允许未经审查使用裸宿主指针。 +6. Native 插件只有明确可信时才能在进程内运行;Worker 隔离崩溃不等于 OS 安全沙箱;现有 strict L1 的拒绝策略不因注册了新端点而自动改变。 +7. 新端点实现在适配器/Provider 仓库升级即可上线,**不需要修改表达式解析器、`Value` 主类型或普通 Runtime 调用分派**(前提是 Resource ABI 版本可兼容)。 + +**重要的实际限制**:新设备/OS 原语有时仍需编写 Rust/C/系统 SDK 代码;若希望连这些原语也用 `.mq.md` 实现,未来只能通过受控的原生编译/FFI 系统接口探索。那不再是“纯计算沙盒不调用外部”,且要接受平台专用、安全与可验证性的代价。 + +### 4.5 安全模式和 I/O 兼容规则 + +旧 `pure` 模式已有 console 输出例外(在当前版本中为明确有权限的 stdout/缓冲输出),不应在迁移过程中暗中撤销。目标安全行为: + +| 模式 | 目标支持 | 强制约束 | +|---|---|---| +| pure | 纯计算、受资源预算的 Memory/值、兼容的已授权输出缓冲 | 不提供任意文件/网络/进程/插件权限;已有 stdout 的兼容例外单独标记 | +| project | 明确授权范围内的标准文件、控制台及经过支持矩阵列出的网络资源 | grant 与实际 Provider 操作相交;项目根目录保护,拒绝路径/域名等越权 | +| strict | 保留已验证的隔离功能 | 未验证的资源端点均 fail-closed,严禁把 project 的新功能直接当作 strict 可用 | + +为了验收“print 已迁移”,不仅要打印正常,还须验证不授予 console 的权限拒绝、输出预算、trace、stderr/stdout 分离、CRLF/换行兼容、部分输出和中文 UTF-8 字节一致性。 + +--- + +## 5. 内置能力分类与迁移总表 + +### 5.1 分类规则(以功能性质,而不是实现语言) + +- **C / Computational Core**:没有它,`.mq.md` 无法表达基本状态转换或构造相应运行时值;其语义在语言规范中冻结。 +- **S / Standard Library**:仅使用 C、现有 Marqdo 模块和已授权 Resource 端点即可实现,发布独立 `.mq.md` 代码。 +- **E / Environment Endpoint**:不可由程序纯计算产生的外部效应,交给 Resource/Provider 端点;其**适配器代码**可用 Rust/C/平台 SDK 实现。 +- **O / Optional Accelerator**:语义上有可用的 Marqdo 参考实现,但经真实测量证明性能不足,才允许可替换优化实现;不得成为唯一实现。 +- **D / Developer Tooling**:编译器、诊断、源码导航、代码检查和包安装工具的宿主实现,不属于运行时标准库函数,不要求迁入 `.mq.md`。 + +分类时必须先展示:输入/输出/错误、是否可观测外部效应、依赖哪个 C 原语、能否实现同语义 `.mq.md` 版本、性能与兼容的验收方法。 + +### 5.2 现有能力迁移矩阵 + +| 现有/相关能力 | 目标类别 | 迁移决策 | 必要条件及注意事项 | +|---|---|---|---| +| 整数/浮点/布尔运算、比较、`and/or`、函数/循环 | C | **保留** | 解释器自身计算,不外包给标准库 | +| `len/长度` | C + S | 基础集合长度观察留 C;高级 `count`/统计放 S | 内核不得泄露内部 Rust 容器作为 API | +| `type/类型` | C | 保留最小类型标识 | 避免不可信反射到真实宿主对象 | +| `str/文本` | C 基本值转换 + S 复合值格式化 | 保留现有基本值转换的兼容语义;新增标准库 `format` | 现有 `str(List/Map/Table)` **不是**可用接口,不能用它替代 `Value::display()` | +| `int/整数`, `float/小数` | C/经验证可 S 化的解析 | 数值语义和精确舍入留 C;高阶解析/format 放 S | Unicode、整数精度、NaN/Inf、负零、超大数兼容 | +| `print/打印` | S,兼容入口 | **迁至标准库**;Legacy Builtin 过渡期代理 | 必须先具备全类型格式化、UTF-8、stdout 流调用和模块自动加载 | +| `table()/表()` | 值语义 C + 表格构造/验证 S | 分阶段迁移;避免一次删除 `Value::Table` | 必须保留表头/行宽、空表、格式化与相等性的兼容语义 | +| `fs.read` | S + E | 高级文本读入迁至 `fs.read_text`;文件流打开/字节读保留 E | EOF、部分读、编码错误、安全路径和句柄关闭 | +| `fs.write` | S + E | 高级文本写入迁至 `fs.write_text`;文件流字节写保留 E | 原子文件替换语义需额外资源操作,不能仅凭流承诺 | +| `ref()` | C / 文档运行时 | **保留**,不等于内存指针 | 静态知识地址与动态 Memory 身份不同 | +| `math.parse/eval/substitute` | S + O/保留兼容端点 | 分阶段实施 `.mq.md` 可用数学表达和替换;原生路径可留优化 | 不得假定复杂符号计算很快即可迁入;以语义覆盖优先 | +| `math.simplify/differentiate` | S 算法 + 可选 O | 不应把未实现功能变成默认 Rust 承诺 | 当前部分操作标记 unsupported,迁移要以真实通过用例为准 | +| `foreign.run/call` | E / 兼容互操作 | **保留为可选能力**,不得成为默认标准库算法必需品 | pure 与 strict 权限限制仍生效 | +| `plugin.invoke` | E / 资源扩展 | 保留为兼容/受控 Provider 入口,长期统一资源契约 | 不新增逐函数 HostOp;把注册与授权分开 | +| `List` 操作、`Map` 访问 | C 基础结构 + S 算法 | 补齐构造/读取/更新的闭包;排序/filter/merge 等迁 S | 不能靠每新增算法就新增 builtin | +| GFM Table 解析/AST | D + 语言数据契约 | 保留在 Markdown parser;不能退化为 stdlib 自行猜 AST | 标准库只处理解析后逻辑 Table 的运算 | +| GFM Table 查询/关联/过滤/渲染 | S | **迁至独立库** | 使用公共 Table 结构观察与构造能力 | +| 字节缓冲区、分配器、`malloc/free` | S | **Marqdo 自己编写** | 基于 Region,提供可追踪块元数据与空间复用 | +| UTF-8/UTF-16/CSV/JSON/二进制协议 | S(必要时 O) | **Marqdo 自己编写** | 输入字节、输出字节、错误恢复、增长写入均已闭合 | +| 控制台、文件、Socket 字节流 | E | 由可信环境提供端点 | 没有这些端点不能完成物理 I/O | +| 时钟、熵、OS 进程、设备操作 | E(按需) | 独立授权 Provider | 不应伪装成通过内存/寄存器“自然涌现” | +| 编译器/包管理/`inspect`/`check`/Trace | D/C 边界 | 不强制改写为 `.mq.md` | 改工具架构需另立目标,勿混进语言核心迁移 | + +### 5.3 防止重复定义:内核与 stdlib 的公开名称策略 + +内核描述的是**基础操作语义**,标准库通过普通可导入模块定义 `std.io.print`、`std.text.utf8_encode`、`std.table.join` 等。文档中的模块名属于例子,不要求增加命名空间语法。 + +现有 `print/打印` 已是稳定的 Builtin 名称且在 `eval_call` 有优先级,迁入标准库之前需要决定: + +1. 新代码推荐通过 `import` 使用 `std.io.打印` / `std.io.print`(用现有导入映射与模块函数解析)。 +2. 保留 `打印(...)` / `print(...)` 作为兼容别名;解析器由内置分派**转向标准库声明/函数引用**,不得继续执行旧 Rust `Value::display()` 路线。 +3. 如果语法解析优先匹配 Builtin,必须调整**名称解析的兼容分派**,确保不出现同一个名称随机解析到两个实现。记录覆盖/遮蔽规则的 ADR。 +4. 标准库函数必须普通可检查、可导航、可单测,执行追踪需要记录其 `.mq.md` 源 span,而不是全部伪装成黑盒 `HostOp::Print`。 +5. 在完整兼容窗口关闭前,旧 Builtin 可以作为受测试的回退路径,但默认必须优先执行 stdlib,且两条路径的输出比较必须通过。 + +### 5.4 打印:第一个完整迁移的具体拆分 + +当前行为(源码事实):`Builtin 打印`→`HostOp::Print`→`Value::display()`→追加 `\n`→输出预算→缓冲 stdout;当前 `str()` 不支持 `List/Map/Table/Formula/Code/FunctionRef/PluginHandle`。 + +目标行为: + +```text +std.io.print(value) + ├─ std.format.display(value) # 按语言值语义递归格式化 + │ ├─ type / len / index / keys # 最小结构观察(核心) + │ └─ std.numeric.format # 数值与特殊值的稳定字符串表示 + ├─ std.text.utf8_encode(text + "\n") + └─ std.io.write_all(stdout_handle, bytes) + └─ Stream.write(handle, region, offset, count) # 唯一外部动作 +``` + +**迁移不能跳过的细节:** + +- `Value::display()` 旧行为是兼容基准:Int 任意精度、浮点格式(NaN/Inf/负零)、Bool/None/Unit、Text、List、Map、Table、Formula/Code/FunctionRef/PluginHandle。需要逐类型 golden;不允许悄悄泄露 Resource/Memory 真实内部 ID。 +- `std.format.display()` 在 Marqdo 中必须**真能遍历复合值**,不能偷偷调用一个隐藏的 Rust `format_anything()` 将整件事移回 Runtime。 +- 对于 `Float` 的精确十进制最短往返表示,如果完整用 `.mq.md` 编写极其昂贵,允许把**格式转换所需的最小数值原语**留在 C 类,并说明它仅负责数值表示规则、并非 `print()` 本身。应首先有经规范化的可观测结果。 +- `write_all` 处理短写和失败,注意重复重试可能重复输出或产生已写前缀;输出预算应在实际写入数量上计费,且不能借包装函数绕过。对 console 格式化的燃料/内存也必须计费。 +- 标准输出内容与编译器日志、trace、`--json` 通道保持隔离。 +- 首版可支持保持 `print` 完全一致,同时引入 `write_text` / `write_bytes` 作为标准库独立能力;用户未授权写标准输出时,仍必须拒绝。 + +示例(**目标 `.mq.md`,非已实现语法,`std` 对应正常模块导入**): + +````markdown +## 打印 + +先将任何可打印值格式化为文本,再编码为 UTF-8 字节。 + +**显示文字 = format.display(值)** + +**数据 = text.utf8_encode(显示文字 + "\n")** + +> io.write_all(io.stdout(), 数据) +```` + +这些 `format.display`、`text.utf8_encode` 和 `io.write_all` 必须有真实的 `.mq.md` 定义;其最终字节写入依赖已有授权 `Stream.write` 而不是新的 `HostOp::PrintJson` 等。 + +--- +## 6. GFM 表格重构:不是降级为字节,而是从物理表示中解放语义 + +### 6.1 三层必须分开 + +当前 `Table { header: Vec, rows: Vec> }` 的优点是简单、正确地把表格作为 Runtime 值;缺点是将结构和实现绑在一起。新方案要分开: + +| 层 | 应维持的内容 | 允许改变的内容 | +|---|---|---| +| **GFM 源文档** | canonical Markdown AST `table/row/cell`、单元格边界、源码 span、渲染与惰性规则 | parser 内部缓存/索引实现;不能改 GFM 的含义 | +| **Table 逻辑值** | 有序表头、行的有序性、单元格值与可见类型、行宽、相等性/格式化/错误规则 | 如何查询/更新、是否支持共享持久内部结构,由 ADR 决定 | +| **Table 物理存储** | 与逻辑值一致的可观察结果 | `Vec>`、持久向量、列式数值缓冲、按需视图、Memory-backed 数值列 | + +**绝不能**把 Markdown 表格的 `|` 分隔符理解为一个字节内存地址。GFM 源单元格与 Runtime `Memory` 的 byte index 不是同一类地址;纯数字列可以优化为 Memory 背后字节块,但不强迫文本/公式/结构化值也采用该布局。 + +### 6.2 新的 Table 对外抽象(语义,不冻结具体类型名) + +建议保持 `Value::Table` 作为独立的逻辑值类型,以免破坏现有程序的 `type`、`len`、format、错误类别与 wire 结构。其公开接口允许: + +```text +row_count(table) -> Int +column_count(table) -> Int +column_name(table, index) -> Text +cell(table, row, col) -> Value +set_cell(table, row, col, v)-> Table # 产出新逻辑值;原表不被隐式修改 +append_row(table, row) -> Table +``` + +**注意:** 以上是逻辑操作而不是要求六个 Rust Builtin。`row_count/column_name/cell` 属于有必要的最小结构观察;`set_cell/append_row` 应优先用可用的通用 List/Map 值重建规则在 `.mq.md` 中实现。若 `Table` 的内部表示变为封闭、不可遍历的 Rust 结构,就必须在核心提供足够的公开投影操作,供标准库重建逻辑表格。 + +重构必须支持: + +- 零行/零列(按当前规范兼容)、重复列名处理(遵照当前规则,不能随意变更)、Unicode header、行宽验证、单元格类型检查。 +- 表格复制保持旧值语义:`b=a` 后逻辑上修改 `b`,不能让 `a` 在未经显式共享授权的情况下发生可见改变。 +- 表格计算可用 `filter/map/join/group/sort` 等 `.mq.md` 标准库函数组合,不因为每增加一种表格操作就修改 `builtins_impl.rs`。 +- 原始 Markdown 中诸如 `**公式**`、链接和反引号在单元格内仍属于**惰性文档内容**。任何“解析某单元格为可执行表达式”的行为必须在显式调用的解析函数中发生,并遵循独立安全边界。 +- 必须保留源文档的可寻址单位与审计来源;Table 逻辑值的快照与原始 Markdown 文本不是同一件事,不能无授权回写源文件。 + +### 6.3 物理存储的演进步骤 + +**T0 保留 Rust `Vec>` 实现**,先让 Marqdo 程序能够通用地访问、构建、变换 Table。此时即使内部仍是 Rust 的 `Vec`,计算功能已经不再依赖新写专门算法的 Rust。 + +**T1 将 Table 从直接字段访问改为抽象的受控存储接口**(Runtime 内部)。为逻辑 `Table` 定义稳定的访问/构造语义;内部可用共享不可变数据结构或写时复制减少复制成本,不能改变用户可观察语义。 + +**T2 可选 Column Store**:只有已验证的大型数值表场景,才尝试让连续列数据存储在 Memory/Buffer 中。列必须有明确元素类型、编码、行数与缺失值规则;异构列及来源信息仍可由托管值保存。 + +**T3 可选 View**:对列切片、行过滤、投影提供只读逻辑视图;若存在共享可变源存储,必须明确版本/快照/引用有效性,不能无声明就让普通 `Table` 变成共享可变容器。 + +**否决**:为每一种表格表现形式增加 `DataFrame/TableView/ColumnarTable/GFMTable/MemoryTable` 五个并行内置类。只允许明确的逻辑 Table 值与可替换存储实现;专门优化作为标准库/后端能力。 + +### 6.4 表格迁移的验收样例 + +输入 GFM 文档: + +```markdown +| 名称 | 数值 | 说明 | +| --- | ---: | --- | +| 甲 | 10 | `x + 1` | +| 乙 | 20 | **不执行** | +``` + +验收断言: + +1. `marqdo check` 和渲染结果与原文兼容;单元格中 `**不执行**` 仍不执行。 +2. Table 逻辑行数为 2,列顺序固定,`说明` 列内容按当前 GFM 数据值映射规范表示,而非误执行代码。 +3. `.mq.md` 的 `table.filter` 能选择 `数值` 大于某阈值的行,不触发新的 Rust Builtin;后续 `table.join/group` 同理。 +4. 表格值复制与变换不使原表发生副作用;大数据优化后仍保持同样结果。 +5. 对 GFM 表格的来源引用、错误 span、序列化与 `Value::display()` 有 golden 比较,避免丢失文档原本的知识联系。 + +--- + +## 7. 不仅要“能算”,还要证明**可组合性在复杂程序中闭合** + +### 7.1 闭包测试的基本定义 + +设 `K` 为核准后的 Marqdo Core 计算能力,`E` 为已经存在且获授权的 OS Resource 端点,`L(K,E)` 为**仅以 `.mq.md` 编写**、允许调用 `K/E` 的所有高层算法及应用。 + +对于高层能力 `F`,要求提供明确构造: + +```text +F ∈ L(K,E) +``` + +这不是“理论图灵完备”的宣称,而是要求**真实程序可运行**且达到:可恢复错误、足够的结构访问/构造、可检查的权限、可控的内存/时间开销和离线可重现。 + +如 `F` 无法实现,必须归因于以下之一: + +- **K 缺口**:无法组合出所需的计算/存储能力,考虑扩展最小通用原语; +- **E 缺口**:OS 没有向程序授予所需原始能力,新增**独立 Provider/端点**而非新增算法 Builtin; +- **性能缺口**:语义可以实现但实用性能不足,先分析 VM 开销和优化编译器,允许**可替换**的原生优化; +- **表达/人体工程缺口**:需要大量重复样板但语义已足够,优先用 `.mq.md` 高层封装,而非新增关键字; +- **规范/安全缺口**:语义不明确或权限不足,先规定边界而非偷开后门。 + +### 7.2 标准闭包任务矩阵 + +| 测试 | `.mq.md` 必须实现的主体 | 需要的最小 K | 需要的 E | 禁止取巧方式 | +|---|---|---|---|---| +| C01 多种分配器 | bump、first-fit、free-list 复用、合并 | 区域大小/增长/字节索引、整数、循环 | 无 | 用 `std::alloc` 为每块建立特殊 HostOp | +| C02 链表/二叉树/图 | 节点布局、插入/删改/遍历、循环引用 | Region、整数地址、函数/循环 | 无 | 直接调用 Rust `LinkedList`/`petgraph` | +| C03 二进制编解码 | LE/BE u16/u32/u64、变长整数、位域 | 算术、字节读写、可恢复错误 | 无 | 将 `pack/unpack` 全部设为 Rust 内置 | +| C04 Unicode/文本 | UTF-8 编码、解码、非法序列策略 | Text 观察、Memory、整数及文本构造 | 无 | Python codecs/隐藏 Rust UTF8 parser 为唯一实现 | +| C05 JSON/CSV | 动态输入、解析、构造、格式化、错误定位 | 动态 List/Map/Text 构造、索引、错误返回 | 无 | `serde_json` 作为唯一实现 | +| C06 GFM Table 算法 | filter/map/join/sort/group/列投影 | Table 逻辑访问、动态 List/Map 构造 | 无 | 每新操作增加 `HostOp::TableX` | +| C07 自举打印 | 任意允许的 Value→规范文本→UTF-8→输出 | `type`/结构观察/格式化/内存 | 已授权 stdout Stream | 仍用 HostOp::Print 执行 `Value::display()` | +| C08 多轮交互 CLI | 提示、读一行、解析用户输入、循环输出 | 文本/字节/错误、控制流 | stdin、stdout | 特增 Rust `input()` | +| C09 文件复制与格式化 | 分块复制、短读短写、文件格式解析 | Buffer/Memory、循环、结果错误 | 授权文件资源流 | 特增 Rust `copy_file()` | +| C10 HTTP/1.1 客户端核心 | 请求构造、流处理、首部/chunked、连接关闭 | Memory/Text/错误/集合 | 已连接 TCP Stream | Rust HTTP 客户端做全部报文解析 | +| C11 自编简单字节码机 | 虚拟寄存器、指令解码、内存、分支、堆栈 | Memory、整数/位/条件、动态函数 | 无 | 在 Rust 中写整台 VM,Marqdo 只调用 | +| C12 科学计算准备 | 矩阵/稀疏矩阵布局、切片索引、数值迭代 | Region/数值/循环、精确索引 | 无 | 先要求 NumPy/Python 内核才能运行 | + +**加严的反规避规则**:C01–C06、C11–C12 在测试环境中必须禁用 Python、Node、Foreign、动态原生插件以及所有与任务等价的专用 Host Builtin;C07–C10 仅保留声明过的 Resource 底层端点,不得允许平台提供整套高级任务实现。 + +### 7.3 不属于“纯计算闭包”的功能:必须另外审计 + +| 能力 | 为何无法靠 K 单独获得 | 基础交付要求 | +|---|---|---| +| `read_line()` 里的实际按键输入 | 真实数据来自 OS/stdin | stdin 资源端点,用户授权和 EOF 语义 | +| 文件/目录、网络、进程 | OS 对象不在用户的虚拟 Memory 内 | 细粒度的权限资源及平台支持矩阵 | +| HTTPS | TCP 只是字节流,TLS 要协议、安全随机数、证书信任和安全实现 | 可用受审计 TLS Provider 或安全实现,不能以裸 HTTP 解析冒充 | +| 高精度时间与熵 | 不是确定性计算结果 | 时钟/随机资源端点,不允许把伪随机当作加密安全熵 | +| CPU/GPU 大规模并行 | 普通 load/store 不定义数据竞争、原子性与设备执行 | 独立调度/同步或异构后端,另行语义批准 | +| QPU | 量子态不可被普通字节地址任意读取/复制 | 正式量子设备端点及资源限制 | + +### 7.4 “不需要继续写 Rust”的强制证明方式 + +以 **三类不修改核心** 测试作为工程门禁: + +- **扩展算法**:在一个固定的 `marqdo` 可执行文件上,安装新 `.mq.md` `std.table.join`、`std.json.parse`、`std.memory.buddy`,不得重编译 CLI,不调用 Foreign。 +- **扩展系统高层功能**:已有文件、TCP 流端点时,安装新的 `.mq.md` 文件格式解析器、网络协议解析器,只增加标准库,不增加 Provider。 +- **扩展真正系统原始功能**:引入新的测试 Resource Provider(如具备 `read/write` 的虚拟终端设备),不得改 Runtime/Expression/HostOp;**允许**单独实现和安装 Provider。其权限与版本协议必须能被现有 CLI 识别。 + +前两项失败意味着内核/标准库尚不充分。第三项失败意味着系统扩展架构不稳固。三者不是一个问题,不得用某一项通过代替另外两项。 + +### 7.5 与 Python/Rust 实现的性能比较应如何解读 + +该 RFC 不承诺初版字节解释器的吞吐能比肩优化后的 Rust/C。性能验收需要分成两个层次: + +**语义与正确性优先门禁**:同样的测试输入、确定性结果、正确内存边界、正确错误/输出与算法复杂度表现;不得用“Rust 版本很快”代替 Marqdo 版本通过。 + +**工程性能评估**:记录 `wall_time / peak_RSS / CPU / fuel / bytes_moved / allocations / copy_count`,与 Rust 原生和其他参考语言比较。初期不设无依据的“必须达到 80% 原生速度”的硬指标;若性能瓶颈来自逐字节 Rust↔Marqdo 调用/隐藏复制,先优化执行路径、批量传输、字节码/JIT,再讨论原生加速。任何 O(1) 原语若退化为 O(n) 隐式全表复制,必须单独报告。 + +### 7.6 三种验收样例(目标伪代码,非现有可运行程序) + +**A. 用 Marqdo 实现堆分配器而不调用 Rust `malloc` 算法**: + +````markdown +## 分配 + +内存管理器保存 region、next、free-list 等元数据。 + +1. `存在足够的空闲块` + > 复用空闲块并更新元数据 +2. `没有足够的空闲块` + > 需要时增长(region) + > 更新区域中的尾指针 + +*分配得到的偏移量* +```` + +**B. 打印与读取由标准库完成**: + +````markdown +## main + +**输入 = io.read_line()** + +**输出 = "收到:" + 输入** + +> io.print(输出) +```` + +**C. 同一个 GFM 表格运行不同的 `.mq.md` 算法**: + +````markdown +## 分析数据 + +| 名称 | 分数 | +| --- | ---: | +| 甲 | 87 | +| 乙 | 92 | + +表格是数据,不在渲染时执行;函数通过显式数据引用取得逻辑 Table。 + +**结果 = table.filter(输入表, 过滤函数)** + +*结果* +```` + +B/C 中的函数名、输入参数和文档表格访问均仅为目标接口示意,须在真正实现时遵守冻结的 `##` 函数声明、参数推断/导入和文档资源寻址规则。**不能为了示例好看就把尚不存在的语法当成现实。** + +--- +## 8. 重构目标架构、仓库组织与实际代码改动 + +### 8.1 独立标准库的仓库和发行方式 + +推荐另建 `cflmy/marqdo-stdlib`,**分仓开发**,但**官方发行版必须搭载可离线使用的兼容版本**。避免用户刚安装 CLI 就因为网络不可用而无法调用 `打印`。 + +```text +marqdo-cli/ # 语言、执行器、系统边界与工具 + crates/marqdo-markdown/ # GFM AST + crates/marqdo-expression/ # expression / Place + crates/marqdo-value/ # Value、MemoryHandle、ResourceHandle + crates/marqdo-ir/ # 语义 IR 与 callable 描述 + crates/marqdo-runtime/ # MemoryStore、解释器、预算/诊断 + crates/marqdo-host/ # Capability 与底层流/资源适配 + crates/marqdo-plugin-abi/ # 版本化 Provider ABI(复用现有) + crates/marqdo-package/ # 锁定的标准库依赖/版本兼容 + crates/cli/ # run/call/check/inspect 与打包 + spec/ # 语义、错误、权限、wire 的规范 + docs/architecture/ # RFC-CM-0001/0002 + ADR + +marqdo-stdlib/ # 全新独立仓库(尚不存在,拟建) + core/ # Result、通用结构处理 + memory/ # allocator、指针编码、arena + collections/ # list、map、tree、graph、queue + numeric/ # 整数位操作/数值算法 + text/ # UTF-8、格式化、缓冲、parser + io/ # stdin/out、read_line、write_all + fs/ # 文件流读写与文件格式辅助 + data/ # csv、json、GFM Table 变换 + net/ # 可用 socket 上的字节协议算法 + examples/ + tests/ + compatibility/ # Runtime 核心能力要求和版本约束 +``` + +标准库中的 `core` 指**标准库基础算法**,不是第二个 Runtime;不能定义与 CLI 冲突的 `Value`、`MemoryHandle` 或安全许可规则。 + +#### 发行契约 + +- CLI 发布包随附 `marqdo-stdlib` 的固定 digest/版本,可离线解析,不需要联网、Git 可用或隐式权限。 +- 锁文件记录 stdlib 实例、内容摘要和要求的核心能力版本;不能只校验“模块路径存在”。 +- 分仓主干可以更快演进,但官方发行组合由兼容 CI 验证,升级库不强迫重编译 CLI;库若要求新的核心原语,版本范围必须准确拒绝旧运行时。 +- 用户自定义标准库版本可采用显式导入/锁定;默认兼容别名解析可使用随发行包固定的标准库,不得让外部项目文件偷偷覆盖 `std.io.print`。 +- 严格保留文档模块的源文件与 span,`inspect`/调试器能看到标准库真实 `.mq.md` 调用堆栈。 + +### 8.2 crate 级最小改动计划 + +| 改动批次 | 源码位置 | 允许修改的内容 | 明确禁止 | +|---|---|---|---| +| A | `marqdo-expression` | Place 左值语法/AST、span 与错误定位 | 为 `print/table/malloc` 增加关键字 | +| B | `marqdo-frontend`, `marqdo-ir` | 映射 `AssignTarget`,声明基础内存操作、受控资源操作元数据 | 新的 Markdown 特殊块解释器 | +| C | `marqdo-value` | `MemoryHandle`、`ResourceHandle` 的语言值外壳;逻辑 Table API | 公开宿主 `*mut u8` 或 `Vec` 地址 | +| D | `marqdo-runtime` | `MemoryStore`、有界索引读写、增长、调用/返回值生命周期、异常/预算/trace;最小结构操作 | 链表、CSV、JSON、HTTP、完整 malloc 等专项算法 | +| E | `marqdo-host` | 将 OS 字节 I/O 从 `Print` 分离;能力检查与 Stream 适配;复用 Provider | 每新增一个高层 API 增加一个 `HostOp` | +| F | `marqdo-plugin-abi` / `CallableRegistry` | 在确有需要时增补资源协议、缓冲区交换及版本元数据 | 平行重建第二套插件系统 | +| G | `marqdo-package`, `cli` | bundled stdlib、模块解析优先级、离线发版、兼容链接与显示 | 在编译/导入时默默授予额外 OS 权限 | +| H | `spec/*`、tests | 冻结新核心语义、能力矩阵、wire、AST golden、跨平台证据 | 只改实现不改规范与测试 | + +### 8.3 Runtime 的抽象状态与资源预算 + +提议以一次 `RunContext` 表示执行范围: + +```text +RunContext { + program_ir + frames # 现有调用帧、局部变量与中间值 + memory_store # region_id -> {length, max_length, bytes, generation} + resource_table # resource_id -> {provider, rights, state, generation} + capabilities # 有效 grant(非 manifest 的请求本身) + budgets # fuel, memory, stack, output, I/O, deadlines + trace_context # source spans、语义事件、资源调用审计 +} +``` + +以上为逻辑组织方式,不要求把当前 Rust 代码改成一个巨大 `RunContext` 结构体。重点是所有功能在执行时**必须共享这些约束**,不能让标准库通过另一条内部通道绕过预算或权限。 + +内存预算至少区分虚拟区域**声明的最大容量**和**已提交字节**;对于物理稀疏页的实现要明示计费规则。程序要求增长时,任何预算失败都不能产生半增长状态。 + +### 8.4 需要重新设计的 `HostOp` 兼容窗口 + +迁移不能从一开始就删除 `HostOp::Print` / `FsRead` / `FsWrite`。建议: + +```text +旧 Marqdo 程序 新 Marqdo 程序 + print(value) std.io.print(value) + │ │ + │(compat alias) │ + └─────────────┬─────────────┘ + ▼ + 标准库函数实现 + ▼ + resource.stdout / Stream.write + ▼ + CapabilityHost/OS +``` + +与 Rust 路线并存时要显式测试以下阶段: + +- **Compat-A**:旧 Builtin 仍是默认,新的 `std.io.print` 通过显式导入调用;运行差异比对。 +- **Compat-B**:旧 `print` 别名路由到 stdlib(可通过调用注册表映射到模块函数),旧 Rust 实现只用于受限对照测试与迁移回退;确保签名、返回 `Unit`、异常与权限兼容。 +- **Compat-C**:所有端到端测试通过后,从公开 Builtin 实现中移除 `Print` 特例(是否保留过渡兼容名由 ADR/版本策略决定)。 + +**审查红线**:最终的 `打印` 调用栈不得再落到 Rust `Value::display()` 全量格式化;stdout 的物理字节写入保留在系统边界,这是正常且必要的。 + +### 8.5 异常和输出的执行语义 + +新的标准库会调用大量可能失败的功能。应明示: + +- `read` 可能短读、EOF、暂不可读;`write` 可能短写,部分写入不能假装没有发生。 +- `close` 的失败和二次关闭如何表现、是否自动释放、程序退出如何关闭未关闭资源。 +- 错误由底层端点返回到 `.mq.md` 的结构化值,用户能在普通分支里处理;禁止“Rust 遇到 OS 错误自动终止整个解释器”成为常态。 +- 输出预算在底层 Stream 中计费,不能因为换成 `std.io.write_all` 就被绕过;错误与权限审计记录在独立通道。 +- 宿主 I/O 引起的非确定性不能被误标记为纯计算可重放;需要 record/replay 时应记录输入字节摘要/授权信息与实际读取结果,遵守敏感数据日志政策。 + +--- + +## 9. 实施路线:每个阶段都有阻断门禁,不允许无限制扩张 + +整个迁移**采用纵向闭环**。只完成“抽象层设计”而没有真实 `.mq.md` 端到端应用,不算完成。 + +### P0:冻结旧语义与精确基线 + +**工作**: + +- 在最新代码提交上确认旧 AST golden、数值和 GFM Table 兼容用例、 `print` 全类型输出、Host 权限/预算/Trace、std::fs 行为、安全矩阵和锁文件。 +- 对本 RFC 与 RFC-CM-0001 的所有用户可观察变化开 ADR,明确哪些是向后兼容、哪些必须升级语言语义版本。 +- 确保现有 CI/发布候选状态不被未完成的计算重构污染;新功能独立分支,稳定发布线路独立维护。 + +**通过**:可运行基准套件有可复制命令、固定 SHA、平台、预期结果和失败输出;规范中所有修改点列出。 + +### P1:Memory 与位置赋值的单一闭环 + +**工作**: + +- `Memory` 有身份、创建/大小/增长、单字节读取和写入;Memory 值共享而 List/Map/Table 不改变既有值语义。 +- `AssignTarget::Index` 仅接受合法的 Memory;其他目标失败;大型/负地址/溢出、0 字节区域、边界、增长失败明确诊断。 +- Memory 不跨线协议成为可伪造句柄;运行时负责释放与配额;trace 来源保留。 + +**通过**:区域隔离、别名、内存增长、回收、序列化拒绝、fuel/预算/跨函数传递及旧索引语义测试全部通过;不存在新增高级数据结构 HostOp。 + +### P2:证明通用值与存储共同形成计算闭包 + +**工作**: + +- 用 `.mq.md` 写 bump allocator → free-list allocator → 链表/树/图、字节 codec。 +- 补齐最小的 List/Map/Text/Table 结构观察、动态构造和可恢复错误;不得为了实现 JSON、Table 或 print 每次再增加一个任务特化 Rust 函数。 +- 用 `.mq.md` 实现 UTF-8、JSON/CSV 的至少一种完整解析/生成路径,并能处理恶意/无效输入。 + +**通过**:C01–C06、C11 在禁止 Foreign/Plugin 的测试环境里通过;动态数组不靠固定长度源代码;没有重新编译 Rust 才能新增算法的步骤。 + +### P3:标准资源接口与真正的 I/O 闭环 + +**工作**: + +- 先在 `project` 模式为已授权 stdin/stdout/文件实现有界字节流,保留原有 console 权限和输出预算。 +- 明确 ResourceHandle 生命周期、参数校验、EOF/partial/error 和 Provider 注册协议;已有 ABI 能用则复用,仅在确实无法正确传递字节/句柄时升级 ABI。 +- 实现 `.mq.md` `io.read_line/write_all/utf8/print` 的完整调用链,不依赖 Rust `Input`/`Print` 专用操作。 + +**通过**:C07–C09 在不允许 Host 高级 API 的测试环境通过;虚拟测试 Provider 不改 `HostOp` 就能实现 Stream;纯模式无新增权限,strict 对未验证功能 fail-closed。 + +### P4:GFM Table 的内部语义与标准库迁移 + +**工作**: + +- 逻辑值访问/构造接口稳定化;保持 `Value::Table` 语义兼容和 GFM AST 惰性。 +- `.mq.md` 实现表格过滤、投影、排序、关联、聚合、转 GFM 文本;必要时将物理行存储抽象化,但不强制使用 Memory。 +- 仅在证明存在明确性能收益时试验 Column Store/Memory-backed 数值列。 + +**通过**:C06 完整、行列的语义/错误/格式化兼容;跨实现存储测试结果一致;零 Rust 专项 Table 算法;旧 AST golden 不变。 + +### P5:外部可扩展性与完整系统构造 + +**工作**: + +- 接入经授权的 TCP Stream / 目录操作等经过支持矩阵定义的端点;用 `.mq.md` 写 HTTP 请求、响应与 chunked 解析(**不假装 HTTPS/TLS 已经安全实现**)。 +- 引入第三方测试 Provider,例如 mock sensor 或虚拟流设备,证明无需修改解释器/HostOp 就可扩展。 +- 在原有 package/lock 上实现 stdlib 单独版本与离线捆绑,补充模块名称遮蔽和兼容门禁。 + +**通过**:C10 与新增 Provider 测试通过;新高层系统功能只提交 `.mq.md` 文件、测试和文档;不修改 `crates/marqdo-runtime` / `crates/marqdo-host`。 + +### P6:独立性能与科学计算能力(在 P0–P5 稳定后) + +**工作**: + +- 收集 profile 和燃料、内存、真实吞吐证据;定位逐字节解释器开销与结构复制成本。 +- 在不改变公开语义前提下尝试字节码解释器、宽度化字节操作、区域高效复制/视图、CPU 数值后端;并行原子/同步另开独立 ADR。 +- 量子设备、GPU 应通过明确的资源/编译后端协议接入,不把 Qubit 当作普通 Memory,也不声称会自然获得加速。 + +**通过**:优化实现与参考解释器对确定性用例一致;性能和资源预算可复现实验,不能为了速度静默改语义或绕过权限。 + +### 9.1 为什么严格按这个顺序 + +- 没有 P1,无法真正用 Marqdo 构造低层结构; +- 没有 P2,直接迁移 print/table 只会让标准库重新求助 Rust; +- 没有 P3,`打印` 迁移不完整,因为它依赖真实 stdout; +- 没有 P4,Table 只是把旧内核包了一层新 API; +- 没有 P5,无法证明“新系统高级能力不修改 Host”; +- P6 过早进入,会把基础语义尚未稳定的问题掩藏在编译器和硬件后端复杂度中。 + +**在任一门禁失败时不向后推进并行的大规模迁移**,而是定位到底缺少 C、E、性能还是可读性,再对现有设计做**最小修正**。 + +--- + +## 10. 全量验收规范:CI 必须能自动判断“完成” + +### 10.1 规范兼容(强制阻断) + +| 编号 | 测试 | 硬性通过标准 | +|---|---|---| +| A01 | 原始 Markdown AST golden | 普通链接、表格、段落、标题与内联 CodeSpan 的 AST 映射不变;新增 Place 仅在合法执行负载中解析 | +| A02 | 原始值语义 | Int/Float/Text/List/Map/Table、调用、拷贝、数值错误与原规范等价 | +| A03 | Memory 区域别名 | `a=m` 与 `b=m` 改动同一内存;不同区域相同偏移互不影响 | +| A04 | Memory 增长与边界 | 成功零初始化、保留旧字节、失败原子回滚;极大 BigInt/负索引/越界拒绝 | +| A05 | 句柄与线协议安全 | 伪造/陈旧的 Memory/Resource 句柄不可使用、不可跨运行盲用,错误不 panic | +| A06 | Place 赋值合法性 | 仅目标允许的位置可写;表达式基址/索引/RHS 求值一次且顺序一致;不悄悄改变 List 值语义 | +| A07 | 错误恢复 | parser/IO 的可恢复失败可被 `.mq.md` 处理;fatal 错误仍有 span 与稳定错误类别 | + +### 10.2 标准库独立性(强制阻断) + +| 编号 | 测试 | 硬性通过标准 | +|---|---|---| +| B01 | 分配器 | `.mq.md` 成功实现 allocate/free/reuse,碎片/合并/重复 free 情况可测试,不需 Rust `malloc` 特例 | +| B02 | 复杂数据结构 | 链表、树、图、散列表/其他自实现容器能动态扩展、插入、删除、遍历 | +| B03 | 二进制处理 | 字节序、整数编码、畸形字节序列、增长、越界处理均在 `.mq.md` 中实现 | +| B04 | UTF-8/JSON/CSV | 复杂数据可解析与序列化,错误可被 `.mq.md` 用户程序捕获 | +| B05 | 打印格式化兼容 | `print` 的所有旧 Value golden 输出一致;新资源句柄不泄露敏感身份 | +| B06 | 表格语义/操作 | GFM 逻辑值可 filter/join/group,原表不被变更,来源与格式化无回归 | +| B07 | 真正无宿主算法 | 固定 CLI 二进制、禁用 Python/Node/Foreign/native plugin,新增 `.mq.md` 实现 C01–C06/C11/C12 的选定用例 | + +### 10.3 系统交互和扩展(强制阻断) + +| 编号 | 测试 | 硬性通过标准 | +|---|---|---| +| S01 | stdin/stdout 多轮交互 | 使用 `.mq.md` 读行、编码、打印,支持中文、EOF、空行、长行、无换行尾部 | +| S02 | 短读短写 | 强制每次最多传输 1–3 字节,标准库能完成任务,无丢失/重复/无限循环 | +| S03 | 文件字节流 | 授权路径可读写,拒绝路径穿越/未授权路径,关闭/异常正确 | +| S04 | Provider 扩展 | 新独立 Provider 通过现有 ABI 演进的契约进入,无须修改 Runtime/HostOp/重编 CLI | +| S05 | 无 grant 失败 | Resource 没有授权则无外部效果,明确错误及审计;manifest 声明不能等价于 grant | +| S06 | strict 安全 | 未经真实 OS 隔离验证的 I/O Provider 在 strict 中拒绝,不能静默降级到 project | +| S07 | print 真实迁移 | 通过 trace/调用信息证明确实进入 `.mq.md` format/write_all,而非隐藏旧 `HostOp::Print` 逻辑 | +| S08 | 资源预算与渠道 | 输出限额、内存限额、fuel、stdout/stderr/JSON 分离、一致的错误码有效 | + +### 10.4 发布、性能与知识追踪(强制阻断) + +| 编号 | 测试 | 硬性通过标准 | +|---|---|---| +| R01 | 离线标准库 | clean environment 安装官方 CLI 后,不联网可调用默认标准库,且记录 stdlib digest | +| R02 | 独立升级 | 新 `.mq.md` stdlib 在兼容的旧 CLI 二进制中运行;不兼容的版本清楚拒绝 | +| R03 | 平台矩阵 | Linux/macOS/Windows 的可用功能与拒绝路径有可复现证据;未支持的平台功能明确 unavailable | +| R04 | 跨实现等价 | Table 不同物理存储/Memory 实现的结果、错误、输出、span 与原规范一致 | +| R05 | 性能画像 | 可复现记录字节吞吐、GC/内存占用、调用次数、fuel、表格复制成本;不隐瞒严重退化 | +| R06 | 知识溯源 | `inspect`、trace、SourceSpan 显示 stdlib 文件调用及 Memory 语义事件;GFM 表格/文档链接不意外执行 | + +### 10.5 CI 的“冻结宿主”实验 + +必须至少有一条专门流水线,其构建后先**冻结 CLI 可执行文件哈希**,之后执行: + +```text +build CLI once → checksum(Marqdo binary) = H + │ + ├── 在独立 stdlib 仓库新增算法模块 + ├── 仅修改 .mq.md、manifest/lock、测试与文档 + ├── 禁用 foreign / native plugins / task-specific host builtins + ├── 运行算法闭包与 I/O 测试(仅授予所需底层端点) + └── assert checksum(Marqdo binary) == H +``` + +测试应同时通过静态检查(是否新增 Rust Builtin/HostOp、是否调用 Foreign)和动态审计(是否偷偷进入专用 Rust 算法路径)。不能只看最终输出是否正确;否则一个隐藏的 Rust `json.parse()` 也可能蒙混过关。 + +### 10.6 负面验收与安全边界 + +- 动态分配器可以被恶意/错误程序写坏**同一区域中的其他块**;这属于明确的裸内存风险,不能标为“被沙盒完全防护”。 +- `memory[huge_integer]`、负索引、非法字节、未授权网络/文件、伪造句柄、关闭后使用:均不得崩溃宿主、越权访问其他区域或被吞成无错误结果。 +- `Resource` 返回的敏感数据不能无条件完整落盘为 Trace;Trace 默认只写操作类型、授权、偏移、字节数及必要摘要。 +- native 插件误配置与 Worker 的权限边界应分别验证;普通 Worker 不是 OS 隔离,未满足 strict L1 的实际隔离要求必须拒绝。 +- 内存增长后 Provider 不得继续使用旧 `Vec` 的真实地址;任何可能有别名的缓冲借用必须先复制或正确固定生命周期。 +- 库函数可以 fail-fast,但不能无视已经完成的部分外部效应;正常的解析失败要有可恢复结果。 + +--- +## 11. 需要做出的正式决策(ADR 清单) + +**本 RFC 是推荐架构,不应让开发者在没有决策记录的情况下自行猜测。** 下列事项需要分别冻结;这些 ADR 的目的恰恰是防止将来凭方便临时加入越来越多的 Rust Builtin。 + +| ADR | 待冻结问题 | 本 RFC 推荐决定 | 重要性 | +|---|---|---|---| +| CM-A01 | Memory 区域身份与生命周期 | 一次运行内不可伪造句柄,复制保留共享身份;区域运行结束释放;内部块由 stdlib 管理 | **阻断 P1** | +| CM-A02 | 内存大小、增长与预算 | 多区域、按字节寻址,64 KiB 候选逻辑页,增长失败不修改原内存 | **阻断 P1** | +| CM-A03 | 位置赋值与求值顺序 | Local 重绑定、Memory Index 可写;普通 List/Map/Table 原语义不变 | **阻断 P1** | +| CM-A04 | 复合值的最小结构契约 | List/Map/Table 的安全遍历和动态构造;支持自举 format/parser | **阻断 P2** | +| CM-A05 | 可恢复错误模型 | 以现有数据值和分支优先表达 Result;明确 EOF/partial/permission/timeout | **阻断 P2/P3** | +| CM-A06 | Stream/Resource 操作协议 | 权限签发句柄、Buffer 读写、可注册端点、短读短写、generation | **阻断 P3** | +| CM-A07 | 标准库模块命名与内置别名 | stdlib 正常模块解析,`print/打印` 有透明兼容转发 | **阻断 P3** | +| CM-A08 | GFM Table 逻辑值与物理存储 | 保留独立 Table 语义/惰性 AST,表格算法迁 stdlib,存储可替换 | **阻断 P4** | +| CM-A09 | Provider ABI 的必要最小升级 | 优先复用现有 ABI v1/Worker/CallableRegistry,按证据扩展字节资源契约 | **阻断 P3/P5** | +| CM-A10 | stdlib 的版本/锁定/离线发版 | 独立仓库,固定随包版本,兼容性与内容摘要受 CI 管理 | **阻断 P5** | +| CM-A11 | strict/proj/pure 的新资源权限 | 维持 fail-closed,不把新 I/O 自动宣布为 strict 可用 | **阻断 P3** | +| CM-A12 | 外部原生能力的未来扩展 | 不开放一般任意 syscall/raw host pointer;设备 Provider 单独授权 | 后续 | +| CM-A13 | 并行共享内存与原子操作 | 当前顺序内存为主;未有原子性规范前不允许共享写无序化 | 后续 | + +**每项 ADR 的模板**应同时列出:非目标、现有行为、候选语义、反例、可组合性证明/反例、兼容策略、权限和错误、性能/预算影响、自动测试、撤回条件。 + +--- + +## 12. 为什么这种架构优于“继续增加 Rust Host 函数” + +### 12.1 可扩展性与“核心稳定”的关系 + +采用本 RFC 后,新增 `std.table.join`、`std.memory.buddy`、`std.io.read_line`、`std.text.utf8_decode`,都只需要标准库提交,**它们的语言执行语义已经由 Core 所定义**。这些算法可在不改 CLI 的环境中升级与单测。 + +继续增加 `HostOp::Xxx` 的路线则恰恰相反:每个新功能需要固定枚举、静态签名、权限、Runtime 分派、Rust 实现、CLI 重发和多平台测试;它的维护成本基本随功能数量线性增长,并强化了语言是“宿主 API 的封装器”的印象。 + +但**最少 Rust 代码并非优化目标**:存在足以保证值、内存、字节、I/O 原子语义的 Rust 实现是正常且必要的。真正需要减少的是“每实现一项高级算法,就必须增加一项独有的宿主实现”。 + +### 12.2 两套存储是否破坏统一性? + +保留 `List/Map/Table` 等托管值,并引入用户可编程字节 Memory,确实形成两个存储领域。这不是完全零成本:需要明确定义值语义、复制和引用、相等性、序列化、资源预算。 + +但强制统一为一个暴露指针的字节堆,会要求所有标准库作者手工编码字符串、任意精度整数、异构表格和垃圾回收根,且会使 Markdown 值与机器布局紧密耦合。**统一应该体现在“共享一套计算与调用语义、公共错误/预算/追踪规则”,不应强制所有东西使用相同字节布局。** + +将来若需要更深层统一,可以先证明同一个抽象存储接口可以拥有不同后端,再谨慎替换实现,而不是现在推倒 `Value` 系统。 + +### 12.3 为什么不暴露真实寄存器或机器码 + +真实寄存器不是跨 CPU 平台可移植的语言值,硬件寄存器调配应该由后端完成。若程序想表达虚拟 CPU 或相关论文算法,允许在 Memory 中**自己构造寄存器文件和指令解释器**,比给语言增设 32 个寄存器关键字更符合扩展性目标。 + +直接注入机器码可以作为经明确授信的可选互操作手段,但它会绕过抽象 Memory 和权限模型的可验证界限,不应变成普通标准库运行的依赖。未来原生编译 Marqdo 本身,则属于后端优化/部署能力,不能反向改变计算语义。 + +### 12.4 面向并行与量子计算的长期优势与明确限制 + +**优势**:显式 Memory 区域、逻辑地址、值计算和透明资源边界,允许描述缓冲区布局、批量数据准备、重复试验、串行解释器参考执行、数据变换;可用于将来多后端的语义一致性测试。 + +**限制**:GPU 的并行 Kernel、设备内存同步以及 QPU 的门/测量语义不由内存读写自动导出。真正的并行共享写需要额外的原子性/调度契约,量子态不可视为可以任意 `mem[i]` 访问的普通字节;这些内容**必须以后端/资源层独立验证,不能悄悄加进 Core**。 + +--- + +## 13. 与 RFC-CM-0001 的继承、修订和增补关系 + +| RFC-CM-0001 结论 | RFC-CM-0002 的处理 | +|---|---| +| 托管值 + 多区域线性内存 + Resource | **完整继承**,明确各自生命周期与共同预算 | +| 内存通过普通索引访问、写入 | **继承**,补充 Place AST、求值与错误规则 | +| 自定义 `malloc/free` 与数据结构放 stdlib | **继承并强化**,以闭包测试禁止 Rust 专项算法捷径 | +| Resource ABI 提供 OS 最后一跳 | **继承并细化**,定义流端点、授权、短读短写与不可伪造句柄 | +| GFM Table 是文档值,不是内存本体 | **继承并扩展**,明确定义逻辑 Table、物理存储、迁移与兼容 | +| 核心保留普通值 | **继承并补齐**,指出 List/Map/Table 的通用构造/遍历是 `print/JSON` 自举前置条件 | +| `print/fs.read/fs.write` 仍在固定 HostOp | **新增迁移**,高层操作移入 stdlib,底层字节端点保留 OS 边界 | +| stdlib 独立仓库、CLI 随包离线提供 | **继承并细化**,建立版本/digest/CI 与兼容别名策略 | +| 验收以分配器、CLI 与系统资源为核心 | **升级为闭包验证**,增加 Table、泛型格式化、UTF-8、JSON、HTTP、虚拟 CPU 等领域 | + +此 RFC 不改变 RFC-CM-0001 的安全非目标,也**不宣称已在仓库实施**。若实施中证明原方案缺少必要机制,应提出精确反例并修改 ADR,不应以“Rust 比较快”直接恢复一条高层 Builtin 后门。 + +--- + +## 14. 开发纪律:每次增加功能前必须填写的审查记录 + +建议把以下检查问题放进 PR 模板(并通过 lint/CI 自动核验能自动核验的部分): + +```text +新增能力名称: +使用场景与可观察语义: +是否仅涉及普通计算/数据结构? +是否需要新的 OS/设备外部效果? +已存在的 Core 原语、Resource 端点是什么? +最小 .mq.md 构造实现(文件链接与测试): +是否新增 Markdown 节点执行语义 / Builtin / HostOp / ABI?为什么无法避免? +是否引入可以被现有功能组合代替的特化原语? +与现有 List/Map/Table、Memory、权限、Trace、错误规则的兼容性: +是否有无 Foreign/无 Native Plugin 的独立运行测试: +若提出原生优化,是否有可替换的 Marqdo 参考实现与性能数据: +平台支持与 strict 隔离证据: +版本与发行兼容: +``` + +**变更批准规则**:仅仅“API 很常用”“Rust 更方便”“某个第三方库已经实现了”“目前性能不太好”,都**不足以单独成为新增语言核心原语的理由**。必须说明现有底层原语的真实语义缺口、不可接受的复杂度或无法弥补的性能瓶颈,并提供可重复的证据。 + +**阻断性的反面信号**: + +- 新增一个 parser/formatter 就修改 `crates/marqdo-runtime/src/builtins_impl.rs`; +- 新增一个业务功能就修改 `HostOp` 枚举; +- 新的“标准库”主体逻辑其实是 Rust ABI 插件,`.mq.md` 只做一行转发; +- 表格迁移后只剩下裸字节 Memory,高级 Table 值无法自然读取或组合; +- 为了方便 I/O,绕过 Resource 权限或让用户传递真实宿主指针; +- 只完成 `memory.new`,不证明用户能够用它写独立的分配器与复杂程序; +- 只有 happy path,未测试虚假句柄、错误、权限、跨平台和部分 I/O; +- 为了看起来“底层”,增加寄存器、机器码或专用硬件关键字而没有计算闭包证据。 + +--- + +## 15. 最终架构裁决建议 + +**建议采纳**: + +1. **语言核心是少量可组合的计算机制**:已有函数/值/控制流 + 通用结构观察/构造 + 可写位置 + 多区域可增长 Memory + 可恢复错误。 +2. **OS 交互通过 Resource 能力边界进行**:已授权流/文件/网络/时钟/设备资源具有版本化契约;高层功能尽量由 `.mq.md` 完成,最后一跳保留可信系统适配器。 +3. **标准库真正独立**:`print/格式化/utf8/io.read_line/fs.*_text/table.*`、自定义内存分配器、JSON/CSV 和其他一般算法优先使用 `.mq.md`;随官方 CLI 捆绑固定兼容版本,同时允许独立发布。 +4. **GFM Table 是逻辑数据和知识文档价值,而非存储指针**:重构内部布局、公开逻辑操作接口、迁移高级算法,但保留 Markdown 惰性和既有值语义。 +5. **迁移需要可证伪的门禁**:冻结 CLI 二进制,不新增 Rust/HostOp,在独立 stdlib 中完成可增长数据结构、泛型打印、文本/表格解析、交互式 CLI、已授权网络协议解析;不满足即不能宣称“自主计算/自主系统扩展闭环完成”。 + +**最终边界声明**: + +> **Marqdo 可以摆脱“每个新功能都依赖 Rust/ABI 现成算法”的开发方式,不能摆脱“任何真实计算都由某个执行环境执行、与操作系统交互必须跨越受控平台边界”的事实。** +> +> 优秀的计算内核不在于直接支持多少个功能,而在于以**精确、有限、可组合**的基本语义,让标准库不断生长,同时不迫使 CLI 的语法、解释器和 Host 随之无界膨胀。 + +本 RFC 建议作为下一阶段开发的**上位约束文档**;首次开发只推进 P0→P2(计算闭包),经验证后推进 P3→P5(系统标准库迁移)。当前 stable/release 收敛与架构试验应当分支隔离,**不得将本 RFC 的未完成代码混入已声明受支持的稳定发行版**。 + +--- + +## 附录 A. 术语表 + +| 术语 | 本 RFC 中的含义 | +|---|---| +| Core | Marqdo 的语言计算语义/不可由自身算法构造的基础机制 | +| Managed Value | Runtime 管理存储且按语言值规则处理的数值、文本、List/Map/Table 等 | +| Memory / Region | 有运行时签发身份、按字节寻址、可增长的虚拟存储区域 | +| Place | 在表达式语义中可被写入的位置(例如局部槽、Memory 索引) | +| Pointer | 由区域引用和偏移构成的逻辑地址,不等于宿主裸指针 | +| Resource | 由可信平台授予的外部对象与权限句柄,不是普通内存 | +| Provider | 对某个授权、版本化的低层资源接口提供具体 OS/设备实现的组件 | +| Stdlib | 用 `.mq.md` 为主编写、随发行包提供、可独立升级的高级库 | +| Computational Closure | 仅靠已批准 Core + 既有权限端点,能在 `.mq.md` 自行构造目标功能的可检验性质 | +| HostOp | 当前 Rust Host 中的固定操作枚举;迁移中保留兼容路径,最终不再承载高层标准库增长 | + +## 附录 B. 依据与可追溯参考 + +**Marqdo 当前代码(固定提交)**: + +- [`spec/evaluation.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/evaluation.md) +- [`spec/expression.ebnf`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/expression.ebnf) +- [`spec/mapping.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/mapping.md) +- [`spec/references.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/references.md) +- [`spec/security.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md) +- [`spec/plugin-abi.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/plugin-abi.md) +- [`spec/package-format.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/package-format.md) +- [`crates/marqdo-value/src/value.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-value/src/value.rs) +- [`crates/marqdo-runtime/src/builtins_impl.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-runtime/src/builtins_impl.rs) +- [`crates/marqdo-runtime/src/interpreter.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-runtime/src/interpreter.rs) +- [`crates/marqdo-host/src/host.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-host/src/host.rs) + +**对照的外部架构原则**(只用于解释取舍,不代表 Marqdo 已实现): + +- [WebAssembly Core 2 工作草案(2026-10-09)](https://www.w3.org/TR/2026/CRD-wasm-core-2-20261009/):抽象线性内存、`memory.grow`、越界访问陷阱、内存复制及可观测执行语义。具体规范版本应在 ADR 中固定,**不要求 Marqdo 逐字复制 Wasm**。 +- [Rust `core`/`alloc`/`std` 文档](https://doc.rust-lang.org/std/):分层区分核心计算、堆分配和平台能力,不要求所有高级库均禁止原生实现。 +- [WASI Component Model 官方文档](https://component-model.bytecodealliance.org/design/wit.html):描述外部接口与资源交互的类型契约;本 RFC 只参考原则,不强制引入完整组件模型。 +- [Linux 用户态 ABI](https://docs.kernel.org/admin-guide/abi.html):真实操作系统接口与平台限制是不可消除的边界,不能当作无权限的语言内置操作。 +- 上一份设计基线:`MARQDO_COMPUTE_MACHINE_RFC_0001.md`(本次对话已创建的原始计算模型 RFC)。 + +--- + +**状态总结**:本文件已完成架构论证、组件划分、迁移分类、复杂场景闭包矩阵、逐阶段实施门禁和自动验收设计;**尚未修改仓库代码、尚未创建标准库仓库、尚未运行提议中的新功能测试、尚未批准任何 ADR**。 diff --git a/docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md b/docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md new file mode 100644 index 0000000..7e9f8de --- /dev/null +++ b/docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md @@ -0,0 +1,498 @@ +# RFC-CM-0001:Marqdo 自主计算机器、可寻址内存与系统交互边界 + +> **状态**:设计提案(Draft,未实施,未批准) +> **日期**:2026-11-20 +> **建议归档路径**:`docs/architecture/RFC-CM-0001-compute-machine.md` +> **验证基线**:`cflmy/marqdo-cli`,Git 提交 `a113d894962d8c00eb02efb42e0e73f8b084a1fa`(检索于 2026-11-20) +> **兼容性**:涉及 `Value`、表达式赋值、运行时、主机能力和线协议的变更,必须通过 ADR、规范版本与相应测试;本 RFC 不声明任何新接口目前可用。 + +--- + +## 摘要 + +Marqdo 的优势不应只是用 Markdown 排版代码,而应是:在不更换 Markdown 语法角色的前提下,以**自身定义的计算语义**构造算法、数据结构、运行时工具和知识制品。现有解释器已提供整数、浮点、文本、列表、映射、控制流和函数调用,但尚未提供**由 Marqdo 程序控制的可寻址字节存储**;其赋值左侧仅支持标识符,系统交互集中在一组固定 `HostOp` 与外部插件能力中。这妨碍了用 `.mq.md` 自己编写内存分配器、二进制协议、共享节点结构、虚拟机器等低层算法。 + +**提议:以“托管值 + 多个可增长的线性内存区域 + 不透明系统资源”为计算机器的三个基础领域。** + +1. 保留现有 `Value` 计算体系,不强迫一切数据都变成字节地址。 +2. 引入一个拥有独立身份、按字节索引、可增长、具有边界的 `Memory` 值;每个区域内部线性寻址,物理实现可以是 Vec、分页或其他表示。 +3. 扩展现有“位置赋值”:允许对适当的索引目标写入,如 `m[8] = 42`,而不是另创 Markdown 语法或几十个关键字。 +4. 为创建、测量和增长内存定义**极少量计算原语**;让 `malloc/free`、内存池、链表、编码、缓冲区等**首先在 `.mq.md` 标准库**中实现。 +5. 将终端、文件、网络、进程、设备定义为**外部资源**,经稳定、可授权、可替换的底层适配接口交互;Marqdo 自身可以实现高级系统库与协议,但系统调用或硬件访问的**最后一步不能凭空消失**。 +6. CPU 寄存器由解释器/编译后端管理;裸机器码、GPU/QPU、共享内存原子操作暂不进入本 RFC 的基础语义。 + +这是一种**可编程的抽象机器**,而不是“在每种新需求前追加 Rust Host 函数”的框架,也不是把 x86/RISC-V 指令直接变成 Markdown 语法。 + +## 1. 设计动机:究竟希望解决什么 + +### 1.1 真实诉求 + +Marqdo 必须能在不借助 Python、JavaScript 或宿主语言提供专门算法函数的前提下,**通过自己的控制流和基础存储机制**构造原先不存在的高级能力。例如:实现内存分配器、二叉树、哈希表、字节码解释器、文件格式解析器、数值算法。对于 OS 和硬件,要求的是**高级功能主要由 Marqdo 实现,底层权限与系统调用接口保持稳定**,而不是“所有跨越内核边界的指令都神奇地由 Markdown 自己完成”。 + +### 1.2 非目标 + +- 不追求复刻 CPU 的真实寄存器、指令流水线、页表与异常处理体系。 +- 不把所有 Markdown `[]()` 链接重新定义为动态指针。链接继续表示文档/符号寻址,不伪装运行时对象地址。 +- 不把 GFM 表格当成实际运行内存。表格可以用于数据/状态的可读表达与快照,但不是存储管理器。 +- 不要求标准库全部使用 `.mq.md`;原生加速可以有,但**不是实现一般算法的必要条件**。 +- 不在第一期承诺用户代码内存安全、数据竞争自由、GPU 加速或跨平台无差异系统调用。 +- 不支持在无权限环境中直接执行任意机器码或任意 syscall。 + +### 1.3 设计原则 + +**P1 语义最小而充分**:首先增加无法由现有表达式自然实现的基础机制,不先增加高级 API 或新关键词。 +**P2 高级能力在语言中生长**:新增算法/数据结构不修改解释器、不修改 `HostOp`、不需要重新编译 CLI。 +**P3 读写位置统一**:索引读取和索引赋值遵循同一类“位置”语义;`Memory` 的读取/写入是基础例子,普通列表的值语义不得无声改变。 +**P4 内外边界诚实**:内存操作是语言计算;操作系统 I/O 依赖权限、平台接口和底层实现。 +**P5 可读性**:普通 Markdown 文档仍然可读;`##`/`**`/`*`/列表/blockquote 的现有角色不因本 RFC 改变。 +**P6 安全与可验证性**:将可验证的保证限定在区域边界和权限边界;不对自定义分配器的块级错误作虚假保证。 +**P7 零隐式权限**:能够导入模块、持有字符串或解析文档,不代表拥有文件、网络或设备访问权。 + +## 2. 现有仓库事实及需要保留的设计 + +依据提交 `a113d894...` 的源代码和规范: + +| 项目 | 现状 | 对本 RFC 的约束 | +|---|---|---| +| `marqdo-value` | `Unit/None/Bool/Int/Float/Text/List/Map/Table/Formula/Code/FunctionRef/PluginHandle`,没有普通可寻址 `Memory` 类型 | 应新增专门的存储引用值,而非把现有 List 改成默认共享对象 | +| `marqdo-expression` | 索引 `a[i]`/成员 `a.x` 可读取;`assignment = identifier = expression` | 最小语法修改是赋值目标扩展,不是新 Markdown 节点 | +| `marqdo-runtime` | `Frame.locals: BTreeMap`;`Assign` 以 `locals.insert` 重绑定 | 局部变量重新赋值应保持既有含义;共享可变性仅属于显式 `Memory` | +| `marqdo-frontend` | Markdown AST 是语法角色唯一权威;GFM Table 惰性 | 不更改表格、链接的执行语义;表达式子语言可增补语义 | +| `marqdo-ir` | 内置函数元数据固定且需更新规范 | 新增基础内存原语需要小型且审慎的语言变更;高层库不应无限扩展 Builtin 表 | +| `marqdo-host` | 固定 `HostOp`,包括 Print、Math、Foreign、FsRead/FsWrite、PluginInvoke | 不应将每一个新的标准库函数继续塞入固定 HostOp | +| 插件 ABI v1 | C ABI、worker、能力请求已有雏形 | 可复用函数注册与 Worker,不能错误地把 C ABI 当沙箱 | +| `pure/project/strict` | `pure` 禁止外部 I/O grant;`strict` Linux 探测成功后只支持 L1 范围,其他平台 fail closed | 新系统资源功能首先在项目模式开发;strict 须经新的隔离验证,不得默许开启 | +| 包管理 | path/Git 依赖与 lock v3 已实现;版本范围部分尚未强制 | 独立标准库可利用现有依赖体系,但兼容版本需要明确验证 | + +**保留的价值**:现有整数运算、函数调用、条件和循环不必推翻;现有 AST 映射、资源边界和可追踪执行事件是 Marqdo 与脚本转译器的根本区别之一。 + +源码(固定版本): +- [`spec/expression.ebnf`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/expression.ebnf) +- [`spec/mapping.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/mapping.md) +- [`spec/evaluation.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/evaluation.md) +- [`crates/marqdo-value/src/value.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-value/src/value.rs) +- [`crates/marqdo-runtime/src/interpreter.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-runtime/src/interpreter.rs) +- [`spec/security.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md) + +## 3. 总体架构与唯一的语义边界 + +```text + .mq.md 文档 GFM Table/Link/标题/正文仍按 Markdown 正常渲染 + │ + ▼ + Canonical Markdown AST ──▶ 已有 Frontend ──▶ Marqdo IR + │ + ┌───────────────────┤ + ▼ ▼ + 值与控制 存储与位置 + Int/List/Map MemoryRegion + 函数/赋值/循环 字节读写/增长 + └─────────┬─────────┘ + │ + Marqdo 计算核心 + │ + 需要访问外部世界时才穿越边界 + ▼ + Resource capability gate + │ │ │ + console file network/device + │ │ │ + Platform Adapter / OS ABI +``` + +逻辑上的执行状态可以表述为: + +`State = (Control, Frames, ManagedValues, RegionStore, ResourceTable, Budget)`。 + +- `Control`:当前 IR 节点和执行顺序;保留已有控制流。 +- `Frames/ManagedValues`:现有 `Value`、局部变量和调用语义。 +- `RegionStore`:一组可增长字节区域,每个区域按 `(region identity, offset)` 寻址。 +- `ResourceTable`:操作系统资源的不透明句柄与授权信息,**不得与内存地址混用**。 +- `Budget`:fuel、调用深度、内存限额、I/O 限额、截止时间等。 + +此设计**不规定**必须先创建自定义字节码 VM。现有树遍历解释器可直接执行新内存语义;将来引入字节码、JIT、LLVM/Cranelift 等属于可选执行方式,必须保持可观察语义。 + +## 4. 三种基础领域:值、内存、资源 + +### 4.1 托管值:维持已有语义 + +`Int/Text/List/Map/Table/Formula/Code` 等继续是运行时管理的值。普通 `b = a` 不得偷偷将普通列表变成共享可变指针。现有对象可以由 Rust 分配内存——**实现语言不等于语言语义**。这个领域负责易用性、文档处理、不可变数据等。 + +### 4.2 Memory:拥有身份的字节区域 + +新增 `Value::Memory`(名称暂定),含不可伪造的运行时区域引用。逻辑区域内部是一串字节,下标为非负整数;不同区域即使下标相同也不是同一个位置。 + +- **区域身份**:由本次运行中的内存实例签发;`Memory` 复制/传参时保留指向相同区域的身份(共享,不复制所有字节)。 +- **地址**:`(MemoryHandle, offset)`;`offset` 只在所属区域内有意义;整数本身不是获得其他区域权限的凭据。 +- **内容**:每个元素是 `0..255` 的一个字节;初始字节为 0。 +- **大小**:区域长度是页大小的整数倍;建议初期采用 `64 KiB` 页,以简化语义与工具链对照,不要求底层真的按页分配。 +- **增长**:只增长,不缩小;旧地址偏移不变;新加入的字节全部为零;不得突破区域/运行全局限额。 +- **读取**:`m[i] -> Int(0..255)`;越界、非法整数或已失效区域均是类型/存储错误。 +- **写入**:`m[i] = b`,`b` 必须是 0..255 的整数,不隐式截断;越界拒绝。 +- **相等性**:两个 Memory 引用的相等比较区域身份;不比较全部字节;Memory 不定义大小排序。 +- **序列化**:不允许将活动 `Memory` 句柄伪装成数值或跨进程永久有效地址。允许显式**复制字节内容**为快照,需限额;跨进程传输需要显式复制或另立经过授权的共享内存协议。 +- **生命周期**:第一版只在单次执行内存在;由 Runtime 保持活动句柄可达性,引用消失后可回收,至少在运行结束时回收全部区域。**不增加用户可见的 `free(region)`**,以免引入悬空区域句柄。用户的 `malloc/free` 管理的是**区域内部的块**。 +- **内存安全范围**:Runtime 保证单次操作不访问该 Memory 区域之外;不保证用户自编分配器不会越块、重复 free、释放后复用错误。 + +此设计不要求新增独立的 `Pointer` 原始类型。**首版指针可由 `Memory` 引用和 `offset` 组合表达**;是否将这一对优化为一等 `Pointer`,由实际性能与调用需求决定,而非现在先引入语法。 + +### 4.3 Resource:与外部世界交互的独立对象 + +`ResourceHandle` 是由可信平台/Provider 签发的不透明句柄,代表文件、控制台流、网络连接或专门设备;含授权访问模式与生命周期。**它不是 `Memory`,不能索引成任意 RAM,也不能通过伪造整数创建。** + +其常见操作是 `open/read/write/close`,但具体合法操作取决于资源接口。操作系统错误、EOF、部分读写、超时、取消都需要保留独立结果和错误语义。不得假定一次 `read` 总能填满缓冲区、一次 `write` 总能输出所有字节。 + +### 4.4 为什么要划分,而不是单一“万能地址空间” + +`Memory` 代表运行时可管理的存储;`Resource` 代表外部效果。普通内存读写是局部状态改变,终端输入/网络读取可能随外部世界变化。把二者统一为可以随意解引用的指针,会隐藏权限和不可重复性,也会使知识证明与回放语义失真。 + +## 5. 可编程动态内存:到底哪些必须内置 + +### 5.1 最小计算内核操作 + +这里定义**能力**,不冻结最终函数名: + +| 基本能力 | 候选用户接口 | 运行时职责 | +|---|---|---| +| 创建区域 | `memory.new(initial_pages, max_pages)` | 签发不可伪造引用,检查预算并分配零初始化区域 | +| 查询大小 | `memory.size(m)` | 返回页数,不泄露真实宿主指针 | +| 增长区域 | `memory.grow(m, pages)` | 原有字节与偏移不变,新字节为零;成功返回原页数,失败返回可区分错误 | +| 读取一字节 | `m[i]` | 边界检查,读取 `0..255` | +| 写入一字节 | `m[i] = b` | 边界和值域检查,更新同一 Memory 区域 | + +`memory.new/size/grow` 是操作而非关键词;`m[i]` 利用当前已有的读取表达式,`m[i] = b` 只扩展现有赋值目标语义。这是此 RFC 要求的**唯一用户可见表达式语法增量**。 + +`len(m)` 可在另一个 ADR 中统一成字节长度而非另设 `memory.size`;这里暂不混淆“页数”和“字节数”。是否以页或字节为 `grow` 单位也应通过 ADR 固定,本文的原型推荐页。 + +### 5.2 位置赋值(Place)如何实现 + +对左值 `x` 与 `m[i]`,**赋值含义不同但遵循一个语法规则**: + +- `x = value`:仍重新绑定当前函数帧局部变量,现有语义不变。 +- `m[i] = byte`:先按既有从左到右顺序求值 `m` 与 `i`,检查目标为 `Memory` 并检查偏移和值,然后更新同一内存实例。 +- `list[i] = value`:**不在第一个阶段承诺**。是否将列表修改为值更新/可变对象更新,要另行定义,不能顺便改变现有列表复制与索引语义。 +- `obj.field = value`:首版也不必实现。只有在明确记录值/对象语义后再拓展。 + +表达式编译结构的最小增量是 `Assign { target: PlaceExpr, value: Expr }`,其中 `PlaceExpr` 初期只有 `Local(name)`、`Index(base,index)`;索引目标仅接受 Memory,其他索引仍只读。无需新 Markdown 节点。 + +### 5.3 自定义指针与分配器 + +`malloc(n)` 在 `.mq.md` 标准库里持有一个区域 `m` 和元数据指针 `next`,向调用方返回区域内偏移(或 `{region:m, offset:p}` 组合值)。`free` 仅将已分配块放回空闲链表;若空间不足,调用 `memory.grow`。对齐、大小、空闲块链表、合并策略都由 `.mq.md` 编写,**不要求 Rust 为每一种分配算法做特殊 HostOp**。 + +例:块格式 `[size:u32][next:u32][flags:u8][payload...]`。首版只支持按字节读取,`u32` 的编码/解码可以使用四次索引、`//` 和 `%` 在 `.mq.md` 中实现。若该路径确实过慢,后来可增加 `load_u32` 等**语义等价、可验证的加速实现**,不能因此把完整分配器写到 Host。 + +分配器可建立:Arena、first-fit、buddy、slab、链表/图、字符串字节编码、编译器符号表、虚拟 CPU 的寄存器文件等。**内存块的逻辑安全由标准库分配器定义**;调用者拿到任意偏移后仍有越块写的能力,这属于明确的低层风险。 + +### 5.4 内存模型形式定义(草案) + +令 `R` 是区域身份集合,`M[r]` 是 r 区域的字节序列,`len(M[r])` 是其有效长度。 + +- `read(r,i) = M[r][i]`,前提 `0 ≤ i < len(M[r])`。 +- `write(r,i,b)` 将 `M[r][i]` 更新为 `b`,前提 `0 ≤ b < 256` 且 `i` 合法;其余字节不变。 +- `grow(r,k)` 要求 `k ≥ 0`,若预算允许,尾部追加 `k × PAGE` 个零字节,旧位置的字节值不变;否则返回显式失败且区域不变。 +- 不同区域互不别名:`r1 != r2` 意味着修改 `M[r1]` 不影响 `M[r2]`。 +- `m2 = m1` 若 m1 为 Memory,则二者指向同一 `r`;若为普通 `List`,原有值语义不变。 + +实现必须防止巨大 BigInt 偏移到 `usize` 转换时溢出、`offset + length` 整数溢出、预算计算溢出。任何错误不得 panic、越界读写或产生非确定性内存破坏。 + +## 6. 非线性存储、寄存器与加速的恰当位置 + +### 6.1 非线性内存不需要独立关键字 + +- **逻辑非线性结构**(链表、树、图):在一个或多个字节区域中存储偏移/引用即可;不要求新类型。 +- **物理非连续内存**:运行时可以用分页、稀疏映射或 Vec 实现同一线性语义;用户无需知道物理页布局。 +- **多设备内存**:未来可能带有设备特定权限与同步约束;不要将 CPU 内存随意解释成 GPU/QPU 地址。优先经专用受控资源/后端接入。 + +### 6.2 不加入真实 CPU 寄存器接口 + +当前局部变量与表达式临时值已经构成“虚拟寄存器”意义上的计算槽。编译到机器码时,可以由后端分配硬件寄存器。用户要模拟寄存器堆,可以用 Memory 自己实现;特殊硬件寄存器则是设备 I/O,而不是普通变量。 + +### 6.3 并行和原子操作 + +本 RFC 的内存规则初始定义为**单线程或按明确顺序执行**。绝不能声称 `m[i] = m[i]+1` 在多线程之间自动原子。未来若允许多个执行任务共享 Memory,必须另行设计同步/原子操作、数据竞争行为和访问序;也可以先采用无共享可变内存的任务隔离。新增原语应依据不可由当前操作正确组合而来的能力,而非先堆锁/屏障关键字。 + +### 6.4 GPU/QPU 不由普通 Memory 自动获得 + +Memory 支持字节缓冲区与数据布局,但 GPU 计算仍需要执行后端与数据传输,真实量子资源则不是可随意读取的字节地址。Marqdo 可用独立后端/服务处理硬件特有语义,内核应保持硬件无关。 + +## 7. 系统交互:能否不依赖 Rust/Python 实现高级功能? + +### 7.1 必须给出的诚实答案 + +**可以**:大量系统功能由 Marqdo 标准库直接编写,包括文本与字节编码、目录遍历算法、协议解析、HTTP 报文、部分网络协议逻辑、缓存、日志格式、CLI 交互逻辑、文件格式读写、数据库协议客户端中的高层逻辑。程序运行时无需执行 Python 或 JavaScript,也不需要为每个高级库函数增加 Rust 分支。 + +**不能无条件做到**:如果当前运行方式是一个 Rust 编写的解释器,那么跨越用户态与操作系统的**最后一步**必须由解释器内建的系统适配器、已授权插件/worker、或其他可信底层机器代码执行。Markdown 源码不能在无相应运行环境、系统调用 ABI 和权限的情况下直接控制硬件。即使将来 Marqdo 编译成独立原生二进制,它也仍通过操作系统 ABI 或设备驱动与外界交互,只是**不再需要 Rust 解释器常驻运行**。 + +**不可混淆的四种独立性:** + +| 层面 | 能否达到 | 含义 | +|---|---|---| +| A. 算法独立 | **可以,近期目标** | 排序、内存分配、解析器等算法均在 `.mq.md` 中 | +| B. 标准库独立 | **可以,近期目标** | 新标准库函数不修改 Rust Runtime/HostOp,单独版本发布 | +| C. 系统接口独立 | **有条件可以** | 已有的 OS 原语/权限可组合构造新功能;新增设备/OS 原语需相应适配器,但不改语言核心 | +| D. 运行时/宿主实现独立 | **中长期目标** | 独立编译成机器码/Wasm、或自举解释器,不依赖 Rust 解释器运行;仍依赖 OS ABI/硬件 | + +**这里的“无需修改宿主”不是“无底层实现”。** 应将“高级逻辑用 Marqdo 实现”与“最后一跳由可信 OS adapter 执行”区分清楚。 + +### 7.2 最小系统资源操作模型 + +建议**不增加任何 I/O 关键字**,而以可注册、版本化的 `Resource` 接口提供 OS 能力。候选调用形式(未实施): + +- `resource.open(kind, descriptor, rights) -> ResourceHandle` +- `resource.read(handle, memory, offset, max_bytes) -> count | EOF/error` +- `resource.write(handle, memory, offset, bytes) -> count | error` +- `resource.close(handle) -> Result` + +其中: + +1. `kind` 例如 `stdio`、`file`、`socket`,必须由可信平台适配器或已安装的 Provider 明确注册,**不能因为传入任意字符串就自动获得能力**。 +2. `descriptor` 是声明请求的结构化参数,例如项目内文件名、网络目标,必须验证并授权;绝非任意可解引用真实指针。 +3. `rights` 是请求的权能;实际授权为“请求 ∩ 操作者 grant ∩ Provider 支持”,拒绝必须显式报错。 +4. `read/write` 通过 Memory 区域交换原始字节,边界检查和缓冲区借用由 Runtime/Adapter 保证;不得允许 I/O Adapter 长期保存未固定的宿主 Rust 指针。 +5. 区分 EOF/0 字节暂时无进展/取消/系统错误;支持部分读写,标准库负责重试与缓冲。 +6. `close` 撤销对应资源句柄;已关闭句柄、重用 ID 必须借助 generation/注册表检查,不能错误访问新资源。 +7. 对设备特殊操作,不强行滥用 `read/write`:必要时可由具体 Provider 暴露专门的**版本化、授权操作**;不在通用内核中新增一个无条件允许任意 syscall 的 `control` 后门。 + +`resource.*` 属于**平台接口层**而不是纯计算内核;即使调用看起来是普通函数,执行前必须检查权限。现有 Host 能力检查可成为迁移入口,但不应为每个新高层标准库函数创建新的 `HostOp` 变体。 + +### 7.3 应该怎样真正避免“每扩展一次就修改 Host”? + +将目前的固定分派进一步分成: + +```text +普通 Marqdo 标准库函数 + ├── 调用 Marqdo 语言计算原语(无需 Host) + └── 仅在外部交互时使用稳定的 Resource ABI + │ + Capability Gate + │ + Provider/Platform Registry + │ + ┌────────────────┴───────────────┐ + ▼ ▼ + 标准平台适配器 独立第三方 Provider + 文件/终端/网络 新设备/专用系统能力 +``` + +关键是**一次性稳定可扩展的外部操作契约**,而不是一张不断增长的 Host 功能目录。 + +现有 C ABI v1、worker 与 `CallableRegistry` 应先尽量复用;新接口可在不破坏 v1 的条件下增补版本化注册信息。如需升级,必须有兼容矩阵与迁移机制。**Provider 负责它新增的原始能力;`.mq.md` 标准库负责高级功能。** + +### 7.4 两种系统访问路线及取舍 + +| 路线 | 优点 | 风险/限制 | 建议 | +|---|---|---|---| +| 能力化 `Resource` ABI | 统一权限与跨平台行为;适合安全分层 | 新的真正 OS 原语仍需要 Adapter/Provider | **默认且必须支持** | +| 直接的原生 syscall/FFI 边界 | 能较快覆盖大量平台特性 | 平台/ABI 依赖、地址搬运、权限绕过、维护和可验证性问题 | 仅可信、本机、显式授权的可选高级接口;默认 pure/strict 禁止 | + +在 Linux 等平台,即使“直接 syscall”,也要处理目标架构调用约定、数字差异、结构体布局、指针与长度、errno 以及 OS 策略;Windows/macOS 并不提供与 Linux 一模一样的 syscall ABI。不得将其伪装成普适的语言核心。 + +**无悖论地满足用户目标的办法**:向 Marqdo 暴露一个小而稳定的**操作系统能力边界**,然后让标准库以 Marqdo 自己的程序逻辑组合出系统高级功能。这是可实现的强独立性;完全消灭系统边界不是现实目标。 + +### 7.5 端到端例子 A:完全用 Marqdo 构建交互式 CLI + +**先决条件**:可信平台适配器提供 stdin/stdout 的已授权资源,Marqdo 运行时支持 Memory 操作。 + +1. 标准库 `io.read_line` 在 `.mq.md` 中申请/增长缓冲区,循环调用资源 `read`,查找换行字节。 +2. 标准库 `text.utf8_decode` 用 `.mq.md` 处理 UTF-8(正确处理非法序列;编码策略可配置)。 +3. 用户程序将文本解析为数字,用现有 `Int` 算术和条件循环处理。 +4. 标准库 `io.write_text` 使用 `.mq.md` 的 UTF-8 编码,再经 `write` 输出字节。 + +**全流程无需 Python/Node,也不需要 Host 特地实现 `input()`。** 然而最底层 stdin/stdout 仍由 OS 适配器调用操作系统接口。 + +### 7.6 端到端例子 B:新增 HTTP 客户端能力 + +**先决条件**:系统适配器已经提供经授权的网络连接资源、DNS/地址解析的可用机制,以及必要的加密能力或可插拔 TLS Provider。 + +1. 标准库构造 HTTP 请求行、首部和消息体的字节序列。 +2. 使用 `resource.write` 处理部分发送;使用 `resource.read` 实现缓冲接收。 +3. `.mq.md` 解析状态行、首部、chunked 编码等;实现重试、缓存、超时逻辑。 +4. 测试 HTTPS 时,必须使用可靠 TLS 实现(可由已审计 Provider 提供;“不依赖外部语言写协议逻辑”不意味着应该自己手写未经验证的 TLS)。 + +网络的“最后一跳”必须存在,且不同平台的 TCP/TLS 能力可能需要不同 Provider;不能只靠 Memory 推导出 socket。 + +### 7.7 端到端例子 C:科研加速与 QPU + +- 经典数值模拟:`.mq.md` 可以在 Memory 中构造矩阵、压缩存储、数据布局算法,后续由独立计算后端加速。 +- QPU:量子设备是具有权限、调度与门集约束的资源,不是普通 Memory;通过专用 Provider/Backend 提交计算、接收结果。 +- 这种接口可保持标准库和上层代理/知识文档不绑定 Rust 宿主,但并不能保证算法自动获得 GPU/QPU 性能或精确的量子物理语义。 + +## 8. 安全、权限与错误模型 + +| 保护目标 | 首版保证 | 明确不能保证 | +|---|---|---| +| 虚拟 Memory | 区域越界、非法索引、增长限额、无效句柄失败 | 区域内 malloc/free 的块级内存安全 | +| 不透明 Resource | 授权身份、访问模式、资源关闭和 generation 检查 | 平台适配器/不可信原生插件本身绝对安全 | +| pure | 内部算术、内存与受预算控制的逻辑计算;不获得外部 I/O | 允许直接 syscall 或任意外部访问 | +| project | 对批准的文件/终端/网络资源进行操作,使用能力化 API | 等价于 OS 级进程沙箱 | +| strict | 维持现有 fail-closed 策略;新 I/O Provider 必须另行经过 OS 级安全验收 | 当前版本自动支持所有外部能力 | + +进一步要求: + +- 全局/单区域内存预算、增长次数与 fuel 限额、操作可取消性;内存失败不得半成功。 +- 路径资源应使用受控根路径和可验证的 OS 打开策略,避免字符串检查后再打开造成 TOCTOU;网络权限需要处理名称解析/重绑定等问题。 +- 插件/worker 不因拥有 API 名字自动获得权限;原生进程内插件不能被当作不可信沙盒。 +- Memory/Resource 活动句柄不应直接跨线协议序列化;跨边界传输使用字节复制、快照或显式授权的资源传递协议。 +- 诊断保持来源 Markdown 节点位置;可选追踪读取/写入的逻辑区域、偏移和长度,不能将外部文件秘密或原始敏感字节无条件写日志。 + +## 9. 与当前 Marqdo 语法和知识编译的结合 + +这一方案故意**只修改最少的表达式语义**。 + +- `##` 函数、`**` 执行、`*` 返回、Markdown 列表与引用规则继续维持规范。 +- `[]()` 是**文档/知识的静态寻址**;动态 Memory 的字节偏移属于执行期位置,两者不强行共享 URI 语法。 +- 普通 GFM 表格是文档值;可用于初始化/展示内存快照,但不会自动变成内存分配器。 +- `memory.new(...)` 与 `resource.open(...)` 都可按普通函数调用表达;不新增 `memory`、`pointer`、`asm`、`register` 这样的语法关键字。 +- 内存状态变化可选输出可读 GFM Table 供 View/Proof 使用;证明必须以抽象执行语义为依据,而不是假设表格文本本身就是执行存储。 + +示例(**候选语法,当前尚不可运行**): + +````markdown +## 研究一个字节缓冲区 + +此函数演示“可引用的存储区域”与普通 Markdown 叙述并存。 + +**区域 = memory.new(1, 4)** + +**`区域[8] = 42`** + +**另一个名字 = 区域** + +**`另一个名字[9] = 99`** + +*区域[8] + 区域[9]* +```` + +预期结果 141。注意这里 `另一个名字 = 区域` 分享 Memory 身份;一般列表/表格并不因此被改造成共享字节缓冲区。 + +## 10. 可行性和备选方案评定 + +| 设计 | 语言自主性 | 高层开发便利性 | 性能潜力 | 安全/复杂度 | 结论 | +|---|---|---|---|---|---| +| 继续只扩充 HostOp | 低 | 表面高 | 取决于 Rust 实现 | Host 随功能膨胀 | 否决为主要路线 | +| 只有托管对象堆/GC | 中 | 高 | 对一般数据不错 | 需要身份/生命周期语义 | 保留现有托管值,但不足以满足低层目标 | +| **多区域字节内存 + 托管值 + Resource** | **高** | **高** | 后端可优化,字节操作起初较慢 | 明确的区域边界;块级风险可见 | **推荐** | +| 任意裸指针 + 直接机器码 | 极低平台独立性 | 低 | 最高上限 | 高风险、难验证 | 只可选的可信原生扩展 | +| 纯 GFM 表格作堆内存 | 低至中 | 对展示很高 | 非常低 | 动态状态/身份混乱 | 否决 | + +该推荐不是宣称“最优”已经被数学证明,而是在现有语法、实现成本、扩展性与安全边界下的**工程选择**。 + +## 11. CLI/Runtime/标准库仓库的具体职责 + +### 11.1 `marqdo-cli`:保留小而稳定的内核 + +- `marqdo-value`:新增不可伪造的 Memory 值表示;必要时添加 ResourceHandle,注意现有 PluginHandle 不能随意直接复用为 OS 资源句柄。 +- `marqdo-expression`:`AssignTarget` 的 Local / Index 变体;保留原有表达式读取。 +- `marqdo-frontend`:更新表达式 AST 对应 IR 和定位信息;不改变 Markdown 语义映射。 +- `marqdo-runtime`:Memory 实例生命周期/增长/边界检查/预算;执行位置写入;参考解释器。 +- `marqdo-host`:逐步提供可注册资源操作的能力边界;必须保持旧 `HostOp` 兼容窗口,不能一次砍断所有旧调用。 +- `marqdo-package` / CLI:发布标准库兼容信息,继续利用已存在的 path/Git pinned lock;资源授权仍由真实操作者提供。 + +### 11.2 独立 `marqdo-stdlib`:以 Marqdo 实现通用功能 + +建议独立仓库 `cflmy/marqdo-stdlib`,首期目录: + +```text +marqdo-stdlib/ + core/ # 字节编码、整数工具、可恢复错误封装 + memory/ # arena, allocator, free-list, binary codecs + collections/ # list/tree/graph/hashmap 等数据结构算法 + text/ # UTF-8, parsing + io/ # buffered readers/writers, read_line + data/ # csv, json, 自定义格式 + tests/ # 以 marqdo run/call 执行的标准库回归程序 +``` + +高层函数应在 `.mq.md` 里,按普通模块导入。发行版携带一个经测试的 stdlib 版本,支持离线运行;单独发布更新时,要求明确的 Runtime 兼容范围与锁定。标准库可有原生加速 Provider,但不能把它作为常规功能的唯一实现路径。 + +### 11.3 独立 Platform Providers + +系统 Adapter 可以由 Rust、C、汇编或将来的 Marqdo 原生编译代码实现。当前 CLI 不应为普通库函数增加 HostOp;对真正的新 OS 原语,若标准 Provider 不支持,用户必须安装/编写扩展 Provider,并获得相关权限。**不是每个新设备都能只靠修改 `.mq.md` 获得访问能力。** + +## 12. 分阶段实施(范围严格受控) + +### R0:冻结设计与当前基线(无新功能) + +- 此 RFC 和对应 ADR 经审阅,确认 `Memory` 值语义、`index assignment`、错误类别、线协议边界及安全模式。 +- 修复已有跨平台 CI/安全收敛问题;固定解释器行为的回归样例。 +- 建立一页对照表:新增语义是否需要 `HostOp`、新关键字、新 Markdown 节点。首版目标均为“不需要”。 + +### R1:最小可寻址存储(核心里程碑) + +- `Memory` 区域:创建、大小、增长、读取一字节、写入一字节;实现总/单区域预算、错误恢复与 trace。 +- 支持 `**\`m[8] = 42\`**` 这类现有 Markdown 负载中的索引赋值。 +- 不开放裸指针、不引入新寄存器、不改旧 List/Map/Table 的赋值与复制行为。 +- 运行跨区域隔离与别名测试;报错不能导致原有区域被污染。 + +### R2:Marqdo 自举的内存与算法库 + +- `.mq.md` 顺序分配器;随后 free-list 可复用分配器。 +- 不修改 Runtime/Host 编写并执行:可增长链表、二叉树、简单虚拟 CPU、UTF-8/二进制解析器。 +- 提供内存利用率、分配吞吐、碎片率、执行时间及解释器 fuel 的测量;避免只用“功能跑通”证明成熟度。 + +### R3:最小系统资源闭环 + +- 在项目模式先实现 `stdio` 与受控项目文件的 `open/read/write/close`;**功能可用性不代表 strict 安全已通过**。 +- `.mq.md` 标准库实现 `read_line`、`write_text`、文件二进制复制与 UTF-8 编码。 +- 资源注册层允许 Provider 扩展;验证一个第三方资源端点不修改解释器编译即可接入(是否需要实现新的系统适配器需明示)。 +- 对当前 `HostOp` 逐步适配,保留兼容期,不能强行把所有第三方都迁移到新的不成熟 ABI。 + +### R4:可选增强(不阻断基础版本) + +- 索引/字段赋值的一般化、字节多宽度原语及位运算、快照、增量编译、优化执行器。 +- 需要共享内存并行时,再设计原子/同步与数据竞争语义。 +- 独立网络/TLS、设备、GPU、QPU 后端与原生编译目标。 + +**必须遵循的顺序**:R1 完整正确 → R2 验证计算自主性 → R3 证明外部交互可组合;不先把 R4 的设备能力和并发类型系统塞入 CLI。 + +## 13. 端到端验收清单 + +| 编号 | 验收项目 | 判定标准 | +|---|---|---| +| T1 | 语义一致性 | 旧的局部赋值与 List/Map/Table 行为完全不变,旧 Markdown AST golden 通过 | +| T2 | Memory 隔离 | `m1[0]=x` 不影响 `m2[0]`;`m2=m1` 后写入可相互观察 | +| T3 | 增长与错误 | 成功增长保留旧字节且新字节为 0;负索引、越界、超限增长、巨大 BigInt 均明确失败且不破坏状态 | +| T4 | 自建分配器 | `.mq.md` 实现 malloc/free,释放块重复利用;高低地址和碎片测试通过 | +| T5 | 高级数据结构 | 不改 Rust 核心,仅新增 `.mq.md` 实现可增长链表、树、二进制协议解析器 | +| T6 | 标准库独立升级 | 旧 CLI 二进制保持不变,安装一个与之兼容的新 `.mq.md` 库并成功使用 | +| T7 | 终端闭环 | `marqdo run` 仅使用标准库加基础 OS Adapter,完成多轮输入/输出,无 Python/Node 调用 | +| T8 | 系统资源授权 | 无 grant 无法 open;已关闭/伪造 handle 无法访问;项目路径无法越权;所有拒绝有诊断和必要审计 | +| T9 | 安全模式 | `pure` 不允许外部 I/O;当前 `strict` 不自动放开新 Provider;真实隔离与失败路径有测试 | +| T10 | 扩展接口 | 新高层 I/O/协议函数不增加任何 HostOp;新 Provider 通过约定的独立注册/版本机制接入 | +| T11 | 兼容/性能 | Windows/macOS/Linux 的支持矩阵真实披露;可定量报告字节访问吞吐、内存增长和 Python/Rust baseline 差距 | +| T12 | 文档与证明 | Memory 状态可导出可读快照;执行状态引用源 Markdown span;不将数据表格误认成内存本体 | + +## 14. 需要 ADR 决断而不是隐含假设的开放问题 + +本 RFC 采用一条**可实施的推荐路线**,但下列决策会改变公开语义,须单独冻结: + +1. **页语义**:推荐 64 KiB 页、只增长;是否与外部目标标准保持一致,预算如何计入已保留与已提交物理内存。 +2. **Memory 的值/调用传递**:推荐复制引用保持身份,区域可达则存活;禁止隐式跨线协议序列化。 +3. **Memory 引用寿命**:推荐首版只在一次执行期间有效,采用自动回收或至少结束时回收;内部块分配是 stdlib 职责。 +4. **Place AST**:推荐首版只增 Local + Memory Index;其他集合的索引赋值须另行评议。 +5. **OS Resource 最小 ABI**:是否先使用现有插件注册机制包装,而不立即新增第二套并行插件协议。 +6. **跨平台支持范围**:纯计算 API 保持统一;系统资源能力按平台明示 availability,不能假定所有设备/系统调用均存在。 +7. **实验数据**:R2 执行效率可能较低,是否加入 32/64 位宽读写及优化编译由实测结果决定,而非预设几十种关键字。 + +## 15. 最终建议与决策摘要 + +**建议通过的架构选择:** + +- **语言核心**:保留现有托管值与控制流,新增**显式、多区域、可增长的字节 Memory**;用现有索引表达式扩展可写位置,不创建新的 Markdown 语法层。 +- **内存管理能力**:`Memory` 区域由 Runtime 提供并保护区域边界;`malloc/free`、块布局、链表、树、二进制编码等由 `.mq.md` 标准库自己构造。 +- **系统能力**:独立、授权、可替换的 `Resource` 边界;标准库在 Marqdo 中组合 I/O,OS 的最后一跳仍由可信本地 Adapter/Provider/编译后端承担。 +- **扩展性**:不为每个库函数增加 Rust `HostOp`;只有新增*真正不可由现有原语表达*的硬件/OS 能力,才需要扩展适配器。 +- **安全性**:区分区域边界安全、块级内存安全、进程隔离和 OS 权限;不能用“沙盒”一词掩盖缺少的保证。 +- **长期演进**:以此作为实现自主数据结构、字节码 VM、优化执行器与量子/科学计算后端的基础,但不许声称三个内存原语就能自动产生 GPU/QPU 加速。 + +**核心结论:** 如果“摆脱宿主语言依赖”指的是**程序逻辑、标准库算法和高级系统功能不依赖宿主语言来实现**,这条路线可以达成;如果指的是**无需任何底层可信机器代码和操作系统接口即可访问硬件**,则不可能。对真正的独立编程语言而言,恰当目标是**自己定义计算、自己构建高层库、清楚且可扩展地跨越不可消除的系统边界**。 + +--- + +## 参考资料(外部设计依据,不是 Marqdo 已实现功能的证据) + +1. [WebAssembly Core Specification 3.0(2026-09-11)](https://www.w3.org/TR/wasm-core-2/):线性内存、内存增长、抽象机器、核心与环境 API 的分离;注意其保证为线性内存边界安全,不保证客户程序自己设计的内部分配器没有逻辑错误。 +2. [Rust `core` / `alloc` / `std` 文档](https://doc.rust-lang.org/std/):计算基础、分配与平台库的分层思路;并不意味着必须用 Rust 编写 Marqdo 的标准库算法。 +3. [Linux Kernel ABI documentation](https://docs.kernel.org/admin-guide/abi.html):OS 用户态接口的稳定性、适用范围和版本边界。 +4. [Linux Kernel ioctl interfaces](https://docs.kernel.org/driver-api/ioctl.html):设备特定能力并不都能归约为普通 RAM 读写,ABI 应明确数据布局与操作语义。 +5. [Marqdo 当前安全模型](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)及[插件 ABI v1](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/plugin-abi.md)。 diff --git a/docs/STATUS.md b/docs/STATUS.md index 4768ca8..74160d7 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -71,7 +71,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL | 2 | Cross-platform strict `cfg` | **Fixed locally; native CI UNVERIFIED** | `#[cfg]` dispatch + non-Linux regression test | | 3 | Strict probe→launch race | **Fixed locally; GitHub Linux CI UNVERIFIED** | fail-closed CLI contract + probe-OK/launch-EAGAIN seam test | | 4 | Parallel security matrix isolation | **Fixed** | unique paths + helper `OnceLock`; 7/7 in parallel workspace run | -| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 44 semantic, 52 compile-negative, 12 runtime-negative fixtures | +| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 68 semantic, 52 compile-negative, 21 runtime-negative fixtures | | 6 | Condition header + source spans | **Fixed** | executable tails rejected; literal-inside-backtick mapping tests | | 7 | Independent Tool consumers | **Done locally** | `sample-view`/`sample-knowledge`/`sample-proof` E2E test | | 8 | lock v3 reproducibility | **Implemented; tamper test added** | `package_env` PASS | @@ -86,7 +86,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL - Frozen Markdown profile `marqdo.md.v1` with 160 AST goldens and explicit CRLF byte semantics. -- Native Rust interpreter with 44 semantic goldens, 52 compile-error fixtures, +- Native Rust interpreter with 68 semantic goldens, 52 compile-error fixtures, and runtime error source attribution. - Native `**` power is pinned end-to-end: right associativity, unary-minus precedence, parentheses, float negative exponents, branch/loop conditions, @@ -109,6 +109,31 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL version verification. - Linux strict L1: Bubblewrap namespace supervisor, runtime probe, fail-closed launch, child self-verification, 7/7 adversarial matrix. +- Addressable memory (RFC-CM P1, ADR-0022): opaque `Memory` values shared by + handle identity, region isolation, zero-filled 64 KiB page growth, bounded + byte read/write, `memory.new/size/grow`, `m[i] = byte`, run and per-region + budgets, host/wire rejection, Analysis v2 target projection. +- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): success is + `{ok:true,value:...}`, failure is `{ok:false,code,message[,details]}` as an + ordinary Map; expected conditions return it, invariant violations stay + fatal. A `.mq.md` first-fit/free-list allocator reuses blocks, splits free + blocks, coalesces address-adjacent blocks and returns `alloc.no_space` + without aborting. +- CM-A07 Compat-A (ADR-0027): explicit `.mq.md` `io.写值` chain + (`format.显示` → UTF-8 encode → `Memory` → `resource.write(stdout)`) matches + builtin `打印` for the covered value set; builtin remains default and no + alias is routed. +- Resource streams (RFC-CM CM-A06, ADR-0025): opaque host-issued `Resource` + handles with `resource.open/read/write/close`; bytes move only through + `Memory`; stdout write is capability/output-budget checked and project-file + read/write is path/grant checked; file writes buffer until `close` and + never flush on drop; a `.mq.md` chunked copy loop reproduces a file without a + Rust `copy_file`; strict refuses every resource open until a provider is + verified. Semantic fixture 66 and CLI/host tests. +- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023): + value-semantics `list.append/concat`, `map.keys/has/get/set/remove`, + Unicode scalar `text.codepoints`/`text.from_codepoint`, and non-negative + `table.columns/row/cell`. Pure builtins; no mutation, no new syntax. ## 4. Open blockers and limits @@ -121,25 +146,59 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL fixture lane, Unix-gated worker tests). 4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed broker services; unsupported requests must return `capability.unavailable`. -5. uv/npm/venv provisioning is not implemented. Only a pre-provisioned, +5. Resource streams are a **P3 first slice** only: project-file read and + buffered write plus stdout, but no directories, network, process, device, + generic provider or stdlib `print` migration; strict refuses every + `resource.open` until the provider passes the isolation lane. +6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned, digest-pinned directory with a regular-file executable is supported. -6. No Windows/macOS RC archive exists from this host. The release-candidate +7. No Windows/macOS RC archive exists from this host. The release-candidate workflow can build macOS on GitHub, but it has not run for this SHA. -7. No release tag/binary publication. The release decision is **DO NOT +8. No release tag/binary publication. The release decision is **DO NOT RELEASE** until the native CI matrix and a security-lane CI URL exist. -## 5. Evidence files +## 5. RFC-CM compute-machine status + +The two 2026-11-20 architecture RFCs are design input, not implemented +behavior. This table records only what is in this tree now. + +| RFC phase | State | Evidence / limit | +|---|---|---| +| P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged | +| P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) | +| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), and a quoted/escaped CSV row parser (68). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Still missing: separate `marqdo-stdlib` repo, JSON floats, table group, EOF/partial stream contract, and a frozen-CLI closure lane. | +| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict refuses all opens. Fixture 66 + CLI/host tests. CM-A07 is **Compat-A only**: `examples/stdlib` provides an explicitly imported `.mq.md` `format/utf8/io` chain whose output matches builtin `打印` for Unit/None/Bool/Int/Float/Text/List/Map/Table; builtin `打印` remains default, no alias routing, and Formula/Code/FunctionRef/PluginHandle formatting is still unobservable. Still missing: Compat-B/C, directories, network, generic provider registry. | +| P4 Table logical/physical migration | **NOT IMPLEMENTED** | `Value::Table` remains `Vec>`; no stdlib table algorithms | +| P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) | +| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work | + +The P1 memory domain is pure computation: it needs no grant and is available in +`pure`, `project` and `strict` (bounded by the memory budgets). It adds no +`HostOp`, no Markdown role and no CLI command. `Memory` handles cannot cross +the wire or a host/plugin/foreign boundary. Block-level safety of a +user-written allocator is explicitly **not** claimed. + +## 6. Evidence files - [`evidence/2026-11-20-local-gate.md`](evidence/2026-11-20-local-gate.md) — earlier local gate and Windows autocrlf reproduction. - [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md) — current VM lane, environment, full log and workspace test log. +- [`evidence/2026-11-20-resource-stream-p3.md`](evidence/2026-11-20-resource-stream-p3.md) + — CM-A06 first slice: stdout write, project-file chunked read, strict refusal, + wire/host boundary rejection and test commands. +- [`evidence/2026-11-20-resource-file-write-p3.md`](evidence/2026-11-20-resource-file-write-p3.md) + — buffered project-file write, flush-on-close, buffer ceiling, grants and + CLI write-then-read evidence. +- [`evidence/2026-11-20-stdlib-print-compat-a.md`](evidence/2026-11-20-stdlib-print-compat-a.md) + — CM-A07 Compat-A `.mq.md` print prototype, side-by-side output comparison, + covered/uncovered value types and the Compat-B blocker. - [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md) — G0..G12 matrix and RC decision. - [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict L1/L2 limits. -## 6. Reproduce +## 7. Reproduce ```bash export PATH="$HOME/.cargo/bin:$PATH" diff --git a/docs/adr/0022-addressable-memory-and-place-assignment.md b/docs/adr/0022-addressable-memory-and-place-assignment.md new file mode 100644 index 0000000..6df6915 --- /dev/null +++ b/docs/adr/0022-addressable-memory-and-place-assignment.md @@ -0,0 +1,183 @@ +# ADR-0022: Addressable `Memory` and Memory-index place assignment (RFC-CM P1) + +Status: **Accepted** +Date: 2026-11-20 +Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md` (CM-A01/CM-A02/CM-A03), +`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (P0/P1), ADR-0007, +ADR-0011, ADR-0017, ADR-0021 +Supersedes: the v1 rule that assignment targets are bare identifiers only +Specs: `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, +`spec/analysis-v2.md`, `spec/security.md` +Evidence: `docs/evidence/2026-11-20-memory-p1.md` + +## Context + +RFC-CM-0001/0002 define three domains: managed values, addressable regions and +capability resources. The first two are a language change; the third is an OS +boundary and is deliberately **not** part of this ADR. + +Before this decision the interpreter had no writable place other than a local +slot. `Value` had no byte region, `a[i]` was read-only, and assignment was +parsed only as `identifier = expression`. A `.mq.md` program therefore could +not express an allocator, an encoded record, a free list or a byte protocol +without a new Rust builtin for every algorithm. That is the closure gap +RFC-CM-0002 §2 identifies. + +The decision must not: + +* turn managed `List`/`Map`/`Table` values into shared mutable objects; +* expose a host pointer, `usize` address or procedural `free(region)`; +* let an active handle cross the value wire or an ABI boundary; +* add a Markdown node, keyword or `HostOp` for memory; +* claim block-level memory safety for user allocators; +* weaken `strict` or grant any external I/O. + +## Decision + +### 1. `Value::Memory` is an opaque, run-scoped identity + +`MemoryHandle` (`crates/marqdo-value/src/value.rs`) is +`(run_id, region_id, generation)` with private fields and a runtime-only +constructor. It is not `Serialize`/`Deserialize`, is never built from user +input, and carries no host address. + +Copying the value, passing it as an argument and storing it in a list or map +all copy the handle and preserve region identity. `==`/`!=` compare identity +only; `Memory` has no ordering comparison. `display()`/`repr()` render the +literal `Memory`, never the internal identity. + +### 2. Regions are grow-only, bounded byte arrays + +Frozen semantics (CM-A01/CM-A02): + +| Property | Decision | +|---|---| +| Page size | 64 KiB logical page (`PAGE_SIZE`) | +| Initial content | Every byte is `0` | +| Growth | `memory.grow` only increases; old offsets/bytes unchanged; new bytes zero | +| Failure atomicity | A rejected create/grow allocates, commits and mutates nothing | +| Shrink/free | No shrink and no user-visible region free; regions are dropped with the run | +| Address | `(region identity, non-negative offset)`; an integer alone is never a credential | +| Read | `m[i] -> Int(0..=255)` | +| Write | `m[i] = byte`, `byte` must be `Int(0..=255)`, no truncation | +| Isolation | Different regions never alias; handle copies of one region do | +| Visibility | `memory.size` returns committed pages, not host capacity or pointers | + +Limits in `MemoryLimits`: + +* `DEFAULT_MAX_TOTAL_BYTES = 64 MiB` committed across the run; +* `DEFAULT_MAX_REGION_BYTES = 16 MiB` per region, which also caps declared + `max_pages` at 256; +* budget arithmetic uses checked `u64`/`usize` operations; allocation uses + `try_reserve` so a host OOM is a structured `resource.exhausted` instead of + a panic. + +Limits are a runtime API (`marqdo_runtime::MemoryLimits`, `run_with_limits`) +and are overridable on `run`/`call` with `--max-memory ` (total) and +`--max-region-memory ` (per region). The defaults are stated in +`spec/evaluation.md`. Both flags apply in `pure`, `project` and `strict`; they +grant no external capability. + +### 3. The only new writable place is a `Memory` index + +`AssignTarget` gains `Index { base, index, span }` in the expression AST and +`PlaceTarget` mirrors it in the IR. The surface syntax remains a `strong` +node containing `target = expression`; no Markdown role, heading or list +meaning changes. + +* `x = value` still rebinds the current frame's local slot. +* `m[i] = byte` evaluates **base, then index, then RHS**, exactly once each. + All three checks (base is `Memory`, offset in range, byte in `0..=255`) + happen before the single byte write, so a failed statement does not write. +* `list[i] = value`, `map[key] = value` and `obj.field = value` are not + defined. Member targets are a parse error; a non-`Memory` index base is + `runtime.type`. No managed collection changes value semantics. +* Nested index targets are accepted syntactically; the innermost base must + still evaluate to a `Memory` value at run time. +* Assignment remains statement-only; `x = y = 1` is still a parse error. + +This is CM-A03. + +### 4. Handles cannot cross the wire or the host boundary + +* `to_wire(Memory)` emits only `{"type":"memory"}` so an accidental transfer + is visible. `from_wire` rejects that tag with `WireError::Invalid`; a + round-trip is intentionally impossible (no `wire_version` bump: the value + shape is rejected, not newly accepted). +* `Host::invoke` rejects any argument tree containing `Memory` before dispatch + (`runtime.type`), so plugins, foreign kernels, math, file and print adapters + never see a handle. A future explicit byte-snapshot operation must copy bytes + into a managed value and is a separate ADR. +* `HostOp` is unchanged. `memory.new/size/grow` and index read/write are + interpreter operations, not capability-gated host effects. + +### 5. Error categories are stable and distinct + +| Situation | Code | +|---|---| +| Negative/unrepresentable offset, offset >= region length | `runtime.index` | +| Unknown, stale or cross-run handle; invalid page count (negative, > budget declaration arithmetic) | `runtime.memory` | +| Wrong operand type; byte outside `0..=255`; non-Memory index-assignment base | `runtime.type` | +| Declared max, region budget or total budget exceeded | `resource.exhausted` | + +No operation panics on malformed input or forged handles. `expect` is used only +after the same handle has been validated in the same call frame. + +### 6. Builtins are part of the language surface, not `HostOp`s + +`memory.new(initial_pages, max_pages) -> Memory`, +`memory.size(memory) -> Int`, `memory.grow(memory, pages) -> Int` are added to +`marqdo-ir::builtins::BUILTINS` and `spec/evaluation.md`. They are pure +(`effectful: false`): `pure` mode may use them, `strict` needs no new provider, +and no capability grant is consulted. + +### 7. Compatibility + +* Local assignment IR keeps its `target` string in Analysis v2; an index + target is an additive object (`{"kind":"index",...}`). Consumers that only + handle locals continue to work. +* Expression AST/statement serde gains an enum tag; this is a pre-1.0 internal + contract documented in `spec/expression.ebnf`. +* Existing Markdown AST goldens, value semantics, `List`/`Map`/`Table` + behaviour and all old builtins are unchanged. +* `Memory` is additive to the value type set. Old programs never produce it. + ABI wire consumers see a new rejected tag, not a renumbered existing tag. + +## Non-goals + +* No `Pointer` value, raw pointer, `free(region)`, registers or machine code. +* No index/field assignment for managed collections; no implicit mutation. +* No multi-threaded sharing, atomics or data-race guarantees. +* No serialization of live regions; no cross-process shared memory. +* No `ResourceHandle`, `Stream`, files, network or OS syscalls. Those are + CM-A05/CM-A06/CM-A09/CM-A11 and later RFC phases. +* No block-level safety claim for a user-written allocator; the runtime only + bounds a single access to its region. +* No allocator or stdlib API in P1; `--max-memory` only bounds the runtime. + +## Consequences + +* A `.mq.md` program can now build byte encoders, arenas, free lists and + virtual CPU register files without new Rust code. P2 (the Marqdo-written + stdlib and closure fixtures) becomes testable. +* The interpreter owns one `MemoryStore` per run; `run_with_limits` exposes + the budget. This is runtime state, not a second `HostOp` table. +* Trace gains `memory.new`, `memory.grow` and `memory.write` semantic events + without exposing handles or byte secrets. +* The remaining RFC decisions (CM-A04 and later) stay separate. In particular, + generic structural observation/construction, recoverable `Result`, table + algorithm migration and the resource/Stream ABI are **not** implemented by + this ADR. + +## Verification + +`docs/evidence/2026-11-20-memory-p1.md` records: + +* parser/AST tests for index and rejected member targets; +* runtime store tests for isolation, aliasing, zero-fill, atomic failed grow, + negative/huge offsets, forged/stale handles, byte range and budgets; +* semantic goldens `45`–`47` and runtime negatives `13`–`18`; +* wire rejection test and host-boundary rejection test; +* CLI example `examples/memory-buffer.mq.md` returning `141`; +* Analysis v2 index-target projection; +* `cargo fmt --check`, `cargo clippy -- -D warnings`, `cargo test --workspace`. diff --git a/docs/adr/0023-minimal-structure-observation-and-construction.md b/docs/adr/0023-minimal-structure-observation-and-construction.md new file mode 100644 index 0000000..5b30d05 --- /dev/null +++ b/docs/adr/0023-minimal-structure-observation-and-construction.md @@ -0,0 +1,147 @@ +# ADR-0023: Minimal structural observation and construction (RFC-CM CM-A04) + +Status: **Accepted** +Date: 2026-11-20 +Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`, +`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A04, P2), +ADR-0007, ADR-0022 +Specs: `spec/evaluation.md`, `spec/expression.ebnf`, `spec/analysis-v2.md` +Evidence: `docs/evidence/2026-11-20-structure-api-p2.md` + +## Context + +RFC-CM-0002 §3.5 identifies a closure gap: a `.mq.md` program can read a +`List`/`Map`/`Table` and print it through the host, but it cannot generally +**construct** the same structures dynamically or **observe** all of their +logical parts. That makes `format`, JSON/CSV parsers and table algorithms +impossible to write in `.mq.md` without adding a task-specific Rust function +for each algorithm. + +The current core already has: + +* `List` literals, index reads (including negative offsets), `长度`; +* `Map` record literals, `.field` reads, `长度`; +* `Text` concatenation, character indexing and `长度`; +* `Table` construction through `表(header, rows)`, `长度` (rows), display and + wire round-trip. + +It lacks: append/concat, dynamic map update/enumeration, Unicode scalar +conversion, and table column/row/cell projection. These are observation and +construction primitives, not algorithms. Sorting, filtering, joining, +serialising and parsing remain standard-library work. + +## Decision + +Add a small, namespaced, **pure** builtin set. No new Markdown node, keyword, +`HostOp`, capability or mutation semantics is introduced. All constructors +return **new** values; an input collection is never modified in place. This +keeps INV-02 and preserves existing `List`/`Map`/`Table` copy semantics. + +### 1. `List` + +| Signature | Semantics | +|---|---| +| `list.append(list: List, value) -> List` | New list with `value` at the end; the input list is unchanged. | +| `list.concat(left: List, right: List) -> List` | New list containing `left` then `right`; both inputs unchanged. | + +Neither aliases the other list's element storage: values are copied according +to their normal value semantics (`Memory` handles keep region identity). + +### 2. `Map` + +| Signature | Semantics | +|---|---| +| `map.keys(map: Map) -> List` | `Text` keys in canonical sorted order (`BTreeMap` order, the order used by `display`). | +| `map.has(map: Map, key: Text) -> Bool` | Whether the key exists. | +| `map.get(map: Map, key: Text) -> Value` | Stored value, or `None` when absent. Use `map.has` when `None` is a legal stored value. | +| `map.set(map: Map, key: Text, value) -> Map` | New map with the key set; the input map is unchanged. | +| `map.remove(map: Map, key: Text) -> Map` | New map without the key (missing key is a no-op); the input map is unchanged. | + +Keys are always `Text`; arbitrary-value keys are not introduced. `map.get` +returning `None` is a *lookup result*, not the language's recoverable-error +contract; a full `Result` convention remains CM-A05 and is not frozen here. + +### 3. `Text` + +`Text` remains a Unicode scalar-value sequence, not a byte array. These +primitives make the existing representation observable without introducing a +second text type or implicit UTF-8 conversion. + +| Signature | Semantics | +|---|---| +| `text.codepoints(text: Text) -> List` | Unicode scalar values (`Int`) in code-point order. | +| `text.from_codepoint(codepoint: Int) -> Text or None` | One-scalar `Text`; `None` for a surrogate or a value outside `0..=0x10FFFF`. | + +UTF-8 encoding/decoding is **not** a core primitive. It can be written in +`.mq.md` using bytes, `text.codepoints` and `text.from_codepoint`; this keeps +the byte/code-point distinction explicit. + +### 4. `Table` + +`Value::Table` stays the logical value; its physical `Vec>` layout +is not part of this contract. Indices are non-negative row/column coordinates +(unlike `List` negative indexing) because a table cell is a document +coordinate. Bounds failures are `runtime.index`. + +| Signature | Semantics | +|---|---| +| `table.columns(table: Table) -> List` | Header names (`Text`) in column order. | +| `table.row(table: Table, row: Int) -> List` | A new list of the row's cells in column order. | +| `table.cell(table: Table, row: Int, column: Int) -> Value` | The cell value. | + +No `table.set_cell`/`append_row` builtin is added. Rebuilding a table is a +value-level operation that can be written with `表(header, rows)` once list +construction exists; adding a Rust table algorithm would violate the closure +goal. + +### 5. Error and compatibility rules + +* Wrong collection/text/table type or key type → `runtime.type`. +* Table row/column outside the logical shape → `runtime.index`. +* `text.from_codepoint` never throws for an invalid scalar; it returns `None`. +* `map.get` never throws for a missing key; it returns `None`. +* Existing `List`/`Map`/`Table` behavior, display, wire encoding and Markdown + laziness are unchanged. +* The builtins are `effectful: false`, available in `pure`, `project` and + `strict`, and add no capability. They do not consume output budget. +* Names are frozen as written; aliases are not added. Namespaced calls use the + existing `Expr::Member` resolution path. + +## Non-goals + +* No `set_cell`, `append_row`, `sort`, `filter`, `join`, `group`, JSON/CSV or + other algorithm builtins; those belong in `.mq.md`. +* No mutable/aliased List or Map, no in-place mutation, no shared structural + sharing protocol. +* No arbitrary-value map keys, no ordered-map insertion semantics, no + duplicate-key policy change. +* No UTF-8/UTF-16 conversion primitive, no byte-array text type. +* No `Result`/`try`/exception syntax; CM-A05 stays open. +* No change to Table storage, GFM parsing, source spans or `ref` semantics. + +## Consequences + +* A `.mq.md` program can build maps and lists dynamically, enumerate keys, + project table cells and encode Unicode scalars. This unblocks a Marqdo- + written formatter, table filter/projection and a JSON/CSV parser without + per-algorithm Rust. +* The builtin table grows by 12 pure entries. The migration rule remains: + only observation/construction primitives may enter core; algorithms do not. +* Because constructors are value-returning, repeated `map.set` in a loop is + O(n²) for a BTreeMap clone. That is acceptable for P2 semantics; a + persistent/structural-sharing representation is a P6 performance decision, + not a semantic one. +* `map.get` returning `None` for absence is deliberate and documented; a + future CM-A05 `Result` convention must not silently reinterpret existing + programs. + +## Verification + +`docs/evidence/2026-11-20-structure-api-p2.md` records: + +* ADR/spec deltas and builtin metadata; +* semantic fixtures for value-semantics list/map operations, code-point + round-trips, table projection and invalid code points; +* negative fixtures for wrong types and table bounds; +* unit tests, `cargo fmt --check`, `cargo clippy -- -D warnings` and + `cargo test --workspace`. diff --git a/docs/adr/0024-recoverable-result-convention.md b/docs/adr/0024-recoverable-result-convention.md new file mode 100644 index 0000000..c3f6cfe --- /dev/null +++ b/docs/adr/0024-recoverable-result-convention.md @@ -0,0 +1,162 @@ +# ADR-0024: Recoverable result convention (RFC-CM CM-A05) + +Status: **Accepted** +Date: 2026-11-20 +Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`, +`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A05, P2/P3), +ADR-0023 +Specs: `spec/evaluation.md` +Evidence: `docs/evidence/2026-11-20-result-and-free-list-p2.md` + +## Context + +A standard library cannot be written in `.mq.md` if every expected condition +(no free block, invalid input, missing field, later end-of-file or a short +read) must abort the whole run. The core has structured `RuntimeError`s with +stable codes, but no way for an ordinary function to return a failure that the +caller can inspect and handle in a normal branch. RFC-CM-0002 §3.6 recommends a +`Result` data contract using existing `Map` values and control flow, without +`try`/`catch` or exception syntax. + +This ADR freezes the **data convention**, not a new runtime type. A separate +core `Result` value, algebraic types or automatic propagation would be a +larger change and is deliberately not introduced. + +## Decision + +### 1. Shape + +A recoverable result is an ordinary `Map`: + +```text +success: {ok: true, value: } +failure: {ok: false, code: , message: [, details: ]} +``` + +* `ok` is always present and always `Bool`. +* On success, `value` is present. It may itself be `None`, a `List`, a `Map`, + a `Table`, a `Memory` handle or any other value. +* On failure, `code` and `message` are present. `code` is a lowercase + dot-separated stable identifier; `message` is human-readable text. +* `details` is optional and, when present, is a `Map` of additional structured + facts. It must not contain host pointers, resource handles or secret bytes. +* No other fields are required. Consumers must tolerate additive fields. + +The constructors are ordinary `.mq.md` functions, for example: + +```text +## 成功 +`(值)` +*{ok: true, value: 值}* + +## 失败 +`(代码, 消息)` +*{ok: false, code: 代码, message: 消息}* +``` + +They are **not** builtins. A library may choose different local function names; +the map shape is the contract. + +### 2. What is recoverable, what stays fatal + +Recoverable (`Result`): + +* a user allocator has no fitting block (`alloc.no_space`); +* a parser rejects malformed input or hits an unexpected end + (`parse.invalid_syntax`, `parse.unexpected_eof`); +* a codec rejects an invalid sequence (`codec.invalid_sequence`); +* a lookup that uses an explicit result because `None` is a legal value; +* later, a resource reports EOF/partial/pending as a **success payload** with + its own byte counts, not as a fatal error (P3 contract). + +Fatal `RuntimeError` (unchanged): + +* violating a language invariant: wrong operand type, out-of-bounds managed + collection access, unknown/stale `Memory` handle, memory budget exhaustion, + division by zero, call arity, unknown symbol; +* a capability denial (authorization is not a recoverable business result); +* an internal invariant failure. + +A library must not convert a fatal invariant into an `Err` by catching it; +there is no catch mechanism and none is proposed here. It must validate inputs +before calling an operation whose failure is fatal, or rely on its own +pre-checks. + +### 3. Codes and namespaces + +`code` is stable within the library that defines it. Recommended prefixes: + +| Prefix | Owner | +|---|---| +| `alloc.*` | allocator libraries | +| `parse.*` | parsers | +| `codec.*` | byte/text codecs | +| `table.*` | table algorithms | +| `io.*` | stream wrappers (P3, when the resource contract exists) | + +Codes are compared as `Text`; a caller must not depend on the `message` text. +Libraries may add codes; removing or renaming one is a breaking change for that +library. + +### 4. EOF and partial I/O (forward contract) + +When the Stream/Resource API arrives (CM-A06), a successful read must +distinguish: + +```text +{ok: true, value: {count: Int, eof: Bool, pending: Bool}} +``` + +EOF is not an `Err`; a short read is not an `Err`; a connection reset or a +permission failure is. The exact stream payload is frozen by CM-A06, not by +this ADR. This ADR only reserves the principle. + +### 5. Why not a new type or syntax + +* A dedicated `Value::Result` would change the value model, wire format, + display, equality and every exhaustive `Value` match for a convention that + existing `Map` plus `ok` already expresses. +* `try`/`catch` would add a control-flow construct that bypasses ordinary + branches, contradicts the RFC's preference for `Result` values and needs new + grammar, scope and error rules. +* Ordinary `Map` results are visible to `type`, `length`, `map.get` and wire + encoding today, and can be built by `.mq.md` libraries without a new ABI. + +## Compatibility + +* Additive: existing builtins and programs are unchanged. +* `map.get` returning `None` for a missing key remains a lookup convention, + not a `Result`; callers that need to distinguish a stored `None` use + `map.has` (ADR-0023). +* Runtime error categories and codes are unchanged. +* No wire version bump: a `Result` is an ordinary `Map` already covered by the + wire format. + +## Non-goals + +* No automatic `?`/propagation, exceptions, panic recovery or stack unwinding. +* No `Result` type, generics, pattern matching or exhaustive `Ok`/`Err` + typing. +* No change to `memory.grow` or other core operations from fatal errors to + `Result`; a library may pre-check and return `Err` instead. +* No resource/stream implementation; CM-A06 remains open. +* No promise that every existing library will adopt the convention. + +## Consequences + +* `.mq.md` algorithms can return and branch on failure without aborting the + run. This is a prerequisite for a Marqdo-written allocator, parser and + eventually `io.read_line`. +* Because the convention is untyped, a caller can inspect the wrong shape; + `类型(result)`, `map.has(result, "ok")` and the frozen field names are the + checks. A future typed-value ADR may improve this without breaking the data + shape. +* The allocator fixture in `54`/`55` demonstrates success and failure maps and + ordinary branching, with no Rust algorithm. + +## Verification + +`docs/evidence/2026-11-20-result-and-free-list-p2.md` records a `.mq.md` +first-fit/free-list allocator that returns `Result` maps, reuses freed blocks, +splits free blocks, coalesces address-adjacent free blocks and returns +`alloc.no_space` without aborting the run. diff --git a/docs/adr/0025-minimal-resource-stream-abi.md b/docs/adr/0025-minimal-resource-stream-abi.md new file mode 100644 index 0000000..bad6239 --- /dev/null +++ b/docs/adr/0025-minimal-resource-stream-abi.md @@ -0,0 +1,173 @@ +# ADR-0025: Minimal Resource/Stream ABI and byte transfer through `Memory` (RFC-CM CM-A06) + +Status: **Accepted** +Date: 2026-11-20 +Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`, +`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A06, CM-A09, +CM-A11, P3), ADR-0022, ADR-0024 +Specs: `spec/evaluation.md`, `spec/security.md`, `spec/value-wire.md` +Evidence: `docs/evidence/2026-11-20-resource-stream-p3.md` + +## Context + +P1/P2 give `.mq.md` managed values, addressable `Memory`, structure primitives, +a recoverable `Result` convention and a substantial set of in-language +algorithms. Nothing yet lets a standard-library function perform a real byte +transfer to the outside world without a dedicated `HostOp` per high-level +function. + +CM-A06 asks for a stable, authorized, versioned resource boundary: obtain a +granted resource, call the operations it promises, release it. CM-A09 says to +extend the existing host/ABI rather than build a second plugin system. CM-A11 +says strict must stay fail-closed until a provider is actually verified. + +This ADR freezes a **first, deliberately small slice**: + +* one opaque host-issued `ResourceHandle` value; +* `resource.open/read/write/close` as pure language builtins, not new `HostOp` + variants; +* byte transfer through `Memory`, never through host pointers; +* stdio `stdout` write and project-file **read** only; +* `Result` maps for EOF/closed/write counts, fatal errors for invalid handle + or capability denial. + +File write, directories, network, processes, TLS, asynchronous I/O and a +generic provider registry remain open. + +## Decision + +### 1. Opaque handle + +`Value::ResourceHandle(ResourceHandle)` is a host-issued opaque identity with +private `(run_id, resource_id, generation)` fields and a hidden constructor. +It is not `Serialize`/`Deserialize`, is rejected by the wire, is never +synthesized from a number or a JSON value, and renders as `Resource` in +`display`/`repr` without exposing the id. + +It is a **separate domain** from `Memory`: a handle is not indexable, not +byte-addressable and not a capability token the user can manufacture. + +### 2. First resource kinds + +| Kind | Descriptor | Rights | Availability | +|---|---|---|---| +| `stdio` | `stdout` | `write` | `pure` and `project` when the console grant is present; `strict` rejects until verified | +| `file` | project-relative path | `read` | `project` with an `fs_read` grant | + +`file` streams are opened by reading a bounded regular file (existing +`ProjectFs::MAX_READ_BYTES = 8 MiB` limit) into host-owned bytes; `read` +returns successive chunks and `eof` when the position reaches the end. This is +honest about being a first provider: it does not expose a host file descriptor, +does not support seeking, writing, directories, network, processes, devices or +paths outside the project root. + +Any other kind/descriptor/right combination is an explicit +`resource.unsupported`/`permission.denied` diagnostic, never a silent fallback. + +### 3. Builtins + +```text +resource.open(kind: Text, descriptor: Text, rights: List) -> Resource +resource.read(handle: Resource, memory: Memory, offset: Int, max_bytes: Int) + -> {ok:true, value:{count:Int, eof:Bool}} | {ok:false, code, message} +resource.write(handle: Resource, memory: Memory, offset: Int, length: Int) + -> {ok:true, value:{count:Int}} | {ok:false, code, message} +resource.close(handle: Resource) + -> {ok:true, value:None} | {ok:false, code, message} +``` + +* `resource.open` is effectful and capability-checked at execution time. +* `resource.read` validates `offset` and the remaining buffer **before** asking + the provider, so a failed range check cannot consume stream data. It caps the + request to the remaining region, writes the returned bytes into `Memory` and + returns the actual count and EOF flag. EOF is a **success** with `eof:true`. +* `resource.write` copies bytes out of `Memory` only for the duration of the + call. A short write is reported as a count; the caller loops. It never + stores a host pointer into a region that could later grow. +* `resource.close` closes the host slot. Reading/writing/closing an already + closed handle returns `{ok:false, code:"resource.closed", ...}` (recoverable). + A forged, cross-run or stale handle is `runtime.resource` (fatal). +* All four are pure interpreter operations over host trait methods; they add no + `HostOp` variant. Invalid capacity/bounds remains `runtime.index`; wrong + operand types remain `runtime.type`. + +### 4. Authorization and modes + +* Authorization is the intersection of the requested operation, the effective + `Capabilities` grant and the mode: + * `stdio/stdout/write` requires `console` (the same grant `打印` uses); + * `file/.../read` requires an `fs_read` pattern match and project-root path + validation (`..`, absolute paths and symlink escapes rejected); +* `pure`: stdio stdout is allowed because console is already a pure-mode + console grant; **file resources are refused by the mode check**, even if a + caller constructs a `Capabilities` set with a matching `fs_read` pattern. + Network/process resources do not exist in this slice. +* `project`: stdio stdout and explicitly granted project-file reads. +* `strict`: **every** `resource.open` is refused with `permission.denied`. + Strict L1 is unchanged and no new provider is silently enabled. A future ADR + must add real OS-level evidence before strict accepts streams. +* Manifest capability requests are not grants; only the operator-provided + effective set is consulted. + +### 5. Errors, budgets and tracing + +New stable categories: + +| Code | Meaning | +|---|---| +| `runtime.resource` | forged/cross-run/stale handle, invalid resource state | +| `resource.closed` | read/write/close on an already closed handle (recoverable) | +| `resource.unsupported` | kind/descriptor/rights not implemented in this slice | + +`resource.write` to stdout charges the existing output budget and appends to +the host output buffer exactly like `打印`; capability denial is +`permission.denied`. Trace records `resource.open`, `resource.read`, +`resource.write` and `resource.close` with kind/count/eof metadata, never raw +file bytes or handles. + +### 6. Wire and boundary + +* `to_wire(Resource)` emits `{"type":"resource"}` only; decoding it is a hard + error (same rule as `Memory`). +* The generic `Host::invoke` boundary rejects arguments containing a resource + handle, so plugins/foreign kernels/file adapters never receive one. +* Plugin output cannot mint a handle: the constructor is host-only and the + value is not decodable from plugin wire. + +### 7. Non-goals + +* No `print` migration in this ADR; that is CM-A07 and a later increment. +* File **write** is added by [ADR-0026](0026-buffered-project-file-write-streams.md), + not this decision; this ADR remains read-only for files. +* No file write, directory listing, rename, metadata, network, DNS, TLS, + process, clock, random or device operations. +* No generic third-party provider registration; CM-A09 remains open. +* No zero-copy, async, deadlines per stream, seek or shared memory. +* No strict-mode resource support and no claim that host-in-process I/O is a + sandbox. + +## Compatibility + +* Additive value variant; old programs never produce `Resource`. +* New builtins only; existing `fs.read/write`, `打印`, plugin and foreign + behavior is unchanged. +* No wire version bump: the new tag is rejected, not accepted. +* New error categories are additive; existing categories/codes are unchanged. + +## Consequences + +* `.mq.md` can implement `write_text`-style output by encoding text into + `Memory` (fixture 59) and calling `resource.write`, and can implement + chunked file readers with `resource.read`/`Result`/EOF. +* The remaining high-level I/O surface can grow in the standard library without + a new `HostOp` per function. +* The resource table is host state per run; closed slots are not reused in this + slice, so generation checks are straightforward. +* Strict remains conservative; `.mq.md` resource code runs in `pure` (stdout) + or `project` (file read) only. + +## Verification + +`docs/evidence/2026-11-20-resource-stream-p3.md` records unit tests for grants, +mode refusal, traversal rejection, forged/closed handles, output budget, +file chunking/EOF, wire rejection and a `.mq.md` stdout fixture. diff --git a/docs/adr/0026-buffered-project-file-write-streams.md b/docs/adr/0026-buffered-project-file-write-streams.md new file mode 100644 index 0000000..1cc8e14 --- /dev/null +++ b/docs/adr/0026-buffered-project-file-write-streams.md @@ -0,0 +1,115 @@ +# ADR-0026: Buffered project-file write streams (CM-A06 extension) + +Status: **Accepted** +Date: 2026-11-20 +Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` +(CM-A06, CM-A09, CM-A11), ADR-0024, ADR-0025 +Specs: `spec/evaluation.md`, `spec/security.md` +Evidence: `docs/evidence/2026-11-20-resource-file-write-p3.md` + +## Context + +ADR-0025 froze the first resource ABI slice with `stdio/stdout` write and +project-file **read**. A standard library cannot implement file copy, file +format writers or "write then verify" without a write-capable file resource. +The RFC's P3 target is explicit `open/read/write/close` for project files in +`project` mode; strict must remain fail-closed. + +Direct per-write file I/O would expose partial writes and OS error timing to +every caller and would make it easy to leave a half-written project file. This +ADR adds the smallest honest write path: an in-memory buffer per file writer, +flushed through the existing project path policy on `close`. + +## Decision + +### 1. Provider shape + +Add a third stream shape to the host resource table (ADR-0025): + +```text +resource.open("file", "", ["write"]) -> Resource +``` + +* `project` mode only, with an `fs_write` glob grant matching the normalised + relative path. +* `pure` still refuses file resources by mode check; `strict` still refuses + every `resource.open` (CM-A11 fail-closed). +* Absolute paths, `..`, symlink escapes and non-regular targets are rejected by + `ProjectFs` exactly like the existing `fs.write`. +* A writer owns an in-memory byte buffer. `resource.write` appends bytes and + returns `{ok:true, value:{count}}`; it does **not** touch the filesystem. +* `resource.close` flushes the buffer through `ProjectFs::write_bytes` and then + marks the handle closed. If the flush fails, the handle stays open with its + buffer intact and the error is reported; the caller may retry or close again. +* If the program exits without closing, no file is written. This is a + deliberate "no flush on drop" rule: a crash or a missing `close` must not + silently publish a partial file. +* `resource.write` is rejected on a file reader; `resource.read` is rejected on + a file writer; stdout remains write-only. Wrong-direction use is + `resource.unsupported`. +* A writer buffer is capped at the existing 8 MiB project read/write ceiling + (`MAX_READ_BYTES`). Exceeding it is `resource.exhausted` and the append is + atomic: the buffer and file are unchanged. + +### 2. Rights and modes + +| Kind | Descriptor | Rights | Mode | Grant | +|---|---|---|---|---| +| `file` | project-relative path | `["read"]` | project | `fs_read` glob | +| `file` | project-relative path | `["write"]` | project | `fs_write` glob | +| `stdio` | `stdout` | `["write"]` | pure/project | `console` | + +Mixed rights (`["read","write"]`) and other kinds/rights are +`resource.unsupported` in this slice. There is no truncate/append/seek flag: +every writer starts empty and replaces the file on flush. + +### 3. Errors, budgets and tracing + +* `resource.closed` for read/write/close on a closed handle (recoverable). +* `runtime.resource` for forged/cross-run/stale handles (fatal). +* `resource.exhausted` for a buffer over the byte ceiling. +* `permission.denied` for a missing mode/grant or a path policy failure. +* `resource.unsupported` for a direction/rights combination not implemented. +* `resource.write` on a file writer does not charge the program output budget; + the buffer is bounded by the file ceiling. `resource.close` performs one + `ProjectFs::write_bytes` call and records `resource.close` with the byte + count, never the contents. +* No partial file is visible on a failed close because the write is a single + `std::fs::write` to the validated project path (the same atomicity level as + the existing `fs.write`; no temp-file rename is claimed). + +### 4. Explicit non-goals + +* No append/truncate/seek flags, no random access, no directory or metadata + operations. +* No file write in pure or strict mode. +* No network/process/device/clock/random resources. +* No generic provider registry (CM-A09), no async/nonblocking I/O. +* No claim that `ProjectFs::write_bytes` is crash-atomic; it is a single + validated write, not a transactional replace. +* No CM-A07 `print` migration in this ADR. + +## Compatibility + +* Additive to ADR-0025: existing `file` read and `stdio` write behavior is + unchanged; `file` write was previously `resource.unsupported`. +* No new `Value` variant, `HostOp`, Markdown node or wire shape. +* Error categories/codes are unchanged; the new `resource.exhausted` diagnostic + reuses the existing stable `resource.exhausted` code family. + +## Consequences + +* `.mq.md` can now implement file copy and bounded format writers by opening a + read stream and a write stream, transferring bytes through `Memory`, and + closing both. This is C09's core shape without a `copy_file` Rust builtin. +* Buffered close gives deterministic "all or nothing within one write call" + behavior and keeps OS write errors at a point where the caller still has a + Result path. +* Large files require chunked writes; the 8 MiB ceiling is the first-slice + limit, not a permanent design. + +## Verification + +`docs/evidence/2026-11-20-resource-file-write-p3.md` records host unit tests +(grants, pure/strict refusal, traversal, buffer ceiling, retry-after-failure), +CLI tests for write-then-read and missing grants, and the full workspace gate. diff --git a/docs/adr/0027-print-migration-compat-a.md b/docs/adr/0027-print-migration-compat-a.md new file mode 100644 index 0000000..00c2ee4 --- /dev/null +++ b/docs/adr/0027-print-migration-compat-a.md @@ -0,0 +1,103 @@ +# ADR-0027: CM-A07 print-migration staging and the Compat-A `.mq.md` stdlib prototype + +Status: **Accepted (Compat-A only; Compat-B/C pending)** +Date: 2026-11-20 +Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` +(CM-A07, CM-A10, P3), ADR-0023, ADR-0024, ADR-0025, ADR-0026 +Specs: `spec/evaluation.md` +Evidence: `docs/evidence/2026-11-20-stdlib-print-compat-a.md` + +## Context + +RFC-CM-0002 §5.4 and CM-A07 require `打印` to stop being a hidden Rust +`Value::display()` path. The migration has three compatibility stages: + +* **Compat-A**: a real `.mq.md` print implementation exists and is callable + through explicit imports; the legacy builtin remains the default. +* **Compat-B**: the `print/打印` name routes to the stdlib implementation + (or an alias table maps it), while old behavior is comparison-tested. +* **Compat-C**: the legacy `HostOp::Print` formatting path is removed from the + public builtin implementation. + +This ADR freezes the staging policy and implements **Compat-A only**. It does +not change `打印` resolution, does not remove `HostOp::Print`, and does not +claim CM-A07 complete. + +## Decision + +### 1. Stage policy + +* A stage is only complete when its comparison evidence exists: + * A: explicit stdlib functions exist and their output is compared with the + builtin for the covered value set; + * B: name resolution tests prove a single implementation is selected and old + callers still behave; + * C: all old goldens pass with no `HostOp::Print` formatting path. +* Stages are additive; no stage may silently make a second implementation + reachable for the same name. +* `strict` behavior is unchanged by stdlib-only code; a stdlib print still + needs the console/resource grant. + +### 2. Compat-A implementation + +Add a normal `.mq.md` module package under `examples/stdlib/`: + +```text +examples/stdlib/format.mq.md # 显示 / 列表显示 / 映射显示 / 表格显示 / 转义文本 +examples/stdlib/text.mq.md # utf8编码 (Memory bytes) +examples/stdlib/io.mq.md # 写文本 / 写值 (resource.open stdout + write/close) +examples/stdlib/demo.mq.md # side-by-side comparison with 打印 +``` + +`io.写值(值)` = `format.显示(值) + "\n"` → `text.utf8编码` → `Memory` → +`resource.open("stdio","stdout",["write"])` → `resource.write` → `close`. +This uses only the public core/resource contracts; it is ordinary imported +`.mq.md` code, not a new builtin. + +Covered value types: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, `List`, +`Map`, `Table`. Formatting matches the builtin `Value::display()` for the +checked corpus, including Text quoting/escaping inside collections, canonical +map key order and table layout. + +**Not covered yet**: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The +language currently has no public observation for the module/name/handle +identity needed to reproduce `Value::display()`/`repr()` for those types. +Adding such observation would be a CM-A04 extension and is out of scope. + +### 3. Compatibility rules + +* `打印` remains the default builtin in this slice. `io.写值` is only reachable + through an explicit import alias; there is no implicit module, bundled stdlib + resolution or global name shadowing. +* No `HostOp` is added or removed. Both paths write through the console/output + budget; the stdlib path additionally goes through the resource ABI. +* Stdlib modules are ordinary Markdown documents: no special parser, no + embedded Rust formatter, no direct formatting syscall. +* A future Compat-B ADR must define name precedence, bundled module lookup, + stdlib version/digest (CM-A10) and failure behavior; this ADR does not. + +### 4. Non-goals + +* No `print` alias routing, no removal of `HostOp::Print`. +* No bundled stdlib install/version/digest or offline distribution. +* No formatting of Formula/Code/FunctionRef/PluginHandle. +* No locale-aware, width-aware or float-format changes. +* No change to Markdown rendering, `strict`, or the wire/ABI. +* No claim that the examples directory is the shipped standard library. + +## Consequences + +* The target call chain (`format.display` → `utf8_encode` → `write_all` → + `resource.write`) is exercised end to end in `.mq.md` and can be compared to + the builtin. +* The missing observation for `Formula/Code/FunctionRef/PluginHandle` is now an + explicit, documented blocker for Compat-B rather than an implicit assumption. +* Because the stdlib is not bundled, ordinary users still get the builtin; the + prototype does not yet reduce the runtime's formatting surface. + +## Verification + +`docs/evidence/2026-11-20-stdlib-print-compat-a.md` records the side-by-side +demo (`io.写值` vs `打印` for a nested list and a table), the CLI test asserting +byte-identical lines, and the full workspace gate. It also records the covered +and uncovered value types. diff --git a/docs/architecture/README.md b/docs/architecture/README.md new file mode 100644 index 0000000..111eabe --- /dev/null +++ b/docs/architecture/README.md @@ -0,0 +1,38 @@ +# Architecture RFCs: compute machine and stdlib migration + +Status: **Draft RFCs, partially implemented**. This page is an index; the +normative RFC text stays at its reviewable path under `docs/`. + +| Document | Status | Implemented part | +|---|---|---| +| [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). | +| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict refuses all opens; ADR-0027 Compat-A `.mq.md` print prototype, builtin still default). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser for arrays, objects, strings with escapes, integers/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68). No separate stdlib repo, JSON floats, network/generic providers, P4 Table migration, P5 providers or P6 performance work. | + +The RFCs are architecture constraints, not evidence that a feature exists. The +implementation status lives in `docs/STATUS.md`; frozen behavior is in +`spec/`; decisions are in `docs/adr/`. + +## Decisions already frozen + +| RFC item | ADR | State | +|---|---|---| +| CM-A01 Memory identity/lifetime | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted | +| CM-A02 page size, growth, budgets | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted | +| CM-A03 place assignment and evaluation order | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted | +| CM-A04 minimal structure observation/construction | [0023](../adr/0023-minimal-structure-observation-and-construction.md) | Accepted | +| CM-A05 recoverable `Result` convention | [0024](../adr/0024-recoverable-result-convention.md) | Accepted | +| CM-A06 minimal Resource/Stream ABI | [0025](../adr/0025-minimal-resource-stream-abi.md) | Accepted (first slice) | +| CM-A06 buffered project-file write | [0026](../adr/0026-buffered-project-file-write-streams.md) | Accepted | +| CM-A07 print migration staging / Compat-A | [0027](../adr/0027-print-migration-compat-a.md) | Accepted (Compat-A only) | +| CM-A07 Compat-B/C, stdlib packaging, Table storage, generic providers, strict resources | — | Not decided; do not implement by implication | + +## What a future phase must not do + +* add a top-level CLI command or Markdown keyword for memory, allocation or I/O; +* add a `HostOp` for every new stdlib function; +* turn managed `List`/`Map`/`Table` into aliased mutable storage; +* expose a host pointer, let a handle cross the wire, or claim block-level + safety for user allocators; +* make `strict` accept an unverified provider by default; +* claim an RFC phase is complete without the closure and negative evidence its + phase gate requires. diff --git a/docs/evidence/2026-11-20-memory-p1.md b/docs/evidence/2026-11-20-memory-p1.md new file mode 100644 index 0000000..c80786e --- /dev/null +++ b/docs/evidence/2026-11-20-memory-p1.md @@ -0,0 +1,130 @@ +# RFC-CM P1 evidence: addressable Memory and place assignment + +Date: 2026-11-20 (Asia/Shanghai) +Repository: `marqdo-cli` +Baseline commit: `a113d894962d8c00eb02efb42e0e73f8b084a1fa` (`docs: bind +README/COMPATIBILITY/CHANGELOG to the convergence SHA`) +Working tree: RFC-CM P1 changes, **not committed** at the time of this report. +Decision record: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md) +Scope: RFC-CM-0001 R1 / RFC-CM-0002 P1 only. P2–P6 are **not** implemented. + +## Environment + +* Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0. +* `PATH=/root/.cargo/bin:$PATH` +* `CARGO_HOME=/root/work/marqdo-cli/.cache/cargo` +* `CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target` +* No GitHub credentials in this checkout; this is **local** evidence, not a + GitHub Actions result. Windows/macOS lanes were not run. + +## Static and workspace gate + +```text +cargo fmt --all -- --check PASS +cargo clippy --workspace --all-targets -- -D warnings PASS +cargo test --workspace --locked PASS + 47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests) +``` + +The 2 ignored tests are the existing `marqdo-plugin-sdk` doc-tests, not new +memory tests. + +## Focused evidence + +```text +cargo test -p marqdo-runtime --lib memory + 8 passed; 0 failed + regions_are_isolated_and_handles_share_identity + create_is_zero_filled_and_reports_pages + grow_keeps_old_bytes_and_zero_fills_new_pages + failed_grow_does_not_change_region_or_budget + invalid_offsets_are_structured_errors + create_validates_page_counts_and_budget + forged_and_stale_handles_are_rejected + page_arithmetic_never_overflows + +cargo test -p marqdo-runtime --test interpreter memory + 3 passed; 0 failed + memory_region_is_shared_by_handle_identity + memory_handle_cannot_reach_the_host_boundary + memory_budget_is_enforced_without_changing_the_region + +cargo test -p marqdo-value --lib memory + 2 passed; 0 failed + memory_display_never_leaks_the_runtime_handle + memory_handles_are_not_transferable + +cargo test -p marqdo-expression --lib memory_index + 1 passed; 0 failed + memory_index_assignment_parses_as_a_place + +cargo test -p marqdo-cli \ + --test conformance_semantics --test negative \ + --test analysis_trace --test examples_r6 + semantic_goldens_match_end_to_end_execution ok + runtime_negative_goldens ok + analysis_v2_projects_memory_place_targets_additively ok + addressable_memory_example_runs_through_the_cli ok + cli_memory_budget_override_fails_closed ok + (20 tests total in these four binaries, 0 failed) +``` + +## End-to-end CLI + +```text +$ marqdo run examples/memory-buffer.mq.md --json +{"success": true, "value": {"type": "int", "decimal": "141"}, ...} + +$ marqdo run examples/memory-buffer.mq.md --max-memory 0 --json +{"success": false, "error": {"code": "resource.exhausted", ...}, ...} +``` + +The example creates one 64 KiB page with a declared maximum of four pages, +writes two bytes through two names that share one region, and returns +`42 + 99 = 141`. + +## Fixtures + +Semantic goldens: + +* `tests/conformance/semantics/45-memory-buffer.mq.md` — shared identity + (`另一个名字 = 区域`) and two byte writes. +* `tests/conformance/semantics/46-memory-grow-zero-fill.mq.md` — zero-page + create, `grow` returning the old count, old bytes preserved, new byte zero, + `memory.size`. +* `tests/conformance/semantics/47-memory-isolation-and-alias.mq.md` — two + regions do not alias at the same offset; handle equality is identity. + +Runtime negatives: + +| Fixture | Expected code | +|---|---| +| `13-memory-out-of-bounds` | `runtime.index` | +| `14-memory-byte-range` | `runtime.type` | +| `15-index-assign-non-memory` | `runtime.type` | +| `16-memory-huge-offset` | `runtime.index` | +| `17-memory-grow-beyond-max` | `resource.exhausted` | +| `18-memory-negative-pages` | `runtime.memory` | + +## What this does not prove + +* No `.mq.md` stdlib repository or allocator/closure fixture yet (RFC-CM P2). +* No `Resource`, `Stream`, byte-transfer ABI, stdlib `print` migration or + `Table` storage abstraction (P3–P5). +* No Windows/macOS evidence and no remote CI URL. +* No claim of block-level memory safety for a user-written allocator; the + runtime only bounds each access to its region. +* No claim that `strict` accepts new I/O: memory is pure computation and no + `HostOp` or provider was added. + +## Reproduce + +```bash +export PATH=/root/.cargo/bin:$PATH +export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo +export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target +cargo fmt --all -- --check +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace --locked +cargo run -p marqdo-cli -- run examples/memory-buffer.mq.md --json +``` diff --git a/docs/evidence/2026-11-20-memory-p2-closure.md b/docs/evidence/2026-11-20-memory-p2-closure.md new file mode 100644 index 0000000..3261667 --- /dev/null +++ b/docs/evidence/2026-11-20-memory-p2-closure.md @@ -0,0 +1,74 @@ +# RFC-CM P2 partial evidence: `.mq.md`-written codec, allocator and list + +Date: 2026-11-20 (Asia/Shanghai) +Repository: `marqdo-cli` +Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes. +Related: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md), +[P1 evidence](2026-11-20-memory-p1.md), +[RFC-CM-0002 §7.2](../../docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md). +Scope: **partial P2 only**. The RFC-CM-0002 P2 gate is **not** passed. + +> Historical slice: this report predates ADR-0023 (CM-A04 structure API), +> ADR-0024 (CM-A05 Result), the coalescing allocator (fixture 55) and the +> binary tree/CSV fixtures (56–57). Its "what remains" list is superseded by +> [`2026-11-20-structure-api-p2.md`](2026-11-20-structure-api-p2.md) and +> [`2026-11-20-result-and-free-list-p2.md`](2026-11-20-result-and-free-list-p2.md); +> the P2 gate itself is still not passed. + +## What was added without Rust algorithm code + +Three semantic fixtures are ordinary `.mq.md` documents. They call only +language builtins (`memory.new/size/grow`), indexing, arithmetic, functions, +branches and loops. No Rust crate was changed for the algorithms themselves. + +| Fixture | `.mq.md` subject | Result | +|---|---|---| +| `tests/conformance/semantics/48-memory-bump-allocator.mq.md` | big-endian u32 encode/decode and a 4-byte-header bump allocator; allocates 8, 16, 8 bytes and writes a byte | `[4, 12, 28, 36, 255]` | +| `tests/conformance/semantics/49-memory-linked-list.mq.md` | allocates three 8-byte nodes (`value:u8`, `next:u32`), links them and traverses the list | `[4, 12, 20, 60]` | +| `tests/conformance/semantics/50-memory-allocation-failure-atomic.mq.md` | a too-large allocation returns `None` without moving the header; the next allocation still starts at 4 | `[None, 4, 4, 65536]` | + +Observed CLI results (all `success: true`): + +```text +48: [4, 12, 28, 36, 255] +49: [4, 12, 20, 60] +50: [None, 4, 4, 65536] +``` + +`cargo test -p marqdo-cli --test conformance_semantics` passes with these three +fixtures included. + +## Why this is not the P2 gate + +RFC-CM-0002 §9/P2 requires C01–C06 and C11 to pass in an environment with +Foreign/native plugins disabled, and requires a minimal API for dynamic +List/Map/Table construction and observation plus recoverable errors. This +slice does **not** provide: + +* duplicate-free/allocator adversarial tests (C01 now has a first-fit + free-list allocator with reuse, splitting and coalescing, fixture 55); +* graph/hash-table structures (C02 now has a singly linked list and a binary + search tree, fixtures 49 and 56, but no graph/hash table); +* general List/Map/Table construction, mutation-free rebuilding or key + enumeration (CM-A04 is undecided); +* a `Result`/recoverable-error contract for parser/IO/EOF/partial conditions + (CM-A05 is undecided); +* an independent `marqdo-stdlib` repository, version/digest contract or + frozen-CLI-binary closure lane (the RFC §10.5 experiment). + +It also does not add a stdlib API to the CLI; the fixtures are conformance +documents, not a shipped standard library. + +## Reproduce + +```bash +export PATH=/root/.cargo/bin:$PATH +export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo +export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target +for f in 48-memory-bump-allocator 49-memory-linked-list \ + 50-memory-allocation-failure-atomic; do + cargo run -q -p marqdo-cli -- run \ + tests/conformance/semantics/$f.mq.md --json +done +cargo test -p marqdo-cli --test conformance_semantics +``` diff --git a/docs/evidence/2026-11-20-resource-file-write-p3.md b/docs/evidence/2026-11-20-resource-file-write-p3.md new file mode 100644 index 0000000..17f5a9c --- /dev/null +++ b/docs/evidence/2026-11-20-resource-file-write-p3.md @@ -0,0 +1,104 @@ +# RFC-CM CM-A06 extension evidence: buffered project-file writes + +Date: 2026-11-20 (Asia/Shanghai) +Repository: `marqdo-cli` +Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes. +Decision record: [ADR-0026](../adr/0026-buffered-project-file-write-streams.md) +Related: [ADR-0025](2026-11-20-resource-stream-p3.md), +[STATUS §5](../STATUS.md). +Scope: project-file **write** extension only; no network/process/directory. + +## Environment and gate + +Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0. + +```text +cargo fmt --all -- --check PASS +cargo clippy --workspace --all-targets -- -D warnings PASS +cargo test --workspace --locked PASS + 406 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests) +``` + +No GitHub CI is available from this checkout; this is local evidence. + +## Implemented behavior + +`resource.open("file", "", ["write"])`: + +* project mode only; requires an `fs_write` glob grant and the same + `ProjectFs` path policy (reject absolute, `..`, symlink escapes); +* pure mode refuses file resources by mode; strict refuses every resource open; +* the writer owns an in-memory buffer; `resource.write` appends and returns + `{ok:true, value:{count}}` without touching the filesystem; +* `resource.close` flushes through `ProjectFs::write_bytes` and then marks the + handle closed; a failed flush leaves the handle open with its buffer; +* an unclosed writer never creates the file (no flush on drop); +* buffer ceiling is the existing 8 MiB file limit; an oversized append is + `resource.exhausted` and leaves both buffer and file unchanged; +* `resource.read` on a writer, and `resource.write` on a reader, are + `resource.unsupported`; +* no new `Value`, `HostOp`, Markdown node or wire shape. + +## Tests + +### Host integration (`crates/marqdo-host/tests/resource_streams.rs`, 9) + +```text +pure_mode_writes_stdout_through_a_resource +stdout_write_obeys_the_output_budget_atomically +strict_mode_refuses_every_resource_open +resource_handles_cannot_cross_the_generic_host_boundary +pure_mode_refuses_file_resources_even_with_a_grant +project_file_writes_buffer_until_close_and_require_a_grant +file_write_buffer_has_a_hard_ceiling +project_file_reads_are_chunked_and_require_a_grant +traversal_and_unsupported_rights_are_refused +``` + +The write test asserts the file does not exist before `close`, appears with +`abcdef` after `close`, closed read/close return `resource.closed`, and the +same open without `fs_write` is `permission.denied`. + +### CLI integration (`crates/cli/tests/resource_streams.rs`, 4) + +```text +project_file_stream_writes_on_close_then_reads_back + run index.mq.md --allow-write out.bin --allow-read out.bin --json + write count 3, close ok, read count 3 eof true, bytes 65/66/67 + out.bin on disk = "ABC" + +file_copy_is_written_in_mqmd_through_chunked_streams + .mq.md opens data.txt for read and out.bin for write, + loops resource.read(16) / resource.write(count) until eof, + closes both; total 6; out.bin on disk = "abcdef" + (C09 file copy without a Rust copy_file builtin) + +project_file_stream_reads_chunks_into_memory +file_stream_requires_a_read_grant +``` + +### Host table unit tests (`crates/marqdo-host/src/resource.rs`, 3) + +Chunked EOF, forged/closed handles, stdout-not-readable and file-not-writable +(read-only direction) remain covered; the writer uses the same handle checks. + +## What this does not prove + +* No append/truncate/seek, directories, metadata, network, process, clock, + random, device or generic provider registry. +* No crash-atomic replace: `ProjectFs::write_bytes` is one validated write, not + a temp-file rename. +* No file write in `pure` or `strict`. +* No CM-A07 stdlib `print` migration. +* No Windows/macOS evidence and no remote CI URL. + +## Reproduce + +```bash +export PATH=/root/.cargo/bin:$PATH +export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo +export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target +cargo test -p marqdo-host --test resource_streams +cargo test -p marqdo-cli --test resource_streams +cargo test --workspace --locked +``` diff --git a/docs/evidence/2026-11-20-resource-stream-p3.md b/docs/evidence/2026-11-20-resource-stream-p3.md new file mode 100644 index 0000000..efcd878 --- /dev/null +++ b/docs/evidence/2026-11-20-resource-stream-p3.md @@ -0,0 +1,144 @@ +# RFC-CM CM-A06 evidence: minimal Resource/Stream ABI + +Date: 2026-11-20 (Asia/Shanghai) +Repository: `marqdo-cli` +Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes. +Decision record: [ADR-0025](../adr/0025-minimal-resource-stream-abi.md) +Related: [P1](2026-11-20-memory-p1.md), +[P2 result/allocator](2026-11-20-result-and-free-list-p2.md), +[STATUS §5](../STATUS.md). +Scope: **first CM-A06 slice only**. Not the full P3 migration. + +## Environment and gate + +Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0. + +```text +cargo fmt --all -- --check PASS +cargo clippy --workspace --all-targets -- -D warnings PASS +cargo test --workspace --locked PASS + 402 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests) +``` + +No GitHub CI is available from this checkout; this is local evidence. + +## Implemented slice + +### Value and wire + +* `Value::Resource(ResourceHandle)` is an opaque host-issued handle; fields are + private, the constructor is host-only, and `display`/`repr` render only + `Resource`. +* `to_wire(Resource)` emits only `{"type":"resource"}`; `from_wire` rejects the + tag. A `Resource` inside a list/map/table is equally rejected on decode. +* The generic `Host::invoke` boundary rejects resources before dispatch, like + `Memory`. Plugins/foreign kernels never receive either handle kind. + +### Builtins + +```text +resource.open(kind, descriptor, rights) -> Resource +resource.read(handle, memory, offset, max_bytes) -> Result {count, eof} +resource.write(handle, memory, offset, length) -> Result {count} +resource.close(handle) -> Result +``` + +* `read` validates the destination region **before** reading from the provider + and caps the request to the remaining bytes. +* EOF is `{ok:true, value:{count:0, eof:true}}`, not an error. +* Closed handles return `{ok:false, code:"resource.closed"}`; forged/cross-run/ + stale handles are fatal `runtime.resource`; unsupported kind/rights are + `resource.unsupported`; missing grant is `permission.denied`. +* Bytes move only as copied `Memory` ranges for the duration of the call. No + host pointer is stored in a region. + +### Providers in this slice + +| Kind | Descriptor | Rights | Mode | Behavior | +|---|---|---|---|---| +| `stdio` | `stdout` | `write` | pure/project | appends to the program output buffer after the output-budget check | +| `file` | project-relative path | `read` | project | bounded regular file loaded at open; chunked reads with EOF | +| anything else | — | — | — | explicit `resource.unsupported` | + +* `strict` refuses every `resource.open` with `permission.denied` (CM-A11 + fail-closed); no provider is silently enabled. +* `pure` allows only stdio stdout (console grant), exactly the existing + console exception; file resources are refused by an explicit mode check, + even if a caller hand-constructs a matching `fs_read` grant. +* File paths still go through `ProjectFs::resolve` (absolute paths, `..`, + symlink escapes rejected) and require an `fs_read` glob grant. +* File size is bounded by the existing 8 MiB read limit. +* `resource.write` to stdout charges the existing output budget; a budget + failure produces no output. There is no separate I/O-byte budget yet. + +## Tests + +### Host integration (`crates/marqdo-host/tests/resource_streams.rs`, 7) + +```text +pure_mode_writes_stdout_through_a_resource +stdout_write_obeys_the_output_budget_atomically +strict_mode_refuses_every_resource_open +resource_handles_cannot_cross_the_generic_host_boundary +pure_mode_refuses_file_resources_even_with_a_grant +project_file_reads_are_chunked_and_require_a_grant +traversal_and_unsupported_rights_are_refused +``` + +### Host table unit tests (`crates/marqdo-host/src/resource.rs`, 3) + +```text +file_reads_are_chunked_with_eof +forged_and_closed_handles_are_rejected +stdout_is_not_readable_and_file_is_not_writable +``` + +### CLI integration (`crates/cli/tests/resource_streams.rs`, 2) + +```text +project_file_stream_reads_chunks_into_memory + data.txt = "abcdef" + run index.mq.md --allow-read data.txt --json + results: counts 2/4/0, eof false/true/true, bytes 97..102 + resource.close => ok:true + +file_stream_requires_a_read_grant + same document without --allow-read + exit 1, error.code = permission.denied, no bytes read +``` + +### Semantic fixture 66 + +`tests/conformance/semantics/66-resource-stdout-write.mq.md`: + +* `.mq.md` UTF-8 encoder writes `"你好,资源!\n"` into `Memory`; +* `resource.open("stdio","stdout",["write"])`; +* `resource.write` reports count 19; +* `resource.close` succeeds; +* a second `resource.read` on the closed handle returns + `{ok:false,code:"resource.closed"}`; +* captured program stdout is exactly `你好,资源!\n`. + +## What this does not prove + +* No file write, directory listing, rename, metadata, network, DNS, TLS, + process, clock, random or device resources. +* No generic third-party provider registry (CM-A09) and no Worker/ABI byte + transfer. +* No stdlib `print` migration (CM-A07); `打印` still uses `HostOp::Print`. +* No strict-mode stream support. Strict L1 is unchanged and refuses opens. +* No async/nonblocking/seek/zero-copy and no per-stream deadline. +* No Windows/macOS native lane and no remote CI URL. +* Host-in-process I/O is not an OS sandbox; project mode remains cooperative. + +## Reproduce + +```bash +export PATH=/root/.cargo/bin:$PATH +export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo +export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target +cargo test -p marqdo-host +cargo test -p marqdo-cli --test resource_streams +cargo test -p marqdo-cli --test conformance_semantics +cargo test --workspace --locked +``` diff --git a/docs/evidence/2026-11-20-result-and-free-list-p2.md b/docs/evidence/2026-11-20-result-and-free-list-p2.md new file mode 100644 index 0000000..5d56379 --- /dev/null +++ b/docs/evidence/2026-11-20-result-and-free-list-p2.md @@ -0,0 +1,361 @@ +# RFC-CM CM-A05 evidence: `Result` convention and first-fit free-list allocator + +Date: 2026-11-20 (Asia/Shanghai) +Repository: `marqdo-cli` +Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes. +Decision records: [ADR-0024](../adr/0024-recoverable-result-convention.md), +[ADR-0023](../adr/0023-minimal-structure-observation-and-construction.md) +Related: [P1](2026-11-20-memory-p1.md), +[P2 closure](2026-11-20-memory-p2-closure.md), +[structure API](2026-11-20-structure-api-p2.md). +Scope: CM-A05 convention, a `.mq.md` first-fit allocator, and the later P2 +fixtures 56–59 (binary tree, CSV splitter, hash table, UTF-8 encoder). Full P2 +gate still **not** passed. + +## Environment and gate + +Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0. + +```text +cargo fmt --all -- --check PASS +cargo clippy --workspace --all-targets -- -D warnings PASS +cargo test --workspace --locked PASS + 47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests) +``` + +No GitHub CI is available from this checkout; this is local evidence. + +## `Result` convention (ADR-0024) + +Success and failure are ordinary `Map` values: + +```text +success: {ok: true, value: } +failure: {ok: false, code: , message: [, details: ]} +``` + +No `Result` value type, exception, `try`/`catch` or automatic propagation was +added. Expected conditions return this shape; wrong types, out-of-bounds, +invalid/stale handles, budget exhaustion and capability denial remain fatal +`RuntimeError`s. + +## Allocator fixture 55 + +`tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md` is a +pure `.mq.md` first-fit allocator over `Memory`: + +* 8-byte block header: total block size (`u32`) and free-list next (`u32`); +* region header: free-list head at offset 0, bump pointer at offset 4; +* allocation rounds the request up to an 8-byte multiple; +* first fit walks the free list, splits a fitting block when the remainder is + at least 16 bytes, otherwise unlinks it; +* no suitable free block falls back to the bump pointer; +* no space returns `{ok:false, code:"alloc.no_space", message:...}`; +* the free list is address-sorted; `释放` inserts a block and coalesces it + with an address-adjacent successor and predecessor without erasing payload + bytes. + +Observed result (compact wire values): + +```text +甲 (8 bytes) -> {ok:true, value:16} +乙 (16 bytes) -> {ok:true, value:32} +丙 (8 bytes) -> {ok:true, value:16} # reuses 甲's freed block +丁 (4 bytes) -> {ok:true, value:16} # reuses the same block +戊 (16 bytes) -> {ok:true, value:32} # reuses 乙's freed block +己 (64 bytes) -> {ok:true, value:56} +庚 (8 bytes) -> {ok:true, value:56} # split from 己 +辛 (16 bytes) -> {ok:true, value:72} # split remainder +壬 (8 bytes) -> {ok:true, value:96} # split remainder +癸 (8 bytes) -> {ok:true, value:112} # exact remainder +failure -> {ok:false, code:"alloc.no_space", + message:"没有足够的空闲空间"} +payload bytes -> 44, 55, 66, 77 at 庚/辛/壬/癸 +``` + +A second region isolates coalescing from bump allocation: + +```text +甲2 8B, 乙2 8B, 丙2 8B -> payloads 16, 32, 48 +fill 65472B -> consumes the rest of the 64 KiB region +free 甲2, 乙2, 丙2 -> address-adjacent blocks merge to one 48-byte block +丁2 40B (need 48) -> {ok:true, value:16} # only the merged block fits +戊2 8B -> {ok:false, alloc.no_space} # bump space exhausted +``` + +Without coalescing, `丁2` would see only three 16-byte fragments and the +exhausted bump pointer, so it would fail; the observed success at offset 16 +proves the merge. + +The payload addresses do not overlap: +`庚=56..71`, `辛=72..95`, `壬=96..111`, `癸=112..127`. The earlier bytes +`甲=16..23`, `乙=32..47`, `戊=32..47` show deliberate reuse after `释放` +(the block is not zeroed; callers must initialise a block they allocate). + +`cargo test -p marqdo-cli --test conformance_semantics` passes with fixture 55 +included. `cargo run -p marqdo-cli -- run +tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md --json` +returns `success: true` with the checked-in expected value. + +## Binary tree fixture 56 + +`tests/conformance/semantics/56-memory-binary-tree.mq.md` is a second +`.mq.md` structure over `Memory`: + +* 16-byte nodes: one-byte value at offset 0, left child offset at 4, right + child offset at 8; `0` is the empty child; +* region header at offset 0 is the bump pointer, initialised to 8; +* `插入` is recursive and writes the child link through its parent block; +* `遍历` is recursive in-order and builds the result with `list.concat`; +* inserted `[5, 3, 8, 1, 4, 7, 9]`, returned: + +```text +root=8 +in-order=[1,3,4,5,7,8,9] +root.left=24 +root.right=40 +``` + +No Rust tree/container algorithm, new `HostOp`, `Value` variant or Markdown +syntax was added. `cargo test -p marqdo-cli --test conformance_semantics` +passes with fixture 56 included. + +## CSV row splitter fixture 57 + +`tests/conformance/semantics/57-csv-row-splitter.mq.md` splits a single CSV row +using only `.mq.md` and the CM-A04 primitives: + +* `text.codepoints` enumerates Unicode scalar values; +* `text.from_codepoint` reconstructs each character; +* `list.append` builds the field list, `list.concat` appends the final field; +* comma is the separator; empty fields are preserved. + +Observed: + +```text +"甲,乙,,丙" -> ["甲","乙","","丙"] +"" -> [""] +"a,b" -> ["a","b"] +``` + +Quoted fields, embedded commas, doubled quotes and line breaks are **not** +implemented. This is a C05 demonstration, not a claim that CSV is complete. + +## Hash table fixture 58 + +`tests/conformance/semantics/58-memory-hash-table.mq.md` implements an +open-addressing hash table over `Memory`: + +* 8-byte header: element count at 0, slot count at 4; slots from offset 8, + 8 bytes each (`key:u32`, `value:u32`); +* key `0` means empty, so business keys start at 1; +* hash is `key % capacity` with linear probing; +* existing keys update in place without incrementing the count; a full table + returns `{ok:false, code:"hash.full", ...}`. + +Observed: + +```text +capacity 32: insert 5->50, 37->370 (collision at slot 5), 12->120, update 5->55 +lookup 5 => 55, 37 => 370, 12 => 120, 99 => None +count => 3 +capacity 2: insert 1, 2, then 4 => hash.full +``` + +No Rust hash container, `HostOp` or new `Value` type was added. + +## UTF-8 encoder fixture 59 + +`tests/conformance/semantics/59-utf8-encoder.mq.md` writes UTF-8 bytes into +`Memory` using `text.codepoints` and byte indexing: + +* code points below `0x80` -> 1 byte; +* below `0x800` -> 2 bytes; below `0x10000` -> 3 bytes; otherwise 4 bytes; +* the fixture encodes `"Aé中𝄞"` to + `[65, 195, 169, 228, 184, 173, 240, 157, 132, 158]` with length 10. + +## UTF-8 decoder fixture 60 + +`tests/conformance/semantics/60-utf8-decoder.mq.md` decodes UTF-8 from +`Memory` and validates every sequence: + +* `utf8.truncated` when a multi-byte sequence runs past the region length; +* `utf8.invalid_continuation` when a byte outside `0x80..0xBF` appears in a + continuation position; +* `utf8.invalid_scalar` for overlong 3-byte forms, surrogates and values above + `0x10FFFF`; +* `utf8.invalid_lead` for `0x80..0xC1` and `0xF5..0xFF` lead bytes. + +Observed: + +```text +[65,195,169,228,184,173] -> ok "Aé中" +[228,184] -> utf8.truncated +[228,65,173] -> utf8.invalid_continuation +[237,160,128] -> utf8.invalid_scalar +[240,157,132,158] -> ok "𝄞" +``` + +Together with fixture 59 this closes the C04 encode/decode demonstration using +only `.mq.md`; no Rust UTF-8 codec is called. + +## Graph adjacency fixture 61 + +`tests/conformance/semantics/61-memory-graph-adjacency.mq.md` stores a directed +graph in `Memory`: + +* 16-byte node: one-byte value at 0, edge-list head `u32` at 4; +* 8-byte edge: target node offset at 0, next edge offset at 4; +* edges are head-inserted, so traversal returns the most recently added edge + first; +* nodes `甲/乙/丙` at offsets 8/24/40, edges `甲→乙`, `甲→丙`, `乙→丙`. + +Observed: + +```text +甲=8, 乙=24, 丙=40 +邻接(甲) => [40, 24] # 丙 then 乙 (head insertion) +邻接(乙) => [40] +邻接(丙) => [] +``` + +No Rust graph container, `HostOp` or new `Value` type was added. + +## JSON subset parser fixture 62 + +`tests/conformance/semantics/62-json-array-parser.mq.md` is a recursive +descent parser over `text.codepoints`: + +* supports nested arrays and objects, strings with standard escapes including + `\uXXXX`, non-negative and negative integers, `true`/`false`/`null`, + whitespace, and returns `{ok,value,next}`; +* errors are `Result` maps: `json.unexpected_eof`, `json.invalid_array`, + `json.invalid_object`, `json.invalid_string`, `json.invalid_number`, + `json.invalid_literal`, `json.unexpected_token`; +* floats are explicitly out of scope. + +Observed: + +```text +" [1, 23, [4, 5], -6] " -> ok [1,23,[4,5],-6], next=20 +"[1 2]" -> json.invalid_array +"[1," -> json.unexpected_eof +{"a": 1, "b": [true, null, "说\"好"]} + -> ok {a:1, b:[true,null,"说\"好"]}, next=35 +"\"a\\nb\"" -> ok "a\nb", next=6 +``` + +No Rust JSON parser, new `Value` type or `HostOp` was added. + +## Bytecode VM fixture 63 + +`tests/conformance/semantics/63-memory-bytecode-vm.mq.md` is a small stack +bytecode machine written entirely in `.mq.md`: + +* program bytes at offset 0; stack at a caller-provided `Memory` offset; +* instruction set: `1=PUSH imm8`, `2=ADD`, `3=MUL`, `5=JMP`, `6=JZ`, `7=HALT`; +* state (`pc`, `sp`, running, error) is ordinary local state; the stack and + program live in `Memory`; +* `(5 + 7) * 3` executes to `{ok:true, value:36}`; +* a `JZ` program jumps over a `PUSH 99` and returns `{ok:true, value:42}`. + +No Rust VM, opcode enum, `HostOp` or new `Value` type was added. + +## Integer codecs fixture 64 + +`tests/conformance/semantics/64-memory-integer-codecs.mq.md` provides generic +big-endian and variable-length integer codecs over `Memory`: + +* `写大端(内存,基址,值,宽度)` writes the least significant byte last, so the + result is big-endian for widths 2, 4 and 8; +* `读大端(内存,基址,宽度)` rebuilds the integer with `value * 256 + byte`; +* `写变长`/`读变长` are LEB128 (7-bit continuation) with the consumed byte + count. + +Observed: + +```text +u16 4660 -> 4660 +u32 305419896 -> 305419896 +u64 1311768467463790320 -> 1311768467463790320 +u64 18446744073709551615 -> 18446744073709551615 +varint 123456789 -> 4 bytes, decoded 123456789, consumed 4 +last two bytes of u64::MAX -> 255, 255 +``` + +No Rust codec builtin or new `Value` type was added. + +## Table sort fixture 65 + +`tests/conformance/semantics/65-table-sort-stdlib.mq.md` sorts a table by a +column using only CM-A04 primitives: + +* `table.row` extracts rows, `table.cell` compares the sort column; +* `插入有序` builds a new row list with `list.append` and binary condition + branches (insertion sort); +* `表(列名, 已排序)` rebuilds the result; the input table is unchanged. + +Observed: + +```text +原表 rows: 甲87, 乙92, 丙71 +升序 rows: 丙71, 甲87, 乙92 +原表: unchanged +``` + +No Rust table sort, `HostOp` or new `Value` type was added. + +## Table join fixture 67 + +`tests/conformance/semantics/67-table-join-stdlib.mq.md` joins two tables on a +column using only CM-A04 primitives: + +* nested `table.row` / `table.cell` loops compare the join keys; +* matches append `list.concat(left_row, right_row)` to the output rows; +* `表(list.concat(headers...), rows)` rebuilds the result; both inputs are + unchanged. + +Observed: left `甲/乙/丙`, right `乙/甲/丁` on column 0 yields the two matched +rows `甲 87 甲 B` and `乙 92 乙 A`. No Rust join or `HostOp` was added. + +## Quoted CSV fixture 68 + +`tests/conformance/semantics/68-csv-quoted-row.mq.md` parses one CSV row with +quoted fields: + +* `"..."` quotes fields, `""` is a literal quote, commas inside quotes are + preserved; +* empty fields are preserved; +* multi-line fields and whitespace trimming are out of scope. + +Observed: + +```text +甲,"乙,丙","说""好" -> ["甲", "乙,丙", "说"好"] +a,,c -> ["a", "", "c"] +"" -> [""] +"a""b",c -> ["a"b", "c"] +``` + +No Rust CSV parser or new `Value` type was added. + +## What remains open + +* JSON floats, table group, and an independent `marqdo-stdlib` repository; +* CM-A06 `Resource`/`Stream` byte-transfer ABI; EOF/partial semantics are + reserved by ADR-0024 but not implemented; +* CM-A07–CM-A11 (stdlib naming, Table storage, provider ABI, packaging, + strict resource policy) and P3–P6; +* the RFC §10.5 frozen-CLI closure lane and Windows/macOS evidence. + +## Reproduce + +```bash +export PATH=/root/.cargo/bin:$PATH +export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo +export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target +cargo test -p marqdo-cli --test conformance_semantics +cargo run -q -p marqdo-cli -- run \ + tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md --json +cargo test --workspace --locked +``` diff --git a/docs/evidence/2026-11-20-stdlib-print-compat-a.md b/docs/evidence/2026-11-20-stdlib-print-compat-a.md new file mode 100644 index 0000000..138c825 --- /dev/null +++ b/docs/evidence/2026-11-20-stdlib-print-compat-a.md @@ -0,0 +1,94 @@ +# RFC-CM CM-A07 Compat-A evidence: `.mq.md` print prototype + +Date: 2026-11-20 (Asia/Shanghai) +Repository: `marqdo-cli` +Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes. +Decision record: [ADR-0027](../adr/0027-print-migration-compat-a.md) +Related: [Resource P3](2026-11-20-resource-stream-p3.md), +[file write](2026-11-20-resource-file-write-p3.md), [STATUS §5](../STATUS.md). +Scope: **Compat-A only**. Builtin `打印` is not migrated; no alias routing and +no removal of `HostOp::Print`. + +## Environment and gate + +Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0. + +```text +cargo fmt --all -- --check PASS +cargo clippy --workspace --all-targets -- -D warnings PASS +cargo test --workspace --locked PASS + 407 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests) +``` + +No GitHub CI is available from this checkout; this is local evidence. + +## Prototype + +Ordinary `.mq.md` modules under `examples/stdlib/`: + +| File | Responsibility | +|---|---| +| `format.mq.md` | `显示` for Unit/None/Bool/Int/Float/Text/List/Map/Table, `值显示` (repr), `转义文本`, `列表显示`, `映射显示`, `表格显示`, `连接文本` | +| `text.mq.md` | `utf8编码(文本, 内存, 基址)` — 1–4 byte UTF-8 into `Memory` | +| `io.mq.md` | `写文本` (encode → `resource.write(stdout)` → close) and `写值` (format + newline) | +| `demo.mq.md` | side-by-side `io.写值(x)` and builtin `打印(x)` for a nested list and a table | + +The call chain is exactly the RFC target shape: + +```text +io.写值(value) + -> format.显示(value) # .mq.md value formatting, no Rust display() + -> text.utf8编码(text, memory) # .mq.md UTF-8 encoder + -> resource.open(stdout) + -> resource.write(memory, ...) # bytes through Memory + -> resource.close(stdout) +``` + +No `HostOp`, no new builtin, no parser change and no special stdlib dispatch. +The module is loaded only through an explicit Markdown import alias. + +## Output comparison + +```text +$ marqdo run examples/stdlib/demo.mq.md +["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None] +["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None] +Table[名称 | 分数]{甲 | 87; 乙 | 92} +Table[名称 | 分数]{甲 | 87; 乙 | 92} +``` + +The first and third lines are `io.写值`; the second and fourth are builtin +`打印`. They are byte-identical. + +CLI test: +`crates/cli/tests/examples_r6.rs::mqmd_stdlib_print_output_matches_builtin_print` +asserts exactly four lines, pairwise equality, and the exact list/table text. + +## Covered and uncovered values + +Covered by Compat-A comparison: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, +`List`, `Map`, `Table`. + +Not covered: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The language +still has no public observation for the module/name/handle identity needed to +reproduce `Value::display()`/`repr()` for those types; adding it is a CM-A04 +extension, not part of this slice. This is the explicit blocker for Compat-B. + +## What is not migrated + +* `打印`/`print` still resolves to the existing builtin and `HostOp::Print`. +* No alias table, no bundled stdlib resolution, no `std` package. +* No stdlib version/digest or offline distribution (CM-A10). +* No Compat-B/C comparison lane or removal of the Rust `Value::display()` path. +* The demo/stdlib is an `examples/` prototype, not a shipped standard library. + +## Reproduce + +```bash +export PATH=/root/.cargo/bin:$PATH +export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo +export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target +cargo run -q -p marqdo-cli -- run examples/stdlib/demo.mq.md +cargo test -p marqdo-cli --test examples_r6 +cargo test --workspace --locked +``` diff --git a/docs/evidence/2026-11-20-structure-api-p2.md b/docs/evidence/2026-11-20-structure-api-p2.md new file mode 100644 index 0000000..5b5f8fe --- /dev/null +++ b/docs/evidence/2026-11-20-structure-api-p2.md @@ -0,0 +1,110 @@ +# RFC-CM CM-A04 evidence: minimal structure observation and construction + +Date: 2026-11-20 (Asia/Shanghai) +Repository: `marqdo-cli` +Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes. +Decision record: [ADR-0023](../adr/0023-minimal-structure-observation-and-construction.md) +Related: [P1 memory evidence](2026-11-20-memory-p1.md), +[P2 closure evidence](2026-11-20-memory-p2-closure.md), +[STATUS §5](../STATUS.md). +Scope: CM-A04 only. The full RFC-CM-0002 P2 gate is **not** passed. + +## Environment and gate + +Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0. + +```text +cargo fmt --all -- --check PASS +cargo clippy --workspace --all-targets -- -D warnings PASS +cargo test --workspace --locked PASS + 47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests) +``` + +The 2 ignored tests are the pre-existing `marqdo-plugin-sdk` doc-tests. No +GitHub CI was available from this checkout; this is local evidence. + +## Builtins added (pure, no `HostOp`, no capability) + +| Name | Semantics | +|---|---| +| `list.append` | new list with one appended value; input unchanged | +| `list.concat` | new concatenated list; inputs unchanged | +| `map.keys` | Text keys in canonical sorted order | +| `map.has` | key membership | +| `map.get` | stored value or `None`; `map.has` disambiguates a stored `None` | +| `map.set` | new map with the key; input unchanged | +| `map.remove` | new map without the key; missing key is a no-op | +| `text.codepoints` | Unicode scalar values as `Int`, in order | +| `text.from_codepoint` | one-scalar Text, or `None` for surrogate/out-of-range | +| `table.columns` | header names in column order | +| `table.row` | new list of row cells; non-negative coordinate | +| `table.cell` | cell value; non-negative row/column | + +No `sort`, `filter`, `join`, `group`, JSON/CSV, `table.set_cell`, mutable +List/Map or new Markdown syntax was added. Constructors never mutate their +arguments; this preserves INV-02. + +## Semantic fixtures + +```text +51-list-map-value-semantics: + original list [1,2] unchanged; appended [1,2,3] + original map {a:1,b:2} unchanged; map.set c=3 then remove a + keys after remove => ["b","c"] + map.has(original, "c") => false + map.get(original, "z") => None + list.concat([1],[2,3]) => [1,2,3] + +52-text-codepoints: + text.codepoints("aé中") => [97, 233, 20013] + text.from_codepoint(20013) => "中" + text.from_codepoint(0xD800) => None + text.from_codepoint(0x110000) => None + +53-table-projection: + columns => ["名称","分数"] + row 1 => ["乙",92] + cell(0,1) => 87 + 长度(table) => 2 + +54-table-filter-stdlib: + 原表 3 rows, threshold 80 + 结果 table has 2 rows (甲 87, 乙 92) + 原表 is still 3 rows and byte-for-byte the same logical table +``` + +Fixture 54 is the first `.mq.md` algorithm over the new primitives: it uses +`table.columns`, `table.cell`, `table.row`, `list.append` and `表` to rebuild a +filtered table. It is a conformance document, not a shipped `table.filter` +builtin; no Rust table algorithm was added. + +CLI confirms all three return `success: true` with the checked-in wire values. + +## Runtime negatives + +| Fixture | Expected code | Observed | +|---|---|---| +| `19-list-append-wrong-type` | `runtime.type` | `runtime.type` | +| `20-table-cell-out-of-bounds` | `runtime.index` | `runtime.index` | +| `21-map-key-not-text` | `runtime.type` | `runtime.type` | + +## Focused command + +```text +cargo test -p marqdo-cli --test conformance_semantics --test negative + semantic_goldens_match_end_to_end_execution ... ok + runtime_negative_goldens ... ok +``` + +## What remains open + +* CM-A05 recoverable `Result`/EOF/partial model. This report predates ADR-0024, + which now freezes the `Result` map convention; the Stream EOF/partial part + remains CM-A06. `map.get` returning `None` is a lookup convention, **not** + that contract. +* CM-A06 `Resource`/`Stream` ABI, CM-A07 stdlib naming/alias, CM-A08 Table + storage/physical migration, CM-A09 provider ABI, CM-A10 stdlib packaging, + CM-A11 strict resource policy. +* A separate `marqdo-stdlib` repository, graph containers, a full JSON parser, + quoted/escaped CSV and the RFC §10.5 frozen-CLI closure lane. +* Windows/macOS native lanes. No remote CI URL exists for this working tree. diff --git a/examples/memory-buffer.mq.md b/examples/memory-buffer.mq.md new file mode 100644 index 0000000..1af321c --- /dev/null +++ b/examples/memory-buffer.mq.md @@ -0,0 +1,14 @@ +## main + +此函数演示“可引用的存储区域”与普通 Markdown 叙述并存。 +区域按 64 KiB 逻辑页增长,副本共享同一区域身份。 + +**区域 = memory.new(1, 4)** + +**`区域[8] = 42`** + +**另一个名字 = 区域** + +**`另一个名字[9] = 99`** + +*区域[8] + 区域[9]* diff --git a/examples/stdlib/demo.mq.md b/examples/stdlib/demo.mq.md new file mode 100644 index 0000000..4afef4f --- /dev/null +++ b/examples/stdlib/demo.mq.md @@ -0,0 +1,20 @@ +--- +import: + io: "./io.mq.md" +--- + +## main + +**列表 = ["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]** + +> io.写值(列表) + +> 打印(列表) + +**表格 = 表(["名称", "分数"], [["甲", 87], ["乙", 92]])** + +> io.写值(表格) + +> 打印(表格) + +*[类型(列表), 类型(表格)]* diff --git a/examples/stdlib/format.mq.md b/examples/stdlib/format.mq.md new file mode 100644 index 0000000..78c0307 --- /dev/null +++ b/examples/stdlib/format.mq.md @@ -0,0 +1,209 @@ +## 显示 + +`(值)` + +按语言值语义格式化,用于与内置 `打印` 的输出对照。 +当前只覆盖 Unit/None/Bool/Int/Float/Text/List/Map/Table;Formula/Code/ +FunctionRef/PluginHandle 仍缺少公开观察接口。 + +**类别 = 类型(值)** + +1. `类别 == "Unit"` + + *"Unit"* + +2. `类别 == "None"` + + *"None"* + +3. `类别 == "Text"` + + *值* + +4. `类别 == "List"` + + *列表显示(值)* + +5. `类别 == "Map"` + + *映射显示(值)* + +6. `类别 == "Table"` + + *表格显示(值)* + +7. `true` + + *文本(值)* + +## 列表显示 + +`(列表)` + +**结果 = "["** + +**下标 = 0** + +- `下标 < 长度(列表)` + + 1. `下标 > 0` + + **结果 = 结果 + ", "** + + 2. `true` + + **忽略 = 0** + + **结果 = 结果 + 值显示(列表[下标])** + + **下标 = 下标 + 1** + +*结果 + "]"* + +## 映射显示 + +`(映射)` + +**键 = map.keys(映射)** + +**结果 = "{"** + +**下标 = 0** + +- `下标 < 长度(键)` + + 1. `下标 > 0` + + **结果 = 结果 + ", "** + + 2. `true` + + **忽略 = 0** + + **当前键 = 键[下标]** + + **结果 = 结果 + 当前键 + ": " + 值显示(map.get(映射, 当前键))** + + **下标 = 下标 + 1** + +*结果 + "}"* + +## 表格显示 + +`(表格)` + +**列 = table.columns(表格)** + +**结果 = "Table[" + 连接文本(列, " | ") + "]{"** + +**行号 = 0** + +- `行号 < 长度(表格)` + + 1. `行号 > 0` + + **结果 = 结果 + "; "** + + 2. `true` + + **忽略 = 0** + + **行 = table.row(表格, 行号)** + + **单元格 = []** + + **列号 = 0** + + - `列号 < 长度(行)` + + **单元格 = list.append(单元格, 显示(行[列号]))** + + **列号 = 列号 + 1** + + **结果 = 结果 + 连接文本(单元格, " | ")** + + **行号 = 行号 + 1** + +*结果 + "}"* + +## 连接文本 + +`(列表, 分隔)` + +**结果 = ""** + +**下标 = 0** + +- `下标 < 长度(列表)` + + 1. `下标 > 0` + + **结果 = 结果 + 分隔** + + 2. `true` + + **忽略 = 0** + + **结果 = 结果 + 列表[下标]** + + **下标 = 下标 + 1** + +*结果* + +## 值显示 + +`(值)` + +列表/映射/表格内部使用 repr 语义:Text 带引号并转义,其它值递归显示。 + +**类别 = 类型(值)** + +1. `类别 == "Text"` + + *转义文本(值)* + +2. `true` + + *显示(值)* + +## 转义文本 + +`(文本)` + +**点 = text.codepoints(文本)** + +**`结果 = "\""`** + +**下标 = 0** + +- `下标 < 长度(点)` + + **码点 = 点[下标]** + + 1. `码点 == 34` + + **`结果 = 结果 + "\\\""`** + + 2. `码点 == 92` + + **`结果 = 结果 + "\\\\"`** + + 3. `码点 == 10` + + **`结果 = 结果 + "\\n"`** + + 4. `码点 == 9` + + **`结果 = 结果 + "\\t"`** + + 5. `码点 == 13` + + **`结果 = 结果 + "\\r"`** + + 6. `true` + + **结果 = 结果 + text.from_codepoint(码点)** + + **下标 = 下标 + 1** + +*`结果 + "\""`* diff --git a/examples/stdlib/io.mq.md b/examples/stdlib/io.mq.md new file mode 100644 index 0000000..1e5a687 --- /dev/null +++ b/examples/stdlib/io.mq.md @@ -0,0 +1,35 @@ +--- +import: + format: "./format.mq.md" + text: "./text.mq.md" +--- + +## 写文本 + +`(文本)` + +把文本编码为 UTF-8 并写入已授权的 stdout 资源流。 + +**内存 = memory.new(1, 1)** + +**长度 = text.utf8编码(文本, 内存, 0)** + +**流 = resource.open("stdio", "stdout", ["write"])** + +> resource.write(流, 内存, 0, 长度) + +> resource.close(流) + +*None* + +## 写值 + +`(值)` + +按语言值语义格式化后输出,并追加换行。 + +**文字 = format.显示(值) + "\n"** + +> 写文本(文字) + +*None* diff --git a/examples/stdlib/text.mq.md b/examples/stdlib/text.mq.md new file mode 100644 index 0000000..ce13721 --- /dev/null +++ b/examples/stdlib/text.mq.md @@ -0,0 +1,55 @@ +## utf8编码 + +`(文本, 内存, 基址)` + +把 Unicode 标量序列按 UTF-8 写入 `Memory`,返回字节长度。 + +**点 = text.codepoints(文本)** + +**下标 = 0** + +**偏移 = 0** + +- `下标 < 长度(点)` + + **码点 = 点[下标]** + + 1. `码点 < 128` + + **`内存[基址 + 偏移] = 码点`** + + **偏移 = 偏移 + 1** + + 2. `码点 < 2048` + + **`内存[基址 + 偏移] = 192 + 码点 // 64`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 2** + + 3. `码点 < 65536` + + **`内存[基址 + 偏移] = 224 + 码点 // 4096`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 // 64 % 64`** + + **`内存[基址 + 偏移 + 2] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 3** + + 4. `true` + + **`内存[基址 + 偏移] = 240 + 码点 // 262144`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 // 4096 % 64`** + + **`内存[基址 + 偏移 + 2] = 128 + 码点 // 64 % 64`** + + **`内存[基址 + 偏移 + 3] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 4** + + **下标 = 下标 + 1** + +*偏移* diff --git a/spec/analysis-v2.md b/spec/analysis-v2.md index f3ac688..79f0b65 100644 --- a/spec/analysis-v2.md +++ b/spec/analysis-v2.md @@ -1,6 +1,7 @@ # Marqdo Analysis v2 -Status: **implemented by `marqdo check --emit ` (S2).** Supersedes +Status: **implemented by `marqdo check --emit ` (S2); ADR-0022 adds an +additive index target shape.** Supersedes Analysis v1. v1 was never released; it is kept as history and is not written. ## 1. Invariants @@ -82,6 +83,24 @@ index, call). `imports`, `references` and `calls` remain static facts; a Markdown reference is not a call. `effects` remains `["unknown"]` (a conservative summary, not a purity proof). +An `assign` node keeps the v1 string `target` for a local rebinding. Addressable +memory adds one additive target form (ADR-0022); consumers that only understand +local assignments can ignore an object target: + +```json +{"kind":"assign", + "target":{"kind":"index", + "base":{"kind":"variable","name":"内存","span":{"start":23,"end":26}}, + "index":{"kind":"literal","type":"int","value":"0","span":{"start":27,"end":28}}, + "span":{"start":23,"end":29}}, + "value":{"kind":"literal","type":"int","value":"7","span":{"start":32,"end":33}}, + "span":{"start":18,"end":35}} +``` + +`target.kind` is `"index"`; `base` and `index` use the same expression node +shapes as every other expression. Analysis is static: it records the place but +never allocates a region or executes the write. + ## 4. Resources `resources` is an array, not a count. v2 projects exactly the two resource diff --git a/spec/evaluation.md b/spec/evaluation.md index f48fe40..a77c605 100644 --- a/spec/evaluation.md +++ b/spec/evaluation.md @@ -1,6 +1,7 @@ # Marqdo Evaluation Semantics v1 -Status: **frozen for M0.5**. Changes to observable behaviour require an ADR. +Status: **frozen for M0.5; extended by ADR-0022 (`Memory`) and ADR-0024 (recoverable results)**. +Changes to observable behaviour require an ADR. ## 1. Values @@ -19,6 +20,8 @@ Status: **frozen for M0.5**. Changes to observable behaviour require an ADR. | `Code` | inert code fence + provenance; execution is explicit | | `FunctionRef` | reference to a named function in a module | | `PluginHandle` | opaque host-owned plugin export handle | +| `Memory` | opaque runtime-owned addressable byte region (ADR-0022); copies share identity, offsets are region-relative | +| `Resource` | opaque host-issued resource/stream handle (ADR-0025); not indexable, not wire-transferable, rendered as `Resource` | There is no truthiness: `if`-like conditions accept `Bool` only. There are no implicit string↔number conversions; conversion builtins must be called. @@ -52,6 +55,15 @@ implicit string↔number conversions; conversion builtins must be called. * Assignment binds/updates a local slot in the current frame. It never writes an outer frame. There is no module-level assignment in v1: assignment outside a function body is an error. +* The only other writable place is an index into a `Memory` region + (`m[offset] = byte`, ADR-0022). List, Map and Table values keep their v1 + value semantics; `list[i] = value`, `map[key] = value` and `obj.field = value` + are not defined and are rejected at run time or parse time rather than + silently changing those values into shared mutable objects. +* For `m[offset] = byte` the operands are evaluated exactly once in the order + **base, offset, right-hand side**. The target must evaluate to `Memory`, the + offset to a non-negative `Int` inside the region and the value to an `Int` + in `0..=255`; otherwise the statement fails without writing anything. * `and`/`or` short-circuit and require `Bool` operands. * `Return` (an `emph` node) unwinds through branch/loop bodies to the enclosing function frame. @@ -62,6 +74,112 @@ implicit string↔number conversions; conversion builtins must be called. * Function calls recurse; recursion depth is bounded by the stack budget. Mutual recursion is allowed. +### 3.1 Addressable `Memory` (ADR-0022) + +`Memory` is a separate storage domain from managed `List`/`Map`/`Table` +values. It exists so `.mq.md` programs can construct allocators, encodings and +data structures without a new Markdown node, keyword or host operation. + +* **Creation.** `memory.new(initial_pages, max_pages)` creates a zero-filled + region and returns its handle. Both arguments are non-negative `Int`s; + `initial_pages <= max_pages`, and `max_pages` must fit the per-region + budget. Rejection is atomic: no region or committed byte is created. +* **Page size.** One logical page is 64 KiB (`PAGE_SIZE`). Print, trace and + storage use bytes; page counts never mix with byte offsets. +* **Size and growth.** `memory.size(memory)` returns committed pages. + `memory.grow(memory, pages)` is grow-only and returns the **old** page + count; old offsets keep their bytes, every new byte is zero. A failed grow + (declared maximum, region budget or total budget) leaves the region and the + budget unchanged. Growing by zero pages is a no-op returning the current + count. Regions never shrink and there is no user-visible `free(region)`. +* **Read/write.** `m[offset]` returns an `Int` in `0..=255`; `m[offset] = byte` + writes one byte. Offsets are non-negative and region-relative; negative, + out-of-range and unrepresentable `BigInt` offsets are `runtime.index`. + Out-of-range byte values are `runtime.type`. No implicit truncation or + wrapping occurs. +* **Identity and equality.** Copying a `Memory` value, passing it to a + function or storing it in a collection shares the same region. Two handles + are `==` only when they designate the same region instance; `Memory` has no + ordering comparison and no byte-content equality operator. +* **Lifetime.** A region lives for one run and is dropped with the runtime. + Block-level ownership (`malloc`/`free`) is a standard-library concern; the + runtime only guarantees that a single access stays inside its region. +* **Diagnostics.** Unknown, stale or cross-run handles are + `runtime.memory`; budget failures are `resource.exhausted`; a non-`Memory` + index-assignment base is `runtime.type`. +* **Boundary.** A live handle never crosses the wire: `to_wire` emits only + `{"type":"memory"}`, decoding that tag is a `WireError::Invalid`, and every + host/plugin/foreign call rejects arguments containing `Memory`. +* **Display.** `Value::display()` renders `Memory` without the internal + region identity; `类型(m)` returns `"Memory"`. + +### 3.2 Recoverable results (ADR-0024) + +A recoverable failure is an ordinary `Map`, not a new value type or syntax: + +```text +success: {ok: true, value: } +failure: {ok: false, code: , message: [, details: ]} +``` + +`ok` is always a `Bool`. Success carries `value`; failure carries a stable +dot-separated `code` and a human-readable `message`. Constructors such as +`成功`/`失败` are ordinary `.mq.md` functions, not builtins. + +* Expected conditions (no free block, malformed parser input, invalid byte + sequence, and later EOF/short I/O results) should use this shape so a caller + can branch on `ok`. +* Language-invariant violations (wrong type, out-of-bounds, unknown/stale + `Memory` handle, budget exhaustion, capability denial) remain fatal + `RuntimeError`s. There is no catch mechanism and a library must not try to + convert them into `Err`. +* `map.get` returning `None` is a lookup result, not this convention; use + `map.has` to distinguish a stored `None` (ADR-0023). +* `code` is stable within its library; callers must not branch on `message`. + Recommended prefixes include `alloc.*`, `parse.*`, `codec.*`, `table.*` and, + later, `io.*`. +* Stream EOF/partial semantics belong to the resource ABI (CM-A06); this + section only reserves the shape `{ok: true, value: {count, eof, ...}}` for + later, without defining it. + +### 3.3 Resource streams (ADR-0025) + +`Resource` is a host-issued opaque handle to an external stream, in a separate +domain from `Memory`. It is not indexable, not a byte address and cannot be +constructed from a number, document or wire value. + +```text +resource.open(kind: Text, descriptor: Text, rights: List) -> Resource +resource.read(handle, memory: Memory, offset: Int, max_bytes: Int) + -> {ok:true, value:{count:Int, eof:Bool}} | {ok:false, code, message} +resource.write(handle, memory: Memory, offset: Int, length: Int) + -> {ok:true, value:{count:Int}} | {ok:false, code, message} +resource.close(handle) + -> {ok:true, value:None} | {ok:false, code, message} +``` + +* Providers: `stdio/stdout` with `["write"]` (pure/project, console grant); + `file` with `["read"]` (project, `fs_read` grant, bounded by the 8 MiB file + limit); `file` with `["write"]` (project, `fs_write` grant, 8 MiB buffered + write flushed on `close`, ADR-0026). Mixed rights and other kinds are + `resource.unsupported`. +* `resource.read` checks the destination `Memory` range before calling the + provider, caps the request to the remaining buffer, writes the returned bytes + and reports the actual count plus `eof`. EOF is success, not an error. +* `resource.write` copies bytes out of `Memory` for the call only; a short write + is a count. stdout output is charged to the output budget and delivered to + the program stdout channel. A project-file write stream appends to a bounded + host buffer and does not touch the filesystem until `close`; a failed close + leaves the handle open with its buffer, and an unclosed writer never creates + the file. +* Reading/writing/closing an already closed handle returns + `{ok:false, code:"resource.closed"}`. Forged/cross-run/stale handles are + fatal `runtime.resource`; unsupported kind/rights/direction are + `resource.unsupported`; capability/path denial is `permission.denied`; a + writer buffer above 8 MiB is `resource.exhausted` and appends atomically. +* `Memory` arguments never cross the host boundary as handles: the interpreter + copies bytes for the call. Resource handles never cross the value wire. + ## 4. Errors All failures are structured. Compile-time problems produce diagnostics @@ -69,7 +187,11 @@ All failures are structured. Compile-time problems produce diagnostics problems produce `RuntimeError { category, span, message, frames, cause }` with a stable category code. Runtime errors stop the current function and propagate with a call stack. A host/foreign/plugin failure never panics the -interpreter; it becomes a structured error. +interpreter; it becomes a structured error. Addressable-memory failures use +the stable codes `runtime.memory` (unknown/stale handle, invalid page counts), +`runtime.index` (offset outside the region, including negative or huge +offsets), `runtime.type` (wrong operand type or byte value outside `0..=255`) +and `resource.exhausted` (region or run budget). ## 5. Parameter inference (static, name based) @@ -98,10 +220,17 @@ explicit signature paragraph allows positional arguments. Every run has explicit limits (all configurable, all reported as `resource.exhausted`): evaluation fuel, call stack depth, total output bytes, -wall-clock time, and host process count. Loops consume fuel per iteration and -expressions per node so that a hostile document cannot hang the CLI. Strict -mode additionally enforces limits at the OS level where available; if the -platform cannot provide that, strict mode refuses to run (see `security.md`). +wall-clock time, host process count, and addressable memory. Addressable +memory has two independent budgets (ADR-0022): total committed bytes across +all regions (`DEFAULT_MAX_TOTAL_BYTES = 64 MiB`) and committed bytes per +region (`DEFAULT_MAX_REGION_BYTES = 16 MiB`, hence at most 256 pages of +64 KiB), overridable with `--max-memory` and `--max-region-memory`. Memory is +a pure computation: it needs no capability grant and is available in `pure`, +`project` and `strict` modes, subject to these budgets. +Loops consume fuel per iteration and expressions per node so that a hostile +document cannot hang the CLI. Strict mode additionally enforces limits at the +OS level where available; if the platform cannot provide that, strict mode +refuses to run (see `security.md`). ## 7. Determinism @@ -125,6 +254,41 @@ host boundary. Names are stable; adding one requires a spec update. | `小数` / `float` | `(x) -> Float` | from Int/Float/Text | | `表` / `table` | `(header: List, rows: List) -> Table` | validates row widths | | `ref` | `(target: Text) -> FunctionRef/Formula/Code` | resolves a document address; module-local, refuses ambiguous resource units (`ref.ambiguous`) | +| `memory.new` | `(initial_pages: Int, max_pages: Int) -> Memory` | creates a zero-filled region; budgets are checked before allocation (ADR-0022) | +| `memory.size` | `(memory: Memory) -> Int` | committed page count | +| `memory.grow` | `(memory: Memory, pages: Int) -> Int` | grow-only; returns the previous page count, new bytes are zero | +| `list.append` | `(list: List, value) -> List` | new list with one appended value; input unchanged (ADR-0023) | +| `list.concat` | `(left: List, right: List) -> List` | new concatenated list; inputs unchanged | +| `map.keys` | `(map: Map) -> List` | Text keys in canonical sorted order | +| `map.has` | `(map: Map, key: Text) -> Bool` | key membership | +| `map.get` | `(map: Map, key: Text) -> Value` | stored value or `None`; use `map.has` when `None` is a legal value | +| `map.set` | `(map: Map, key: Text, value) -> Map` | new map with the key; input unchanged | +| `map.remove` | `(map: Map, key: Text) -> Map` | new map without the key; missing key is a no-op | +| `text.codepoints` | `(text: Text) -> List` | Unicode scalar values as `Int`, in order | +| `text.from_codepoint` | `(codepoint: Int) -> Text or None` | `None` for a surrogate or a value outside `0..=0x10FFFF` | +| `table.columns` | `(table: Table) -> List` | header names in column order | +| `table.row` | `(table: Table, row: Int) -> List` | new list of row cells; non-negative coordinate | +| `table.cell` | `(table: Table, row: Int, column: Int) -> Value` | cell value; non-negative coordinates | +| `resource.open` | `(kind: Text, descriptor: Text, rights: List) -> Resource` | capability-checked stream open (ADR-0025) | +| `resource.read` | `(handle, memory: Memory, offset: Int, max_bytes: Int) -> Result` | bounded chunk read into Memory; `{count,eof}` | +| `resource.write` | `(handle, memory: Memory, offset: Int, length: Int) -> Result` | bounded byte write; `{count}` | +| `resource.close` | `(handle) -> Result` | releases the host slot | | `math.parse/eval/substitute` | see `spec/math.md` | P5; `math.simplify`/`math.differentiate` are declared but return `math.unsupported` in this build | | `foreign.run/call` | see `spec/foreign.md` | P5, capability-gated | | `fs.read/write` | see `spec/security.md` | capability-gated | + +The legacy `打印/print` builtin remains the default and still uses the host +output path. `examples/stdlib/` contains an explicitly imported `.mq.md` +`format/utf8/io` prototype (ADR-0027 Compat-A) whose output is comparison-tested +against `打印` for Unit/None/Bool/Int/Float/Text/List/Map/Table; it does **not** +replace the builtin, and Formula/Code/FunctionRef/PluginHandle formatting is not +yet expressible in the standard library. + +The `list.*`, `map.*`, `text.*` and `table.*` entries are observation and +construction primitives, not algorithms (ADR-0023). They return new values and +never mutate their arguments. Sorting, filtering, joining, grouping, JSON/CSV +and table transformations remain standard-library work: adding one must not +add a builtin. `map.get` returns `None` for a missing key; `map.has` is the +unambiguous membership check when a stored `None` is possible. `table.row` and +`table.cell` use non-negative coordinates, while `List` indexing keeps its v1 +negative-index rule. diff --git a/spec/expression.ebnf b/spec/expression.ebnf index 9cd7626..cc6cf68 100644 --- a/spec/expression.ebnf +++ b/spec/expression.ebnf @@ -1,5 +1,6 @@ (* Marqdo Expression Grammar v1 - Status: frozen for M0.5. This is the only grammar for executable payloads. + Status: frozen for M0.5, extended by ADR-0022 with Memory-index assignment + targets. This is the only grammar for executable payloads. It is a small independent language: it is NOT Markdown and NOT translated to Python. Markdown AST decides where a payload exists; this grammar decides what it means. @@ -10,9 +11,13 @@ program = [ statement ] , EOF ; statement = assignment | expression ; -assignment = identifier , "=" , expression ; -(* "=" is a single "=" token; "==" is a distinct token. Assignment targets - are bare identifiers only: no member or index assignment in v1. *) +assignment = assign_target , "=" , expression ; +assign_target = identifier + | postfix_expr , "[" , expression , "]" ; +(* v1 left-hand sides are a local identifier or one index place. The index + target writes one byte into an addressable Memory region; other base values + are rejected at run time. Member assignment (`a.b = 1`) is a parse error. + Chained assignment is a parse error. See ADR-0022. *) expression = or_expr ; or_expr = and_expr , { "or" , and_expr } ; diff --git a/spec/security.md b/spec/security.md index cfacb6c..68ecc38 100644 --- a/spec/security.md +++ b/spec/security.md @@ -27,8 +27,8 @@ Normative implementation: | Mode | Intended guarantee | Actual status in this build | |---|---|---| | `pure` | No external I/O; memory computation and budgets only. | **Enforced.** `Capabilities::pure()` = `console`, `math` only; every FS/foreign/plugin grant flag makes the CLI exit 2 with "pure mode prohibits external I/O grants". | -| `project` | Operator-approved project resources under `CapabilityHost` + bounded resources. | **Enforced for the implemented operations.** Defaults equal `pure`; `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin` add grants. Sandboxing is cooperative (in-process checks), not OS isolation. | -| `strict` | Untrusted documents under real OS isolation. | **Linux: enforced when the runtime probe passes (ADR-0021).** `PlatformCapabilities::detect()` runs a real Bubblewrap smoke test; success reports `Available { mechanism }`, failure reports `Unavailable { reason }`. Strict L1 supports the interpreter, console and native Math only; `--allow-read/--allow-write/--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file` are rejected with exit 2. macOS/Windows and Linux hosts without the mechanisms remain `Unavailable`; there is **no fallback** to an unrestricted process. | +| `project` | Operator-approved project resources under `CapabilityHost` + bounded resources. | **Enforced for the implemented operations.** Defaults equal `pure`; `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin` add grants. `resource.open` supports stdout write, granted project-file read and granted buffered project-file write (ADR-0025/0026); network/process remain unsupported. Sandboxing is cooperative (in-process checks), not OS isolation. | +| `strict` | Untrusted documents under real OS isolation. | **Linux: enforced when the runtime probe passes (ADR-0021).** `resource.open` is **refused** in strict until a provider passes the isolation lane (CM-A11). `PlatformCapabilities::detect()` runs a real Bubblewrap smoke test; success reports `Available { mechanism }`, failure reports `Unavailable { reason }`. Strict L1 supports the interpreter, console and native Math only; `--allow-read/--allow-write/--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file` are rejected with exit 2. macOS/Windows and Linux hosts without the mechanisms remain `Unavailable`; there is **no fallback** to an unrestricted process. | `marqdo inspect --platform` prints the detection report, for example: @@ -55,10 +55,10 @@ set for one run: | Field | Default (`pure`/`project`) | Used by | |---|---|---| -| `console: bool` | `true` | currently **not checked** by `op_print`; print is always allowed and only bounded by the output budget (open item) | +| `console: bool` | `true` | `op_print` and `resource.open("stdio","stdout",["write"])`; a missing grant is `permission.denied` and output is also bounded by the output budget | | `math: bool` | `true` | `math.*` host ops | -| `fs_read: Vec` | empty | `fs.read` | -| `fs_write: Vec` | empty | `fs.write` | +| `fs_read: Vec` | empty | `fs.read` and project `resource.open("file", path, ["read"])` | +| `fs_write: Vec` | empty | `fs.write` and project `resource.open("file", path, ["write"])` | | `net_hosts: Vec` | empty | `Capabilities::check_net` exists, but no host op or builtin performs network access in this build | | `process: bool` | `false` | required together with `foreign` for `foreign.run`/`foreign.call` | | `foreign: bool` | `false` | foreign kernels | @@ -144,6 +144,8 @@ no host bypass for tests: tests use the same `StdHost`/`Capabilities` types. | call stack depth | `DEFAULT_STACK_LIMIT = 256` | checked before every function call → `resource.exhausted` | | program output bytes | `DEFAULT_OUTPUT_BYTES = 1 MiB` | `print` output and `fs.read` contents are charged; overflow → `resource.exhausted` | | wall clock | none by default | `--timeout ` sets a deadline; checked between statements/iterations and before loop conditions → `resource.exhausted`. It does **not** preempt a running host call. | +| addressable memory | 64 MiB committed total, 16 MiB per region, 64 KiB pages; `--max-memory` / `--max-region-memory` override | Runtime-owned `MemoryStore` (ADR-0022); create/grow checks both budgets before allocation and failed growth is atomic → `resource.exhausted`. Memory is pure computation and needs no capability grant in any mode. | +| resource streams | stdout writes charged to the existing output budget; file reads and writer buffers bounded by the 8 MiB per-file limit | Host-owned resource table (ADR-0025/0026); output budget is checked before a stdout write is buffered; a failed check produces no bytes; writer buffers flush only on `close`. No separate I/O byte budget yet. | CLI overrides (`run`/`call`): `--fuel`, `--stack`, `--max-output`, `--timeout`. Additional fixed limits: diff --git a/spec/value-wire.md b/spec/value-wire.md index 11d0394..315a24f 100644 --- a/spec/value-wire.md +++ b/spec/value-wire.md @@ -1,7 +1,9 @@ # Marqdo Value Wire Format v1 -Status: **frozen for M0.5**. Changes require an ADR, a `wire_version` bump and -cross-ABI compatibility tests. +Status: **frozen for M0.5; extended by ADR-0022 (non-transferable `Memory`)**. +Changes require an ADR, a `wire_version` bump and cross-ABI compatibility +tests. Adding a rejected tag is not a version bump; accepting a new value shape +is. Canonical implementation: `crates/marqdo-value/src/wire.rs` (`to_wire`, `from_wire`, `from_wire_with_limits`, `WireLimits`, `WireError`). @@ -14,12 +16,20 @@ documented in [`foreign.md`](foreign.md) and implemented in ## 1. Why a typed wire format JSON has one numeric type. Marqdo has arbitrary-precision `Int`, IEEE-754 -`Float` (including `NaN` and both infinities) and 13 distinct runtime value +`Float` (including `NaN` and both infinities) and 15 distinct runtime value types (`crates/marqdo-value/src/value.rs`). Encoding an `Int` as a JSON number would silently lose digits above 2^53 and would make `Int(1)` and `Float(1.0)` indistinguishable after a round trip. The wire format is therefore **fully type-tagged** and is not plain JSON. +Addressable `Memory` and host `Resource` handles are deliberately **not +transferable**. `to_wire` emits +`{"type":"memory"}` only so that an accidental transfer is visible; decoding +that tag is always `WireError::Invalid`, and the host boundary rejects any +argument containing a live handle before an ABI call. Copying bytes requires an +explicit standard-library snapshot to a managed value, not serializing a +handle (ADR-0022). + The rules that follow from this: 1. Every value is a JSON **object** with a mandatory `"type"` string. @@ -81,6 +91,8 @@ The table is normative and mirrors `encode`/`decode` in | `Code` | `{"type":"code","language":"python","source":"...","content_hash":""}` plus optional `"unit"` and `"span"` | `language` and `content_hash` default to `""` on decode when absent; `source` is required. | | `FunctionRef` | `{"type":"function_ref","module":"index.mq.md","name":"main"}` | Both fields required strings. | | `PluginHandle` | `{"type":"plugin_handle","plugin":"demo.solver","export":"solver.add","id":1}` | `id` must be a non-negative JSON integer (`as_u64`). | +| `Memory` | `{"type":"memory"}` | **Never decodable.** No handle, address or bytes are encoded; a round trip is intentionally impossible (`wire::tests::memory_handles_are_not_transferable`). | +| `Resource` | `{"type":"resource"}` | **Never decodable.** Host-private stream handle; `from_wire` rejects the tag (`resource_handles_are_not_transferable`, ADR-0025). | Optional fields (`unit`, `span`, `display`, `language`, `content_hash`) are emitted only when present on the value. Encoders always emit `display` for diff --git a/tests/README.md b/tests/README.md index 73506f7..934c913 100644 --- a/tests/README.md +++ b/tests/README.md @@ -15,11 +15,11 @@ no test commits, copies or rewrites files outside a throw-away temp directory. tests/ ├── conformance/ │ ├── ast/ 160 × *.mq.md + sibling *.ast.json goldens -│ └── semantics/ 44 × *.mq.md + sibling *.expected.json goldens +│ └── semantics/ 68 × *.mq.md + sibling *.expected.json goldens ├── negative/ │ ├── *.mq.md 52 × compile-time hard-error fixtures + *.expected.json │ ├── warnings/ 6 × inert-executable warning fixtures + *.expected.json -│ └── runtime/ 12 × runtime-error fixtures + *.expected.json +│ └── runtime/ 21 × runtime-error fixtures + *.expected.json └── README.md this file ``` @@ -69,6 +69,130 @@ The power/control-flow slice is pinned by: * `crates/marqdo-frontend/src/payload.rs` span tests and `crates/cli/tests/integration_cli.rs` CLI span/E2E assertions. +The addressable-memory slice (RFC-CM P1, ADR-0022) is pinned by: + +* semantic fixtures 45–47: sharing a region by handle, per-region isolation, + `memory.size`, `memory.grow` returning the old count, preserved old bytes and + zero-filled new pages; +* runtime negatives 13–18: out-of-bounds and huge offsets, byte values outside + `0..=255`, index assignment on a `List`, growth beyond the declared maximum + and negative page counts; +* `tests/negative/runtime/15-index-assign-non-memory.mq.md` also guards the v1 + value semantics of `List` (it must not become shared mutable memory); +* `crates/marqdo-runtime/src/memory.rs` unit tests (aliasing, atomic failed + growth, forged/stale handles, page arithmetic) and + `crates/marqdo-runtime/tests/interpreter.rs` host-boundary/budget tests; +* `crates/marqdo-value/src/wire.rs` proves live handles are not transferable; +* `examples/memory-buffer.mq.md` and the `examples_r6` CLI test prove the + `.mq.md` surface end to end. + +Partial RFC-CM P2 (compute closure) is demonstrated, not certified, by: + +* semantic fixtures 48–50: a big-endian u32 codec and bump allocator, three + allocated nodes linked and traversed, and a failed allocation that leaves + the allocator header unchanged. These documents use only language builtins + and control flow; no Rust algorithm is added for them. +* The full P2 gate also needs a free-list allocator with reuse/coalescing, + tree/graph structures, a recoverable `Result` model and an independent + stdlib repository. Those are **not** implemented. + +CM-A04 structure primitives (ADR-0023) are pinned by: + +* semantic fixtures 51–54: value semantics of `list.append`/`map.set`/ + `map.remove` (inputs unchanged, keys sorted, `map.has`/`map.get`), + Unicode scalar round-trip through `text.codepoints`/ + `text.from_codepoint` (including invalid scalars → `None`), + `table.columns`/`table.row`/`table.cell` projection, and a `.mq.md` + `过滤` function that rebuilds a filtered GFM-style table without mutating + the input or adding a Rust table algorithm; +* runtime negatives 19–21: `list.append` on a Map, table row out of bounds, + and a non-Text `map.get` key; +* `crates/marqdo-runtime/src/builtins_impl.rs`; no algorithm builtin such as + sort/filter/join/JSON/CSV is added. + +CM-A05 recoverable results and the first-fit allocator (ADR-0024) are pinned +by: + +* semantic fixture 55: a `.mq.md` first-fit/free-list allocator with 8-byte + block headers, free-block reuse, block splitting, address-adjacent + coalescing (a second region fills its bump space, so only the merged block + can satisfy a 40-byte request), a too-large request returning + `{ok:false, code:"alloc.no_space", ...}`, and payload writes that show the + reused blocks do not overlap; +* the `Result` shape is an ordinary Map; no `Result` value type, exception, + `try`/`catch` or propagation syntax is added. The ADR records the + success/failure field contract and the fatal-vs-recoverable boundary; +* semantic fixture 56: a `Memory`-backed binary search tree with 16-byte nodes, + recursive insertion and recursive in-order traversal built from + `list.concat`; the returned root/left/right offsets and sorted values pin + node layout and traversal; +* semantic fixture 57: a `.mq.md` CSV single-row splitter using + `text.codepoints`, `text.from_codepoint`, `list.append` and `list.concat`; + it preserves empty fields and does **not** pretend to support quoted or + escaped CSV; +* semantic fixture 58: an open-addressing hash table with linear probing, + collision handling (keys 5 and 37), value update, not-found `None`, element + count, and a full-table `hash.full` Result in a second region; +* semantic fixture 59: a `.mq.md` UTF-8 encoder that writes 1–4 byte sequences + into `Memory` from `text.codepoints`, returning the byte length and bytes; +* semantic fixture 60: a validating `.mq.md` UTF-8 decoder over `Memory` with + explicit `utf8.truncated`, `utf8.invalid_continuation`, `utf8.invalid_scalar` + and `utf8.invalid_lead` failures; valid ASCII, 2/3/4-byte and mixed text is + decoded; +* semantic fixture 61: a `Memory`-backed graph adjacency list with 16-byte nodes + and 8-byte edges, head-inserted edges and a neighbor traversal; no Rust graph + container or new `Value` type; +* semantic fixture 62: a recursive `.mq.md` JSON subset parser for nested + arrays and objects, strings with standard escapes including `\uXXXX`, + integers (including negative), `true`/`false`/`null`, whitespace, `next` + positions and `Result` errors (`json.unexpected_eof`, `json.invalid_array`, + `json.invalid_object`, `json.invalid_string`, `json.invalid_number`, + `json.invalid_literal`, `json.unexpected_token`). Floats are explicitly out + of scope; +* semantic fixture 63: a `Memory`-backed stack bytecode VM with instruction + decode, PUSH/ADD/MUL/JMP/JZ/HALT, a stack region and two programs (arithmetic + and conditional branch); no Rust VM or interpreter loop was added; +* semantic fixture 64: `.mq.md` big-endian 16/32/64-bit encoders/decoders and + LEB128 varint encode/decode, including the `u64` maximum and byte-level + round-trips; +* semantic fixture 65: a `.mq.md` table sort built on `table.row`, + `table.columns`, `table.cell`, `表` and `list.append`; insertion sort + rebuilds a new table and leaves the input table unchanged; +* semantic fixture 67: a `.mq.md` table equality join over two tables, using + `table.row/columns/cell`, `list.concat`, `list.append` and `表`; both input + tables remain unchanged; +* semantic fixture 68: a quoted/escaped CSV single-row parser using Unicode + scalar primitives; `""` is a literal quote and commas inside quotes are + preserved. Multi-line fields and whitespace trimming are out of scope. + +RFC-CM CM-A07 Compat-A (ADR-0027) is pinned by: + +* `examples/stdlib/{format,text,io}.mq.md` and `examples/stdlib/demo.mq.md`; +* `crates/cli/tests/examples_r6.rs::mqmd_stdlib_print_output_matches_builtin_print`, + which runs the demo and asserts `io.写值` and builtin `打印` lines are + byte-identical for a nested List/Map/None value and a Table; +* the demo covers Unit/None/Bool? (Bool appears inside the nested list), + Int, Float, Text, List, Map and Table. Formula/Code/FunctionRef/PluginHandle + are explicitly not covered; `打印` is not yet routed to the stdlib. + +RFC-CM CM-A06 resource streams (ADR-0025) are pinned by: + +* semantic fixture 66: `.mq.md` UTF-8 text encoded into `Memory`, written to a + `stdio/stdout` resource, closed, then re-used to prove + `{ok:false,code:"resource.closed"}`; the expected `stdout` is the decoded + text; +* `crates/marqdo-host/tests/resource_streams.rs` (9): pure-mode stdout write, + output-budget atomicity, strict refusal, project-file chunking/EOF, missing + grant, traversal rejection and unsupported rights/kinds; +* `crates/cli/tests/resource_streams.rs`: real `marqdo run` with + `--allow-read data.txt`, three chunked reads into `Memory` plus EOF, the same + document failing with `permission.denied` without the grant, and a + write-then-read program with `--allow-write/--allow-read out.bin` proving the + file is created only at `resource.close`, and a chunked `.mq.md` file copy + (`resource.read` → `resource.write`) that reproduces `data.txt` in `out.bin`; +* `crates/marqdo-host/src/resource.rs` unit tests for forged/closed handles, + stdout read/file write unsupported, and chunked EOF. + Fixture discovery is directory-based and sorted by file name, so the order is deterministic on every platform. The AST and semantic suites fail loudly if the corpus shrinks below 100 / 25 fixtures, and the negative suite fails below 30 diff --git a/tests/conformance/semantics/45-memory-buffer.expected.json b/tests/conformance/semantics/45-memory-buffer.expected.json new file mode 100644 index 0000000..a292b48 --- /dev/null +++ b/tests/conformance/semantics/45-memory-buffer.expected.json @@ -0,0 +1,9 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "int", + "decimal": "141" + } +} diff --git a/tests/conformance/semantics/45-memory-buffer.mq.md b/tests/conformance/semantics/45-memory-buffer.mq.md new file mode 100644 index 0000000..e3c4cdf --- /dev/null +++ b/tests/conformance/semantics/45-memory-buffer.mq.md @@ -0,0 +1,14 @@ +## main + +This function demonstrates an addressable byte region next to ordinary prose. +A copied name shares the same region identity. + +**区域 = memory.new(1, 4)** + +**`区域[8] = 42`** + +**另一个名字 = 区域** + +**`另一个名字[9] = 99`** + +*区域[8] + 区域[9]* diff --git a/tests/conformance/semantics/46-memory-grow-zero-fill.expected.json b/tests/conformance/semantics/46-memory-grow-zero-fill.expected.json new file mode 100644 index 0000000..cb23ed3 --- /dev/null +++ b/tests/conformance/semantics/46-memory-grow-zero-fill.expected.json @@ -0,0 +1,14 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + {"type": "int", "decimal": "0"}, + {"type": "int", "decimal": "1"}, + {"type": "int", "decimal": "9"}, + {"type": "int", "decimal": "0"} + ] + } +} diff --git a/tests/conformance/semantics/46-memory-grow-zero-fill.mq.md b/tests/conformance/semantics/46-memory-grow-zero-fill.mq.md new file mode 100644 index 0000000..0b578ac --- /dev/null +++ b/tests/conformance/semantics/46-memory-grow-zero-fill.mq.md @@ -0,0 +1,14 @@ +## main + +A zero-page region is legal. Growing it returns the previous page count, +preserves old bytes and zero-fills every new byte. + +**内存 = memory.new(0, 2)** + +**旧页数 = memory.grow(内存, 1)** + +**内存[0] = 9** + +**页数 = memory.size(内存)** + +*[旧页数, 页数, 内存[0], 内存[65535]]* diff --git a/tests/conformance/semantics/47-memory-isolation-and-alias.expected.json b/tests/conformance/semantics/47-memory-isolation-and-alias.expected.json new file mode 100644 index 0000000..ec21dec --- /dev/null +++ b/tests/conformance/semantics/47-memory-isolation-and-alias.expected.json @@ -0,0 +1,15 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + {"type": "int", "decimal": "5"}, + {"type": "int", "decimal": "6"}, + {"type": "int", "decimal": "7"}, + {"type": "bool", "value": true}, + {"type": "bool", "value": false} + ] + } +} diff --git a/tests/conformance/semantics/47-memory-isolation-and-alias.mq.md b/tests/conformance/semantics/47-memory-isolation-and-alias.mq.md new file mode 100644 index 0000000..9bca1cd --- /dev/null +++ b/tests/conformance/semantics/47-memory-isolation-and-alias.mq.md @@ -0,0 +1,18 @@ +## main + +Two regions never alias even at the same offset. A copied name does share +the region identity, and equality compares that identity, not the bytes. + +**甲 = memory.new(1, 1)** + +**乙 = memory.new(1, 1)** + +**`甲[0] = 5`** + +**`乙[0] = 6`** + +**别名 = 甲** + +**`别名[1] = 7`** + +*[甲[0], 乙[0], 甲[1], 甲 == 别名, 甲 == 乙]* diff --git a/tests/conformance/semantics/48-memory-bump-allocator.expected.json b/tests/conformance/semantics/48-memory-bump-allocator.expected.json new file mode 100644 index 0000000..fe2da8b --- /dev/null +++ b/tests/conformance/semantics/48-memory-bump-allocator.expected.json @@ -0,0 +1,15 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + {"type": "int", "decimal": "4"}, + {"type": "int", "decimal": "12"}, + {"type": "int", "decimal": "28"}, + {"type": "int", "decimal": "36"}, + {"type": "int", "decimal": "255"} + ] + } +} diff --git a/tests/conformance/semantics/48-memory-bump-allocator.mq.md b/tests/conformance/semantics/48-memory-bump-allocator.mq.md new file mode 100644 index 0000000..ba8fc45 --- /dev/null +++ b/tests/conformance/semantics/48-memory-bump-allocator.mq.md @@ -0,0 +1,72 @@ +## 写32 + +`(内存, 基址, 值)` + +把非负 Int 的低 32 位按大端字节写入区域。 + +**字节3 = 值 % 256** + +**字节2 = 值 // 256 % 256** + +**字节1 = 值 // 65536 % 256** + +**字节0 = 值 // 16777216 % 256** + +**`内存[基址] = 字节0`** + +**`内存[基址 + 1] = 字节1`** + +**`内存[基址 + 2] = 字节2`** + +**`内存[基址 + 3] = 字节3`** + +*None* + +## 读32 + +`(内存, 基址)` + +同样按大端读取四个字节。 + +*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]* + +## 分配 + +`(内存, 容量, 大小)` + +区域头四个字节保存下一个空闲偏移。这是顺序分配器:free 只把块交还给调用方, +不在这里实现复用。 + +**当前位置 = 读32(内存, 0)** + +1. `当前位置 + 大小 <= 容量` + + > 写32(内存, 0, 当前位置 + 大小) + + *当前位置* + +2. `true` + + *None* + +## main + +**内存 = memory.new(1, 1)** + +**`内存[0] = 0`** + +**`内存[1] = 0`** + +**`内存[2] = 0`** + +**`内存[3] = 4`** + +**第一个 = 分配(内存, 65536, 8)** + +**第二个 = 分配(内存, 65536, 16)** + +**第三个 = 分配(内存, 65536, 8)** + +**`内存[第一个] = 255`** + +*[第一个, 第二个, 第三个, 读32(内存, 0), 内存[第一个]]* diff --git a/tests/conformance/semantics/49-memory-linked-list.expected.json b/tests/conformance/semantics/49-memory-linked-list.expected.json new file mode 100644 index 0000000..dafd343 --- /dev/null +++ b/tests/conformance/semantics/49-memory-linked-list.expected.json @@ -0,0 +1,14 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + {"type": "int", "decimal": "4"}, + {"type": "int", "decimal": "12"}, + {"type": "int", "decimal": "20"}, + {"type": "int", "decimal": "60"} + ] + } +} diff --git a/tests/conformance/semantics/49-memory-linked-list.mq.md b/tests/conformance/semantics/49-memory-linked-list.mq.md new file mode 100644 index 0000000..83d176b --- /dev/null +++ b/tests/conformance/semantics/49-memory-linked-list.mq.md @@ -0,0 +1,93 @@ +## 写32 + +`(内存, 基址, 值)` + +**字节3 = 值 % 256** + +**字节2 = 值 // 256 % 256** + +**字节1 = 值 // 65536 % 256** + +**字节0 = 值 // 16777216 % 256** + +**`内存[基址] = 字节0`** + +**`内存[基址 + 1] = 字节1`** + +**`内存[基址 + 2] = 字节2`** + +**`内存[基址 + 3] = 字节3`** + +*None* + +## 读32 + +`(内存, 基址)` + +*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]* + +## 分配 + +`(内存, 容量, 大小)` + +**当前位置 = 读32(内存, 0)** + +1. `当前位置 + 大小 <= 容量` + + > 写32(内存, 0, 当前位置 + 大小) + + *当前位置* + +2. `true` + + *None* + +## 节点 + +`(内存, 值, 下一个)` + +节点是 8 字节块:偏移 0 存一个字节的值,偏移 1..4 存 next 偏移;0 表示空。 + +**地址 = 分配(内存, 65536, 8)** + +**`内存[地址] = 值`** + +> 写32(内存, 地址 + 1, 下一个) + +*地址* + +## 遍历求和 + +`(内存, 头)` + +**当前 = 头** + +**总和 = 0** + +- `当前 != 0` + + **总和 = 总和 + 内存[当前]** + + **当前 = 读32(内存, 当前 + 1)** + +*总和* + +## main + +**内存 = memory.new(1, 1)** + +**`内存[0] = 0`** + +**`内存[1] = 0`** + +**`内存[2] = 0`** + +**`内存[3] = 4`** + +**甲 = 节点(内存, 10, 0)** + +**乙 = 节点(内存, 20, 甲)** + +**丙 = 节点(内存, 30, 乙)** + +*[甲, 乙, 丙, 遍历求和(内存, 丙)]* diff --git a/tests/conformance/semantics/50-memory-allocation-failure-atomic.expected.json b/tests/conformance/semantics/50-memory-allocation-failure-atomic.expected.json new file mode 100644 index 0000000..d9d017e --- /dev/null +++ b/tests/conformance/semantics/50-memory-allocation-failure-atomic.expected.json @@ -0,0 +1,14 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + {"type": "none"}, + {"type": "int", "decimal": "4"}, + {"type": "int", "decimal": "4"}, + {"type": "int", "decimal": "65536"} + ] + } +} diff --git a/tests/conformance/semantics/50-memory-allocation-failure-atomic.mq.md b/tests/conformance/semantics/50-memory-allocation-failure-atomic.mq.md new file mode 100644 index 0000000..7833c61 --- /dev/null +++ b/tests/conformance/semantics/50-memory-allocation-failure-atomic.mq.md @@ -0,0 +1,67 @@ +## 写32 + +`(内存, 基址, 值)` + +**字节3 = 值 % 256** + +**字节2 = 值 // 256 % 256** + +**字节1 = 值 // 65536 % 256** + +**字节0 = 值 // 16777216 % 256** + +**`内存[基址] = 字节0`** + +**`内存[基址 + 1] = 字节1`** + +**`内存[基址 + 2] = 字节2`** + +**`内存[基址 + 3] = 字节3`** + +*None* + +## 读32 + +`(内存, 基址)` + +*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]* + +## 分配 + +`(内存, 容量, 大小)` + +**当前位置 = 读32(内存, 0)** + +1. `当前位置 + 大小 <= 容量` + + > 写32(内存, 0, 当前位置 + 大小) + + *当前位置* + +2. `true` + + *None* + +## main + +分配失败必须保持区域头不变,之后的分配仍从原位置继续。 + +**内存 = memory.new(1, 1)** + +**`内存[0] = 0`** + +**`内存[1] = 0`** + +**`内存[2] = 0`** + +**`内存[3] = 4`** + +**失败 = 分配(内存, 65536, 65536)** + +**失败后位置 = 读32(内存, 0)** + +**成功 = 分配(内存, 65536, 65532)** + +**最终位置 = 读32(内存, 0)** + +*[失败, 失败后位置, 成功, 最终位置]* diff --git a/tests/conformance/semantics/51-list-map-value-semantics.expected.json b/tests/conformance/semantics/51-list-map-value-semantics.expected.json new file mode 100644 index 0000000..77e76ad --- /dev/null +++ b/tests/conformance/semantics/51-list-map-value-semantics.expected.json @@ -0,0 +1,45 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + { + "type": "list", + "items": [ + {"type": "int", "decimal": "1"}, + {"type": "int", "decimal": "2"} + ] + }, + { + "type": "list", + "items": [ + {"type": "int", "decimal": "1"}, + {"type": "int", "decimal": "2"}, + {"type": "int", "decimal": "3"} + ] + }, + {"type": "int", "decimal": "2"}, + {"type": "int", "decimal": "2"}, + {"type": "int", "decimal": "2"}, + { + "type": "list", + "items": [ + {"type": "text", "value": "b"}, + {"type": "text", "value": "c"} + ] + }, + {"type": "bool", "value": false}, + {"type": "none"}, + { + "type": "list", + "items": [ + {"type": "int", "decimal": "1"}, + {"type": "int", "decimal": "2"}, + {"type": "int", "decimal": "3"} + ] + } + ] + } +} diff --git a/tests/conformance/semantics/51-list-map-value-semantics.mq.md b/tests/conformance/semantics/51-list-map-value-semantics.mq.md new file mode 100644 index 0000000..e5cba46 --- /dev/null +++ b/tests/conformance/semantics/51-list-map-value-semantics.mq.md @@ -0,0 +1,18 @@ +## main + +Lists and maps keep value semantics: constructors return new values and the +inputs are unchanged. + +**原列表 = [1, 2]** + +**新列表 = list.append(原列表, 3)** + +**原映射 = {a: 1, b: 2}** + +**新映射 = map.set(原映射, "c", 3)** + +**删除后 = map.remove(新映射, "a")** + +**键 = map.keys(删除后)** + +*[原列表, 新列表, 原映射.b, 新映射.b, 删除后.b, 键, map.has(原映射, "c"), map.get(原映射, "z"), list.concat([1], [2, 3])]* diff --git a/tests/conformance/semantics/52-text-codepoints.expected.json b/tests/conformance/semantics/52-text-codepoints.expected.json new file mode 100644 index 0000000..87fe67a --- /dev/null +++ b/tests/conformance/semantics/52-text-codepoints.expected.json @@ -0,0 +1,22 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + { + "type": "list", + "items": [ + {"type": "int", "decimal": "97"}, + {"type": "int", "decimal": "233"}, + {"type": "int", "decimal": "20013"} + ] + }, + {"type": "text", "value": "中"}, + {"type": "none"}, + {"type": "none"}, + {"type": "int", "decimal": "1"} + ] + } +} diff --git a/tests/conformance/semantics/52-text-codepoints.mq.md b/tests/conformance/semantics/52-text-codepoints.mq.md new file mode 100644 index 0000000..f33a70c --- /dev/null +++ b/tests/conformance/semantics/52-text-codepoints.mq.md @@ -0,0 +1,7 @@ +## main + +**点 = text.codepoints("aé中")** + +**回 = text.from_codepoint(20013)** + +*[点, 回, text.from_codepoint(55296), text.from_codepoint(1114112), 长度(回)]* diff --git a/tests/conformance/semantics/53-table-projection.expected.json b/tests/conformance/semantics/53-table-projection.expected.json new file mode 100644 index 0000000..4ebc8f9 --- /dev/null +++ b/tests/conformance/semantics/53-table-projection.expected.json @@ -0,0 +1,26 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + { + "type": "list", + "items": [ + {"type": "text", "value": "名称"}, + {"type": "text", "value": "分数"} + ] + }, + { + "type": "list", + "items": [ + {"type": "text", "value": "乙"}, + {"type": "int", "decimal": "92"} + ] + }, + {"type": "int", "decimal": "87"}, + {"type": "int", "decimal": "2"} + ] + } +} diff --git a/tests/conformance/semantics/53-table-projection.mq.md b/tests/conformance/semantics/53-table-projection.mq.md new file mode 100644 index 0000000..2de5f6b --- /dev/null +++ b/tests/conformance/semantics/53-table-projection.mq.md @@ -0,0 +1,5 @@ +## main + +**数据 = 表(["名称", "分数"], [["甲", 87], ["乙", 92]])** + +*[table.columns(数据), table.row(数据, 1), table.cell(数据, 0, 1), 长度(数据)]* diff --git a/tests/conformance/semantics/54-table-filter-stdlib.expected.json b/tests/conformance/semantics/54-table-filter-stdlib.expected.json new file mode 100644 index 0000000..83bd221 --- /dev/null +++ b/tests/conformance/semantics/54-table-filter-stdlib.expected.json @@ -0,0 +1,42 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "type": "list", + "items": [ + { + "type": "table", + "header": ["名称", "分数"], + "rows": [ + [ + {"type": "text", "value": "甲"}, + {"type": "int", "decimal": "87"} + ], + [ + {"type": "text", "value": "乙"}, + {"type": "int", "decimal": "92"} + ], + [ + {"type": "text", "value": "丙"}, + {"type": "int", "decimal": "71"} + ] + ] + }, + { + "type": "table", + "header": ["名称", "分数"], + "rows": [ + [ + {"type": "text", "value": "甲"}, + {"type": "int", "decimal": "87"} + ], + [ + {"type": "text", "value": "乙"}, + {"type": "int", "decimal": "92"} + ] + ] + } + ] + } +} diff --git a/tests/conformance/semantics/54-table-filter-stdlib.mq.md b/tests/conformance/semantics/54-table-filter-stdlib.mq.md new file mode 100644 index 0000000..684396a --- /dev/null +++ b/tests/conformance/semantics/54-table-filter-stdlib.mq.md @@ -0,0 +1,31 @@ +## 过滤 + +`(数据, 阈值)` + +只用 table 观察原语和 list 构造原语重建一个新表;输入表不被修改。 + +**列 = table.columns(数据)** + +**结果行 = []** + +**行号 = 0** + +- `行号 < 长度(数据)` + + **分数 = table.cell(数据, 行号, 1)** + + 1. `分数 > 阈值` + + **结果行 = list.append(结果行, table.row(数据, 行号))** + + **行号 = 行号 + 1** + +*表(列, 结果行)* + +## main + +**原表 = 表(["名称", "分数"], [["甲", 87], ["乙", 92], ["丙", 71]])** + +**结果 = 过滤(原表, 80)** + +*[原表, 结果]* diff --git a/tests/conformance/semantics/55-first-fit-free-list-allocator.expected.json b/tests/conformance/semantics/55-first-fit-free-list-allocator.expected.json new file mode 100644 index 0000000..bbdec4b --- /dev/null +++ b/tests/conformance/semantics/55-first-fit-free-list-allocator.expected.json @@ -0,0 +1,262 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "16", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "32", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "16", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "16", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "32", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "56", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "56", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "72", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "96", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "112", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "code": { + "type": "text", + "value": "alloc.no_space" + }, + "message": { + "type": "text", + "value": "没有足够的空闲空间" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "16", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "32", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "48", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "16", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "code": { + "type": "text", + "value": "alloc.no_space" + }, + "message": { + "type": "text", + "value": "没有足够的空闲空间" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + }, + { + "decimal": "33", + "type": "int" + }, + { + "decimal": "22", + "type": "int" + }, + { + "decimal": "33", + "type": "int" + }, + { + "decimal": "33", + "type": "int" + }, + { + "decimal": "22", + "type": "int" + }, + { + "decimal": "44", + "type": "int" + }, + { + "decimal": "55", + "type": "int" + }, + { + "decimal": "66", + "type": "int" + }, + { + "decimal": "77", + "type": "int" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md b/tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md new file mode 100644 index 0000000..17e3b2f --- /dev/null +++ b/tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md @@ -0,0 +1,308 @@ +## 写32 + +`(内存, 基址, 值)` + +**字节3 = 值 % 256** + +**字节2 = 值 // 256 % 256** + +**字节1 = 值 // 65536 % 256** + +**字节0 = 值 // 16777216 % 256** + +**`内存[基址] = 字节0`** + +**`内存[基址 + 1] = 字节1`** + +**`内存[基址 + 2] = 字节2`** + +**`内存[基址 + 3] = 字节3`** + +*None* + +## 读32 + +`(内存, 基址)` + +*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]* + +## 成功 + +`(值)` + +*{ok: true, value: 值}* + +## 失败 + +`(代码, 消息)` + +*{ok: false, code: 代码, message: 消息}* + +## 初始化 + +`(内存)` + +区域头:偏移 0 是空闲链表头,偏移 4 是 bump 指针;首个块从 8 开始。 + +> 写32(内存, 0, 0) + +> 写32(内存, 4, 8) + +*成功(None)* + +## 查找 + +`(内存, 需求)` + +返回 `[前一个空闲块, 当前空闲块, 当前块大小]`;没有合适块时返回零值。 + +**前一个 = 0** + +**当前 = 读32(内存, 0)** + +- `当前 != 0` + + **块大小 = 读32(内存, 当前)** + + 1. `块大小 >= 需求` + + *[前一个, 当前, 块大小]* + + **前一个 = 当前** + + **当前 = 读32(内存, 当前 + 4)** + +*[0, 0, 0]* + +## 分配 + +`(内存, 容量, 请求大小)` + +first-fit:先查空闲链表,必要时切分;否则从 bump 指针处取新块。 +块头 8 字节:偏移 0 是总块大小,偏移 4 是空闲链表 next。返回 payload 偏移。 + +**需求 = (请求大小 + 8 + 7) // 8 * 8** + +**位置 = 查找(内存, 需求)** + +**前一个 = 位置[0]** + +**当前 = 位置[1]** + +**块大小 = 位置[2]** + +1. `当前 != 0` + + **余量 = 块大小 - 需求** + + **下一个 = 读32(内存, 当前 + 4)** + + 1. `余量 >= 16` + + **余块 = 当前 + 需求** + + > 写32(内存, 余块, 余量) + + > 写32(内存, 余块 + 4, 下一个) + + 1. `前一个 == 0` + + > 写32(内存, 0, 余块) + + 2. `true` + + > 写32(内存, 前一个 + 4, 余块) + + > 写32(内存, 当前, 需求) + + *成功(当前 + 8)* + + 2. `true` + + 1. `前一个 == 0` + + > 写32(内存, 0, 下一个) + + 2. `true` + + > 写32(内存, 前一个 + 4, 下一个) + + *成功(当前 + 8)* + +2. `true` + + **下一个空位 = 读32(内存, 4)** + + 1. `下一个空位 + 需求 <= 容量` + + > 写32(内存, 下一个空位, 需求) + + > 写32(内存, 4, 下一个空位 + 需求) + + *成功(下一个空位 + 8)* + + 2. `true` + + *失败("alloc.no_space", "没有足够的空闲空间")* + +## 插入空闲 + +`(内存, 块)` + +空闲链表按地址升序;插入后先与后继合并,再与前去合并。 + +**前一个 = 0** + +**当前 = 读32(内存, 0)** + +- `当前 != 0 and 当前 < 块` + + **前一个 = 当前** + + **当前 = 读32(内存, 当前 + 4)** + +> 写32(内存, 块 + 4, 当前) + +1. `前一个 == 0` + + > 写32(内存, 0, 块) + +2. `true` + + > 写32(内存, 前一个 + 4, 块) + +> 合并(内存, 前一个, 块, 当前) + +*成功(None)* + +## 合并 + +`(内存, 前一个, 块, 当前)` + +相邻块只有在地址连续时才合并,避免产生虚假的大块。 + +**块大小 = 读32(内存, 块)** + +1. `当前 != 0 and 块 + 块大小 == 当前` + + **合并大小 = 块大小 + 读32(内存, 当前)** + + > 写32(内存, 块, 合并大小) + + > 写32(内存, 块 + 4, 读32(内存, 当前 + 4)) + +2. `true` + + **忽略后继合并 = 0** + +**前一个大小 = 0** + +1. `前一个 != 0` + + **前一个大小 = 读32(内存, 前一个)** + +2. `true` + + **忽略前去读取 = 0** + +后继合并完成后再处理前驱;两个判断必须分开执行。 + +1. `前一个 != 0 and 前一个 + 前一个大小 == 块` + + **前一个新大小 = 前一个大小 + 读32(内存, 块)** + + > 写32(内存, 前一个, 前一个新大小) + + > 写32(内存, 前一个 + 4, 读32(内存, 块 + 4)) + +2. `true` + + **忽略前去合并 = 0** + +*成功(None)* + +## 释放 + +`(内存, 地址)` + +把 payload 偏移换算回块头,按地址插入空闲链表并由“合并”处理相邻块。 + +**块 = 地址 - 8** + +> 插入空闲(内存, 块) + +*成功(None)* + +## main + +**内存 = memory.new(1, 1)** + +> 初始化(内存) + +**甲 = 分配(内存, 65536, 8)** + +**乙 = 分配(内存, 65536, 16)** + +**`内存[甲.value] = 11`** + +**`内存[乙.value] = 22`** + +> 释放(内存, 甲.value) + +**丙 = 分配(内存, 65536, 8)** + +**`内存[丙.value] = 33`** + +> 释放(内存, 乙.value) + +> 释放(内存, 丙.value) + +**丁 = 分配(内存, 65536, 4)** + +**戊 = 分配(内存, 65536, 16)** + +**己 = 分配(内存, 65536, 64)** + +> 释放(内存, 己.value) + +**庚 = 分配(内存, 65536, 8)** + +**辛 = 分配(内存, 65536, 16)** + +**壬 = 分配(内存, 65536, 8)** + +**癸 = 分配(内存, 65536, 8)** + +**`内存[庚.value] = 44`** + +**`内存[辛.value] = 55`** + +**`内存[壬.value] = 66`** + +**`内存[癸.value] = 77`** + +**失败结果 = 分配(内存, 65536, 100000)** + +**内存2 = memory.new(1, 1)** + +> 初始化(内存2) + +**甲2 = 分配(内存2, 65536, 8)** + +**乙2 = 分配(内存2, 65536, 8)** + +**丙2 = 分配(内存2, 65536, 8)** + +> 分配(内存2, 65536, 65472) + +> 释放(内存2, 甲2.value) + +> 释放(内存2, 乙2.value) + +> 释放(内存2, 丙2.value) + +**丁2 = 分配(内存2, 65536, 40)** + +**戊2 = 分配(内存2, 65536, 8)** + +*[甲, 乙, 丙, 丁, 戊, 己, 庚, 辛, 壬, 癸, 失败结果, 甲2, 乙2, 丙2, 丁2, 戊2, 内存[甲.value], 内存[乙.value], 内存[丙.value], 内存[丁.value], 内存[戊.value], 内存[庚.value], 内存[辛.value], 内存[壬.value], 内存[癸.value]]* diff --git a/tests/conformance/semantics/56-memory-binary-tree.expected.json b/tests/conformance/semantics/56-memory-binary-tree.expected.json new file mode 100644 index 0000000..ce9c410 --- /dev/null +++ b/tests/conformance/semantics/56-memory-binary-tree.expected.json @@ -0,0 +1,55 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "decimal": "8", + "type": "int" + }, + { + "items": [ + { + "decimal": "1", + "type": "int" + }, + { + "decimal": "3", + "type": "int" + }, + { + "decimal": "4", + "type": "int" + }, + { + "decimal": "5", + "type": "int" + }, + { + "decimal": "7", + "type": "int" + }, + { + "decimal": "8", + "type": "int" + }, + { + "decimal": "9", + "type": "int" + } + ], + "type": "list" + }, + { + "decimal": "24", + "type": "int" + }, + { + "decimal": "40", + "type": "int" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/56-memory-binary-tree.mq.md b/tests/conformance/semantics/56-memory-binary-tree.mq.md new file mode 100644 index 0000000..29dc859 --- /dev/null +++ b/tests/conformance/semantics/56-memory-binary-tree.mq.md @@ -0,0 +1,112 @@ +## 写32 + +`(内存, 基址, 值)` + +**字节3 = 值 % 256** + +**字节2 = 值 // 256 % 256** + +**字节1 = 值 // 65536 % 256** + +**字节0 = 值 // 16777216 % 256** + +**`内存[基址] = 字节0`** + +**`内存[基址 + 1] = 字节1`** + +**`内存[基址 + 2] = 字节2`** + +**`内存[基址 + 3] = 字节3`** + +*None* + +## 读32 + +`(内存, 基址)` + +*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]* + +## 新节点 + +`(内存, 值)` + +每个节点 16 字节:偏移 0 存一个字节的值,偏移 4 存左子,偏移 8 存右子;0 表示空。 +节点值必须落在 0..255。 + +**位置 = 读32(内存, 0)** + +**`内存[位置] = 值`** + +> 写32(内存, 位置 + 4, 0) + +> 写32(内存, 位置 + 8, 0) + +> 写32(内存, 0, 位置 + 16) + +*位置* + +## 插入 + +`(内存, 根, 值)` + +1. `根 == 0` + + *新节点(内存, 值)* + +2. `值 < 内存[根]` + + **左 = 插入(内存, 读32(内存, 根 + 4), 值)** + + > 写32(内存, 根 + 4, 左) + + *根* + +3. `true` + + **右 = 插入(内存, 读32(内存, 根 + 8), 值)** + + > 写32(内存, 根 + 8, 右) + + *根* + +## 遍历 + +`(内存, 根)` + +中序遍历,返回从小到大的值列表。 + +1. `根 == 0` + + *[]* + +**左 = 遍历(内存, 读32(内存, 根 + 4))** + +**右 = 遍历(内存, 读32(内存, 根 + 8))** + +*list.concat(list.concat(左, [内存[根]]), 右)* + +## main + +**内存 = memory.new(1, 1)** + +**`内存[0] = 0`** + +**`内存[1] = 0`** + +**`内存[2] = 0`** + +**`内存[3] = 8`** + +**根 = 0** + +**值表 = [5, 3, 8, 1, 4, 7, 9]** + +**下标 = 0** + +- `下标 < 长度(值表)` + + **根 = 插入(内存, 根, 值表[下标])** + + **下标 = 下标 + 1** + +*[根, 遍历(内存, 根), 读32(内存, 根 + 4), 读32(内存, 根 + 8)]* diff --git a/tests/conformance/semantics/57-csv-row-splitter.expected.json b/tests/conformance/semantics/57-csv-row-splitter.expected.json new file mode 100644 index 0000000..16ee878 --- /dev/null +++ b/tests/conformance/semantics/57-csv-row-splitter.expected.json @@ -0,0 +1,53 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "items": [ + { + "type": "text", + "value": "甲" + }, + { + "type": "text", + "value": "乙" + }, + { + "type": "text", + "value": "" + }, + { + "type": "text", + "value": "丙" + } + ], + "type": "list" + }, + { + "items": [ + { + "type": "text", + "value": "" + } + ], + "type": "list" + }, + { + "items": [ + { + "type": "text", + "value": "a" + }, + { + "type": "text", + "value": "b" + } + ], + "type": "list" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/57-csv-row-splitter.mq.md b/tests/conformance/semantics/57-csv-row-splitter.mq.md new file mode 100644 index 0000000..fbbc5e1 --- /dev/null +++ b/tests/conformance/semantics/57-csv-row-splitter.mq.md @@ -0,0 +1,35 @@ +## 分割 + +`(文本)` + +这是 CSV 单行拆分的最小实现:只处理逗号分隔,不支持引号或转义;空字段保留。 + +**结果 = []** + +**当前 = ""** + +**点 = text.codepoints(文本)** + +**下标 = 0** + +- `下标 < 长度(点)` + + **字符 = text.from_codepoint(点[下标])** + + 1. `字符 == ","` + + **结果 = list.append(结果, 当前)** + + **当前 = ""** + + 2. `true` + + **当前 = 当前 + 字符** + + **下标 = 下标 + 1** + +*list.concat(结果, [当前])* + +## main + +*[分割("甲,乙,,丙"), 分割(""), 分割("a,b")]* diff --git a/tests/conformance/semantics/58-memory-hash-table.expected.json b/tests/conformance/semantics/58-memory-hash-table.expected.json new file mode 100644 index 0000000..64bbfd5 --- /dev/null +++ b/tests/conformance/semantics/58-memory-hash-table.expected.json @@ -0,0 +1,66 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "type": "none" + } + }, + "type": "map" + }, + { + "decimal": "55", + "type": "int" + }, + { + "decimal": "370", + "type": "int" + }, + { + "decimal": "120", + "type": "int" + }, + { + "type": "none" + }, + { + "decimal": "3", + "type": "int" + }, + { + "entries": { + "code": { + "type": "text", + "value": "hash.full" + }, + "message": { + "type": "text", + "value": "哈希表已满" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + }, + { + "decimal": "10", + "type": "int" + }, + { + "decimal": "20", + "type": "int" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/58-memory-hash-table.mq.md b/tests/conformance/semantics/58-memory-hash-table.mq.md new file mode 100644 index 0000000..c96253b --- /dev/null +++ b/tests/conformance/semantics/58-memory-hash-table.mq.md @@ -0,0 +1,164 @@ +## 写32 + +`(内存, 基址, 值)` + +**字节3 = 值 % 256** + +**字节2 = 值 // 256 % 256** + +**字节1 = 值 // 65536 % 256** + +**字节0 = 值 // 16777216 % 256** + +**`内存[基址] = 字节0`** + +**`内存[基址 + 1] = 字节1`** + +**`内存[基址 + 2] = 字节2`** + +**`内存[基址 + 3] = 字节3`** + +*None* + +## 读32 + +`(内存, 基址)` + +*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]* + +## 成功 + +`(值)` + +*{ok: true, value: 值}* + +## 失败 + +`(代码, 消息)` + +*{ok: false, code: 代码, message: 消息}* + +## 初始化 + +`(内存, 容量)` + +头部 8 字节:0..3 是元素数,4..7 是槽数。槽从偏移 8 开始,每槽 8 字节: +键 u32 + 值 u32。键 0 表示空槽,因此业务键必须从 1 开始。 + +> 写32(内存, 0, 0) + +> 写32(内存, 4, 容量) + +*成功(None)* + +## 插入 + +`(内存, 键, 值)` + +线性探测;键已存在时只更新值,不增加元素数。 + +**容量 = 读32(内存, 4)** + +**位置 = 键 % 容量** + +**尝试 = 0** + +**完成 = false** + +- `尝试 < 容量 and not 完成` + + **槽 = 8 + (位置 + 尝试) % 容量 * 8** + + **槽键 = 读32(内存, 槽)** + + 1. `槽键 == 0` + + > 写32(内存, 槽, 键) + + > 写32(内存, 槽 + 4, 值) + + > 写32(内存, 0, 读32(内存, 0) + 1) + + **完成 = true** + + 2. `槽键 == 键` + + > 写32(内存, 槽 + 4, 值) + + **完成 = true** + + 3. `true` + + **尝试 = 尝试 + 1** + +1. `完成` + + *成功(None)* + +2. `true` + + *失败("hash.full", "哈希表已满")* + +## 查找 + +`(内存, 键)` + +遇到空槽即停止;找不到返回 None。 + +**容量 = 读32(内存, 4)** + +**位置 = 键 % 容量** + +**尝试 = 0** + +**结果 = None** + +- `尝试 < 容量` + + **槽 = 8 + (位置 + 尝试) % 容量 * 8** + + **槽键 = 读32(内存, 槽)** + + 1. `槽键 == 键` + + **结果 = 读32(内存, 槽 + 4)** + + **尝试 = 容量** + + 2. `槽键 == 0` + + **尝试 = 容量** + + 3. `true` + + **尝试 = 尝试 + 1** + +*结果* + +## main + +业务键从 1 开始。37 与 5 在 32 个槽中同余,用来验证线性探测。 + +**内存 = memory.new(1, 1)** + +> 初始化(内存, 32) + +**插入5 = 插入(内存, 5, 50)** + +> 插入(内存, 37, 370) + +> 插入(内存, 12, 120) + +> 插入(内存, 5, 55) + +**内存2 = memory.new(1, 1)** + +> 初始化(内存2, 2) + +> 插入(内存2, 1, 10) + +> 插入(内存2, 2, 20) + +**表满 = 插入(内存2, 4, 40)** + +*[插入5, 查找(内存, 5), 查找(内存, 37), 查找(内存, 12), 查找(内存, 99), 读32(内存, 0), 表满, 查找(内存2, 1), 查找(内存2, 2)]* diff --git a/tests/conformance/semantics/59-utf8-encoder.expected.json b/tests/conformance/semantics/59-utf8-encoder.expected.json new file mode 100644 index 0000000..e3ae065 --- /dev/null +++ b/tests/conformance/semantics/59-utf8-encoder.expected.json @@ -0,0 +1,59 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "decimal": "10", + "type": "int" + }, + { + "items": [ + { + "decimal": "65", + "type": "int" + }, + { + "decimal": "195", + "type": "int" + }, + { + "decimal": "169", + "type": "int" + }, + { + "decimal": "228", + "type": "int" + }, + { + "decimal": "184", + "type": "int" + }, + { + "decimal": "173", + "type": "int" + }, + { + "decimal": "240", + "type": "int" + }, + { + "decimal": "157", + "type": "int" + }, + { + "decimal": "132", + "type": "int" + }, + { + "decimal": "158", + "type": "int" + } + ], + "type": "list" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/59-utf8-encoder.mq.md b/tests/conformance/semantics/59-utf8-encoder.mq.md new file mode 100644 index 0000000..897c8fe --- /dev/null +++ b/tests/conformance/semantics/59-utf8-encoder.mq.md @@ -0,0 +1,74 @@ +## 编码 + +`(文本, 内存, 基址)` + +把 Unicode 标量序列按 UTF-8 编码写入 `Memory`,返回字节长度。 +这是 `.mq.md` 自己实现的编码逻辑,不是 Rust 内建 UTF-8 转换。 + +**点 = text.codepoints(文本)** + +**下标 = 0** + +**偏移 = 0** + +- `下标 < 长度(点)` + + **码点 = 点[下标]** + + 1. `码点 < 128` + + **`内存[基址 + 偏移] = 码点`** + + **偏移 = 偏移 + 1** + + 2. `码点 < 2048` + + **`内存[基址 + 偏移] = 192 + 码点 // 64`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 2** + + 3. `码点 < 65536` + + **`内存[基址 + 偏移] = 224 + 码点 // 4096`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 // 64 % 64`** + + **`内存[基址 + 偏移 + 2] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 3** + + 4. `true` + + **`内存[基址 + 偏移] = 240 + 码点 // 262144`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 // 4096 % 64`** + + **`内存[基址 + 偏移 + 2] = 128 + 码点 // 64 % 64`** + + **`内存[基址 + 偏移 + 3] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 4** + + **下标 = 下标 + 1** + +*偏移* + +## main + +**内存 = memory.new(1, 1)** + +**长度 = 编码("Aé中𝄞", 内存, 0)** + +**字节 = []** + +**下标 = 0** + +- `下标 < 长度` + + **字节 = list.append(字节, 内存[下标])** + + **下标 = 下标 + 1** + +*[长度, 字节]* diff --git a/tests/conformance/semantics/60-utf8-decoder.expected.json b/tests/conformance/semantics/60-utf8-decoder.expected.json new file mode 100644 index 0000000..b074b67 --- /dev/null +++ b/tests/conformance/semantics/60-utf8-decoder.expected.json @@ -0,0 +1,87 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "type": "text", + "value": "Aé中" + } + }, + "type": "map" + }, + { + "entries": { + "code": { + "type": "text", + "value": "utf8.truncated" + }, + "message": { + "type": "text", + "value": "UTF-8 解码失败" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + }, + { + "entries": { + "code": { + "type": "text", + "value": "utf8.invalid_continuation" + }, + "message": { + "type": "text", + "value": "UTF-8 解码失败" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + }, + { + "entries": { + "code": { + "type": "text", + "value": "utf8.invalid_scalar" + }, + "message": { + "type": "text", + "value": "UTF-8 解码失败" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "type": "text", + "value": "𝄞" + } + }, + "type": "map" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/60-utf8-decoder.mq.md b/tests/conformance/semantics/60-utf8-decoder.mq.md new file mode 100644 index 0000000..d570b08 --- /dev/null +++ b/tests/conformance/semantics/60-utf8-decoder.mq.md @@ -0,0 +1,185 @@ +## 成功 + +`(值)` + +*{ok: true, value: 值}* + +## 失败 + +`(代码, 消息)` + +*{ok: false, code: 代码, message: 消息}* + +## 续字节 + +`(字节)` + +*字节 >= 128 and 字节 < 192* + +## 解码2 + +`(内存, 基址, 长度, 偏移, 文本)` + +1. `偏移 + 1 >= 长度` + + *[文本, 偏移, "utf8.truncated"]* + +**b1 = 内存[基址 + 偏移 + 1]** + +1. `not 续字节(b1)` + + *[文本, 偏移, "utf8.invalid_continuation"]* + +**码点 = (内存[基址 + 偏移] - 192) * 64 + (b1 - 128)** + +1. `码点 < 128` + + *[文本, 偏移, "utf8.invalid_scalar"]* + +*[文本 + text.from_codepoint(码点), 偏移 + 2, ""]* + +## 解码3 + +`(内存, 基址, 长度, 偏移, 文本)` + +1. `偏移 + 2 >= 长度` + + *[文本, 偏移, "utf8.truncated"]* + +**b1 = 内存[基址 + 偏移 + 1]** + +**b2 = 内存[基址 + 偏移 + 2]** + +1. `not (续字节(b1) and 续字节(b2))` + + *[文本, 偏移, "utf8.invalid_continuation"]* + +**码点 = (内存[基址 + 偏移] - 224) * 4096 + (b1 - 128) * 64 + (b2 - 128)** + +1. `码点 < 2048 or (码点 >= 55296 and 码点 <= 57343)` + + *[文本, 偏移, "utf8.invalid_scalar"]* + +*[文本 + text.from_codepoint(码点), 偏移 + 3, ""]* + +## 解码4 + +`(内存, 基址, 长度, 偏移, 文本)` + +1. `偏移 + 3 >= 长度` + + *[文本, 偏移, "utf8.truncated"]* + +**b1 = 内存[基址 + 偏移 + 1]** + +**b2 = 内存[基址 + 偏移 + 2]** + +**b3 = 内存[基址 + 偏移 + 3]** + +1. `not (续字节(b1) and 续字节(b2) and 续字节(b3))` + + *[文本, 偏移, "utf8.invalid_continuation"]* + +**码点 = (内存[基址 + 偏移] - 240) * 262144 + (b1 - 128) * 4096 + (b2 - 128) * 64 + (b3 - 128)** + +1. `码点 < 65536 or 码点 > 1114111` + + *[文本, 偏移, "utf8.invalid_scalar"]* + +*[文本 + text.from_codepoint(码点), 偏移 + 4, ""]* + +## 解码 + +`(内存, 基址, 长度)` + +**偏移 = 0** + +**文本 = ""** + +**错误 = ""** + +- `偏移 < 长度 and 错误 == ""` + + **首 = 内存[基址 + 偏移]** + + 1. `首 < 128` + + **文本 = 文本 + text.from_codepoint(首)** + + **偏移 = 偏移 + 1** + + 2. `首 < 194` + + **错误 = "utf8.invalid_lead"** + + 3. `首 < 224` + + **结果 = 解码2(内存, 基址, 长度, 偏移, 文本)** + + **文本 = 结果[0]** + + **偏移 = 结果[1]** + + **错误 = 结果[2]** + + 4. `首 < 240` + + **结果 = 解码3(内存, 基址, 长度, 偏移, 文本)** + + **文本 = 结果[0]** + + **偏移 = 结果[1]** + + **错误 = 结果[2]** + + 5. `首 < 245` + + **结果 = 解码4(内存, 基址, 长度, 偏移, 文本)** + + **文本 = 结果[0]** + + **偏移 = 结果[1]** + + **错误 = 结果[2]** + + 6. `true` + + **错误 = "utf8.invalid_lead"** + +1. `错误 == ""` + + *成功(文本)* + +2. `true` + + *失败(错误, "UTF-8 解码失败")* + +## 写字节 + +`(内存, 基址, 字节)` + +**下标 = 0** + +- `下标 < 长度(字节)` + + **`内存[基址 + 下标] = 字节[下标]`** + + **下标 = 下标 + 1** + +*None* + +## main + +**内存 = memory.new(1, 1)** + +> 写字节(内存, 0, [65, 195, 169, 228, 184, 173]) + +> 写字节(内存, 16, [228, 184]) + +> 写字节(内存, 32, [228, 65, 173]) + +> 写字节(内存, 48, [237, 160, 128]) + +> 写字节(内存, 64, [240, 157, 132, 158]) + +*[解码(内存, 0, 6), 解码(内存, 16, 2), 解码(内存, 32, 3), 解码(内存, 48, 3), 解码(内存, 64, 4)]* diff --git a/tests/conformance/semantics/61-memory-graph-adjacency.expected.json b/tests/conformance/semantics/61-memory-graph-adjacency.expected.json new file mode 100644 index 0000000..44e3f34 --- /dev/null +++ b/tests/conformance/semantics/61-memory-graph-adjacency.expected.json @@ -0,0 +1,48 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "decimal": "8", + "type": "int" + }, + { + "decimal": "24", + "type": "int" + }, + { + "decimal": "40", + "type": "int" + }, + { + "items": [ + { + "decimal": "40", + "type": "int" + }, + { + "decimal": "24", + "type": "int" + } + ], + "type": "list" + }, + { + "items": [ + { + "decimal": "40", + "type": "int" + } + ], + "type": "list" + }, + { + "items": [], + "type": "list" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/61-memory-graph-adjacency.mq.md b/tests/conformance/semantics/61-memory-graph-adjacency.mq.md new file mode 100644 index 0000000..dce9236 --- /dev/null +++ b/tests/conformance/semantics/61-memory-graph-adjacency.mq.md @@ -0,0 +1,113 @@ +## 写32 + +`(内存, 基址, 值)` + +**字节3 = 值 % 256** + +**字节2 = 值 // 256 % 256** + +**字节1 = 值 // 65536 % 256** + +**字节0 = 值 // 16777216 % 256** + +**`内存[基址] = 字节0`** + +**`内存[基址 + 1] = 字节1`** + +**`内存[基址 + 2] = 字节2`** + +**`内存[基址 + 3] = 字节3`** + +*None* + +## 读32 + +`(内存, 基址)` + +*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]* + +## 分配 + +`(内存, 大小)` + +简单 bump 分配;调用方保证容量足够。 + +**位置 = 读32(内存, 0)** + +> 写32(内存, 0, 位置 + 大小) + +*位置* + +## 新节点 + +`(内存, 值)` + +节点 16 字节:偏移 0 存一个字节的值,偏移 4 存边链表头;0 表示空。 + +**位置 = 分配(内存, 16)** + +**`内存[位置] = 值`** + +> 写32(内存, 位置 + 4, 0) + +*位置* + +## 加边 + +`(内存, 源, 目标)` + +边 8 字节:目标 u32 + 下一条边 u32。头插到源节点的边链表。 + +**边 = 分配(内存, 8)** + +> 写32(内存, 边, 目标) + +> 写32(内存, 边 + 4, 读32(内存, 源 + 4)) + +> 写32(内存, 源 + 4, 边) + +*None* + +## 邻接 + +`(内存, 节点)` + +返回节点值列表;顺序是头插顺序(后加的边在前)。 + +**当前 = 读32(内存, 节点 + 4)** + +**结果 = []** + +- `当前 != 0` + + **结果 = list.append(结果, 读32(内存, 当前))** + + **当前 = 读32(内存, 当前 + 4)** + +*结果* + +## main + +**内存 = memory.new(1, 1)** + +**`内存[0] = 0`** + +**`内存[1] = 0`** + +**`内存[2] = 0`** + +**`内存[3] = 8`** + +**甲 = 新节点(内存, 1)** + +**乙 = 新节点(内存, 2)** + +**丙 = 新节点(内存, 3)** + +> 加边(内存, 甲, 乙) + +> 加边(内存, 甲, 丙) + +> 加边(内存, 乙, 丙) + +*[甲, 乙, 丙, 邻接(内存, 甲), 邻接(内存, 乙), 邻接(内存, 丙)]* diff --git a/tests/conformance/semantics/62-json-array-parser.expected.json b/tests/conformance/semantics/62-json-array-parser.expected.json new file mode 100644 index 0000000..c24968a --- /dev/null +++ b/tests/conformance/semantics/62-json-array-parser.expected.json @@ -0,0 +1,142 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "entries": { + "next": { + "decimal": "20", + "type": "int" + }, + "ok": { + "type": "bool", + "value": true + }, + "value": { + "items": [ + { + "decimal": "1", + "type": "int" + }, + { + "decimal": "23", + "type": "int" + }, + { + "items": [ + { + "decimal": "4", + "type": "int" + }, + { + "decimal": "5", + "type": "int" + } + ], + "type": "list" + }, + { + "decimal": "-6", + "type": "int" + } + ], + "type": "list" + } + }, + "type": "map" + }, + { + "entries": { + "code": { + "type": "text", + "value": "json.invalid_array" + }, + "message": { + "type": "text", + "value": "数组缺少逗号或 ]" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + }, + { + "entries": { + "code": { + "type": "text", + "value": "json.unexpected_eof" + }, + "message": { + "type": "text", + "value": "数组未闭合" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + }, + { + "entries": { + "next": { + "decimal": "35", + "type": "int" + }, + "ok": { + "type": "bool", + "value": true + }, + "value": { + "entries": { + "a": { + "decimal": "1", + "type": "int" + }, + "b": { + "items": [ + { + "type": "bool", + "value": true + }, + { + "type": "none" + }, + { + "type": "text", + "value": "说\"好" + } + ], + "type": "list" + } + }, + "type": "map" + } + }, + "type": "map" + }, + { + "entries": { + "next": { + "decimal": "6", + "type": "int" + }, + "ok": { + "type": "bool", + "value": true + }, + "value": { + "type": "text", + "value": "a\nb" + } + }, + "type": "map" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/62-json-array-parser.mq.md b/tests/conformance/semantics/62-json-array-parser.mq.md new file mode 100644 index 0000000..067ad00 --- /dev/null +++ b/tests/conformance/semantics/62-json-array-parser.mq.md @@ -0,0 +1,431 @@ +## 成功值 + +`(值, 下一位置)` + +*{ok: true, value: 值, next: 下一位置}* + +## 失败 + +`(代码, 消息)` + +*{ok: false, code: 代码, message: 消息}* + +## 是空白 + +`(码点)` + +*码点 == 32 or 码点 == 9 or 码点 == 10 or 码点 == 13* + +## 是数字 + +`(码点)` + +*码点 >= 48 and 码点 <= 57* + +## 跳过空白 + +`(点, 位置)` + +**当前 = 位置** + +- `当前 < 长度(点) and 是空白(点[当前])` + + **当前 = 当前 + 1** + +*当前* + +## 匹配 + +`(点, 位置, 文本)` + +**目标 = text.codepoints(文本)** + +**下标 = 0** + +**结果 = true** + +- `下标 < 长度(目标) and 结果` + + 1. `位置 + 下标 >= 长度(点)` + + **结果 = false** + + 2. `点[位置 + 下标] != 目标[下标]` + + **结果 = false** + + 3. `true` + + **继续 = 0** + + **下标 = 下标 + 1** + +*结果* + +## 解析数字 + +`(点, 位置)` + +**负数 = false** + +**当前 = 位置** + +1. `text.from_codepoint(点[当前]) == "-"` + + **负数 = true** + + **当前 = 当前 + 1** + +2. `true` + + **非负 = 0** + +**值 = 0** + +**开始 = 当前** + +- `当前 < 长度(点) and 是数字(点[当前])` + + **值 = 值 * 10 + 点[当前] - 48** + + **当前 = 当前 + 1** + +1. `当前 == 开始` + + *失败("json.invalid_number", "数字缺少位数")* + +1. `负数` + + *成功值(0 - 值, 当前)* + +*成功值(值, 当前)* + +## 十六进制值 + +`(码点)` + +1. `码点 >= 48 and 码点 <= 57` + + *码点 - 48* + +2. `码点 >= 65 and 码点 <= 70` + + *码点 - 55* + +3. `码点 >= 97 and 码点 <= 102` + + *码点 - 87* + +*0 - 1* + +## 解析字符串 + +`(点, 位置)` + +**当前 = 位置 + 1** + +**结果 = ""** + +**完成 = false** + +**错误 = ""** + +- `当前 < 长度(点) and not 完成 and 错误 == ""` + + **码点 = 点[当前]** + + 1. `码点 == 34` + + **完成 = true** + + **当前 = 当前 + 1** + + 2. `码点 == 92` + + 1. `当前 + 1 >= 长度(点)` + + **错误 = "json.invalid_escape"** + + 2. `true` + + **转义码点 = 点[当前 + 1]** + + 1. `转义码点 == 34` + + **结果 = 结果 + text.from_codepoint(34)** + + **当前 = 当前 + 2** + + 2. `转义码点 == 92` + + **结果 = 结果 + text.from_codepoint(92)** + + **当前 = 当前 + 2** + + 3. `转义码点 == 47` + + **结果 = 结果 + text.from_codepoint(47)** + + **当前 = 当前 + 2** + + 4. `转义码点 == 98` + + **结果 = 结果 + text.from_codepoint(8)** + + **当前 = 当前 + 2** + + 5. `转义码点 == 102` + + **结果 = 结果 + text.from_codepoint(12)** + + **当前 = 当前 + 2** + + 6. `转义码点 == 110` + + **结果 = 结果 + text.from_codepoint(10)** + + **当前 = 当前 + 2** + + 7. `转义码点 == 114` + + **结果 = 结果 + text.from_codepoint(13)** + + **当前 = 当前 + 2** + + 8. `转义码点 == 116` + + **结果 = 结果 + text.from_codepoint(9)** + + **当前 = 当前 + 2** + + 9. `转义码点 == 117` + + 1. `当前 + 5 >= 长度(点)` + + **错误 = "json.invalid_escape"** + + 2. `true` + + **数字 = 十六进制值(点[当前 + 2]) * 4096 + 十六进制值(点[当前 + 3]) * 256 + 十六进制值(点[当前 + 4]) * 16 + 十六进制值(点[当前 + 5])** + + 1. `数字 < 0 or (数字 >= 55296 and 数字 <= 57343)` + + **错误 = "json.invalid_escape"** + + 2. `true` + + **结果 = 结果 + text.from_codepoint(数字)** + + **当前 = 当前 + 6** + + 10. `true` + + **错误 = "json.invalid_escape"** + + 3. `true` + + **结果 = 结果 + text.from_codepoint(码点)** + + **当前 = 当前 + 1** + +1. `错误 != ""` + + *失败("json.invalid_string", "字符串转义无效")* + +1. `not 完成` + + *失败("json.unexpected_eof", "字符串未闭合")* + +*成功值(结果, 当前)* + +## 解析对象 + +`(点, 位置)` + +*解析对象项(点, 位置 + 1, {})* + +## 解析对象项 + +`(点, 位置, 已解析)` + +**位置 = 跳过空白(点, 位置)** + +1. `位置 >= 长度(点)` + + *失败("json.unexpected_eof", "对象未闭合")* + +**当前 = text.from_codepoint(点[位置])** + +1. `当前 == "}"` + + *成功值(已解析, 位置 + 1)* + +**键结果 = 解析字符串(点, 位置)** + +1. `not 键结果.ok` + + *键结果* + +**键 = 键结果.value** + +**位置 = 跳过空白(点, 键结果.next)** + +1. `位置 >= 长度(点)` + + *失败("json.unexpected_eof", "对象缺少冒号")* + +1. `text.from_codepoint(点[位置]) != ":"` + + *失败("json.invalid_object", "对象缺少冒号")* + +**位置 = 跳过空白(点, 位置 + 1)** + +**值结果 = 解析值(点, 位置)** + +1. `not 值结果.ok` + + *值结果* + +**新的已解析 = map.set(已解析, 键, 值结果.value)** + +**位置 = 跳过空白(点, 值结果.next)** + +1. `位置 >= 长度(点)` + + *失败("json.unexpected_eof", "对象未闭合")* + +**分隔 = text.from_codepoint(点[位置])** + +1. `分隔 == ","` + + *解析对象项(点, 位置 + 1, 新的已解析)* + +2. `分隔 == "}"` + + *成功值(新的已解析, 位置 + 1)* + +3. `true` + + *失败("json.invalid_object", "对象缺少逗号或 }")* + +## 解析值 + +`(点, 位置)` + +**位置 = 跳过空白(点, 位置)** + +1. `位置 >= 长度(点)` + + *失败("json.unexpected_eof", "需要值")* + +**首 = text.from_codepoint(点[位置])** + +**首码点 = 点[位置]** + +1. `首码点 == 34` + + *解析字符串(点, 位置)* + +2. `首码点 == 123` + + *解析对象(点, 位置)* + +3. `首 == "["` + + *解析数组(点, 位置)* + +4. `首 == "-" or 是数字(点[位置])` + + *解析数字(点, 位置)* + +5. `首 == "t"` + + 1. `匹配(点, 位置, "true")` + + *成功值(true, 位置 + 4)* + + 2. `true` + + *失败("json.invalid_literal", "无效字面量")* + +6. `首 == "f"` + + 1. `匹配(点, 位置, "false")` + + *成功值(false, 位置 + 5)* + + 2. `true` + + *失败("json.invalid_literal", "无效字面量")* + +7. `首 == "n"` + + 1. `匹配(点, 位置, "null")` + + *成功值(None, 位置 + 4)* + + 2. `true` + + *失败("json.invalid_literal", "无效字面量")* + +8. `true` + + *失败("json.unexpected_token", "不支持的值")* + +## 解析数组 + +`(点, 位置)` + +*解析数组项(点, 位置 + 1, [])* + +## 解析数组项 + +`(点, 位置, 已解析)` + +**位置 = 跳过空白(点, 位置)** + +1. `位置 >= 长度(点)` + + *失败("json.unexpected_eof", "数组未闭合")* + +**当前 = text.from_codepoint(点[位置])** + +1. `当前 == "]"` + + *成功值(已解析, 位置 + 1)* + +**项 = 解析值(点, 位置)** + +1. `not 项.ok` + + *项* + +**新的已解析 = list.append(已解析, 项.value)** + +**位置 = 跳过空白(点, 项.next)** + +1. `位置 >= 长度(点)` + + *失败("json.unexpected_eof", "数组未闭合")* + +**分隔 = text.from_codepoint(点[位置])** + +1. `分隔 == ","` + + *解析数组项(点, 位置 + 1, 新的已解析)* + +2. `分隔 == "]"` + + *成功值(新的已解析, 位置 + 1)* + +3. `true` + + *失败("json.invalid_array", "数组缺少逗号或 ]")* + +## main + +**`对象结果 = 解析值(text.codepoints("{\"a\": 1, \"b\": [true, null, \"说\\\"好\"]}"), 0)`** + +**`字符串结果 = 解析值(text.codepoints("\"a\\nb\""), 0)`** + +*[解析值(text.codepoints(" [1, 23, [4, 5], -6] "), 0), 解析值(text.codepoints("[1 2]"), 0), 解析值(text.codepoints("[1,"), 0), 对象结果, 字符串结果]* diff --git a/tests/conformance/semantics/63-memory-bytecode-vm.expected.json b/tests/conformance/semantics/63-memory-bytecode-vm.expected.json new file mode 100644 index 0000000..78cd0ad --- /dev/null +++ b/tests/conformance/semantics/63-memory-bytecode-vm.expected.json @@ -0,0 +1,36 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "36", + "type": "int" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "decimal": "42", + "type": "int" + } + }, + "type": "map" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/63-memory-bytecode-vm.mq.md b/tests/conformance/semantics/63-memory-bytecode-vm.mq.md new file mode 100644 index 0000000..f9f94a1 --- /dev/null +++ b/tests/conformance/semantics/63-memory-bytecode-vm.mq.md @@ -0,0 +1,130 @@ +## 成功 + +`(值)` + +*{ok: true, value: 值}* + +## 失败 + +`(代码, 消息)` + +*{ok: false, code: 代码, message: 消息}* + +## 写字节 + +`(内存, 基址, 字节)` + +**下标 = 0** + +- `下标 < 长度(字节)` + + **`内存[基址 + 下标] = 字节[下标]`** + + **下标 = 下标 + 1** + +*None* + +## 运行 + +`(内存, 程序长度, 栈基址)` + +程序从偏移 0 开始,栈从 `栈基址` 开始。指令集: +1 = PUSH 立即数,2 = ADD,3 = MUL,5 = JMP,6 = JZ,7 = HALT。 + +**指令指针 = 0** + +**栈指针 = 0** + +**运行中 = true** + +**错误 = ""** + +- `指令指针 < 程序长度 and 运行中 and 错误 == ""` + + **操作码 = 内存[指令指针]** + + 1. `操作码 == 1` + + **`内存[栈基址 + 栈指针] = 内存[指令指针 + 1]`** + + **栈指针 = 栈指针 + 1** + + **指令指针 = 指令指针 + 2** + + 2. `操作码 == 2` + + **右 = 内存[栈基址 + 栈指针 - 1]** + + **左 = 内存[栈基址 + 栈指针 - 2]** + + **`内存[栈基址 + 栈指针 - 2] = 左 + 右`** + + **栈指针 = 栈指针 - 1** + + **指令指针 = 指令指针 + 1** + + 3. `操作码 == 3` + + **右 = 内存[栈基址 + 栈指针 - 1]** + + **左 = 内存[栈基址 + 栈指针 - 2]** + + **`内存[栈基址 + 栈指针 - 2] = 左 * 右`** + + **栈指针 = 栈指针 - 1** + + **指令指针 = 指令指针 + 1** + + 4. `操作码 == 5` + + **指令指针 = 内存[指令指针 + 1]** + + 5. `操作码 == 6` + + **条件 = 内存[栈基址 + 栈指针 - 1]** + + **栈指针 = 栈指针 - 1** + + 1. `条件 == 0` + + **指令指针 = 内存[指令指针 + 1]** + + 2. `true` + + **指令指针 = 指令指针 + 2** + + 6. `操作码 == 7` + + **运行中 = false** + + 7. `true` + + **错误 = "vm.invalid_opcode"** + +1. `错误 != ""` + + *失败("vm.invalid_opcode", "未知指令")* + +1. `栈指针 == 0` + + *失败("vm.empty_stack", "没有结果")* + +*成功(内存[栈基址 + 栈指针 - 1])* + +## main + +**内存 = memory.new(1, 1)** + +第一段程序计算 `(5 + 7) * 3`。 + +> 写字节(内存, 0, [1, 5, 1, 7, 2, 1, 3, 3, 7]) + +**算术结果 = 运行(内存, 9, 1024)** + +第二段程序在条件为 0 时跳到偏移 8,压入 42 后停机。 + +> 写字节(内存, 0, [1, 0, 6, 8, 1, 99, 7, 0, 1, 42, 7]) + +**分支结果 = 运行(内存, 11, 2048)** + +*[算术结果, 分支结果]* diff --git a/tests/conformance/semantics/64-memory-integer-codecs.expected.json b/tests/conformance/semantics/64-memory-integer-codecs.expected.json new file mode 100644 index 0000000..d481f7c --- /dev/null +++ b/tests/conformance/semantics/64-memory-integer-codecs.expected.json @@ -0,0 +1,46 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "decimal": "4660", + "type": "int" + }, + { + "decimal": "305419896", + "type": "int" + }, + { + "decimal": "1311768467463790320", + "type": "int" + }, + { + "decimal": "18446744073709551615", + "type": "int" + }, + { + "decimal": "4", + "type": "int" + }, + { + "decimal": "123456789", + "type": "int" + }, + { + "decimal": "4", + "type": "int" + }, + { + "decimal": "255", + "type": "int" + }, + { + "decimal": "255", + "type": "int" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/64-memory-integer-codecs.mq.md b/tests/conformance/semantics/64-memory-integer-codecs.mq.md new file mode 100644 index 0000000..d6ace4f --- /dev/null +++ b/tests/conformance/semantics/64-memory-integer-codecs.mq.md @@ -0,0 +1,108 @@ +## 写大端 + +`(内存, 基址, 值, 宽度)` + +从最低字节开始写,放到 `基址 + 宽度 - 1 - 下标`,因此最终是大端序。 +调用方保证 `值` 能放进 `宽度` 字节。 + +**当前 = 值** + +**下标 = 0** + +- `下标 < 宽度` + + **`内存[基址 + 宽度 - 1 - 下标] = 当前 % 256`** + + **当前 = 当前 // 256** + + **下标 = 下标 + 1** + +*None* + +## 读大端 + +`(内存, 基址, 宽度)` + +**值 = 0** + +**下标 = 0** + +- `下标 < 宽度` + + **值 = 值 * 256 + 内存[基址 + 下标]** + + **下标 = 下标 + 1** + +*值* + +## 写变长 + +`(内存, 基址, 值)` + +LEB128:每字节低 7 位,最高位表示是否还有后续;返回写入字节数。 + +**当前 = 值** + +**偏移 = 0** + +- `当前 >= 128` + + **`内存[基址 + 偏移] = 当前 % 128 + 128`** + + **当前 = 当前 // 128** + + **偏移 = 偏移 + 1** + +**`内存[基址 + 偏移] = 当前`** + +*偏移 + 1* + +## 读变长 + +`(内存, 基址)` + +返回 `[值, 消耗字节数]`。 + +**值 = 0** + +**倍数 = 1** + +**偏移 = 0** + +**继续 = true** + +- `继续` + + **字节 = 内存[基址 + 偏移]** + + **值 = 值 + 字节 % 128 * 倍数** + + 1. `字节 < 128` + + **继续 = false** + + 2. `true` + + **倍数 = 倍数 * 128** + + **偏移 = 偏移 + 1** + +*[值, 偏移 + 1]* + +## main + +**内存 = memory.new(1, 1)** + +> 写大端(内存, 0, 4660, 2) + +> 写大端(内存, 16, 305419896, 4) + +> 写大端(内存, 32, 1311768467463790320, 8) + +> 写大端(内存, 48, 18446744073709551615, 8) + +**变长长度 = 写变长(内存, 128, 123456789)** + +**变长 = 读变长(内存, 128)** + +*[读大端(内存, 0, 2), 读大端(内存, 16, 4), 读大端(内存, 32, 8), 读大端(内存, 48, 8), 变长长度, 变长[0], 变长[1], 内存[48], 内存[55]]* diff --git a/tests/conformance/semantics/65-table-sort-stdlib.expected.json b/tests/conformance/semantics/65-table-sort-stdlib.expected.json new file mode 100644 index 0000000..6b172b9 --- /dev/null +++ b/tests/conformance/semantics/65-table-sort-stdlib.expected.json @@ -0,0 +1,88 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "header": [ + "名称", + "分数" + ], + "rows": [ + [ + { + "type": "text", + "value": "甲" + }, + { + "decimal": "87", + "type": "int" + } + ], + [ + { + "type": "text", + "value": "乙" + }, + { + "decimal": "92", + "type": "int" + } + ], + [ + { + "type": "text", + "value": "丙" + }, + { + "decimal": "71", + "type": "int" + } + ] + ], + "type": "table" + }, + { + "header": [ + "名称", + "分数" + ], + "rows": [ + [ + { + "type": "text", + "value": "丙" + }, + { + "decimal": "71", + "type": "int" + } + ], + [ + { + "type": "text", + "value": "甲" + }, + { + "decimal": "87", + "type": "int" + } + ], + [ + { + "type": "text", + "value": "乙" + }, + { + "decimal": "92", + "type": "int" + } + ] + ], + "type": "table" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/65-table-sort-stdlib.mq.md b/tests/conformance/semantics/65-table-sort-stdlib.mq.md new file mode 100644 index 0000000..0385f6c --- /dev/null +++ b/tests/conformance/semantics/65-table-sort-stdlib.mq.md @@ -0,0 +1,69 @@ +## 插入有序 + +`(已排序, 行, 列)` + +把一行插入到已按 `列` 升序排列的行列表中;返回新列表,不修改输入。 + +**新列表 = []** + +**已插入 = false** + +**下标 = 0** + +- `下标 < 长度(已排序)` + + **当前行 = 已排序[下标]** + + 1. `not 已插入 and 当前行[列] > 行[列]` + + **新列表 = list.append(新列表, 行)** + + **新列表 = list.append(新列表, 当前行)** + + **已插入 = true** + + 2. `true` + + **新列表 = list.append(新列表, 当前行)** + + **下标 = 下标 + 1** + +1. `not 已插入` + + **新列表 = list.append(新列表, 行)** + +2. `true` + + **忽略 = 0** + +*新列表* + +## 排序 + +`(数据, 列)` + +按列号升序排序;输入表不变,输出新表。 + +**列名 = table.columns(数据)** + +**行数 = 长度(数据)** + +**已排序 = []** + +**下标 = 0** + +- `下标 < 行数` + + **已排序 = 插入有序(已排序, table.row(数据, 下标), 列)** + + **下标 = 下标 + 1** + +*表(列名, 已排序)* + +## main + +**原表 = 表(["名称", "分数"], [["甲", 87], ["乙", 92], ["丙", 71]])** + +**升序 = 排序(原表, 1)** + +*[原表, 升序]* diff --git a/tests/conformance/semantics/66-resource-stdout-write.expected.json b/tests/conformance/semantics/66-resource-stdout-write.expected.json new file mode 100644 index 0000000..296b589 --- /dev/null +++ b/tests/conformance/semantics/66-resource-stdout-write.expected.json @@ -0,0 +1,61 @@ +{ + "args": {}, + "function": "main", + "stdout": "你好,资源!\n", + "value": { + "items": [ + { + "type": "text", + "value": "Resource" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "entries": { + "count": { + "decimal": "19", + "type": "int" + } + }, + "type": "map" + } + }, + "type": "map" + }, + { + "entries": { + "ok": { + "type": "bool", + "value": true + }, + "value": { + "type": "none" + } + }, + "type": "map" + }, + { + "entries": { + "code": { + "type": "text", + "value": "resource.closed" + }, + "message": { + "type": "text", + "value": "Resource handle is closed" + }, + "ok": { + "type": "bool", + "value": false + } + }, + "type": "map" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/66-resource-stdout-write.mq.md b/tests/conformance/semantics/66-resource-stdout-write.mq.md new file mode 100644 index 0000000..4148397 --- /dev/null +++ b/tests/conformance/semantics/66-resource-stdout-write.mq.md @@ -0,0 +1,69 @@ +## 编码 + +`(文本, 内存, 基址)` + +**点 = text.codepoints(文本)** + +**下标 = 0** + +**偏移 = 0** + +- `下标 < 长度(点)` + + **码点 = 点[下标]** + + 1. `码点 < 128` + + **`内存[基址 + 偏移] = 码点`** + + **偏移 = 偏移 + 1** + + 2. `码点 < 2048` + + **`内存[基址 + 偏移] = 192 + 码点 // 64`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 2** + + 3. `码点 < 65536` + + **`内存[基址 + 偏移] = 224 + 码点 // 4096`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 // 64 % 64`** + + **`内存[基址 + 偏移 + 2] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 3** + + 4. `true` + + **`内存[基址 + 偏移] = 240 + 码点 // 262144`** + + **`内存[基址 + 偏移 + 1] = 128 + 码点 // 4096 % 64`** + + **`内存[基址 + 偏移 + 2] = 128 + 码点 // 64 % 64`** + + **`内存[基址 + 偏移 + 3] = 128 + 码点 % 64`** + + **偏移 = 偏移 + 4** + + **下标 = 下标 + 1** + +*偏移* + +## main + +**内存 = memory.new(1, 1)** + +**长度 = 编码("你好,资源!\n", 内存, 0)** + +**标准输出 = resource.open("stdio", "stdout", ["write"])** + +**写入 = resource.write(标准输出, 内存, 0, 长度)** + +**关闭 = resource.close(标准输出)** + +**再次读取 = resource.read(标准输出, 内存, 0, 1)** + +*[类型(标准输出), 写入, 关闭, 再次读取]* diff --git a/tests/conformance/semantics/67-table-join-stdlib.expected.json b/tests/conformance/semantics/67-table-join-stdlib.expected.json new file mode 100644 index 0000000..849efe8 --- /dev/null +++ b/tests/conformance/semantics/67-table-join-stdlib.expected.json @@ -0,0 +1,135 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "header": [ + "名称", + "分数" + ], + "rows": [ + [ + { + "type": "text", + "value": "甲" + }, + { + "decimal": "87", + "type": "int" + } + ], + [ + { + "type": "text", + "value": "乙" + }, + { + "decimal": "92", + "type": "int" + } + ], + [ + { + "type": "text", + "value": "丙" + }, + { + "decimal": "71", + "type": "int" + } + ] + ], + "type": "table" + }, + { + "header": [ + "名称", + "等级" + ], + "rows": [ + [ + { + "type": "text", + "value": "乙" + }, + { + "type": "text", + "value": "A" + } + ], + [ + { + "type": "text", + "value": "甲" + }, + { + "type": "text", + "value": "B" + } + ], + [ + { + "type": "text", + "value": "丁" + }, + { + "type": "text", + "value": "C" + } + ] + ], + "type": "table" + }, + { + "header": [ + "名称", + "分数", + "名称", + "等级" + ], + "rows": [ + [ + { + "type": "text", + "value": "甲" + }, + { + "decimal": "87", + "type": "int" + }, + { + "type": "text", + "value": "甲" + }, + { + "type": "text", + "value": "B" + } + ], + [ + { + "type": "text", + "value": "乙" + }, + { + "decimal": "92", + "type": "int" + }, + { + "type": "text", + "value": "乙" + }, + { + "type": "text", + "value": "A" + } + ] + ], + "type": "table" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/67-table-join-stdlib.mq.md b/tests/conformance/semantics/67-table-join-stdlib.mq.md new file mode 100644 index 0000000..8b3c020 --- /dev/null +++ b/tests/conformance/semantics/67-table-join-stdlib.mq.md @@ -0,0 +1,45 @@ +## 连接 + +`(左, 右, 左列, 右列)` + +按 equality 连接两张表;输出表头是左表头接右表头,原表不变。 + +**表头 = list.concat(table.columns(左), table.columns(右))** + +**结果行 = []** + +**左行号 = 0** + +- `左行号 < 长度(左)` + + **左行 = table.row(左, 左行号)** + + **键 = 左行[左列]** + + **右行号 = 0** + + - `右行号 < 长度(右)` + + **右行 = table.row(右, 右行号)** + + 1. `右行[右列] == 键` + + **结果行 = list.append(结果行, list.concat(左行, 右行))** + + 2. `true` + + **继续 = 0** + + **右行号 = 右行号 + 1** + + **左行号 = 左行号 + 1** + +*表(表头, 结果行)* + +## main + +**左表 = 表(["名称", "分数"], [["甲", 87], ["乙", 92], ["丙", 71]])** + +**右表 = 表(["名称", "等级"], [["乙", "A"], ["甲", "B"], ["丁", "C"]])** + +*[左表, 右表, 连接(左表, 右表, 0, 0)]* diff --git a/tests/conformance/semantics/68-csv-quoted-row.expected.json b/tests/conformance/semantics/68-csv-quoted-row.expected.json new file mode 100644 index 0000000..fdd6875 --- /dev/null +++ b/tests/conformance/semantics/68-csv-quoted-row.expected.json @@ -0,0 +1,66 @@ +{ + "args": {}, + "function": "main", + "stdout": "", + "value": { + "items": [ + { + "items": [ + { + "type": "text", + "value": "甲" + }, + { + "type": "text", + "value": "乙,丙" + }, + { + "type": "text", + "value": "说\"好" + } + ], + "type": "list" + }, + { + "items": [ + { + "type": "text", + "value": "a" + }, + { + "type": "text", + "value": "" + }, + { + "type": "text", + "value": "c" + } + ], + "type": "list" + }, + { + "items": [ + { + "type": "text", + "value": "" + } + ], + "type": "list" + }, + { + "items": [ + { + "type": "text", + "value": "a\"b" + }, + { + "type": "text", + "value": "c" + } + ], + "type": "list" + } + ], + "type": "list" + } +} diff --git a/tests/conformance/semantics/68-csv-quoted-row.mq.md b/tests/conformance/semantics/68-csv-quoted-row.mq.md new file mode 100644 index 0000000..dfcf01b --- /dev/null +++ b/tests/conformance/semantics/68-csv-quoted-row.mq.md @@ -0,0 +1,66 @@ +## 解析 + +`(文本)` + +单行 CSV:逗号分隔、双引号包裹、`""` 表示一个字面双引号。 +不处理换行、多行字段或首尾空白。 + +**点 = text.codepoints(文本)** + +**下标 = 0** + +**字段 = ""** + +**字段表 = []** + +**引号内 = false** + +- `下标 < 长度(点)` + + **字符 = text.from_codepoint(点[下标])** + + 1. `引号内 and 字符 == "\""` + + 1. `下标 + 1 < 长度(点) and text.from_codepoint(点[下标 + 1]) == "\""` + + **`字段 = 字段 + "\""`** + + **下标 = 下标 + 2** + + 2. `true` + + **引号内 = false** + + **下标 = 下标 + 1** + + 2. `引号内` + + **字段 = 字段 + 字符** + + **下标 = 下标 + 1** + + 3. `字符 == "\""` + + **引号内 = true** + + **下标 = 下标 + 1** + + 4. `字符 == ","` + + **字段表 = list.append(字段表, 字段)** + + **字段 = ""** + + **下标 = 下标 + 1** + + 5. `true` + + **字段 = 字段 + 字符** + + **下标 = 下标 + 1** + +*list.concat(字段表, [字段])* + +## main + +*`[解析("甲,\"乙,丙\",\"说\"\"好\""), 解析("a,,c"), 解析("\"\""), 解析("\"a\"\"b\",c")]`* diff --git a/tests/negative/runtime/13-memory-out-of-bounds.expected.json b/tests/negative/runtime/13-memory-out-of-bounds.expected.json new file mode 100644 index 0000000..6ab53b5 --- /dev/null +++ b/tests/negative/runtime/13-memory-out-of-bounds.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "runtime.index" + ] +} diff --git a/tests/negative/runtime/13-memory-out-of-bounds.mq.md b/tests/negative/runtime/13-memory-out-of-bounds.mq.md new file mode 100644 index 0000000..e58aab5 --- /dev/null +++ b/tests/negative/runtime/13-memory-out-of-bounds.mq.md @@ -0,0 +1,5 @@ +## main + +**内存 = memory.new(1, 1)** + +*内存[65536]* diff --git a/tests/negative/runtime/14-memory-byte-range.expected.json b/tests/negative/runtime/14-memory-byte-range.expected.json new file mode 100644 index 0000000..deec1d6 --- /dev/null +++ b/tests/negative/runtime/14-memory-byte-range.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "runtime.type" + ] +} diff --git a/tests/negative/runtime/14-memory-byte-range.mq.md b/tests/negative/runtime/14-memory-byte-range.mq.md new file mode 100644 index 0000000..29735ff --- /dev/null +++ b/tests/negative/runtime/14-memory-byte-range.mq.md @@ -0,0 +1,5 @@ +## main + +**内存 = memory.new(1, 1)** + +**`内存[0] = 256`** diff --git a/tests/negative/runtime/15-index-assign-non-memory.expected.json b/tests/negative/runtime/15-index-assign-non-memory.expected.json new file mode 100644 index 0000000..deec1d6 --- /dev/null +++ b/tests/negative/runtime/15-index-assign-non-memory.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "runtime.type" + ] +} diff --git a/tests/negative/runtime/15-index-assign-non-memory.mq.md b/tests/negative/runtime/15-index-assign-non-memory.mq.md new file mode 100644 index 0000000..69a058b --- /dev/null +++ b/tests/negative/runtime/15-index-assign-non-memory.mq.md @@ -0,0 +1,8 @@ +## main + +List values keep their v1 value semantics: index assignment is not defined +for them and must not silently turn a list into shared mutable memory. + +**列表 = [1, 2]** + +**`列表[0] = 9`** diff --git a/tests/negative/runtime/16-memory-huge-offset.expected.json b/tests/negative/runtime/16-memory-huge-offset.expected.json new file mode 100644 index 0000000..6ab53b5 --- /dev/null +++ b/tests/negative/runtime/16-memory-huge-offset.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "runtime.index" + ] +} diff --git a/tests/negative/runtime/16-memory-huge-offset.mq.md b/tests/negative/runtime/16-memory-huge-offset.mq.md new file mode 100644 index 0000000..06a1d34 --- /dev/null +++ b/tests/negative/runtime/16-memory-huge-offset.mq.md @@ -0,0 +1,5 @@ +## main + +**内存 = memory.new(1, 1)** + +*内存[999999999999999999999999999999999999999]* diff --git a/tests/negative/runtime/17-memory-grow-beyond-max.expected.json b/tests/negative/runtime/17-memory-grow-beyond-max.expected.json new file mode 100644 index 0000000..8abf84f --- /dev/null +++ b/tests/negative/runtime/17-memory-grow-beyond-max.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "resource.exhausted" + ] +} diff --git a/tests/negative/runtime/17-memory-grow-beyond-max.mq.md b/tests/negative/runtime/17-memory-grow-beyond-max.mq.md new file mode 100644 index 0000000..73b5073 --- /dev/null +++ b/tests/negative/runtime/17-memory-grow-beyond-max.mq.md @@ -0,0 +1,5 @@ +## main + +**内存 = memory.new(1, 1)** + +*memory.grow(内存, 1)* diff --git a/tests/negative/runtime/18-memory-negative-pages.expected.json b/tests/negative/runtime/18-memory-negative-pages.expected.json new file mode 100644 index 0000000..aee4d32 --- /dev/null +++ b/tests/negative/runtime/18-memory-negative-pages.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "runtime.memory" + ] +} diff --git a/tests/negative/runtime/18-memory-negative-pages.mq.md b/tests/negative/runtime/18-memory-negative-pages.mq.md new file mode 100644 index 0000000..c288597 --- /dev/null +++ b/tests/negative/runtime/18-memory-negative-pages.mq.md @@ -0,0 +1,3 @@ +## main + +*[memory.new(0 - 1, 1)]* diff --git a/tests/negative/runtime/19-list-append-wrong-type.expected.json b/tests/negative/runtime/19-list-append-wrong-type.expected.json new file mode 100644 index 0000000..deec1d6 --- /dev/null +++ b/tests/negative/runtime/19-list-append-wrong-type.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "runtime.type" + ] +} diff --git a/tests/negative/runtime/19-list-append-wrong-type.mq.md b/tests/negative/runtime/19-list-append-wrong-type.mq.md new file mode 100644 index 0000000..74d754b --- /dev/null +++ b/tests/negative/runtime/19-list-append-wrong-type.mq.md @@ -0,0 +1,3 @@ +## main + +*list.append({a: 1}, 2)* diff --git a/tests/negative/runtime/20-table-cell-out-of-bounds.expected.json b/tests/negative/runtime/20-table-cell-out-of-bounds.expected.json new file mode 100644 index 0000000..6ab53b5 --- /dev/null +++ b/tests/negative/runtime/20-table-cell-out-of-bounds.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "runtime.index" + ] +} diff --git a/tests/negative/runtime/20-table-cell-out-of-bounds.mq.md b/tests/negative/runtime/20-table-cell-out-of-bounds.mq.md new file mode 100644 index 0000000..b498642 --- /dev/null +++ b/tests/negative/runtime/20-table-cell-out-of-bounds.mq.md @@ -0,0 +1,5 @@ +## main + +**数据 = 表(["a"], [[1]])** + +*table.cell(数据, 1, 0)* diff --git a/tests/negative/runtime/21-map-key-not-text.expected.json b/tests/negative/runtime/21-map-key-not-text.expected.json new file mode 100644 index 0000000..deec1d6 --- /dev/null +++ b/tests/negative/runtime/21-map-key-not-text.expected.json @@ -0,0 +1,7 @@ +{ + "function": "main", + "args": {}, + "codes": [ + "runtime.type" + ] +} diff --git a/tests/negative/runtime/21-map-key-not-text.mq.md b/tests/negative/runtime/21-map-key-not-text.mq.md new file mode 100644 index 0000000..87c54c1 --- /dev/null +++ b/tests/negative/runtime/21-map-key-not-text.mq.md @@ -0,0 +1,5 @@ +## main + +**数据 = {a: 1}** + +*map.get(数据, 1)*