feat(rfc-cm): baseline P0-P3 compute machine, stdlib and resource slices

Implements the first vertical slices of RFC-CM-0001/0002 on top of a113d89:

- P1: opaque Memory regions (64 KiB pages, budgets, grow-only), byte
  read/write, Memory-index place assignment, wire/host-boundary rejection.
- CM-A04/A05: value-semantics List/Map/Text/Table primitives and the
  recoverable Result map convention.
- CM-A06: opaque Resource handles, resource.open/read/write/close, stdout
  write, granted project-file read and buffered write, strict fail-closed.
- CM-A07 Compat-A: explicit .mq.md format/utf8/io chain matching builtin
  print for the covered value types; builtin remains default.
- P2 fixtures: allocators, list/tree/graph/hash, integer codecs, UTF-8
  encode/decode, CSV/JSON subsets, table filter/sort/join, bytecode VM.
- ADRs 0022-0027, spec updates, evidence reports and 68 semantic goldens.

Gate: cargo fmt --check, clippy -D warnings, 407 tests passed.
This commit is contained in:
cflmy committed 2026-11-20 21:16:54 +08:00
1 parent a113d89496
commit 92bfbf2568
127 files changed
+11351 -141

No files matched your search

+75
View File
@@ -43,9 +43,84 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
without rewriting `marqdo.lock` or the materialized `.marqdo/env`.
- G-C lane honours `CARGO_TARGET_DIR`, so a run from a clean `git archive` with
an external target directory uses the binary it just built.
- Addressable `Memory` (RFC-CM-0001 P1, ADR-0022): an opaque run-scoped
region value with 64 KiB zero-filled pages, grow-only capacity, bounded
byte read/write, `memory.new/size/grow`, and `m[offset] = byte`. Copies share
the region identity; different regions never alias. Default budgets are
64 MiB committed per run and 16 MiB per region; `--max-memory` and
`--max-region-memory` override the budgets on `run`/`call`.
- Memory-index place assignment in the expression/IR model
(`AssignTarget::Index` / `PlaceTarget::Index`) with the frozen evaluation
order base, index, RHS and no change to `List`/`Map`/`Table` value semantics.
- Stable `runtime.memory` error category for invalid/stale handles and page
counts; `runtime.index` for offsets, `runtime.type` for operand/byte-range
errors, `resource.exhausted` for budgets.
- Analysis v2 projects an index assignment target as an additive
`{"kind":"index",...}` object while local targets keep the v1 string.
- `examples/memory-buffer.mq.md`, semantic goldens 45–47 and runtime negatives
13–18 cover sharing, isolation, growth, zero-fill, offsets, byte range,
non-Memory index assignment, declared-max growth and negative page counts.
- Partial RFC-CM P2 closure evidence (`.mq.md` only, no Rust algorithm
additions): fixtures 48–50 implement a big-endian u32 codec, a bump
allocator, node allocation and a linked-list traversal/sum, and prove that a
failed allocation leaves the allocator header unchanged. This is **not** the
full P2 gate: JSON floats, table group and a frozen-CLI stdlib closure lane
are still missing.
- `Memory` handles are never transferable: `to_wire` emits a bare rejected tag,
`from_wire` fails, and every host/plugin/foreign call rejects arguments that
contain a handle.
- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
`list.append/concat`, `map.keys/has/get/set/remove`,
`text.codepoints/from_codepoint`, and `table.columns/row/cell`. All are pure,
value-returning builtins; inputs are never mutated and no new syntax is
added. Semantic fixtures 51–53 and runtime negatives 19–21 cover value
semantics, Unicode scalars, table projection, wrong types and bounds.
- `.mq.md` table filter (`过滤`) built on `table.columns/row/cell`,
`list.append` and `表`, with a semantic golden showing the input table is
unchanged (fixture 54). No `table.filter`/`HostOp` is added.
- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): an ordinary Map
`{ok:true,value}` / `{ok:false,code,message[,details]}`, constructed by
`.mq.md` functions, with expected failures recoverable and language
invariant violations still fatal. No `Result` type, exception or
`try`/`catch` syntax.
- `.mq.md` first-fit free-list allocator (fixture 55): 8-byte block headers,
free-block reuse, block splitting, address-adjacent coalescing (tested in a
second region whose bump space is exhausted, so only a merged block can
satisfy the request), non-overlapping reused payloads and `alloc.no_space`
returned as a `Result` without aborting the run. No allocator Rust builtin or
`HostOp` was added.
- `.mq.md` P2 closures beyond the first slice: binary tree insert/in-order
traversal (56), CSV row splitter (57), open-addressing hash table (58),
UTF-8 encoder/validating decoder (59–60), graph adjacency list (61),
recursive JSON parser for arrays/objects/strings (62), stack bytecode VM
(63), 16/32/64-bit + LEB128 codecs (64), table sort (65), table equality join
(67) and quoted/escaped CSV (68). No Rust algorithm, container or parser was
added for these; algorithms remain `.mq.md` code. JSON floats remain open.
- Resource/stream first slice (RFC-CM CM-A06, ADR-0025): opaque host-issued
`Resource` handles, `resource.open/read/write/close`, bytes transferred
through `Memory`, stdout write in pure/project under console/output budgets,
granted project-file chunked read in project, EOF as success, closed-handle
Result errors, forged-handle fatal errors, wire/host-boundary rejection, and
strict refusing every resource open. Semantic fixture 66 and host/CLI tests.
- Buffered project-file write streams (ADR-0026): `file` + `["write"]` requires
an `fs_write` grant, buffers in the host, flushes only on `close`, never
flushes on drop, has an 8 MiB ceiling, and is refused in pure/strict. A
`.mq.md` chunked copy loop reproduces a file without a Rust `copy_file`.
- CM-A07 **Compat-A** prototype (ADR-0027): `examples/stdlib/` provides
`format.显示`, UTF-8 encoding and `io.写值` (`resource.write(stdout)`), and
its output is comparison-tested byte-for-byte against builtin `打印` for
Unit/None/Bool/Int/Float/Text/List/Map/Table. Builtin `打印` remains the
default; no alias routing, bundled stdlib or `HostOp::Print` removal; the
other value types lack public observation.
### Changed
- Assignment targets are no longer restricted to bare identifiers: a single
index place is accepted at statement level. Member assignment stays a parse
error; chained assignment stays a parse error. `spec/expression.ebnf`,
`spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md` and
`spec/security.md` document the new semantics.
- `docs/STABLE_RELEASE_IMPLEMENTATION.md` is the single next-phase
implementation plan. Older plan documents are historical input, not current
status.
+3 -1
View File
@@ -26,7 +26,9 @@ portable/tested locally but the native platform lane has not run here;
| Feature | Ubuntu Linux x86_64 | macOS arm64 | Windows x86_64 |
|---|---|---|---|
| Markdown profile / AST goldens (160) | PASS | UNVERIFIED (cfg fix + CRLF fix) | UNVERIFIED (cfg fix + CRLF fix) |
| Rust interpreter / semantics (44) | PASS | UNVERIFIED | UNVERIFIED |
| Rust interpreter / semantics (68) | PASS | UNVERIFIED | UNVERIFIED |
| Addressable `Memory` P1 (ADR-0022) | PASS | UNVERIFIED | UNVERIFIED |
| Resource streams CM-A06 (ADR-0025/0026: stdout + project file read/write) | PASS | UNVERIFIED | **FAIL-CLOSED** (`strict` refuses resource opens) |
| `run` / `call` / `check` / `inspect` / `fmt` | PASS | UNVERIFIED | UNVERIFIED |
| lock v3 path/Git graph + sync + tamper | PASS | UNVERIFIED | UNVERIFIED |
| Analysis v2 / Trace v2 + Python consumers | PASS | UNVERIFIED | UNVERIFIED |
+23 -2
View File
@@ -81,12 +81,33 @@ executes anything.
## What works (implemented and tested)
* **P0 contracts** — frozen Markdown profile, AST → program mapping, strict
expression grammar, evaluation semantics; 160 AST goldens, 44 semantic
goldens, 52 compile-error fixtures and 12 runtime-error fixtures
expression grammar, evaluation semantics; 160 AST goldens, 68 semantic
goldens, 52 compile-error fixtures and 21 runtime-error fixtures
(`tests/`).
* **P1 interpreter** — native Rust tree-walk interpreter: values, scopes,
functions, static parameter inference, assignments, branches, loops,
returns, recursion, structured errors, fuel/stack/output budgets.
* **Addressable memory (RFC-CM P1)** — opaque run-scoped `Memory` regions with
64 KiB zero-filled pages, grow-only `memory.new/size/grow`, bounded byte
read/write, `m[i] = byte`, run/per-region budgets, host-boundary and wire
rejection (ADR-0022).
* **Minimal structure primitives (RFC-CM CM-A04)** — value-semantics
`list.append/concat`, `map.keys/has/get/set/remove`, Unicode scalar
`text.codepoints/from_codepoint`, and non-negative `table.columns/row/cell`;
pure builtins with no mutation or new syntax (ADR-0023).
* **Recoverable results (RFC-CM CM-A05)** — `{ok:true,value}` /
`{ok:false,code,message}` as an ordinary Map, with a `.mq.md` first-fit
free-list allocator demonstrating reuse, splitting and `alloc.no_space`
recovery (ADR-0024). No `Result` type or exception syntax.
* **Stdlib print prototype (RFC-CM CM-A07 Compat-A)** — ordinary imported
`.mq.md` `format.显示` + UTF-8 + `resource.write(stdout)` matches builtin
`打印` for the covered value set; the builtin is still the default and no
alias is routed (ADR-0027).
* **Resource streams (RFC-CM CM-A06)** — opaque host-issued `Resource` handles;
`resource.open/read/write/close` transfer bytes through `Memory`; stdout
write in pure/project, granted project-file read, and buffered project-file
write flushed on `close` in project; strict refuses all resource opens until
verified (ADR-0025/0026).
* **P2 references + CLI** — explicit YAML front-matter imports, `ModuleId`/
`UnitId`/anchors, import cycle detection, multi-module calls,
`run/call/check/inspect/fmt`, reference validation (`check --refs`) and the
+30 -5
View File
@@ -8,7 +8,9 @@
use std::path::Path;
use marqdo_expression::{BinaryOp, Expr, UnaryOp};
use marqdo_ir::{CallTarget, Function, Module, ModuleId, Resource, SignatureKind, Stmt, UnitKind};
use marqdo_ir::{
CallTarget, Function, Module, ModuleId, PlaceTarget, Resource, SignatureKind, Stmt, UnitKind,
};
use marqdo_span::Span;
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
@@ -232,13 +234,31 @@ fn function_json(function: &Function) -> Value {
})
}
/// Project an assignment place. Local targets keep the v1 string form; the new
/// index target is an additive object so existing consumers that compare the
/// local `target` string are unaffected (Analysis v2, ADR-0022).
fn place_json(place: &PlaceTarget) -> Value {
match place {
PlaceTarget::Local { name, .. } => Value::String(name.clone()),
PlaceTarget::Index { base, index, span } => json!({
"kind": "index",
"base": expr_json(base),
"index": expr_json(index),
"span": span_json(*span),
}),
}
}
fn stmt_json(stmt: &Stmt) -> Value {
match stmt {
Stmt::Assign {
name, value, span, ..
target,
value,
span,
..
} => json!({
"kind": "assign",
"target": name,
"target": place_json(target),
"value": expr_json(value),
"span": span_json(*span),
}),
@@ -497,9 +517,14 @@ fn collect_calls(
) {
for stmt in body {
match stmt {
Stmt::Assign { value, .. } | Stmt::Eval { expr: value, .. } => {
expr_calls(project, module, function, value, calls)
Stmt::Assign { target, value, .. } => {
if let PlaceTarget::Index { base, index, .. } = target {
expr_calls(project, module, function, base, calls);
expr_calls(project, module, function, index, calls);
}
expr_calls(project, module, function, value, calls);
}
Stmt::Eval { expr: value, .. } => expr_calls(project, module, function, value, calls),
Stmt::Return { value, .. } => expr_calls(project, module, function, value, calls),
Stmt::Call { call, span } => calls.push(json!({
"function": function.id.to_string(),
+15 -2
View File
@@ -4,7 +4,7 @@ use std::path::{Path, PathBuf};
use std::time::{Duration, Instant};
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_runtime::{run, EvaluatedArg, RuntimeError, Value};
use marqdo_runtime::{run_with_limits, EvaluatedArg, MemoryLimits, RuntimeError, Value};
use marqdo_sandbox::{PlatformCapabilities, SandboxMode};
use marqdo_span::Span;
use serde_json::json;
@@ -32,6 +32,10 @@ pub struct SandboxSettings {
pub stack: Option<usize>,
/// Output limit override.
pub max_output: Option<usize>,
/// Total committed addressable-memory budget override (bytes).
pub max_memory: Option<usize>,
/// Per-region committed addressable-memory budget override (bytes).
pub max_region_memory: Option<usize>,
/// Wall-clock timeout in seconds.
pub timeout: Option<u64>,
/// Explicitly trusted plugin manifests to load.
@@ -884,12 +888,21 @@ pub fn execute(
.iter()
.map(|(name, value)| EvaluatedArg::named(name.clone(), value.clone(), Span::default()))
.collect();
let result = run(
let memory_limits = MemoryLimits {
max_total_bytes: settings
.max_memory
.unwrap_or(marqdo_runtime::DEFAULT_MAX_TOTAL_BYTES),
max_region_bytes: settings
.max_region_memory
.unwrap_or(marqdo_runtime::DEFAULT_MAX_REGION_BYTES),
};
let result = run_with_limits(
&project.program,
&mut host,
&project.entry,
&function_name,
runtime_args,
memory_limits,
);
if let Some(path) = &settings.trace_file {
let status = match &result {
+8
View File
@@ -103,6 +103,12 @@ struct SandboxArgs {
/// Override the program output byte limit.
#[arg(long = "max-output")]
max_output: Option<usize>,
/// Override the total committed addressable-memory budget in bytes.
#[arg(long = "max-memory", value_name = "BYTES")]
max_memory: Option<usize>,
/// Override the per-region committed addressable-memory budget in bytes.
#[arg(long = "max-region-memory", value_name = "BYTES")]
max_region_memory: Option<usize>,
/// Wall-clock timeout in seconds.
#[arg(long)]
timeout: Option<u64>,
@@ -137,6 +143,8 @@ impl SandboxArgs {
fuel: self.fuel,
stack: self.stack,
max_output: self.max_output,
max_memory: self.max_memory,
max_region_memory: self.max_region_memory,
timeout: self.timeout,
plugins: self.plugin.clone(),
trust_native_plugin: self.trust_native_plugin,
+28
View File
@@ -181,6 +181,34 @@ fn analysis_v2_identity_is_location_independent() {
assert_eq!(analysis["project"]["identity_kind"], "manifest");
}
#[test]
fn analysis_v2_projects_memory_place_targets_additively() {
let dir = TempDir::new("analysis-place");
let source = "## main\n\n**内存 = memory.new(1, 2)**\n\n**`内存[0] = 7`**\n\n*内存[0]*\n";
write(dir.path(), "index.mq.md", source);
let target = dir.path().join("analysis.json");
let output = cli(
dir.path(),
&["check", "index.mq.md", "--emit", target.to_str().unwrap()],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let analysis: serde_json::Value =
serde_json::from_slice(&std::fs::read(&target).unwrap()).unwrap();
let body = analysis["modules"][0]["functions"][0]["body"]
.as_array()
.expect("body is an array");
assert_eq!(body[0]["target"], "内存");
assert_eq!(body[1]["target"]["kind"], "index");
assert_eq!(body[1]["target"]["base"]["kind"], "variable");
assert_eq!(body[1]["target"]["index"]["kind"], "literal");
assert_eq!(body[1]["target"]["index"]["type"], "int");
}
#[test]
fn analysis_v2_projects_addressable_formula_and_code_resources() {
let dir = TempDir::new("analysis-resources");
+43
View File
@@ -24,6 +24,49 @@ fn stderr(output: &Output) -> String {
String::from_utf8_lossy(&output.stderr).to_string()
}
#[test]
fn mqmd_stdlib_print_output_matches_builtin_print() {
let output = cli(&["run", "examples/stdlib/demo.mq.md"]);
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
let stdout = String::from_utf8_lossy(&output.stdout);
let lines: Vec<&str> = stdout.lines().collect();
assert_eq!(lines.len(), 4, "{stdout:?}");
assert_eq!(lines[0], lines[1], "list formatting differs from 打印");
assert_eq!(lines[2], lines[3], "table formatting differs from 打印");
assert_eq!(
lines[0],
r#"["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]"#
);
assert_eq!(lines[2], "Table[名称 | 分数]{甲 | 87; 乙 | 92}");
}
#[test]
fn addressable_memory_example_runs_through_the_cli() {
let output = cli(&["run", "examples/memory-buffer.mq.md", "--json"]);
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
let report: serde_json::Value =
serde_json::from_slice(&output.stdout).expect("run --json emits JSON");
assert_eq!(report["success"], true);
assert_eq!(report["value"]["type"], "int");
assert_eq!(report["value"]["decimal"], "141");
}
#[test]
fn cli_memory_budget_override_fails_closed() {
let output = cli(&[
"run",
"examples/memory-buffer.mq.md",
"--max-memory",
"0",
"--json",
]);
assert_eq!(output.status.code(), Some(1), "{}", stderr(&output));
let report: serde_json::Value =
serde_json::from_slice(&output.stdout).expect("run --json emits JSON");
assert_eq!(report["success"], false);
assert_eq!(report["error"]["code"], "resource.exhausted");
}
#[test]
fn multi_version_packages_run_without_an_implicit_override() {
let output = cli(&["run", "examples/packages-multi-version/index.mq.md"]);
+235
View File
@@ -0,0 +1,235 @@
//! End-to-end resource stream tests through the CLI.
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
struct TempDir(PathBuf);
impl TempDir {
fn new(tag: &str) -> Self {
let path =
std::env::temp_dir().join(format!("marqdo-resource-test-{}-{tag}", std::process::id()));
let _ = std::fs::remove_dir_all(&path);
std::fs::create_dir_all(&path).unwrap();
Self(path)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn cli(dir: &Path, args: &[&str]) -> Output {
Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(args)
.current_dir(dir)
.output()
.expect("run marqdo")
}
fn write_file(path: &Path, contents: &str) {
std::fs::write(path, contents).unwrap();
}
const READER: &str = r#"## main
**内存 = memory.new(1, 1)**
**流 = resource.open("file", "data.txt", ["read"])**
**第一次 = resource.read(流, 内存, 0, 2)**
**第二次 = resource.read(流, 内存, 2, 4)**
**第三次 = resource.read(流, 内存, 6, 1)**
**关闭 = resource.close(流)**
*[第一次.value.count, 第一次.value.eof, 第二次.value.count, 第二次.value.eof, 第三次.value.count, 第三次.value.eof, 内存[0], 内存[1], 内存[2], 内存[3], 内存[4], 内存[5], 关闭.ok]*
"#;
#[test]
fn project_file_stream_reads_chunks_into_memory() {
let dir = TempDir::new("file-read");
write_file(&dir.path().join("data.txt"), "abcdef");
write_file(&dir.path().join("index.mq.md"), READER);
let output = cli(
dir.path(),
&["run", "index.mq.md", "--allow-read", "data.txt", "--json"],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
let values: Vec<&serde_json::Value> = report["value"]["items"]
.as_array()
.expect("list result")
.iter()
.collect();
assert_eq!(values[0]["decimal"], "2");
assert_eq!(values[1]["value"], false);
assert_eq!(values[2]["decimal"], "4");
assert_eq!(values[3]["value"], true);
assert_eq!(values[4]["decimal"], "0");
assert_eq!(values[5]["value"], true);
for (index, expected) in ["97", "98", "99", "100", "101", "102"]
.into_iter()
.enumerate()
{
assert_eq!(values[6 + index]["decimal"], expected);
}
assert_eq!(values[12]["value"], true);
}
const WRITER: &str = r#"## main
**内存 = memory.new(1, 1)**
**`内存[0] = 65`**
**`内存[1] = 66`**
**`内存[2] = 67`**
**写 = resource.open("file", "out.bin", ["write"])**
**写入 = resource.write(写, 内存, 0, 3)**
**关闭写 = resource.close(写)**
**读 = resource.open("file", "out.bin", ["read"])**
**读出 = resource.read(读, 内存, 0, 3)**
**关闭读 = resource.close(读)**
*[写入.value.count, 关闭写.ok, 读出.value.count, 读出.value.eof, 内存[0], 内存[1], 内存[2], 关闭读.ok]*
"#;
#[test]
fn project_file_stream_writes_on_close_then_reads_back() {
let dir = TempDir::new("file-write");
write_file(&dir.path().join("index.mq.md"), WRITER);
let output = cli(
dir.path(),
&[
"run",
"index.mq.md",
"--allow-write",
"out.bin",
"--allow-read",
"out.bin",
"--json",
],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
let values = report["value"]["items"].as_array().expect("list result");
assert_eq!(values[0]["decimal"], "3");
assert_eq!(values[1]["value"], true);
assert_eq!(values[2]["decimal"], "3");
assert_eq!(values[3]["value"], true);
assert_eq!(values[4]["decimal"], "65");
assert_eq!(values[5]["decimal"], "66");
assert_eq!(values[6]["decimal"], "67");
assert_eq!(values[7]["value"], true);
assert_eq!(std::fs::read(dir.path().join("out.bin")).unwrap(), b"ABC");
}
const COPY: &str = r#"## main
**输入 = resource.open("file", "data.txt", ["read"])**
**输出 = resource.open("file", "out.bin", ["write"])**
**内存 = memory.new(1, 1)**
**总数 = 0**
**继续 = true**
- `继续`
**块 = resource.read(输入, 内存, 0, 16)**
1. `块.ok`
> resource.write(输出, 内存, 0, 块.value.count)
**总数 = 总数 + 块.value.count**
**继续 = not 块.value.eof**
2. `true`
**继续 = false**
**关闭读 = resource.close(输入)**
**关闭写 = resource.close(输出)**
*[总数, 关闭读.ok, 关闭写.ok]*
"#;
#[test]
fn file_copy_is_written_in_mqmd_through_chunked_streams() {
let dir = TempDir::new("file-copy");
write_file(&dir.path().join("data.txt"), "abcdef");
write_file(&dir.path().join("index.mq.md"), COPY);
let output = cli(
dir.path(),
&[
"run",
"index.mq.md",
"--allow-read",
"data.txt",
"--allow-write",
"out.bin",
"--json",
],
);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
let values = report["value"]["items"].as_array().expect("list result");
assert_eq!(values[0]["decimal"], "6");
assert_eq!(values[1]["value"], true);
assert_eq!(values[2]["value"], true);
assert_eq!(
std::fs::read(dir.path().join("out.bin")).unwrap(),
b"abcdef"
);
}
#[test]
fn file_stream_requires_a_read_grant() {
let dir = TempDir::new("file-denied");
write_file(&dir.path().join("data.txt"), "abcdef");
write_file(&dir.path().join("index.mq.md"), READER);
let output = cli(dir.path(), &["run", "index.mq.md", "--json"]);
assert_eq!(output.status.code(), Some(1));
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], false);
assert_eq!(report["error"]["code"], "permission.denied");
}
+39 -4
View File
@@ -195,15 +195,50 @@ impl Expr {
}
}
/// The payload of a statement.
/// The writable target on the left of `=` (ADR-0022).
///
/// v1 accepted only [`AssignTarget::Local`]. RFC-CM-0001 adds the second
/// variant because addressable `Memory` needs a writable place; no other
/// collection becomes mutable. Both variants use the same assignment syntax
/// and the same statement position.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum StatementKind {
/// `<bare identifier> = <expression>`; the only assignment form in v1.
Assign {
#[serde(tag = "place", rename_all = "snake_case")]
pub enum AssignTarget {
/// `name = value`: rebind a local slot in the current frame.
Local {
/// Assigned identifier.
name: String,
/// Span of the identifier token (not the whole assignment).
name_span: Span,
},
/// `base[index] = value`: write one byte into an addressable `Memory`.
Index {
/// Base expression, evaluated first.
base: Box<Expr>,
/// Index expression, evaluated second.
index: Box<Expr>,
/// Span from the base start through the closing `]`.
span: Span,
},
}
impl AssignTarget {
/// Best known source span of the assignment target.
pub fn span(&self) -> Span {
match self {
AssignTarget::Local { name_span, .. } => *name_span,
AssignTarget::Index { span, .. } => *span,
}
}
}
/// The payload of a statement.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum StatementKind {
/// `target = expression`; see [`AssignTarget`] for the writable places.
Assign {
/// Assigned place.
target: AssignTarget,
/// Right-hand side expression.
value: Expr,
},
+5 -3
View File
@@ -36,8 +36,8 @@
//! The grammar is frozen in `spec/expression.ebnf`. In order of decreasing
//! precedence: prefix `not` / `-` / `+`, `**` (right associative), postfix
//! calls `.name` / `[index]`, `* / // %`, `+ -`, a single non-associative
//! comparison, `and`, `or`. Assignment is only available at statement start
//! and only for a bare identifier target.
//! comparison, `and`, `or`. Assignment is only available at statement start;
//! its target is a bare identifier or an index expression (`m[8] = 42`).
//!
//! Notable lexical rules:
//!
@@ -84,7 +84,9 @@ use std::fmt;
use serde::Serialize;
pub use ast::{Arg, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp};
pub use ast::{
Arg, AssignTarget, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp,
};
pub use marqdo_span::Span;
pub use parser::MAX_NESTING_DEPTH;
+106 -40
View File
@@ -17,7 +17,9 @@
//! recursing into any sub-expression so adversarial inputs fail with a
//! structured [`ParseError`] instead of overflowing the stack.
use crate::ast::{Arg, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp};
use crate::ast::{
Arg, AssignTarget, BinaryOp, Expr, Ident, RecordEntry, Statement, StatementKind, UnaryOp,
};
use crate::lexer::{self, Token, TokenKind};
use crate::ParseError;
use marqdo_span::Span;
@@ -47,21 +49,16 @@ pub(crate) fn parse_statement(source: &str) -> Result<Statement, ParseError> {
return Err(parser.error_here("expected a statement, found end of input"));
}
let start = parser.peek().span.start;
if let TokenKind::Ident(_) = parser.peek().kind {
if matches!(
parser.peek_n(1).map(|token| &token.kind),
Some(TokenKind::Assign)
) {
return parser.assignment(start);
}
let left = parser.expression()?;
if matches!(parser.peek().kind, TokenKind::Assign) {
return parser.assignment(start, left);
}
let expr = parser.expression()?;
if !parser.at_eof() {
return Err(parser.trailing_statement_error(&expr));
return Err(parser.trailing_statement_error());
}
let end = parser.previous_end();
Ok(Statement {
kind: StatementKind::Expr(expr),
kind: StatementKind::Expr(left),
span: Span::new(start, end),
})
}
@@ -201,11 +198,8 @@ impl Parser {
// Statements
// ------------------------------------------------------------------
fn assignment(&mut self, start: usize) -> Result<Statement, ParseError> {
let TokenKind::Ident(name) = self.peek().kind.clone() else {
return Err(self.error_here("expected an identifier on the left of `=`"));
};
let name_token = self.bump();
fn assignment(&mut self, start: usize, left: Expr) -> Result<Statement, ParseError> {
let target = self.assign_target(left)?;
self.bump(); // `=`
if self.at_eof() {
return Err(self.error_here("expected an expression after `=`"));
@@ -226,29 +220,40 @@ impl Parser {
}
let end = self.previous_end();
Ok(Statement {
kind: StatementKind::Assign {
name,
name_span: name_token.span,
value,
},
kind: StatementKind::Assign { target, value },
span: Span::new(start, end),
})
}
fn trailing_statement_error(&self, expr: &Expr) -> ParseError {
if matches!(self.peek().kind, TokenKind::Assign) {
return if matches!(expr, Expr::Member { .. } | Expr::Index { .. }) {
self.error_here(
"invalid assignment target: member and index assignment are not supported \
in v1; assign a bare identifier",
)
} else {
self.error_here(
"invalid assignment target: assignment requires a bare identifier on the \
left of `=`",
)
};
/// Turn the already-parsed left expression into a writable target.
///
/// v1 only writes local slots. Addressable `Memory` adds index targets;
/// member assignment and every other shape stay hard parse errors so an
/// invalid executable shape never silently becomes prose (ADR-0022).
fn assign_target(&self, left: Expr) -> Result<AssignTarget, ParseError> {
match left {
Expr::Ident(Ident { name, span }) => Ok(AssignTarget::Local {
name,
name_span: span,
}),
Expr::Index { base, index, span } => Ok(AssignTarget::Index { base, index, span }),
Expr::Member { span, .. } => Err(ParseError::new(
span,
"member assignment is not supported in v1; only a local name or a \
Memory index is writable",
)),
other => Err(ParseError::new(
other.span(),
format!(
"invalid assignment target: {} is not a writable place; assign a local \
name or a Memory index",
describe_expr(&other)
),
)),
}
}
fn trailing_statement_error(&self) -> ParseError {
self.error_here(format!(
"unexpected {} after expression",
self.describe_current()
@@ -705,6 +710,24 @@ fn is_keyword(kind: &TokenKind) -> bool {
)
}
fn describe_expr(expr: &Expr) -> &'static str {
match expr {
Expr::Int(_) => "an integer literal",
Expr::Float(_) => "a float literal",
Expr::Text(_) => "a text literal",
Expr::Bool(_) => "a boolean literal",
Expr::None => "`None`",
Expr::Ident(_) => "an identifier",
Expr::List(_) => "a list literal",
Expr::Record(_) => "a record literal",
Expr::Unary { .. } => "a unary expression",
Expr::Binary { .. } => "a binary expression",
Expr::Call { .. } => "a call expression",
Expr::Member { .. } => "a member access",
Expr::Index { .. } => "an index expression",
}
}
fn describe_token(kind: &TokenKind) -> String {
match kind {
TokenKind::Int(raw) => format!("integer literal `{raw}`"),
@@ -1154,14 +1177,19 @@ mod tests {
assert_eq!(
stmt.kind,
StatementKind::Assign {
name: "x".to_owned(),
name_span: Span::new(0, 1),
target: AssignTarget::Local {
name: "x".to_owned(),
name_span: Span::new(0, 1),
},
value: Expr::Int(BigInt::from(1)),
}
);
let stmt = st("结果 = 阶乘(n=3)");
match &stmt.kind {
StatementKind::Assign { name, value, .. } => {
StatementKind::Assign {
target: AssignTarget::Local { name, .. },
value,
} => {
assert_eq!(name, "结果");
assert_eq!(sexpr(value), "(call 阶乘 n=3)");
}
@@ -1170,6 +1198,40 @@ mod tests {
assert_eq!(st("x = y + 1").span(), Span::new(0, 9));
}
#[test]
fn memory_index_assignment_parses_as_a_place() {
let stmt = st("m[8] = 42");
let StatementKind::Assign { target, value } = &stmt.kind else {
panic!("expected assignment, got {:?}", stmt.kind);
};
let AssignTarget::Index { base, index, span } = target else {
panic!("expected index target, got {target:?}");
};
assert_eq!(
**base,
Expr::Ident(Ident {
name: "m".to_owned(),
span: Span::new(0, 1),
})
);
assert_eq!(**index, Expr::Int(BigInt::from(8)));
assert_eq!(*span, Span::new(0, 4));
assert_eq!(*value, Expr::Int(BigInt::from(42)));
assert_eq!(stmt.span, Span::new(0, 9));
// Nested targets are syntactically places; the runtime decides whether
// the base value is a writable Memory region.
let stmt = st("outer[0][1] = 2");
let StatementKind::Assign {
target: AssignTarget::Index { base, .. },
..
} = &stmt.kind
else {
panic!("expected nested index target, got {:?}", stmt.kind);
};
assert!(matches!(base.as_ref(), Expr::Index { .. }));
}
#[test]
fn expression_statements_parse() {
assert_eq!(st("打印(1)").kind, StatementKind::Expr(ex("打印(1)")));
@@ -1185,8 +1247,8 @@ mod tests {
"x =",
"x = 1 = 2",
"a.b = 1",
"a[0] = 1",
"1 = 2",
"f() = 1",
"= 1",
"x = 1 2",
] {
@@ -1194,7 +1256,8 @@ mod tests {
assert_eq!(err.code, "expr.parse", "input {bad:?}");
}
assert!(st_err("x = 1 = 2").message.contains("chained assignment"));
assert!(st_err("a.b = 1").message.contains("member and index"));
assert!(st_err("a.b = 1").message.contains("member assignment"));
assert!(st_err("1 = 2").message.contains("writable place"));
assert!(st_err("x =").message.contains("after `=`"));
}
@@ -1228,7 +1291,10 @@ mod tests {
let stmt = st(" x = 1 ");
assert_eq!(stmt.span(), Span::new(2, 7));
match stmt.kind {
StatementKind::Assign { name_span, .. } => assert_eq!(name_span, Span::new(2, 3)),
StatementKind::Assign {
target: AssignTarget::Local { name_span, .. },
..
} => assert_eq!(name_span, Span::new(2, 3)),
other => panic!("expected assignment, got {other:?}"),
}
+13 -5
View File
@@ -8,7 +8,7 @@
use std::collections::{BTreeMap, BTreeSet};
use marqdo_expression::{Arg, Expr, Ident};
use marqdo_ir::{builtins, Module, ModuleId, Param, Program, SignatureKind, Stmt};
use marqdo_ir::{builtins, Module, ModuleId, Param, PlaceTarget, Program, SignatureKind, Stmt};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
@@ -109,7 +109,10 @@ pub fn validate_program(program: &Program) -> Vec<(ModuleId, Diagnostic)> {
fn collect_assigned(stmts: &[Stmt], out: &mut BTreeSet<String>) {
for stmt in stmts {
match stmt {
Stmt::Assign { name, .. } => {
Stmt::Assign {
target: PlaceTarget::Local { name, .. },
..
} => {
out.insert(name.clone());
}
Stmt::Branch { arms, .. } => {
@@ -138,9 +141,14 @@ impl<'a> CallChecker<'a> {
fn stmts(&mut self, stmts: &[Stmt]) {
for stmt in stmts {
match stmt {
Stmt::Assign { value, .. }
| Stmt::Eval { expr: value, .. }
| Stmt::Return { value, .. } => {
Stmt::Assign { target, value, .. } => {
if let PlaceTarget::Index { base, index, .. } = target {
self.expr(base);
self.expr(index);
}
self.expr(value);
}
Stmt::Eval { expr: value, .. } | Stmt::Return { value, .. } => {
self.expr(value);
}
Stmt::Call { call, .. } => {
+41 -6
View File
@@ -17,7 +17,7 @@
use std::collections::{BTreeMap, BTreeSet};
use marqdo_expression::{Arg, Expr, Ident, RecordEntry};
use marqdo_ir::{BranchArm, Stmt};
use marqdo_ir::{BranchArm, PlaceTarget, Stmt};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
@@ -178,9 +178,24 @@ impl<'a> Analyzer<'a> {
fn analyze_stmts(&mut self, stmts: &[Stmt], mut state: State) -> State {
for stmt in stmts {
state = match stmt {
Stmt::Assign { name, value, .. } => {
self.read_expr(value, &state);
self.assign(name, &mut state);
Stmt::Assign { target, value, .. } => {
match target {
PlaceTarget::Local { name, .. } => {
// Preserve v1 order: the RHS is read before the
// local rebinding makes the name definitely
// assigned.
self.read_expr(value, &state);
self.assign(name, &mut state);
}
PlaceTarget::Index { base, index, .. } => {
// Runtime order for an index place is base, index,
// RHS; reads in the place are free reads like any
// other.
self.read_expr(base, &state);
self.read_expr(index, &state);
self.read_expr(value, &state);
}
}
state
}
Stmt::Eval { expr, .. } => {
@@ -312,8 +327,10 @@ mod tests {
fn assign(name: &str, value: &str) -> Stmt {
Stmt::Assign {
name: name.to_string(),
name_span: Span::new(0, 0),
target: PlaceTarget::Local {
name: name.to_string(),
name_span: Span::new(0, 0),
},
value: parse_expression(value).unwrap(),
span: Span::new(0, 0),
}
@@ -334,6 +351,24 @@ mod tests {
.collect()
}
#[test]
fn index_assignment_reads_its_base_and_index_without_binding_a_local() {
let stmts = vec![
Stmt::Assign {
target: PlaceTarget::Index {
base: parse_expression("m").unwrap(),
index: parse_expression("i").unwrap(),
span: Span::new(0, 0),
},
value: parse_expression("9").unwrap(),
span: Span::new(0, 0),
},
ret("m[0]"),
];
// `m` and `i` are free reads; the place never rebinds `m` as a local.
assert_eq!(params(&stmts), vec!["i".to_string(), "m".to_string()]);
}
#[test]
fn infers_free_reads() {
// n * 阶乘(n=n-1)
+17 -9
View File
@@ -6,9 +6,9 @@
//! diagnostics and never fall back to prose.
use marqdo_expression::{
parse_arguments, parse_expression, parse_statement, Arg, Expr, StatementKind,
parse_arguments, parse_expression, parse_statement, Arg, AssignTarget, Expr, StatementKind,
};
use marqdo_ir::{CallSite, CallTarget, ImportDecl, Stmt};
use marqdo_ir::{CallSite, CallTarget, ImportDecl, PlaceTarget, Stmt};
use marqdo_markdown::{normalize_name, MarkdownDocument, MdKind, MdNode};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
@@ -143,13 +143,21 @@ impl<'a> Mapper<'a> {
Ok(mut statement) => {
segments.rewrite_statement(&mut statement);
Some(match statement.kind {
StatementKind::Assign {
name,
name_span,
value,
} => Stmt::Assign {
name,
name_span,
StatementKind::Assign { target, value } => Stmt::Assign {
target: match target {
AssignTarget::Local { name, name_span } => {
PlaceTarget::Local { name, name_span }
}
AssignTarget::Index {
base,
index,
span: target_span,
} => PlaceTarget::Index {
base: *base,
index: *index,
span: target_span,
},
},
value,
span: node.span,
},
+12 -6
View File
@@ -5,7 +5,7 @@
//! string, so its spans are payload-relative; this module maps them back to the
//! original file so diagnostics point at real source.
use marqdo_expression::{Arg, Expr, RecordEntry, Statement, StatementKind};
use marqdo_expression::{Arg, AssignTarget, Expr, RecordEntry, Statement, StatementKind};
use marqdo_markdown::{MdKind, MdNode, PayloadError};
use marqdo_span::Span;
@@ -140,11 +140,17 @@ impl PayloadSegments {
/// Map and rewrite every span in a parsed statement.
pub fn rewrite_statement(&self, statement: &mut Statement) {
statement.span = self.map_span(statement.span);
if let StatementKind::Assign {
name_span, value, ..
} = &mut statement.kind
{
*name_span = self.map_span(*name_span);
if let StatementKind::Assign { target, value } = &mut statement.kind {
match target {
AssignTarget::Local { name_span, .. } => {
*name_span = self.map_span(*name_span);
}
AssignTarget::Index { base, index, span } => {
*span = self.map_span(*span);
self.rewrite_expr(base);
self.rewrite_expr(index);
}
}
self.rewrite_expr(value);
}
if let StatementKind::Expr(expr) = &mut statement.kind {
+30 -9
View File
@@ -83,6 +83,22 @@ impl ProjectFs {
/// Read a UTF-8 file, bounded by [`MAX_READ_BYTES`].
pub fn read(&self, user_path: &str, span: Option<Span>) -> Result<String, RuntimeError> {
let bytes = self.read_bytes(user_path, span)?;
String::from_utf8(bytes).map_err(|_| {
RuntimeError::new(
ErrorCategory::Type,
span,
format!("`{user_path}` is not valid UTF-8"),
)
})
}
/// Read a file as raw bytes, bounded by [`MAX_READ_BYTES`].
///
/// Used by the resource stream provider; path/capability checks are the
/// caller's responsibility for grants, while this method still rejects
/// traversal and symlink escapes.
pub fn read_bytes(&self, user_path: &str, span: Option<Span>) -> Result<Vec<u8>, RuntimeError> {
let (absolute, relative) = self.resolve(user_path, span)?;
let metadata = std::fs::metadata(&absolute).map_err(|error| {
RuntimeError::new(
@@ -118,19 +134,12 @@ impl ProjectFs {
if !canonical.starts_with(&self.root) {
return Err(invalid_path(span, "path escapes the project root"));
}
let bytes = std::fs::read(&canonical).map_err(|error| {
std::fs::read(&canonical).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot read `{relative}`: {error}"),
)
})?;
String::from_utf8(bytes).map_err(|_| {
RuntimeError::new(
ErrorCategory::Type,
span,
format!("`{relative}` is not valid UTF-8"),
)
})
}
@@ -140,6 +149,18 @@ impl ProjectFs {
user_path: &str,
contents: &str,
span: Option<Span>,
) -> Result<(), RuntimeError> {
self.write_bytes(user_path, contents.as_bytes(), span)
}
/// Write raw bytes inside the project root, creating parent directories.
///
/// This is a single validated write, not a transactional replace.
pub fn write_bytes(
&self,
user_path: &str,
bytes: &[u8],
span: Option<Span>,
) -> Result<(), RuntimeError> {
let (absolute, relative) = self.resolve(user_path, span)?;
// Validate the closest existing ancestor stays inside the root.
@@ -169,7 +190,7 @@ impl ProjectFs {
)
})?;
}
std::fs::write(&absolute, contents).map_err(|error| {
std::fs::write(&absolute, bytes).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
+257 -1
View File
@@ -12,7 +12,7 @@ use marqdo_plugin_abi::{
ErrorKind as PluginErrorKind, LoadedPlugin, PluginError, PluginKind, PluginManifest,
};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, RuntimeError, Value};
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value};
use crate::budget::Budget;
use crate::capability::{Capabilities, CapabilityAudit, CapabilityBroker};
@@ -20,6 +20,7 @@ use crate::context::ExecutionContext;
use crate::fs::ProjectFs;
use crate::handles::{contains_plugin_handle, PluginHandleTable};
use crate::plugin_worker::WorkerPlugin;
use crate::resource::{ResourceRead, ResourceTable};
use crate::trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState};
use marqdo_sandbox::{PlatformCapabilities, SandboxMode};
@@ -79,6 +80,65 @@ pub trait Host {
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError>;
/// Open a resource/stream by kind, descriptor and requested rights.
///
/// Default: unavailable. Only the standard host implements the first
/// resource kinds (ADR-0025).
fn resource_open(
&mut self,
_kind: &str,
_descriptor: &str,
_rights: &[String],
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"this host does not provide resource streams",
))
}
/// Read up to `max_bytes` from a resource into host memory.
fn resource_read(
&mut self,
_handle: ResourceHandle,
_max_bytes: usize,
ctx: &ExecutionContext,
) -> Result<ResourceRead, RuntimeError> {
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"this host does not provide resource streams",
))
}
/// Write bytes to a resource; returns the accepted count.
fn resource_write(
&mut self,
_handle: ResourceHandle,
_bytes: &[u8],
ctx: &ExecutionContext,
) -> Result<usize, RuntimeError> {
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"this host does not provide resource streams",
))
}
/// Close a resource handle.
fn resource_close(
&mut self,
_handle: ResourceHandle,
ctx: &ExecutionContext,
) -> Result<(), RuntimeError> {
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"this host does not provide resource streams",
))
}
/// Consume evaluation fuel.
fn consume_fuel(&mut self, amount: u64, span: Option<Span>) -> Result<(), RuntimeError>;
@@ -143,6 +203,8 @@ pub struct StdHost {
broker_audit: Arc<Mutex<Vec<CapabilityAudit>>>,
/// Host-issued opaque plugin handles.
pub handles: PluginHandleTable,
/// Host-owned resource/stream table (ADR-0025).
resources: ResourceTable,
semantic_events: Vec<SemanticEvent>,
next_event_seq: u64,
trace_state: TraceState,
@@ -203,6 +265,7 @@ impl StdHost {
vendor_capabilities: BTreeSet::new(),
broker_audit: Arc::new(Mutex::new(Vec::new())),
handles: PluginHandleTable::default(),
resources: ResourceTable::new(),
semantic_events: Vec::new(),
next_event_seq: 0,
trace_state: TraceState::default(),
@@ -526,6 +589,96 @@ impl StdHost {
self.fs.write(&path, &contents, ctx.span)?;
Ok(Value::Unit)
}
fn open_resource_stream(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
if self.mode == SandboxMode::Strict {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"resource streams are not verified in strict mode and are refused",
));
}
if self.mode == SandboxMode::Pure && kind == "file" {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"pure mode does not provide project file resources",
));
}
let requested: BTreeSet<&str> = rights.iter().map(String::as_str).collect();
match kind {
"stdio" => {
if descriptor != "stdout" || requested != BTreeSet::from(["write"]) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"the first stream slice supports only stdio/stdout with rights [write]",
));
}
if !self.capabilities.console {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"the console capability is required for stdio/stdout",
));
}
let handle = self.resources.open_stdout();
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some("kind=stdio descriptor=stdout"),
);
Ok(Value::Resource(handle))
}
"file" => {
if requested == BTreeSet::from(["read"]) {
self.capabilities.check_read(descriptor, ctx.span)?;
let (_, relative) = self.fs.resolve(descriptor, ctx.span)?;
let bytes = self.fs.read_bytes(descriptor, ctx.span)?;
let handle = self.resources.open_file(relative, bytes);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some("kind=file rights=read"),
);
return Ok(Value::Resource(handle));
}
if requested == BTreeSet::from(["write"]) {
self.capabilities.check_write(descriptor, ctx.span)?;
let (_, relative) = self.fs.resolve(descriptor, ctx.span)?;
let handle = self.resources.open_file_writer(relative);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some("kind=file rights=write"),
);
return Ok(Value::Resource(handle));
}
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"project file streams support exactly [read] or [write] in this build",
))
}
other => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
format!("resource kind `{other}` is not implemented in this build"),
)),
}
}
}
impl Host for StdHost {
@@ -535,6 +688,31 @@ impl Host for StdHost {
args: Vec<Value>,
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
// Addressable memory is language-local state. Handles must not reach a
// plugin, foreign kernel or file adapter, where they could not be
// interpreted safely (RFC-CM-0001 §8, ADR-0022 §4).
if args.iter().any(Value::contains_memory) {
return Err(RuntimeError::new(
ErrorCategory::Type,
ctx.span,
format!(
"Memory values cannot cross the host boundary (`{}`); copy bytes into \
Text explicitly first",
op.name()
),
));
}
if args.iter().any(Value::contains_resource) {
return Err(RuntimeError::new(
ErrorCategory::Type,
ctx.span,
format!(
"Resource handles cannot cross the generic host boundary (`{}`); use \
resource.* operations",
op.name()
),
));
}
let started = Instant::now();
let result = self.dispatch(op, args, ctx);
let summary = match &result {
@@ -562,6 +740,84 @@ impl Host for StdHost {
result
}
fn resource_open(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
self.open_resource_stream(kind, descriptor, rights, ctx)
}
fn resource_read(
&mut self,
handle: ResourceHandle,
max_bytes: usize,
ctx: &ExecutionContext,
) -> Result<ResourceRead, RuntimeError> {
let kind = self.resources.kind(handle, ctx.span)?;
let read = self.resources.read(handle, max_bytes, ctx.span)?;
self.record_semantic(
"resource.read",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!(
"kind={kind} count={} eof={}",
read.bytes.len(),
read.eof
)),
);
Ok(read)
}
fn resource_write(
&mut self,
handle: ResourceHandle,
bytes: &[u8],
ctx: &ExecutionContext,
) -> Result<usize, RuntimeError> {
let kind = self.resources.kind(handle, ctx.span)?;
let count = self.resources.write(handle, bytes, ctx.span)?;
if kind == "stdio" {
self.budget.consume_output(count, ctx.span)?;
self.output.extend_from_slice(bytes);
}
self.record_semantic(
"resource.write",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind={kind} count={count}")),
);
Ok(count)
}
fn resource_close(
&mut self,
handle: ResourceHandle,
ctx: &ExecutionContext,
) -> Result<(), RuntimeError> {
let pending = self.resources.pending_file_write(handle, ctx.span)?;
let count = match pending {
Some((relative, bytes)) => {
self.fs.write_bytes(&relative, &bytes, ctx.span)?;
bytes.len()
}
None => 0,
};
self.resources.close(handle, ctx.span)?;
self.record_semantic(
"resource.close",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("flushed_bytes={count}")),
);
Ok(())
}
fn consume_fuel(&mut self, amount: u64, span: Option<Span>) -> Result<(), RuntimeError> {
self.budget.consume_fuel(amount, span)
}
+2
View File
@@ -12,6 +12,7 @@ pub mod handles;
pub mod host;
pub mod kernels;
mod plugin_worker;
pub mod resource;
pub mod trace;
pub use budget::{Budget, DEFAULT_FUEL, DEFAULT_OUTPUT_BYTES, DEFAULT_STACK_LIMIT};
@@ -21,4 +22,5 @@ pub use fs::ProjectFs;
pub use handles::{contains_plugin_handle, PluginHandleTable};
pub use host::{arity, expect_text, Host, HostOp, StdHost};
pub use marqdo_sandbox::{PlatformCapabilities, SandboxMode, StrictIsolation};
pub use resource::{ResourceRead, ResourceTable};
pub use trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState, TraceTruncation};
+390
View File
@@ -0,0 +1,390 @@
//! Host-owned resource/stream table (ADR-0025).
//!
//! The first slice deliberately supports only stdout writes and bounded
//! project-file reads loaded at open. Handles are run-scoped and opaque; the
//! table validates run identity, slot identity and generation on every call.
use std::collections::BTreeMap;
use std::sync::atomic::{AtomicU64, Ordering};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError};
static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1);
/// Bytes returned by one stream read.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ResourceRead {
/// Bytes actually read (may be empty at EOF).
pub bytes: Vec<u8>,
/// Whether the stream has reached EOF.
pub eof: bool,
}
enum Stream {
Stdout {
closed: bool,
},
File {
relative: String,
bytes: Vec<u8>,
position: usize,
closed: bool,
},
FileWriter {
relative: String,
bytes: Vec<u8>,
closed: bool,
},
}
struct Slot {
generation: u64,
stream: Stream,
}
impl Slot {
fn closed(&self) -> bool {
match &self.stream {
Stream::Stdout { closed } => *closed,
Stream::File { closed, .. } => *closed,
Stream::FileWriter { closed, .. } => *closed,
}
}
}
/// Host resource table for one run.
pub struct ResourceTable {
run_id: u64,
next_resource: u64,
slots: BTreeMap<u64, Slot>,
}
impl Default for ResourceTable {
fn default() -> Self {
Self::new()
}
}
impl ResourceTable {
/// Create an empty table with a fresh run identity.
pub fn new() -> Self {
Self {
run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed),
next_resource: 1,
slots: BTreeMap::new(),
}
}
/// Number of live (not necessarily open) slots.
pub fn slot_count(&self) -> usize {
self.slots.len()
}
/// Register the pre-opened stdout stream.
pub fn open_stdout(&mut self) -> ResourceHandle {
self.insert(Stream::Stdout { closed: false })
}
/// Register a project-file read stream opened from already-bounded bytes.
pub fn open_file(&mut self, relative: impl Into<String>, bytes: Vec<u8>) -> ResourceHandle {
self.insert(Stream::File {
relative: relative.into(),
bytes,
position: 0,
closed: false,
})
}
/// Register an empty buffered project-file write stream.
pub fn open_file_writer(&mut self, relative: impl Into<String>) -> ResourceHandle {
self.insert(Stream::FileWriter {
relative: relative.into(),
bytes: Vec::new(),
closed: false,
})
}
fn insert(&mut self, stream: Stream) -> ResourceHandle {
let id = self.next_resource;
self.next_resource += 1;
self.slots.insert(
id,
Slot {
generation: 1,
stream,
},
);
ResourceHandle::from_host(self.run_id, id, 1)
}
fn slot(&self, handle: ResourceHandle, span: Option<Span>) -> Result<&Slot, RuntimeError> {
if handle.run_id() != self.run_id {
return Err(RuntimeError::new(
ErrorCategory::ResourceFault,
span,
"Resource handle belongs to a different host run",
));
}
let Some(slot) = self.slots.get(&handle.resource_id()) else {
return Err(RuntimeError::new(
ErrorCategory::ResourceFault,
span,
"unknown Resource handle",
));
};
if slot.generation != handle.generation() {
return Err(RuntimeError::new(
ErrorCategory::ResourceFault,
span,
"stale Resource handle",
));
}
Ok(slot)
}
fn slot_mut(
&mut self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<&mut Slot, RuntimeError> {
self.slot(handle, span)?;
self.slots
.get_mut(&handle.resource_id())
.ok_or_else(|| unknown_handle(span))
}
/// Read up to `max_bytes` from a file stream.
///
/// Reads at EOF return `count=0, eof=true`. A read on a closed stream is
/// `resource.closed`; stdout is not readable.
pub fn read(
&mut self,
handle: ResourceHandle,
max_bytes: usize,
span: Option<Span>,
) -> Result<ResourceRead, RuntimeError> {
let slot = self.slot_mut(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is closed",
));
}
match &mut slot.stream {
Stream::File {
bytes, position, ..
} => {
let start = *position;
let end = start.saturating_add(max_bytes).min(bytes.len());
let chunk = bytes[start..end].to_vec();
*position = end;
Ok(ResourceRead {
bytes: chunk,
eof: end >= bytes.len(),
})
}
Stream::Stdout { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"stdout does not support resource.read",
)),
Stream::FileWriter { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"a file write stream does not support resource.read",
)),
}
}
/// Validate a stdout write and return the accepted byte count.
///
/// The caller performs the output-budget accounting and buffer append; this
/// method only checks handle identity, closed state and stream kind.
pub fn write(
&mut self,
handle: ResourceHandle,
bytes: &[u8],
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let slot = self.slot_mut(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is closed",
));
}
match &mut slot.stream {
Stream::Stdout { .. } => Ok(bytes.len()),
Stream::File { relative, .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
format!("project file `{relative}` was opened read-only"),
)),
Stream::FileWriter {
relative,
bytes: buffer,
..
} => {
let next = buffer.len().checked_add(bytes.len());
let Some(next) = next else {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("file write buffer for `{relative}` overflows"),
));
};
if next > crate::fs::MAX_READ_BYTES as usize {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"file write buffer for `{relative}` exceeds the {} byte limit",
crate::fs::MAX_READ_BYTES
),
));
}
buffer.extend_from_slice(bytes);
Ok(bytes.len())
}
}
}
/// Close a stream. A second close is `resource.closed`.
pub fn close(
&mut self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<(), RuntimeError> {
let slot = self.slot_mut(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is already closed",
));
}
match &mut slot.stream {
Stream::Stdout { closed } => *closed = true,
Stream::File { closed, .. } => *closed = true,
Stream::FileWriter { closed, .. } => *closed = true,
}
Ok(())
}
/// Return the buffered bytes of a file write stream without closing it.
///
/// `None` means the handle is not a file writer (stdout or a reader). The
/// caller must call [`Self::close`] only after the flush succeeds, so a
/// failed flush leaves the writer open with its buffer intact.
pub fn pending_file_write(
&self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<Option<(String, Vec<u8>)>, RuntimeError> {
let slot = self.slot(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is closed",
));
}
match &slot.stream {
Stream::FileWriter {
relative, bytes, ..
} => Ok(Some((relative.clone(), bytes.clone()))),
_ => Ok(None),
}
}
/// Descriptor kind for diagnostics/trace, if the handle is valid.
pub fn kind(
&self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<&'static str, RuntimeError> {
let slot = self.slot(handle, span)?;
Ok(match slot.stream {
Stream::Stdout { .. } => "stdio",
Stream::File { .. } | Stream::FileWriter { .. } => "file",
})
}
}
fn unknown_handle(span: Option<Span>) -> RuntimeError {
RuntimeError::new(
ErrorCategory::ResourceFault,
span,
"unknown Resource handle",
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn file_reads_are_chunked_with_eof() {
let mut table = ResourceTable::new();
let handle = table.open_file("data.txt", b"abcdef".to_vec());
assert_eq!(
table.read(handle, 2, None).unwrap(),
ResourceRead {
bytes: b"ab".to_vec(),
eof: false
}
);
assert_eq!(
table.read(handle, 8, None).unwrap(),
ResourceRead {
bytes: b"cdef".to_vec(),
eof: true
}
);
assert_eq!(
table.read(handle, 1, None).unwrap(),
ResourceRead {
bytes: Vec::new(),
eof: true
}
);
}
#[test]
fn forged_and_closed_handles_are_rejected() {
let mut table = ResourceTable::new();
let handle = table.open_stdout();
let forged = ResourceHandle::from_host(handle.run_id() + 1, handle.resource_id(), 1);
assert_eq!(
table.write(forged, b"x", None).unwrap_err().code(),
"runtime.resource"
);
table.close(handle, None).unwrap();
assert_eq!(
table.write(handle, b"x", None).unwrap_err().code(),
"resource.closed"
);
assert_eq!(
table.close(handle, None).unwrap_err().code(),
"resource.closed"
);
}
#[test]
fn stdout_is_not_readable_and_file_is_not_writable() {
let mut table = ResourceTable::new();
let stdout = table.open_stdout();
assert_eq!(
table.read(stdout, 1, None).unwrap_err().code(),
"resource.unsupported"
);
let file = table.open_file("x.txt", Vec::new());
assert_eq!(
table.write(file, b"x", None).unwrap_err().code(),
"resource.unsupported"
);
}
}
@@ -0,0 +1,205 @@
//! Resource/stream host boundary tests (ADR-0025).
use std::path::Path;
use marqdo_host::{Capabilities, ExecutionContext, Host, HostOp, StdHost};
use marqdo_sandbox::SandboxMode;
use marqdo_value::{ErrorCategory, Value};
fn context() -> ExecutionContext {
ExecutionContext::new("index.mq.md".to_string(), None)
}
fn handle(value: Value) -> marqdo_value::ResourceHandle {
match value {
Value::Resource(handle) => handle,
other => panic!("expected a Resource handle, got {other:?}"),
}
}
fn write_project_file(root: &Path, name: &str, contents: &[u8]) {
std::fs::write(root.join(name), contents).unwrap();
}
#[test]
fn pure_mode_writes_stdout_through_a_resource() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let ctx = context();
let handle = handle(
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"hello", &ctx).unwrap(), 5);
assert_eq!(host.take_output(), b"hello");
host.resource_close(handle, &ctx).unwrap();
let error = host.resource_close(handle, &ctx).unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceClosed);
assert_eq!(error.code(), "resource.closed");
}
#[test]
fn stdout_write_obeys_the_output_budget_atomically() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
host.budget.max_output_bytes = 3;
let ctx = context();
let handle = handle(
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
.unwrap(),
);
let error = host.resource_write(handle, b"hello", &ctx).unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
assert!(host.take_output().is_empty());
}
#[test]
fn strict_mode_refuses_every_resource_open() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Strict, Capabilities::pure(), dir.path());
let error = host
.resource_open("stdio", "stdout", &["write".to_string()], &context())
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn resource_handles_cannot_cross_the_generic_host_boundary() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let ctx = context();
let handle = handle(
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
.unwrap(),
);
let error = host
.invoke(HostOp::Print, vec![Value::Resource(handle)], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::Type);
assert!(error.message.contains("Resource handles"));
}
#[test]
fn pure_mode_refuses_file_resources_even_with_a_grant() {
let dir = tempfile::tempdir().unwrap();
write_project_file(dir.path(), "a.txt", b"abcdef");
let mut host = StdHost::new(
SandboxMode::Pure,
Capabilities::pure().allow_read("a.txt"),
dir.path(),
);
let error = host
.resource_open("file", "a.txt", &["read".to_string()], &context())
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn project_file_writes_buffer_until_close_and_require_a_grant() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_write("out.bin"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open("file", "out.bin", &["write".to_string()], &ctx)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"abc", &ctx).unwrap(), 3);
assert_eq!(host.resource_write(handle, b"def", &ctx).unwrap(), 3);
assert!(
!dir.path().join("out.bin").exists(),
"write must not flush early"
);
host.resource_close(handle, &ctx).unwrap();
assert_eq!(
std::fs::read(dir.path().join("out.bin")).unwrap(),
b"abcdef"
);
let error = host.resource_read(handle, 1, &ctx).unwrap_err();
assert_eq!(error.code(), "resource.closed");
let second = host.resource_close(handle, &ctx).unwrap_err();
assert_eq!(second.code(), "resource.closed");
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = denied
.resource_open("file", "out.bin", &["write".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn file_write_buffer_has_a_hard_ceiling() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_write("big.bin"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open("file", "big.bin", &["write".to_string()], &ctx)
.unwrap(),
);
let oversized = vec![0u8; marqdo_host::fs::MAX_READ_BYTES as usize + 1];
let error = host.resource_write(handle, &oversized, &ctx).unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
assert_eq!(host.resource_write(handle, b"ok", &ctx).unwrap(), 2);
host.resource_close(handle, &ctx).unwrap();
assert_eq!(std::fs::read(dir.path().join("big.bin")).unwrap(), b"ok");
}
#[test]
fn project_file_reads_are_chunked_and_require_a_grant() {
let dir = tempfile::tempdir().unwrap();
write_project_file(dir.path(), "a.txt", b"abcdef");
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_read("a.txt"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open("file", "a.txt", &["read".to_string()], &ctx)
.unwrap(),
);
let first = host.resource_read(handle, 2, &ctx).unwrap();
assert_eq!(first.bytes, b"ab");
assert!(!first.eof);
let second = host.resource_read(handle, 8, &ctx).unwrap();
assert_eq!(second.bytes, b"cdef");
assert!(second.eof);
assert!(host.resource_read(handle, 1, &ctx).unwrap().eof);
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = denied
.resource_open("file", "a.txt", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn traversal_and_unsupported_rights_are_refused() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_read("**"),
dir.path(),
);
let ctx = context();
let traversal = host
.resource_open("file", "../secret", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(traversal.category, ErrorCategory::PermissionDenied);
let write_rights = host
.resource_open("stdio", "stdout", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(write_rights.category, ErrorCategory::ResourceUnsupported);
let unknown_kind = host
.resource_open("socket", "example.test", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(unknown_kind.category, ErrorCategory::ResourceUnsupported);
}
+157
View File
@@ -167,6 +167,144 @@ const FS_WRITE: Builtin = Builtin {
named_only: false,
effectful: true,
};
const MEMORY_NEW: Builtin = Builtin {
name: "memory.new",
aliases: &[],
params: &[req("initial_pages"), req("max_pages")],
named_only: false,
effectful: false,
};
const MEMORY_SIZE: Builtin = Builtin {
name: "memory.size",
aliases: &[],
params: &[req("memory")],
named_only: false,
effectful: false,
};
const MEMORY_GROW: Builtin = Builtin {
name: "memory.grow",
aliases: &[],
params: &[req("memory"), req("pages")],
named_only: false,
effectful: false,
};
const LIST_APPEND: Builtin = Builtin {
name: "list.append",
aliases: &[],
params: &[req("list"), req("value")],
named_only: false,
effectful: false,
};
const LIST_CONCAT: Builtin = Builtin {
name: "list.concat",
aliases: &[],
params: &[req("left"), req("right")],
named_only: false,
effectful: false,
};
const MAP_KEYS: Builtin = Builtin {
name: "map.keys",
aliases: &[],
params: &[req("map")],
named_only: false,
effectful: false,
};
const MAP_HAS: Builtin = Builtin {
name: "map.has",
aliases: &[],
params: &[req("map"), req("key")],
named_only: false,
effectful: false,
};
const MAP_GET: Builtin = Builtin {
name: "map.get",
aliases: &[],
params: &[req("map"), req("key")],
named_only: false,
effectful: false,
};
const MAP_SET: Builtin = Builtin {
name: "map.set",
aliases: &[],
params: &[req("map"), req("key"), req("value")],
named_only: false,
effectful: false,
};
const MAP_REMOVE: Builtin = Builtin {
name: "map.remove",
aliases: &[],
params: &[req("map"), req("key")],
named_only: false,
effectful: false,
};
const TEXT_CODEPOINTS: Builtin = Builtin {
name: "text.codepoints",
aliases: &[],
params: &[req("text")],
named_only: false,
effectful: false,
};
const TEXT_FROM_CODEPOINT: Builtin = Builtin {
name: "text.from_codepoint",
aliases: &[],
params: &[req("codepoint")],
named_only: false,
effectful: false,
};
const TABLE_COLUMNS: Builtin = Builtin {
name: "table.columns",
aliases: &[],
params: &[req("table")],
named_only: false,
effectful: false,
};
const TABLE_ROW: Builtin = Builtin {
name: "table.row",
aliases: &[],
params: &[req("table"), req("row")],
named_only: false,
effectful: false,
};
const RESOURCE_OPEN: Builtin = Builtin {
name: "resource.open",
aliases: &[],
params: &[req("kind"), req("descriptor"), req("rights")],
named_only: false,
effectful: true,
};
const RESOURCE_READ: Builtin = Builtin {
name: "resource.read",
aliases: &[],
params: &[
req("handle"),
req("memory"),
req("offset"),
req("max_bytes"),
],
named_only: false,
effectful: true,
};
const RESOURCE_WRITE: Builtin = Builtin {
name: "resource.write",
aliases: &[],
params: &[req("handle"), req("memory"), req("offset"), req("length")],
named_only: false,
effectful: true,
};
const RESOURCE_CLOSE: Builtin = Builtin {
name: "resource.close",
aliases: &[],
params: &[req("handle")],
named_only: false,
effectful: true,
};
const TABLE_CELL: Builtin = Builtin {
name: "table.cell",
aliases: &[],
params: &[req("table"), req("row"), req("column")],
named_only: false,
effectful: false,
};
/// All builtins known to the language. Host availability is decided at call
/// time by the capability layer, not here.
@@ -188,6 +326,25 @@ pub const BUILTINS: &[Builtin] = &[
FOREIGN_CALL,
FS_READ,
FS_WRITE,
MEMORY_NEW,
MEMORY_SIZE,
MEMORY_GROW,
LIST_APPEND,
LIST_CONCAT,
MAP_KEYS,
MAP_HAS,
MAP_GET,
MAP_SET,
MAP_REMOVE,
TEXT_CODEPOINTS,
TEXT_FROM_CODEPOINT,
TABLE_COLUMNS,
TABLE_ROW,
TABLE_CELL,
RESOURCE_OPEN,
RESOURCE_READ,
RESOURCE_WRITE,
RESOURCE_CLOSE,
];
/// Find a builtin by canonical name or alias.
+40 -5
View File
@@ -224,16 +224,51 @@ pub struct Function {
pub body: Vec<Stmt>,
}
/// Writable place of an assignment (ADR-0022).
///
/// The Markdown surface is unchanged: both variants are `strong` nodes whose
/// payload contains `target = expression`. `Local` rebinds a frame slot;
/// `Index` writes one byte into an addressable runtime `Memory` region and is
/// rejected for every other base value.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(tag = "place", rename_all = "snake_case")]
pub enum PlaceTarget {
/// Local slot rebinding.
Local {
/// Target name.
name: String,
/// Span of the target name.
name_span: Span,
},
/// `base[index] = value`; only a `Memory` base is writable.
Index {
/// Base expression, evaluated first.
base: Expr,
/// Index expression, evaluated second.
index: Expr,
/// Span from the base start through the closing `]`.
span: Span,
},
}
impl PlaceTarget {
/// Best known source span of the place.
pub fn span(&self) -> Span {
match self {
PlaceTarget::Local { name_span, .. } => *name_span,
PlaceTarget::Index { span, .. } => *span,
}
}
}
/// An executable statement.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(tag = "stmt", rename_all = "snake_case")]
pub enum Stmt {
/// Local assignment (`strong` node with `name = expr`).
/// Assignment to a local slot or a `Memory` index.
Assign {
/// Target name.
name: String,
/// Span of the target name (approximated by the payload span).
name_span: Span,
/// Writable place.
target: PlaceTarget,
/// Value expression.
value: Expr,
/// Span of the whole statement.
+511 -3
View File
@@ -6,13 +6,13 @@
use std::collections::BTreeMap;
use marqdo_host::HostOp;
use marqdo_host::{HostOp, ResourceRead};
use marqdo_ir::builtins::Builtin;
use marqdo_ir::Resource;
use marqdo_span::Span;
use marqdo_value::{
float_to_bigint, CodeValue, ErrorCategory, FormulaValue, FunctionRef, RuntimeError, Table,
Value,
float_to_bigint, CodeValue, ErrorCategory, FormulaValue, FunctionRef, ResourceHandle,
RuntimeError, Table, Value,
};
use crate::interpreter::Interpreter;
@@ -187,6 +187,292 @@ pub(crate) fn dispatch(
&module,
)
}
"memory.new" => {
let initial_pages = page_count(take(&bound, "initial_pages", builtin, span)?, span)?;
let max_pages = page_count(take(&bound, "max_pages", builtin, span)?, span)?;
let handle = interpreter
.memory_mut()
.create(initial_pages, max_pages, span)?;
interpreter.record_memory_event(
"memory.new",
&module,
span,
&format!("initial_pages={initial_pages} max_pages={max_pages}"),
);
Ok(Value::Memory(handle))
}
"memory.size" => {
let memory = take(&bound, "memory", builtin, span)?;
let memory_type = memory.type_name();
let Value::Memory(handle) = memory else {
return Err(RuntimeError::type_error(
span,
format!("memory.size expects a Memory value, got {memory_type}"),
));
};
let pages = interpreter.memory().size(handle, span)?;
Ok(pages_value(pages, span)?)
}
"memory.grow" => {
let memory = take(&bound, "memory", builtin, span)?;
let memory_type = memory.type_name();
let Value::Memory(handle) = memory else {
return Err(RuntimeError::type_error(
span,
format!("memory.grow expects a Memory value, got {memory_type}"),
));
};
let pages = page_count(take(&bound, "pages", builtin, span)?, span)?;
let old_pages = interpreter.memory_mut().grow(handle, pages, span)?;
interpreter.record_memory_event(
"memory.grow",
&module,
span,
&format!("pages={pages} old_pages={old_pages}"),
);
Ok(pages_value(old_pages, span)?)
}
"list.append" => {
let list = take(&bound, "list", builtin, span)?;
let value = take(&bound, "value", builtin, span)?;
let list_type = list.type_name();
let Value::List(mut items) = list else {
return Err(RuntimeError::type_error(
span,
format!("list.append expects a List, got {list_type}"),
));
};
items.push(value);
Ok(Value::List(items))
}
"list.concat" => {
let left = take(&bound, "left", builtin, span)?;
let right = take(&bound, "right", builtin, span)?;
let left_type = left.type_name();
let Value::List(mut items) = left else {
return Err(RuntimeError::type_error(
span,
format!("list.concat left operand must be a List, got {left_type}"),
));
};
let right_type = right.type_name();
let Value::List(other) = right else {
return Err(RuntimeError::type_error(
span,
format!("list.concat right operand must be a List, got {right_type}"),
));
};
items.extend(other);
Ok(Value::List(items))
}
"map.keys" => {
let value = take(&bound, "map", builtin, span)?;
let value_type = value.type_name();
let Value::Map(map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.keys expects a Map, got {value_type}"),
));
};
Ok(Value::List(
map.keys().cloned().map(Value::Text).collect::<Vec<_>>(),
))
}
"map.has" => {
let value = take(&bound, "map", builtin, span)?;
let key = map_key(take(&bound, "key", builtin, span)?, "map.has", span)?;
let value_type = value.type_name();
let Value::Map(map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.has expects a Map, got {value_type}"),
));
};
Ok(Value::Bool(map.contains_key(&key)))
}
"map.get" => {
let value = take(&bound, "map", builtin, span)?;
let key = map_key(take(&bound, "key", builtin, span)?, "map.get", span)?;
let value_type = value.type_name();
let Value::Map(map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.get expects a Map, got {value_type}"),
));
};
Ok(map.get(&key).cloned().unwrap_or(Value::None))
}
"map.set" => {
let value = take(&bound, "map", builtin, span)?;
let key = map_key(take(&bound, "key", builtin, span)?, "map.set", span)?;
let new_value = take(&bound, "value", builtin, span)?;
let value_type = value.type_name();
let Value::Map(mut map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.set expects a Map, got {value_type}"),
));
};
map.insert(key, new_value);
Ok(Value::Map(map))
}
"map.remove" => {
let value = take(&bound, "map", builtin, span)?;
let key = map_key(take(&bound, "key", builtin, span)?, "map.remove", span)?;
let value_type = value.type_name();
let Value::Map(mut map) = value else {
return Err(RuntimeError::type_error(
span,
format!("map.remove expects a Map, got {value_type}"),
));
};
map.remove(&key);
Ok(Value::Map(map))
}
"text.codepoints" => {
let value = take(&bound, "text", builtin, span)?;
let value_type = value.type_name();
let Value::Text(text) = value else {
return Err(RuntimeError::type_error(
span,
format!("text.codepoints expects Text, got {value_type}"),
));
};
Ok(Value::List(
text.chars()
.map(|character| Value::int(i64::from(character as u32)))
.collect(),
))
}
"text.from_codepoint" => {
let value = take(&bound, "codepoint", builtin, span)?;
let value_type = value.type_name();
let Value::Int(codepoint) = value else {
return Err(RuntimeError::type_error(
span,
format!("text.from_codepoint expects an Int, got {value_type}"),
));
};
let Some(codepoint) = num_traits::ToPrimitive::to_u32(&codepoint) else {
return Ok(Value::None);
};
Ok(match char::from_u32(codepoint) {
Some(character) => Value::Text(character.to_string()),
None => Value::None,
})
}
"table.columns" => {
let value = take(&bound, "table", builtin, span)?;
let value_type = value.type_name();
let Value::Table(table) = value else {
return Err(RuntimeError::type_error(
span,
format!("table.columns expects a Table, got {value_type}"),
));
};
Ok(Value::List(
table.header.into_iter().map(Value::Text).collect(),
))
}
"table.row" => {
let value = take(&bound, "table", builtin, span)?;
let row = take(&bound, "row", builtin, span)?;
let value_type = value.type_name();
let Value::Table(table) = value else {
return Err(RuntimeError::type_error(
span,
format!("table.row expects a Table, got {value_type}"),
));
};
let position = table_index(row, table.rows.len(), "row", span)?;
Ok(Value::List(table.rows[position].clone()))
}
"table.cell" => {
let value = take(&bound, "table", builtin, span)?;
let row = take(&bound, "row", builtin, span)?;
let column = take(&bound, "column", builtin, span)?;
let value_type = value.type_name();
let Value::Table(table) = value else {
return Err(RuntimeError::type_error(
span,
format!("table.cell expects a Table, got {value_type}"),
));
};
let row = table_index(row, table.rows.len(), "row", span)?;
let column = table_index(column, table.header.len(), "column", span)?;
Ok(table.rows[row][column].clone())
}
"resource.open" => {
let kind = expect_text_value(take(&bound, "kind", builtin, span)?, span)?;
let descriptor = expect_text_value(take(&bound, "descriptor", builtin, span)?, span)?;
let rights = rights_list(take(&bound, "rights", builtin, span)?, span)?;
interpreter.resource_open(&kind, &descriptor, &rights, &module, span)
}
"resource.read" => {
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
let memory = memory_handle(
take(&bound, "memory", builtin, span)?,
"resource.read",
span,
)?;
let offset = take(&bound, "offset", builtin, span)?;
let offset_int = int_value(offset, "resource.read offset", span)?;
let offset_value = Value::Int(offset_int.clone());
let max_bytes = positive_count(
take(&bound, "max_bytes", builtin, span)?,
"resource.read max_bytes",
span,
)?;
let available = interpreter.memory().byte_len(memory, span)?;
let start = non_negative_usize(&offset_value, "resource.read offset", span)?;
if start >= available {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("resource.read offset {start} is at or beyond the Memory length {available}"),
));
}
let limit = max_bytes.min(available - start);
let result = interpreter.resource_read(handle, limit, &module, span);
match result {
Ok(read) => {
interpreter
.memory_mut()
.write_bytes(memory, &offset_int, &read.bytes, span)?;
Ok(resource_ok(resource_read_payload(&read)))
}
Err(error) => resource_recoverable(error),
}
}
"resource.write" => {
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
let memory = memory_handle(
take(&bound, "memory", builtin, span)?,
"resource.write",
span,
)?;
let offset = take(&bound, "offset", builtin, span)?;
let offset_int = int_value(offset, "resource.write offset", span)?;
let length = non_negative_usize(
&take(&bound, "length", builtin, span)?,
"resource.write length",
span,
)?;
let bytes = interpreter
.memory()
.read_bytes(memory, &offset_int, length, span)?;
match interpreter.resource_write(handle, &bytes, &module, span) {
Ok(count) => Ok(resource_ok(resource_write_payload(count))),
Err(error) => resource_recoverable(error),
}
}
"resource.close" => {
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
match interpreter.resource_close(handle, &module, span) {
Ok(()) => Ok(resource_ok(Value::None)),
Err(error) => resource_recoverable(error),
}
}
other => Err(RuntimeError::new(
ErrorCategory::KernelUnavailable,
span,
@@ -210,6 +496,228 @@ fn take(
})
}
/// Validate a page-count argument: an `Int` in the non-negative `u64` range.
fn page_count(value: Value, span: Option<Span>) -> Result<u64, RuntimeError> {
let value_type = value.type_name();
let Value::Int(count) = value else {
return Err(RuntimeError::type_error(
span,
format!("memory page count must be an Int, got {value_type}"),
));
};
num_traits::ToPrimitive::to_u64(&count).ok_or_else(|| {
RuntimeError::new(
ErrorCategory::MemoryFault,
span,
format!("memory page count {count} must be a non-negative integer"),
)
})
}
/// Convert a page count back to a language `Int`.
fn pages_value(pages: u64, span: Option<Span>) -> Result<Value, RuntimeError> {
i64::try_from(pages).map(Value::int).map_err(|_| {
RuntimeError::new(
ErrorCategory::MemoryFault,
span,
format!("page count {pages} does not fit an Int"),
)
})
}
/// Require a `Text` map key and return it.
fn map_key(value: Value, builtin: &str, span: Option<Span>) -> Result<String, RuntimeError> {
let value_type = value.type_name();
match value {
Value::Text(key) => Ok(key),
_ => Err(RuntimeError::type_error(
span,
format!("{builtin} key must be Text, got {value_type}"),
)),
}
}
/// Require a Text value.
fn expect_text_value(value: Value, span: Option<Span>) -> Result<String, RuntimeError> {
let value_type = value.type_name();
let Value::Text(text) = value else {
return Err(RuntimeError::type_error(
span,
format!("expected Text, got {value_type}"),
));
};
Ok(text)
}
/// Require a List of Text rights.
fn rights_list(value: Value, span: Option<Span>) -> Result<Vec<String>, RuntimeError> {
let value_type = value.type_name();
let Value::List(items) = value else {
return Err(RuntimeError::type_error(
span,
format!("resource rights must be a List, got {value_type}"),
));
};
let mut rights = Vec::with_capacity(items.len());
for item in items {
let item_type = item.type_name();
let Value::Text(right) = item else {
return Err(RuntimeError::type_error(
span,
format!("resource rights must contain Text, got {item_type}"),
));
};
rights.push(right);
}
Ok(rights)
}
/// Require an opaque resource handle.
fn resource_handle(value: Value, span: Option<Span>) -> Result<ResourceHandle, RuntimeError> {
let value_type = value.type_name();
let Value::Resource(handle) = value else {
return Err(RuntimeError::type_error(
span,
format!("expected a Resource handle, got {value_type}"),
));
};
Ok(handle)
}
/// Require an opaque Memory handle.
fn memory_handle(
value: Value,
builtin: &str,
span: Option<Span>,
) -> Result<marqdo_value::MemoryHandle, RuntimeError> {
let value_type = value.type_name();
let Value::Memory(handle) = value else {
return Err(RuntimeError::type_error(
span,
format!("{builtin} memory must be a Memory value, got {value_type}"),
));
};
Ok(handle)
}
/// Require an Int value.
fn int_value(
value: Value,
what: &str,
span: Option<Span>,
) -> Result<num_bigint::BigInt, RuntimeError> {
let value_type = value.type_name();
let Value::Int(value) = value else {
return Err(RuntimeError::type_error(
span,
format!("{what} must be an Int, got {value_type}"),
));
};
Ok(value)
}
/// Require a non-negative usize.
fn non_negative_usize(
value: &Value,
what: &str,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let value_type = value.type_name();
let Value::Int(value) = value else {
return Err(RuntimeError::type_error(
span,
format!("{what} must be an Int, got {value_type}"),
));
};
num_traits::ToPrimitive::to_usize(value).ok_or_else(|| {
RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("{what} {value} must be a non-negative integer in range"),
)
})
}
/// Require a strictly positive byte count.
fn positive_count(value: Value, what: &str, span: Option<Span>) -> Result<usize, RuntimeError> {
let count = non_negative_usize(&value, what, span)?;
if count == 0 {
return Err(RuntimeError::type_error(
span,
format!("{what} must be greater than zero"),
));
}
Ok(count)
}
fn resource_ok(value: Value) -> Value {
let mut map = BTreeMap::new();
map.insert("ok".to_string(), Value::Bool(true));
map.insert("value".to_string(), value);
Value::Map(map)
}
fn resource_err(code: &str, message: &str) -> Value {
let mut map = BTreeMap::new();
map.insert("ok".to_string(), Value::Bool(false));
map.insert("code".to_string(), Value::Text(code.to_string()));
map.insert("message".to_string(), Value::Text(message.to_string()));
Value::Map(map)
}
fn resource_read_payload(read: &ResourceRead) -> Value {
let mut map = BTreeMap::new();
map.insert("count".to_string(), Value::int(read.bytes.len() as i64));
map.insert("eof".to_string(), Value::Bool(read.eof));
Value::Map(map)
}
fn resource_write_payload(count: usize) -> Value {
let mut map = BTreeMap::new();
map.insert("count".to_string(), Value::int(count as i64));
Value::Map(map)
}
/// Convert a closed-handle failure into a recoverable Result; other errors are
/// fatal (ADR-0025).
fn resource_recoverable(error: RuntimeError) -> Result<Value, RuntimeError> {
if error.category == ErrorCategory::ResourceClosed {
return Ok(resource_err("resource.closed", &error.message));
}
Err(error)
}
/// Validate a non-negative table row/column coordinate.
fn table_index(
value: Value,
len: usize,
what: &str,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let value_type = value.type_name();
let Value::Int(index) = value else {
return Err(RuntimeError::type_error(
span,
format!("table {what} must be an Int, got {value_type}"),
));
};
let Some(position) = num_traits::ToPrimitive::to_usize(&index) else {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("table {what} {index} is out of bounds for length {len}"),
));
};
if position >= len {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("table {what} {position} is out of bounds for length {len}"),
));
}
Ok(position)
}
fn formula_text(value: Value, span: Option<Span>) -> Result<String, RuntimeError> {
match value {
Value::Formula(formula) => Ok(formula.latex),
+207 -8
View File
@@ -3,12 +3,13 @@
use std::collections::BTreeMap;
use marqdo_expression::{BinaryOp, Expr, RecordEntry};
use marqdo_host::{ExecutionContext, Host, HostOp};
use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, Program, Stmt};
use marqdo_host::{ExecutionContext, Host, HostOp, ResourceRead};
use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, PlaceTarget, Program, Stmt};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, RuntimeError, Value};
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value};
use crate::builtins_impl::FrameView;
use crate::memory::{MemoryLimits, MemoryStore};
use crate::value_ops;
@@ -68,6 +69,7 @@ pub struct Interpreter<'a> {
program: &'a Program,
host: &'a mut dyn Host,
depth: usize,
memory: MemoryStore,
}
/// Run a named function with already-evaluated arguments.
@@ -77,11 +79,31 @@ pub fn run(
module: &ModuleId,
function: &str,
args: Vec<EvaluatedArg>,
) -> Result<RunOutcome, RuntimeError> {
run_with_limits(
program,
host,
module,
function,
args,
MemoryLimits::default(),
)
}
/// Run with explicit addressable-memory limits (ADR-0022).
pub fn run_with_limits(
program: &Program,
host: &mut dyn Host,
module: &ModuleId,
function: &str,
args: Vec<EvaluatedArg>,
limits: MemoryLimits,
) -> Result<RunOutcome, RuntimeError> {
let mut interpreter = Interpreter {
program,
host,
depth: 0,
memory: MemoryStore::new(limits),
};
let value = interpreter.call_by_name(module, function, args, None)?;
Ok(RunOutcome { value })
@@ -300,10 +322,25 @@ impl<'a> Interpreter<'a> {
self.host.consume_fuel(1, Some(stmt.span()))?;
self.host.check_deadline(Some(stmt.span()))?;
match stmt {
Stmt::Assign { name, value, .. } => {
let value = self.eval_expr(value, frame)?;
frame.locals.insert(name.clone(), value);
}
Stmt::Assign { target, value, .. } => match target {
PlaceTarget::Local { name, .. } => {
let value = self.eval_expr(value, frame)?;
frame.locals.insert(name.clone(), value);
}
PlaceTarget::Index { base, index, span } => {
// Frozen order (ADR-0022): base, index, then RHS.
let base_value = self.eval_expr(base, frame)?;
let index_value = self.eval_expr(index, frame)?;
let byte_value = self.eval_expr(value, frame)?;
self.assign_memory_index(
base_value,
index_value,
byte_value,
Some(*span),
frame,
)?;
}
},
Stmt::Eval { expr, .. } => {
self.eval_expr(expr, frame)?;
}
@@ -417,6 +454,62 @@ impl<'a> Interpreter<'a> {
self.call_by_name_from(&module, &name, args, Some(span), Some(&caller))
}
/// Commit one byte into an addressable region after all operand checks.
///
/// The three operands are already evaluated, so a rejected value never
/// produces a partial write (ADR-0022 §3).
fn assign_memory_index(
&mut self,
base: Value,
index: Value,
byte: Value,
span: Option<Span>,
frame: &Frame,
) -> Result<(), RuntimeError> {
let base_type = base.type_name();
let Value::Memory(handle) = base else {
return Err(RuntimeError::new(
ErrorCategory::Type,
span,
format!(
"index assignment requires a Memory base in v1, got {base_type}; List, Map \
and Table values keep their existing value semantics"
),
));
};
let index_type = index.type_name();
let Value::Int(offset) = index else {
return Err(RuntimeError::type_error(
span,
format!("Memory index must be an Int, got {index_type}"),
));
};
let byte_type = byte.type_name();
let byte_repr = byte.display();
let Value::Int(byte) = byte else {
return Err(RuntimeError::type_error(
span,
format!("Memory byte value must be an Int in 0..255, got {byte_type}"),
));
};
let Some(byte) = num_traits::ToPrimitive::to_u8(&byte) else {
return Err(RuntimeError::type_error(
span,
format!("Memory byte value {byte_repr} is outside 0..255"),
));
};
self.memory.write(handle, &offset, byte, span)?;
let module = frame.function.module.canonical();
self.host.record_semantic(
"memory.write",
Some(&frame.function.to_string()),
Some(&module),
span,
Some("byte"),
);
Ok(())
}
fn eval_args(
&mut self,
args: &[marqdo_expression::Arg],
@@ -528,7 +621,22 @@ impl<'a> Interpreter<'a> {
Expr::Index { base, index, span } => {
let base_value = self.eval_expr(base, frame)?;
let index_value = self.eval_expr(index, frame)?;
index_value_of(base_value, index_value, Some(*span))
match base_value {
Value::Memory(handle) => {
let Value::Int(offset) = index_value else {
return Err(RuntimeError::type_error(
Some(*span),
format!(
"Memory index must be an Int, got {}",
index_value.type_name()
),
));
};
let byte = self.memory.read(handle, &offset, Some(*span))?;
Ok(Value::int(i64::from(byte)))
}
other => index_value_of(other, index_value, Some(*span)),
}
}
}
}
@@ -701,6 +809,97 @@ impl<'a> Interpreter<'a> {
self.program
}
/// Addressable-memory store for builtins and place writes.
pub(crate) fn memory(&self) -> &MemoryStore {
&self.memory
}
/// Mutable addressable-memory store for `memory.new`/`memory.grow`.
pub(crate) fn memory_mut(&mut self) -> &mut MemoryStore {
&mut self.memory
}
/// Record a semantic event for a builtin that is not a host operation.
pub(crate) fn record_memory_event(
&mut self,
kind: &str,
module: &ModuleId,
span: Option<Span>,
detail: &str,
) {
let module = module.canonical();
self.host
.record_semantic(kind, None, Some(&module), span, Some(detail));
}
/// Open a resource/stream through the host boundary.
pub(crate) fn resource_open(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
module: &ModuleId,
span: Option<Span>,
) -> Result<Value, RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_open(kind, descriptor, rights, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Read from a resource stream.
pub(crate) fn resource_read(
&mut self,
handle: ResourceHandle,
max_bytes: usize,
module: &ModuleId,
span: Option<Span>,
) -> Result<ResourceRead, RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_read(handle, max_bytes, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Write bytes to a resource stream.
pub(crate) fn resource_write(
&mut self,
handle: ResourceHandle,
bytes: &[u8],
module: &ModuleId,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_write(handle, bytes, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Close a resource stream.
pub(crate) fn resource_close(
&mut self,
handle: ResourceHandle,
module: &ModuleId,
span: Option<Span>,
) -> Result<(), RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_close(handle, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Call a builtin-read host operation on behalf of a frame's module.
pub(crate) fn host_invoke(
&mut self,
+5 -1
View File
@@ -8,8 +8,12 @@
mod builtins_impl;
mod interpreter;
mod memory;
mod value_ops;
pub use interpreter::{run, EvaluatedArg, Interpreter, RunOutcome};
pub use interpreter::{run, run_with_limits, EvaluatedArg, Interpreter, RunOutcome};
pub use memory::{
MemoryLimits, MemoryStore, DEFAULT_MAX_REGION_BYTES, DEFAULT_MAX_TOTAL_BYTES, PAGE_SIZE,
};
pub use marqdo_value::{RuntimeError, Value};
+600
View File
@@ -0,0 +1,600 @@
//! Runtime-owned addressable byte regions (RFC-CM-0001 / ADR-0022).
//!
//! A [`MemoryStore`] owns every region created during one run. Language values
//! carry only an opaque [`MemoryHandle`]; offsets are region-relative integers
//! and are never host pointers. Region identity is issued here and cannot be
//! forged from a document, a number or wire data.
//!
//! Semantics frozen by ADR-0022:
//!
//! * a region is a byte array with a 64 KiB logical page size; new bytes are
//! zero-filled;
//! * regions grow only, never shrink; a failed grow leaves the region and the
//! committed budget unchanged;
//! * copies of a `Memory` value share one region, while different regions
//! never alias;
//! * every access is bounds-checked against the region and the run budget, and
//! overflow in offset/page arithmetic is a structured error, never a panic;
//! * regions live for the whole run and are dropped with the store. There is
//! no user-visible `free(region)`.
use std::collections::BTreeMap;
use std::sync::atomic::{AtomicU64, Ordering};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, MemoryHandle, RuntimeError};
use num_bigint::BigInt;
use num_traits::ToPrimitive;
/// Logical page size for every region.
pub const PAGE_SIZE: usize = 64 * 1024;
/// Default maximum committed bytes across all regions in one run (64 MiB).
pub const DEFAULT_MAX_TOTAL_BYTES: usize = 64 * 1024 * 1024;
/// Default maximum committed bytes for one region (16 MiB).
pub const DEFAULT_MAX_REGION_BYTES: usize = 16 * 1024 * 1024;
static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1);
/// Bounds for addressable memory in one run.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct MemoryLimits {
/// Maximum bytes committed across all live regions.
pub max_total_bytes: usize,
/// Maximum bytes committed by a single region (`max_pages` is further
/// capped by this value).
pub max_region_bytes: usize,
}
impl Default for MemoryLimits {
fn default() -> Self {
Self {
max_total_bytes: DEFAULT_MAX_TOTAL_BYTES,
max_region_bytes: DEFAULT_MAX_REGION_BYTES,
}
}
}
impl MemoryLimits {
/// Maximum page count a single region may declare or reach.
pub fn max_region_pages(&self) -> u64 {
(self.max_region_bytes / PAGE_SIZE) as u64
}
}
struct Region {
bytes: Vec<u8>,
/// Declared maximum size for this region.
max_bytes: usize,
generation: u64,
}
/// All addressable regions created by one run.
pub struct MemoryStore {
run_id: u64,
next_region: u64,
regions: BTreeMap<u64, Region>,
committed_bytes: usize,
limits: MemoryLimits,
}
impl MemoryStore {
/// Create an empty store with a fresh, unguessable run identity.
pub fn new(limits: MemoryLimits) -> Self {
Self {
run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed),
next_region: 1,
regions: BTreeMap::new(),
committed_bytes: 0,
limits,
}
}
/// Limits in force for this store.
pub fn limits(&self) -> MemoryLimits {
self.limits
}
/// Bytes currently committed across all regions.
pub fn committed_bytes(&self) -> usize {
self.committed_bytes
}
/// Number of live regions.
pub fn region_count(&self) -> usize {
self.regions.len()
}
/// Create a zero-filled region.
///
/// `initial_pages` is committed immediately; `max_pages` is the declared
/// capacity that `grow` may not exceed. Both are checked against the
/// region and total budgets before anything is allocated, so a rejected
/// create leaves the store unchanged.
pub fn create(
&mut self,
initial_pages: u64,
max_pages: u64,
span: Option<Span>,
) -> Result<MemoryHandle, RuntimeError> {
if initial_pages > max_pages {
return Err(memory_error(
span,
format!(
"memory.new initial_pages ({initial_pages}) must not exceed max_pages \
({max_pages})"
),
));
}
if max_pages > self.limits.max_region_pages() {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory.new max_pages ({max_pages}) exceeds the per-region limit of {} \
pages ({PAGE_SIZE} bytes each)",
self.limits.max_region_pages()
),
));
}
let max_bytes = checked_bytes(max_pages, span, "region capacity")?;
if max_bytes > self.limits.max_region_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory.new max_pages ({max_pages}) exceeds the per-region byte budget of {}",
self.limits.max_region_bytes
),
));
}
let initial_bytes = checked_bytes(initial_pages, span, "initial region size")?;
let committed = self
.committed_bytes
.checked_add(initial_bytes)
.ok_or_else(|| budget_error(span, "memory budget overflow"))?;
if committed > self.limits.max_total_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory budget exceeded: committing {initial_bytes} bytes would reach \
{committed} of {}",
self.limits.max_total_bytes
),
));
}
let mut bytes = Vec::new();
bytes.try_reserve_exact(initial_bytes).map_err(|_| {
RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("cannot allocate {initial_bytes} bytes of addressable memory"),
)
})?;
bytes.resize(initial_bytes, 0);
let region_id = self.next_region;
self.next_region = self
.next_region
.checked_add(1)
.ok_or_else(|| memory_error(span, "memory region identity space exhausted"))?;
self.regions.insert(
region_id,
Region {
bytes,
max_bytes,
generation: 1,
},
);
self.committed_bytes = committed;
Ok(MemoryHandle::from_runtime(self.run_id, region_id, 1))
}
/// Current length of a region in pages.
pub fn size(&self, handle: MemoryHandle, span: Option<Span>) -> Result<u64, RuntimeError> {
let region = self.region(handle, span)?;
Ok((region.bytes.len() / PAGE_SIZE) as u64)
}
/// Grow a region by whole pages and return its previous page count.
///
/// A failed grow is atomic: the region keeps its old length and bytes and
/// the committed budget is unchanged.
pub fn grow(
&mut self,
handle: MemoryHandle,
pages: u64,
span: Option<Span>,
) -> Result<u64, RuntimeError> {
if pages == 0 {
return self.size(handle, span);
}
let region = self.region(handle, span)?;
let old_bytes = region.bytes.len();
let old_pages = (old_bytes / PAGE_SIZE) as u64;
let additional = checked_bytes(pages, span, "memory.grow pages")?;
let new_bytes = old_bytes
.checked_add(additional)
.ok_or_else(|| memory_error(span, "memory.grow size overflows the address space"))?;
if new_bytes > region.max_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory.grow would grow the region to {new_bytes} bytes, beyond its \
declared maximum of {} bytes",
region.max_bytes
),
));
}
let committed = self
.committed_bytes
.checked_add(additional)
.ok_or_else(|| budget_error(span, "memory budget overflow"))?;
if committed > self.limits.max_total_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"memory budget exceeded: growing by {additional} bytes would reach \
{committed} of {}",
self.limits.max_total_bytes
),
));
}
let region = self
.regions
.get_mut(&handle.region_id())
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
region.bytes.try_reserve(additional).map_err(|_| {
RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("cannot grow addressable memory by {additional} bytes"),
)
})?;
region.bytes.resize(new_bytes, 0);
self.committed_bytes = committed;
Ok(old_pages)
}
/// Total byte length of a region.
pub fn byte_len(
&self,
handle: MemoryHandle,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
Ok(self.region(handle, span)?.bytes.len())
}
/// Read a contiguous byte range. Used by resource I/O adapters.
pub fn read_bytes(
&self,
handle: MemoryHandle,
offset: &BigInt,
length: usize,
span: Option<Span>,
) -> Result<Vec<u8>, RuntimeError> {
let region = self.region(handle, span)?;
let start = range_offset(offset, region.bytes.len(), length, span)?;
Ok(region.bytes[start..start + length].to_vec())
}
/// Write a contiguous byte range atomically after a full bounds check.
pub fn write_bytes(
&mut self,
handle: MemoryHandle,
offset: &BigInt,
bytes: &[u8],
span: Option<Span>,
) -> Result<(), RuntimeError> {
let len = self.region(handle, span)?.bytes.len();
let start = range_offset(offset, len, bytes.len(), span)?;
let region = self
.regions
.get_mut(&handle.region_id())
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
region.bytes[start..start + bytes.len()].copy_from_slice(bytes);
Ok(())
}
/// Read one byte. Offsets are non-negative and region-relative.
pub fn read(
&self,
handle: MemoryHandle,
offset: &BigInt,
span: Option<Span>,
) -> Result<u8, RuntimeError> {
let region = self.region(handle, span)?;
let position = checked_offset(offset, region.bytes.len(), span)?;
Ok(region.bytes[position])
}
/// Write one byte. The caller must have validated the byte range.
pub fn write(
&mut self,
handle: MemoryHandle,
offset: &BigInt,
byte: u8,
span: Option<Span>,
) -> Result<(), RuntimeError> {
let len = self.region(handle, span)?.bytes.len();
let position = checked_offset(offset, len, span)?;
let region = self
.regions
.get_mut(&handle.region_id())
.ok_or_else(|| memory_error(span, "unknown Memory region handle"))?;
region.bytes[position] = byte;
Ok(())
}
fn region(&self, handle: MemoryHandle, span: Option<Span>) -> Result<&Region, RuntimeError> {
if handle.run_id() != self.run_id {
return Err(memory_error(
span,
"Memory handle belongs to a different execution run",
));
}
let Some(region) = self.regions.get(&handle.region_id()) else {
return Err(memory_error(span, "unknown Memory region handle"));
};
if region.generation != handle.generation() {
return Err(memory_error(span, "stale Memory region handle"));
}
Ok(region)
}
}
fn checked_bytes(pages: u64, span: Option<Span>, what: &str) -> Result<usize, RuntimeError> {
let bytes = pages
.checked_mul(PAGE_SIZE as u64)
.ok_or_else(|| memory_error(span, format!("{what} overflows the address space")))?;
usize::try_from(bytes).map_err(|_| {
RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("{what} ({pages} pages) does not fit this platform's address space"),
)
})
}
/// Validate an offset plus a byte count against the region length.
fn range_offset(
offset: &BigInt,
len: usize,
length: usize,
span: Option<Span>,
) -> Result<usize, RuntimeError> {
if offset.sign() == num_bigint::Sign::Minus {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
"Memory offset must be a non-negative integer",
));
}
let Some(start) = offset.to_usize() else {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("Memory offset {offset} is out of bounds for region length {len}"),
));
};
let end = start.checked_add(length).ok_or_else(|| {
memory_error(
span,
"Memory offset plus length overflows the address space",
)
})?;
if end > len {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("Memory range {start}..{end} is out of bounds for region length {len}"),
));
}
Ok(start)
}
fn checked_offset(offset: &BigInt, len: usize, span: Option<Span>) -> Result<usize, RuntimeError> {
if offset.sign() == num_bigint::Sign::Minus {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
"Memory offset must be a non-negative integer",
));
}
let Some(position) = offset.to_usize() else {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("Memory offset {offset} is out of bounds for region length {len}"),
));
};
if position >= len {
return Err(RuntimeError::new(
ErrorCategory::IndexOutOfBounds,
span,
format!("Memory offset {offset} is out of bounds for region length {len}"),
));
}
Ok(position)
}
fn memory_error(span: Option<Span>, message: impl Into<String>) -> RuntimeError {
RuntimeError::new(ErrorCategory::MemoryFault, span, message)
}
fn budget_error(span: Option<Span>, message: impl Into<String>) -> RuntimeError {
RuntimeError::new(ErrorCategory::ResourceExhausted, span, message)
}
#[cfg(test)]
mod tests {
use super::*;
fn int(value: impl Into<BigInt>) -> BigInt {
value.into()
}
fn store() -> MemoryStore {
MemoryStore::new(MemoryLimits::default())
}
#[test]
fn regions_are_isolated_and_handles_share_identity() {
let mut store = store();
let first = store.create(1, 2, None).unwrap();
let second = store.create(1, 2, None).unwrap();
let alias = first;
store.write(first, &int(0), 7, None).unwrap();
store.write(second, &int(0), 9, None).unwrap();
assert_eq!(store.read(first, &int(0), None).unwrap(), 7);
assert_eq!(store.read(alias, &int(0), None).unwrap(), 7);
assert_eq!(store.read(second, &int(0), None).unwrap(), 9);
assert_eq!(first, alias);
assert_ne!(first, second);
}
#[test]
fn create_is_zero_filled_and_reports_pages() {
let mut store = store();
let memory = store.create(1, 4, None).unwrap();
assert_eq!(store.size(memory, None).unwrap(), 1);
assert_eq!(PAGE_SIZE, 64 * 1024);
for offset in [0usize, 1, PAGE_SIZE - 1] {
assert_eq!(store.read(memory, &int(offset as u64), None).unwrap(), 0);
}
// Zero-page creation is legal but every access is out of bounds.
let empty = store.create(0, 1, None).unwrap();
assert_eq!(store.size(empty, None).unwrap(), 0);
assert_eq!(
store.read(empty, &int(0), None).unwrap_err().category,
ErrorCategory::IndexOutOfBounds
);
}
#[test]
fn grow_keeps_old_bytes_and_zero_fills_new_pages() {
let mut store = store();
let memory = store.create(1, 2, None).unwrap();
store.write(memory, &int(3), 200, None).unwrap();
assert_eq!(store.grow(memory, 1, None).unwrap(), 1);
assert_eq!(store.size(memory, None).unwrap(), 2);
assert_eq!(store.read(memory, &int(3), None).unwrap(), 200);
assert_eq!(store.read(memory, &int(PAGE_SIZE as u64), None).unwrap(), 0);
store
.write(memory, &int(PAGE_SIZE as u64 + 5), 42, None)
.unwrap();
assert_eq!(
store
.read(memory, &int(PAGE_SIZE as u64 + 5), None)
.unwrap(),
42
);
// Zero-page growth is a no-op returning the old count.
assert_eq!(store.grow(memory, 0, None).unwrap(), 2);
}
#[test]
fn failed_grow_does_not_change_region_or_budget() {
let mut store = MemoryStore::new(MemoryLimits {
max_total_bytes: PAGE_SIZE * 2,
max_region_bytes: PAGE_SIZE * 4,
});
let memory = store.create(1, 2, None).unwrap();
store.write(memory, &int(0), 1, None).unwrap();
let before = store.committed_bytes();
let error = store.grow(memory, 2, None).unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceExhausted);
assert_eq!(store.size(memory, None).unwrap(), 1);
assert_eq!(store.read(memory, &int(0), None).unwrap(), 1);
assert_eq!(store.committed_bytes(), before);
}
#[test]
fn invalid_offsets_are_structured_errors() {
let mut store = store();
let memory = store.create(1, 1, None).unwrap();
for offset in [int(-1), int(PAGE_SIZE as u64)] {
let error = store.read(memory, &offset, None).unwrap_err();
assert_eq!(error.category, ErrorCategory::IndexOutOfBounds);
}
let huge: BigInt = "999999999999999999999999999999999999999".parse().unwrap();
assert_eq!(
store.read(memory, &huge, None).unwrap_err().category,
ErrorCategory::IndexOutOfBounds
);
assert_eq!(
store.write(memory, &huge, 1, None).unwrap_err().category,
ErrorCategory::IndexOutOfBounds
);
assert_eq!(
store
.write(memory, &int(PAGE_SIZE as u64), 1, None)
.unwrap_err()
.category,
ErrorCategory::IndexOutOfBounds
);
}
#[test]
fn create_validates_page_counts_and_budget() {
let mut store = MemoryStore::new(MemoryLimits {
max_total_bytes: PAGE_SIZE * 3,
max_region_bytes: PAGE_SIZE * 2,
});
assert_eq!(
store.create(2, 1, None).unwrap_err().category,
ErrorCategory::MemoryFault
);
assert_eq!(
store.create(1, 3, None).unwrap_err().category,
ErrorCategory::ResourceExhausted
);
let _first = store.create(2, 2, None).unwrap();
let _second = store.create(1, 1, None).unwrap();
// Total budget is now exactly three pages.
assert_eq!(
store.create(1, 1, None).unwrap_err().category,
ErrorCategory::ResourceExhausted
);
assert_eq!(store.committed_bytes(), PAGE_SIZE * 3);
}
#[test]
fn forged_and_stale_handles_are_rejected() {
let mut store = store();
let memory = store.create(1, 1, None).unwrap();
let forged = MemoryHandle::from_runtime(memory.run_id() + 1, memory.region_id(), 1);
assert_eq!(
store.read(forged, &int(0), None).unwrap_err().category,
ErrorCategory::MemoryFault
);
let stale = MemoryHandle::from_runtime(memory.run_id(), memory.region_id(), 2);
assert_eq!(
store.read(stale, &int(0), None).unwrap_err().category,
ErrorCategory::MemoryFault
);
let unknown = MemoryHandle::from_runtime(memory.run_id(), 999, 1);
assert_eq!(
store.write(unknown, &int(0), 1, None).unwrap_err().category,
ErrorCategory::MemoryFault
);
}
#[test]
fn page_arithmetic_never_overflows() {
let mut store = MemoryStore::new(MemoryLimits {
max_total_bytes: usize::MAX,
max_region_bytes: usize::MAX,
});
assert_eq!(
store.create(u64::MAX, u64::MAX, None).unwrap_err().category,
ErrorCategory::ResourceExhausted
);
}
}
+26
View File
@@ -107,6 +107,32 @@ fn compare(
}
}))
}
(Value::Resource(a), Value::Resource(b)) => {
return Ok(Value::Bool(match op {
BinaryOp::Eq => a == b,
BinaryOp::Ne => a != b,
_ => {
return Err(type_error(
span,
"Resource ordering comparisons are not supported".to_string(),
))
}
}));
}
(Value::Memory(a), Value::Memory(b)) => {
// Identity, not byte content: two handles for the same region are
// equal, regions with identical bytes are not (ADR-0022 §1).
return Ok(Value::Bool(match op {
BinaryOp::Eq => a == b,
BinaryOp::Ne => a != b,
_ => {
return Err(type_error(
span,
"Memory ordering comparisons are not supported".to_string(),
))
}
}));
}
(a, b) => {
if op == BinaryOp::Eq {
return Ok(Value::Bool(false));
+44 -1
View File
@@ -2,7 +2,7 @@
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_ir::{ModuleId, Program};
use marqdo_runtime::{run, EvaluatedArg, RuntimeError, Value};
use marqdo_runtime::{run, run_with_limits, EvaluatedArg, MemoryLimits, RuntimeError, Value};
use marqdo_sandbox::SandboxMode;
use marqdo_span::Span;
@@ -170,3 +170,46 @@ fn ambiguous_ref_resource_is_an_error() {
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "ref.ambiguous");
}
#[test]
fn memory_region_is_shared_by_handle_identity() {
let source = "## f\n\n**m = memory.new(1, 2)**\n\n**n = m**\n\n**`m[3] = 7`**\n\n*[n[3], m == n, memory.size(n)]*\n";
let value = run_named(source, "f", &[]).unwrap();
assert_eq!(
value,
Value::List(vec![Value::int(7), Value::Bool(true), Value::int(1)])
);
}
#[test]
fn memory_handle_cannot_reach_the_host_boundary() {
let source = "## f\n\n**m = memory.new(1, 1)**\n\n> 打印(m)\n\n*0*\n";
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "runtime.type");
assert!(
error
.message
.contains("Memory values cannot cross the host boundary"),
"unexpected message: {}",
error.message
);
}
#[test]
fn memory_budget_is_enforced_without_changing_the_region() {
let program = build("## f\n\n**m = memory.new(1, 1)**\n\n*[m[0], memory.grow(m, 1)]*\n");
let mut host = host();
let error = run_with_limits(
&program,
&mut host,
&ModuleId::from("index.mq.md"),
"f",
Vec::new(),
MemoryLimits {
max_total_bytes: 64 * 1024,
max_region_bytes: 64 * 1024 * 2,
},
)
.unwrap_err();
assert_eq!(error.code(), "resource.exhausted");
}
+16 -1
View File
@@ -23,8 +23,19 @@ pub enum ErrorCategory {
NotCallable,
/// A map key or record field does not exist.
UnknownField,
/// A list/map index is out of bounds.
/// A list/map/memory index is out of bounds.
IndexOutOfBounds,
/// An addressable-memory operation failed for a reason other than a plain
/// out-of-bounds read: invalid page counts, an unknown/stale handle, or an
/// arithmetic overflow in an address.
MemoryFault,
/// A resource handle is forged, cross-run or stale.
ResourceFault,
/// A resource operation used an already-closed handle (recoverable as a
/// `Result` by the builtin).
ResourceClosed,
/// The requested resource kind/descriptor/rights are not implemented.
ResourceUnsupported,
/// A resource budget was exhausted (fuel, stack, output size).
ResourceExhausted,
/// A capability check denied the operation.
@@ -60,6 +71,10 @@ impl ErrorCategory {
ErrorCategory::NotCallable => "call.not_callable",
ErrorCategory::UnknownField => "runtime.unknown_field",
ErrorCategory::IndexOutOfBounds => codes::RUNTIME_INDEX,
ErrorCategory::MemoryFault => "runtime.memory",
ErrorCategory::ResourceFault => "runtime.resource",
ErrorCategory::ResourceClosed => "resource.closed",
ErrorCategory::ResourceUnsupported => "resource.unsupported",
ErrorCategory::ResourceExhausted => codes::RESOURCE_EXHAUSTED,
ErrorCategory::PermissionDenied => codes::PERMISSION_DENIED,
ErrorCategory::KernelUnavailable => codes::KERNEL_UNAVAILABLE,
+1 -1
View File
@@ -15,6 +15,6 @@ pub use error::{CallFrame, ErrorCategory, RuntimeError};
pub use marqdo_span::{LineCol, Span};
pub use value::{
bigint_to_exact_f64, float_to_bigint, format_float, CodeValue, FormulaValue, FunctionRef,
PluginHandle, Table, Value,
MemoryHandle, PluginHandle, ResourceHandle, Table, Value,
};
pub use wire::{from_wire, to_wire, WireError, WIRE_VERSION};
+139
View File
@@ -75,6 +75,88 @@ pub struct PluginHandle {
pub id: u64,
}
/// Opaque identity of one runtime-managed byte region (ADR-0022).
///
/// A handle is issued only by the runtime's `MemoryStore`. Copying a `Memory`
/// value or passing it as an argument preserves this identity, so two names
/// can observe each other's writes. The fields are deliberately not part of
/// any wire format: [`crate::wire`] rejects the `memory` tag on decode, and no
/// public constructor accepts untrusted data. There is no `usize` address in
/// the language value; offsets are region-relative integers.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct MemoryHandle {
run: u64,
region: u64,
generation: u64,
}
impl MemoryHandle {
/// Create a handle. Only the runtime store may call this; users cannot
/// construct one from a number, a document or wire data.
#[doc(hidden)]
pub const fn from_runtime(run: u64, region: u64, generation: u64) -> Self {
Self {
run,
region,
generation,
}
}
/// Run identity this region belongs to.
pub const fn run_id(&self) -> u64 {
self.run
}
/// Region identity within the run.
pub const fn region_id(&self) -> u64 {
self.region
}
/// Generation of the region slot (guards a future reuse policy).
pub const fn generation(&self) -> u64 {
self.generation
}
}
/// Opaque host-issued handle to an external resource/stream (ADR-0025).
///
/// Fields are private and the constructor is host-only. A handle is never
/// built from wire data or a user integer; it is a different domain from
/// `Memory` and cannot be indexed.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct ResourceHandle {
run: u64,
resource: u64,
generation: u64,
}
impl ResourceHandle {
/// Host-only constructor.
#[doc(hidden)]
pub const fn from_host(run: u64, resource: u64, generation: u64) -> Self {
Self {
run,
resource,
generation,
}
}
/// Host run identity.
pub const fn run_id(&self) -> u64 {
self.run
}
/// Resource slot identity.
pub const fn resource_id(&self) -> u64 {
self.resource
}
/// Slot generation.
pub const fn generation(&self) -> u64 {
self.generation
}
}
/// A Marqdo runtime value.
#[derive(Debug, Clone, PartialEq)]
pub enum Value {
@@ -104,6 +186,11 @@ pub enum Value {
FunctionRef(FunctionRef),
/// Host-owned plugin handle.
PluginHandle(PluginHandle),
/// Runtime-owned addressable byte region (RFC-CM-0001). Copies share the
/// same region identity; they are not byte snapshots.
Memory(MemoryHandle),
/// Host-owned external resource/stream handle (ADR-0025).
Resource(ResourceHandle),
}
impl Value {
@@ -123,6 +210,8 @@ impl Value {
Value::Code(_) => "Code",
Value::FunctionRef(_) => "FunctionRef",
Value::PluginHandle(_) => "PluginHandle",
Value::Memory(_) => "Memory",
Value::Resource(_) => "Resource",
}
}
@@ -200,6 +289,44 @@ impl Value {
Value::Code(c) => format!("Code({:?}, {} bytes)", c.language, c.source.len()),
Value::FunctionRef(r) => format!("FunctionRef({}::{})", r.module, r.name),
Value::PluginHandle(h) => format!("PluginHandle({}::{})", h.plugin, h.export),
// Never render the runtime handle: internal region identity is not
// observable program data (RFC-CM-0001 §8).
Value::Memory(_) => "Memory".to_string(),
// Resource identity is host-private and never program data.
Value::Resource(_) => "Resource".to_string(),
}
}
/// Whether this value contains a live `Memory` handle anywhere.
///
/// Used at the host/plugin/foreign boundary, which must reject handles
/// instead of serializing them.
pub fn contains_memory(&self) -> bool {
match self {
Value::Memory(_) => true,
Value::List(items) => items.iter().any(Value::contains_memory),
Value::Map(entries) => entries.values().any(Value::contains_memory),
Value::Table(table) => table
.rows
.iter()
.flat_map(|row| row.iter())
.any(Value::contains_memory),
_ => false,
}
}
/// Whether this value contains a live `Resource` handle anywhere.
pub fn contains_resource(&self) -> bool {
match self {
Value::Resource(_) => true,
Value::List(items) => items.iter().any(Value::contains_resource),
Value::Map(entries) => entries.values().any(Value::contains_resource),
Value::Table(table) => table
.rows
.iter()
.flat_map(|row| row.iter())
.any(Value::contains_resource),
_ => false,
}
}
@@ -325,6 +452,18 @@ mod tests {
);
}
#[test]
fn memory_display_never_leaks_the_runtime_handle() {
let memory = Value::Memory(MemoryHandle::from_runtime(41, 7, 1));
assert_eq!(memory.type_name(), "Memory");
assert_eq!(memory.display(), "Memory");
assert_eq!(memory.repr(), "Memory");
assert!(!memory.display().contains("41"));
assert!(!memory.repr().contains('7'));
assert!(memory.contains_memory());
assert!(Value::List(vec![memory]).contains_memory());
}
#[test]
fn big_integers_are_arbitrary_precision() {
let big: BigInt = "123456789012345678901234567890".parse().unwrap();
+40
View File
@@ -145,6 +145,12 @@ fn encode(value: &Value) -> Json {
"export": handle.export,
"id": handle.id,
}),
// Active memory handles never cross the wire (RFC-CM-0001 §8).
// The tag is emitted so an accidental transfer is visible instead of
// silently becoming a number; decoding it is a hard error.
Value::Memory(_) => json!({"type": "memory"}),
// Resource handles are host-private; same non-transferable rule.
Value::Resource(_) => json!({"type": "resource"}),
}
}
@@ -321,6 +327,12 @@ fn decode(json: &Json, depth: usize, limits: &WireLimits) -> Result<Value, WireE
.as_u64()
.ok_or_else(|| WireError::Invalid("plugin_handle.id must be an integer".into()))?,
})),
"memory" => Err(WireError::Invalid(
"Memory handles cannot cross the wire; copy bytes explicitly".into(),
)),
"resource" => Err(WireError::Invalid(
"Resource handles cannot cross the wire".into(),
)),
other => Err(WireError::UnknownType(other.to_string())),
}
}
@@ -393,6 +405,34 @@ mod tests {
assert!(matches!(decoded, Value::Float(f) if f == f64::NEG_INFINITY));
}
#[test]
fn memory_handles_are_not_transferable() {
let value = Value::Memory(crate::MemoryHandle::from_runtime(1, 2, 1));
let wire = to_wire(&value);
assert_eq!(wire, json!({"type": "memory"}));
assert_eq!(
from_wire(&wire),
Err(WireError::Invalid(
"Memory handles cannot cross the wire; copy bytes explicitly".into()
))
);
// A hand-written fake handle is rejected by the same tag rule.
assert!(from_wire(&json!({"type": "memory", "region": 2})).is_err());
}
#[test]
fn resource_handles_are_not_transferable() {
let value = Value::Resource(crate::ResourceHandle::from_host(1, 2, 1));
let wire = to_wire(&value);
assert_eq!(wire, json!({"type": "resource"}));
assert_eq!(
from_wire(&wire),
Err(WireError::Invalid(
"Resource handles cannot cross the wire".into()
))
);
}
#[test]
fn untrusted_wire_is_validated() {
assert_eq!(
File diff suppressed because it is too large. Load diff
+498
View File
@@ -0,0 +1,498 @@
# RFC-CM-0001:Marqdo 自主计算机器、可寻址内存与系统交互边界
> **状态**:设计提案(Draft,未实施,未批准)
> **日期**:2026-11-20
> **建议归档路径**:`docs/architecture/RFC-CM-0001-compute-machine.md`
> **验证基线**:`cflmy/marqdo-cli`,Git 提交 `a113d894962d8c00eb02efb42e0e73f8b084a1fa`(检索于 2026-11-20)
> **兼容性**:涉及 `Value`、表达式赋值、运行时、主机能力和线协议的变更,必须通过 ADR、规范版本与相应测试;本 RFC 不声明任何新接口目前可用。
---
## 摘要
Marqdo 的优势不应只是用 Markdown 排版代码,而应是:在不更换 Markdown 语法角色的前提下,以**自身定义的计算语义**构造算法、数据结构、运行时工具和知识制品。现有解释器已提供整数、浮点、文本、列表、映射、控制流和函数调用,但尚未提供**由 Marqdo 程序控制的可寻址字节存储**;其赋值左侧仅支持标识符,系统交互集中在一组固定 `HostOp` 与外部插件能力中。这妨碍了用 `.mq.md` 自己编写内存分配器、二进制协议、共享节点结构、虚拟机器等低层算法。
**提议:以“托管值 + 多个可增长的线性内存区域 + 不透明系统资源”为计算机器的三个基础领域。**
1. 保留现有 `Value` 计算体系,不强迫一切数据都变成字节地址。
2. 引入一个拥有独立身份、按字节索引、可增长、具有边界的 `Memory` 值;每个区域内部线性寻址,物理实现可以是 Vec、分页或其他表示。
3. 扩展现有“位置赋值”:允许对适当的索引目标写入,如 `m[8] = 42`,而不是另创 Markdown 语法或几十个关键字。
4. 为创建、测量和增长内存定义**极少量计算原语**;让 `malloc/free`、内存池、链表、编码、缓冲区等**首先在 `.mq.md` 标准库**中实现。
5. 将终端、文件、网络、进程、设备定义为**外部资源**,经稳定、可授权、可替换的底层适配接口交互;Marqdo 自身可以实现高级系统库与协议,但系统调用或硬件访问的**最后一步不能凭空消失**。
6. CPU 寄存器由解释器/编译后端管理;裸机器码、GPU/QPU、共享内存原子操作暂不进入本 RFC 的基础语义。
这是一种**可编程的抽象机器**,而不是“在每种新需求前追加 Rust Host 函数”的框架,也不是把 x86/RISC-V 指令直接变成 Markdown 语法。
## 1. 设计动机:究竟希望解决什么
### 1.1 真实诉求
Marqdo 必须能在不借助 Python、JavaScript 或宿主语言提供专门算法函数的前提下,**通过自己的控制流和基础存储机制**构造原先不存在的高级能力。例如:实现内存分配器、二叉树、哈希表、字节码解释器、文件格式解析器、数值算法。对于 OS 和硬件,要求的是**高级功能主要由 Marqdo 实现,底层权限与系统调用接口保持稳定**,而不是“所有跨越内核边界的指令都神奇地由 Markdown 自己完成”。
### 1.2 非目标
- 不追求复刻 CPU 的真实寄存器、指令流水线、页表与异常处理体系。
- 不把所有 Markdown `[]()` 链接重新定义为动态指针。链接继续表示文档/符号寻址,不伪装运行时对象地址。
- 不把 GFM 表格当成实际运行内存。表格可以用于数据/状态的可读表达与快照,但不是存储管理器。
- 不要求标准库全部使用 `.mq.md`;原生加速可以有,但**不是实现一般算法的必要条件**。
- 不在第一期承诺用户代码内存安全、数据竞争自由、GPU 加速或跨平台无差异系统调用。
- 不支持在无权限环境中直接执行任意机器码或任意 syscall。
### 1.3 设计原则
**P1 语义最小而充分**:首先增加无法由现有表达式自然实现的基础机制,不先增加高级 API 或新关键词。
**P2 高级能力在语言中生长**:新增算法/数据结构不修改解释器、不修改 `HostOp`、不需要重新编译 CLI。
**P3 读写位置统一**:索引读取和索引赋值遵循同一类“位置”语义;`Memory` 的读取/写入是基础例子,普通列表的值语义不得无声改变。
**P4 内外边界诚实**:内存操作是语言计算;操作系统 I/O 依赖权限、平台接口和底层实现。
**P5 可读性**:普通 Markdown 文档仍然可读;`##`/`**`/`*`/列表/blockquote 的现有角色不因本 RFC 改变。
**P6 安全与可验证性**:将可验证的保证限定在区域边界和权限边界;不对自定义分配器的块级错误作虚假保证。
**P7 零隐式权限**:能够导入模块、持有字符串或解析文档,不代表拥有文件、网络或设备访问权。
## 2. 现有仓库事实及需要保留的设计
依据提交 `a113d894...` 的源代码和规范:
| 项目 | 现状 | 对本 RFC 的约束 |
|---|---|---|
| `marqdo-value` | `Unit/None/Bool/Int/Float/Text/List/Map/Table/Formula/Code/FunctionRef/PluginHandle`,没有普通可寻址 `Memory` 类型 | 应新增专门的存储引用值,而非把现有 List 改成默认共享对象 |
| `marqdo-expression` | 索引 `a[i]`/成员 `a.x` 可读取;`assignment = identifier = expression` | 最小语法修改是赋值目标扩展,不是新 Markdown 节点 |
| `marqdo-runtime` | `Frame.locals: BTreeMap<String, Value>`;`Assign` 以 `locals.insert` 重绑定 | 局部变量重新赋值应保持既有含义;共享可变性仅属于显式 `Memory` |
| `marqdo-frontend` | Markdown AST 是语法角色唯一权威;GFM Table 惰性 | 不更改表格、链接的执行语义;表达式子语言可增补语义 |
| `marqdo-ir` | 内置函数元数据固定且需更新规范 | 新增基础内存原语需要小型且审慎的语言变更;高层库不应无限扩展 Builtin 表 |
| `marqdo-host` | 固定 `HostOp`,包括 Print、Math、Foreign、FsRead/FsWrite、PluginInvoke | 不应将每一个新的标准库函数继续塞入固定 HostOp |
| 插件 ABI v1 | C ABI、worker、能力请求已有雏形 | 可复用函数注册与 Worker,不能错误地把 C ABI 当沙箱 |
| `pure/project/strict` | `pure` 禁止外部 I/O grant;`strict` Linux 探测成功后只支持 L1 范围,其他平台 fail closed | 新系统资源功能首先在项目模式开发;strict 须经新的隔离验证,不得默许开启 |
| 包管理 | path/Git 依赖与 lock v3 已实现;版本范围部分尚未强制 | 独立标准库可利用现有依赖体系,但兼容版本需要明确验证 |
**保留的价值**:现有整数运算、函数调用、条件和循环不必推翻;现有 AST 映射、资源边界和可追踪执行事件是 Marqdo 与脚本转译器的根本区别之一。
源码(固定版本):
- [`spec/expression.ebnf`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/expression.ebnf)
- [`spec/mapping.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/mapping.md)
- [`spec/evaluation.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/evaluation.md)
- [`crates/marqdo-value/src/value.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-value/src/value.rs)
- [`crates/marqdo-runtime/src/interpreter.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-runtime/src/interpreter.rs)
- [`spec/security.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)
## 3. 总体架构与唯一的语义边界
```text
.mq.md 文档 GFM Table/Link/标题/正文仍按 Markdown 正常渲染
│
▼
Canonical Markdown AST ──▶ 已有 Frontend ──▶ Marqdo IR
│
┌───────────────────┤
▼ ▼
值与控制 存储与位置
Int/List/Map MemoryRegion
函数/赋值/循环 字节读写/增长
└─────────┬─────────┘
│
Marqdo 计算核心
│
需要访问外部世界时才穿越边界
▼
Resource capability gate
│ │ │
console file network/device
│ │ │
Platform Adapter / OS ABI
```
逻辑上的执行状态可以表述为:
`State = (Control, Frames, ManagedValues, RegionStore, ResourceTable, Budget)`。
- `Control`:当前 IR 节点和执行顺序;保留已有控制流。
- `Frames/ManagedValues`:现有 `Value`、局部变量和调用语义。
- `RegionStore`:一组可增长字节区域,每个区域按 `(region identity, offset)` 寻址。
- `ResourceTable`:操作系统资源的不透明句柄与授权信息,**不得与内存地址混用**。
- `Budget`:fuel、调用深度、内存限额、I/O 限额、截止时间等。
此设计**不规定**必须先创建自定义字节码 VM。现有树遍历解释器可直接执行新内存语义;将来引入字节码、JIT、LLVM/Cranelift 等属于可选执行方式,必须保持可观察语义。
## 4. 三种基础领域:值、内存、资源
### 4.1 托管值:维持已有语义
`Int/Text/List/Map/Table/Formula/Code` 等继续是运行时管理的值。普通 `b = a` 不得偷偷将普通列表变成共享可变指针。现有对象可以由 Rust 分配内存——**实现语言不等于语言语义**。这个领域负责易用性、文档处理、不可变数据等。
### 4.2 Memory:拥有身份的字节区域
新增 `Value::Memory`(名称暂定),含不可伪造的运行时区域引用。逻辑区域内部是一串字节,下标为非负整数;不同区域即使下标相同也不是同一个位置。
- **区域身份**:由本次运行中的内存实例签发;`Memory` 复制/传参时保留指向相同区域的身份(共享,不复制所有字节)。
- **地址**:`(MemoryHandle, offset)`;`offset` 只在所属区域内有意义;整数本身不是获得其他区域权限的凭据。
- **内容**:每个元素是 `0..255` 的一个字节;初始字节为 0。
- **大小**:区域长度是页大小的整数倍;建议初期采用 `64 KiB` 页,以简化语义与工具链对照,不要求底层真的按页分配。
- **增长**:只增长,不缩小;旧地址偏移不变;新加入的字节全部为零;不得突破区域/运行全局限额。
- **读取**:`m[i] -> Int(0..255)`;越界、非法整数或已失效区域均是类型/存储错误。
- **写入**:`m[i] = b`,`b` 必须是 0..255 的整数,不隐式截断;越界拒绝。
- **相等性**:两个 Memory 引用的相等比较区域身份;不比较全部字节;Memory 不定义大小排序。
- **序列化**:不允许将活动 `Memory` 句柄伪装成数值或跨进程永久有效地址。允许显式**复制字节内容**为快照,需限额;跨进程传输需要显式复制或另立经过授权的共享内存协议。
- **生命周期**:第一版只在单次执行内存在;由 Runtime 保持活动句柄可达性,引用消失后可回收,至少在运行结束时回收全部区域。**不增加用户可见的 `free(region)`**,以免引入悬空区域句柄。用户的 `malloc/free` 管理的是**区域内部的块**。
- **内存安全范围**:Runtime 保证单次操作不访问该 Memory 区域之外;不保证用户自编分配器不会越块、重复 free、释放后复用错误。
此设计不要求新增独立的 `Pointer` 原始类型。**首版指针可由 `Memory` 引用和 `offset` 组合表达**;是否将这一对优化为一等 `Pointer`,由实际性能与调用需求决定,而非现在先引入语法。
### 4.3 Resource:与外部世界交互的独立对象
`ResourceHandle` 是由可信平台/Provider 签发的不透明句柄,代表文件、控制台流、网络连接或专门设备;含授权访问模式与生命周期。**它不是 `Memory`,不能索引成任意 RAM,也不能通过伪造整数创建。**
其常见操作是 `open/read/write/close`,但具体合法操作取决于资源接口。操作系统错误、EOF、部分读写、超时、取消都需要保留独立结果和错误语义。不得假定一次 `read` 总能填满缓冲区、一次 `write` 总能输出所有字节。
### 4.4 为什么要划分,而不是单一“万能地址空间”
`Memory` 代表运行时可管理的存储;`Resource` 代表外部效果。普通内存读写是局部状态改变,终端输入/网络读取可能随外部世界变化。把二者统一为可以随意解引用的指针,会隐藏权限和不可重复性,也会使知识证明与回放语义失真。
## 5. 可编程动态内存:到底哪些必须内置
### 5.1 最小计算内核操作
这里定义**能力**,不冻结最终函数名:
| 基本能力 | 候选用户接口 | 运行时职责 |
|---|---|---|
| 创建区域 | `memory.new(initial_pages, max_pages)` | 签发不可伪造引用,检查预算并分配零初始化区域 |
| 查询大小 | `memory.size(m)` | 返回页数,不泄露真实宿主指针 |
| 增长区域 | `memory.grow(m, pages)` | 原有字节与偏移不变,新字节为零;成功返回原页数,失败返回可区分错误 |
| 读取一字节 | `m[i]` | 边界检查,读取 `0..255` |
| 写入一字节 | `m[i] = b` | 边界和值域检查,更新同一 Memory 区域 |
`memory.new/size/grow` 是操作而非关键词;`m[i]` 利用当前已有的读取表达式,`m[i] = b` 只扩展现有赋值目标语义。这是此 RFC 要求的**唯一用户可见表达式语法增量**。
`len(m)` 可在另一个 ADR 中统一成字节长度而非另设 `memory.size`;这里暂不混淆“页数”和“字节数”。是否以页或字节为 `grow` 单位也应通过 ADR 固定,本文的原型推荐页。
### 5.2 位置赋值(Place)如何实现
对左值 `x` 与 `m[i]`,**赋值含义不同但遵循一个语法规则**:
- `x = value`:仍重新绑定当前函数帧局部变量,现有语义不变。
- `m[i] = byte`:先按既有从左到右顺序求值 `m` 与 `i`,检查目标为 `Memory` 并检查偏移和值,然后更新同一内存实例。
- `list[i] = value`:**不在第一个阶段承诺**。是否将列表修改为值更新/可变对象更新,要另行定义,不能顺便改变现有列表复制与索引语义。
- `obj.field = value`:首版也不必实现。只有在明确记录值/对象语义后再拓展。
表达式编译结构的最小增量是 `Assign { target: PlaceExpr, value: Expr }`,其中 `PlaceExpr` 初期只有 `Local(name)`、`Index(base,index)`;索引目标仅接受 Memory,其他索引仍只读。无需新 Markdown 节点。
### 5.3 自定义指针与分配器
`malloc(n)` 在 `.mq.md` 标准库里持有一个区域 `m` 和元数据指针 `next`,向调用方返回区域内偏移(或 `{region:m, offset:p}` 组合值)。`free` 仅将已分配块放回空闲链表;若空间不足,调用 `memory.grow`。对齐、大小、空闲块链表、合并策略都由 `.mq.md` 编写,**不要求 Rust 为每一种分配算法做特殊 HostOp**。
例:块格式 `[size:u32][next:u32][flags:u8][payload...]`。首版只支持按字节读取,`u32` 的编码/解码可以使用四次索引、`//` 和 `%` 在 `.mq.md` 中实现。若该路径确实过慢,后来可增加 `load_u32` 等**语义等价、可验证的加速实现**,不能因此把完整分配器写到 Host。
分配器可建立:Arena、first-fit、buddy、slab、链表/图、字符串字节编码、编译器符号表、虚拟 CPU 的寄存器文件等。**内存块的逻辑安全由标准库分配器定义**;调用者拿到任意偏移后仍有越块写的能力,这属于明确的低层风险。
### 5.4 内存模型形式定义(草案)
令 `R` 是区域身份集合,`M[r]` 是 r 区域的字节序列,`len(M[r])` 是其有效长度。
- `read(r,i) = M[r][i]`,前提 `0 ≤ i < len(M[r])`。
- `write(r,i,b)` 将 `M[r][i]` 更新为 `b`,前提 `0 ≤ b < 256` 且 `i` 合法;其余字节不变。
- `grow(r,k)` 要求 `k ≥ 0`,若预算允许,尾部追加 `k × PAGE` 个零字节,旧位置的字节值不变;否则返回显式失败且区域不变。
- 不同区域互不别名:`r1 != r2` 意味着修改 `M[r1]` 不影响 `M[r2]`。
- `m2 = m1` 若 m1 为 Memory,则二者指向同一 `r`;若为普通 `List`,原有值语义不变。
实现必须防止巨大 BigInt 偏移到 `usize` 转换时溢出、`offset + length` 整数溢出、预算计算溢出。任何错误不得 panic、越界读写或产生非确定性内存破坏。
## 6. 非线性存储、寄存器与加速的恰当位置
### 6.1 非线性内存不需要独立关键字
- **逻辑非线性结构**(链表、树、图):在一个或多个字节区域中存储偏移/引用即可;不要求新类型。
- **物理非连续内存**:运行时可以用分页、稀疏映射或 Vec 实现同一线性语义;用户无需知道物理页布局。
- **多设备内存**:未来可能带有设备特定权限与同步约束;不要将 CPU 内存随意解释成 GPU/QPU 地址。优先经专用受控资源/后端接入。
### 6.2 不加入真实 CPU 寄存器接口
当前局部变量与表达式临时值已经构成“虚拟寄存器”意义上的计算槽。编译到机器码时,可以由后端分配硬件寄存器。用户要模拟寄存器堆,可以用 Memory 自己实现;特殊硬件寄存器则是设备 I/O,而不是普通变量。
### 6.3 并行和原子操作
本 RFC 的内存规则初始定义为**单线程或按明确顺序执行**。绝不能声称 `m[i] = m[i]+1` 在多线程之间自动原子。未来若允许多个执行任务共享 Memory,必须另行设计同步/原子操作、数据竞争行为和访问序;也可以先采用无共享可变内存的任务隔离。新增原语应依据不可由当前操作正确组合而来的能力,而非先堆锁/屏障关键字。
### 6.4 GPU/QPU 不由普通 Memory 自动获得
Memory 支持字节缓冲区与数据布局,但 GPU 计算仍需要执行后端与数据传输,真实量子资源则不是可随意读取的字节地址。Marqdo 可用独立后端/服务处理硬件特有语义,内核应保持硬件无关。
## 7. 系统交互:能否不依赖 Rust/Python 实现高级功能?
### 7.1 必须给出的诚实答案
**可以**:大量系统功能由 Marqdo 标准库直接编写,包括文本与字节编码、目录遍历算法、协议解析、HTTP 报文、部分网络协议逻辑、缓存、日志格式、CLI 交互逻辑、文件格式读写、数据库协议客户端中的高层逻辑。程序运行时无需执行 Python 或 JavaScript,也不需要为每个高级库函数增加 Rust 分支。
**不能无条件做到**:如果当前运行方式是一个 Rust 编写的解释器,那么跨越用户态与操作系统的**最后一步**必须由解释器内建的系统适配器、已授权插件/worker、或其他可信底层机器代码执行。Markdown 源码不能在无相应运行环境、系统调用 ABI 和权限的情况下直接控制硬件。即使将来 Marqdo 编译成独立原生二进制,它也仍通过操作系统 ABI 或设备驱动与外界交互,只是**不再需要 Rust 解释器常驻运行**。
**不可混淆的四种独立性:**
| 层面 | 能否达到 | 含义 |
|---|---|---|
| A. 算法独立 | **可以,近期目标** | 排序、内存分配、解析器等算法均在 `.mq.md` 中 |
| B. 标准库独立 | **可以,近期目标** | 新标准库函数不修改 Rust Runtime/HostOp,单独版本发布 |
| C. 系统接口独立 | **有条件可以** | 已有的 OS 原语/权限可组合构造新功能;新增设备/OS 原语需相应适配器,但不改语言核心 |
| D. 运行时/宿主实现独立 | **中长期目标** | 独立编译成机器码/Wasm、或自举解释器,不依赖 Rust 解释器运行;仍依赖 OS ABI/硬件 |
**这里的“无需修改宿主”不是“无底层实现”。** 应将“高级逻辑用 Marqdo 实现”与“最后一跳由可信 OS adapter 执行”区分清楚。
### 7.2 最小系统资源操作模型
建议**不增加任何 I/O 关键字**,而以可注册、版本化的 `Resource` 接口提供 OS 能力。候选调用形式(未实施):
- `resource.open(kind, descriptor, rights) -> ResourceHandle`
- `resource.read(handle, memory, offset, max_bytes) -> count | EOF/error`
- `resource.write(handle, memory, offset, bytes) -> count | error`
- `resource.close(handle) -> Result`
其中:
1. `kind` 例如 `stdio`、`file`、`socket`,必须由可信平台适配器或已安装的 Provider 明确注册,**不能因为传入任意字符串就自动获得能力**。
2. `descriptor` 是声明请求的结构化参数,例如项目内文件名、网络目标,必须验证并授权;绝非任意可解引用真实指针。
3. `rights` 是请求的权能;实际授权为“请求 ∩ 操作者 grant ∩ Provider 支持”,拒绝必须显式报错。
4. `read/write` 通过 Memory 区域交换原始字节,边界检查和缓冲区借用由 Runtime/Adapter 保证;不得允许 I/O Adapter 长期保存未固定的宿主 Rust 指针。
5. 区分 EOF/0 字节暂时无进展/取消/系统错误;支持部分读写,标准库负责重试与缓冲。
6. `close` 撤销对应资源句柄;已关闭句柄、重用 ID 必须借助 generation/注册表检查,不能错误访问新资源。
7. 对设备特殊操作,不强行滥用 `read/write`:必要时可由具体 Provider 暴露专门的**版本化、授权操作**;不在通用内核中新增一个无条件允许任意 syscall 的 `control` 后门。
`resource.*` 属于**平台接口层**而不是纯计算内核;即使调用看起来是普通函数,执行前必须检查权限。现有 Host 能力检查可成为迁移入口,但不应为每个新高层标准库函数创建新的 `HostOp` 变体。
### 7.3 应该怎样真正避免“每扩展一次就修改 Host”?
将目前的固定分派进一步分成:
```text
普通 Marqdo 标准库函数
├── 调用 Marqdo 语言计算原语(无需 Host)
└── 仅在外部交互时使用稳定的 Resource ABI
│
Capability Gate
│
Provider/Platform Registry
│
┌────────────────┴───────────────┐
▼ ▼
标准平台适配器 独立第三方 Provider
文件/终端/网络 新设备/专用系统能力
```
关键是**一次性稳定可扩展的外部操作契约**,而不是一张不断增长的 Host 功能目录。
现有 C ABI v1、worker 与 `CallableRegistry` 应先尽量复用;新接口可在不破坏 v1 的条件下增补版本化注册信息。如需升级,必须有兼容矩阵与迁移机制。**Provider 负责它新增的原始能力;`.mq.md` 标准库负责高级功能。**
### 7.4 两种系统访问路线及取舍
| 路线 | 优点 | 风险/限制 | 建议 |
|---|---|---|---|
| 能力化 `Resource` ABI | 统一权限与跨平台行为;适合安全分层 | 新的真正 OS 原语仍需要 Adapter/Provider | **默认且必须支持** |
| 直接的原生 syscall/FFI 边界 | 能较快覆盖大量平台特性 | 平台/ABI 依赖、地址搬运、权限绕过、维护和可验证性问题 | 仅可信、本机、显式授权的可选高级接口;默认 pure/strict 禁止 |
在 Linux 等平台,即使“直接 syscall”,也要处理目标架构调用约定、数字差异、结构体布局、指针与长度、errno 以及 OS 策略;Windows/macOS 并不提供与 Linux 一模一样的 syscall ABI。不得将其伪装成普适的语言核心。
**无悖论地满足用户目标的办法**:向 Marqdo 暴露一个小而稳定的**操作系统能力边界**,然后让标准库以 Marqdo 自己的程序逻辑组合出系统高级功能。这是可实现的强独立性;完全消灭系统边界不是现实目标。
### 7.5 端到端例子 A:完全用 Marqdo 构建交互式 CLI
**先决条件**:可信平台适配器提供 stdin/stdout 的已授权资源,Marqdo 运行时支持 Memory 操作。
1. 标准库 `io.read_line` 在 `.mq.md` 中申请/增长缓冲区,循环调用资源 `read`,查找换行字节。
2. 标准库 `text.utf8_decode` 用 `.mq.md` 处理 UTF-8(正确处理非法序列;编码策略可配置)。
3. 用户程序将文本解析为数字,用现有 `Int` 算术和条件循环处理。
4. 标准库 `io.write_text` 使用 `.mq.md` 的 UTF-8 编码,再经 `write` 输出字节。
**全流程无需 Python/Node,也不需要 Host 特地实现 `input()`。** 然而最底层 stdin/stdout 仍由 OS 适配器调用操作系统接口。
### 7.6 端到端例子 B:新增 HTTP 客户端能力
**先决条件**:系统适配器已经提供经授权的网络连接资源、DNS/地址解析的可用机制,以及必要的加密能力或可插拔 TLS Provider。
1. 标准库构造 HTTP 请求行、首部和消息体的字节序列。
2. 使用 `resource.write` 处理部分发送;使用 `resource.read` 实现缓冲接收。
3. `.mq.md` 解析状态行、首部、chunked 编码等;实现重试、缓存、超时逻辑。
4. 测试 HTTPS 时,必须使用可靠 TLS 实现(可由已审计 Provider 提供;“不依赖外部语言写协议逻辑”不意味着应该自己手写未经验证的 TLS)。
网络的“最后一跳”必须存在,且不同平台的 TCP/TLS 能力可能需要不同 Provider;不能只靠 Memory 推导出 socket。
### 7.7 端到端例子 C:科研加速与 QPU
- 经典数值模拟:`.mq.md` 可以在 Memory 中构造矩阵、压缩存储、数据布局算法,后续由独立计算后端加速。
- QPU:量子设备是具有权限、调度与门集约束的资源,不是普通 Memory;通过专用 Provider/Backend 提交计算、接收结果。
- 这种接口可保持标准库和上层代理/知识文档不绑定 Rust 宿主,但并不能保证算法自动获得 GPU/QPU 性能或精确的量子物理语义。
## 8. 安全、权限与错误模型
| 保护目标 | 首版保证 | 明确不能保证 |
|---|---|---|
| 虚拟 Memory | 区域越界、非法索引、增长限额、无效句柄失败 | 区域内 malloc/free 的块级内存安全 |
| 不透明 Resource | 授权身份、访问模式、资源关闭和 generation 检查 | 平台适配器/不可信原生插件本身绝对安全 |
| pure | 内部算术、内存与受预算控制的逻辑计算;不获得外部 I/O | 允许直接 syscall 或任意外部访问 |
| project | 对批准的文件/终端/网络资源进行操作,使用能力化 API | 等价于 OS 级进程沙箱 |
| strict | 维持现有 fail-closed 策略;新 I/O Provider 必须另行经过 OS 级安全验收 | 当前版本自动支持所有外部能力 |
进一步要求:
- 全局/单区域内存预算、增长次数与 fuel 限额、操作可取消性;内存失败不得半成功。
- 路径资源应使用受控根路径和可验证的 OS 打开策略,避免字符串检查后再打开造成 TOCTOU;网络权限需要处理名称解析/重绑定等问题。
- 插件/worker 不因拥有 API 名字自动获得权限;原生进程内插件不能被当作不可信沙盒。
- Memory/Resource 活动句柄不应直接跨线协议序列化;跨边界传输使用字节复制、快照或显式授权的资源传递协议。
- 诊断保持来源 Markdown 节点位置;可选追踪读取/写入的逻辑区域、偏移和长度,不能将外部文件秘密或原始敏感字节无条件写日志。
## 9. 与当前 Marqdo 语法和知识编译的结合
这一方案故意**只修改最少的表达式语义**。
- `##` 函数、`**` 执行、`*` 返回、Markdown 列表与引用规则继续维持规范。
- `[]()` 是**文档/知识的静态寻址**;动态 Memory 的字节偏移属于执行期位置,两者不强行共享 URI 语法。
- 普通 GFM 表格是文档值;可用于初始化/展示内存快照,但不会自动变成内存分配器。
- `memory.new(...)` 与 `resource.open(...)` 都可按普通函数调用表达;不新增 `memory`、`pointer`、`asm`、`register` 这样的语法关键字。
- 内存状态变化可选输出可读 GFM Table 供 View/Proof 使用;证明必须以抽象执行语义为依据,而不是假设表格文本本身就是执行存储。
示例(**候选语法,当前尚不可运行**):
````markdown
## 研究一个字节缓冲区
此函数演示“可引用的存储区域”与普通 Markdown 叙述并存。
**区域 = memory.new(1, 4)**
**`区域[8] = 42`**
**另一个名字 = 区域**
**`另一个名字[9] = 99`**
*区域[8] + 区域[9]*
````
预期结果 141。注意这里 `另一个名字 = 区域` 分享 Memory 身份;一般列表/表格并不因此被改造成共享字节缓冲区。
## 10. 可行性和备选方案评定
| 设计 | 语言自主性 | 高层开发便利性 | 性能潜力 | 安全/复杂度 | 结论 |
|---|---|---|---|---|---|
| 继续只扩充 HostOp | 低 | 表面高 | 取决于 Rust 实现 | Host 随功能膨胀 | 否决为主要路线 |
| 只有托管对象堆/GC | 中 | 高 | 对一般数据不错 | 需要身份/生命周期语义 | 保留现有托管值,但不足以满足低层目标 |
| **多区域字节内存 + 托管值 + Resource** | **高** | **高** | 后端可优化,字节操作起初较慢 | 明确的区域边界;块级风险可见 | **推荐** |
| 任意裸指针 + 直接机器码 | 极低平台独立性 | 低 | 最高上限 | 高风险、难验证 | 只可选的可信原生扩展 |
| 纯 GFM 表格作堆内存 | 低至中 | 对展示很高 | 非常低 | 动态状态/身份混乱 | 否决 |
该推荐不是宣称“最优”已经被数学证明,而是在现有语法、实现成本、扩展性与安全边界下的**工程选择**。
## 11. CLI/Runtime/标准库仓库的具体职责
### 11.1 `marqdo-cli`:保留小而稳定的内核
- `marqdo-value`:新增不可伪造的 Memory 值表示;必要时添加 ResourceHandle,注意现有 PluginHandle 不能随意直接复用为 OS 资源句柄。
- `marqdo-expression`:`AssignTarget` 的 Local / Index 变体;保留原有表达式读取。
- `marqdo-frontend`:更新表达式 AST 对应 IR 和定位信息;不改变 Markdown 语义映射。
- `marqdo-runtime`:Memory 实例生命周期/增长/边界检查/预算;执行位置写入;参考解释器。
- `marqdo-host`:逐步提供可注册资源操作的能力边界;必须保持旧 `HostOp` 兼容窗口,不能一次砍断所有旧调用。
- `marqdo-package` / CLI:发布标准库兼容信息,继续利用已存在的 path/Git pinned lock;资源授权仍由真实操作者提供。
### 11.2 独立 `marqdo-stdlib`:以 Marqdo 实现通用功能
建议独立仓库 `cflmy/marqdo-stdlib`,首期目录:
```text
marqdo-stdlib/
core/ # 字节编码、整数工具、可恢复错误封装
memory/ # arena, allocator, free-list, binary codecs
collections/ # list/tree/graph/hashmap 等数据结构算法
text/ # UTF-8, parsing
io/ # buffered readers/writers, read_line
data/ # csv, json, 自定义格式
tests/ # 以 marqdo run/call 执行的标准库回归程序
```
高层函数应在 `.mq.md` 里,按普通模块导入。发行版携带一个经测试的 stdlib 版本,支持离线运行;单独发布更新时,要求明确的 Runtime 兼容范围与锁定。标准库可有原生加速 Provider,但不能把它作为常规功能的唯一实现路径。
### 11.3 独立 Platform Providers
系统 Adapter 可以由 Rust、C、汇编或将来的 Marqdo 原生编译代码实现。当前 CLI 不应为普通库函数增加 HostOp;对真正的新 OS 原语,若标准 Provider 不支持,用户必须安装/编写扩展 Provider,并获得相关权限。**不是每个新设备都能只靠修改 `.mq.md` 获得访问能力。**
## 12. 分阶段实施(范围严格受控)
### R0:冻结设计与当前基线(无新功能)
- 此 RFC 和对应 ADR 经审阅,确认 `Memory` 值语义、`index assignment`、错误类别、线协议边界及安全模式。
- 修复已有跨平台 CI/安全收敛问题;固定解释器行为的回归样例。
- 建立一页对照表:新增语义是否需要 `HostOp`、新关键字、新 Markdown 节点。首版目标均为“不需要”。
### R1:最小可寻址存储(核心里程碑)
- `Memory` 区域:创建、大小、增长、读取一字节、写入一字节;实现总/单区域预算、错误恢复与 trace。
- 支持 `**\`m[8] = 42\`**` 这类现有 Markdown 负载中的索引赋值。
- 不开放裸指针、不引入新寄存器、不改旧 List/Map/Table 的赋值与复制行为。
- 运行跨区域隔离与别名测试;报错不能导致原有区域被污染。
### R2:Marqdo 自举的内存与算法库
- `.mq.md` 顺序分配器;随后 free-list 可复用分配器。
- 不修改 Runtime/Host 编写并执行:可增长链表、二叉树、简单虚拟 CPU、UTF-8/二进制解析器。
- 提供内存利用率、分配吞吐、碎片率、执行时间及解释器 fuel 的测量;避免只用“功能跑通”证明成熟度。
### R3:最小系统资源闭环
- 在项目模式先实现 `stdio` 与受控项目文件的 `open/read/write/close`;**功能可用性不代表 strict 安全已通过**。
- `.mq.md` 标准库实现 `read_line`、`write_text`、文件二进制复制与 UTF-8 编码。
- 资源注册层允许 Provider 扩展;验证一个第三方资源端点不修改解释器编译即可接入(是否需要实现新的系统适配器需明示)。
- 对当前 `HostOp` 逐步适配,保留兼容期,不能强行把所有第三方都迁移到新的不成熟 ABI。
### R4:可选增强(不阻断基础版本)
- 索引/字段赋值的一般化、字节多宽度原语及位运算、快照、增量编译、优化执行器。
- 需要共享内存并行时,再设计原子/同步与数据竞争语义。
- 独立网络/TLS、设备、GPU、QPU 后端与原生编译目标。
**必须遵循的顺序**:R1 完整正确 → R2 验证计算自主性 → R3 证明外部交互可组合;不先把 R4 的设备能力和并发类型系统塞入 CLI。
## 13. 端到端验收清单
| 编号 | 验收项目 | 判定标准 |
|---|---|---|
| T1 | 语义一致性 | 旧的局部赋值与 List/Map/Table 行为完全不变,旧 Markdown AST golden 通过 |
| T2 | Memory 隔离 | `m1[0]=x` 不影响 `m2[0]`;`m2=m1` 后写入可相互观察 |
| T3 | 增长与错误 | 成功增长保留旧字节且新字节为 0;负索引、越界、超限增长、巨大 BigInt 均明确失败且不破坏状态 |
| T4 | 自建分配器 | `.mq.md` 实现 malloc/free,释放块重复利用;高低地址和碎片测试通过 |
| T5 | 高级数据结构 | 不改 Rust 核心,仅新增 `.mq.md` 实现可增长链表、树、二进制协议解析器 |
| T6 | 标准库独立升级 | 旧 CLI 二进制保持不变,安装一个与之兼容的新 `.mq.md` 库并成功使用 |
| T7 | 终端闭环 | `marqdo run` 仅使用标准库加基础 OS Adapter,完成多轮输入/输出,无 Python/Node 调用 |
| T8 | 系统资源授权 | 无 grant 无法 open;已关闭/伪造 handle 无法访问;项目路径无法越权;所有拒绝有诊断和必要审计 |
| T9 | 安全模式 | `pure` 不允许外部 I/O;当前 `strict` 不自动放开新 Provider;真实隔离与失败路径有测试 |
| T10 | 扩展接口 | 新高层 I/O/协议函数不增加任何 HostOp;新 Provider 通过约定的独立注册/版本机制接入 |
| T11 | 兼容/性能 | Windows/macOS/Linux 的支持矩阵真实披露;可定量报告字节访问吞吐、内存增长和 Python/Rust baseline 差距 |
| T12 | 文档与证明 | Memory 状态可导出可读快照;执行状态引用源 Markdown span;不将数据表格误认成内存本体 |
## 14. 需要 ADR 决断而不是隐含假设的开放问题
本 RFC 采用一条**可实施的推荐路线**,但下列决策会改变公开语义,须单独冻结:
1. **页语义**:推荐 64 KiB 页、只增长;是否与外部目标标准保持一致,预算如何计入已保留与已提交物理内存。
2. **Memory 的值/调用传递**:推荐复制引用保持身份,区域可达则存活;禁止隐式跨线协议序列化。
3. **Memory 引用寿命**:推荐首版只在一次执行期间有效,采用自动回收或至少结束时回收;内部块分配是 stdlib 职责。
4. **Place AST**:推荐首版只增 Local + Memory Index;其他集合的索引赋值须另行评议。
5. **OS Resource 最小 ABI**:是否先使用现有插件注册机制包装,而不立即新增第二套并行插件协议。
6. **跨平台支持范围**:纯计算 API 保持统一;系统资源能力按平台明示 availability,不能假定所有设备/系统调用均存在。
7. **实验数据**:R2 执行效率可能较低,是否加入 32/64 位宽读写及优化编译由实测结果决定,而非预设几十种关键字。
## 15. 最终建议与决策摘要
**建议通过的架构选择:**
- **语言核心**:保留现有托管值与控制流,新增**显式、多区域、可增长的字节 Memory**;用现有索引表达式扩展可写位置,不创建新的 Markdown 语法层。
- **内存管理能力**:`Memory` 区域由 Runtime 提供并保护区域边界;`malloc/free`、块布局、链表、树、二进制编码等由 `.mq.md` 标准库自己构造。
- **系统能力**:独立、授权、可替换的 `Resource` 边界;标准库在 Marqdo 中组合 I/O,OS 的最后一跳仍由可信本地 Adapter/Provider/编译后端承担。
- **扩展性**:不为每个库函数增加 Rust `HostOp`;只有新增*真正不可由现有原语表达*的硬件/OS 能力,才需要扩展适配器。
- **安全性**:区分区域边界安全、块级内存安全、进程隔离和 OS 权限;不能用“沙盒”一词掩盖缺少的保证。
- **长期演进**:以此作为实现自主数据结构、字节码 VM、优化执行器与量子/科学计算后端的基础,但不许声称三个内存原语就能自动产生 GPU/QPU 加速。
**核心结论:** 如果“摆脱宿主语言依赖”指的是**程序逻辑、标准库算法和高级系统功能不依赖宿主语言来实现**,这条路线可以达成;如果指的是**无需任何底层可信机器代码和操作系统接口即可访问硬件**,则不可能。对真正的独立编程语言而言,恰当目标是**自己定义计算、自己构建高层库、清楚且可扩展地跨越不可消除的系统边界**。
---
## 参考资料(外部设计依据,不是 Marqdo 已实现功能的证据)
1. [WebAssembly Core Specification 3.0(2026-09-11)](https://www.w3.org/TR/wasm-core-2/):线性内存、内存增长、抽象机器、核心与环境 API 的分离;注意其保证为线性内存边界安全,不保证客户程序自己设计的内部分配器没有逻辑错误。
2. [Rust `core` / `alloc` / `std` 文档](https://doc.rust-lang.org/std/):计算基础、分配与平台库的分层思路;并不意味着必须用 Rust 编写 Marqdo 的标准库算法。
3. [Linux Kernel ABI documentation](https://docs.kernel.org/admin-guide/abi.html):OS 用户态接口的稳定性、适用范围和版本边界。
4. [Linux Kernel ioctl interfaces](https://docs.kernel.org/driver-api/ioctl.html):设备特定能力并不都能归约为普通 RAM 读写,ABI 应明确数据布局与操作语义。
5. [Marqdo 当前安全模型](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)及[插件 ABI v1](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/plugin-abi.md)。
+66 -7
View File
@@ -71,7 +71,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
| 2 | Cross-platform strict `cfg` | **Fixed locally; native CI UNVERIFIED** | `#[cfg]` dispatch + non-Linux regression test |
| 3 | Strict probe→launch race | **Fixed locally; GitHub Linux CI UNVERIFIED** | fail-closed CLI contract + probe-OK/launch-EAGAIN seam test |
| 4 | Parallel security matrix isolation | **Fixed** | unique paths + helper `OnceLock`; 7/7 in parallel workspace run |
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 44 semantic, 52 compile-negative, 12 runtime-negative fixtures |
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 68 semantic, 52 compile-negative, 21 runtime-negative fixtures |
| 6 | Condition header + source spans | **Fixed** | executable tails rejected; literal-inside-backtick mapping tests |
| 7 | Independent Tool consumers | **Done locally** | `sample-view`/`sample-knowledge`/`sample-proof` E2E test |
| 8 | lock v3 reproducibility | **Implemented; tamper test added** | `package_env` PASS |
@@ -86,7 +86,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
- Frozen Markdown profile `marqdo.md.v1` with 160 AST goldens and explicit
CRLF byte semantics.
- Native Rust interpreter with 44 semantic goldens, 52 compile-error fixtures,
- Native Rust interpreter with 68 semantic goldens, 52 compile-error fixtures,
and runtime error source attribution.
- Native `**` power is pinned end-to-end: right associativity, unary-minus
precedence, parentheses, float negative exponents, branch/loop conditions,
@@ -109,6 +109,31 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
version verification.
- Linux strict L1: Bubblewrap namespace supervisor, runtime probe, fail-closed
launch, child self-verification, 7/7 adversarial matrix.
- Addressable memory (RFC-CM P1, ADR-0022): opaque `Memory` values shared by
handle identity, region isolation, zero-filled 64 KiB page growth, bounded
byte read/write, `memory.new/size/grow`, `m[i] = byte`, run and per-region
budgets, host/wire rejection, Analysis v2 target projection.
- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): success is
`{ok:true,value:...}`, failure is `{ok:false,code,message[,details]}` as an
ordinary Map; expected conditions return it, invariant violations stay
fatal. A `.mq.md` first-fit/free-list allocator reuses blocks, splits free
blocks, coalesces address-adjacent blocks and returns `alloc.no_space`
without aborting.
- CM-A07 Compat-A (ADR-0027): explicit `.mq.md` `io.写值` chain
(`format.显示` → UTF-8 encode → `Memory` → `resource.write(stdout)`) matches
builtin `打印` for the covered value set; builtin remains default and no
alias is routed.
- Resource streams (RFC-CM CM-A06, ADR-0025): opaque host-issued `Resource`
handles with `resource.open/read/write/close`; bytes move only through
`Memory`; stdout write is capability/output-budget checked and project-file
read/write is path/grant checked; file writes buffer until `close` and
never flush on drop; a `.mq.md` chunked copy loop reproduces a file without a
Rust `copy_file`; strict refuses every resource open until a provider is
verified. Semantic fixture 66 and CLI/host tests.
- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
value-semantics `list.append/concat`, `map.keys/has/get/set/remove`,
Unicode scalar `text.codepoints`/`text.from_codepoint`, and non-negative
`table.columns/row/cell`. Pure builtins; no mutation, no new syntax.
## 4. Open blockers and limits
@@ -121,25 +146,59 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
fixture lane, Unix-gated worker tests).
4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed
broker services; unsupported requests must return `capability.unavailable`.
5. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
5. Resource streams are a **P3 first slice** only: project-file read and
buffered write plus stdout, but no directories, network, process, device,
generic provider or stdlib `print` migration; strict refuses every
`resource.open` until the provider passes the isolation lane.
6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
digest-pinned directory with a regular-file executable is supported.
6. No Windows/macOS RC archive exists from this host. The release-candidate
7. No Windows/macOS RC archive exists from this host. The release-candidate
workflow can build macOS on GitHub, but it has not run for this SHA.
7. No release tag/binary publication. The release decision is **DO NOT
8. No release tag/binary publication. The release decision is **DO NOT
RELEASE** until the native CI matrix and a security-lane CI URL exist.
## 5. Evidence files
## 5. RFC-CM compute-machine status
The two 2026-11-20 architecture RFCs are design input, not implemented
behavior. This table records only what is in this tree now.
| RFC phase | State | Evidence / limit |
|---|---|---|
| P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged |
| P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) |
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), and a quoted/escaped CSV row parser (68). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Still missing: separate `marqdo-stdlib` repo, JSON floats, table group, EOF/partial stream contract, and a frozen-CLI closure lane. |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict refuses all opens. Fixture 66 + CLI/host tests. CM-A07 is **Compat-A only**: `examples/stdlib` provides an explicitly imported `.mq.md` `format/utf8/io` chain whose output matches builtin `打印` for Unit/None/Bool/Int/Float/Text/List/Map/Table; builtin `打印` remains default, no alias routing, and Formula/Code/FunctionRef/PluginHandle formatting is still unobservable. Still missing: Compat-B/C, directories, network, generic provider registry. |
| P4 Table logical/physical migration | **NOT IMPLEMENTED** | `Value::Table` remains `Vec<Vec<Value>>`; no stdlib table algorithms |
| P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) |
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
The P1 memory domain is pure computation: it needs no grant and is available in
`pure`, `project` and `strict` (bounded by the memory budgets). It adds no
`HostOp`, no Markdown role and no CLI command. `Memory` handles cannot cross
the wire or a host/plugin/foreign boundary. Block-level safety of a
user-written allocator is explicitly **not** claimed.
## 6. Evidence files
- [`evidence/2026-11-20-local-gate.md`](evidence/2026-11-20-local-gate.md) —
earlier local gate and Windows autocrlf reproduction.
- [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md)
— current VM lane, environment, full log and workspace test log.
- [`evidence/2026-11-20-resource-stream-p3.md`](evidence/2026-11-20-resource-stream-p3.md)
— CM-A06 first slice: stdout write, project-file chunked read, strict refusal,
wire/host boundary rejection and test commands.
- [`evidence/2026-11-20-resource-file-write-p3.md`](evidence/2026-11-20-resource-file-write-p3.md)
— buffered project-file write, flush-on-close, buffer ceiling, grants and
CLI write-then-read evidence.
- [`evidence/2026-11-20-stdlib-print-compat-a.md`](evidence/2026-11-20-stdlib-print-compat-a.md)
— CM-A07 Compat-A `.mq.md` print prototype, side-by-side output comparison,
covered/uncovered value types and the Compat-B blocker.
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
— G0..G12 matrix and RC decision.
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
L1/L2 limits.
## 6. Reproduce
## 7. Reproduce
```bash
export PATH="$HOME/.cargo/bin:$PATH"
@@ -0,0 +1,183 @@
# ADR-0022: Addressable `Memory` and Memory-index place assignment (RFC-CM P1)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md` (CM-A01/CM-A02/CM-A03),
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (P0/P1), ADR-0007,
ADR-0011, ADR-0017, ADR-0021
Supersedes: the v1 rule that assignment targets are bare identifiers only
Specs: `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`,
`spec/analysis-v2.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-memory-p1.md`
## Context
RFC-CM-0001/0002 define three domains: managed values, addressable regions and
capability resources. The first two are a language change; the third is an OS
boundary and is deliberately **not** part of this ADR.
Before this decision the interpreter had no writable place other than a local
slot. `Value` had no byte region, `a[i]` was read-only, and assignment was
parsed only as `identifier = expression`. A `.mq.md` program therefore could
not express an allocator, an encoded record, a free list or a byte protocol
without a new Rust builtin for every algorithm. That is the closure gap
RFC-CM-0002 §2 identifies.
The decision must not:
* turn managed `List`/`Map`/`Table` values into shared mutable objects;
* expose a host pointer, `usize` address or procedural `free(region)`;
* let an active handle cross the value wire or an ABI boundary;
* add a Markdown node, keyword or `HostOp` for memory;
* claim block-level memory safety for user allocators;
* weaken `strict` or grant any external I/O.
## Decision
### 1. `Value::Memory` is an opaque, run-scoped identity
`MemoryHandle` (`crates/marqdo-value/src/value.rs`) is
`(run_id, region_id, generation)` with private fields and a runtime-only
constructor. It is not `Serialize`/`Deserialize`, is never built from user
input, and carries no host address.
Copying the value, passing it as an argument and storing it in a list or map
all copy the handle and preserve region identity. `==`/`!=` compare identity
only; `Memory` has no ordering comparison. `display()`/`repr()` render the
literal `Memory`, never the internal identity.
### 2. Regions are grow-only, bounded byte arrays
Frozen semantics (CM-A01/CM-A02):
| Property | Decision |
|---|---|
| Page size | 64 KiB logical page (`PAGE_SIZE`) |
| Initial content | Every byte is `0` |
| Growth | `memory.grow` only increases; old offsets/bytes unchanged; new bytes zero |
| Failure atomicity | A rejected create/grow allocates, commits and mutates nothing |
| Shrink/free | No shrink and no user-visible region free; regions are dropped with the run |
| Address | `(region identity, non-negative offset)`; an integer alone is never a credential |
| Read | `m[i] -> Int(0..=255)` |
| Write | `m[i] = byte`, `byte` must be `Int(0..=255)`, no truncation |
| Isolation | Different regions never alias; handle copies of one region do |
| Visibility | `memory.size` returns committed pages, not host capacity or pointers |
Limits in `MemoryLimits`:
* `DEFAULT_MAX_TOTAL_BYTES = 64 MiB` committed across the run;
* `DEFAULT_MAX_REGION_BYTES = 16 MiB` per region, which also caps declared
`max_pages` at 256;
* budget arithmetic uses checked `u64`/`usize` operations; allocation uses
`try_reserve` so a host OOM is a structured `resource.exhausted` instead of
a panic.
Limits are a runtime API (`marqdo_runtime::MemoryLimits`, `run_with_limits`)
and are overridable on `run`/`call` with `--max-memory <BYTES>` (total) and
`--max-region-memory <BYTES>` (per region). The defaults are stated in
`spec/evaluation.md`. Both flags apply in `pure`, `project` and `strict`; they
grant no external capability.
### 3. The only new writable place is a `Memory` index
`AssignTarget` gains `Index { base, index, span }` in the expression AST and
`PlaceTarget` mirrors it in the IR. The surface syntax remains a `strong`
node containing `target = expression`; no Markdown role, heading or list
meaning changes.
* `x = value` still rebinds the current frame's local slot.
* `m[i] = byte` evaluates **base, then index, then RHS**, exactly once each.
All three checks (base is `Memory`, offset in range, byte in `0..=255`)
happen before the single byte write, so a failed statement does not write.
* `list[i] = value`, `map[key] = value` and `obj.field = value` are not
defined. Member targets are a parse error; a non-`Memory` index base is
`runtime.type`. No managed collection changes value semantics.
* Nested index targets are accepted syntactically; the innermost base must
still evaluate to a `Memory` value at run time.
* Assignment remains statement-only; `x = y = 1` is still a parse error.
This is CM-A03.
### 4. Handles cannot cross the wire or the host boundary
* `to_wire(Memory)` emits only `{"type":"memory"}` so an accidental transfer
is visible. `from_wire` rejects that tag with `WireError::Invalid`; a
round-trip is intentionally impossible (no `wire_version` bump: the value
shape is rejected, not newly accepted).
* `Host::invoke` rejects any argument tree containing `Memory` before dispatch
(`runtime.type`), so plugins, foreign kernels, math, file and print adapters
never see a handle. A future explicit byte-snapshot operation must copy bytes
into a managed value and is a separate ADR.
* `HostOp` is unchanged. `memory.new/size/grow` and index read/write are
interpreter operations, not capability-gated host effects.
### 5. Error categories are stable and distinct
| Situation | Code |
|---|---|
| Negative/unrepresentable offset, offset >= region length | `runtime.index` |
| Unknown, stale or cross-run handle; invalid page count (negative, > budget declaration arithmetic) | `runtime.memory` |
| Wrong operand type; byte outside `0..=255`; non-Memory index-assignment base | `runtime.type` |
| Declared max, region budget or total budget exceeded | `resource.exhausted` |
No operation panics on malformed input or forged handles. `expect` is used only
after the same handle has been validated in the same call frame.
### 6. Builtins are part of the language surface, not `HostOp`s
`memory.new(initial_pages, max_pages) -> Memory`,
`memory.size(memory) -> Int`, `memory.grow(memory, pages) -> Int` are added to
`marqdo-ir::builtins::BUILTINS` and `spec/evaluation.md`. They are pure
(`effectful: false`): `pure` mode may use them, `strict` needs no new provider,
and no capability grant is consulted.
### 7. Compatibility
* Local assignment IR keeps its `target` string in Analysis v2; an index
target is an additive object (`{"kind":"index",...}`). Consumers that only
handle locals continue to work.
* Expression AST/statement serde gains an enum tag; this is a pre-1.0 internal
contract documented in `spec/expression.ebnf`.
* Existing Markdown AST goldens, value semantics, `List`/`Map`/`Table`
behaviour and all old builtins are unchanged.
* `Memory` is additive to the value type set. Old programs never produce it.
ABI wire consumers see a new rejected tag, not a renumbered existing tag.
## Non-goals
* No `Pointer` value, raw pointer, `free(region)`, registers or machine code.
* No index/field assignment for managed collections; no implicit mutation.
* No multi-threaded sharing, atomics or data-race guarantees.
* No serialization of live regions; no cross-process shared memory.
* No `ResourceHandle`, `Stream`, files, network or OS syscalls. Those are
CM-A05/CM-A06/CM-A09/CM-A11 and later RFC phases.
* No block-level safety claim for a user-written allocator; the runtime only
bounds a single access to its region.
* No allocator or stdlib API in P1; `--max-memory` only bounds the runtime.
## Consequences
* A `.mq.md` program can now build byte encoders, arenas, free lists and
virtual CPU register files without new Rust code. P2 (the Marqdo-written
stdlib and closure fixtures) becomes testable.
* The interpreter owns one `MemoryStore` per run; `run_with_limits` exposes
the budget. This is runtime state, not a second `HostOp` table.
* Trace gains `memory.new`, `memory.grow` and `memory.write` semantic events
without exposing handles or byte secrets.
* The remaining RFC decisions (CM-A04 and later) stay separate. In particular,
generic structural observation/construction, recoverable `Result`, table
algorithm migration and the resource/Stream ABI are **not** implemented by
this ADR.
## Verification
`docs/evidence/2026-11-20-memory-p1.md` records:
* parser/AST tests for index and rejected member targets;
* runtime store tests for isolation, aliasing, zero-fill, atomic failed grow,
negative/huge offsets, forged/stale handles, byte range and budgets;
* semantic goldens `45`–`47` and runtime negatives `13`–`18`;
* wire rejection test and host-boundary rejection test;
* CLI example `examples/memory-buffer.mq.md` returning `141`;
* Analysis v2 index-target projection;
* `cargo fmt --check`, `cargo clippy -- -D warnings`, `cargo test --workspace`.
@@ -0,0 +1,147 @@
# ADR-0023: Minimal structural observation and construction (RFC-CM CM-A04)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A04, P2),
ADR-0007, ADR-0022
Specs: `spec/evaluation.md`, `spec/expression.ebnf`, `spec/analysis-v2.md`
Evidence: `docs/evidence/2026-11-20-structure-api-p2.md`
## Context
RFC-CM-0002 §3.5 identifies a closure gap: a `.mq.md` program can read a
`List`/`Map`/`Table` and print it through the host, but it cannot generally
**construct** the same structures dynamically or **observe** all of their
logical parts. That makes `format`, JSON/CSV parsers and table algorithms
impossible to write in `.mq.md` without adding a task-specific Rust function
for each algorithm.
The current core already has:
* `List` literals, index reads (including negative offsets), `长度`;
* `Map` record literals, `.field` reads, `长度`;
* `Text` concatenation, character indexing and `长度`;
* `Table` construction through `表(header, rows)`, `长度` (rows), display and
wire round-trip.
It lacks: append/concat, dynamic map update/enumeration, Unicode scalar
conversion, and table column/row/cell projection. These are observation and
construction primitives, not algorithms. Sorting, filtering, joining,
serialising and parsing remain standard-library work.
## Decision
Add a small, namespaced, **pure** builtin set. No new Markdown node, keyword,
`HostOp`, capability or mutation semantics is introduced. All constructors
return **new** values; an input collection is never modified in place. This
keeps INV-02 and preserves existing `List`/`Map`/`Table` copy semantics.
### 1. `List`
| Signature | Semantics |
|---|---|
| `list.append(list: List, value) -> List` | New list with `value` at the end; the input list is unchanged. |
| `list.concat(left: List, right: List) -> List` | New list containing `left` then `right`; both inputs unchanged. |
Neither aliases the other list's element storage: values are copied according
to their normal value semantics (`Memory` handles keep region identity).
### 2. `Map`
| Signature | Semantics |
|---|---|
| `map.keys(map: Map) -> List` | `Text` keys in canonical sorted order (`BTreeMap` order, the order used by `display`). |
| `map.has(map: Map, key: Text) -> Bool` | Whether the key exists. |
| `map.get(map: Map, key: Text) -> Value` | Stored value, or `None` when absent. Use `map.has` when `None` is a legal stored value. |
| `map.set(map: Map, key: Text, value) -> Map` | New map with the key set; the input map is unchanged. |
| `map.remove(map: Map, key: Text) -> Map` | New map without the key (missing key is a no-op); the input map is unchanged. |
Keys are always `Text`; arbitrary-value keys are not introduced. `map.get`
returning `None` is a *lookup result*, not the language's recoverable-error
contract; a full `Result` convention remains CM-A05 and is not frozen here.
### 3. `Text`
`Text` remains a Unicode scalar-value sequence, not a byte array. These
primitives make the existing representation observable without introducing a
second text type or implicit UTF-8 conversion.
| Signature | Semantics |
|---|---|
| `text.codepoints(text: Text) -> List` | Unicode scalar values (`Int`) in code-point order. |
| `text.from_codepoint(codepoint: Int) -> Text or None` | One-scalar `Text`; `None` for a surrogate or a value outside `0..=0x10FFFF`. |
UTF-8 encoding/decoding is **not** a core primitive. It can be written in
`.mq.md` using bytes, `text.codepoints` and `text.from_codepoint`; this keeps
the byte/code-point distinction explicit.
### 4. `Table`
`Value::Table` stays the logical value; its physical `Vec<Vec<Value>>` layout
is not part of this contract. Indices are non-negative row/column coordinates
(unlike `List` negative indexing) because a table cell is a document
coordinate. Bounds failures are `runtime.index`.
| Signature | Semantics |
|---|---|
| `table.columns(table: Table) -> List` | Header names (`Text`) in column order. |
| `table.row(table: Table, row: Int) -> List` | A new list of the row's cells in column order. |
| `table.cell(table: Table, row: Int, column: Int) -> Value` | The cell value. |
No `table.set_cell`/`append_row` builtin is added. Rebuilding a table is a
value-level operation that can be written with `表(header, rows)` once list
construction exists; adding a Rust table algorithm would violate the closure
goal.
### 5. Error and compatibility rules
* Wrong collection/text/table type or key type → `runtime.type`.
* Table row/column outside the logical shape → `runtime.index`.
* `text.from_codepoint` never throws for an invalid scalar; it returns `None`.
* `map.get` never throws for a missing key; it returns `None`.
* Existing `List`/`Map`/`Table` behavior, display, wire encoding and Markdown
laziness are unchanged.
* The builtins are `effectful: false`, available in `pure`, `project` and
`strict`, and add no capability. They do not consume output budget.
* Names are frozen as written; aliases are not added. Namespaced calls use the
existing `Expr::Member` resolution path.
## Non-goals
* No `set_cell`, `append_row`, `sort`, `filter`, `join`, `group`, JSON/CSV or
other algorithm builtins; those belong in `.mq.md`.
* No mutable/aliased List or Map, no in-place mutation, no shared structural
sharing protocol.
* No arbitrary-value map keys, no ordered-map insertion semantics, no
duplicate-key policy change.
* No UTF-8/UTF-16 conversion primitive, no byte-array text type.
* No `Result`/`try`/exception syntax; CM-A05 stays open.
* No change to Table storage, GFM parsing, source spans or `ref` semantics.
## Consequences
* A `.mq.md` program can build maps and lists dynamically, enumerate keys,
project table cells and encode Unicode scalars. This unblocks a Marqdo-
written formatter, table filter/projection and a JSON/CSV parser without
per-algorithm Rust.
* The builtin table grows by 12 pure entries. The migration rule remains:
only observation/construction primitives may enter core; algorithms do not.
* Because constructors are value-returning, repeated `map.set` in a loop is
O(n²) for a BTreeMap clone. That is acceptable for P2 semantics; a
persistent/structural-sharing representation is a P6 performance decision,
not a semantic one.
* `map.get` returning `None` for absence is deliberate and documented; a
future CM-A05 `Result` convention must not silently reinterpret existing
programs.
## Verification
`docs/evidence/2026-11-20-structure-api-p2.md` records:
* ADR/spec deltas and builtin metadata;
* semantic fixtures for value-semantics list/map operations, code-point
round-trips, table projection and invalid code points;
* negative fixtures for wrong types and table bounds;
* unit tests, `cargo fmt --check`, `cargo clippy -- -D warnings` and
`cargo test --workspace`.
@@ -0,0 +1,162 @@
# ADR-0024: Recoverable result convention (RFC-CM CM-A05)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A05, P2/P3),
ADR-0023
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-20-result-and-free-list-p2.md`
## Context
A standard library cannot be written in `.mq.md` if every expected condition
(no free block, invalid input, missing field, later end-of-file or a short
read) must abort the whole run. The core has structured `RuntimeError`s with
stable codes, but no way for an ordinary function to return a failure that the
caller can inspect and handle in a normal branch. RFC-CM-0002 §3.6 recommends a
`Result` data contract using existing `Map` values and control flow, without
`try`/`catch` or exception syntax.
This ADR freezes the **data convention**, not a new runtime type. A separate
core `Result` value, algebraic types or automatic propagation would be a
larger change and is deliberately not introduced.
## Decision
### 1. Shape
A recoverable result is an ordinary `Map`:
```text
success: {ok: true, value: <any Value>}
failure: {ok: false, code: <Text>, message: <Text> [, details: <Map>]}
```
* `ok` is always present and always `Bool`.
* On success, `value` is present. It may itself be `None`, a `List`, a `Map`,
a `Table`, a `Memory` handle or any other value.
* On failure, `code` and `message` are present. `code` is a lowercase
dot-separated stable identifier; `message` is human-readable text.
* `details` is optional and, when present, is a `Map` of additional structured
facts. It must not contain host pointers, resource handles or secret bytes.
* No other fields are required. Consumers must tolerate additive fields.
The constructors are ordinary `.mq.md` functions, for example:
```text
## 成功
`(值)`
*{ok: true, value: 值}*
## 失败
`(代码, 消息)`
*{ok: false, code: 代码, message: 消息}*
```
They are **not** builtins. A library may choose different local function names;
the map shape is the contract.
### 2. What is recoverable, what stays fatal
Recoverable (`Result`):
* a user allocator has no fitting block (`alloc.no_space`);
* a parser rejects malformed input or hits an unexpected end
(`parse.invalid_syntax`, `parse.unexpected_eof`);
* a codec rejects an invalid sequence (`codec.invalid_sequence`);
* a lookup that uses an explicit result because `None` is a legal value;
* later, a resource reports EOF/partial/pending as a **success payload** with
its own byte counts, not as a fatal error (P3 contract).
Fatal `RuntimeError` (unchanged):
* violating a language invariant: wrong operand type, out-of-bounds managed
collection access, unknown/stale `Memory` handle, memory budget exhaustion,
division by zero, call arity, unknown symbol;
* a capability denial (authorization is not a recoverable business result);
* an internal invariant failure.
A library must not convert a fatal invariant into an `Err` by catching it;
there is no catch mechanism and none is proposed here. It must validate inputs
before calling an operation whose failure is fatal, or rely on its own
pre-checks.
### 3. Codes and namespaces
`code` is stable within the library that defines it. Recommended prefixes:
| Prefix | Owner |
|---|---|
| `alloc.*` | allocator libraries |
| `parse.*` | parsers |
| `codec.*` | byte/text codecs |
| `table.*` | table algorithms |
| `io.*` | stream wrappers (P3, when the resource contract exists) |
Codes are compared as `Text`; a caller must not depend on the `message` text.
Libraries may add codes; removing or renaming one is a breaking change for that
library.
### 4. EOF and partial I/O (forward contract)
When the Stream/Resource API arrives (CM-A06), a successful read must
distinguish:
```text
{ok: true, value: {count: Int, eof: Bool, pending: Bool}}
```
EOF is not an `Err`; a short read is not an `Err`; a connection reset or a
permission failure is. The exact stream payload is frozen by CM-A06, not by
this ADR. This ADR only reserves the principle.
### 5. Why not a new type or syntax
* A dedicated `Value::Result` would change the value model, wire format,
display, equality and every exhaustive `Value` match for a convention that
existing `Map` plus `ok` already expresses.
* `try`/`catch` would add a control-flow construct that bypasses ordinary
branches, contradicts the RFC's preference for `Result` values and needs new
grammar, scope and error rules.
* Ordinary `Map` results are visible to `type`, `length`, `map.get` and wire
encoding today, and can be built by `.mq.md` libraries without a new ABI.
## Compatibility
* Additive: existing builtins and programs are unchanged.
* `map.get` returning `None` for a missing key remains a lookup convention,
not a `Result`; callers that need to distinguish a stored `None` use
`map.has` (ADR-0023).
* Runtime error categories and codes are unchanged.
* No wire version bump: a `Result` is an ordinary `Map` already covered by the
wire format.
## Non-goals
* No automatic `?`/propagation, exceptions, panic recovery or stack unwinding.
* No `Result` type, generics, pattern matching or exhaustive `Ok`/`Err`
typing.
* No change to `memory.grow` or other core operations from fatal errors to
`Result`; a library may pre-check and return `Err` instead.
* No resource/stream implementation; CM-A06 remains open.
* No promise that every existing library will adopt the convention.
## Consequences
* `.mq.md` algorithms can return and branch on failure without aborting the
run. This is a prerequisite for a Marqdo-written allocator, parser and
eventually `io.read_line`.
* Because the convention is untyped, a caller can inspect the wrong shape;
`类型(result)`, `map.has(result, "ok")` and the frozen field names are the
checks. A future typed-value ADR may improve this without breaking the data
shape.
* The allocator fixture in `54`/`55` demonstrates success and failure maps and
ordinary branching, with no Rust algorithm.
## Verification
`docs/evidence/2026-11-20-result-and-free-list-p2.md` records a `.mq.md`
first-fit/free-list allocator that returns `Result` maps, reuses freed blocks,
splits free blocks, coalesces address-adjacent free blocks and returns
`alloc.no_space` without aborting the run.
@@ -0,0 +1,173 @@
# ADR-0025: Minimal Resource/Stream ABI and byte transfer through `Memory` (RFC-CM CM-A06)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A06, CM-A09,
CM-A11, P3), ADR-0022, ADR-0024
Specs: `spec/evaluation.md`, `spec/security.md`, `spec/value-wire.md`
Evidence: `docs/evidence/2026-11-20-resource-stream-p3.md`
## Context
P1/P2 give `.mq.md` managed values, addressable `Memory`, structure primitives,
a recoverable `Result` convention and a substantial set of in-language
algorithms. Nothing yet lets a standard-library function perform a real byte
transfer to the outside world without a dedicated `HostOp` per high-level
function.
CM-A06 asks for a stable, authorized, versioned resource boundary: obtain a
granted resource, call the operations it promises, release it. CM-A09 says to
extend the existing host/ABI rather than build a second plugin system. CM-A11
says strict must stay fail-closed until a provider is actually verified.
This ADR freezes a **first, deliberately small slice**:
* one opaque host-issued `ResourceHandle` value;
* `resource.open/read/write/close` as pure language builtins, not new `HostOp`
variants;
* byte transfer through `Memory`, never through host pointers;
* stdio `stdout` write and project-file **read** only;
* `Result` maps for EOF/closed/write counts, fatal errors for invalid handle
or capability denial.
File write, directories, network, processes, TLS, asynchronous I/O and a
generic provider registry remain open.
## Decision
### 1. Opaque handle
`Value::ResourceHandle(ResourceHandle)` is a host-issued opaque identity with
private `(run_id, resource_id, generation)` fields and a hidden constructor.
It is not `Serialize`/`Deserialize`, is rejected by the wire, is never
synthesized from a number or a JSON value, and renders as `Resource` in
`display`/`repr` without exposing the id.
It is a **separate domain** from `Memory`: a handle is not indexable, not
byte-addressable and not a capability token the user can manufacture.
### 2. First resource kinds
| Kind | Descriptor | Rights | Availability |
|---|---|---|---|
| `stdio` | `stdout` | `write` | `pure` and `project` when the console grant is present; `strict` rejects until verified |
| `file` | project-relative path | `read` | `project` with an `fs_read` grant |
`file` streams are opened by reading a bounded regular file (existing
`ProjectFs::MAX_READ_BYTES = 8 MiB` limit) into host-owned bytes; `read`
returns successive chunks and `eof` when the position reaches the end. This is
honest about being a first provider: it does not expose a host file descriptor,
does not support seeking, writing, directories, network, processes, devices or
paths outside the project root.
Any other kind/descriptor/right combination is an explicit
`resource.unsupported`/`permission.denied` diagnostic, never a silent fallback.
### 3. Builtins
```text
resource.open(kind: Text, descriptor: Text, rights: List) -> Resource
resource.read(handle: Resource, memory: Memory, offset: Int, max_bytes: Int)
-> {ok:true, value:{count:Int, eof:Bool}} | {ok:false, code, message}
resource.write(handle: Resource, memory: Memory, offset: Int, length: Int)
-> {ok:true, value:{count:Int}} | {ok:false, code, message}
resource.close(handle: Resource)
-> {ok:true, value:None} | {ok:false, code, message}
```
* `resource.open` is effectful and capability-checked at execution time.
* `resource.read` validates `offset` and the remaining buffer **before** asking
the provider, so a failed range check cannot consume stream data. It caps the
request to the remaining region, writes the returned bytes into `Memory` and
returns the actual count and EOF flag. EOF is a **success** with `eof:true`.
* `resource.write` copies bytes out of `Memory` only for the duration of the
call. A short write is reported as a count; the caller loops. It never
stores a host pointer into a region that could later grow.
* `resource.close` closes the host slot. Reading/writing/closing an already
closed handle returns `{ok:false, code:"resource.closed", ...}` (recoverable).
A forged, cross-run or stale handle is `runtime.resource` (fatal).
* All four are pure interpreter operations over host trait methods; they add no
`HostOp` variant. Invalid capacity/bounds remains `runtime.index`; wrong
operand types remain `runtime.type`.
### 4. Authorization and modes
* Authorization is the intersection of the requested operation, the effective
`Capabilities` grant and the mode:
* `stdio/stdout/write` requires `console` (the same grant `打印` uses);
* `file/.../read` requires an `fs_read` pattern match and project-root path
validation (`..`, absolute paths and symlink escapes rejected);
* `pure`: stdio stdout is allowed because console is already a pure-mode
console grant; **file resources are refused by the mode check**, even if a
caller constructs a `Capabilities` set with a matching `fs_read` pattern.
Network/process resources do not exist in this slice.
* `project`: stdio stdout and explicitly granted project-file reads.
* `strict`: **every** `resource.open` is refused with `permission.denied`.
Strict L1 is unchanged and no new provider is silently enabled. A future ADR
must add real OS-level evidence before strict accepts streams.
* Manifest capability requests are not grants; only the operator-provided
effective set is consulted.
### 5. Errors, budgets and tracing
New stable categories:
| Code | Meaning |
|---|---|
| `runtime.resource` | forged/cross-run/stale handle, invalid resource state |
| `resource.closed` | read/write/close on an already closed handle (recoverable) |
| `resource.unsupported` | kind/descriptor/rights not implemented in this slice |
`resource.write` to stdout charges the existing output budget and appends to
the host output buffer exactly like `打印`; capability denial is
`permission.denied`. Trace records `resource.open`, `resource.read`,
`resource.write` and `resource.close` with kind/count/eof metadata, never raw
file bytes or handles.
### 6. Wire and boundary
* `to_wire(Resource)` emits `{"type":"resource"}` only; decoding it is a hard
error (same rule as `Memory`).
* The generic `Host::invoke` boundary rejects arguments containing a resource
handle, so plugins/foreign kernels/file adapters never receive one.
* Plugin output cannot mint a handle: the constructor is host-only and the
value is not decodable from plugin wire.
### 7. Non-goals
* No `print` migration in this ADR; that is CM-A07 and a later increment.
* File **write** is added by [ADR-0026](0026-buffered-project-file-write-streams.md),
not this decision; this ADR remains read-only for files.
* No file write, directory listing, rename, metadata, network, DNS, TLS,
process, clock, random or device operations.
* No generic third-party provider registration; CM-A09 remains open.
* No zero-copy, async, deadlines per stream, seek or shared memory.
* No strict-mode resource support and no claim that host-in-process I/O is a
sandbox.
## Compatibility
* Additive value variant; old programs never produce `Resource`.
* New builtins only; existing `fs.read/write`, `打印`, plugin and foreign
behavior is unchanged.
* No wire version bump: the new tag is rejected, not accepted.
* New error categories are additive; existing categories/codes are unchanged.
## Consequences
* `.mq.md` can implement `write_text`-style output by encoding text into
`Memory` (fixture 59) and calling `resource.write`, and can implement
chunked file readers with `resource.read`/`Result`/EOF.
* The remaining high-level I/O surface can grow in the standard library without
a new `HostOp` per function.
* The resource table is host state per run; closed slots are not reused in this
slice, so generation checks are straightforward.
* Strict remains conservative; `.mq.md` resource code runs in `pure` (stdout)
or `project` (file read) only.
## Verification
`docs/evidence/2026-11-20-resource-stream-p3.md` records unit tests for grants,
mode refusal, traversal rejection, forged/closed handles, output budget,
file chunking/EOF, wire rejection and a `.mq.md` stdout fixture.
@@ -0,0 +1,115 @@
# ADR-0026: Buffered project-file write streams (CM-A06 extension)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A06, CM-A09, CM-A11), ADR-0024, ADR-0025
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-resource-file-write-p3.md`
## Context
ADR-0025 froze the first resource ABI slice with `stdio/stdout` write and
project-file **read**. A standard library cannot implement file copy, file
format writers or "write then verify" without a write-capable file resource.
The RFC's P3 target is explicit `open/read/write/close` for project files in
`project` mode; strict must remain fail-closed.
Direct per-write file I/O would expose partial writes and OS error timing to
every caller and would make it easy to leave a half-written project file. This
ADR adds the smallest honest write path: an in-memory buffer per file writer,
flushed through the existing project path policy on `close`.
## Decision
### 1. Provider shape
Add a third stream shape to the host resource table (ADR-0025):
```text
resource.open("file", "<project-relative path>", ["write"]) -> Resource
```
* `project` mode only, with an `fs_write` glob grant matching the normalised
relative path.
* `pure` still refuses file resources by mode check; `strict` still refuses
every `resource.open` (CM-A11 fail-closed).
* Absolute paths, `..`, symlink escapes and non-regular targets are rejected by
`ProjectFs` exactly like the existing `fs.write`.
* A writer owns an in-memory byte buffer. `resource.write` appends bytes and
returns `{ok:true, value:{count}}`; it does **not** touch the filesystem.
* `resource.close` flushes the buffer through `ProjectFs::write_bytes` and then
marks the handle closed. If the flush fails, the handle stays open with its
buffer intact and the error is reported; the caller may retry or close again.
* If the program exits without closing, no file is written. This is a
deliberate "no flush on drop" rule: a crash or a missing `close` must not
silently publish a partial file.
* `resource.write` is rejected on a file reader; `resource.read` is rejected on
a file writer; stdout remains write-only. Wrong-direction use is
`resource.unsupported`.
* A writer buffer is capped at the existing 8 MiB project read/write ceiling
(`MAX_READ_BYTES`). Exceeding it is `resource.exhausted` and the append is
atomic: the buffer and file are unchanged.
### 2. Rights and modes
| Kind | Descriptor | Rights | Mode | Grant |
|---|---|---|---|---|
| `file` | project-relative path | `["read"]` | project | `fs_read` glob |
| `file` | project-relative path | `["write"]` | project | `fs_write` glob |
| `stdio` | `stdout` | `["write"]` | pure/project | `console` |
Mixed rights (`["read","write"]`) and other kinds/rights are
`resource.unsupported` in this slice. There is no truncate/append/seek flag:
every writer starts empty and replaces the file on flush.
### 3. Errors, budgets and tracing
* `resource.closed` for read/write/close on a closed handle (recoverable).
* `runtime.resource` for forged/cross-run/stale handles (fatal).
* `resource.exhausted` for a buffer over the byte ceiling.
* `permission.denied` for a missing mode/grant or a path policy failure.
* `resource.unsupported` for a direction/rights combination not implemented.
* `resource.write` on a file writer does not charge the program output budget;
the buffer is bounded by the file ceiling. `resource.close` performs one
`ProjectFs::write_bytes` call and records `resource.close` with the byte
count, never the contents.
* No partial file is visible on a failed close because the write is a single
`std::fs::write` to the validated project path (the same atomicity level as
the existing `fs.write`; no temp-file rename is claimed).
### 4. Explicit non-goals
* No append/truncate/seek flags, no random access, no directory or metadata
operations.
* No file write in pure or strict mode.
* No network/process/device/clock/random resources.
* No generic provider registry (CM-A09), no async/nonblocking I/O.
* No claim that `ProjectFs::write_bytes` is crash-atomic; it is a single
validated write, not a transactional replace.
* No CM-A07 `print` migration in this ADR.
## Compatibility
* Additive to ADR-0025: existing `file` read and `stdio` write behavior is
unchanged; `file` write was previously `resource.unsupported`.
* No new `Value` variant, `HostOp`, Markdown node or wire shape.
* Error categories/codes are unchanged; the new `resource.exhausted` diagnostic
reuses the existing stable `resource.exhausted` code family.
## Consequences
* `.mq.md` can now implement file copy and bounded format writers by opening a
read stream and a write stream, transferring bytes through `Memory`, and
closing both. This is C09's core shape without a `copy_file` Rust builtin.
* Buffered close gives deterministic "all or nothing within one write call"
behavior and keeps OS write errors at a point where the caller still has a
Result path.
* Large files require chunked writes; the 8 MiB ceiling is the first-slice
limit, not a permanent design.
## Verification
`docs/evidence/2026-11-20-resource-file-write-p3.md` records host unit tests
(grants, pure/strict refusal, traversal, buffer ceiling, retry-after-failure),
CLI tests for write-then-read and missing grants, and the full workspace gate.
+103
View File
@@ -0,0 +1,103 @@
# ADR-0027: CM-A07 print-migration staging and the Compat-A `.mq.md` stdlib prototype
Status: **Accepted (Compat-A only; Compat-B/C pending)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A07, CM-A10, P3), ADR-0023, ADR-0024, ADR-0025, ADR-0026
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-20-stdlib-print-compat-a.md`
## Context
RFC-CM-0002 §5.4 and CM-A07 require `打印` to stop being a hidden Rust
`Value::display()` path. The migration has three compatibility stages:
* **Compat-A**: a real `.mq.md` print implementation exists and is callable
through explicit imports; the legacy builtin remains the default.
* **Compat-B**: the `print/打印` name routes to the stdlib implementation
(or an alias table maps it), while old behavior is comparison-tested.
* **Compat-C**: the legacy `HostOp::Print` formatting path is removed from the
public builtin implementation.
This ADR freezes the staging policy and implements **Compat-A only**. It does
not change `打印` resolution, does not remove `HostOp::Print`, and does not
claim CM-A07 complete.
## Decision
### 1. Stage policy
* A stage is only complete when its comparison evidence exists:
* A: explicit stdlib functions exist and their output is compared with the
builtin for the covered value set;
* B: name resolution tests prove a single implementation is selected and old
callers still behave;
* C: all old goldens pass with no `HostOp::Print` formatting path.
* Stages are additive; no stage may silently make a second implementation
reachable for the same name.
* `strict` behavior is unchanged by stdlib-only code; a stdlib print still
needs the console/resource grant.
### 2. Compat-A implementation
Add a normal `.mq.md` module package under `examples/stdlib/`:
```text
examples/stdlib/format.mq.md # 显示 / 列表显示 / 映射显示 / 表格显示 / 转义文本
examples/stdlib/text.mq.md # utf8编码 (Memory bytes)
examples/stdlib/io.mq.md # 写文本 / 写值 (resource.open stdout + write/close)
examples/stdlib/demo.mq.md # side-by-side comparison with 打印
```
`io.写值(值)` = `format.显示(值) + "\n"` → `text.utf8编码` → `Memory` →
`resource.open("stdio","stdout",["write"])` → `resource.write` → `close`.
This uses only the public core/resource contracts; it is ordinary imported
`.mq.md` code, not a new builtin.
Covered value types: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, `List`,
`Map`, `Table`. Formatting matches the builtin `Value::display()` for the
checked corpus, including Text quoting/escaping inside collections, canonical
map key order and table layout.
**Not covered yet**: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The
language currently has no public observation for the module/name/handle
identity needed to reproduce `Value::display()`/`repr()` for those types.
Adding such observation would be a CM-A04 extension and is out of scope.
### 3. Compatibility rules
* `打印` remains the default builtin in this slice. `io.写值` is only reachable
through an explicit import alias; there is no implicit module, bundled stdlib
resolution or global name shadowing.
* No `HostOp` is added or removed. Both paths write through the console/output
budget; the stdlib path additionally goes through the resource ABI.
* Stdlib modules are ordinary Markdown documents: no special parser, no
embedded Rust formatter, no direct formatting syscall.
* A future Compat-B ADR must define name precedence, bundled module lookup,
stdlib version/digest (CM-A10) and failure behavior; this ADR does not.
### 4. Non-goals
* No `print` alias routing, no removal of `HostOp::Print`.
* No bundled stdlib install/version/digest or offline distribution.
* No formatting of Formula/Code/FunctionRef/PluginHandle.
* No locale-aware, width-aware or float-format changes.
* No change to Markdown rendering, `strict`, or the wire/ABI.
* No claim that the examples directory is the shipped standard library.
## Consequences
* The target call chain (`format.display` → `utf8_encode` → `write_all` →
`resource.write`) is exercised end to end in `.mq.md` and can be compared to
the builtin.
* The missing observation for `Formula/Code/FunctionRef/PluginHandle` is now an
explicit, documented blocker for Compat-B rather than an implicit assumption.
* Because the stdlib is not bundled, ordinary users still get the builtin; the
prototype does not yet reduce the runtime's formatting surface.
## Verification
`docs/evidence/2026-11-20-stdlib-print-compat-a.md` records the side-by-side
demo (`io.写值` vs `打印` for a nested list and a table), the CLI test asserting
byte-identical lines, and the full workspace gate. It also records the covered
and uncovered value types.
+38
View File
@@ -0,0 +1,38 @@
# Architecture RFCs: compute machine and stdlib migration
Status: **Draft RFCs, partially implemented**. This page is an index; the
normative RFC text stays at its reviewable path under `docs/`.
| Document | Status | Implemented part |
|---|---|---|
| [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). |
| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict refuses all opens; ADR-0027 Compat-A `.mq.md` print prototype, builtin still default). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser for arrays, objects, strings with escapes, integers/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68). No separate stdlib repo, JSON floats, network/generic providers, P4 Table migration, P5 providers or P6 performance work. |
The RFCs are architecture constraints, not evidence that a feature exists. The
implementation status lives in `docs/STATUS.md`; frozen behavior is in
`spec/`; decisions are in `docs/adr/`.
## Decisions already frozen
| RFC item | ADR | State |
|---|---|---|
| CM-A01 Memory identity/lifetime | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
| CM-A02 page size, growth, budgets | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
| CM-A03 place assignment and evaluation order | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
| CM-A04 minimal structure observation/construction | [0023](../adr/0023-minimal-structure-observation-and-construction.md) | Accepted |
| CM-A05 recoverable `Result` convention | [0024](../adr/0024-recoverable-result-convention.md) | Accepted |
| CM-A06 minimal Resource/Stream ABI | [0025](../adr/0025-minimal-resource-stream-abi.md) | Accepted (first slice) |
| CM-A06 buffered project-file write | [0026](../adr/0026-buffered-project-file-write-streams.md) | Accepted |
| CM-A07 print migration staging / Compat-A | [0027](../adr/0027-print-migration-compat-a.md) | Accepted (Compat-A only) |
| CM-A07 Compat-B/C, stdlib packaging, Table storage, generic providers, strict resources | — | Not decided; do not implement by implication |
## What a future phase must not do
* add a top-level CLI command or Markdown keyword for memory, allocation or I/O;
* add a `HostOp` for every new stdlib function;
* turn managed `List`/`Map`/`Table` into aliased mutable storage;
* expose a host pointer, let a handle cross the wire, or claim block-level
safety for user allocators;
* make `strict` accept an unverified provider by default;
* claim an RFC phase is complete without the closure and negative evidence its
phase gate requires.
+130
View File
@@ -0,0 +1,130 @@
# RFC-CM P1 evidence: addressable Memory and place assignment
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline commit: `a113d894962d8c00eb02efb42e0e73f8b084a1fa` (`docs: bind
README/COMPATIBILITY/CHANGELOG to the convergence SHA`)
Working tree: RFC-CM P1 changes, **not committed** at the time of this report.
Decision record: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md)
Scope: RFC-CM-0001 R1 / RFC-CM-0002 P1 only. P2–P6 are **not** implemented.
## Environment
* Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
* `PATH=/root/.cargo/bin:$PATH`
* `CARGO_HOME=/root/work/marqdo-cli/.cache/cargo`
* `CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target`
* No GitHub credentials in this checkout; this is **local** evidence, not a
GitHub Actions result. Windows/macOS lanes were not run.
## Static and workspace gate
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests)
```
The 2 ignored tests are the existing `marqdo-plugin-sdk` doc-tests, not new
memory tests.
## Focused evidence
```text
cargo test -p marqdo-runtime --lib memory
8 passed; 0 failed
regions_are_isolated_and_handles_share_identity
create_is_zero_filled_and_reports_pages
grow_keeps_old_bytes_and_zero_fills_new_pages
failed_grow_does_not_change_region_or_budget
invalid_offsets_are_structured_errors
create_validates_page_counts_and_budget
forged_and_stale_handles_are_rejected
page_arithmetic_never_overflows
cargo test -p marqdo-runtime --test interpreter memory
3 passed; 0 failed
memory_region_is_shared_by_handle_identity
memory_handle_cannot_reach_the_host_boundary
memory_budget_is_enforced_without_changing_the_region
cargo test -p marqdo-value --lib memory
2 passed; 0 failed
memory_display_never_leaks_the_runtime_handle
memory_handles_are_not_transferable
cargo test -p marqdo-expression --lib memory_index
1 passed; 0 failed
memory_index_assignment_parses_as_a_place
cargo test -p marqdo-cli \
--test conformance_semantics --test negative \
--test analysis_trace --test examples_r6
semantic_goldens_match_end_to_end_execution ok
runtime_negative_goldens ok
analysis_v2_projects_memory_place_targets_additively ok
addressable_memory_example_runs_through_the_cli ok
cli_memory_budget_override_fails_closed ok
(20 tests total in these four binaries, 0 failed)
```
## End-to-end CLI
```text
$ marqdo run examples/memory-buffer.mq.md --json
{"success": true, "value": {"type": "int", "decimal": "141"}, ...}
$ marqdo run examples/memory-buffer.mq.md --max-memory 0 --json
{"success": false, "error": {"code": "resource.exhausted", ...}, ...}
```
The example creates one 64 KiB page with a declared maximum of four pages,
writes two bytes through two names that share one region, and returns
`42 + 99 = 141`.
## Fixtures
Semantic goldens:
* `tests/conformance/semantics/45-memory-buffer.mq.md` — shared identity
(`另一个名字 = 区域`) and two byte writes.
* `tests/conformance/semantics/46-memory-grow-zero-fill.mq.md` — zero-page
create, `grow` returning the old count, old bytes preserved, new byte zero,
`memory.size`.
* `tests/conformance/semantics/47-memory-isolation-and-alias.mq.md` — two
regions do not alias at the same offset; handle equality is identity.
Runtime negatives:
| Fixture | Expected code |
|---|---|
| `13-memory-out-of-bounds` | `runtime.index` |
| `14-memory-byte-range` | `runtime.type` |
| `15-index-assign-non-memory` | `runtime.type` |
| `16-memory-huge-offset` | `runtime.index` |
| `17-memory-grow-beyond-max` | `resource.exhausted` |
| `18-memory-negative-pages` | `runtime.memory` |
## What this does not prove
* No `.mq.md` stdlib repository or allocator/closure fixture yet (RFC-CM P2).
* No `Resource`, `Stream`, byte-transfer ABI, stdlib `print` migration or
`Table` storage abstraction (P3–P5).
* No Windows/macOS evidence and no remote CI URL.
* No claim of block-level memory safety for a user-written allocator; the
runtime only bounds each access to its region.
* No claim that `strict` accepts new I/O: memory is pure computation and no
`HostOp` or provider was added.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace --locked
cargo run -p marqdo-cli -- run examples/memory-buffer.mq.md --json
```
@@ -0,0 +1,74 @@
# RFC-CM P2 partial evidence: `.mq.md`-written codec, allocator and list
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes.
Related: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md),
[P1 evidence](2026-11-20-memory-p1.md),
[RFC-CM-0002 §7.2](../../docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md).
Scope: **partial P2 only**. The RFC-CM-0002 P2 gate is **not** passed.
> Historical slice: this report predates ADR-0023 (CM-A04 structure API),
> ADR-0024 (CM-A05 Result), the coalescing allocator (fixture 55) and the
> binary tree/CSV fixtures (56–57). Its "what remains" list is superseded by
> [`2026-11-20-structure-api-p2.md`](2026-11-20-structure-api-p2.md) and
> [`2026-11-20-result-and-free-list-p2.md`](2026-11-20-result-and-free-list-p2.md);
> the P2 gate itself is still not passed.
## What was added without Rust algorithm code
Three semantic fixtures are ordinary `.mq.md` documents. They call only
language builtins (`memory.new/size/grow`), indexing, arithmetic, functions,
branches and loops. No Rust crate was changed for the algorithms themselves.
| Fixture | `.mq.md` subject | Result |
|---|---|---|
| `tests/conformance/semantics/48-memory-bump-allocator.mq.md` | big-endian u32 encode/decode and a 4-byte-header bump allocator; allocates 8, 16, 8 bytes and writes a byte | `[4, 12, 28, 36, 255]` |
| `tests/conformance/semantics/49-memory-linked-list.mq.md` | allocates three 8-byte nodes (`value:u8`, `next:u32`), links them and traverses the list | `[4, 12, 20, 60]` |
| `tests/conformance/semantics/50-memory-allocation-failure-atomic.mq.md` | a too-large allocation returns `None` without moving the header; the next allocation still starts at 4 | `[None, 4, 4, 65536]` |
Observed CLI results (all `success: true`):
```text
48: [4, 12, 28, 36, 255]
49: [4, 12, 20, 60]
50: [None, 4, 4, 65536]
```
`cargo test -p marqdo-cli --test conformance_semantics` passes with these three
fixtures included.
## Why this is not the P2 gate
RFC-CM-0002 §9/P2 requires C01–C06 and C11 to pass in an environment with
Foreign/native plugins disabled, and requires a minimal API for dynamic
List/Map/Table construction and observation plus recoverable errors. This
slice does **not** provide:
* duplicate-free/allocator adversarial tests (C01 now has a first-fit
free-list allocator with reuse, splitting and coalescing, fixture 55);
* graph/hash-table structures (C02 now has a singly linked list and a binary
search tree, fixtures 49 and 56, but no graph/hash table);
* general List/Map/Table construction, mutation-free rebuilding or key
enumeration (CM-A04 is undecided);
* a `Result`/recoverable-error contract for parser/IO/EOF/partial conditions
(CM-A05 is undecided);
* an independent `marqdo-stdlib` repository, version/digest contract or
frozen-CLI-binary closure lane (the RFC §10.5 experiment).
It also does not add a stdlib API to the CLI; the fixtures are conformance
documents, not a shipped standard library.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
for f in 48-memory-bump-allocator 49-memory-linked-list \
50-memory-allocation-failure-atomic; do
cargo run -q -p marqdo-cli -- run \
tests/conformance/semantics/$f.mq.md --json
done
cargo test -p marqdo-cli --test conformance_semantics
```
@@ -0,0 +1,104 @@
# RFC-CM CM-A06 extension evidence: buffered project-file writes
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes.
Decision record: [ADR-0026](../adr/0026-buffered-project-file-write-streams.md)
Related: [ADR-0025](2026-11-20-resource-stream-p3.md),
[STATUS §5](../STATUS.md).
Scope: project-file **write** extension only; no network/process/directory.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
406 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Implemented behavior
`resource.open("file", "<project-relative path>", ["write"])`:
* project mode only; requires an `fs_write` glob grant and the same
`ProjectFs` path policy (reject absolute, `..`, symlink escapes);
* pure mode refuses file resources by mode; strict refuses every resource open;
* the writer owns an in-memory buffer; `resource.write` appends and returns
`{ok:true, value:{count}}` without touching the filesystem;
* `resource.close` flushes through `ProjectFs::write_bytes` and then marks the
handle closed; a failed flush leaves the handle open with its buffer;
* an unclosed writer never creates the file (no flush on drop);
* buffer ceiling is the existing 8 MiB file limit; an oversized append is
`resource.exhausted` and leaves both buffer and file unchanged;
* `resource.read` on a writer, and `resource.write` on a reader, are
`resource.unsupported`;
* no new `Value`, `HostOp`, Markdown node or wire shape.
## Tests
### Host integration (`crates/marqdo-host/tests/resource_streams.rs`, 9)
```text
pure_mode_writes_stdout_through_a_resource
stdout_write_obeys_the_output_budget_atomically
strict_mode_refuses_every_resource_open
resource_handles_cannot_cross_the_generic_host_boundary
pure_mode_refuses_file_resources_even_with_a_grant
project_file_writes_buffer_until_close_and_require_a_grant
file_write_buffer_has_a_hard_ceiling
project_file_reads_are_chunked_and_require_a_grant
traversal_and_unsupported_rights_are_refused
```
The write test asserts the file does not exist before `close`, appears with
`abcdef` after `close`, closed read/close return `resource.closed`, and the
same open without `fs_write` is `permission.denied`.
### CLI integration (`crates/cli/tests/resource_streams.rs`, 4)
```text
project_file_stream_writes_on_close_then_reads_back
run index.mq.md --allow-write out.bin --allow-read out.bin --json
write count 3, close ok, read count 3 eof true, bytes 65/66/67
out.bin on disk = "ABC"
file_copy_is_written_in_mqmd_through_chunked_streams
.mq.md opens data.txt for read and out.bin for write,
loops resource.read(16) / resource.write(count) until eof,
closes both; total 6; out.bin on disk = "abcdef"
(C09 file copy without a Rust copy_file builtin)
project_file_stream_reads_chunks_into_memory
file_stream_requires_a_read_grant
```
### Host table unit tests (`crates/marqdo-host/src/resource.rs`, 3)
Chunked EOF, forged/closed handles, stdout-not-readable and file-not-writable
(read-only direction) remain covered; the writer uses the same handle checks.
## What this does not prove
* No append/truncate/seek, directories, metadata, network, process, clock,
random, device or generic provider registry.
* No crash-atomic replace: `ProjectFs::write_bytes` is one validated write, not
a temp-file rename.
* No file write in `pure` or `strict`.
* No CM-A07 stdlib `print` migration.
* No Windows/macOS evidence and no remote CI URL.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-host --test resource_streams
cargo test -p marqdo-cli --test resource_streams
cargo test --workspace --locked
```
@@ -0,0 +1,144 @@
# RFC-CM CM-A06 evidence: minimal Resource/Stream ABI
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes.
Decision record: [ADR-0025](../adr/0025-minimal-resource-stream-abi.md)
Related: [P1](2026-11-20-memory-p1.md),
[P2 result/allocator](2026-11-20-result-and-free-list-p2.md),
[STATUS §5](../STATUS.md).
Scope: **first CM-A06 slice only**. Not the full P3 migration.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
402 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Implemented slice
### Value and wire
* `Value::Resource(ResourceHandle)` is an opaque host-issued handle; fields are
private, the constructor is host-only, and `display`/`repr` render only
`Resource`.
* `to_wire(Resource)` emits only `{"type":"resource"}`; `from_wire` rejects the
tag. A `Resource` inside a list/map/table is equally rejected on decode.
* The generic `Host::invoke` boundary rejects resources before dispatch, like
`Memory`. Plugins/foreign kernels never receive either handle kind.
### Builtins
```text
resource.open(kind, descriptor, rights) -> Resource
resource.read(handle, memory, offset, max_bytes) -> Result {count, eof}
resource.write(handle, memory, offset, length) -> Result {count}
resource.close(handle) -> Result
```
* `read` validates the destination region **before** reading from the provider
and caps the request to the remaining bytes.
* EOF is `{ok:true, value:{count:0, eof:true}}`, not an error.
* Closed handles return `{ok:false, code:"resource.closed"}`; forged/cross-run/
stale handles are fatal `runtime.resource`; unsupported kind/rights are
`resource.unsupported`; missing grant is `permission.denied`.
* Bytes move only as copied `Memory` ranges for the duration of the call. No
host pointer is stored in a region.
### Providers in this slice
| Kind | Descriptor | Rights | Mode | Behavior |
|---|---|---|---|---|
| `stdio` | `stdout` | `write` | pure/project | appends to the program output buffer after the output-budget check |
| `file` | project-relative path | `read` | project | bounded regular file loaded at open; chunked reads with EOF |
| anything else | — | — | — | explicit `resource.unsupported` |
* `strict` refuses every `resource.open` with `permission.denied` (CM-A11
fail-closed); no provider is silently enabled.
* `pure` allows only stdio stdout (console grant), exactly the existing
console exception; file resources are refused by an explicit mode check,
even if a caller hand-constructs a matching `fs_read` grant.
* File paths still go through `ProjectFs::resolve` (absolute paths, `..`,
symlink escapes rejected) and require an `fs_read` glob grant.
* File size is bounded by the existing 8 MiB read limit.
* `resource.write` to stdout charges the existing output budget; a budget
failure produces no output. There is no separate I/O-byte budget yet.
## Tests
### Host integration (`crates/marqdo-host/tests/resource_streams.rs`, 7)
```text
pure_mode_writes_stdout_through_a_resource
stdout_write_obeys_the_output_budget_atomically
strict_mode_refuses_every_resource_open
resource_handles_cannot_cross_the_generic_host_boundary
pure_mode_refuses_file_resources_even_with_a_grant
project_file_reads_are_chunked_and_require_a_grant
traversal_and_unsupported_rights_are_refused
```
### Host table unit tests (`crates/marqdo-host/src/resource.rs`, 3)
```text
file_reads_are_chunked_with_eof
forged_and_closed_handles_are_rejected
stdout_is_not_readable_and_file_is_not_writable
```
### CLI integration (`crates/cli/tests/resource_streams.rs`, 2)
```text
project_file_stream_reads_chunks_into_memory
data.txt = "abcdef"
run index.mq.md --allow-read data.txt --json
results: counts 2/4/0, eof false/true/true, bytes 97..102
resource.close => ok:true
file_stream_requires_a_read_grant
same document without --allow-read
exit 1, error.code = permission.denied, no bytes read
```
### Semantic fixture 66
`tests/conformance/semantics/66-resource-stdout-write.mq.md`:
* `.mq.md` UTF-8 encoder writes `"你好,资源!\n"` into `Memory`;
* `resource.open("stdio","stdout",["write"])`;
* `resource.write` reports count 19;
* `resource.close` succeeds;
* a second `resource.read` on the closed handle returns
`{ok:false,code:"resource.closed"}`;
* captured program stdout is exactly `你好,资源!\n`.
## What this does not prove
* No file write, directory listing, rename, metadata, network, DNS, TLS,
process, clock, random or device resources.
* No generic third-party provider registry (CM-A09) and no Worker/ABI byte
transfer.
* No stdlib `print` migration (CM-A07); `打印` still uses `HostOp::Print`.
* No strict-mode stream support. Strict L1 is unchanged and refuses opens.
* No async/nonblocking/seek/zero-copy and no per-stream deadline.
* No Windows/macOS native lane and no remote CI URL.
* Host-in-process I/O is not an OS sandbox; project mode remains cooperative.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-host
cargo test -p marqdo-cli --test resource_streams
cargo test -p marqdo-cli --test conformance_semantics
cargo test --workspace --locked
```
@@ -0,0 +1,361 @@
# RFC-CM CM-A05 evidence: `Result` convention and first-fit free-list allocator
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes.
Decision records: [ADR-0024](../adr/0024-recoverable-result-convention.md),
[ADR-0023](../adr/0023-minimal-structure-observation-and-construction.md)
Related: [P1](2026-11-20-memory-p1.md),
[P2 closure](2026-11-20-memory-p2-closure.md),
[structure API](2026-11-20-structure-api-p2.md).
Scope: CM-A05 convention, a `.mq.md` first-fit allocator, and the later P2
fixtures 56–59 (binary tree, CSV splitter, hash table, UTF-8 encoder). Full P2
gate still **not** passed.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## `Result` convention (ADR-0024)
Success and failure are ordinary `Map` values:
```text
success: {ok: true, value: <Value>}
failure: {ok: false, code: <Text>, message: <Text>[, details: <Map>]}
```
No `Result` value type, exception, `try`/`catch` or automatic propagation was
added. Expected conditions return this shape; wrong types, out-of-bounds,
invalid/stale handles, budget exhaustion and capability denial remain fatal
`RuntimeError`s.
## Allocator fixture 55
`tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md` is a
pure `.mq.md` first-fit allocator over `Memory`:
* 8-byte block header: total block size (`u32`) and free-list next (`u32`);
* region header: free-list head at offset 0, bump pointer at offset 4;
* allocation rounds the request up to an 8-byte multiple;
* first fit walks the free list, splits a fitting block when the remainder is
at least 16 bytes, otherwise unlinks it;
* no suitable free block falls back to the bump pointer;
* no space returns `{ok:false, code:"alloc.no_space", message:...}`;
* the free list is address-sorted; `释放` inserts a block and coalesces it
with an address-adjacent successor and predecessor without erasing payload
bytes.
Observed result (compact wire values):
```text
甲 (8 bytes) -> {ok:true, value:16}
乙 (16 bytes) -> {ok:true, value:32}
丙 (8 bytes) -> {ok:true, value:16} # reuses 甲's freed block
丁 (4 bytes) -> {ok:true, value:16} # reuses the same block
戊 (16 bytes) -> {ok:true, value:32} # reuses 乙's freed block
己 (64 bytes) -> {ok:true, value:56}
庚 (8 bytes) -> {ok:true, value:56} # split from 己
辛 (16 bytes) -> {ok:true, value:72} # split remainder
壬 (8 bytes) -> {ok:true, value:96} # split remainder
癸 (8 bytes) -> {ok:true, value:112} # exact remainder
failure -> {ok:false, code:"alloc.no_space",
message:"没有足够的空闲空间"}
payload bytes -> 44, 55, 66, 77 at 庚/辛/壬/癸
```
A second region isolates coalescing from bump allocation:
```text
甲2 8B, 乙2 8B, 丙2 8B -> payloads 16, 32, 48
fill 65472B -> consumes the rest of the 64 KiB region
free 甲2, 乙2, 丙2 -> address-adjacent blocks merge to one 48-byte block
丁2 40B (need 48) -> {ok:true, value:16} # only the merged block fits
戊2 8B -> {ok:false, alloc.no_space} # bump space exhausted
```
Without coalescing, `丁2` would see only three 16-byte fragments and the
exhausted bump pointer, so it would fail; the observed success at offset 16
proves the merge.
The payload addresses do not overlap:
`庚=56..71`, `辛=72..95`, `壬=96..111`, `癸=112..127`. The earlier bytes
`甲=16..23`, `乙=32..47`, `戊=32..47` show deliberate reuse after `释放`
(the block is not zeroed; callers must initialise a block they allocate).
`cargo test -p marqdo-cli --test conformance_semantics` passes with fixture 55
included. `cargo run -p marqdo-cli -- run
tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md --json`
returns `success: true` with the checked-in expected value.
## Binary tree fixture 56
`tests/conformance/semantics/56-memory-binary-tree.mq.md` is a second
`.mq.md` structure over `Memory`:
* 16-byte nodes: one-byte value at offset 0, left child offset at 4, right
child offset at 8; `0` is the empty child;
* region header at offset 0 is the bump pointer, initialised to 8;
* `插入` is recursive and writes the child link through its parent block;
* `遍历` is recursive in-order and builds the result with `list.concat`;
* inserted `[5, 3, 8, 1, 4, 7, 9]`, returned:
```text
root=8
in-order=[1,3,4,5,7,8,9]
root.left=24
root.right=40
```
No Rust tree/container algorithm, new `HostOp`, `Value` variant or Markdown
syntax was added. `cargo test -p marqdo-cli --test conformance_semantics`
passes with fixture 56 included.
## CSV row splitter fixture 57
`tests/conformance/semantics/57-csv-row-splitter.mq.md` splits a single CSV row
using only `.mq.md` and the CM-A04 primitives:
* `text.codepoints` enumerates Unicode scalar values;
* `text.from_codepoint` reconstructs each character;
* `list.append` builds the field list, `list.concat` appends the final field;
* comma is the separator; empty fields are preserved.
Observed:
```text
"甲,乙,,丙" -> ["甲","乙","","丙"]
"" -> [""]
"a,b" -> ["a","b"]
```
Quoted fields, embedded commas, doubled quotes and line breaks are **not**
implemented. This is a C05 demonstration, not a claim that CSV is complete.
## Hash table fixture 58
`tests/conformance/semantics/58-memory-hash-table.mq.md` implements an
open-addressing hash table over `Memory`:
* 8-byte header: element count at 0, slot count at 4; slots from offset 8,
8 bytes each (`key:u32`, `value:u32`);
* key `0` means empty, so business keys start at 1;
* hash is `key % capacity` with linear probing;
* existing keys update in place without incrementing the count; a full table
returns `{ok:false, code:"hash.full", ...}`.
Observed:
```text
capacity 32: insert 5->50, 37->370 (collision at slot 5), 12->120, update 5->55
lookup 5 => 55, 37 => 370, 12 => 120, 99 => None
count => 3
capacity 2: insert 1, 2, then 4 => hash.full
```
No Rust hash container, `HostOp` or new `Value` type was added.
## UTF-8 encoder fixture 59
`tests/conformance/semantics/59-utf8-encoder.mq.md` writes UTF-8 bytes into
`Memory` using `text.codepoints` and byte indexing:
* code points below `0x80` -> 1 byte;
* below `0x800` -> 2 bytes; below `0x10000` -> 3 bytes; otherwise 4 bytes;
* the fixture encodes `"Aé中𝄞"` to
`[65, 195, 169, 228, 184, 173, 240, 157, 132, 158]` with length 10.
## UTF-8 decoder fixture 60
`tests/conformance/semantics/60-utf8-decoder.mq.md` decodes UTF-8 from
`Memory` and validates every sequence:
* `utf8.truncated` when a multi-byte sequence runs past the region length;
* `utf8.invalid_continuation` when a byte outside `0x80..0xBF` appears in a
continuation position;
* `utf8.invalid_scalar` for overlong 3-byte forms, surrogates and values above
`0x10FFFF`;
* `utf8.invalid_lead` for `0x80..0xC1` and `0xF5..0xFF` lead bytes.
Observed:
```text
[65,195,169,228,184,173] -> ok "Aé中"
[228,184] -> utf8.truncated
[228,65,173] -> utf8.invalid_continuation
[237,160,128] -> utf8.invalid_scalar
[240,157,132,158] -> ok "𝄞"
```
Together with fixture 59 this closes the C04 encode/decode demonstration using
only `.mq.md`; no Rust UTF-8 codec is called.
## Graph adjacency fixture 61
`tests/conformance/semantics/61-memory-graph-adjacency.mq.md` stores a directed
graph in `Memory`:
* 16-byte node: one-byte value at 0, edge-list head `u32` at 4;
* 8-byte edge: target node offset at 0, next edge offset at 4;
* edges are head-inserted, so traversal returns the most recently added edge
first;
* nodes `甲/乙/丙` at offsets 8/24/40, edges `甲→乙`, `甲→丙`, `乙→丙`.
Observed:
```text
甲=8, 乙=24, 丙=40
邻接(甲) => [40, 24] # 丙 then 乙 (head insertion)
邻接(乙) => [40]
邻接(丙) => []
```
No Rust graph container, `HostOp` or new `Value` type was added.
## JSON subset parser fixture 62
`tests/conformance/semantics/62-json-array-parser.mq.md` is a recursive
descent parser over `text.codepoints`:
* supports nested arrays and objects, strings with standard escapes including
`\uXXXX`, non-negative and negative integers, `true`/`false`/`null`,
whitespace, and returns `{ok,value,next}`;
* errors are `Result` maps: `json.unexpected_eof`, `json.invalid_array`,
`json.invalid_object`, `json.invalid_string`, `json.invalid_number`,
`json.invalid_literal`, `json.unexpected_token`;
* floats are explicitly out of scope.
Observed:
```text
" [1, 23, [4, 5], -6] " -> ok [1,23,[4,5],-6], next=20
"[1 2]" -> json.invalid_array
"[1," -> json.unexpected_eof
{"a": 1, "b": [true, null, "说\"好"]}
-> ok {a:1, b:[true,null,"说\"好"]}, next=35
"\"a\\nb\"" -> ok "a\nb", next=6
```
No Rust JSON parser, new `Value` type or `HostOp` was added.
## Bytecode VM fixture 63
`tests/conformance/semantics/63-memory-bytecode-vm.mq.md` is a small stack
bytecode machine written entirely in `.mq.md`:
* program bytes at offset 0; stack at a caller-provided `Memory` offset;
* instruction set: `1=PUSH imm8`, `2=ADD`, `3=MUL`, `5=JMP`, `6=JZ`, `7=HALT`;
* state (`pc`, `sp`, running, error) is ordinary local state; the stack and
program live in `Memory`;
* `(5 + 7) * 3` executes to `{ok:true, value:36}`;
* a `JZ` program jumps over a `PUSH 99` and returns `{ok:true, value:42}`.
No Rust VM, opcode enum, `HostOp` or new `Value` type was added.
## Integer codecs fixture 64
`tests/conformance/semantics/64-memory-integer-codecs.mq.md` provides generic
big-endian and variable-length integer codecs over `Memory`:
* `写大端(内存,基址,值,宽度)` writes the least significant byte last, so the
result is big-endian for widths 2, 4 and 8;
* `读大端(内存,基址,宽度)` rebuilds the integer with `value * 256 + byte`;
* `写变长`/`读变长` are LEB128 (7-bit continuation) with the consumed byte
count.
Observed:
```text
u16 4660 -> 4660
u32 305419896 -> 305419896
u64 1311768467463790320 -> 1311768467463790320
u64 18446744073709551615 -> 18446744073709551615
varint 123456789 -> 4 bytes, decoded 123456789, consumed 4
last two bytes of u64::MAX -> 255, 255
```
No Rust codec builtin or new `Value` type was added.
## Table sort fixture 65
`tests/conformance/semantics/65-table-sort-stdlib.mq.md` sorts a table by a
column using only CM-A04 primitives:
* `table.row` extracts rows, `table.cell` compares the sort column;
* `插入有序` builds a new row list with `list.append` and binary condition
branches (insertion sort);
* `表(列名, 已排序)` rebuilds the result; the input table is unchanged.
Observed:
```text
原表 rows: 甲87, 乙92, 丙71
升序 rows: 丙71, 甲87, 乙92
原表: unchanged
```
No Rust table sort, `HostOp` or new `Value` type was added.
## Table join fixture 67
`tests/conformance/semantics/67-table-join-stdlib.mq.md` joins two tables on a
column using only CM-A04 primitives:
* nested `table.row` / `table.cell` loops compare the join keys;
* matches append `list.concat(left_row, right_row)` to the output rows;
* `表(list.concat(headers...), rows)` rebuilds the result; both inputs are
unchanged.
Observed: left `甲/乙/丙`, right `乙/甲/丁` on column 0 yields the two matched
rows `甲 87 甲 B` and `乙 92 乙 A`. No Rust join or `HostOp` was added.
## Quoted CSV fixture 68
`tests/conformance/semantics/68-csv-quoted-row.mq.md` parses one CSV row with
quoted fields:
* `"..."` quotes fields, `""` is a literal quote, commas inside quotes are
preserved;
* empty fields are preserved;
* multi-line fields and whitespace trimming are out of scope.
Observed:
```text
甲,"乙,丙","说""好" -> ["甲", "乙,丙", "说"好"]
a,,c -> ["a", "", "c"]
"" -> [""]
"a""b",c -> ["a"b", "c"]
```
No Rust CSV parser or new `Value` type was added.
## What remains open
* JSON floats, table group, and an independent `marqdo-stdlib` repository;
* CM-A06 `Resource`/`Stream` byte-transfer ABI; EOF/partial semantics are
reserved by ADR-0024 but not implemented;
* CM-A07–CM-A11 (stdlib naming, Table storage, provider ABI, packaging,
strict resource policy) and P3–P6;
* the RFC §10.5 frozen-CLI closure lane and Windows/macOS evidence.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-cli --test conformance_semantics
cargo run -q -p marqdo-cli -- run \
tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md --json
cargo test --workspace --locked
```
@@ -0,0 +1,94 @@
# RFC-CM CM-A07 Compat-A evidence: `.mq.md` print prototype
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes.
Decision record: [ADR-0027](../adr/0027-print-migration-compat-a.md)
Related: [Resource P3](2026-11-20-resource-stream-p3.md),
[file write](2026-11-20-resource-file-write-p3.md), [STATUS §5](../STATUS.md).
Scope: **Compat-A only**. Builtin `打印` is not migrated; no alias routing and
no removal of `HostOp::Print`.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
407 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Prototype
Ordinary `.mq.md` modules under `examples/stdlib/`:
| File | Responsibility |
|---|---|
| `format.mq.md` | `显示` for Unit/None/Bool/Int/Float/Text/List/Map/Table, `值显示` (repr), `转义文本`, `列表显示`, `映射显示`, `表格显示`, `连接文本` |
| `text.mq.md` | `utf8编码(文本, 内存, 基址)` — 1–4 byte UTF-8 into `Memory` |
| `io.mq.md` | `写文本` (encode → `resource.write(stdout)` → close) and `写值` (format + newline) |
| `demo.mq.md` | side-by-side `io.写值(x)` and builtin `打印(x)` for a nested list and a table |
The call chain is exactly the RFC target shape:
```text
io.写值(value)
-> format.显示(value) # .mq.md value formatting, no Rust display()
-> text.utf8编码(text, memory) # .mq.md UTF-8 encoder
-> resource.open(stdout)
-> resource.write(memory, ...) # bytes through Memory
-> resource.close(stdout)
```
No `HostOp`, no new builtin, no parser change and no special stdlib dispatch.
The module is loaded only through an explicit Markdown import alias.
## Output comparison
```text
$ marqdo run examples/stdlib/demo.mq.md
["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]
["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]
Table[名称 | 分数]{甲 | 87; 乙 | 92}
Table[名称 | 分数]{甲 | 87; 乙 | 92}
```
The first and third lines are `io.写值`; the second and fourth are builtin
`打印`. They are byte-identical.
CLI test:
`crates/cli/tests/examples_r6.rs::mqmd_stdlib_print_output_matches_builtin_print`
asserts exactly four lines, pairwise equality, and the exact list/table text.
## Covered and uncovered values
Covered by Compat-A comparison: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`,
`List`, `Map`, `Table`.
Not covered: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The language
still has no public observation for the module/name/handle identity needed to
reproduce `Value::display()`/`repr()` for those types; adding it is a CM-A04
extension, not part of this slice. This is the explicit blocker for Compat-B.
## What is not migrated
* `打印`/`print` still resolves to the existing builtin and `HostOp::Print`.
* No alias table, no bundled stdlib resolution, no `std` package.
* No stdlib version/digest or offline distribution (CM-A10).
* No Compat-B/C comparison lane or removal of the Rust `Value::display()` path.
* The demo/stdlib is an `examples/` prototype, not a shipped standard library.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo run -q -p marqdo-cli -- run examples/stdlib/demo.mq.md
cargo test -p marqdo-cli --test examples_r6
cargo test --workspace --locked
```
@@ -0,0 +1,110 @@
# RFC-CM CM-A04 evidence: minimal structure observation and construction
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes.
Decision record: [ADR-0023](../adr/0023-minimal-structure-observation-and-construction.md)
Related: [P1 memory evidence](2026-11-20-memory-p1.md),
[P2 closure evidence](2026-11-20-memory-p2-closure.md),
[STATUS §5](../STATUS.md).
Scope: CM-A04 only. The full RFC-CM-0002 P2 gate is **not** passed.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests)
```
The 2 ignored tests are the pre-existing `marqdo-plugin-sdk` doc-tests. No
GitHub CI was available from this checkout; this is local evidence.
## Builtins added (pure, no `HostOp`, no capability)
| Name | Semantics |
|---|---|
| `list.append` | new list with one appended value; input unchanged |
| `list.concat` | new concatenated list; inputs unchanged |
| `map.keys` | Text keys in canonical sorted order |
| `map.has` | key membership |
| `map.get` | stored value or `None`; `map.has` disambiguates a stored `None` |
| `map.set` | new map with the key; input unchanged |
| `map.remove` | new map without the key; missing key is a no-op |
| `text.codepoints` | Unicode scalar values as `Int`, in order |
| `text.from_codepoint` | one-scalar Text, or `None` for surrogate/out-of-range |
| `table.columns` | header names in column order |
| `table.row` | new list of row cells; non-negative coordinate |
| `table.cell` | cell value; non-negative row/column |
No `sort`, `filter`, `join`, `group`, JSON/CSV, `table.set_cell`, mutable
List/Map or new Markdown syntax was added. Constructors never mutate their
arguments; this preserves INV-02.
## Semantic fixtures
```text
51-list-map-value-semantics:
original list [1,2] unchanged; appended [1,2,3]
original map {a:1,b:2} unchanged; map.set c=3 then remove a
keys after remove => ["b","c"]
map.has(original, "c") => false
map.get(original, "z") => None
list.concat([1],[2,3]) => [1,2,3]
52-text-codepoints:
text.codepoints("aé中") => [97, 233, 20013]
text.from_codepoint(20013) => "中"
text.from_codepoint(0xD800) => None
text.from_codepoint(0x110000) => None
53-table-projection:
columns => ["名称","分数"]
row 1 => ["乙",92]
cell(0,1) => 87
长度(table) => 2
54-table-filter-stdlib:
原表 3 rows, threshold 80
结果 table has 2 rows (甲 87, 乙 92)
原表 is still 3 rows and byte-for-byte the same logical table
```
Fixture 54 is the first `.mq.md` algorithm over the new primitives: it uses
`table.columns`, `table.cell`, `table.row`, `list.append` and `表` to rebuild a
filtered table. It is a conformance document, not a shipped `table.filter`
builtin; no Rust table algorithm was added.
CLI confirms all three return `success: true` with the checked-in wire values.
## Runtime negatives
| Fixture | Expected code | Observed |
|---|---|---|
| `19-list-append-wrong-type` | `runtime.type` | `runtime.type` |
| `20-table-cell-out-of-bounds` | `runtime.index` | `runtime.index` |
| `21-map-key-not-text` | `runtime.type` | `runtime.type` |
## Focused command
```text
cargo test -p marqdo-cli --test conformance_semantics --test negative
semantic_goldens_match_end_to_end_execution ... ok
runtime_negative_goldens ... ok
```
## What remains open
* CM-A05 recoverable `Result`/EOF/partial model. This report predates ADR-0024,
which now freezes the `Result` map convention; the Stream EOF/partial part
remains CM-A06. `map.get` returning `None` is a lookup convention, **not**
that contract.
* CM-A06 `Resource`/`Stream` ABI, CM-A07 stdlib naming/alias, CM-A08 Table
storage/physical migration, CM-A09 provider ABI, CM-A10 stdlib packaging,
CM-A11 strict resource policy.
* A separate `marqdo-stdlib` repository, graph containers, a full JSON parser,
quoted/escaped CSV and the RFC §10.5 frozen-CLI closure lane.
* Windows/macOS native lanes. No remote CI URL exists for this working tree.
+14
View File
@@ -0,0 +1,14 @@
## main
此函数演示“可引用的存储区域”与普通 Markdown 叙述并存。
区域按 64 KiB 逻辑页增长,副本共享同一区域身份。
**区域 = memory.new(1, 4)**
**`区域[8] = 42`**
**另一个名字 = 区域**
**`另一个名字[9] = 99`**
*区域[8] + 区域[9]*
+20
View File
@@ -0,0 +1,20 @@
---
import:
io: "./io.mq.md"
---
## main
**列表 = ["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]**
> io.写值(列表)
> 打印(列表)
**表格 = 表(["名称", "分数"], [["甲", 87], ["乙", 92]])**
> io.写值(表格)
> 打印(表格)
*[类型(列表), 类型(表格)]*
+209
View File
@@ -0,0 +1,209 @@
## 显示
`(值)`
按语言值语义格式化,用于与内置 `打印` 的输出对照。
当前只覆盖 Unit/None/Bool/Int/Float/Text/List/Map/Table;Formula/Code/
FunctionRef/PluginHandle 仍缺少公开观察接口。
**类别 = 类型(值)**
1. `类别 == "Unit"`
*"Unit"*
2. `类别 == "None"`
*"None"*
3. `类别 == "Text"`
*值*
4. `类别 == "List"`
*列表显示(值)*
5. `类别 == "Map"`
*映射显示(值)*
6. `类别 == "Table"`
*表格显示(值)*
7. `true`
*文本(值)*
## 列表显示
`(列表)`
**结果 = "["**
**下标 = 0**
- `下标 < 长度(列表)`
1. `下标 > 0`
**结果 = 结果 + ", "**
2. `true`
**忽略 = 0**
**结果 = 结果 + 值显示(列表[下标])**
**下标 = 下标 + 1**
*结果 + "]"*
## 映射显示
`(映射)`
**键 = map.keys(映射)**
**结果 = "{"**
**下标 = 0**
- `下标 < 长度(键)`
1. `下标 > 0`
**结果 = 结果 + ", "**
2. `true`
**忽略 = 0**
**当前键 = 键[下标]**
**结果 = 结果 + 当前键 + ": " + 值显示(map.get(映射, 当前键))**
**下标 = 下标 + 1**
*结果 + "}"*
## 表格显示
`(表格)`
**列 = table.columns(表格)**
**结果 = "Table[" + 连接文本(列, " | ") + "]{"**
**行号 = 0**
- `行号 < 长度(表格)`
1. `行号 > 0`
**结果 = 结果 + "; "**
2. `true`
**忽略 = 0**
**行 = table.row(表格, 行号)**
**单元格 = []**
**列号 = 0**
- `列号 < 长度(行)`
**单元格 = list.append(单元格, 显示(行[列号]))**
**列号 = 列号 + 1**
**结果 = 结果 + 连接文本(单元格, " | ")**
**行号 = 行号 + 1**
*结果 + "}"*
## 连接文本
`(列表, 分隔)`
**结果 = ""**
**下标 = 0**
- `下标 < 长度(列表)`
1. `下标 > 0`
**结果 = 结果 + 分隔**
2. `true`
**忽略 = 0**
**结果 = 结果 + 列表[下标]**
**下标 = 下标 + 1**
*结果*
## 值显示
`(值)`
列表/映射/表格内部使用 repr 语义:Text 带引号并转义,其它值递归显示。
**类别 = 类型(值)**
1. `类别 == "Text"`
*转义文本(值)*
2. `true`
*显示(值)*
## 转义文本
`(文本)`
**点 = text.codepoints(文本)**
**`结果 = "\""`**
**下标 = 0**
- `下标 < 长度(点)`
**码点 = 点[下标]**
1. `码点 == 34`
**`结果 = 结果 + "\\\""`**
2. `码点 == 92`
**`结果 = 结果 + "\\\\"`**
3. `码点 == 10`
**`结果 = 结果 + "\\n"`**
4. `码点 == 9`
**`结果 = 结果 + "\\t"`**
5. `码点 == 13`
**`结果 = 结果 + "\\r"`**
6. `true`
**结果 = 结果 + text.from_codepoint(码点)**
**下标 = 下标 + 1**
*`结果 + "\""`*
+35
View File
@@ -0,0 +1,35 @@
---
import:
format: "./format.mq.md"
text: "./text.mq.md"
---
## 写文本
`(文本)`
把文本编码为 UTF-8 并写入已授权的 stdout 资源流。
**内存 = memory.new(1, 1)**
**长度 = text.utf8编码(文本, 内存, 0)**
**流 = resource.open("stdio", "stdout", ["write"])**
> resource.write(流, 内存, 0, 长度)
> resource.close(流)
*None*
## 写值
`(值)`
按语言值语义格式化后输出,并追加换行。
**文字 = format.显示(值) + "\n"**
> 写文本(文字)
*None*
+55
View File
@@ -0,0 +1,55 @@
## utf8编码
`(文本, 内存, 基址)`
把 Unicode 标量序列按 UTF-8 写入 `Memory`,返回字节长度。
**点 = text.codepoints(文本)**
**下标 = 0**
**偏移 = 0**
- `下标 < 长度(点)`
**码点 = 点[下标]**
1. `码点 < 128`
**`内存[基址 + 偏移] = 码点`**
**偏移 = 偏移 + 1**
2. `码点 < 2048`
**`内存[基址 + 偏移] = 192 + 码点 // 64`**
**`内存[基址 + 偏移 + 1] = 128 + 码点 % 64`**
**偏移 = 偏移 + 2**
3. `码点 < 65536`
**`内存[基址 + 偏移] = 224 + 码点 // 4096`**
**`内存[基址 + 偏移 + 1] = 128 + 码点 // 64 % 64`**
**`内存[基址 + 偏移 + 2] = 128 + 码点 % 64`**
**偏移 = 偏移 + 3**
4. `true`
**`内存[基址 + 偏移] = 240 + 码点 // 262144`**
**`内存[基址 + 偏移 + 1] = 128 + 码点 // 4096 % 64`**
**`内存[基址 + 偏移 + 2] = 128 + 码点 // 64 % 64`**
**`内存[基址 + 偏移 + 3] = 128 + 码点 % 64`**
**偏移 = 偏移 + 4**
**下标 = 下标 + 1**
*偏移*
+20 -1
View File
@@ -1,6 +1,7 @@
# Marqdo Analysis v2
Status: **implemented by `marqdo check --emit <path>` (S2).** Supersedes
Status: **implemented by `marqdo check --emit <path>` (S2); ADR-0022 adds an
additive index target shape.** Supersedes
Analysis v1. v1 was never released; it is kept as history and is not written.
## 1. Invariants
@@ -82,6 +83,24 @@ index, call). `imports`, `references` and `calls` remain static facts; a
Markdown reference is not a call. `effects` remains `["unknown"]` (a
conservative summary, not a purity proof).
An `assign` node keeps the v1 string `target` for a local rebinding. Addressable
memory adds one additive target form (ADR-0022); consumers that only understand
local assignments can ignore an object target:
```json
{"kind":"assign",
"target":{"kind":"index",
"base":{"kind":"variable","name":"内存","span":{"start":23,"end":26}},
"index":{"kind":"literal","type":"int","value":"0","span":{"start":27,"end":28}},
"span":{"start":23,"end":29}},
"value":{"kind":"literal","type":"int","value":"7","span":{"start":32,"end":33}},
"span":{"start":18,"end":35}}
```
`target.kind` is `"index"`; `base` and `index` use the same expression node
shapes as every other expression. Analysis is static: it records the place but
never allocates a region or executes the write.
## 4. Resources
`resources` is an array, not a count. v2 projects exactly the two resource
+170 -6
View File
@@ -1,6 +1,7 @@
# Marqdo Evaluation Semantics v1
Status: **frozen for M0.5**. Changes to observable behaviour require an ADR.
Status: **frozen for M0.5; extended by ADR-0022 (`Memory`) and ADR-0024 (recoverable results)**.
Changes to observable behaviour require an ADR.
## 1. Values
@@ -19,6 +20,8 @@ Status: **frozen for M0.5**. Changes to observable behaviour require an ADR.
| `Code` | inert code fence + provenance; execution is explicit |
| `FunctionRef` | reference to a named function in a module |
| `PluginHandle` | opaque host-owned plugin export handle |
| `Memory` | opaque runtime-owned addressable byte region (ADR-0022); copies share identity, offsets are region-relative |
| `Resource` | opaque host-issued resource/stream handle (ADR-0025); not indexable, not wire-transferable, rendered as `Resource` |
There is no truthiness: `if`-like conditions accept `Bool` only. There are no
implicit string↔number conversions; conversion builtins must be called.
@@ -52,6 +55,15 @@ implicit string↔number conversions; conversion builtins must be called.
* Assignment binds/updates a local slot in the current frame. It never writes
an outer frame. There is no module-level assignment in v1: assignment
outside a function body is an error.
* The only other writable place is an index into a `Memory` region
(`m[offset] = byte`, ADR-0022). List, Map and Table values keep their v1
value semantics; `list[i] = value`, `map[key] = value` and `obj.field = value`
are not defined and are rejected at run time or parse time rather than
silently changing those values into shared mutable objects.
* For `m[offset] = byte` the operands are evaluated exactly once in the order
**base, offset, right-hand side**. The target must evaluate to `Memory`, the
offset to a non-negative `Int` inside the region and the value to an `Int`
in `0..=255`; otherwise the statement fails without writing anything.
* `and`/`or` short-circuit and require `Bool` operands.
* `Return` (an `emph` node) unwinds through branch/loop bodies to the
enclosing function frame.
@@ -62,6 +74,112 @@ implicit string↔number conversions; conversion builtins must be called.
* Function calls recurse; recursion depth is bounded by the stack budget.
Mutual recursion is allowed.
### 3.1 Addressable `Memory` (ADR-0022)
`Memory` is a separate storage domain from managed `List`/`Map`/`Table`
values. It exists so `.mq.md` programs can construct allocators, encodings and
data structures without a new Markdown node, keyword or host operation.
* **Creation.** `memory.new(initial_pages, max_pages)` creates a zero-filled
region and returns its handle. Both arguments are non-negative `Int`s;
`initial_pages <= max_pages`, and `max_pages` must fit the per-region
budget. Rejection is atomic: no region or committed byte is created.
* **Page size.** One logical page is 64 KiB (`PAGE_SIZE`). Print, trace and
storage use bytes; page counts never mix with byte offsets.
* **Size and growth.** `memory.size(memory)` returns committed pages.
`memory.grow(memory, pages)` is grow-only and returns the **old** page
count; old offsets keep their bytes, every new byte is zero. A failed grow
(declared maximum, region budget or total budget) leaves the region and the
budget unchanged. Growing by zero pages is a no-op returning the current
count. Regions never shrink and there is no user-visible `free(region)`.
* **Read/write.** `m[offset]` returns an `Int` in `0..=255`; `m[offset] = byte`
writes one byte. Offsets are non-negative and region-relative; negative,
out-of-range and unrepresentable `BigInt` offsets are `runtime.index`.
Out-of-range byte values are `runtime.type`. No implicit truncation or
wrapping occurs.
* **Identity and equality.** Copying a `Memory` value, passing it to a
function or storing it in a collection shares the same region. Two handles
are `==` only when they designate the same region instance; `Memory` has no
ordering comparison and no byte-content equality operator.
* **Lifetime.** A region lives for one run and is dropped with the runtime.
Block-level ownership (`malloc`/`free`) is a standard-library concern; the
runtime only guarantees that a single access stays inside its region.
* **Diagnostics.** Unknown, stale or cross-run handles are
`runtime.memory`; budget failures are `resource.exhausted`; a non-`Memory`
index-assignment base is `runtime.type`.
* **Boundary.** A live handle never crosses the wire: `to_wire` emits only
`{"type":"memory"}`, decoding that tag is a `WireError::Invalid`, and every
host/plugin/foreign call rejects arguments containing `Memory`.
* **Display.** `Value::display()` renders `Memory` without the internal
region identity; `类型(m)` returns `"Memory"`.
### 3.2 Recoverable results (ADR-0024)
A recoverable failure is an ordinary `Map`, not a new value type or syntax:
```text
success: {ok: true, value: <any Value>}
failure: {ok: false, code: <Text>, message: <Text> [, details: <Map>]}
```
`ok` is always a `Bool`. Success carries `value`; failure carries a stable
dot-separated `code` and a human-readable `message`. Constructors such as
`成功`/`失败` are ordinary `.mq.md` functions, not builtins.
* Expected conditions (no free block, malformed parser input, invalid byte
sequence, and later EOF/short I/O results) should use this shape so a caller
can branch on `ok`.
* Language-invariant violations (wrong type, out-of-bounds, unknown/stale
`Memory` handle, budget exhaustion, capability denial) remain fatal
`RuntimeError`s. There is no catch mechanism and a library must not try to
convert them into `Err`.
* `map.get` returning `None` is a lookup result, not this convention; use
`map.has` to distinguish a stored `None` (ADR-0023).
* `code` is stable within its library; callers must not branch on `message`.
Recommended prefixes include `alloc.*`, `parse.*`, `codec.*`, `table.*` and,
later, `io.*`.
* Stream EOF/partial semantics belong to the resource ABI (CM-A06); this
section only reserves the shape `{ok: true, value: {count, eof, ...}}` for
later, without defining it.
### 3.3 Resource streams (ADR-0025)
`Resource` is a host-issued opaque handle to an external stream, in a separate
domain from `Memory`. It is not indexable, not a byte address and cannot be
constructed from a number, document or wire value.
```text
resource.open(kind: Text, descriptor: Text, rights: List) -> Resource
resource.read(handle, memory: Memory, offset: Int, max_bytes: Int)
-> {ok:true, value:{count:Int, eof:Bool}} | {ok:false, code, message}
resource.write(handle, memory: Memory, offset: Int, length: Int)
-> {ok:true, value:{count:Int}} | {ok:false, code, message}
resource.close(handle)
-> {ok:true, value:None} | {ok:false, code, message}
```
* Providers: `stdio/stdout` with `["write"]` (pure/project, console grant);
`file` with `["read"]` (project, `fs_read` grant, bounded by the 8 MiB file
limit); `file` with `["write"]` (project, `fs_write` grant, 8 MiB buffered
write flushed on `close`, ADR-0026). Mixed rights and other kinds are
`resource.unsupported`.
* `resource.read` checks the destination `Memory` range before calling the
provider, caps the request to the remaining buffer, writes the returned bytes
and reports the actual count plus `eof`. EOF is success, not an error.
* `resource.write` copies bytes out of `Memory` for the call only; a short write
is a count. stdout output is charged to the output budget and delivered to
the program stdout channel. A project-file write stream appends to a bounded
host buffer and does not touch the filesystem until `close`; a failed close
leaves the handle open with its buffer, and an unclosed writer never creates
the file.
* Reading/writing/closing an already closed handle returns
`{ok:false, code:"resource.closed"}`. Forged/cross-run/stale handles are
fatal `runtime.resource`; unsupported kind/rights/direction are
`resource.unsupported`; capability/path denial is `permission.denied`; a
writer buffer above 8 MiB is `resource.exhausted` and appends atomically.
* `Memory` arguments never cross the host boundary as handles: the interpreter
copies bytes for the call. Resource handles never cross the value wire.
## 4. Errors
All failures are structured. Compile-time problems produce diagnostics
@@ -69,7 +187,11 @@ All failures are structured. Compile-time problems produce diagnostics
problems produce `RuntimeError { category, span, message, frames, cause }`
with a stable category code. Runtime errors stop the current function and
propagate with a call stack. A host/foreign/plugin failure never panics the
interpreter; it becomes a structured error.
interpreter; it becomes a structured error. Addressable-memory failures use
the stable codes `runtime.memory` (unknown/stale handle, invalid page counts),
`runtime.index` (offset outside the region, including negative or huge
offsets), `runtime.type` (wrong operand type or byte value outside `0..=255`)
and `resource.exhausted` (region or run budget).
## 5. Parameter inference (static, name based)
@@ -98,10 +220,17 @@ explicit signature paragraph allows positional arguments.
Every run has explicit limits (all configurable, all reported as
`resource.exhausted`): evaluation fuel, call stack depth, total output bytes,
wall-clock time, and host process count. Loops consume fuel per iteration and
expressions per node so that a hostile document cannot hang the CLI. Strict
mode additionally enforces limits at the OS level where available; if the
platform cannot provide that, strict mode refuses to run (see `security.md`).
wall-clock time, host process count, and addressable memory. Addressable
memory has two independent budgets (ADR-0022): total committed bytes across
all regions (`DEFAULT_MAX_TOTAL_BYTES = 64 MiB`) and committed bytes per
region (`DEFAULT_MAX_REGION_BYTES = 16 MiB`, hence at most 256 pages of
64 KiB), overridable with `--max-memory` and `--max-region-memory`. Memory is
a pure computation: it needs no capability grant and is available in `pure`,
`project` and `strict` modes, subject to these budgets.
Loops consume fuel per iteration and expressions per node so that a hostile
document cannot hang the CLI. Strict mode additionally enforces limits at the
OS level where available; if the platform cannot provide that, strict mode
refuses to run (see `security.md`).
## 7. Determinism
@@ -125,6 +254,41 @@ host boundary. Names are stable; adding one requires a spec update.
| `小数` / `float` | `(x) -> Float` | from Int/Float/Text |
| `表` / `table` | `(header: List, rows: List) -> Table` | validates row widths |
| `ref` | `(target: Text) -> FunctionRef/Formula/Code` | resolves a document address; module-local, refuses ambiguous resource units (`ref.ambiguous`) |
| `memory.new` | `(initial_pages: Int, max_pages: Int) -> Memory` | creates a zero-filled region; budgets are checked before allocation (ADR-0022) |
| `memory.size` | `(memory: Memory) -> Int` | committed page count |
| `memory.grow` | `(memory: Memory, pages: Int) -> Int` | grow-only; returns the previous page count, new bytes are zero |
| `list.append` | `(list: List, value) -> List` | new list with one appended value; input unchanged (ADR-0023) |
| `list.concat` | `(left: List, right: List) -> List` | new concatenated list; inputs unchanged |
| `map.keys` | `(map: Map) -> List` | Text keys in canonical sorted order |
| `map.has` | `(map: Map, key: Text) -> Bool` | key membership |
| `map.get` | `(map: Map, key: Text) -> Value` | stored value or `None`; use `map.has` when `None` is a legal value |
| `map.set` | `(map: Map, key: Text, value) -> Map` | new map with the key; input unchanged |
| `map.remove` | `(map: Map, key: Text) -> Map` | new map without the key; missing key is a no-op |
| `text.codepoints` | `(text: Text) -> List` | Unicode scalar values as `Int`, in order |
| `text.from_codepoint` | `(codepoint: Int) -> Text or None` | `None` for a surrogate or a value outside `0..=0x10FFFF` |
| `table.columns` | `(table: Table) -> List` | header names in column order |
| `table.row` | `(table: Table, row: Int) -> List` | new list of row cells; non-negative coordinate |
| `table.cell` | `(table: Table, row: Int, column: Int) -> Value` | cell value; non-negative coordinates |
| `resource.open` | `(kind: Text, descriptor: Text, rights: List) -> Resource` | capability-checked stream open (ADR-0025) |
| `resource.read` | `(handle, memory: Memory, offset: Int, max_bytes: Int) -> Result` | bounded chunk read into Memory; `{count,eof}` |
| `resource.write` | `(handle, memory: Memory, offset: Int, length: Int) -> Result` | bounded byte write; `{count}` |
| `resource.close` | `(handle) -> Result` | releases the host slot |
| `math.parse/eval/substitute` | see `spec/math.md` | P5; `math.simplify`/`math.differentiate` are declared but return `math.unsupported` in this build |
| `foreign.run/call` | see `spec/foreign.md` | P5, capability-gated |
| `fs.read/write` | see `spec/security.md` | capability-gated |
The legacy `打印/print` builtin remains the default and still uses the host
output path. `examples/stdlib/` contains an explicitly imported `.mq.md`
`format/utf8/io` prototype (ADR-0027 Compat-A) whose output is comparison-tested
against `打印` for Unit/None/Bool/Int/Float/Text/List/Map/Table; it does **not**
replace the builtin, and Formula/Code/FunctionRef/PluginHandle formatting is not
yet expressible in the standard library.
The `list.*`, `map.*`, `text.*` and `table.*` entries are observation and
construction primitives, not algorithms (ADR-0023). They return new values and
never mutate their arguments. Sorting, filtering, joining, grouping, JSON/CSV
and table transformations remain standard-library work: adding one must not
add a builtin. `map.get` returns `None` for a missing key; `map.has` is the
unambiguous membership check when a stored `None` is possible. `table.row` and
`table.cell` use non-negative coordinates, while `List` indexing keeps its v1
negative-index rule.
+9 -4
View File
@@ -1,5 +1,6 @@
(* Marqdo Expression Grammar v1
Status: frozen for M0.5. This is the only grammar for executable payloads.
Status: frozen for M0.5, extended by ADR-0022 with Memory-index assignment
targets. This is the only grammar for executable payloads.
It is a small independent language: it is NOT Markdown and NOT translated
to Python. Markdown AST decides where a payload exists; this grammar
decides what it means.
@@ -10,9 +11,13 @@
program = [ statement ] , EOF ;
statement = assignment | expression ;
assignment = identifier , "=" , expression ;
(* "=" is a single "=" token; "==" is a distinct token. Assignment targets
are bare identifiers only: no member or index assignment in v1. *)
assignment = assign_target , "=" , expression ;
assign_target = identifier
| postfix_expr , "[" , expression , "]" ;
(* v1 left-hand sides are a local identifier or one index place. The index
target writes one byte into an addressable Memory region; other base values
are rejected at run time. Member assignment (`a.b = 1`) is a parse error.
Chained assignment is a parse error. See ADR-0022. *)
expression = or_expr ;
or_expr = and_expr , { "or" , and_expr } ;
+7 -5
View File
@@ -27,8 +27,8 @@ Normative implementation:
| Mode | Intended guarantee | Actual status in this build |
|---|---|---|
| `pure` | No external I/O; memory computation and budgets only. | **Enforced.** `Capabilities::pure()` = `console`, `math` only; every FS/foreign/plugin grant flag makes the CLI exit 2 with "pure mode prohibits external I/O grants". |
| `project` | Operator-approved project resources under `CapabilityHost` + bounded resources. | **Enforced for the implemented operations.** Defaults equal `pure`; `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin` add grants. Sandboxing is cooperative (in-process checks), not OS isolation. |
| `strict` | Untrusted documents under real OS isolation. | **Linux: enforced when the runtime probe passes (ADR-0021).** `PlatformCapabilities::detect()` runs a real Bubblewrap smoke test; success reports `Available { mechanism }`, failure reports `Unavailable { reason }`. Strict L1 supports the interpreter, console and native Math only; `--allow-read/--allow-write/--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file` are rejected with exit 2. macOS/Windows and Linux hosts without the mechanisms remain `Unavailable`; there is **no fallback** to an unrestricted process. |
| `project` | Operator-approved project resources under `CapabilityHost` + bounded resources. | **Enforced for the implemented operations.** Defaults equal `pure`; `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin` add grants. `resource.open` supports stdout write, granted project-file read and granted buffered project-file write (ADR-0025/0026); network/process remain unsupported. Sandboxing is cooperative (in-process checks), not OS isolation. |
| `strict` | Untrusted documents under real OS isolation. | **Linux: enforced when the runtime probe passes (ADR-0021).** `resource.open` is **refused** in strict until a provider passes the isolation lane (CM-A11). `PlatformCapabilities::detect()` runs a real Bubblewrap smoke test; success reports `Available { mechanism }`, failure reports `Unavailable { reason }`. Strict L1 supports the interpreter, console and native Math only; `--allow-read/--allow-write/--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file` are rejected with exit 2. macOS/Windows and Linux hosts without the mechanisms remain `Unavailable`; there is **no fallback** to an unrestricted process. |
`marqdo inspect <file> --platform` prints the detection report, for example:
@@ -55,10 +55,10 @@ set for one run:
| Field | Default (`pure`/`project`) | Used by |
|---|---|---|
| `console: bool` | `true` | currently **not checked** by `op_print`; print is always allowed and only bounded by the output budget (open item) |
| `console: bool` | `true` | `op_print` and `resource.open("stdio","stdout",["write"])`; a missing grant is `permission.denied` and output is also bounded by the output budget |
| `math: bool` | `true` | `math.*` host ops |
| `fs_read: Vec<String>` | empty | `fs.read` |
| `fs_write: Vec<String>` | empty | `fs.write` |
| `fs_read: Vec<String>` | empty | `fs.read` and project `resource.open("file", path, ["read"])` |
| `fs_write: Vec<String>` | empty | `fs.write` and project `resource.open("file", path, ["write"])` |
| `net_hosts: Vec<String>` | empty | `Capabilities::check_net` exists, but no host op or builtin performs network access in this build |
| `process: bool` | `false` | required together with `foreign` for `foreign.run`/`foreign.call` |
| `foreign: bool` | `false` | foreign kernels |
@@ -144,6 +144,8 @@ no host bypass for tests: tests use the same `StdHost`/`Capabilities` types.
| call stack depth | `DEFAULT_STACK_LIMIT = 256` | checked before every function call → `resource.exhausted` |
| program output bytes | `DEFAULT_OUTPUT_BYTES = 1 MiB` | `print` output and `fs.read` contents are charged; overflow → `resource.exhausted` |
| wall clock | none by default | `--timeout <seconds>` sets a deadline; checked between statements/iterations and before loop conditions → `resource.exhausted`. It does **not** preempt a running host call. |
| addressable memory | 64 MiB committed total, 16 MiB per region, 64 KiB pages; `--max-memory` / `--max-region-memory` override | Runtime-owned `MemoryStore` (ADR-0022); create/grow checks both budgets before allocation and failed growth is atomic → `resource.exhausted`. Memory is pure computation and needs no capability grant in any mode. |
| resource streams | stdout writes charged to the existing output budget; file reads and writer buffers bounded by the 8 MiB per-file limit | Host-owned resource table (ADR-0025/0026); output budget is checked before a stdout write is buffered; a failed check produces no bytes; writer buffers flush only on `close`. No separate I/O byte budget yet. |
CLI overrides (`run`/`call`): `--fuel`, `--stack`, `--max-output`,
`--timeout`. Additional fixed limits:
+15 -3
View File
@@ -1,7 +1,9 @@
# Marqdo Value Wire Format v1
Status: **frozen for M0.5**. Changes require an ADR, a `wire_version` bump and
cross-ABI compatibility tests.
Status: **frozen for M0.5; extended by ADR-0022 (non-transferable `Memory`)**.
Changes require an ADR, a `wire_version` bump and cross-ABI compatibility
tests. Adding a rejected tag is not a version bump; accepting a new value shape
is.
Canonical implementation: `crates/marqdo-value/src/wire.rs`
(`to_wire`, `from_wire`, `from_wire_with_limits`, `WireLimits`, `WireError`).
@@ -14,12 +16,20 @@ documented in [`foreign.md`](foreign.md) and implemented in
## 1. Why a typed wire format
JSON has one numeric type. Marqdo has arbitrary-precision `Int`, IEEE-754
`Float` (including `NaN` and both infinities) and 13 distinct runtime value
`Float` (including `NaN` and both infinities) and 15 distinct runtime value
types (`crates/marqdo-value/src/value.rs`). Encoding an `Int` as a JSON number
would silently lose digits above 2^53 and would make `Int(1)` and
`Float(1.0)` indistinguishable after a round trip. The wire format is
therefore **fully type-tagged** and is not plain JSON.
Addressable `Memory` and host `Resource` handles are deliberately **not
transferable**. `to_wire` emits
`{"type":"memory"}` only so that an accidental transfer is visible; decoding
that tag is always `WireError::Invalid`, and the host boundary rejects any
argument containing a live handle before an ABI call. Copying bytes requires an
explicit standard-library snapshot to a managed value, not serializing a
handle (ADR-0022).
The rules that follow from this:
1. Every value is a JSON **object** with a mandatory `"type"` string.
@@ -81,6 +91,8 @@ The table is normative and mirrors `encode`/`decode` in
| `Code` | `{"type":"code","language":"python","source":"...","content_hash":"<sha256 hex>"}` plus optional `"unit"` and `"span"` | `language` and `content_hash` default to `""` on decode when absent; `source` is required. |
| `FunctionRef` | `{"type":"function_ref","module":"index.mq.md","name":"main"}` | Both fields required strings. |
| `PluginHandle` | `{"type":"plugin_handle","plugin":"demo.solver","export":"solver.add","id":1}` | `id` must be a non-negative JSON integer (`as_u64`). |
| `Memory` | `{"type":"memory"}` | **Never decodable.** No handle, address or bytes are encoded; a round trip is intentionally impossible (`wire::tests::memory_handles_are_not_transferable`). |
| `Resource` | `{"type":"resource"}` | **Never decodable.** Host-private stream handle; `from_wire` rejects the tag (`resource_handles_are_not_transferable`, ADR-0025). |
Optional fields (`unit`, `span`, `display`, `language`, `content_hash`) are
emitted only when present on the value. Encoders always emit `display` for
+126 -2
View File
@@ -15,11 +15,11 @@ no test commits, copies or rewrites files outside a throw-away temp directory.
tests/
├── conformance/
│ ├── ast/ 160 × *.mq.md + sibling *.ast.json goldens
│ └── semantics/ 44 × *.mq.md + sibling *.expected.json goldens
│ └── semantics/ 68 × *.mq.md + sibling *.expected.json goldens
├── negative/
│ ├── *.mq.md 52 × compile-time hard-error fixtures + *.expected.json
│ ├── warnings/ 6 × inert-executable warning fixtures + *.expected.json
│ └── runtime/ 12 × runtime-error fixtures + *.expected.json
│ └── runtime/ 21 × runtime-error fixtures + *.expected.json
└── README.md this file
```
@@ -69,6 +69,130 @@ The power/control-flow slice is pinned by:
* `crates/marqdo-frontend/src/payload.rs` span tests and
`crates/cli/tests/integration_cli.rs` CLI span/E2E assertions.
The addressable-memory slice (RFC-CM P1, ADR-0022) is pinned by:
* semantic fixtures 45–47: sharing a region by handle, per-region isolation,
`memory.size`, `memory.grow` returning the old count, preserved old bytes and
zero-filled new pages;
* runtime negatives 13–18: out-of-bounds and huge offsets, byte values outside
`0..=255`, index assignment on a `List`, growth beyond the declared maximum
and negative page counts;
* `tests/negative/runtime/15-index-assign-non-memory.mq.md` also guards the v1
value semantics of `List` (it must not become shared mutable memory);
* `crates/marqdo-runtime/src/memory.rs` unit tests (aliasing, atomic failed
growth, forged/stale handles, page arithmetic) and
`crates/marqdo-runtime/tests/interpreter.rs` host-boundary/budget tests;
* `crates/marqdo-value/src/wire.rs` proves live handles are not transferable;
* `examples/memory-buffer.mq.md` and the `examples_r6` CLI test prove the
`.mq.md` surface end to end.
Partial RFC-CM P2 (compute closure) is demonstrated, not certified, by:
* semantic fixtures 48–50: a big-endian u32 codec and bump allocator, three
allocated nodes linked and traversed, and a failed allocation that leaves
the allocator header unchanged. These documents use only language builtins
and control flow; no Rust algorithm is added for them.
* The full P2 gate also needs a free-list allocator with reuse/coalescing,
tree/graph structures, a recoverable `Result` model and an independent
stdlib repository. Those are **not** implemented.
CM-A04 structure primitives (ADR-0023) are pinned by:
* semantic fixtures 51–54: value semantics of `list.append`/`map.set`/
`map.remove` (inputs unchanged, keys sorted, `map.has`/`map.get`),
Unicode scalar round-trip through `text.codepoints`/
`text.from_codepoint` (including invalid scalars → `None`),
`table.columns`/`table.row`/`table.cell` projection, and a `.mq.md`
`过滤` function that rebuilds a filtered GFM-style table without mutating
the input or adding a Rust table algorithm;
* runtime negatives 19–21: `list.append` on a Map, table row out of bounds,
and a non-Text `map.get` key;
* `crates/marqdo-runtime/src/builtins_impl.rs`; no algorithm builtin such as
sort/filter/join/JSON/CSV is added.
CM-A05 recoverable results and the first-fit allocator (ADR-0024) are pinned
by:
* semantic fixture 55: a `.mq.md` first-fit/free-list allocator with 8-byte
block headers, free-block reuse, block splitting, address-adjacent
coalescing (a second region fills its bump space, so only the merged block
can satisfy a 40-byte request), a too-large request returning
`{ok:false, code:"alloc.no_space", ...}`, and payload writes that show the
reused blocks do not overlap;
* the `Result` shape is an ordinary Map; no `Result` value type, exception,
`try`/`catch` or propagation syntax is added. The ADR records the
success/failure field contract and the fatal-vs-recoverable boundary;
* semantic fixture 56: a `Memory`-backed binary search tree with 16-byte nodes,
recursive insertion and recursive in-order traversal built from
`list.concat`; the returned root/left/right offsets and sorted values pin
node layout and traversal;
* semantic fixture 57: a `.mq.md` CSV single-row splitter using
`text.codepoints`, `text.from_codepoint`, `list.append` and `list.concat`;
it preserves empty fields and does **not** pretend to support quoted or
escaped CSV;
* semantic fixture 58: an open-addressing hash table with linear probing,
collision handling (keys 5 and 37), value update, not-found `None`, element
count, and a full-table `hash.full` Result in a second region;
* semantic fixture 59: a `.mq.md` UTF-8 encoder that writes 1–4 byte sequences
into `Memory` from `text.codepoints`, returning the byte length and bytes;
* semantic fixture 60: a validating `.mq.md` UTF-8 decoder over `Memory` with
explicit `utf8.truncated`, `utf8.invalid_continuation`, `utf8.invalid_scalar`
and `utf8.invalid_lead` failures; valid ASCII, 2/3/4-byte and mixed text is
decoded;
* semantic fixture 61: a `Memory`-backed graph adjacency list with 16-byte nodes
and 8-byte edges, head-inserted edges and a neighbor traversal; no Rust graph
container or new `Value` type;
* semantic fixture 62: a recursive `.mq.md` JSON subset parser for nested
arrays and objects, strings with standard escapes including `\uXXXX`,
integers (including negative), `true`/`false`/`null`, whitespace, `next`
positions and `Result` errors (`json.unexpected_eof`, `json.invalid_array`,
`json.invalid_object`, `json.invalid_string`, `json.invalid_number`,
`json.invalid_literal`, `json.unexpected_token`). Floats are explicitly out
of scope;
* semantic fixture 63: a `Memory`-backed stack bytecode VM with instruction
decode, PUSH/ADD/MUL/JMP/JZ/HALT, a stack region and two programs (arithmetic
and conditional branch); no Rust VM or interpreter loop was added;
* semantic fixture 64: `.mq.md` big-endian 16/32/64-bit encoders/decoders and
LEB128 varint encode/decode, including the `u64` maximum and byte-level
round-trips;
* semantic fixture 65: a `.mq.md` table sort built on `table.row`,
`table.columns`, `table.cell`, `表` and `list.append`; insertion sort
rebuilds a new table and leaves the input table unchanged;
* semantic fixture 67: a `.mq.md` table equality join over two tables, using
`table.row/columns/cell`, `list.concat`, `list.append` and `表`; both input
tables remain unchanged;
* semantic fixture 68: a quoted/escaped CSV single-row parser using Unicode
scalar primitives; `""` is a literal quote and commas inside quotes are
preserved. Multi-line fields and whitespace trimming are out of scope.
RFC-CM CM-A07 Compat-A (ADR-0027) is pinned by:
* `examples/stdlib/{format,text,io}.mq.md` and `examples/stdlib/demo.mq.md`;
* `crates/cli/tests/examples_r6.rs::mqmd_stdlib_print_output_matches_builtin_print`,
which runs the demo and asserts `io.写值` and builtin `打印` lines are
byte-identical for a nested List/Map/None value and a Table;
* the demo covers Unit/None/Bool? (Bool appears inside the nested list),
Int, Float, Text, List, Map and Table. Formula/Code/FunctionRef/PluginHandle
are explicitly not covered; `打印` is not yet routed to the stdlib.
RFC-CM CM-A06 resource streams (ADR-0025) are pinned by:
* semantic fixture 66: `.mq.md` UTF-8 text encoded into `Memory`, written to a
`stdio/stdout` resource, closed, then re-used to prove
`{ok:false,code:"resource.closed"}`; the expected `stdout` is the decoded
text;
* `crates/marqdo-host/tests/resource_streams.rs` (9): pure-mode stdout write,
output-budget atomicity, strict refusal, project-file chunking/EOF, missing
grant, traversal rejection and unsupported rights/kinds;
* `crates/cli/tests/resource_streams.rs`: real `marqdo run` with
`--allow-read data.txt`, three chunked reads into `Memory` plus EOF, the same
document failing with `permission.denied` without the grant, and a
write-then-read program with `--allow-write/--allow-read out.bin` proving the
file is created only at `resource.close`, and a chunked `.mq.md` file copy
(`resource.read` → `resource.write`) that reproduces `data.txt` in `out.bin`;
* `crates/marqdo-host/src/resource.rs` unit tests for forged/closed handles,
stdout read/file write unsupported, and chunked EOF.
Fixture discovery is directory-based and sorted by file name, so the order is
deterministic on every platform. The AST and semantic suites fail loudly if the
corpus shrinks below 100 / 25 fixtures, and the negative suite fails below 30
@@ -0,0 +1,9 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "int",
"decimal": "141"
}
}
@@ -0,0 +1,14 @@
## main
This function demonstrates an addressable byte region next to ordinary prose.
A copied name shares the same region identity.
**区域 = memory.new(1, 4)**
**`区域[8] = 42`**
**另一个名字 = 区域**
**`另一个名字[9] = 99`**
*区域[8] + 区域[9]*
@@ -0,0 +1,14 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{"type": "int", "decimal": "0"},
{"type": "int", "decimal": "1"},
{"type": "int", "decimal": "9"},
{"type": "int", "decimal": "0"}
]
}
}
@@ -0,0 +1,14 @@
## main
A zero-page region is legal. Growing it returns the previous page count,
preserves old bytes and zero-fills every new byte.
**内存 = memory.new(0, 2)**
**旧页数 = memory.grow(内存, 1)**
**内存[0] = 9**
**页数 = memory.size(内存)**
*[旧页数, 页数, 内存[0], 内存[65535]]*
@@ -0,0 +1,15 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{"type": "int", "decimal": "5"},
{"type": "int", "decimal": "6"},
{"type": "int", "decimal": "7"},
{"type": "bool", "value": true},
{"type": "bool", "value": false}
]
}
}
@@ -0,0 +1,18 @@
## main
Two regions never alias even at the same offset. A copied name does share
the region identity, and equality compares that identity, not the bytes.
**甲 = memory.new(1, 1)**
**乙 = memory.new(1, 1)**
**`甲[0] = 5`**
**`乙[0] = 6`**
**别名 = 甲**
**`别名[1] = 7`**
*[甲[0], 乙[0], 甲[1], 甲 == 别名, 甲 == 乙]*
@@ -0,0 +1,15 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{"type": "int", "decimal": "4"},
{"type": "int", "decimal": "12"},
{"type": "int", "decimal": "28"},
{"type": "int", "decimal": "36"},
{"type": "int", "decimal": "255"}
]
}
}
@@ -0,0 +1,72 @@
## 写32
`(内存, 基址, 值)`
把非负 Int 的低 32 位按大端字节写入区域。
**字节3 = 值 % 256**
**字节2 = 值 // 256 % 256**
**字节1 = 值 // 65536 % 256**
**字节0 = 值 // 16777216 % 256**
**`内存[基址] = 字节0`**
**`内存[基址 + 1] = 字节1`**
**`内存[基址 + 2] = 字节2`**
**`内存[基址 + 3] = 字节3`**
*None*
## 读32
`(内存, 基址)`
同样按大端读取四个字节。
*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]*
## 分配
`(内存, 容量, 大小)`
区域头四个字节保存下一个空闲偏移。这是顺序分配器:free 只把块交还给调用方,
不在这里实现复用。
**当前位置 = 读32(内存, 0)**
1. `当前位置 + 大小 <= 容量`
> 写32(内存, 0, 当前位置 + 大小)
*当前位置*
2. `true`
*None*
## main
**内存 = memory.new(1, 1)**
**`内存[0] = 0`**
**`内存[1] = 0`**
**`内存[2] = 0`**
**`内存[3] = 4`**
**第一个 = 分配(内存, 65536, 8)**
**第二个 = 分配(内存, 65536, 16)**
**第三个 = 分配(内存, 65536, 8)**
**`内存[第一个] = 255`**
*[第一个, 第二个, 第三个, 读32(内存, 0), 内存[第一个]]*
@@ -0,0 +1,14 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{"type": "int", "decimal": "4"},
{"type": "int", "decimal": "12"},
{"type": "int", "decimal": "20"},
{"type": "int", "decimal": "60"}
]
}
}
@@ -0,0 +1,93 @@
## 写32
`(内存, 基址, 值)`
**字节3 = 值 % 256**
**字节2 = 值 // 256 % 256**
**字节1 = 值 // 65536 % 256**
**字节0 = 值 // 16777216 % 256**
**`内存[基址] = 字节0`**
**`内存[基址 + 1] = 字节1`**
**`内存[基址 + 2] = 字节2`**
**`内存[基址 + 3] = 字节3`**
*None*
## 读32
`(内存, 基址)`
*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]*
## 分配
`(内存, 容量, 大小)`
**当前位置 = 读32(内存, 0)**
1. `当前位置 + 大小 <= 容量`
> 写32(内存, 0, 当前位置 + 大小)
*当前位置*
2. `true`
*None*
## 节点
`(内存, 值, 下一个)`
节点是 8 字节块:偏移 0 存一个字节的值,偏移 1..4 存 next 偏移;0 表示空。
**地址 = 分配(内存, 65536, 8)**
**`内存[地址] = 值`**
> 写32(内存, 地址 + 1, 下一个)
*地址*
## 遍历求和
`(内存, 头)`
**当前 = 头**
**总和 = 0**
- `当前 != 0`
**总和 = 总和 + 内存[当前]**
**当前 = 读32(内存, 当前 + 1)**
*总和*
## main
**内存 = memory.new(1, 1)**
**`内存[0] = 0`**
**`内存[1] = 0`**
**`内存[2] = 0`**
**`内存[3] = 4`**
**甲 = 节点(内存, 10, 0)**
**乙 = 节点(内存, 20, 甲)**
**丙 = 节点(内存, 30, 乙)**
*[甲, 乙, 丙, 遍历求和(内存, 丙)]*
@@ -0,0 +1,14 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{"type": "none"},
{"type": "int", "decimal": "4"},
{"type": "int", "decimal": "4"},
{"type": "int", "decimal": "65536"}
]
}
}
@@ -0,0 +1,67 @@
## 写32
`(内存, 基址, 值)`
**字节3 = 值 % 256**
**字节2 = 值 // 256 % 256**
**字节1 = 值 // 65536 % 256**
**字节0 = 值 // 16777216 % 256**
**`内存[基址] = 字节0`**
**`内存[基址 + 1] = 字节1`**
**`内存[基址 + 2] = 字节2`**
**`内存[基址 + 3] = 字节3`**
*None*
## 读32
`(内存, 基址)`
*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]*
## 分配
`(内存, 容量, 大小)`
**当前位置 = 读32(内存, 0)**
1. `当前位置 + 大小 <= 容量`
> 写32(内存, 0, 当前位置 + 大小)
*当前位置*
2. `true`
*None*
## main
分配失败必须保持区域头不变,之后的分配仍从原位置继续。
**内存 = memory.new(1, 1)**
**`内存[0] = 0`**
**`内存[1] = 0`**
**`内存[2] = 0`**
**`内存[3] = 4`**
**失败 = 分配(内存, 65536, 65536)**
**失败后位置 = 读32(内存, 0)**
**成功 = 分配(内存, 65536, 65532)**
**最终位置 = 读32(内存, 0)**
*[失败, 失败后位置, 成功, 最终位置]*
@@ -0,0 +1,45 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{
"type": "list",
"items": [
{"type": "int", "decimal": "1"},
{"type": "int", "decimal": "2"}
]
},
{
"type": "list",
"items": [
{"type": "int", "decimal": "1"},
{"type": "int", "decimal": "2"},
{"type": "int", "decimal": "3"}
]
},
{"type": "int", "decimal": "2"},
{"type": "int", "decimal": "2"},
{"type": "int", "decimal": "2"},
{
"type": "list",
"items": [
{"type": "text", "value": "b"},
{"type": "text", "value": "c"}
]
},
{"type": "bool", "value": false},
{"type": "none"},
{
"type": "list",
"items": [
{"type": "int", "decimal": "1"},
{"type": "int", "decimal": "2"},
{"type": "int", "decimal": "3"}
]
}
]
}
}
@@ -0,0 +1,18 @@
## main
Lists and maps keep value semantics: constructors return new values and the
inputs are unchanged.
**原列表 = [1, 2]**
**新列表 = list.append(原列表, 3)**
**原映射 = {a: 1, b: 2}**
**新映射 = map.set(原映射, "c", 3)**
**删除后 = map.remove(新映射, "a")**
**键 = map.keys(删除后)**
*[原列表, 新列表, 原映射.b, 新映射.b, 删除后.b, 键, map.has(原映射, "c"), map.get(原映射, "z"), list.concat([1], [2, 3])]*
@@ -0,0 +1,22 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{
"type": "list",
"items": [
{"type": "int", "decimal": "97"},
{"type": "int", "decimal": "233"},
{"type": "int", "decimal": "20013"}
]
},
{"type": "text", "value": "中"},
{"type": "none"},
{"type": "none"},
{"type": "int", "decimal": "1"}
]
}
}
@@ -0,0 +1,7 @@
## main
**点 = text.codepoints("aé中")**
**回 = text.from_codepoint(20013)**
*[点, 回, text.from_codepoint(55296), text.from_codepoint(1114112), 长度(回)]*
@@ -0,0 +1,26 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{
"type": "list",
"items": [
{"type": "text", "value": "名称"},
{"type": "text", "value": "分数"}
]
},
{
"type": "list",
"items": [
{"type": "text", "value": "乙"},
{"type": "int", "decimal": "92"}
]
},
{"type": "int", "decimal": "87"},
{"type": "int", "decimal": "2"}
]
}
}
@@ -0,0 +1,5 @@
## main
**数据 = 表(["名称", "分数"], [["甲", 87], ["乙", 92]])**
*[table.columns(数据), table.row(数据, 1), table.cell(数据, 0, 1), 长度(数据)]*
@@ -0,0 +1,42 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"type": "list",
"items": [
{
"type": "table",
"header": ["名称", "分数"],
"rows": [
[
{"type": "text", "value": "甲"},
{"type": "int", "decimal": "87"}
],
[
{"type": "text", "value": "乙"},
{"type": "int", "decimal": "92"}
],
[
{"type": "text", "value": "丙"},
{"type": "int", "decimal": "71"}
]
]
},
{
"type": "table",
"header": ["名称", "分数"],
"rows": [
[
{"type": "text", "value": "甲"},
{"type": "int", "decimal": "87"}
],
[
{"type": "text", "value": "乙"},
{"type": "int", "decimal": "92"}
]
]
}
]
}
}
@@ -0,0 +1,31 @@
## 过滤
`(数据, 阈值)`
只用 table 观察原语和 list 构造原语重建一个新表;输入表不被修改。
**列 = table.columns(数据)**
**结果行 = []**
**行号 = 0**
- `行号 < 长度(数据)`
**分数 = table.cell(数据, 行号, 1)**
1. `分数 > 阈值`
**结果行 = list.append(结果行, table.row(数据, 行号))**
**行号 = 行号 + 1**
*表(列, 结果行)*
## main
**原表 = 表(["名称", "分数"], [["甲", 87], ["乙", 92], ["丙", 71]])**
**结果 = 过滤(原表, 80)**
*[原表, 结果]*
@@ -0,0 +1,262 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "16",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "32",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "16",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "16",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "32",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "56",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "56",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "72",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "96",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "112",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"code": {
"type": "text",
"value": "alloc.no_space"
},
"message": {
"type": "text",
"value": "没有足够的空闲空间"
},
"ok": {
"type": "bool",
"value": false
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "16",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "32",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "48",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "16",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"code": {
"type": "text",
"value": "alloc.no_space"
},
"message": {
"type": "text",
"value": "没有足够的空闲空间"
},
"ok": {
"type": "bool",
"value": false
}
},
"type": "map"
},
{
"decimal": "33",
"type": "int"
},
{
"decimal": "22",
"type": "int"
},
{
"decimal": "33",
"type": "int"
},
{
"decimal": "33",
"type": "int"
},
{
"decimal": "22",
"type": "int"
},
{
"decimal": "44",
"type": "int"
},
{
"decimal": "55",
"type": "int"
},
{
"decimal": "66",
"type": "int"
},
{
"decimal": "77",
"type": "int"
}
],
"type": "list"
}
}
@@ -0,0 +1,308 @@
## 写32
`(内存, 基址, 值)`
**字节3 = 值 % 256**
**字节2 = 值 // 256 % 256**
**字节1 = 值 // 65536 % 256**
**字节0 = 值 // 16777216 % 256**
**`内存[基址] = 字节0`**
**`内存[基址 + 1] = 字节1`**
**`内存[基址 + 2] = 字节2`**
**`内存[基址 + 3] = 字节3`**
*None*
## 读32
`(内存, 基址)`
*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]*
## 成功
`(值)`
*{ok: true, value: 值}*
## 失败
`(代码, 消息)`
*{ok: false, code: 代码, message: 消息}*
## 初始化
`(内存)`
区域头:偏移 0 是空闲链表头,偏移 4 是 bump 指针;首个块从 8 开始。
> 写32(内存, 0, 0)
> 写32(内存, 4, 8)
*成功(None)*
## 查找
`(内存, 需求)`
返回 `[前一个空闲块, 当前空闲块, 当前块大小]`;没有合适块时返回零值。
**前一个 = 0**
**当前 = 读32(内存, 0)**
- `当前 != 0`
**块大小 = 读32(内存, 当前)**
1. `块大小 >= 需求`
*[前一个, 当前, 块大小]*
**前一个 = 当前**
**当前 = 读32(内存, 当前 + 4)**
*[0, 0, 0]*
## 分配
`(内存, 容量, 请求大小)`
first-fit:先查空闲链表,必要时切分;否则从 bump 指针处取新块。
块头 8 字节:偏移 0 是总块大小,偏移 4 是空闲链表 next。返回 payload 偏移。
**需求 = (请求大小 + 8 + 7) // 8 * 8**
**位置 = 查找(内存, 需求)**
**前一个 = 位置[0]**
**当前 = 位置[1]**
**块大小 = 位置[2]**
1. `当前 != 0`
**余量 = 块大小 - 需求**
**下一个 = 读32(内存, 当前 + 4)**
1. `余量 >= 16`
**余块 = 当前 + 需求**
> 写32(内存, 余块, 余量)
> 写32(内存, 余块 + 4, 下一个)
1. `前一个 == 0`
> 写32(内存, 0, 余块)
2. `true`
> 写32(内存, 前一个 + 4, 余块)
> 写32(内存, 当前, 需求)
*成功(当前 + 8)*
2. `true`
1. `前一个 == 0`
> 写32(内存, 0, 下一个)
2. `true`
> 写32(内存, 前一个 + 4, 下一个)
*成功(当前 + 8)*
2. `true`
**下一个空位 = 读32(内存, 4)**
1. `下一个空位 + 需求 <= 容量`
> 写32(内存, 下一个空位, 需求)
> 写32(内存, 4, 下一个空位 + 需求)
*成功(下一个空位 + 8)*
2. `true`
*失败("alloc.no_space", "没有足够的空闲空间")*
## 插入空闲
`(内存, 块)`
空闲链表按地址升序;插入后先与后继合并,再与前去合并。
**前一个 = 0**
**当前 = 读32(内存, 0)**
- `当前 != 0 and 当前 < 块`
**前一个 = 当前**
**当前 = 读32(内存, 当前 + 4)**
> 写32(内存, 块 + 4, 当前)
1. `前一个 == 0`
> 写32(内存, 0, 块)
2. `true`
> 写32(内存, 前一个 + 4, 块)
> 合并(内存, 前一个, 块, 当前)
*成功(None)*
## 合并
`(内存, 前一个, 块, 当前)`
相邻块只有在地址连续时才合并,避免产生虚假的大块。
**块大小 = 读32(内存, 块)**
1. `当前 != 0 and 块 + 块大小 == 当前`
**合并大小 = 块大小 + 读32(内存, 当前)**
> 写32(内存, 块, 合并大小)
> 写32(内存, 块 + 4, 读32(内存, 当前 + 4))
2. `true`
**忽略后继合并 = 0**
**前一个大小 = 0**
1. `前一个 != 0`
**前一个大小 = 读32(内存, 前一个)**
2. `true`
**忽略前去读取 = 0**
后继合并完成后再处理前驱;两个判断必须分开执行。
1. `前一个 != 0 and 前一个 + 前一个大小 == 块`
**前一个新大小 = 前一个大小 + 读32(内存, 块)**
> 写32(内存, 前一个, 前一个新大小)
> 写32(内存, 前一个 + 4, 读32(内存, 块 + 4))
2. `true`
**忽略前去合并 = 0**
*成功(None)*
## 释放
`(内存, 地址)`
把 payload 偏移换算回块头,按地址插入空闲链表并由“合并”处理相邻块。
**块 = 地址 - 8**
> 插入空闲(内存, 块)
*成功(None)*
## main
**内存 = memory.new(1, 1)**
> 初始化(内存)
**甲 = 分配(内存, 65536, 8)**
**乙 = 分配(内存, 65536, 16)**
**`内存[甲.value] = 11`**
**`内存[乙.value] = 22`**
> 释放(内存, 甲.value)
**丙 = 分配(内存, 65536, 8)**
**`内存[丙.value] = 33`**
> 释放(内存, 乙.value)
> 释放(内存, 丙.value)
**丁 = 分配(内存, 65536, 4)**
**戊 = 分配(内存, 65536, 16)**
**己 = 分配(内存, 65536, 64)**
> 释放(内存, 己.value)
**庚 = 分配(内存, 65536, 8)**
**辛 = 分配(内存, 65536, 16)**
**壬 = 分配(内存, 65536, 8)**
**癸 = 分配(内存, 65536, 8)**
**`内存[庚.value] = 44`**
**`内存[辛.value] = 55`**
**`内存[壬.value] = 66`**
**`内存[癸.value] = 77`**
**失败结果 = 分配(内存, 65536, 100000)**
**内存2 = memory.new(1, 1)**
> 初始化(内存2)
**甲2 = 分配(内存2, 65536, 8)**
**乙2 = 分配(内存2, 65536, 8)**
**丙2 = 分配(内存2, 65536, 8)**
> 分配(内存2, 65536, 65472)
> 释放(内存2, 甲2.value)
> 释放(内存2, 乙2.value)
> 释放(内存2, 丙2.value)
**丁2 = 分配(内存2, 65536, 40)**
**戊2 = 分配(内存2, 65536, 8)**
*[甲, 乙, 丙, 丁, 戊, 己, 庚, 辛, 壬, 癸, 失败结果, 甲2, 乙2, 丙2, 丁2, 戊2, 内存[甲.value], 内存[乙.value], 内存[丙.value], 内存[丁.value], 内存[戊.value], 内存[庚.value], 内存[辛.value], 内存[壬.value], 内存[癸.value]]*
@@ -0,0 +1,55 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"decimal": "8",
"type": "int"
},
{
"items": [
{
"decimal": "1",
"type": "int"
},
{
"decimal": "3",
"type": "int"
},
{
"decimal": "4",
"type": "int"
},
{
"decimal": "5",
"type": "int"
},
{
"decimal": "7",
"type": "int"
},
{
"decimal": "8",
"type": "int"
},
{
"decimal": "9",
"type": "int"
}
],
"type": "list"
},
{
"decimal": "24",
"type": "int"
},
{
"decimal": "40",
"type": "int"
}
],
"type": "list"
}
}
@@ -0,0 +1,112 @@
## 写32
`(内存, 基址, 值)`
**字节3 = 值 % 256**
**字节2 = 值 // 256 % 256**
**字节1 = 值 // 65536 % 256**
**字节0 = 值 // 16777216 % 256**
**`内存[基址] = 字节0`**
**`内存[基址 + 1] = 字节1`**
**`内存[基址 + 2] = 字节2`**
**`内存[基址 + 3] = 字节3`**
*None*
## 读32
`(内存, 基址)`
*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]*
## 新节点
`(内存, 值)`
每个节点 16 字节:偏移 0 存一个字节的值,偏移 4 存左子,偏移 8 存右子;0 表示空。
节点值必须落在 0..255。
**位置 = 读32(内存, 0)**
**`内存[位置] = 值`**
> 写32(内存, 位置 + 4, 0)
> 写32(内存, 位置 + 8, 0)
> 写32(内存, 0, 位置 + 16)
*位置*
## 插入
`(内存, 根, 值)`
1. `根 == 0`
*新节点(内存, 值)*
2. `值 < 内存[根]`
**左 = 插入(内存, 读32(内存, 根 + 4), 值)**
> 写32(内存, 根 + 4, 左)
*根*
3. `true`
**右 = 插入(内存, 读32(内存, 根 + 8), 值)**
> 写32(内存, 根 + 8, 右)
*根*
## 遍历
`(内存, 根)`
中序遍历,返回从小到大的值列表。
1. `根 == 0`
*[]*
**左 = 遍历(内存, 读32(内存, 根 + 4))**
**右 = 遍历(内存, 读32(内存, 根 + 8))**
*list.concat(list.concat(左, [内存[根]]), 右)*
## main
**内存 = memory.new(1, 1)**
**`内存[0] = 0`**
**`内存[1] = 0`**
**`内存[2] = 0`**
**`内存[3] = 8`**
**根 = 0**
**值表 = [5, 3, 8, 1, 4, 7, 9]**
**下标 = 0**
- `下标 < 长度(值表)`
**根 = 插入(内存, 根, 值表[下标])**
**下标 = 下标 + 1**
*[根, 遍历(内存, 根), 读32(内存, 根 + 4), 读32(内存, 根 + 8)]*
@@ -0,0 +1,53 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"items": [
{
"type": "text",
"value": "甲"
},
{
"type": "text",
"value": "乙"
},
{
"type": "text",
"value": ""
},
{
"type": "text",
"value": "丙"
}
],
"type": "list"
},
{
"items": [
{
"type": "text",
"value": ""
}
],
"type": "list"
},
{
"items": [
{
"type": "text",
"value": "a"
},
{
"type": "text",
"value": "b"
}
],
"type": "list"
}
],
"type": "list"
}
}
@@ -0,0 +1,35 @@
## 分割
`(文本)`
这是 CSV 单行拆分的最小实现:只处理逗号分隔,不支持引号或转义;空字段保留。
**结果 = []**
**当前 = ""**
**点 = text.codepoints(文本)**
**下标 = 0**
- `下标 < 长度(点)`
**字符 = text.from_codepoint(点[下标])**
1. `字符 == ","`
**结果 = list.append(结果, 当前)**
**当前 = ""**
2. `true`
**当前 = 当前 + 字符**
**下标 = 下标 + 1**
*list.concat(结果, [当前])*
## main
*[分割("甲,乙,,丙"), 分割(""), 分割("a,b")]*
@@ -0,0 +1,66 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"type": "none"
}
},
"type": "map"
},
{
"decimal": "55",
"type": "int"
},
{
"decimal": "370",
"type": "int"
},
{
"decimal": "120",
"type": "int"
},
{
"type": "none"
},
{
"decimal": "3",
"type": "int"
},
{
"entries": {
"code": {
"type": "text",
"value": "hash.full"
},
"message": {
"type": "text",
"value": "哈希表已满"
},
"ok": {
"type": "bool",
"value": false
}
},
"type": "map"
},
{
"decimal": "10",
"type": "int"
},
{
"decimal": "20",
"type": "int"
}
],
"type": "list"
}
}
@@ -0,0 +1,164 @@
## 写32
`(内存, 基址, 值)`
**字节3 = 值 % 256**
**字节2 = 值 // 256 % 256**
**字节1 = 值 // 65536 % 256**
**字节0 = 值 // 16777216 % 256**
**`内存[基址] = 字节0`**
**`内存[基址 + 1] = 字节1`**
**`内存[基址 + 2] = 字节2`**
**`内存[基址 + 3] = 字节3`**
*None*
## 读32
`(内存, 基址)`
*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]*
## 成功
`(值)`
*{ok: true, value: 值}*
## 失败
`(代码, 消息)`
*{ok: false, code: 代码, message: 消息}*
## 初始化
`(内存, 容量)`
头部 8 字节:0..3 是元素数,4..7 是槽数。槽从偏移 8 开始,每槽 8 字节:
键 u32 + 值 u32。键 0 表示空槽,因此业务键必须从 1 开始。
> 写32(内存, 0, 0)
> 写32(内存, 4, 容量)
*成功(None)*
## 插入
`(内存, 键, 值)`
线性探测;键已存在时只更新值,不增加元素数。
**容量 = 读32(内存, 4)**
**位置 = 键 % 容量**
**尝试 = 0**
**完成 = false**
- `尝试 < 容量 and not 完成`
**槽 = 8 + (位置 + 尝试) % 容量 * 8**
**槽键 = 读32(内存, 槽)**
1. `槽键 == 0`
> 写32(内存, 槽, 键)
> 写32(内存, 槽 + 4, 值)
> 写32(内存, 0, 读32(内存, 0) + 1)
**完成 = true**
2. `槽键 == 键`
> 写32(内存, 槽 + 4, 值)
**完成 = true**
3. `true`
**尝试 = 尝试 + 1**
1. `完成`
*成功(None)*
2. `true`
*失败("hash.full", "哈希表已满")*
## 查找
`(内存, 键)`
遇到空槽即停止;找不到返回 None。
**容量 = 读32(内存, 4)**
**位置 = 键 % 容量**
**尝试 = 0**
**结果 = None**
- `尝试 < 容量`
**槽 = 8 + (位置 + 尝试) % 容量 * 8**
**槽键 = 读32(内存, 槽)**
1. `槽键 == 键`
**结果 = 读32(内存, 槽 + 4)**
**尝试 = 容量**
2. `槽键 == 0`
**尝试 = 容量**
3. `true`
**尝试 = 尝试 + 1**
*结果*
## main
业务键从 1 开始。37 与 5 在 32 个槽中同余,用来验证线性探测。
**内存 = memory.new(1, 1)**
> 初始化(内存, 32)
**插入5 = 插入(内存, 5, 50)**
> 插入(内存, 37, 370)
> 插入(内存, 12, 120)
> 插入(内存, 5, 55)
**内存2 = memory.new(1, 1)**
> 初始化(内存2, 2)
> 插入(内存2, 1, 10)
> 插入(内存2, 2, 20)
**表满 = 插入(内存2, 4, 40)**
*[插入5, 查找(内存, 5), 查找(内存, 37), 查找(内存, 12), 查找(内存, 99), 读32(内存, 0), 表满, 查找(内存2, 1), 查找(内存2, 2)]*
@@ -0,0 +1,59 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"decimal": "10",
"type": "int"
},
{
"items": [
{
"decimal": "65",
"type": "int"
},
{
"decimal": "195",
"type": "int"
},
{
"decimal": "169",
"type": "int"
},
{
"decimal": "228",
"type": "int"
},
{
"decimal": "184",
"type": "int"
},
{
"decimal": "173",
"type": "int"
},
{
"decimal": "240",
"type": "int"
},
{
"decimal": "157",
"type": "int"
},
{
"decimal": "132",
"type": "int"
},
{
"decimal": "158",
"type": "int"
}
],
"type": "list"
}
],
"type": "list"
}
}
@@ -0,0 +1,74 @@
## 编码
`(文本, 内存, 基址)`
把 Unicode 标量序列按 UTF-8 编码写入 `Memory`,返回字节长度。
这是 `.mq.md` 自己实现的编码逻辑,不是 Rust 内建 UTF-8 转换。
**点 = text.codepoints(文本)**
**下标 = 0**
**偏移 = 0**
- `下标 < 长度(点)`
**码点 = 点[下标]**
1. `码点 < 128`
**`内存[基址 + 偏移] = 码点`**
**偏移 = 偏移 + 1**
2. `码点 < 2048`
**`内存[基址 + 偏移] = 192 + 码点 // 64`**
**`内存[基址 + 偏移 + 1] = 128 + 码点 % 64`**
**偏移 = 偏移 + 2**
3. `码点 < 65536`
**`内存[基址 + 偏移] = 224 + 码点 // 4096`**
**`内存[基址 + 偏移 + 1] = 128 + 码点 // 64 % 64`**
**`内存[基址 + 偏移 + 2] = 128 + 码点 % 64`**
**偏移 = 偏移 + 3**
4. `true`
**`内存[基址 + 偏移] = 240 + 码点 // 262144`**
**`内存[基址 + 偏移 + 1] = 128 + 码点 // 4096 % 64`**
**`内存[基址 + 偏移 + 2] = 128 + 码点 // 64 % 64`**
**`内存[基址 + 偏移 + 3] = 128 + 码点 % 64`**
**偏移 = 偏移 + 4**
**下标 = 下标 + 1**
*偏移*
## main
**内存 = memory.new(1, 1)**
**长度 = 编码("Aé中𝄞", 内存, 0)**
**字节 = []**
**下标 = 0**
- `下标 < 长度`
**字节 = list.append(字节, 内存[下标])**
**下标 = 下标 + 1**
*[长度, 字节]*
@@ -0,0 +1,87 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"type": "text",
"value": "Aé中"
}
},
"type": "map"
},
{
"entries": {
"code": {
"type": "text",
"value": "utf8.truncated"
},
"message": {
"type": "text",
"value": "UTF-8 解码失败"
},
"ok": {
"type": "bool",
"value": false
}
},
"type": "map"
},
{
"entries": {
"code": {
"type": "text",
"value": "utf8.invalid_continuation"
},
"message": {
"type": "text",
"value": "UTF-8 解码失败"
},
"ok": {
"type": "bool",
"value": false
}
},
"type": "map"
},
{
"entries": {
"code": {
"type": "text",
"value": "utf8.invalid_scalar"
},
"message": {
"type": "text",
"value": "UTF-8 解码失败"
},
"ok": {
"type": "bool",
"value": false
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"type": "text",
"value": "𝄞"
}
},
"type": "map"
}
],
"type": "list"
}
}
@@ -0,0 +1,185 @@
## 成功
`(值)`
*{ok: true, value: 值}*
## 失败
`(代码, 消息)`
*{ok: false, code: 代码, message: 消息}*
## 续字节
`(字节)`
*字节 >= 128 and 字节 < 192*
## 解码2
`(内存, 基址, 长度, 偏移, 文本)`
1. `偏移 + 1 >= 长度`
*[文本, 偏移, "utf8.truncated"]*
**b1 = 内存[基址 + 偏移 + 1]**
1. `not 续字节(b1)`
*[文本, 偏移, "utf8.invalid_continuation"]*
**码点 = (内存[基址 + 偏移] - 192) * 64 + (b1 - 128)**
1. `码点 < 128`
*[文本, 偏移, "utf8.invalid_scalar"]*
*[文本 + text.from_codepoint(码点), 偏移 + 2, ""]*
## 解码3
`(内存, 基址, 长度, 偏移, 文本)`
1. `偏移 + 2 >= 长度`
*[文本, 偏移, "utf8.truncated"]*
**b1 = 内存[基址 + 偏移 + 1]**
**b2 = 内存[基址 + 偏移 + 2]**
1. `not (续字节(b1) and 续字节(b2))`
*[文本, 偏移, "utf8.invalid_continuation"]*
**码点 = (内存[基址 + 偏移] - 224) * 4096 + (b1 - 128) * 64 + (b2 - 128)**
1. `码点 < 2048 or (码点 >= 55296 and 码点 <= 57343)`
*[文本, 偏移, "utf8.invalid_scalar"]*
*[文本 + text.from_codepoint(码点), 偏移 + 3, ""]*
## 解码4
`(内存, 基址, 长度, 偏移, 文本)`
1. `偏移 + 3 >= 长度`
*[文本, 偏移, "utf8.truncated"]*
**b1 = 内存[基址 + 偏移 + 1]**
**b2 = 内存[基址 + 偏移 + 2]**
**b3 = 内存[基址 + 偏移 + 3]**
1. `not (续字节(b1) and 续字节(b2) and 续字节(b3))`
*[文本, 偏移, "utf8.invalid_continuation"]*
**码点 = (内存[基址 + 偏移] - 240) * 262144 + (b1 - 128) * 4096 + (b2 - 128) * 64 + (b3 - 128)**
1. `码点 < 65536 or 码点 > 1114111`
*[文本, 偏移, "utf8.invalid_scalar"]*
*[文本 + text.from_codepoint(码点), 偏移 + 4, ""]*
## 解码
`(内存, 基址, 长度)`
**偏移 = 0**
**文本 = ""**
**错误 = ""**
- `偏移 < 长度 and 错误 == ""`
**首 = 内存[基址 + 偏移]**
1. `首 < 128`
**文本 = 文本 + text.from_codepoint(首)**
**偏移 = 偏移 + 1**
2. `首 < 194`
**错误 = "utf8.invalid_lead"**
3. `首 < 224`
**结果 = 解码2(内存, 基址, 长度, 偏移, 文本)**
**文本 = 结果[0]**
**偏移 = 结果[1]**
**错误 = 结果[2]**
4. `首 < 240`
**结果 = 解码3(内存, 基址, 长度, 偏移, 文本)**
**文本 = 结果[0]**
**偏移 = 结果[1]**
**错误 = 结果[2]**
5. `首 < 245`
**结果 = 解码4(内存, 基址, 长度, 偏移, 文本)**
**文本 = 结果[0]**
**偏移 = 结果[1]**
**错误 = 结果[2]**
6. `true`
**错误 = "utf8.invalid_lead"**
1. `错误 == ""`
*成功(文本)*
2. `true`
*失败(错误, "UTF-8 解码失败")*
## 写字节
`(内存, 基址, 字节)`
**下标 = 0**
- `下标 < 长度(字节)`
**`内存[基址 + 下标] = 字节[下标]`**
**下标 = 下标 + 1**
*None*
## main
**内存 = memory.new(1, 1)**
> 写字节(内存, 0, [65, 195, 169, 228, 184, 173])
> 写字节(内存, 16, [228, 184])
> 写字节(内存, 32, [228, 65, 173])
> 写字节(内存, 48, [237, 160, 128])
> 写字节(内存, 64, [240, 157, 132, 158])
*[解码(内存, 0, 6), 解码(内存, 16, 2), 解码(内存, 32, 3), 解码(内存, 48, 3), 解码(内存, 64, 4)]*
@@ -0,0 +1,48 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"decimal": "8",
"type": "int"
},
{
"decimal": "24",
"type": "int"
},
{
"decimal": "40",
"type": "int"
},
{
"items": [
{
"decimal": "40",
"type": "int"
},
{
"decimal": "24",
"type": "int"
}
],
"type": "list"
},
{
"items": [
{
"decimal": "40",
"type": "int"
}
],
"type": "list"
},
{
"items": [],
"type": "list"
}
],
"type": "list"
}
}
@@ -0,0 +1,113 @@
## 写32
`(内存, 基址, 值)`
**字节3 = 值 % 256**
**字节2 = 值 // 256 % 256**
**字节1 = 值 // 65536 % 256**
**字节0 = 值 // 16777216 % 256**
**`内存[基址] = 字节0`**
**`内存[基址 + 1] = 字节1`**
**`内存[基址 + 2] = 字节2`**
**`内存[基址 + 3] = 字节3`**
*None*
## 读32
`(内存, 基址)`
*内存[基址] * 16777216 + 内存[基址 + 1] * 65536 + 内存[基址 + 2] * 256 + 内存[基址 + 3]*
## 分配
`(内存, 大小)`
简单 bump 分配;调用方保证容量足够。
**位置 = 读32(内存, 0)**
> 写32(内存, 0, 位置 + 大小)
*位置*
## 新节点
`(内存, 值)`
节点 16 字节:偏移 0 存一个字节的值,偏移 4 存边链表头;0 表示空。
**位置 = 分配(内存, 16)**
**`内存[位置] = 值`**
> 写32(内存, 位置 + 4, 0)
*位置*
## 加边
`(内存, 源, 目标)`
边 8 字节:目标 u32 + 下一条边 u32。头插到源节点的边链表。
**边 = 分配(内存, 8)**
> 写32(内存, 边, 目标)
> 写32(内存, 边 + 4, 读32(内存, 源 + 4))
> 写32(内存, 源 + 4, 边)
*None*
## 邻接
`(内存, 节点)`
返回节点值列表;顺序是头插顺序(后加的边在前)。
**当前 = 读32(内存, 节点 + 4)**
**结果 = []**
- `当前 != 0`
**结果 = list.append(结果, 读32(内存, 当前))**
**当前 = 读32(内存, 当前 + 4)**
*结果*
## main
**内存 = memory.new(1, 1)**
**`内存[0] = 0`**
**`内存[1] = 0`**
**`内存[2] = 0`**
**`内存[3] = 8`**
**甲 = 新节点(内存, 1)**
**乙 = 新节点(内存, 2)**
**丙 = 新节点(内存, 3)**
> 加边(内存, 甲, 乙)
> 加边(内存, 甲, 丙)
> 加边(内存, 乙, 丙)
*[甲, 乙, 丙, 邻接(内存, 甲), 邻接(内存, 乙), 邻接(内存, 丙)]*
@@ -0,0 +1,142 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"entries": {
"next": {
"decimal": "20",
"type": "int"
},
"ok": {
"type": "bool",
"value": true
},
"value": {
"items": [
{
"decimal": "1",
"type": "int"
},
{
"decimal": "23",
"type": "int"
},
{
"items": [
{
"decimal": "4",
"type": "int"
},
{
"decimal": "5",
"type": "int"
}
],
"type": "list"
},
{
"decimal": "-6",
"type": "int"
}
],
"type": "list"
}
},
"type": "map"
},
{
"entries": {
"code": {
"type": "text",
"value": "json.invalid_array"
},
"message": {
"type": "text",
"value": "数组缺少逗号或 ]"
},
"ok": {
"type": "bool",
"value": false
}
},
"type": "map"
},
{
"entries": {
"code": {
"type": "text",
"value": "json.unexpected_eof"
},
"message": {
"type": "text",
"value": "数组未闭合"
},
"ok": {
"type": "bool",
"value": false
}
},
"type": "map"
},
{
"entries": {
"next": {
"decimal": "35",
"type": "int"
},
"ok": {
"type": "bool",
"value": true
},
"value": {
"entries": {
"a": {
"decimal": "1",
"type": "int"
},
"b": {
"items": [
{
"type": "bool",
"value": true
},
{
"type": "none"
},
{
"type": "text",
"value": "说\"好"
}
],
"type": "list"
}
},
"type": "map"
}
},
"type": "map"
},
{
"entries": {
"next": {
"decimal": "6",
"type": "int"
},
"ok": {
"type": "bool",
"value": true
},
"value": {
"type": "text",
"value": "a\nb"
}
},
"type": "map"
}
],
"type": "list"
}
}
@@ -0,0 +1,431 @@
## 成功值
`(值, 下一位置)`
*{ok: true, value: 值, next: 下一位置}*
## 失败
`(代码, 消息)`
*{ok: false, code: 代码, message: 消息}*
## 是空白
`(码点)`
*码点 == 32 or 码点 == 9 or 码点 == 10 or 码点 == 13*
## 是数字
`(码点)`
*码点 >= 48 and 码点 <= 57*
## 跳过空白
`(点, 位置)`
**当前 = 位置**
- `当前 < 长度(点) and 是空白(点[当前])`
**当前 = 当前 + 1**
*当前*
## 匹配
`(点, 位置, 文本)`
**目标 = text.codepoints(文本)**
**下标 = 0**
**结果 = true**
- `下标 < 长度(目标) and 结果`
1. `位置 + 下标 >= 长度(点)`
**结果 = false**
2. `点[位置 + 下标] != 目标[下标]`
**结果 = false**
3. `true`
**继续 = 0**
**下标 = 下标 + 1**
*结果*
## 解析数字
`(点, 位置)`
**负数 = false**
**当前 = 位置**
1. `text.from_codepoint(点[当前]) == "-"`
**负数 = true**
**当前 = 当前 + 1**
2. `true`
**非负 = 0**
**值 = 0**
**开始 = 当前**
- `当前 < 长度(点) and 是数字(点[当前])`
**值 = 值 * 10 + 点[当前] - 48**
**当前 = 当前 + 1**
1. `当前 == 开始`
*失败("json.invalid_number", "数字缺少位数")*
1. `负数`
*成功值(0 - 值, 当前)*
*成功值(值, 当前)*
## 十六进制值
`(码点)`
1. `码点 >= 48 and 码点 <= 57`
*码点 - 48*
2. `码点 >= 65 and 码点 <= 70`
*码点 - 55*
3. `码点 >= 97 and 码点 <= 102`
*码点 - 87*
*0 - 1*
## 解析字符串
`(点, 位置)`
**当前 = 位置 + 1**
**结果 = ""**
**完成 = false**
**错误 = ""**
- `当前 < 长度(点) and not 完成 and 错误 == ""`
**码点 = 点[当前]**
1. `码点 == 34`
**完成 = true**
**当前 = 当前 + 1**
2. `码点 == 92`
1. `当前 + 1 >= 长度(点)`
**错误 = "json.invalid_escape"**
2. `true`
**转义码点 = 点[当前 + 1]**
1. `转义码点 == 34`
**结果 = 结果 + text.from_codepoint(34)**
**当前 = 当前 + 2**
2. `转义码点 == 92`
**结果 = 结果 + text.from_codepoint(92)**
**当前 = 当前 + 2**
3. `转义码点 == 47`
**结果 = 结果 + text.from_codepoint(47)**
**当前 = 当前 + 2**
4. `转义码点 == 98`
**结果 = 结果 + text.from_codepoint(8)**
**当前 = 当前 + 2**
5. `转义码点 == 102`
**结果 = 结果 + text.from_codepoint(12)**
**当前 = 当前 + 2**
6. `转义码点 == 110`
**结果 = 结果 + text.from_codepoint(10)**
**当前 = 当前 + 2**
7. `转义码点 == 114`
**结果 = 结果 + text.from_codepoint(13)**
**当前 = 当前 + 2**
8. `转义码点 == 116`
**结果 = 结果 + text.from_codepoint(9)**
**当前 = 当前 + 2**
9. `转义码点 == 117`
1. `当前 + 5 >= 长度(点)`
**错误 = "json.invalid_escape"**
2. `true`
**数字 = 十六进制值(点[当前 + 2]) * 4096 + 十六进制值(点[当前 + 3]) * 256 + 十六进制值(点[当前 + 4]) * 16 + 十六进制值(点[当前 + 5])**
1. `数字 < 0 or (数字 >= 55296 and 数字 <= 57343)`
**错误 = "json.invalid_escape"**
2. `true`
**结果 = 结果 + text.from_codepoint(数字)**
**当前 = 当前 + 6**
10. `true`
**错误 = "json.invalid_escape"**
3. `true`
**结果 = 结果 + text.from_codepoint(码点)**
**当前 = 当前 + 1**
1. `错误 != ""`
*失败("json.invalid_string", "字符串转义无效")*
1. `not 完成`
*失败("json.unexpected_eof", "字符串未闭合")*
*成功值(结果, 当前)*
## 解析对象
`(点, 位置)`
*解析对象项(点, 位置 + 1, {})*
## 解析对象项
`(点, 位置, 已解析)`
**位置 = 跳过空白(点, 位置)**
1. `位置 >= 长度(点)`
*失败("json.unexpected_eof", "对象未闭合")*
**当前 = text.from_codepoint(点[位置])**
1. `当前 == "}"`
*成功值(已解析, 位置 + 1)*
**键结果 = 解析字符串(点, 位置)**
1. `not 键结果.ok`
*键结果*
**键 = 键结果.value**
**位置 = 跳过空白(点, 键结果.next)**
1. `位置 >= 长度(点)`
*失败("json.unexpected_eof", "对象缺少冒号")*
1. `text.from_codepoint(点[位置]) != ":"`
*失败("json.invalid_object", "对象缺少冒号")*
**位置 = 跳过空白(点, 位置 + 1)**
**值结果 = 解析值(点, 位置)**
1. `not 值结果.ok`
*值结果*
**新的已解析 = map.set(已解析, 键, 值结果.value)**
**位置 = 跳过空白(点, 值结果.next)**
1. `位置 >= 长度(点)`
*失败("json.unexpected_eof", "对象未闭合")*
**分隔 = text.from_codepoint(点[位置])**
1. `分隔 == ","`
*解析对象项(点, 位置 + 1, 新的已解析)*
2. `分隔 == "}"`
*成功值(新的已解析, 位置 + 1)*
3. `true`
*失败("json.invalid_object", "对象缺少逗号或 }")*
## 解析值
`(点, 位置)`
**位置 = 跳过空白(点, 位置)**
1. `位置 >= 长度(点)`
*失败("json.unexpected_eof", "需要值")*
**首 = text.from_codepoint(点[位置])**
**首码点 = 点[位置]**
1. `首码点 == 34`
*解析字符串(点, 位置)*
2. `首码点 == 123`
*解析对象(点, 位置)*
3. `首 == "["`
*解析数组(点, 位置)*
4. `首 == "-" or 是数字(点[位置])`
*解析数字(点, 位置)*
5. `首 == "t"`
1. `匹配(点, 位置, "true")`
*成功值(true, 位置 + 4)*
2. `true`
*失败("json.invalid_literal", "无效字面量")*
6. `首 == "f"`
1. `匹配(点, 位置, "false")`
*成功值(false, 位置 + 5)*
2. `true`
*失败("json.invalid_literal", "无效字面量")*
7. `首 == "n"`
1. `匹配(点, 位置, "null")`
*成功值(None, 位置 + 4)*
2. `true`
*失败("json.invalid_literal", "无效字面量")*
8. `true`
*失败("json.unexpected_token", "不支持的值")*
## 解析数组
`(点, 位置)`
*解析数组项(点, 位置 + 1, [])*
## 解析数组项
`(点, 位置, 已解析)`
**位置 = 跳过空白(点, 位置)**
1. `位置 >= 长度(点)`
*失败("json.unexpected_eof", "数组未闭合")*
**当前 = text.from_codepoint(点[位置])**
1. `当前 == "]"`
*成功值(已解析, 位置 + 1)*
**项 = 解析值(点, 位置)**
1. `not 项.ok`
*项*
**新的已解析 = list.append(已解析, 项.value)**
**位置 = 跳过空白(点, 项.next)**
1. `位置 >= 长度(点)`
*失败("json.unexpected_eof", "数组未闭合")*
**分隔 = text.from_codepoint(点[位置])**
1. `分隔 == ","`
*解析数组项(点, 位置 + 1, 新的已解析)*
2. `分隔 == "]"`
*成功值(新的已解析, 位置 + 1)*
3. `true`
*失败("json.invalid_array", "数组缺少逗号或 ]")*
## main
**`对象结果 = 解析值(text.codepoints("{\"a\": 1, \"b\": [true, null, \"说\\\"好\"]}"), 0)`**
**`字符串结果 = 解析值(text.codepoints("\"a\\nb\""), 0)`**
*[解析值(text.codepoints(" [1, 23, [4, 5], -6] "), 0), 解析值(text.codepoints("[1 2]"), 0), 解析值(text.codepoints("[1,"), 0), 对象结果, 字符串结果]*
@@ -0,0 +1,36 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "36",
"type": "int"
}
},
"type": "map"
},
{
"entries": {
"ok": {
"type": "bool",
"value": true
},
"value": {
"decimal": "42",
"type": "int"
}
},
"type": "map"
}
],
"type": "list"
}
}
@@ -0,0 +1,130 @@
## 成功
`(值)`
*{ok: true, value: 值}*
## 失败
`(代码, 消息)`
*{ok: false, code: 代码, message: 消息}*
## 写字节
`(内存, 基址, 字节)`
**下标 = 0**
- `下标 < 长度(字节)`
**`内存[基址 + 下标] = 字节[下标]`**
**下标 = 下标 + 1**
*None*
## 运行
`(内存, 程序长度, 栈基址)`
程序从偏移 0 开始,栈从 `栈基址` 开始。指令集:
1 = PUSH 立即数,2 = ADD,3 = MUL,5 = JMP,6 = JZ,7 = HALT。
**指令指针 = 0**
**栈指针 = 0**
**运行中 = true**
**错误 = ""**
- `指令指针 < 程序长度 and 运行中 and 错误 == ""`
**操作码 = 内存[指令指针]**
1. `操作码 == 1`
**`内存[栈基址 + 栈指针] = 内存[指令指针 + 1]`**
**栈指针 = 栈指针 + 1**
**指令指针 = 指令指针 + 2**
2. `操作码 == 2`
**右 = 内存[栈基址 + 栈指针 - 1]**
**左 = 内存[栈基址 + 栈指针 - 2]**
**`内存[栈基址 + 栈指针 - 2] = 左 + 右`**
**栈指针 = 栈指针 - 1**
**指令指针 = 指令指针 + 1**
3. `操作码 == 3`
**右 = 内存[栈基址 + 栈指针 - 1]**
**左 = 内存[栈基址 + 栈指针 - 2]**
**`内存[栈基址 + 栈指针 - 2] = 左 * 右`**
**栈指针 = 栈指针 - 1**
**指令指针 = 指令指针 + 1**
4. `操作码 == 5`
**指令指针 = 内存[指令指针 + 1]**
5. `操作码 == 6`
**条件 = 内存[栈基址 + 栈指针 - 1]**
**栈指针 = 栈指针 - 1**
1. `条件 == 0`
**指令指针 = 内存[指令指针 + 1]**
2. `true`
**指令指针 = 指令指针 + 2**
6. `操作码 == 7`
**运行中 = false**
7. `true`
**错误 = "vm.invalid_opcode"**
1. `错误 != ""`
*失败("vm.invalid_opcode", "未知指令")*
1. `栈指针 == 0`
*失败("vm.empty_stack", "没有结果")*
*成功(内存[栈基址 + 栈指针 - 1])*
## main
**内存 = memory.new(1, 1)**
第一段程序计算 `(5 + 7) * 3`。
> 写字节(内存, 0, [1, 5, 1, 7, 2, 1, 3, 3, 7])
**算术结果 = 运行(内存, 9, 1024)**
第二段程序在条件为 0 时跳到偏移 8,压入 42 后停机。
> 写字节(内存, 0, [1, 0, 6, 8, 1, 99, 7, 0, 1, 42, 7])
**分支结果 = 运行(内存, 11, 2048)**
*[算术结果, 分支结果]*
@@ -0,0 +1,46 @@
{
"args": {},
"function": "main",
"stdout": "",
"value": {
"items": [
{
"decimal": "4660",
"type": "int"
},
{
"decimal": "305419896",
"type": "int"
},
{
"decimal": "1311768467463790320",
"type": "int"
},
{
"decimal": "18446744073709551615",
"type": "int"
},
{
"decimal": "4",
"type": "int"
},
{
"decimal": "123456789",
"type": "int"
},
{
"decimal": "4",
"type": "int"
},
{
"decimal": "255",
"type": "int"
},
{
"decimal": "255",
"type": "int"
}
],
"type": "list"
}
}
Loaded 100 of 127 files, more files were not shown because too many files have changed in this diff. Show more