feat(rfc-cm): baseline P0-P3 compute machine, stdlib and resource slices

Implements the first vertical slices of RFC-CM-0001/0002 on top of a113d89:

- P1: opaque Memory regions (64 KiB pages, budgets, grow-only), byte
  read/write, Memory-index place assignment, wire/host-boundary rejection.
- CM-A04/A05: value-semantics List/Map/Text/Table primitives and the
  recoverable Result map convention.
- CM-A06: opaque Resource handles, resource.open/read/write/close, stdout
  write, granted project-file read and buffered write, strict fail-closed.
- CM-A07 Compat-A: explicit .mq.md format/utf8/io chain matching builtin
  print for the covered value types; builtin remains default.
- P2 fixtures: allocators, list/tree/graph/hash, integer codecs, UTF-8
  encode/decode, CSV/JSON subsets, table filter/sort/join, bytecode VM.
- ADRs 0022-0027, spec updates, evidence reports and 68 semantic goldens.

Gate: cargo fmt --check, clippy -D warnings, 407 tests passed.
This commit is contained in:
cflmy committed 2026-11-20 21:16:54 +08:00
1 parent a113d89496
commit 92bfbf2568
127 files changed
+11351 -141

No files matched your search

@@ -0,0 +1,183 @@
# ADR-0022: Addressable `Memory` and Memory-index place assignment (RFC-CM P1)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md` (CM-A01/CM-A02/CM-A03),
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (P0/P1), ADR-0007,
ADR-0011, ADR-0017, ADR-0021
Supersedes: the v1 rule that assignment targets are bare identifiers only
Specs: `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`,
`spec/analysis-v2.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-memory-p1.md`
## Context
RFC-CM-0001/0002 define three domains: managed values, addressable regions and
capability resources. The first two are a language change; the third is an OS
boundary and is deliberately **not** part of this ADR.
Before this decision the interpreter had no writable place other than a local
slot. `Value` had no byte region, `a[i]` was read-only, and assignment was
parsed only as `identifier = expression`. A `.mq.md` program therefore could
not express an allocator, an encoded record, a free list or a byte protocol
without a new Rust builtin for every algorithm. That is the closure gap
RFC-CM-0002 §2 identifies.
The decision must not:
* turn managed `List`/`Map`/`Table` values into shared mutable objects;
* expose a host pointer, `usize` address or procedural `free(region)`;
* let an active handle cross the value wire or an ABI boundary;
* add a Markdown node, keyword or `HostOp` for memory;
* claim block-level memory safety for user allocators;
* weaken `strict` or grant any external I/O.
## Decision
### 1. `Value::Memory` is an opaque, run-scoped identity
`MemoryHandle` (`crates/marqdo-value/src/value.rs`) is
`(run_id, region_id, generation)` with private fields and a runtime-only
constructor. It is not `Serialize`/`Deserialize`, is never built from user
input, and carries no host address.
Copying the value, passing it as an argument and storing it in a list or map
all copy the handle and preserve region identity. `==`/`!=` compare identity
only; `Memory` has no ordering comparison. `display()`/`repr()` render the
literal `Memory`, never the internal identity.
### 2. Regions are grow-only, bounded byte arrays
Frozen semantics (CM-A01/CM-A02):
| Property | Decision |
|---|---|
| Page size | 64 KiB logical page (`PAGE_SIZE`) |
| Initial content | Every byte is `0` |
| Growth | `memory.grow` only increases; old offsets/bytes unchanged; new bytes zero |
| Failure atomicity | A rejected create/grow allocates, commits and mutates nothing |
| Shrink/free | No shrink and no user-visible region free; regions are dropped with the run |
| Address | `(region identity, non-negative offset)`; an integer alone is never a credential |
| Read | `m[i] -> Int(0..=255)` |
| Write | `m[i] = byte`, `byte` must be `Int(0..=255)`, no truncation |
| Isolation | Different regions never alias; handle copies of one region do |
| Visibility | `memory.size` returns committed pages, not host capacity or pointers |
Limits in `MemoryLimits`:
* `DEFAULT_MAX_TOTAL_BYTES = 64 MiB` committed across the run;
* `DEFAULT_MAX_REGION_BYTES = 16 MiB` per region, which also caps declared
`max_pages` at 256;
* budget arithmetic uses checked `u64`/`usize` operations; allocation uses
`try_reserve` so a host OOM is a structured `resource.exhausted` instead of
a panic.
Limits are a runtime API (`marqdo_runtime::MemoryLimits`, `run_with_limits`)
and are overridable on `run`/`call` with `--max-memory <BYTES>` (total) and
`--max-region-memory <BYTES>` (per region). The defaults are stated in
`spec/evaluation.md`. Both flags apply in `pure`, `project` and `strict`; they
grant no external capability.
### 3. The only new writable place is a `Memory` index
`AssignTarget` gains `Index { base, index, span }` in the expression AST and
`PlaceTarget` mirrors it in the IR. The surface syntax remains a `strong`
node containing `target = expression`; no Markdown role, heading or list
meaning changes.
* `x = value` still rebinds the current frame's local slot.
* `m[i] = byte` evaluates **base, then index, then RHS**, exactly once each.
All three checks (base is `Memory`, offset in range, byte in `0..=255`)
happen before the single byte write, so a failed statement does not write.
* `list[i] = value`, `map[key] = value` and `obj.field = value` are not
defined. Member targets are a parse error; a non-`Memory` index base is
`runtime.type`. No managed collection changes value semantics.
* Nested index targets are accepted syntactically; the innermost base must
still evaluate to a `Memory` value at run time.
* Assignment remains statement-only; `x = y = 1` is still a parse error.
This is CM-A03.
### 4. Handles cannot cross the wire or the host boundary
* `to_wire(Memory)` emits only `{"type":"memory"}` so an accidental transfer
is visible. `from_wire` rejects that tag with `WireError::Invalid`; a
round-trip is intentionally impossible (no `wire_version` bump: the value
shape is rejected, not newly accepted).
* `Host::invoke` rejects any argument tree containing `Memory` before dispatch
(`runtime.type`), so plugins, foreign kernels, math, file and print adapters
never see a handle. A future explicit byte-snapshot operation must copy bytes
into a managed value and is a separate ADR.
* `HostOp` is unchanged. `memory.new/size/grow` and index read/write are
interpreter operations, not capability-gated host effects.
### 5. Error categories are stable and distinct
| Situation | Code |
|---|---|
| Negative/unrepresentable offset, offset >= region length | `runtime.index` |
| Unknown, stale or cross-run handle; invalid page count (negative, > budget declaration arithmetic) | `runtime.memory` |
| Wrong operand type; byte outside `0..=255`; non-Memory index-assignment base | `runtime.type` |
| Declared max, region budget or total budget exceeded | `resource.exhausted` |
No operation panics on malformed input or forged handles. `expect` is used only
after the same handle has been validated in the same call frame.
### 6. Builtins are part of the language surface, not `HostOp`s
`memory.new(initial_pages, max_pages) -> Memory`,
`memory.size(memory) -> Int`, `memory.grow(memory, pages) -> Int` are added to
`marqdo-ir::builtins::BUILTINS` and `spec/evaluation.md`. They are pure
(`effectful: false`): `pure` mode may use them, `strict` needs no new provider,
and no capability grant is consulted.
### 7. Compatibility
* Local assignment IR keeps its `target` string in Analysis v2; an index
target is an additive object (`{"kind":"index",...}`). Consumers that only
handle locals continue to work.
* Expression AST/statement serde gains an enum tag; this is a pre-1.0 internal
contract documented in `spec/expression.ebnf`.
* Existing Markdown AST goldens, value semantics, `List`/`Map`/`Table`
behaviour and all old builtins are unchanged.
* `Memory` is additive to the value type set. Old programs never produce it.
ABI wire consumers see a new rejected tag, not a renumbered existing tag.
## Non-goals
* No `Pointer` value, raw pointer, `free(region)`, registers or machine code.
* No index/field assignment for managed collections; no implicit mutation.
* No multi-threaded sharing, atomics or data-race guarantees.
* No serialization of live regions; no cross-process shared memory.
* No `ResourceHandle`, `Stream`, files, network or OS syscalls. Those are
CM-A05/CM-A06/CM-A09/CM-A11 and later RFC phases.
* No block-level safety claim for a user-written allocator; the runtime only
bounds a single access to its region.
* No allocator or stdlib API in P1; `--max-memory` only bounds the runtime.
## Consequences
* A `.mq.md` program can now build byte encoders, arenas, free lists and
virtual CPU register files without new Rust code. P2 (the Marqdo-written
stdlib and closure fixtures) becomes testable.
* The interpreter owns one `MemoryStore` per run; `run_with_limits` exposes
the budget. This is runtime state, not a second `HostOp` table.
* Trace gains `memory.new`, `memory.grow` and `memory.write` semantic events
without exposing handles or byte secrets.
* The remaining RFC decisions (CM-A04 and later) stay separate. In particular,
generic structural observation/construction, recoverable `Result`, table
algorithm migration and the resource/Stream ABI are **not** implemented by
this ADR.
## Verification
`docs/evidence/2026-11-20-memory-p1.md` records:
* parser/AST tests for index and rejected member targets;
* runtime store tests for isolation, aliasing, zero-fill, atomic failed grow,
negative/huge offsets, forged/stale handles, byte range and budgets;
* semantic goldens `45`–`47` and runtime negatives `13`–`18`;
* wire rejection test and host-boundary rejection test;
* CLI example `examples/memory-buffer.mq.md` returning `141`;
* Analysis v2 index-target projection;
* `cargo fmt --check`, `cargo clippy -- -D warnings`, `cargo test --workspace`.
@@ -0,0 +1,147 @@
# ADR-0023: Minimal structural observation and construction (RFC-CM CM-A04)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A04, P2),
ADR-0007, ADR-0022
Specs: `spec/evaluation.md`, `spec/expression.ebnf`, `spec/analysis-v2.md`
Evidence: `docs/evidence/2026-11-20-structure-api-p2.md`
## Context
RFC-CM-0002 §3.5 identifies a closure gap: a `.mq.md` program can read a
`List`/`Map`/`Table` and print it through the host, but it cannot generally
**construct** the same structures dynamically or **observe** all of their
logical parts. That makes `format`, JSON/CSV parsers and table algorithms
impossible to write in `.mq.md` without adding a task-specific Rust function
for each algorithm.
The current core already has:
* `List` literals, index reads (including negative offsets), `长度`;
* `Map` record literals, `.field` reads, `长度`;
* `Text` concatenation, character indexing and `长度`;
* `Table` construction through `表(header, rows)`, `长度` (rows), display and
wire round-trip.
It lacks: append/concat, dynamic map update/enumeration, Unicode scalar
conversion, and table column/row/cell projection. These are observation and
construction primitives, not algorithms. Sorting, filtering, joining,
serialising and parsing remain standard-library work.
## Decision
Add a small, namespaced, **pure** builtin set. No new Markdown node, keyword,
`HostOp`, capability or mutation semantics is introduced. All constructors
return **new** values; an input collection is never modified in place. This
keeps INV-02 and preserves existing `List`/`Map`/`Table` copy semantics.
### 1. `List`
| Signature | Semantics |
|---|---|
| `list.append(list: List, value) -> List` | New list with `value` at the end; the input list is unchanged. |
| `list.concat(left: List, right: List) -> List` | New list containing `left` then `right`; both inputs unchanged. |
Neither aliases the other list's element storage: values are copied according
to their normal value semantics (`Memory` handles keep region identity).
### 2. `Map`
| Signature | Semantics |
|---|---|
| `map.keys(map: Map) -> List` | `Text` keys in canonical sorted order (`BTreeMap` order, the order used by `display`). |
| `map.has(map: Map, key: Text) -> Bool` | Whether the key exists. |
| `map.get(map: Map, key: Text) -> Value` | Stored value, or `None` when absent. Use `map.has` when `None` is a legal stored value. |
| `map.set(map: Map, key: Text, value) -> Map` | New map with the key set; the input map is unchanged. |
| `map.remove(map: Map, key: Text) -> Map` | New map without the key (missing key is a no-op); the input map is unchanged. |
Keys are always `Text`; arbitrary-value keys are not introduced. `map.get`
returning `None` is a *lookup result*, not the language's recoverable-error
contract; a full `Result` convention remains CM-A05 and is not frozen here.
### 3. `Text`
`Text` remains a Unicode scalar-value sequence, not a byte array. These
primitives make the existing representation observable without introducing a
second text type or implicit UTF-8 conversion.
| Signature | Semantics |
|---|---|
| `text.codepoints(text: Text) -> List` | Unicode scalar values (`Int`) in code-point order. |
| `text.from_codepoint(codepoint: Int) -> Text or None` | One-scalar `Text`; `None` for a surrogate or a value outside `0..=0x10FFFF`. |
UTF-8 encoding/decoding is **not** a core primitive. It can be written in
`.mq.md` using bytes, `text.codepoints` and `text.from_codepoint`; this keeps
the byte/code-point distinction explicit.
### 4. `Table`
`Value::Table` stays the logical value; its physical `Vec<Vec<Value>>` layout
is not part of this contract. Indices are non-negative row/column coordinates
(unlike `List` negative indexing) because a table cell is a document
coordinate. Bounds failures are `runtime.index`.
| Signature | Semantics |
|---|---|
| `table.columns(table: Table) -> List` | Header names (`Text`) in column order. |
| `table.row(table: Table, row: Int) -> List` | A new list of the row's cells in column order. |
| `table.cell(table: Table, row: Int, column: Int) -> Value` | The cell value. |
No `table.set_cell`/`append_row` builtin is added. Rebuilding a table is a
value-level operation that can be written with `表(header, rows)` once list
construction exists; adding a Rust table algorithm would violate the closure
goal.
### 5. Error and compatibility rules
* Wrong collection/text/table type or key type → `runtime.type`.
* Table row/column outside the logical shape → `runtime.index`.
* `text.from_codepoint` never throws for an invalid scalar; it returns `None`.
* `map.get` never throws for a missing key; it returns `None`.
* Existing `List`/`Map`/`Table` behavior, display, wire encoding and Markdown
laziness are unchanged.
* The builtins are `effectful: false`, available in `pure`, `project` and
`strict`, and add no capability. They do not consume output budget.
* Names are frozen as written; aliases are not added. Namespaced calls use the
existing `Expr::Member` resolution path.
## Non-goals
* No `set_cell`, `append_row`, `sort`, `filter`, `join`, `group`, JSON/CSV or
other algorithm builtins; those belong in `.mq.md`.
* No mutable/aliased List or Map, no in-place mutation, no shared structural
sharing protocol.
* No arbitrary-value map keys, no ordered-map insertion semantics, no
duplicate-key policy change.
* No UTF-8/UTF-16 conversion primitive, no byte-array text type.
* No `Result`/`try`/exception syntax; CM-A05 stays open.
* No change to Table storage, GFM parsing, source spans or `ref` semantics.
## Consequences
* A `.mq.md` program can build maps and lists dynamically, enumerate keys,
project table cells and encode Unicode scalars. This unblocks a Marqdo-
written formatter, table filter/projection and a JSON/CSV parser without
per-algorithm Rust.
* The builtin table grows by 12 pure entries. The migration rule remains:
only observation/construction primitives may enter core; algorithms do not.
* Because constructors are value-returning, repeated `map.set` in a loop is
O(n²) for a BTreeMap clone. That is acceptable for P2 semantics; a
persistent/structural-sharing representation is a P6 performance decision,
not a semantic one.
* `map.get` returning `None` for absence is deliberate and documented; a
future CM-A05 `Result` convention must not silently reinterpret existing
programs.
## Verification
`docs/evidence/2026-11-20-structure-api-p2.md` records:
* ADR/spec deltas and builtin metadata;
* semantic fixtures for value-semantics list/map operations, code-point
round-trips, table projection and invalid code points;
* negative fixtures for wrong types and table bounds;
* unit tests, `cargo fmt --check`, `cargo clippy -- -D warnings` and
`cargo test --workspace`.
@@ -0,0 +1,162 @@
# ADR-0024: Recoverable result convention (RFC-CM CM-A05)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A05, P2/P3),
ADR-0023
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-20-result-and-free-list-p2.md`
## Context
A standard library cannot be written in `.mq.md` if every expected condition
(no free block, invalid input, missing field, later end-of-file or a short
read) must abort the whole run. The core has structured `RuntimeError`s with
stable codes, but no way for an ordinary function to return a failure that the
caller can inspect and handle in a normal branch. RFC-CM-0002 §3.6 recommends a
`Result` data contract using existing `Map` values and control flow, without
`try`/`catch` or exception syntax.
This ADR freezes the **data convention**, not a new runtime type. A separate
core `Result` value, algebraic types or automatic propagation would be a
larger change and is deliberately not introduced.
## Decision
### 1. Shape
A recoverable result is an ordinary `Map`:
```text
success: {ok: true, value: <any Value>}
failure: {ok: false, code: <Text>, message: <Text> [, details: <Map>]}
```
* `ok` is always present and always `Bool`.
* On success, `value` is present. It may itself be `None`, a `List`, a `Map`,
a `Table`, a `Memory` handle or any other value.
* On failure, `code` and `message` are present. `code` is a lowercase
dot-separated stable identifier; `message` is human-readable text.
* `details` is optional and, when present, is a `Map` of additional structured
facts. It must not contain host pointers, resource handles or secret bytes.
* No other fields are required. Consumers must tolerate additive fields.
The constructors are ordinary `.mq.md` functions, for example:
```text
## 成功
`(值)`
*{ok: true, value: 值}*
## 失败
`(代码, 消息)`
*{ok: false, code: 代码, message: 消息}*
```
They are **not** builtins. A library may choose different local function names;
the map shape is the contract.
### 2. What is recoverable, what stays fatal
Recoverable (`Result`):
* a user allocator has no fitting block (`alloc.no_space`);
* a parser rejects malformed input or hits an unexpected end
(`parse.invalid_syntax`, `parse.unexpected_eof`);
* a codec rejects an invalid sequence (`codec.invalid_sequence`);
* a lookup that uses an explicit result because `None` is a legal value;
* later, a resource reports EOF/partial/pending as a **success payload** with
its own byte counts, not as a fatal error (P3 contract).
Fatal `RuntimeError` (unchanged):
* violating a language invariant: wrong operand type, out-of-bounds managed
collection access, unknown/stale `Memory` handle, memory budget exhaustion,
division by zero, call arity, unknown symbol;
* a capability denial (authorization is not a recoverable business result);
* an internal invariant failure.
A library must not convert a fatal invariant into an `Err` by catching it;
there is no catch mechanism and none is proposed here. It must validate inputs
before calling an operation whose failure is fatal, or rely on its own
pre-checks.
### 3. Codes and namespaces
`code` is stable within the library that defines it. Recommended prefixes:
| Prefix | Owner |
|---|---|
| `alloc.*` | allocator libraries |
| `parse.*` | parsers |
| `codec.*` | byte/text codecs |
| `table.*` | table algorithms |
| `io.*` | stream wrappers (P3, when the resource contract exists) |
Codes are compared as `Text`; a caller must not depend on the `message` text.
Libraries may add codes; removing or renaming one is a breaking change for that
library.
### 4. EOF and partial I/O (forward contract)
When the Stream/Resource API arrives (CM-A06), a successful read must
distinguish:
```text
{ok: true, value: {count: Int, eof: Bool, pending: Bool}}
```
EOF is not an `Err`; a short read is not an `Err`; a connection reset or a
permission failure is. The exact stream payload is frozen by CM-A06, not by
this ADR. This ADR only reserves the principle.
### 5. Why not a new type or syntax
* A dedicated `Value::Result` would change the value model, wire format,
display, equality and every exhaustive `Value` match for a convention that
existing `Map` plus `ok` already expresses.
* `try`/`catch` would add a control-flow construct that bypasses ordinary
branches, contradicts the RFC's preference for `Result` values and needs new
grammar, scope and error rules.
* Ordinary `Map` results are visible to `type`, `length`, `map.get` and wire
encoding today, and can be built by `.mq.md` libraries without a new ABI.
## Compatibility
* Additive: existing builtins and programs are unchanged.
* `map.get` returning `None` for a missing key remains a lookup convention,
not a `Result`; callers that need to distinguish a stored `None` use
`map.has` (ADR-0023).
* Runtime error categories and codes are unchanged.
* No wire version bump: a `Result` is an ordinary `Map` already covered by the
wire format.
## Non-goals
* No automatic `?`/propagation, exceptions, panic recovery or stack unwinding.
* No `Result` type, generics, pattern matching or exhaustive `Ok`/`Err`
typing.
* No change to `memory.grow` or other core operations from fatal errors to
`Result`; a library may pre-check and return `Err` instead.
* No resource/stream implementation; CM-A06 remains open.
* No promise that every existing library will adopt the convention.
## Consequences
* `.mq.md` algorithms can return and branch on failure without aborting the
run. This is a prerequisite for a Marqdo-written allocator, parser and
eventually `io.read_line`.
* Because the convention is untyped, a caller can inspect the wrong shape;
`类型(result)`, `map.has(result, "ok")` and the frozen field names are the
checks. A future typed-value ADR may improve this without breaking the data
shape.
* The allocator fixture in `54`/`55` demonstrates success and failure maps and
ordinary branching, with no Rust algorithm.
## Verification
`docs/evidence/2026-11-20-result-and-free-list-p2.md` records a `.mq.md`
first-fit/free-list allocator that returns `Result` maps, reuses freed blocks,
splits free blocks, coalesces address-adjacent free blocks and returns
`alloc.no_space` without aborting the run.
@@ -0,0 +1,173 @@
# ADR-0025: Minimal Resource/Stream ABI and byte transfer through `Memory` (RFC-CM CM-A06)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A06, CM-A09,
CM-A11, P3), ADR-0022, ADR-0024
Specs: `spec/evaluation.md`, `spec/security.md`, `spec/value-wire.md`
Evidence: `docs/evidence/2026-11-20-resource-stream-p3.md`
## Context
P1/P2 give `.mq.md` managed values, addressable `Memory`, structure primitives,
a recoverable `Result` convention and a substantial set of in-language
algorithms. Nothing yet lets a standard-library function perform a real byte
transfer to the outside world without a dedicated `HostOp` per high-level
function.
CM-A06 asks for a stable, authorized, versioned resource boundary: obtain a
granted resource, call the operations it promises, release it. CM-A09 says to
extend the existing host/ABI rather than build a second plugin system. CM-A11
says strict must stay fail-closed until a provider is actually verified.
This ADR freezes a **first, deliberately small slice**:
* one opaque host-issued `ResourceHandle` value;
* `resource.open/read/write/close` as pure language builtins, not new `HostOp`
variants;
* byte transfer through `Memory`, never through host pointers;
* stdio `stdout` write and project-file **read** only;
* `Result` maps for EOF/closed/write counts, fatal errors for invalid handle
or capability denial.
File write, directories, network, processes, TLS, asynchronous I/O and a
generic provider registry remain open.
## Decision
### 1. Opaque handle
`Value::ResourceHandle(ResourceHandle)` is a host-issued opaque identity with
private `(run_id, resource_id, generation)` fields and a hidden constructor.
It is not `Serialize`/`Deserialize`, is rejected by the wire, is never
synthesized from a number or a JSON value, and renders as `Resource` in
`display`/`repr` without exposing the id.
It is a **separate domain** from `Memory`: a handle is not indexable, not
byte-addressable and not a capability token the user can manufacture.
### 2. First resource kinds
| Kind | Descriptor | Rights | Availability |
|---|---|---|---|
| `stdio` | `stdout` | `write` | `pure` and `project` when the console grant is present; `strict` rejects until verified |
| `file` | project-relative path | `read` | `project` with an `fs_read` grant |
`file` streams are opened by reading a bounded regular file (existing
`ProjectFs::MAX_READ_BYTES = 8 MiB` limit) into host-owned bytes; `read`
returns successive chunks and `eof` when the position reaches the end. This is
honest about being a first provider: it does not expose a host file descriptor,
does not support seeking, writing, directories, network, processes, devices or
paths outside the project root.
Any other kind/descriptor/right combination is an explicit
`resource.unsupported`/`permission.denied` diagnostic, never a silent fallback.
### 3. Builtins
```text
resource.open(kind: Text, descriptor: Text, rights: List) -> Resource
resource.read(handle: Resource, memory: Memory, offset: Int, max_bytes: Int)
-> {ok:true, value:{count:Int, eof:Bool}} | {ok:false, code, message}
resource.write(handle: Resource, memory: Memory, offset: Int, length: Int)
-> {ok:true, value:{count:Int}} | {ok:false, code, message}
resource.close(handle: Resource)
-> {ok:true, value:None} | {ok:false, code, message}
```
* `resource.open` is effectful and capability-checked at execution time.
* `resource.read` validates `offset` and the remaining buffer **before** asking
the provider, so a failed range check cannot consume stream data. It caps the
request to the remaining region, writes the returned bytes into `Memory` and
returns the actual count and EOF flag. EOF is a **success** with `eof:true`.
* `resource.write` copies bytes out of `Memory` only for the duration of the
call. A short write is reported as a count; the caller loops. It never
stores a host pointer into a region that could later grow.
* `resource.close` closes the host slot. Reading/writing/closing an already
closed handle returns `{ok:false, code:"resource.closed", ...}` (recoverable).
A forged, cross-run or stale handle is `runtime.resource` (fatal).
* All four are pure interpreter operations over host trait methods; they add no
`HostOp` variant. Invalid capacity/bounds remains `runtime.index`; wrong
operand types remain `runtime.type`.
### 4. Authorization and modes
* Authorization is the intersection of the requested operation, the effective
`Capabilities` grant and the mode:
* `stdio/stdout/write` requires `console` (the same grant `打印` uses);
* `file/.../read` requires an `fs_read` pattern match and project-root path
validation (`..`, absolute paths and symlink escapes rejected);
* `pure`: stdio stdout is allowed because console is already a pure-mode
console grant; **file resources are refused by the mode check**, even if a
caller constructs a `Capabilities` set with a matching `fs_read` pattern.
Network/process resources do not exist in this slice.
* `project`: stdio stdout and explicitly granted project-file reads.
* `strict`: **every** `resource.open` is refused with `permission.denied`.
Strict L1 is unchanged and no new provider is silently enabled. A future ADR
must add real OS-level evidence before strict accepts streams.
* Manifest capability requests are not grants; only the operator-provided
effective set is consulted.
### 5. Errors, budgets and tracing
New stable categories:
| Code | Meaning |
|---|---|
| `runtime.resource` | forged/cross-run/stale handle, invalid resource state |
| `resource.closed` | read/write/close on an already closed handle (recoverable) |
| `resource.unsupported` | kind/descriptor/rights not implemented in this slice |
`resource.write` to stdout charges the existing output budget and appends to
the host output buffer exactly like `打印`; capability denial is
`permission.denied`. Trace records `resource.open`, `resource.read`,
`resource.write` and `resource.close` with kind/count/eof metadata, never raw
file bytes or handles.
### 6. Wire and boundary
* `to_wire(Resource)` emits `{"type":"resource"}` only; decoding it is a hard
error (same rule as `Memory`).
* The generic `Host::invoke` boundary rejects arguments containing a resource
handle, so plugins/foreign kernels/file adapters never receive one.
* Plugin output cannot mint a handle: the constructor is host-only and the
value is not decodable from plugin wire.
### 7. Non-goals
* No `print` migration in this ADR; that is CM-A07 and a later increment.
* File **write** is added by [ADR-0026](0026-buffered-project-file-write-streams.md),
not this decision; this ADR remains read-only for files.
* No file write, directory listing, rename, metadata, network, DNS, TLS,
process, clock, random or device operations.
* No generic third-party provider registration; CM-A09 remains open.
* No zero-copy, async, deadlines per stream, seek or shared memory.
* No strict-mode resource support and no claim that host-in-process I/O is a
sandbox.
## Compatibility
* Additive value variant; old programs never produce `Resource`.
* New builtins only; existing `fs.read/write`, `打印`, plugin and foreign
behavior is unchanged.
* No wire version bump: the new tag is rejected, not accepted.
* New error categories are additive; existing categories/codes are unchanged.
## Consequences
* `.mq.md` can implement `write_text`-style output by encoding text into
`Memory` (fixture 59) and calling `resource.write`, and can implement
chunked file readers with `resource.read`/`Result`/EOF.
* The remaining high-level I/O surface can grow in the standard library without
a new `HostOp` per function.
* The resource table is host state per run; closed slots are not reused in this
slice, so generation checks are straightforward.
* Strict remains conservative; `.mq.md` resource code runs in `pure` (stdout)
or `project` (file read) only.
## Verification
`docs/evidence/2026-11-20-resource-stream-p3.md` records unit tests for grants,
mode refusal, traversal rejection, forged/closed handles, output budget,
file chunking/EOF, wire rejection and a `.mq.md` stdout fixture.
@@ -0,0 +1,115 @@
# ADR-0026: Buffered project-file write streams (CM-A06 extension)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A06, CM-A09, CM-A11), ADR-0024, ADR-0025
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-resource-file-write-p3.md`
## Context
ADR-0025 froze the first resource ABI slice with `stdio/stdout` write and
project-file **read**. A standard library cannot implement file copy, file
format writers or "write then verify" without a write-capable file resource.
The RFC's P3 target is explicit `open/read/write/close` for project files in
`project` mode; strict must remain fail-closed.
Direct per-write file I/O would expose partial writes and OS error timing to
every caller and would make it easy to leave a half-written project file. This
ADR adds the smallest honest write path: an in-memory buffer per file writer,
flushed through the existing project path policy on `close`.
## Decision
### 1. Provider shape
Add a third stream shape to the host resource table (ADR-0025):
```text
resource.open("file", "<project-relative path>", ["write"]) -> Resource
```
* `project` mode only, with an `fs_write` glob grant matching the normalised
relative path.
* `pure` still refuses file resources by mode check; `strict` still refuses
every `resource.open` (CM-A11 fail-closed).
* Absolute paths, `..`, symlink escapes and non-regular targets are rejected by
`ProjectFs` exactly like the existing `fs.write`.
* A writer owns an in-memory byte buffer. `resource.write` appends bytes and
returns `{ok:true, value:{count}}`; it does **not** touch the filesystem.
* `resource.close` flushes the buffer through `ProjectFs::write_bytes` and then
marks the handle closed. If the flush fails, the handle stays open with its
buffer intact and the error is reported; the caller may retry or close again.
* If the program exits without closing, no file is written. This is a
deliberate "no flush on drop" rule: a crash or a missing `close` must not
silently publish a partial file.
* `resource.write` is rejected on a file reader; `resource.read` is rejected on
a file writer; stdout remains write-only. Wrong-direction use is
`resource.unsupported`.
* A writer buffer is capped at the existing 8 MiB project read/write ceiling
(`MAX_READ_BYTES`). Exceeding it is `resource.exhausted` and the append is
atomic: the buffer and file are unchanged.
### 2. Rights and modes
| Kind | Descriptor | Rights | Mode | Grant |
|---|---|---|---|---|
| `file` | project-relative path | `["read"]` | project | `fs_read` glob |
| `file` | project-relative path | `["write"]` | project | `fs_write` glob |
| `stdio` | `stdout` | `["write"]` | pure/project | `console` |
Mixed rights (`["read","write"]`) and other kinds/rights are
`resource.unsupported` in this slice. There is no truncate/append/seek flag:
every writer starts empty and replaces the file on flush.
### 3. Errors, budgets and tracing
* `resource.closed` for read/write/close on a closed handle (recoverable).
* `runtime.resource` for forged/cross-run/stale handles (fatal).
* `resource.exhausted` for a buffer over the byte ceiling.
* `permission.denied` for a missing mode/grant or a path policy failure.
* `resource.unsupported` for a direction/rights combination not implemented.
* `resource.write` on a file writer does not charge the program output budget;
the buffer is bounded by the file ceiling. `resource.close` performs one
`ProjectFs::write_bytes` call and records `resource.close` with the byte
count, never the contents.
* No partial file is visible on a failed close because the write is a single
`std::fs::write` to the validated project path (the same atomicity level as
the existing `fs.write`; no temp-file rename is claimed).
### 4. Explicit non-goals
* No append/truncate/seek flags, no random access, no directory or metadata
operations.
* No file write in pure or strict mode.
* No network/process/device/clock/random resources.
* No generic provider registry (CM-A09), no async/nonblocking I/O.
* No claim that `ProjectFs::write_bytes` is crash-atomic; it is a single
validated write, not a transactional replace.
* No CM-A07 `print` migration in this ADR.
## Compatibility
* Additive to ADR-0025: existing `file` read and `stdio` write behavior is
unchanged; `file` write was previously `resource.unsupported`.
* No new `Value` variant, `HostOp`, Markdown node or wire shape.
* Error categories/codes are unchanged; the new `resource.exhausted` diagnostic
reuses the existing stable `resource.exhausted` code family.
## Consequences
* `.mq.md` can now implement file copy and bounded format writers by opening a
read stream and a write stream, transferring bytes through `Memory`, and
closing both. This is C09's core shape without a `copy_file` Rust builtin.
* Buffered close gives deterministic "all or nothing within one write call"
behavior and keeps OS write errors at a point where the caller still has a
Result path.
* Large files require chunked writes; the 8 MiB ceiling is the first-slice
limit, not a permanent design.
## Verification
`docs/evidence/2026-11-20-resource-file-write-p3.md` records host unit tests
(grants, pure/strict refusal, traversal, buffer ceiling, retry-after-failure),
CLI tests for write-then-read and missing grants, and the full workspace gate.
+103
View File
@@ -0,0 +1,103 @@
# ADR-0027: CM-A07 print-migration staging and the Compat-A `.mq.md` stdlib prototype
Status: **Accepted (Compat-A only; Compat-B/C pending)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A07, CM-A10, P3), ADR-0023, ADR-0024, ADR-0025, ADR-0026
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-20-stdlib-print-compat-a.md`
## Context
RFC-CM-0002 §5.4 and CM-A07 require `打印` to stop being a hidden Rust
`Value::display()` path. The migration has three compatibility stages:
* **Compat-A**: a real `.mq.md` print implementation exists and is callable
through explicit imports; the legacy builtin remains the default.
* **Compat-B**: the `print/打印` name routes to the stdlib implementation
(or an alias table maps it), while old behavior is comparison-tested.
* **Compat-C**: the legacy `HostOp::Print` formatting path is removed from the
public builtin implementation.
This ADR freezes the staging policy and implements **Compat-A only**. It does
not change `打印` resolution, does not remove `HostOp::Print`, and does not
claim CM-A07 complete.
## Decision
### 1. Stage policy
* A stage is only complete when its comparison evidence exists:
* A: explicit stdlib functions exist and their output is compared with the
builtin for the covered value set;
* B: name resolution tests prove a single implementation is selected and old
callers still behave;
* C: all old goldens pass with no `HostOp::Print` formatting path.
* Stages are additive; no stage may silently make a second implementation
reachable for the same name.
* `strict` behavior is unchanged by stdlib-only code; a stdlib print still
needs the console/resource grant.
### 2. Compat-A implementation
Add a normal `.mq.md` module package under `examples/stdlib/`:
```text
examples/stdlib/format.mq.md # 显示 / 列表显示 / 映射显示 / 表格显示 / 转义文本
examples/stdlib/text.mq.md # utf8编码 (Memory bytes)
examples/stdlib/io.mq.md # 写文本 / 写值 (resource.open stdout + write/close)
examples/stdlib/demo.mq.md # side-by-side comparison with 打印
```
`io.写值(值)` = `format.显示(值) + "\n"` → `text.utf8编码` → `Memory` →
`resource.open("stdio","stdout",["write"])` → `resource.write` → `close`.
This uses only the public core/resource contracts; it is ordinary imported
`.mq.md` code, not a new builtin.
Covered value types: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, `List`,
`Map`, `Table`. Formatting matches the builtin `Value::display()` for the
checked corpus, including Text quoting/escaping inside collections, canonical
map key order and table layout.
**Not covered yet**: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The
language currently has no public observation for the module/name/handle
identity needed to reproduce `Value::display()`/`repr()` for those types.
Adding such observation would be a CM-A04 extension and is out of scope.
### 3. Compatibility rules
* `打印` remains the default builtin in this slice. `io.写值` is only reachable
through an explicit import alias; there is no implicit module, bundled stdlib
resolution or global name shadowing.
* No `HostOp` is added or removed. Both paths write through the console/output
budget; the stdlib path additionally goes through the resource ABI.
* Stdlib modules are ordinary Markdown documents: no special parser, no
embedded Rust formatter, no direct formatting syscall.
* A future Compat-B ADR must define name precedence, bundled module lookup,
stdlib version/digest (CM-A10) and failure behavior; this ADR does not.
### 4. Non-goals
* No `print` alias routing, no removal of `HostOp::Print`.
* No bundled stdlib install/version/digest or offline distribution.
* No formatting of Formula/Code/FunctionRef/PluginHandle.
* No locale-aware, width-aware or float-format changes.
* No change to Markdown rendering, `strict`, or the wire/ABI.
* No claim that the examples directory is the shipped standard library.
## Consequences
* The target call chain (`format.display` → `utf8_encode` → `write_all` →
`resource.write`) is exercised end to end in `.mq.md` and can be compared to
the builtin.
* The missing observation for `Formula/Code/FunctionRef/PluginHandle` is now an
explicit, documented blocker for Compat-B rather than an implicit assumption.
* Because the stdlib is not bundled, ordinary users still get the builtin; the
prototype does not yet reduce the runtime's formatting surface.
## Verification
`docs/evidence/2026-11-20-stdlib-print-compat-a.md` records the side-by-side
demo (`io.写值` vs `打印` for a nested list and a table), the CLI test asserting
byte-identical lines, and the full workspace gate. It also records the covered
and uncovered value types.