feat(stdlib): add table logical-value algorithms (CM-A08)

- Add stdlib/data/table.mq.md with select/filter/sort/append/set-cell/join/
  group as value-returning .mq.md functions over table.columns/row/cell.
- Keep Value::Table logical semantics, GFM AST laziness and T0 physical
  storage unchanged; no new table builtin or HostOp (ADR-0032).
- Add stdlib/tests/table.mq.md and a CLI test asserting shapes/order and
  input immutability; refresh the bundle digest.
- Update architecture/status/spec/README/changelog and evidence.

Gate: fmt, clippy -D warnings, 420 tests passed.
This commit is contained in:
cflmy committed 2026-11-20 22:28:52 +08:00
1 parent 49910ece43
commit a8e62a5ba9
12 files changed
+533 -4

No files matched your search

+8 -1
View File
@@ -135,6 +135,10 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
PluginHandle `plugin/export` are read-only observation members (host id
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
- Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md`
provides select/filter/sort/append/set/join/group over the logical
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
storage are unchanged.
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
`format`/`text`/`io`; explicit `std:` imports resolve only inside the bundle;
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
@@ -186,7 +190,7 @@ behavior. This table records only what is in this tree now.
| P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) |
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: publication as a separate external stdlib repository. The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). C01–C06/C11/C12 each have `.mq.md` evidence, including JSON floats and a JSON formatter. |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Still missing: directories, network, generic provider registry, Table storage, strict file/network resources. |
| P4 Table logical/physical migration | **NOT IMPLEMENTED** | `Value::Table` remains `Vec<Vec<Value>>`; no stdlib table algorithms |
| P4 Table logical/physical migration | **PARTIAL** | ADR-0032/CM-A08: `Value::Table` logical semantics unchanged (T0 `Vec<Vec<Value>>`); `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group as value-returning `.mq.md` functions with input immutability. No T1/T2/T3 physical storage, column store or Memory-backed table. |
| P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) |
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
@@ -222,6 +226,9 @@ user-written allocator is explicitly **not** claimed.
- [`evidence/2026-11-20-print-compat-c.md`](evidence/2026-11-20-print-compat-c.md)
— ADR-0031 removal of `HostOp::Print`, no-bundle fail-closed behavior and
embedder compatibility note.
- [`evidence/2026-11-20-table-cm-a08.md`](evidence/2026-11-20-table-cm-a08.md)
— ADR-0032 table logical value, stdlib select/filter/sort/append/set/join/
group, input immutability and T0 storage boundary.
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
— G0..G12 matrix and RC decision.
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
@@ -0,0 +1,96 @@
# ADR-0032: GFM Table logical value, stdlib algorithms and T0 storage (CM-A08)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A08, P2/P4), ADR-0023
Specs: `spec/evaluation.md`, `spec/mapping.md`
Evidence: `docs/evidence/2026-11-20-table-cm-a08.md`
## Context
RFC-CM §6 separates three layers: the GFM source document (canonical Markdown
AST), the logical `Table` value, and the physical storage. The current runtime
uses `Value::Table { header: Vec<String>, rows: Vec<Vec<Value>> }`, which is
correct but concrete. Table algorithms already exist as `.mq.md` conformance
fixtures (filter/sort/join/group), but they are not shipped as a reusable
stdlib module. CM-A08 requires that high-level table algorithms live in the
standard library and that physical storage can change without changing
observable semantics.
## Decision
### 1. Three layers stay separate
* **GFM source**: unchanged. The Markdown parser owns `table/row/cell`,
spans, laziness and rendering. This ADR adds no table execution to Markdown.
* **Logical value**: `Value::Table` keeps ordered headers, ordered rows, cell
values, width validation, equality/display/wire semantics and value-copy
behavior (`b = a` does not alias).
* **Physical storage**: T0 remains `Vec<Vec<Value>>`. This is an implementation
detail; a future T1 controlled interface or T2 column store must preserve the
logical contract. No column store, view or Memory-backed table is introduced
here.
### 2. Standard-library table module
Add `stdlib/data/table.mq.md`, importable as `std:data/table`:
| Function | Semantics |
|---|---|
| `选择列(数据, 列)` | new table with one column, original unchanged |
| `过滤数值大于(数据, 列, 阈值)` | rows whose numeric cell is `>` threshold |
| `排序(数据, 列)` | insertion sort by a numeric column; new table |
| `插入有序(已排序行, 行, 列)` | helper; value-returning |
| `追加行(数据, 行)` | new table with one appended row (width validated by `表`) |
| `设置单元格(数据, 行, 列, 值)` | rebuilds rows; original unchanged |
| `连接(左, 右, 左列, 右列)` | equality join; headers concatenated |
| `分组(数据, 键列, 值列)` | Text key -> List of values |
All functions use only `table.columns/row/cell`, `list.*`, `map.*` and `表`;
none is a new builtin or `HostOp`. The module is ordinary `.mq.md`, loaded
through the verified `std:` bundle and its digest.
### 3. Semantics fixed by this ADR
* Every operation returns a **new** table; inputs are never mutated.
* Row/column coordinates are non-negative; out-of-range is `runtime.index`.
* Numeric algorithms use `Int`/`Float` comparison of cell values; passing a
Text cell where a number is required is a runtime type error, not a silent
coercion.
* GFM cell content such as `**不执行**` or backticked code remains inert data;
no table function parses a cell as executable source.
* Empty tables, zero rows/columns, duplicate headers and Unicode headers keep
their existing behavior; `表` still validates row width.
* Table source spans and document addresses are unchanged; a rebuilt Table is
a logical value, not a rewrite of the source document.
### 4. What is deliberately not done
* No physical storage migration (T1/T2/T3), column store, shared view or
Memory-backed table.
* No `DataFrame`/`TableView` parallel types; `Value::Table` remains the only
table value.
* No per-operation Rust builtin (`table.filter`, `table.join`, ...).
* No mutable table update; `设置单元格` rebuilds.
* No sorting/grouping of heterogeneous mixed-type columns beyond the explicit
numeric/Text contracts.
* No change to GFM parsing or rendering.
## Consequences
* A project can `import: {table: "std:data/table"}` and filter/sort/join/group
tables without new Rust code; this is the P4 migration's first concrete
stdlib artifact.
* The physical representation remains replaceable because algorithms depend
only on the logical observation/construction API.
* Large-table performance is O(n²) for insertion sort/join and copies rows;
P6 performance work (column store, persistent vectors) can change storage
without changing these outputs.
* The bundle digest must be regenerated when this module changes.
## Verification
`docs/evidence/2026-11-20-table-cm-a08.md` records the module self-test and a
CLI project importing `std:data/table` that checks filter/select/append/set/
join/group results and input immutability, plus the full workspace gate.
+3 -2
View File
@@ -6,7 +6,7 @@ normative RFC text stays at its reviewable path under `docs/`.
| Document | Status | Implemented part |
|---|---|---|
| [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). |
| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict refuses all opens; ADR-0027 Compat-A `.mq.md` print prototype, builtin still default). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser/formatter for arrays, objects, strings with escapes, ints/floats/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). No separate external stdlib repo, network/generic providers, P4 Table migration, P5 providers or P6 performance work. |
| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict permits only the stdio console channel; ADR-0027/0030/0031 Compat-C print via bundled stdlib, no legacy host formatter). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser/formatter for arrays, objects, strings with escapes, ints/floats/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). No separate external stdlib repo, network/generic providers, P4 physical storage migration, P5 providers or P6 performance work. |
The RFCs are architecture constraints, not evidence that a feature exists. The
implementation status lives in `docs/STATUS.md`; frozen behavior is in
@@ -28,7 +28,8 @@ implementation status lives in `docs/STATUS.md`; frozen behavior is in
| Full-value stdlib formatting / FunctionRef+PluginHandle observation | [0029](../adr/0029-full-value-formatting-and-handle-observation.md) | Accepted |
| CM-A07 Compat-B print routing | [0030](../adr/0030-print-compat-b-routing.md) | Accepted |
| CM-A07 Compat-C remove legacy print formatting | [0031](../adr/0031-print-compat-c-remove-host-formatting.md) | Accepted |
| Table storage, generic providers, strict file/network resources | — | Not decided; do not implement by implication |
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
| Table physical storage (T1/T2/T3), generic providers, strict file/network resources | — | Not decided; do not implement by implication |
## What a future phase must not do
+84
View File
@@ -0,0 +1,84 @@
# RFC-CM CM-A08 evidence: table logical value and stdlib algorithms
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `49910ec` plus this step's changes.
Decision record: [ADR-0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md)
Related: [P2 table fixtures](2026-11-20-result-and-free-list-p2.md), [STATUS §5](../STATUS.md).
Scope: logical `Value::Table` + stdlib algorithms; **no physical migration**.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
420 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Layers
* GFM source: unchanged parser/AST/laziness.
* Logical value: `Value::Table` keeps ordered headers/rows, width validation,
equality/display/wire and copy (no aliasing).
* Physical storage: T0 `Vec<Vec<Value>>`, unchanged and private to the
implementation; T1/T2/T3 are explicitly not done.
## Stdlib module
`stdlib/data/table.mq.md` (import `std:data/table`):
```text
选择列(数据, 列)
过滤数值大于(数据, 列, 阈值)
排序(数据, 列) / 插入有序(...)
追加行(数据, 行)
设置单元格(数据, 目标行, 目标列, 值)
连接(左, 右, 左列, 右列)
分组(数据, 键列, 值列)
```
All are `.mq.md` functions using `table.columns/row/cell`, `list.*`, `map.*`
and `表`. Every result is a new value; inputs are never mutated.
## Self-test
`stdlib/tests/table.mq.md`:
```text
data = [名称,分数]: 甲87, 乙92, 丙71
选择列(1) -> header [分数]
过滤 >80 -> 甲87, 乙92
排序(1) -> 丙71, 甲87, 乙92
追加行(丁65) -> 4 rows
设置单元格(0,1,99) -> 甲99
连接(左,右,0,0) -> 2 matched rows, 4 columns
分组(类别,值) -> {a:[1,3], b:[2]}
original data -> unchanged
```
`crates/cli/tests/stdlib_table.rs` runs it through the CLI with
`MARQDO_STDLIB_DIR` and asserts every shape/order plus input immutability.
## Not done
* No T1 controlled storage interface, T2 column store, T3 views, or
Memory-backed numeric columns.
* No `DataFrame`/`TableView` value types; no mutable table operations.
* No generic mixed-type sort/group beyond the explicit numeric/Text contracts.
* No change to GFM parsing, rendering, spans or references.
* No external stdlib repository publication and no Windows/macOS evidence.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-cli --test stdlib_table
cargo test --workspace --locked
```