feat(rfc-cm): add JSON floats and compact formatting

- Replace the JSON number scanner with sign/integer/fraction/exponent
  scanning; return Int or Float via 整数/小数 and reject missing digits.
- Add 片段/格式化文本/格式化列表/格式化对象/格式化 recursive formatter.
- Extend fixture 62 with floats ({1,2.5,-3e2,4E-1,{a:1.0},x}) and a compact
  round-trip string.
- Update P2 status: only the separate stdlib package and frozen-CLI closure
  lane remain.

Gate: fmt, clippy -D warnings, 407 tests passed.
This commit is contained in:
cflmy committed 2026-11-20 21:26:56 +08:00
1 parent 5d3a5fb13b
commit 1a978c7a42
7 files changed
+280 -30

No files matched your search

+4 -3
View File
@@ -64,8 +64,8 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
additions): fixtures 48–50 implement a big-endian u32 codec, a bump
allocator, node allocation and a linked-list traversal/sum, and prove that a
failed allocation leaves the allocator header unchanged. This is **not** the
full P2 gate: JSON floats, table group and a frozen-CLI stdlib closure lane
are still missing.
full P2 gate: a separate stdlib package and a frozen-CLI closure lane are
still missing.
- `Memory` handles are never transferable: `to_wire` emits a bare rejected tag,
`from_wire` fails, and every host/plugin/foreign call rejects arguments that
contain a handle.
@@ -96,7 +96,8 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
recursive JSON parser for arrays/objects/strings (62), stack bytecode VM
(63), 16/32/64-bit + LEB128 codecs (64), table sort (65), table equality join
(67) and quoted/escaped CSV (68). No Rust algorithm, container or parser was
added for these; algorithms remain `.mq.md` code. JSON floats remain open.
added for these; algorithms remain `.mq.md` code. JSON float parsing and a
compact formatter are included.
- Resource/stream first slice (RFC-CM CM-A06, ADR-0025): opaque host-issued
`Resource` handles, `resource.open/read/write/close`, bytes transferred
through `Memory`, stdout write in pure/project under console/output budgets,
+1 -1
View File
@@ -166,7 +166,7 @@ behavior. This table records only what is in this tree now.
|---|---|---|
| P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged |
| P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) |
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: JSON floats/formatter, a separate stdlib repository/package, and the frozen-CLI closure lane. C01–C06/C11/C12 now each have `.mq.md` evidence. |
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: a separate stdlib repository/package and the frozen-CLI closure lane. C01–C06/C11/C12 now each have `.mq.md` evidence, including JSON floats and a JSON formatter. |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict refuses all opens. Fixture 66 + CLI/host tests. CM-A07 is **Compat-A only**: `examples/stdlib` provides an explicitly imported `.mq.md` `format/utf8/io` chain whose output matches builtin `打印` for Unit/None/Bool/Int/Float/Text/List/Map/Table; builtin `打印` remains default, no alias routing, and Formula/Code/FunctionRef/PluginHandle formatting is still unobservable. Still missing: Compat-B/C, directories, network, generic provider registry. |
| P4 Table logical/physical migration | **NOT IMPLEMENTED** | `Value::Table` remains `Vec<Vec<Value>>`; no stdlib table algorithms |
| P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) |
+1 -1
View File
@@ -6,7 +6,7 @@ normative RFC text stays at its reviewable path under `docs/`.
| Document | Status | Implemented part |
|---|---|---|
| [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). |
| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict refuses all opens; ADR-0027 Compat-A `.mq.md` print prototype, builtin still default). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser for arrays, objects, strings with escapes, integers/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). No separate stdlib repo, JSON floats, network/generic providers, P4 Table migration, P5 providers or P6 performance work. |
| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict refuses all opens; ADR-0027 Compat-A `.mq.md` print prototype, builtin still default). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser/formatter for arrays, objects, strings with escapes, ints/floats/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). No separate stdlib repo, network/generic providers, P4 Table migration, P5 providers or P6 performance work. |
The RFCs are architecture constraints, not evidence that a feature exists. The
implementation status lives in `docs/STATUS.md`; frozen behavior is in
@@ -227,12 +227,12 @@ No Rust graph container, `HostOp` or new `Value` type was added.
descent parser over `text.codepoints`:
* supports nested arrays and objects, strings with standard escapes including
`\uXXXX`, non-negative and negative integers, `true`/`false`/`null`,
whitespace, and returns `{ok,value,next}`;
`\uXXXX`, integers and floats (including negative and exponent forms),
`true`/`false`/`null`, whitespace, and returns `{ok,value,next}`;
* a compact formatter renders parsed values back to JSON;
* errors are `Result` maps: `json.unexpected_eof`, `json.invalid_array`,
`json.invalid_object`, `json.invalid_string`, `json.invalid_number`,
`json.invalid_literal`, `json.unexpected_token`;
* floats are explicitly out of scope.
`json.invalid_literal`, `json.unexpected_token`.
Observed:
@@ -243,6 +243,9 @@ Observed:
{"a": 1, "b": [true, null, "说\"好"]}
-> ok {a:1, b:[true,null,"说\"好"]}, next=35
"\"a\\nb\"" -> ok "a\nb", next=6
[1, 2.5, -3e2, 4E-1, {"a": 1.0}, "x"]
-> ok with Float 2.5, -300.0, 0.4, 1.0; formatter =>
[1, 2.5, -300.0, 0.4, {"a": 1.0}, "x"]
```
No Rust JSON parser, new `Value` type or `HostOp` was added.
@@ -362,8 +365,7 @@ second free in fixture 55 returns that Result instead of corrupting the list.
## What remains open
* JSON floats/formatter, and an independent `marqdo-stdlib` repository plus
frozen-CLI closure lane;
* an independent `marqdo-stdlib` repository plus frozen-CLI closure lane;
* CM-A06 `Resource`/`Stream` byte-transfer ABI; EOF/partial semantics are
reserved by ADR-0024 but not implemented;
* CM-A07–CM-A11 (stdlib naming, Table storage, provider ABI, packaging,
+6 -6
View File
@@ -142,13 +142,13 @@ by:
* semantic fixture 61: a `Memory`-backed graph adjacency list with 16-byte nodes
and 8-byte edges, head-inserted edges and a neighbor traversal; no Rust graph
container or new `Value` type;
* semantic fixture 62: a recursive `.mq.md` JSON subset parser for nested
* semantic fixture 62: a recursive `.mq.md` JSON parser/formatter for nested
arrays and objects, strings with standard escapes including `\uXXXX`,
integers (including negative), `true`/`false`/`null`, whitespace, `next`
positions and `Result` errors (`json.unexpected_eof`, `json.invalid_array`,
`json.invalid_object`, `json.invalid_string`, `json.invalid_number`,
`json.invalid_literal`, `json.unexpected_token`). Floats are explicitly out
of scope;
ints and floats (including exponents), `true`/`false`/`null`, whitespace,
`next` positions, compact output and `Result` errors
(`json.unexpected_eof`, `json.invalid_array`, `json.invalid_object`,
`json.invalid_string`, `json.invalid_number`, `json.invalid_literal`,
`json.unexpected_token`);
* semantic fixture 63: a `Memory`-backed stack bytecode VM with instruction
decode, PUSH/ADD/MUL/JMP/JZ/HALT, a stack region and two programs (arithmetic
and conditional branch); no Rust VM or interpreter loop was added;
@@ -135,6 +135,57 @@
}
},
"type": "map"
},
{
"entries": {
"next": {
"decimal": "37",
"type": "int"
},
"ok": {
"type": "bool",
"value": true
},
"value": {
"items": [
{
"decimal": "1",
"type": "int"
},
{
"type": "float",
"value": 2.5
},
{
"type": "float",
"value": -300.0
},
{
"type": "float",
"value": 0.4
},
{
"entries": {
"a": {
"type": "float",
"value": 1.0
}
},
"type": "map"
},
{
"type": "text",
"value": "x"
}
],
"type": "list"
}
},
"type": "map"
},
{
"type": "text",
"value": "[1, 2.5, -300.0, 0.4, {\"a\": 1.0}, \"x\"]"
}
],
"type": "list"
@@ -62,43 +62,113 @@
*结果*
## 片段
`(点, 起点, 终点)`
**结果 = ""**
**下标 = 起点**
- `下标 < 终点`
**结果 = 结果 + text.from_codepoint(点[下标])**
**下标 = 下标 + 1**
*结果*
## 解析数字
`(点, 位置)`
**负数 = false**
JSON 数字:可选负号、整数位、可选小数位、可选指数;含 `.`/`e` 返回 Float。
**开始 = 位置**
**当前 = 位置**
1. `text.from_codepoint(点[当前]) == "-"`
**负数 = true**
**当前 = 当前 + 1**
2. `true`
**非负 = 0**
**值 = 0**
**开始 = 当前**
**整数位 = 0**
- `当前 < 长度(点) and 是数字(点[当前])`
**值 = 值 * 10 + 点[当前] - 48**
**整数位 = 整数位 + 1**
**当前 = 当前 + 1**
1. `当前 == 开始`
**有小数 = false**
*失败("json.invalid_number", "数字缺少位数")*
**小数位 = 0**
1. `负数`
1. `当前 < 长度(点) and text.from_codepoint(点[当前]) == "."`
*成功值(0 - 值, 当前)*
**有小数 = true**
*成功值(值, 当前)*
**当前 = 当前 + 1**
- `当前 < 长度(点) and 是数字(点[当前])`
**小数位 = 小数位 + 1**
**当前 = 当前 + 1**
2. `true`
**忽略 = 0**
**指数位 = 0**
**指数标记 = false**
1. `当前 < 长度(点) and (text.from_codepoint(点[当前]) == "e" or text.from_codepoint(点[当前]) == "E")`
**有小数 = true**
**指数标记 = true**
**当前 = 当前 + 1**
1. `当前 < 长度(点) and (text.from_codepoint(点[当前]) == "+" or text.from_codepoint(点[当前]) == "-")`
**当前 = 当前 + 1**
2. `true`
**忽略指数符号 = 0**
- `当前 < 长度(点) and 是数字(点[当前])`
**指数位 = 指数位 + 1**
**当前 = 当前 + 1**
1. `整数位 == 0`
*失败("json.invalid_number", "数字缺少整数位")*
1. `有小数 and not 指数标记 and 小数位 == 0`
*失败("json.invalid_number", "小数缺少位数")*
1. `指数标记 and 指数位 == 0`
*失败("json.invalid_number", "指数缺少位数")*
**原文 = 片段(点, 开始, 当前)**
1. `有小数`
*成功值(小数(原文), 当前)*
*成功值(整数(原文), 当前)*
## 十六进制值
@@ -422,10 +492,136 @@
*失败("json.invalid_array", "数组缺少逗号或 ]")*
## 格式化文本
`(文本)`
**点 = text.codepoints(文本)**
**`结果 = "\""`**
**下标 = 0**
- `下标 < 长度(点)`
**码点 = 点[下标]**
1. `码点 == 34`
**`结果 = 结果 + "\\\""`**
2. `码点 == 92`
**`结果 = 结果 + "\\\\"`**
3. `码点 == 10`
**`结果 = 结果 + "\\n"`**
4. `码点 == 9`
**`结果 = 结果 + "\\t"`**
5. `码点 == 13`
**`结果 = 结果 + "\\r"`**
6. `true`
**结果 = 结果 + text.from_codepoint(码点)**
**下标 = 下标 + 1**
*`结果 + "\""`*
## 格式化列表
`(列表)`
**结果 = "["**
**下标 = 0**
- `下标 < 长度(列表)`
1. `下标 > 0`
**结果 = 结果 + ", "**
2. `true`
**忽略 = 0**
**结果 = 结果 + 格式化(列表[下标])**
**下标 = 下标 + 1**
*结果 + "]"*
## 格式化对象
`(映射)`
**键列表 = map.keys(映射)**
**结果 = "{"**
**下标 = 0**
- `下标 < 长度(键列表)`
1. `下标 > 0`
**结果 = 结果 + ", "**
2. `true`
**忽略 = 0**
**当前键 = 键列表[下标]**
**结果 = 结果 + 格式化文本(当前键) + ": " + 格式化(map.get(映射, 当前键))**
**下标 = 下标 + 1**
*结果 + "}"*
## 格式化
`(值)`
**类别 = 类型(值)**
1. `类别 == "Text"`
*格式化文本(值)*
2. `类别 == "None"`
*"null"*
3. `类别 == "Bool" or 类别 == "Int" or 类别 == "Float"`
*文本(值)*
4. `类别 == "List"`
*格式化列表(值)*
5. `类别 == "Map"`
*格式化对象(值)*
*"null"*
## main
**`对象结果 = 解析值(text.codepoints("{\"a\": 1, \"b\": [true, null, \"说\\\"好\"]}"), 0)`**
**`字符串结果 = 解析值(text.codepoints("\"a\\nb\""), 0)`**
*[解析值(text.codepoints(" [1, 23, [4, 5], -6] "), 0), 解析值(text.codepoints("[1 2]"), 0), 解析值(text.codepoints("[1,"), 0), 对象结果, 字符串结果]*
**`浮点结果 = 解析值(text.codepoints("[1, 2.5, -3e2, 4E-1, {\"a\": 1.0}, \"x\"]"), 0)`**
**格式化结果 = 格式化(浮点结果.value)**
*[解析值(text.codepoints(" [1, 23, [4, 5], -6] "), 0), 解析值(text.codepoints("[1 2]"), 0), 解析值(text.codepoints("[1,"), 0), 对象结果, 字符串结果, 浮点结果, 格式化结果]*