diff --git a/CHANGELOG.md b/CHANGELOG.md index 21f9565..7a239b9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,8 +64,8 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601. additions): fixtures 48–50 implement a big-endian u32 codec, a bump allocator, node allocation and a linked-list traversal/sum, and prove that a failed allocation leaves the allocator header unchanged. This is **not** the - full P2 gate: JSON floats, table group and a frozen-CLI stdlib closure lane - are still missing. + full P2 gate: a separate stdlib package and a frozen-CLI closure lane are + still missing. - `Memory` handles are never transferable: `to_wire` emits a bare rejected tag, `from_wire` fails, and every host/plugin/foreign call rejects arguments that contain a handle. @@ -96,7 +96,8 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601. recursive JSON parser for arrays/objects/strings (62), stack bytecode VM (63), 16/32/64-bit + LEB128 codecs (64), table sort (65), table equality join (67) and quoted/escaped CSV (68). No Rust algorithm, container or parser was - added for these; algorithms remain `.mq.md` code. JSON floats remain open. + added for these; algorithms remain `.mq.md` code. JSON float parsing and a + compact formatter are included. - Resource/stream first slice (RFC-CM CM-A06, ADR-0025): opaque host-issued `Resource` handles, `resource.open/read/write/close`, bytes transferred through `Memory`, stdout write in pure/project under console/output budgets, diff --git a/docs/STATUS.md b/docs/STATUS.md index 19a0824..c983b16 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -166,7 +166,7 @@ behavior. This table records only what is in this tree now. |---|---|---| | P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged | | P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) | -| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: JSON floats/formatter, a separate stdlib repository/package, and the frozen-CLI closure lane. C01–C06/C11/C12 now each have `.mq.md` evidence. | +| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: a separate stdlib repository/package and the frozen-CLI closure lane. C01–C06/C11/C12 now each have `.mq.md` evidence, including JSON floats and a JSON formatter. | | P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict refuses all opens. Fixture 66 + CLI/host tests. CM-A07 is **Compat-A only**: `examples/stdlib` provides an explicitly imported `.mq.md` `format/utf8/io` chain whose output matches builtin `打印` for Unit/None/Bool/Int/Float/Text/List/Map/Table; builtin `打印` remains default, no alias routing, and Formula/Code/FunctionRef/PluginHandle formatting is still unobservable. Still missing: Compat-B/C, directories, network, generic provider registry. | | P4 Table logical/physical migration | **NOT IMPLEMENTED** | `Value::Table` remains `Vec>`; no stdlib table algorithms | | P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) | diff --git a/docs/architecture/README.md b/docs/architecture/README.md index c2aa13c..f59af51 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -6,7 +6,7 @@ normative RFC text stays at its reviewable path under `docs/`. | Document | Status | Implemented part | |---|---|---| | [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). | -| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict refuses all opens; ADR-0027 Compat-A `.mq.md` print prototype, builtin still default). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser for arrays, objects, strings with escapes, integers/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). No separate stdlib repo, JSON floats, network/generic providers, P4 Table migration, P5 providers or P6 performance work. | +| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict refuses all opens; ADR-0027 Compat-A `.mq.md` print prototype, builtin still default). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser/formatter for arrays, objects, strings with escapes, ints/floats/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). No separate stdlib repo, network/generic providers, P4 Table migration, P5 providers or P6 performance work. | The RFCs are architecture constraints, not evidence that a feature exists. The implementation status lives in `docs/STATUS.md`; frozen behavior is in diff --git a/docs/evidence/2026-11-20-result-and-free-list-p2.md b/docs/evidence/2026-11-20-result-and-free-list-p2.md index ac3d570..d04a1fe 100644 --- a/docs/evidence/2026-11-20-result-and-free-list-p2.md +++ b/docs/evidence/2026-11-20-result-and-free-list-p2.md @@ -227,12 +227,12 @@ No Rust graph container, `HostOp` or new `Value` type was added. descent parser over `text.codepoints`: * supports nested arrays and objects, strings with standard escapes including - `\uXXXX`, non-negative and negative integers, `true`/`false`/`null`, - whitespace, and returns `{ok,value,next}`; + `\uXXXX`, integers and floats (including negative and exponent forms), + `true`/`false`/`null`, whitespace, and returns `{ok,value,next}`; +* a compact formatter renders parsed values back to JSON; * errors are `Result` maps: `json.unexpected_eof`, `json.invalid_array`, `json.invalid_object`, `json.invalid_string`, `json.invalid_number`, - `json.invalid_literal`, `json.unexpected_token`; -* floats are explicitly out of scope. + `json.invalid_literal`, `json.unexpected_token`. Observed: @@ -243,6 +243,9 @@ Observed: {"a": 1, "b": [true, null, "说\"好"]} -> ok {a:1, b:[true,null,"说\"好"]}, next=35 "\"a\\nb\"" -> ok "a\nb", next=6 +[1, 2.5, -3e2, 4E-1, {"a": 1.0}, "x"] + -> ok with Float 2.5, -300.0, 0.4, 1.0; formatter => + [1, 2.5, -300.0, 0.4, {"a": 1.0}, "x"] ``` No Rust JSON parser, new `Value` type or `HostOp` was added. @@ -362,8 +365,7 @@ second free in fixture 55 returns that Result instead of corrupting the list. ## What remains open -* JSON floats/formatter, and an independent `marqdo-stdlib` repository plus - frozen-CLI closure lane; +* an independent `marqdo-stdlib` repository plus frozen-CLI closure lane; * CM-A06 `Resource`/`Stream` byte-transfer ABI; EOF/partial semantics are reserved by ADR-0024 but not implemented; * CM-A07–CM-A11 (stdlib naming, Table storage, provider ABI, packaging, diff --git a/tests/README.md b/tests/README.md index 8cfe373..cef5942 100644 --- a/tests/README.md +++ b/tests/README.md @@ -142,13 +142,13 @@ by: * semantic fixture 61: a `Memory`-backed graph adjacency list with 16-byte nodes and 8-byte edges, head-inserted edges and a neighbor traversal; no Rust graph container or new `Value` type; -* semantic fixture 62: a recursive `.mq.md` JSON subset parser for nested +* semantic fixture 62: a recursive `.mq.md` JSON parser/formatter for nested arrays and objects, strings with standard escapes including `\uXXXX`, - integers (including negative), `true`/`false`/`null`, whitespace, `next` - positions and `Result` errors (`json.unexpected_eof`, `json.invalid_array`, - `json.invalid_object`, `json.invalid_string`, `json.invalid_number`, - `json.invalid_literal`, `json.unexpected_token`). Floats are explicitly out - of scope; + ints and floats (including exponents), `true`/`false`/`null`, whitespace, + `next` positions, compact output and `Result` errors + (`json.unexpected_eof`, `json.invalid_array`, `json.invalid_object`, + `json.invalid_string`, `json.invalid_number`, `json.invalid_literal`, + `json.unexpected_token`); * semantic fixture 63: a `Memory`-backed stack bytecode VM with instruction decode, PUSH/ADD/MUL/JMP/JZ/HALT, a stack region and two programs (arithmetic and conditional branch); no Rust VM or interpreter loop was added; diff --git a/tests/conformance/semantics/62-json-array-parser.expected.json b/tests/conformance/semantics/62-json-array-parser.expected.json index c24968a..f7c290a 100644 --- a/tests/conformance/semantics/62-json-array-parser.expected.json +++ b/tests/conformance/semantics/62-json-array-parser.expected.json @@ -135,6 +135,57 @@ } }, "type": "map" + }, + { + "entries": { + "next": { + "decimal": "37", + "type": "int" + }, + "ok": { + "type": "bool", + "value": true + }, + "value": { + "items": [ + { + "decimal": "1", + "type": "int" + }, + { + "type": "float", + "value": 2.5 + }, + { + "type": "float", + "value": -300.0 + }, + { + "type": "float", + "value": 0.4 + }, + { + "entries": { + "a": { + "type": "float", + "value": 1.0 + } + }, + "type": "map" + }, + { + "type": "text", + "value": "x" + } + ], + "type": "list" + } + }, + "type": "map" + }, + { + "type": "text", + "value": "[1, 2.5, -300.0, 0.4, {\"a\": 1.0}, \"x\"]" } ], "type": "list" diff --git a/tests/conformance/semantics/62-json-array-parser.mq.md b/tests/conformance/semantics/62-json-array-parser.mq.md index 067ad00..a7fabd2 100644 --- a/tests/conformance/semantics/62-json-array-parser.mq.md +++ b/tests/conformance/semantics/62-json-array-parser.mq.md @@ -62,43 +62,113 @@ *结果* +## 片段 + +`(点, 起点, 终点)` + +**结果 = ""** + +**下标 = 起点** + +- `下标 < 终点` + + **结果 = 结果 + text.from_codepoint(点[下标])** + + **下标 = 下标 + 1** + +*结果* + ## 解析数字 `(点, 位置)` -**负数 = false** +JSON 数字:可选负号、整数位、可选小数位、可选指数;含 `.`/`e` 返回 Float。 + +**开始 = 位置** **当前 = 位置** 1. `text.from_codepoint(点[当前]) == "-"` - **负数 = true** - **当前 = 当前 + 1** 2. `true` **非负 = 0** -**值 = 0** - -**开始 = 当前** +**整数位 = 0** - `当前 < 长度(点) and 是数字(点[当前])` - **值 = 值 * 10 + 点[当前] - 48** + **整数位 = 整数位 + 1** **当前 = 当前 + 1** -1. `当前 == 开始` +**有小数 = false** - *失败("json.invalid_number", "数字缺少位数")* +**小数位 = 0** -1. `负数` +1. `当前 < 长度(点) and text.from_codepoint(点[当前]) == "."` - *成功值(0 - 值, 当前)* + **有小数 = true** -*成功值(值, 当前)* + **当前 = 当前 + 1** + + - `当前 < 长度(点) and 是数字(点[当前])` + + **小数位 = 小数位 + 1** + + **当前 = 当前 + 1** + +2. `true` + + **忽略 = 0** + +**指数位 = 0** + +**指数标记 = false** + +1. `当前 < 长度(点) and (text.from_codepoint(点[当前]) == "e" or text.from_codepoint(点[当前]) == "E")` + + **有小数 = true** + + **指数标记 = true** + + **当前 = 当前 + 1** + + 1. `当前 < 长度(点) and (text.from_codepoint(点[当前]) == "+" or text.from_codepoint(点[当前]) == "-")` + + **当前 = 当前 + 1** + + 2. `true` + + **忽略指数符号 = 0** + + - `当前 < 长度(点) and 是数字(点[当前])` + + **指数位 = 指数位 + 1** + + **当前 = 当前 + 1** + +1. `整数位 == 0` + + *失败("json.invalid_number", "数字缺少整数位")* + +1. `有小数 and not 指数标记 and 小数位 == 0` + + *失败("json.invalid_number", "小数缺少位数")* + +1. `指数标记 and 指数位 == 0` + + *失败("json.invalid_number", "指数缺少位数")* + +**原文 = 片段(点, 开始, 当前)** + +1. `有小数` + + *成功值(小数(原文), 当前)* + +*成功值(整数(原文), 当前)* ## 十六进制值 @@ -422,10 +492,136 @@ *失败("json.invalid_array", "数组缺少逗号或 ]")* +## 格式化文本 + +`(文本)` + +**点 = text.codepoints(文本)** + +**`结果 = "\""`** + +**下标 = 0** + +- `下标 < 长度(点)` + + **码点 = 点[下标]** + + 1. `码点 == 34` + + **`结果 = 结果 + "\\\""`** + + 2. `码点 == 92` + + **`结果 = 结果 + "\\\\"`** + + 3. `码点 == 10` + + **`结果 = 结果 + "\\n"`** + + 4. `码点 == 9` + + **`结果 = 结果 + "\\t"`** + + 5. `码点 == 13` + + **`结果 = 结果 + "\\r"`** + + 6. `true` + + **结果 = 结果 + text.from_codepoint(码点)** + + **下标 = 下标 + 1** + +*`结果 + "\""`* + +## 格式化列表 + +`(列表)` + +**结果 = "["** + +**下标 = 0** + +- `下标 < 长度(列表)` + + 1. `下标 > 0` + + **结果 = 结果 + ", "** + + 2. `true` + + **忽略 = 0** + + **结果 = 结果 + 格式化(列表[下标])** + + **下标 = 下标 + 1** + +*结果 + "]"* + +## 格式化对象 + +`(映射)` + +**键列表 = map.keys(映射)** + +**结果 = "{"** + +**下标 = 0** + +- `下标 < 长度(键列表)` + + 1. `下标 > 0` + + **结果 = 结果 + ", "** + + 2. `true` + + **忽略 = 0** + + **当前键 = 键列表[下标]** + + **结果 = 结果 + 格式化文本(当前键) + ": " + 格式化(map.get(映射, 当前键))** + + **下标 = 下标 + 1** + +*结果 + "}"* + +## 格式化 + +`(值)` + +**类别 = 类型(值)** + +1. `类别 == "Text"` + + *格式化文本(值)* + +2. `类别 == "None"` + + *"null"* + +3. `类别 == "Bool" or 类别 == "Int" or 类别 == "Float"` + + *文本(值)* + +4. `类别 == "List"` + + *格式化列表(值)* + +5. `类别 == "Map"` + + *格式化对象(值)* + +*"null"* + ## main **`对象结果 = 解析值(text.codepoints("{\"a\": 1, \"b\": [true, null, \"说\\\"好\"]}"), 0)`** **`字符串结果 = 解析值(text.codepoints("\"a\\nb\""), 0)`** -*[解析值(text.codepoints(" [1, 23, [4, 5], -6] "), 0), 解析值(text.codepoints("[1 2]"), 0), 解析值(text.codepoints("[1,"), 0), 对象结果, 字符串结果]* +**`浮点结果 = 解析值(text.codepoints("[1, 2.5, -3e2, 4E-1, {\"a\": 1.0}, \"x\"]"), 0)`** + +**格式化结果 = 格式化(浮点结果.value)** + +*[解析值(text.codepoints(" [1, 23, [4, 5], -6] "), 0), 解析值(text.codepoints("[1 2]"), 0), 解析值(text.codepoints("[1,"), 0), 对象结果, 字符串结果, 浮点结果, 格式化结果]*