feat(json,csv): bound depth and add multi-value/multi-row APIs

This commit is contained in:
Marqdo Agent committed 2026-11-21 10:51:55 +08:00
1 parent efbb8007e1
commit 94a13880c4
10 files changed
+316 -12

No files matched your search

+9
View File
@@ -236,6 +236,10 @@ fn json_test_parses_and_formats_nested_values() {
assert_eq!(items[4]["value"], false);
assert_eq!(items[5]["value"], "json.invalid_array");
assert_eq!(items[6]["value"], "json.trailing");
assert_eq!(items[7]["value"], false);
assert_eq!(items[8]["value"], "json.depth");
assert_eq!(items[9]["value"], true);
assert_eq!(items[10]["items"][2]["items"][0]["decimal"], "3");
}
#[test]
@@ -260,6 +264,11 @@ fn csv_test_parses_and_formats_quoted_rows() {
let quoted = items[4]["items"].as_array().expect("quoted row");
assert_eq!(quoted[0]["value"], "a");
assert_eq!(quoted[1]["value"], "b");
assert_eq!(items[5]["value"], true);
let rows = items[6]["items"].as_array().expect("rows");
assert_eq!(rows.len(), 2);
assert_eq!(rows[1]["items"][0]["value"], "说\"好");
assert_eq!(items[7]["value"], "甲,\"乙,丙\"\n\"说\"\"好\",2");
}
#[test]
+3 -3
View File
@@ -202,14 +202,14 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
over the existing `resource.*` ABI plus `存在`/`元数据`/`列出目录`/`创建目录`/
`删除`/`重命名`/`追加文本`/`读取片段`/`遍历` over `resource.request`
(ADR-0041/0045); `data/json`/`data/csv`
provide recursive JSON and quoted single-row CSV product parsers; `path` and a
`regex` subset are pure `.mq.md`; `datetime` is pure UTC ISO-8601 and
provide recursive JSON (128-depth bound, multi-value) and quoted single/multi-row
CSV product parsers; `path` and a `regex` subset are pure `.mq.md`; `datetime` is pure UTC ISO-8601 and
`clock.现在` reads `clock.now` in project mode (ADR-0042); `env.读取` reads
only exact `--allow-env NAME` grants in project mode (ADR-0043); `process.运行`
runs an exact `--allow-process PROGRAM` grant with 5 s timeout and 1 MiB caps
(ADR-0044, not an OS sandbox); `random.字节` reads explicitly granted
`random.bytes` entropy (ADR-0046). File watch, binary offset reads, streaming
JSON/CSV, a JSON depth bound and regex groups/alternation remain open.
incremental JSON/CSV streaming and regex groups/alternation remain open.
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
`format`/`text`/`io`/`fs`/`data`/`path`/`regex`/`datetime`/`clock`/`env`/`process`/`random`/`compress`; explicit `std:` imports resolve only inside the bundle;
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
+3 -3
View File
@@ -92,8 +92,8 @@ review RFC, not a new crate per provider.
| project file write | `resource.open("file", path, ["write"])`; buffers until `close`, never flush-on-drop | implemented, project mode | `resource_streams`, `stdlib_product` |
| `std:fs/file` | `读文本/写文本/复制文件/存在/元数据/列出目录/创建目录/删除/重命名/追加文本/读取片段/遍历` over `resource.*`; bounded at 8 MiB | implemented; no watch/binary offset read | `stdlib_product` |
| `resource.request` | `resource.request("fs.exists|metadata|list|mkdir|remove|rename", paths)`; project mode, grant-checked, recoverable fs Result | implemented `fs.*`; no seek/append/watch | `resource_streams`, `stdlib_product` |
| `std:data/json` | `json.解析(文本)` -> Result with nested Value; `json.格式化(值)` -> Text | implemented (recursive, recoverable errors) | `stdlib_product` |
| `std:data/csv` | `csv.解析行(文本)` -> Result with List<Text>; `csv.格式化行(列表)` -> Text | implemented (single row, quoted) | `stdlib_product` |
| `std:data/json` | `json.解析/解析多个/格式化`; nested Value, 128-depth bound, recoverable errors | implemented (recursive, multi-value; no streaming) | `stdlib_product` |
| `std:data/csv` | `csv.解析行/解析/格式化行/格式化`; quoted single/multi-row, CRLF, quoted newlines | implemented (no streaming) | `stdlib_product` |
| `std:path` | `path.连接/文件名/目录名/扩展名/规范/是否绝对`; lexical only | implemented (no I/O) | `stdlib_product` |
| `std:regex` | `regex.匹配/搜索` with `^ $ . * + ? [...]` escapes | implemented subset; no groups/alternation | `stdlib_product` |
| `std:datetime` | `datetime.格式化(毫秒)` / `datetime.解析(文本)`; UTC ISO-8601, pre-1970 supported | implemented (pure) | `stdlib_product` |
@@ -130,7 +130,7 @@ that a phase is complete.
| P0 | Documentation drift between `STATUS.md`, the architecture index and the evidence | being fixed by this migration and the docs gate |
| P1 | `std:fs/file` read/write/copy, metadata/directory, append/read_at/walk are implemented; watch and binary offset reads remain open | open |
| P1 | No file watch or binary `read_at` | open; future Resource Operation extension |
| P1 | JSON/CSV product parsers are implemented; no streaming/multi-line CSV or depth-bounded JSON walker | open |
| P1 | JSON/CSV product parsers are implemented with depth bound and multi-value/multi-row; no incremental streaming parser | open |
| P1 | CLI split into `commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into `host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`; `commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the `Host` trait methods and capability checks | split; shared helpers and trait methods remain by design |
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
| P2 | DNS, UDP, TCP listener and TLS client endpoints are implemented; no TLS server/mTLS or custom-CA CLI | open |
@@ -0,0 +1,35 @@
# Evidence: JSON depth/multi-value and CSV multi-row
Date: 2026-11-21 (Asia/Shanghai)
Source baseline: `efbb8007e172be4662348d73903a2a320d9d6ff9`
Repository: `marqdo-cli`
RFC item: P1 JSON/CSV completeness
## Implemented
- `json.解析` now rejects nesting deeper than 128 with `json.depth`; the
depth scanner ignores string contents and escapes.
- `json.解析多个(文本)` parses whitespace-separated JSON values and returns
`{ok:true,value:List}`, failing atomically on the first bad value.
- `csv.解析(文本)` parses multi-row CSV with quoted fields, `""`, CRLF/LF and
quoted newlines; `csv.格式化(行列表)` rebuilds the text.
- `csv.解析行`/`格式化行` remain the single-row API.
## Tests
`stdlib/tests/json.mq.md` covers 130-level input -> `json.depth` and
multi-value parsing; `stdlib/tests/csv.mq.md` covers quoted multi-row input
and formatting. Both run through `std:` imports in the product lane.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace 490 passed, 0 failed, 2 ignored
cargo test -p marqdo-cli --test stdlib_product 16 passed
python3 tools/pack-stdlib.py PASS
```
## Not covered
No incremental/chunked JSON or CSV parser: both still require the full text in
one Text value. No streaming parser state/Resource kind.
+2 -2
View File
@@ -12,8 +12,8 @@ module; a mismatch is a hard error.
- `io/stream.mq.md` — `写全部` / `写文本` with short-write and close checking
- `io/console.mq.md` — `读取行` / `打印` over `stdio/stdin` and `stdio/stdout`
- `demo.mq.md` — side-by-side comparison with builtin `打印`
- `data/json.mq.md` — recursive JSON parser and formatter (recoverable Result)
- `data/csv.mq.md` — quoted single-row CSV parser and formatter
- `data/json.mq.md` — recursive JSON parser/formatter, 128-depth bound, multi-value
- `data/csv.mq.md` — quoted single/multi-row CSV parser and formatter
- `data/table.mq.md` — value-returning table select/filter/sort/append/set/join/group/GFM
- `env.mq.md` — explicitly granted `env.get` names over `resource.request`
- `process.mq.md` — explicitly granted `process.run` with timeout/output caps
+128
View File
@@ -161,3 +161,131 @@
**下标 = 下标 + 1**
*结果*
## 解析
`(文本)`
解析多行 CSV(逗号分隔、双引号、`""` 转义、CRLF/LF 行尾,引号内可含
换行)。返回 `{ok:true, value:List<List<Text>>}`;未闭合引号和不合法
引号返回可恢复错误。
**点 = text.codepoints(文本)**
**下标 = 0**
**字段 = ""**
**当前行 = []**
**行 = []**
**引号内 = false**
**错误 = ""**
- `下标 < 长度(点) and 错误 == ""`
**字符 = text.from_codepoint(点[下标])**
1. `引号内 and 字符 == "\""`
1. `下标 + 1 < 长度(点) and text.from_codepoint(点[下标 + 1]) == "\""`
**`字段 = 字段 + "\""`**
**下标 = 下标 + 2**
2. `true`
**引号内 = false**
**下标 = 下标 + 1**
2. `引号内`
**字段 = 字段 + 字符**
**下标 = 下标 + 1**
3. `字符 == "\""`
1. `字段 == ""`
**引号内 = true**
**下标 = 下标 + 1**
2. `true`
**错误 = "csv.invalid_quote"**
4. `字符 == ","`
**当前行 = list.append(当前行, 字段)**
**字段 = ""**
**下标 = 下标 + 1**
5. `字符 == "\n"`
**当前行 = list.append(当前行, 字段)**
**行 = list.append(行, 当前行)**
**当前行 = []**
**字段 = ""**
**下标 = 下标 + 1**
6. `字符 == "\r"`
**下标 = 下标 + 1**
7. `true`
**字段 = 字段 + 字符**
**下标 = 下标 + 1**
1. `错误 != ""`
*失败(错误, "未加引号字段中出现双引号")*
1. `引号内`
*失败("csv.unterminated_quote", "引号未闭合")*
1. `字段 != "" or 长度(当前行) > 0`
**当前行 = list.append(当前行, 字段)**
**行 = list.append(行, 当前行)**
*成功(行)*
## 格式化
`(行列表)`
**结果 = ""**
**行号 = 0**
- `行号 < 长度(行列表)`
1. `行号 > 0`
**结果 = 结果 + "\n"**
2. `true`
**忽略 = 0**
**结果 = 结果 + 格式化行(行列表[行号])**
**行号 = 行号 + 1**
*结果*
+107 -1
View File
@@ -1,12 +1,91 @@
## 嵌套深度
`(文本)`
按未转义字符串之外的 `{`/`[` 计算最大嵌套深度。
**点 = text.codepoints(文本)**
**下标 = 0**
**深度 = 0**
**最大 = 0**
**字符串内 = false**
**转义 = false**
- `下标 < 长度(点)`
**码点 = 点[下标]**
1. `字符串内`
1. `转义`
**转义 = false**
2. `码点 == 92`
**转义 = true**
3. `码点 == 34`
**字符串内 = false**
4. `true`
**忽略 = 0**
2. `码点 == 34`
**字符串内 = true**
3. `码点 == 123 or 码点 == 91`
**深度 = 深度 + 1**
1. `深度 > 最大`
**最大 = 深度**
2. `true`
**忽略 = 0**
4. `码点 == 125 or 码点 == 93`
1. `深度 > 0`
**深度 = 深度 - 1**
2. `true`
**忽略 = 0**
5. `true`
**忽略 = 0**
**下标 = 下标 + 1**
*最大*
## 解析
`(文本)`
解析一个完整的 JSON 文本。值之后只允许空白;多余输入返回
`json.trailing`。成功结果只保留 `{ok, value}`,不暴露内部游标。
`json.trailing`,嵌套超过 128 层返回 `json.depth`。成功结果只保留
`{ok, value}`,不暴露内部游标。
**点 = text.codepoints(文本)**
1. `嵌套深度(文本) > 128`
*失败("json.depth", "嵌套深度超过 128")*
**结果 = 解析值(点, 0)**
1. `not 结果.ok`
@@ -636,3 +715,30 @@ JSON 数字:可选负号、整数位、可选小数位、可选指数;含 `.
*格式化对象(值)*
*"null"*
## 解析多个
`(文本)`
解析连续多个 JSON 值(空白分隔),返回 `{ok:true, value:List}`。
任一项失败时返回该项的错误,不返回部分结果。
**点 = text.codepoints(文本)**
**位置 = 跳过空白(点, 0)**
**结果 = []**
- `位置 < 长度(点)`
**项 = 解析值(点, 位置)**
1. `not 项.ok`
*项*
**结果 = list.append(结果, 项.value)**
**位置 = 跳过空白(点, 项.next)**
*{ok: true, value: 结果}*
+1 -1
View File
@@ -1,5 +1,5 @@
{
"name": "marqdo-stdlib",
"version": "0.1.0",
"digest": "sha256:1e5087c05a3c1f5ce0581c834a061d12b88cc0e9eed098b766d4ff1a2ecf859e"
"digest": "sha256:770a63bbd1d3c8a62471c0f0c3cb9ee5647656c137a528442f16e3bd63505d5f"
}
+5 -1
View File
@@ -13,4 +13,8 @@ import:
**`框 = csv.解析行("a,\"b\"")`**
*[行.value, csv.格式化行(行.value), 空.value, 坏.code, 框.value]*
**`多行 = csv.解析("甲,\"乙,丙\"\n\"说\"\"好\",2")`**
**格式化 = csv.格式化(多行.value)**
*[行.value, csv.格式化行(行.value), 空.value, 坏.code, 框.value, 多行.ok, 多行.value, 格式化]*
+23 -1
View File
@@ -13,4 +13,26 @@ import:
**`尾随 = json.解析("1 2")`**
*[对象.ok, 对象.value.a, 对象.value.b[2], json.格式化(浮点.value), 失败.ok, 失败.code, 尾随.code]*
**深度文本 = ""**
**深度下标 = 0**
- `深度下标 < 130`
**深度文本 = 深度文本 + "["**
**深度下标 = 深度下标 + 1**
**深度下标 = 0**
- `深度下标 < 130`
**深度文本 = 深度文本 + "]"**
**深度下标 = 深度下标 + 1**
**深度 = json.解析(深度文本)**
**`多个 = json.解析多个("1 2 [3]")`**
*[对象.ok, 对象.value.a, 对象.value.b[2], json.格式化(浮点.value), 失败.ok, 失败.code, 尾随.code, 深度.ok, 深度.code, 多个.ok, 多个.value]*