feat(fs): add append, read_at and recursive walk

This commit is contained in:
Marqdo Agent committed 2026-11-21 10:20:07 +08:00
1 parent 7859c26213
commit 6338fa10e9
13 files changed
+479 -10

No files matched your search

+4 -3
View File
@@ -58,7 +58,7 @@ byte-identical).
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
478 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
479 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
```
The documentation and test migration is local work; re-run this block after it
@@ -200,13 +200,14 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
storage are unchanged. `std:fs/file` provides bounded `读文本`/`写文本`/`复制文件`
over the existing `resource.*` ABI plus `存在`/`元数据`/`列出目录`/`创建目录`/
`删除`/`重命名` over `resource.request` (ADR-0041); `data/json`/`data/csv`
`删除`/`重命名`/`追加文本`/`读取片段`/`遍历` over `resource.request`
(ADR-0041/0045); `data/json`/`data/csv`
provide recursive JSON and quoted single-row CSV product parsers; `path` and a
`regex` subset are pure `.mq.md`; `datetime` is pure UTC ISO-8601 and
`clock.现在` reads `clock.now` in project mode (ADR-0042); `env.读取` reads
only exact `--allow-env NAME` grants in project mode (ADR-0043); `process.运行`
runs an exact `--allow-process PROGRAM` grant with 5 s timeout and 1 MiB caps
(ADR-0044, not an OS sandbox). Append/seek/watch, recursive walk, streaming
(ADR-0044, not an OS sandbox). File watch, binary offset reads, streaming
JSON/CSV, a JSON depth bound, regex groups/alternation, secure random and
compression remain open.
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
+46
View File
@@ -0,0 +1,46 @@
# ADR-0045: `fs.append`, `fs.read_at` and `fs.walk`
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0041
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-21-fs-extended.md`
## Context
ADR-0041 added metadata/directory operations, but file content still required
a whole-file `resource.open` read and a whole-buffer write. That leaves append,
offset reads and recursive directory discovery unavailable to `.mq.md` without
adding Rust algorithms to the standard library.
## Decision
Extend the existing `fs.*` group of `resource.request`:
```text
resource.request("fs.append", [path, text]) -> {ok:true, value:count}
resource.request("fs.read_at", [path, offset, len])-> {ok:true, value:Text}
resource.request("fs.walk", [dir]) -> {ok:true, value:List<Text>}
```
* `fs.append` writes UTF-8 text to the end of a project file, creating it when
missing. It requires `fs_write`, is bounded by the existing 8 MiB file limit,
and never bypasses the path policy.
* `fs.read_at` reads at most `len` bytes from `offset` and returns Text. Both
values are non-negative integers; `len` is capped at 8 MiB. Invalid UTF-8 is
a fatal type error, not silent loss.
* `fs.walk` recursively lists regular files under a directory, relative to the
project root. It requires `fs_read`, skips symlinks, has a depth limit of 64
and a 4096-entry limit. The list is sorted by name at each level.
* All three are project-mode only, use the same path/grant checks as the other
`fs.*` operations, and record one `resource.request` effect event.
* No new builtin or host trait method is added; the operation strings extend
the existing ABI.
## Consequences
* `std:fs/file` exposes `追加文本`, `读取片段` and `遍历` in `.mq.md`.
* Binary offset reads, append handles and filesystem watch remain open; a
watch API needs a separate event model and ADR.
* The operations allocate at most the existing 8 MiB read limit; `walk` is
metadata-only and returns names, not contents.
+4 -3
View File
@@ -90,7 +90,7 @@ review RFC, not a new crate per provider.
| stdio stderr | `resource.open("stdio","stderr",["write"])`; separate from stdout and CLI diagnostics | implemented | `resource_streams`, `stdlib_product` |
| project file read | `resource.open("file", path, ["read"])`; path/grant checked; bounded | implemented, project mode | `resource_streams`, `stdlib_product` |
| project file write | `resource.open("file", path, ["write"])`; buffers until `close`, never flush-on-drop | implemented, project mode | `resource_streams`, `stdlib_product` |
| `std:fs/file` | `fs.读文本`, `fs.写文本`, `fs.复制文件` over `resource.open/read/write/close` and `Memory`; bounded at 8 MiB | implemented (read/write/copy) | `stdlib_product` |
| `std:fs/file` | `读文本/写文本/复制文件/存在/元数据/列出目录/创建目录/删除/重命名/追加文本/读取片段/遍历` over `resource.*`; bounded at 8 MiB | implemented; no watch/binary offset read | `stdlib_product` |
| `resource.request` | `resource.request("fs.exists|metadata|list|mkdir|remove|rename", paths)`; project mode, grant-checked, recoverable fs Result | implemented `fs.*`; no seek/append/watch | `resource_streams`, `stdlib_product` |
| `std:data/json` | `json.解析(文本)` -> Result with nested Value; `json.格式化(值)` -> Text | implemented (recursive, recoverable errors) | `stdlib_product` |
| `std:data/csv` | `csv.解析行(文本)` -> Result with List<Text>; `csv.格式化行(列表)` -> Text | implemented (single row, quoted) | `stdlib_product` |
@@ -125,8 +125,8 @@ that a phase is complete.
|---|---|---|
| P0 | Stdlib release is a bundle in this repo; extraction is deferred (ADR-0039) | accepted, keep in repo |
| P0 | Documentation drift between `STATUS.md`, the architecture index and the evidence | being fixed by this migration and the docs gate |
| P1 | `std:fs/file` read/write/copy and `resource.request` fs metadata/directory operations are implemented; append/seek/watch/recursive walk remain open | open |
| P1 | No file append/seek/watch/recursive directory walk | open; future Resource Operation extension |
| P1 | `std:fs/file` read/write/copy, metadata/directory, append/read_at/walk are implemented; watch and binary offset reads remain open | open |
| P1 | No file watch or binary `read_at` | open; future Resource Operation extension |
| P1 | JSON/CSV product parsers are implemented; no streaming/multi-line CSV or depth-bounded JSON walker | open |
| P1 | CLI split into `commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into `host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`; `commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the `Host` trait methods and capability checks | split; shared helpers and trait methods remain by design |
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
@@ -165,5 +165,6 @@ that a phase is complete.
- [ADR-0042: `clock.now` resource operation](../adr/0042-clock-resource-operation.md) — accepted project-mode Unix-millis clock.
- [ADR-0043: `env.get` resource operation](../adr/0043-env-resource-operation.md) — accepted exact-name environment grants.
- [ADR-0044: `process.run` resource operation](../adr/0044-process-resource-operation.md) — accepted exact-path process grant; not a sandbox.
- [ADR-0045: fs append/read_at/walk](../adr/0045-fs-append-read-at-walk.md) — accepted bounded content and directory walk.
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
+1
View File
@@ -49,6 +49,7 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
| `clock.now` resource operation | [0042](../adr/0042-clock-resource-operation.md) | Accepted (project mode, Unix millis) |
| `env.get` resource operation | [0043](../adr/0043-env-resource-operation.md) | Accepted (exact-name project grant) |
| `process.run` resource operation | [0044](../adr/0044-process-resource-operation.md) | Accepted (exact program grant, project only, not a sandbox) |
| `fs.append`/`fs.read_at`/`fs.walk` | [0045](../adr/0045-fs-append-read-at-walk.md) | Accepted (bounded content, directory walk) |
| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | **Not decided**; do not implement by implication |
## Evidence and status
+39
View File
@@ -0,0 +1,39 @@
# Evidence: `fs.append`, `fs.read_at` and `fs.walk`
Date: 2026-11-21 (Asia/Shanghai)
Source baseline: `7859c26213bda30b34a5379a4ab1f75430967494`
Repository: `marqdo-cli`
Decision record: [ADR-0045](../adr/0045-fs-append-read-at-walk.md)
## Implemented
`resource.request` now supports:
- `fs.append(path, text)` -> `{ok:true, value:count}`; creates the file,
requires `fs_write`, 8 MiB cap.
- `fs.read_at(path, offset, length)` -> `{ok:true, value:Text}`; non-negative
integers, 8 MiB cap, invalid UTF-8 is fatal.
- `fs.walk(dir)` -> `{ok:true, value:List<Text>}`; recursive regular files,
skips symlinks, depth 64, 4096-entry cap.
`std:fs/file` exposes `追加文本`, `读取片段`, `遍历`.
## Tests
`stdlib/tests/fs.mq.md` appends, reads a slice and walks the output directory;
`crates/marqdo-host/tests/resource_streams.rs` pins host behavior and the
negative-offset type error.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace 479 passed, 0 failed, 2 ignored
cargo test -p marqdo-host --test resource_streams 24 passed
cargo test -p marqdo-cli --test stdlib_product 14 passed
python3 tools/pack-stdlib.py PASS
```
## Not covered
No file watch, no binary offset read into `Memory`, no seek/resume handle, no
write-at-offset, no recursive delete and no symlink traversal.