feat(fs): add append, read_at and recursive walk
This commit is contained in:
1 parent
7859c26213
commit
6338fa10e9
13 files changed
+479
-10
No files matched your search
+4
-3
@@ -58,7 +58,7 @@ byte-identical).
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
478 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
479 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
```
|
||||
|
||||
The documentation and test migration is local work; re-run this block after it
|
||||
@@ -200,13 +200,14 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
|
||||
storage are unchanged. `std:fs/file` provides bounded `读文本`/`写文本`/`复制文件`
|
||||
over the existing `resource.*` ABI plus `存在`/`元数据`/`列出目录`/`创建目录`/
|
||||
`删除`/`重命名` over `resource.request` (ADR-0041); `data/json`/`data/csv`
|
||||
`删除`/`重命名`/`追加文本`/`读取片段`/`遍历` over `resource.request`
|
||||
(ADR-0041/0045); `data/json`/`data/csv`
|
||||
provide recursive JSON and quoted single-row CSV product parsers; `path` and a
|
||||
`regex` subset are pure `.mq.md`; `datetime` is pure UTC ISO-8601 and
|
||||
`clock.现在` reads `clock.now` in project mode (ADR-0042); `env.读取` reads
|
||||
only exact `--allow-env NAME` grants in project mode (ADR-0043); `process.运行`
|
||||
runs an exact `--allow-process PROGRAM` grant with 5 s timeout and 1 MiB caps
|
||||
(ADR-0044, not an OS sandbox). Append/seek/watch, recursive walk, streaming
|
||||
(ADR-0044, not an OS sandbox). File watch, binary offset reads, streaming
|
||||
JSON/CSV, a JSON depth bound, regex groups/alternation, secure random and
|
||||
compression remain open.
|
||||
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# ADR-0045: `fs.append`, `fs.read_at` and `fs.walk`
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Relates to: ADR-0041
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-21-fs-extended.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0041 added metadata/directory operations, but file content still required
|
||||
a whole-file `resource.open` read and a whole-buffer write. That leaves append,
|
||||
offset reads and recursive directory discovery unavailable to `.mq.md` without
|
||||
adding Rust algorithms to the standard library.
|
||||
|
||||
## Decision
|
||||
|
||||
Extend the existing `fs.*` group of `resource.request`:
|
||||
|
||||
```text
|
||||
resource.request("fs.append", [path, text]) -> {ok:true, value:count}
|
||||
resource.request("fs.read_at", [path, offset, len])-> {ok:true, value:Text}
|
||||
resource.request("fs.walk", [dir]) -> {ok:true, value:List<Text>}
|
||||
```
|
||||
|
||||
* `fs.append` writes UTF-8 text to the end of a project file, creating it when
|
||||
missing. It requires `fs_write`, is bounded by the existing 8 MiB file limit,
|
||||
and never bypasses the path policy.
|
||||
* `fs.read_at` reads at most `len` bytes from `offset` and returns Text. Both
|
||||
values are non-negative integers; `len` is capped at 8 MiB. Invalid UTF-8 is
|
||||
a fatal type error, not silent loss.
|
||||
* `fs.walk` recursively lists regular files under a directory, relative to the
|
||||
project root. It requires `fs_read`, skips symlinks, has a depth limit of 64
|
||||
and a 4096-entry limit. The list is sorted by name at each level.
|
||||
* All three are project-mode only, use the same path/grant checks as the other
|
||||
`fs.*` operations, and record one `resource.request` effect event.
|
||||
* No new builtin or host trait method is added; the operation strings extend
|
||||
the existing ABI.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `std:fs/file` exposes `追加文本`, `读取片段` and `遍历` in `.mq.md`.
|
||||
* Binary offset reads, append handles and filesystem watch remain open; a
|
||||
watch API needs a separate event model and ADR.
|
||||
* The operations allocate at most the existing 8 MiB read limit; `walk` is
|
||||
metadata-only and returns names, not contents.
|
||||
@@ -90,7 +90,7 @@ review RFC, not a new crate per provider.
|
||||
| stdio stderr | `resource.open("stdio","stderr",["write"])`; separate from stdout and CLI diagnostics | implemented | `resource_streams`, `stdlib_product` |
|
||||
| project file read | `resource.open("file", path, ["read"])`; path/grant checked; bounded | implemented, project mode | `resource_streams`, `stdlib_product` |
|
||||
| project file write | `resource.open("file", path, ["write"])`; buffers until `close`, never flush-on-drop | implemented, project mode | `resource_streams`, `stdlib_product` |
|
||||
| `std:fs/file` | `fs.读文本`, `fs.写文本`, `fs.复制文件` over `resource.open/read/write/close` and `Memory`; bounded at 8 MiB | implemented (read/write/copy) | `stdlib_product` |
|
||||
| `std:fs/file` | `读文本/写文本/复制文件/存在/元数据/列出目录/创建目录/删除/重命名/追加文本/读取片段/遍历` over `resource.*`; bounded at 8 MiB | implemented; no watch/binary offset read | `stdlib_product` |
|
||||
| `resource.request` | `resource.request("fs.exists|metadata|list|mkdir|remove|rename", paths)`; project mode, grant-checked, recoverable fs Result | implemented `fs.*`; no seek/append/watch | `resource_streams`, `stdlib_product` |
|
||||
| `std:data/json` | `json.解析(文本)` -> Result with nested Value; `json.格式化(值)` -> Text | implemented (recursive, recoverable errors) | `stdlib_product` |
|
||||
| `std:data/csv` | `csv.解析行(文本)` -> Result with List<Text>; `csv.格式化行(列表)` -> Text | implemented (single row, quoted) | `stdlib_product` |
|
||||
@@ -125,8 +125,8 @@ that a phase is complete.
|
||||
|---|---|---|
|
||||
| P0 | Stdlib release is a bundle in this repo; extraction is deferred (ADR-0039) | accepted, keep in repo |
|
||||
| P0 | Documentation drift between `STATUS.md`, the architecture index and the evidence | being fixed by this migration and the docs gate |
|
||||
| P1 | `std:fs/file` read/write/copy and `resource.request` fs metadata/directory operations are implemented; append/seek/watch/recursive walk remain open | open |
|
||||
| P1 | No file append/seek/watch/recursive directory walk | open; future Resource Operation extension |
|
||||
| P1 | `std:fs/file` read/write/copy, metadata/directory, append/read_at/walk are implemented; watch and binary offset reads remain open | open |
|
||||
| P1 | No file watch or binary `read_at` | open; future Resource Operation extension |
|
||||
| P1 | JSON/CSV product parsers are implemented; no streaming/multi-line CSV or depth-bounded JSON walker | open |
|
||||
| P1 | CLI split into `commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into `host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`; `commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the `Host` trait methods and capability checks | split; shared helpers and trait methods remain by design |
|
||||
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
|
||||
@@ -165,5 +165,6 @@ that a phase is complete.
|
||||
- [ADR-0042: `clock.now` resource operation](../adr/0042-clock-resource-operation.md) — accepted project-mode Unix-millis clock.
|
||||
- [ADR-0043: `env.get` resource operation](../adr/0043-env-resource-operation.md) — accepted exact-name environment grants.
|
||||
- [ADR-0044: `process.run` resource operation](../adr/0044-process-resource-operation.md) — accepted exact-path process grant; not a sandbox.
|
||||
- [ADR-0045: fs append/read_at/walk](../adr/0045-fs-append-read-at-walk.md) — accepted bounded content and directory walk.
|
||||
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
|
||||
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
|
||||
@@ -49,6 +49,7 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
| `clock.now` resource operation | [0042](../adr/0042-clock-resource-operation.md) | Accepted (project mode, Unix millis) |
|
||||
| `env.get` resource operation | [0043](../adr/0043-env-resource-operation.md) | Accepted (exact-name project grant) |
|
||||
| `process.run` resource operation | [0044](../adr/0044-process-resource-operation.md) | Accepted (exact program grant, project only, not a sandbox) |
|
||||
| `fs.append`/`fs.read_at`/`fs.walk` | [0045](../adr/0045-fs-append-read-at-walk.md) | Accepted (bounded content, directory walk) |
|
||||
| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | **Not decided**; do not implement by implication |
|
||||
|
||||
## Evidence and status
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
# Evidence: `fs.append`, `fs.read_at` and `fs.walk`
|
||||
|
||||
Date: 2026-11-21 (Asia/Shanghai)
|
||||
Source baseline: `7859c26213bda30b34a5379a4ab1f75430967494`
|
||||
Repository: `marqdo-cli`
|
||||
Decision record: [ADR-0045](../adr/0045-fs-append-read-at-walk.md)
|
||||
|
||||
## Implemented
|
||||
|
||||
`resource.request` now supports:
|
||||
|
||||
- `fs.append(path, text)` -> `{ok:true, value:count}`; creates the file,
|
||||
requires `fs_write`, 8 MiB cap.
|
||||
- `fs.read_at(path, offset, length)` -> `{ok:true, value:Text}`; non-negative
|
||||
integers, 8 MiB cap, invalid UTF-8 is fatal.
|
||||
- `fs.walk(dir)` -> `{ok:true, value:List<Text>}`; recursive regular files,
|
||||
skips symlinks, depth 64, 4096-entry cap.
|
||||
|
||||
`std:fs/file` exposes `追加文本`, `读取片段`, `遍历`.
|
||||
|
||||
## Tests
|
||||
|
||||
`stdlib/tests/fs.mq.md` appends, reads a slice and walks the output directory;
|
||||
`crates/marqdo-host/tests/resource_streams.rs` pins host behavior and the
|
||||
negative-offset type error.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace 479 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-host --test resource_streams 24 passed
|
||||
cargo test -p marqdo-cli --test stdlib_product 14 passed
|
||||
python3 tools/pack-stdlib.py PASS
|
||||
```
|
||||
|
||||
## Not covered
|
||||
|
||||
No file watch, no binary offset read into `Memory`, no seek/resume handle, no
|
||||
write-at-offset, no recursive delete and no symlink traversal.
|
||||
Reference in new issue
Block a user