feat(resource): add grant-checked fs resource operations
This commit is contained in:
1 parent
cd90f059b4
commit
fbe36bcc43
16 files changed
+812
-31
No files matched your search
+13
-8
@@ -58,7 +58,7 @@ byte-identical).
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
464 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
467 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
```
|
||||
|
||||
The documentation and test migration is local work; re-run this block after it
|
||||
@@ -199,7 +199,9 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
provides select/filter/sort/append/set/join/group/`转GFM` over the logical
|
||||
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
|
||||
storage are unchanged. `std:fs/file` provides bounded `读文本`/`写文本`/`复制文件`
|
||||
over the existing `resource.*` ABI; metadata/directory operations remain open.
|
||||
over the existing `resource.*` ABI plus `存在`/`元数据`/`列出目录`/`创建目录`/
|
||||
`删除`/`重命名` over `resource.request` (ADR-0041). Append/seek/watch and a
|
||||
recursive walk remain open.
|
||||
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
|
||||
`format`/`text`/`io`/`fs`; explicit `std:` imports resolve only inside the bundle;
|
||||
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
|
||||
@@ -217,9 +219,11 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
stderr are separate capability/output-budget checked buffers; project-file
|
||||
read/write is path/grant checked; file writes buffer until `close` and
|
||||
never flush on drop; a `.mq.md` chunked copy loop reproduces a file without a
|
||||
Rust `copy_file`; strict permits only the stdio stdout console channel and
|
||||
refuses stdin/stderr/file/TCP/provider resources (CM-A11 lane PASS). Semantic
|
||||
fixture 66 and CLI/host tests.
|
||||
Rust `copy_file`; `resource.request` adds non-streaming `fs.exists`/
|
||||
`fs.metadata`/`fs.list`/`fs.mkdir`/`fs.remove`/`fs.rename` (project mode,
|
||||
grant-checked, recoverable fs Result); strict permits only the stdio stdout
|
||||
console channel and refuses stdin/stderr/file/TCP/provider resources (CM-A11
|
||||
lane PASS). Semantic fixture 66 and CLI/host tests.
|
||||
- Test and documentation migration (review RFC-CM P0/P2):
|
||||
`crates/cli/tests/docs_consistency.rs` pins one current overview, the
|
||||
status/index links and SHA-pinned evidence; `crates/cli/tests/architecture_boundaries.rs`
|
||||
@@ -248,9 +252,10 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed
|
||||
broker services; unsupported requests must return `capability.unavailable`.
|
||||
5. Resource streams cover project-file read/write (including `std:fs/file`
|
||||
read/write/copy), stdout, compatible print, trusted in-process providers and
|
||||
IP-literal TCP; file metadata/directories, process, device, DNS/TLS/HTTP, UDP
|
||||
and provider packaging remain open. Strict permits only stdio and refuses the
|
||||
read/write/copy), non-streaming `resource.request` `fs.*` metadata/directory
|
||||
operations, stdout, compatible print, trusted in-process providers and
|
||||
IP-literal TCP; append/seek/watch, process, device, DNS/TLS/HTTP, UDP and
|
||||
provider packaging remain open. Strict permits only stdio and refuses the
|
||||
rest (CM-A11 lane PASS).
|
||||
6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
|
||||
digest-pinned directory with a regular-file executable is supported.
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# ADR-0041: Non-streaming Resource Operation for filesystem metadata and directories
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Relates to: ADR-0025, ADR-0026, ADR-0040
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-21-fs-file-product.md`, `docs/evidence/2026-11-21-resource-operation.md`
|
||||
|
||||
## Context
|
||||
|
||||
`resource.open/read/write/close` is a byte-stream ABI. It cannot express
|
||||
operations that do not return a byte stream: `exists`, `metadata`, directory
|
||||
listing, `mkdir`, `remove` and `rename`. Adding one Rust builtin per operation,
|
||||
or faking them by reading a directory as a file, would either grow the language
|
||||
surface without a contract or overstate what a stream can do.
|
||||
|
||||
The audit RFC's P1 items require a separate, explicit capability class for
|
||||
these non-streaming operations, not a silent `HostOp` for every filesystem
|
||||
call.
|
||||
|
||||
## Decision
|
||||
|
||||
Add one builtin and one host trait method:
|
||||
|
||||
```text
|
||||
resource.request(operation: Text, arguments: List) -> Result
|
||||
Host::resource_request(operation: &str, arguments: Vec<Value>, ctx) -> Result<Value, RuntimeError>
|
||||
```
|
||||
|
||||
* The first version registers only the `fs.*` operation group. An unknown
|
||||
operation is fatal `resource.unsupported`; it is not a recoverable Result.
|
||||
* Arguments must be `Text` paths. `Memory` and `Resource` handles are rejected
|
||||
before dispatch, so they still cannot cross the host boundary.
|
||||
* Every call is project-mode only. `pure` and `strict` return
|
||||
`permission.denied`; this does not widen strict L1.
|
||||
* Grants: `fs.exists`/`fs.metadata`/`fs.list` require `fs_read`; `fs.mkdir`/
|
||||
`fs.remove` require `fs_write`; `fs.rename` requires `fs_write` on both
|
||||
paths. Grants are checked before path resolution or any I/O.
|
||||
* Paths use the same `ProjectFs::resolve` policy: relative, no `..`, no
|
||||
absolute path, canonical target must stay inside the project root.
|
||||
* Expected filesystem conditions are recoverable `{ok:false, code, message}`:
|
||||
`fs.not_found`, `fs.not_directory`, `fs.exists`, `fs.not_empty`. Capability
|
||||
denial, traversal, unsupported operations and I/O failures stay fatal
|
||||
`RuntimeError`s.
|
||||
* `remove` deletes a file or an **empty** directory; it never recurses.
|
||||
`rename` refuses an existing target with `fs.exists` instead of overwriting.
|
||||
* `mkdir` is single-level: a missing parent returns `fs.not_found`.
|
||||
* Every call records one semantic `resource.request` and one effect event
|
||||
with `op=fs.<name>`. No bytes move through the operation; file content still
|
||||
uses `resource.read`/`resource.write`.
|
||||
|
||||
## Consequences
|
||||
|
||||
* The language surface grows by exactly one builtin (`resource.request`), not
|
||||
one per operation. The operation strings are a versioned stdlib ABI, not new
|
||||
Markdown syntax.
|
||||
* A future directory walk, seek, stat or process operation can reuse the same
|
||||
shape without inventing a new `HostOp`; it still needs its own ADR and
|
||||
capability group.
|
||||
* `std:fs/file` is a `.mq.md` facade over both ABIs: byte streams for content,
|
||||
`resource.request` for metadata/directories. No Rust algorithm is added.
|
||||
@@ -18,7 +18,7 @@ platform".
|
||||
| Dimension | Current state | Source |
|
||||
|---|---|---|
|
||||
| Implemented | Rust workspace `0.1.0`: Markdown/frontend/IR, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, bundled stdlib, CLI composition root | `Cargo.toml`, `crates/`, `stdlib/` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **464 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline | `docs/STATUS.md` §3 |
|
||||
| Tested locally | `cargo test --workspace --locked`: **467 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline | `docs/STATUS.md` §3 |
|
||||
| Bundled stdlib released | In-repo bundle `marqdo-stdlib@0.1.0`, SHA-256 digest verified before loading; it is **not** an external registry package | `stdlib/stdlib.json`, `stdlib/README.md` |
|
||||
| Multi-platform verified | Linux local gate is the only executed lane here; Windows/macOS native lanes are **UNVERIFIED**, and the GitHub security lane remains a **VM-only** gate | `docs/STATUS.md` §1, §4 |
|
||||
|
||||
@@ -87,6 +87,7 @@ review RFC, not a new crate per provider.
|
||||
| project file read | `resource.open("file", path, ["read"])`; path/grant checked; bounded | implemented, project mode | `resource_streams`, `stdlib_product` |
|
||||
| project file write | `resource.open("file", path, ["write"])`; buffers until `close`, never flush-on-drop | implemented, project mode | `resource_streams`, `stdlib_product` |
|
||||
| `std:fs/file` | `fs.读文本`, `fs.写文本`, `fs.复制文件` over `resource.open/read/write/close` and `Memory`; bounded at 8 MiB | implemented (read/write/copy) | `stdlib_product` |
|
||||
| `resource.request` | `resource.request("fs.exists|metadata|list|mkdir|remove|rename", paths)`; project mode, grant-checked, recoverable fs Result | implemented `fs.*`; no seek/append/watch | `resource_streams`, `stdlib_product` |
|
||||
| file metadata / directory / rename / remove / mkdir | not part of the current Resource contract | **not implemented**; needs a separate Resource Operation ADR | — |
|
||||
| TCP | `resource.open("tcp", ip:port, ["read","write"])`; exact IP grant, 5 s timeout | implemented; no DNS/TLS/server socket/UDP | `tcp_endpoint`, `http_over_tcp` |
|
||||
| HTTP/1.1 | `stdlib/net/http.mq.md`: status/header/chunked-byte parse, request build | implemented over an already-authorized TCP stream; not a DNS/TLS client | `stdlib_http`, `http_over_tcp`, `stdlib_product` |
|
||||
@@ -112,8 +113,8 @@ that a phase is complete.
|
||||
|---|---|---|
|
||||
| P0 | Stdlib release is a bundle in this repo; extraction is deferred (ADR-0039) | accepted, keep in repo |
|
||||
| P0 | Documentation drift between `STATUS.md`, the architecture index and the evidence | being fixed by this migration and the docs gate |
|
||||
| P1 | `std:fs/file` has read/write/copy; metadata, `exists`, `list`, `mkdir`, `remove` and `rename` need a Resource Operation | open |
|
||||
| P1 | No file stat/list/rename/remove/mkdir/append/seek capability | open; requires a Resource Operation ADR |
|
||||
| P1 | `std:fs/file` read/write/copy and `resource.request` fs metadata/directory operations are implemented; append/seek/watch/recursive walk remain open | open |
|
||||
| P1 | No file append/seek/watch/recursive directory walk | open; future Resource Operation extension |
|
||||
| P1 | JSON/CSV/UTF-8 decode remain mainly under `tests/conformance` | open as product stdlib API |
|
||||
| P1 | `crates/cli/src/commands.rs` and `crates/marqdo-host/src/host.rs` still mix multiple responsibilities | open refactor, behavior-preserving |
|
||||
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
|
||||
@@ -148,5 +149,6 @@ that a phase is complete.
|
||||
- [Implementation status](../STATUS.md) — executed state and blockers.
|
||||
- [Language/frontmatter blueprint](../DEVELOPMENT_BLUEPRINT.md) — design baseline; not current status.
|
||||
- [Stable-release implementation plan](../STABLE_RELEASE_IMPLEMENTATION.md) — next-phase plan; not implemented behavior.
|
||||
- [ADR-0041: non-streaming resource operations](../adr/0041-resource-operation-fs.md) — accepted `resource.request` `fs.*` contract.
|
||||
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
|
||||
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
|
||||
@@ -45,6 +45,7 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
| CM-A09 C ABI provider worker binding v1 | [0038](../adr/0038-c-abi-provider-worker-binding.md) | Accepted (trusted helper) |
|
||||
| Stdlib extraction deferral | [0039](../adr/0039-stdlib-extraction-intent.md) | Accepted (in-repo for now) |
|
||||
| Stdio stdin/stderr plus stdlib stream/console APIs | [0040](../adr/0040-stdio-stdin-stderr-and-stdlib-streams.md) | Accepted (stdin read, separate stdout/stderr, product stdlib) |
|
||||
| Non-streaming `resource.request` `fs.*` operations | [0041](../adr/0041-resource-operation-fs.md) | Accepted (project mode, grant-checked, recoverable fs Result) |
|
||||
| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | **Not decided**; do not implement by implication |
|
||||
|
||||
## Evidence and status
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# Evidence: non-streaming Resource Operation (`fs.*`)
|
||||
|
||||
Date: 2026-11-21 (Asia/Shanghai)
|
||||
Source baseline: `cd90f059b4a91993f89fba5ae63b400dac4bd2c4`
|
||||
Repository: `marqdo-cli`
|
||||
Decision record: [ADR-0041](../adr/0041-resource-operation-fs.md)
|
||||
RFC item: P1 file metadata/directory operations
|
||||
|
||||
## Implemented
|
||||
|
||||
`resource.request(operation, arguments)` adds one builtin and one host trait
|
||||
method. The first operation group is `fs.*`:
|
||||
|
||||
| Operation | Arguments | Grant | Recoverable failures |
|
||||
|---|---|---|---|
|
||||
| `fs.exists` | path | `fs_read` | — |
|
||||
| `fs.metadata` | path | `fs_read` | `fs.not_found` |
|
||||
| `fs.list` | directory | `fs_read` | `fs.not_found`, `fs.not_directory` |
|
||||
| `fs.mkdir` | path | `fs_write` | `fs.exists`, `fs.not_found` |
|
||||
| `fs.remove` | path | `fs_write` | `fs.not_found`, `fs.not_empty` |
|
||||
| `fs.rename` | from, to | `fs_write` on both | `fs.not_found`, `fs.exists` |
|
||||
|
||||
`pure`/`strict` return `permission.denied`; traversal, absolute paths,
|
||||
unknown operations and I/O failures stay fatal. `remove` never recurses and
|
||||
`rename` never overwrites. No file bytes move through this ABI.
|
||||
|
||||
`stdlib/fs/file.mq.md` now exposes `存在`, `元数据`, `是文件`, `是目录`,
|
||||
`列出目录`, `创建目录`, `删除` and `重命名` over `resource.request`, still in
|
||||
`.mq.md`.
|
||||
|
||||
## Commands
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace 467 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-host --test resource_streams 20 passed
|
||||
cargo test -p marqdo-cli --test stdlib_product 6 passed
|
||||
python3 tools/pack-stdlib.py PASS
|
||||
```
|
||||
|
||||
## Not covered
|
||||
|
||||
No append/seek/watch, recursive removal or recursive directory walk. No
|
||||
process/device/DNS/TLS/UDP operation. Those need a later ADR and capability
|
||||
group. `fs.exists`/metadata follow symlinks only when the canonical target
|
||||
stays inside the project root.
|
||||
Reference in new issue
Block a user