feat(host): add in-process ResourceProvider v1 (CM-A09 slice)

- Add ResourceProvider v1 trait with name/protocol/kinds/open/read/write/close;
  host still issues the opaque language Resource handle, provider tokens stay
  private.
- Add StdHost::register_resource_provider with protocol/version, reserved-name
  and duplicate-kind checks; add Capabilities.allow_resource_provider grants.
- Provider kinds are project-mode only; pure/strict refuse them; in-process
  providers are trusted and not a sandbox (ADR-0033).
- Reuse resource.* and Result shapes; no parser/Value/runtime/HostOp changes.
- Add mock read/write provider and rejection tests, docs/spec/status/evidence.

Gate: fmt, clippy -D warnings, 423 tests passed.
This commit is contained in:
cflmy committed 2026-11-20 22:38:16 +08:00
1 parent a8e62a5ba9
commit 1be4662049
13 files changed
+640 -18

No files matched your search

+8 -1
View File
@@ -135,6 +135,10 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
PluginHandle `plugin/export` are read-only observation members (host id
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
- In-process ResourceProvider v1 (CM-A09 first slice, ADR-0033): trusted
providers register additional resource kinds without parser/runtime/HostOp
changes; project-mode + name grant required; pure/strict refuse provider
kinds; C/Worker transport remains open.
- Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md`
provides select/filter/sort/append/set/join/group over the logical
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
@@ -191,7 +195,7 @@ behavior. This table records only what is in this tree now.
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: publication as a separate external stdlib repository. The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). C01–C06/C11/C12 each have `.mq.md` evidence, including JSON floats and a JSON formatter. |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Still missing: directories, network, generic provider registry, Table storage, strict file/network resources. |
| P4 Table logical/physical migration | **PARTIAL** | ADR-0032/CM-A08: `Value::Table` logical semantics unchanged (T0 `Vec<Vec<Value>>`); `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group as value-returning `.mq.md` functions with input immutability. No T1/T2/T3 physical storage, column store or Memory-backed table. |
| P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) |
| P5 third-party Provider / TCP / HTTP | **PARTIAL** | ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, network/socket/TCP/HTTP, clock/random/process/device providers, or provider package installation. |
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
The P1 memory domain is pure computation: it needs no grant and is available in
@@ -229,6 +233,9 @@ user-written allocator is explicitly **not** claimed.
- [`evidence/2026-11-20-table-cm-a08.md`](evidence/2026-11-20-table-cm-a08.md)
— ADR-0032 table logical value, stdlib select/filter/sort/append/set/join/
group, input immutability and T0 storage boundary.
- [`evidence/2026-11-20-provider-abi-cm-a09.md`](evidence/2026-11-20-provider-abi-cm-a09.md)
— ADR-0033 in-process ResourceProvider v1, grants/mode checks, mock provider
read/write and registration rejection cases.
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
— G0..G12 matrix and RC decision.
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
@@ -0,0 +1,123 @@
# ADR-0033: In-process ResourceProvider v1 (CM-A09 first slice)
Status: **Accepted (in-process slice only)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A11, P3/P5), ADR-0025, ADR-0026, ADR-0031
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-provider-abi-cm-a09.md`
## Context
ADR-0025/0026 hard-code two resource kinds (`stdio/stdout`, project `file`) in
the host. CM-A09 requires a versioned, discoverable provider contract so a new
device/OS endpoint can be added **without** changing the expression parser,
`Value` model, runtime dispatch or `HostOp` enum. The RFC warns against a
second shadow plugin ecosystem: providers should reuse the same host/ABI
infrastructure where possible.
The full C ABI/Worker resource protocol is a larger artifact (C structs, wire
buffers, ownership, process isolation). This ADR freezes the **in-process
trusted provider** slice first; a Worker/C ABI binding can implement the same
contract later. In-process providers are explicitly **not** a sandbox.
## Decision
### 1. Provider contract v1
```rust
pub trait ResourceProvider: Send {
fn name(&self) -> &str; // stable grant name
fn protocol_version(&self) -> u32 { 1 } // must be 1 in this slice
fn kinds(&self) -> &[&str]; // resource kinds it serves
fn open(&mut self, kind: &str, descriptor: &str, rights: &[String],
span: Option<Span>) -> Result<u64, RuntimeError>; // provider token
fn read(&mut self, token: u64, max_bytes: usize,
span: Option<Span>) -> Result<ResourceRead, RuntimeError>;
fn write(&mut self, token: u64, bytes: &[u8],
span: Option<Span>) -> Result<usize, RuntimeError>;
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError>;
}
```
* `ResourceRead { bytes, eof }` is the same host type used by built-in streams.
* The **host** still issues the opaque language `ResourceHandle`; the provider
token is private to the provider and never enters a `Value`, wire message or
trace.
* Bytes cross the provider boundary as copied slices for the duration of a
call; no provider may retain a `Memory` pointer across calls or after a
`grow`.
* `open` returns a provider token; `read/write/close` must reject unknown,
stale or closed tokens with a structured error. The host checks the language
handle before dispatching.
### 2. Registration
`StdHost::register_resource_provider(Box<dyn ResourceProvider>)`:
* rejects `protocol_version != 1`;
* rejects an empty `kinds()` list;
* rejects a provider name or kind that is empty, `stdio`, `file`, or already
registered (no silent shadowing);
* stores the provider under each declared kind and its stable name for grants.
Registration is a host/embedding API, not a language import. Importing a
`.mq.md` package never registers or authorizes a provider. Installation and
authorization remain separate (INV-06).
### 3. Authorization and modes
* `Capabilities` gains `resource_providers: Vec<String>` and
`allow_resource_provider(name)`; a matching name is required at
`resource.open` time.
* `project` only. `pure` and `strict` refuse provider kinds even when
registered; strict remains fail-closed (CM-A11). Existing stdio/file rules
are unchanged.
* A provider's own rights/descriptor validation is not a substitute for the
grant; denial is `permission.denied` with a diagnostic.
* In-process provider code shares the host address space and is **not**
sandboxed. Only explicitly trusted providers may be registered; the API
documentation and ADR state this.
### 4. Runtime surface
No new `Value`, `HostOp`, Markdown node or grammar. A language program still
writes:
```text
resource.open("<kind>", "<descriptor>", ["read"])
```
The existing `resource.read/write/close` and `Result` shapes are reused. The
provider name is used only for the grant check and trace metadata; the language
does not need to know it beyond the grant.
### 5. What remains for CM-A09
* C ABI / Worker resource transport (buffer handoff, ownership, crash isolation,
version negotiation through the existing plugin ABI).
* Provider discovery/installation from packages, signatures and platform
support matrices.
* Network/socket/device providers themselves.
* Strict-mode provider verification.
This ADR is a contract for trusted in-process providers, not a second plugin
ecosystem.
## Consequences
* A new endpoint can be implemented as a `ResourceProvider` and registered
without touching parser/runtime/`HostOp`; language code uses the same
`resource.*` calls.
* The provider token cannot be forged into a language handle; only the host
creates handles.
* In-process providers can crash/block the host; they are trusted. Worker C ABI
is still needed for untrusted extensions.
* `Capabilities` grows one grant list; CLI exposure is deferred because there
is no language-level provider installer yet.
## Verification
`docs/evidence/2026-11-20-provider-abi-cm-a09.md` records a mock read/write
provider registered without parser/runtime changes, grant/pure/strict denial,
protocol/kind conflict registration errors and the full workspace gate.
+2 -1
View File
@@ -29,7 +29,8 @@ implementation status lives in `docs/STATUS.md`; frozen behavior is in
| CM-A07 Compat-B print routing | [0030](../adr/0030-print-compat-b-routing.md) | Accepted |
| CM-A07 Compat-C remove legacy print formatting | [0031](../adr/0031-print-compat-c-remove-host-formatting.md) | Accepted |
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
| Table physical storage (T1/T2/T3), generic providers, strict file/network resources | — | Not decided; do not implement by implication |
| CM-A09 in-process ResourceProvider v1 | [0033](../adr/0033-in-process-resource-provider-v1.md) | Accepted (trusted in-process slice) |
| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication |
## What a future phase must not do
@@ -0,0 +1,90 @@
# RFC-CM CM-A09 evidence: in-process ResourceProvider v1
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a8e62a5` plus this step's changes.
Decision record: [ADR-0033](../adr/0033-in-process-resource-provider-v1.md)
Related: [resource streams](2026-11-20-resource-stream-p3.md), [STATUS §5](../STATUS.md).
Scope: trusted in-process provider contract only; C/Worker transport remains open.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
423 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Contract
`ResourceProvider` v1:
```text
name() -> stable grant name
protocol_version() -> 1
kinds() -> resource kinds
open(kind, descriptor, rights) -> provider token
read(token, max_bytes) -> {bytes, eof}
write(token, bytes) -> count
close(token)
```
The host still issues the language `ResourceHandle`; the provider token never
enters a `Value`, wire message or trace. Bytes are copied for the call duration;
providers must not retain Memory pointers.
## Registration and authorization
* `StdHost::register_resource_provider` rejects protocol != 1, empty kinds,
reserved `stdio`/`file` names/kinds and duplicate kind registration.
* `Capabilities.resource_providers` + `allow_resource_provider(name)` add the
grant; `resource.open` requires the provider name to be granted.
* Project mode only. Pure and strict refuse provider kinds; registration grants
nothing by itself.
* In-process providers are trusted and **not** sandboxed; the ADR states this
explicitly.
## Tests
`crates/marqdo-host/tests/resource_streams.rs` (12 total; 3 provider tests):
```text
registered_provider_streams_work_with_a_grant_in_project_mode
mock sensor: read ab/cdef with eof, write 1, close, second close resource.closed
provider_requires_a_name_grant_and_project_mode
no grant -> permission.denied
pure -> permission.denied
strict -> permission.denied
provider_registration_rejects_versions_reserved_names_and_conflicts
version 2, reserved stdio kind, duplicate kind -> resource.unsupported
```
No parser, `Value`, runtime dispatch or `HostOp` change was required; the mock
provider is registered through a host API and the language sees the existing
`resource.*` calls.
## Not done
* C ABI / Worker resource transport: buffer handoff, ownership, crash/timeout
isolation and version negotiation through the plugin ABI.
* Provider package installation, signatures, discovery and platform matrix.
* Network/socket/TCP/HTTP, clock, random, process, directory, device providers.
* Strict-mode provider verification.
* No external `marqdo-stdlib` repository publication.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-host --test resource_streams
cargo test --workspace --locked
```