feat(host): add in-process ResourceProvider v1 (CM-A09 slice)
- Add ResourceProvider v1 trait with name/protocol/kinds/open/read/write/close; host still issues the opaque language Resource handle, provider tokens stay private. - Add StdHost::register_resource_provider with protocol/version, reserved-name and duplicate-kind checks; add Capabilities.allow_resource_provider grants. - Provider kinds are project-mode only; pure/strict refuse them; in-process providers are trusted and not a sandbox (ADR-0033). - Reuse resource.* and Result shapes; no parser/Value/runtime/HostOp changes. - Add mock read/write provider and rejection tests, docs/spec/status/evidence. Gate: fmt, clippy -D warnings, 423 tests passed.
This commit is contained in:
1 parent
a8e62a5ba9
commit
1be4662049
13 files changed
+640
-18
No files matched your search
+8
-1
@@ -135,6 +135,10 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
PluginHandle `plugin/export` are read-only observation members (host id
|
||||
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
|
||||
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
|
||||
- In-process ResourceProvider v1 (CM-A09 first slice, ADR-0033): trusted
|
||||
providers register additional resource kinds without parser/runtime/HostOp
|
||||
changes; project-mode + name grant required; pure/strict refuse provider
|
||||
kinds; C/Worker transport remains open.
|
||||
- Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md`
|
||||
provides select/filter/sort/append/set/join/group over the logical
|
||||
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
|
||||
@@ -191,7 +195,7 @@ behavior. This table records only what is in this tree now.
|
||||
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: publication as a separate external stdlib repository. The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). C01–C06/C11/C12 each have `.mq.md` evidence, including JSON floats and a JSON formatter. |
|
||||
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Still missing: directories, network, generic provider registry, Table storage, strict file/network resources. |
|
||||
| P4 Table logical/physical migration | **PARTIAL** | ADR-0032/CM-A08: `Value::Table` logical semantics unchanged (T0 `Vec<Vec<Value>>`); `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group as value-returning `.mq.md` functions with input immutability. No T1/T2/T3 physical storage, column store or Memory-backed table. |
|
||||
| P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) |
|
||||
| P5 third-party Provider / TCP / HTTP | **PARTIAL** | ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, network/socket/TCP/HTTP, clock/random/process/device providers, or provider package installation. |
|
||||
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
|
||||
|
||||
The P1 memory domain is pure computation: it needs no grant and is available in
|
||||
@@ -229,6 +233,9 @@ user-written allocator is explicitly **not** claimed.
|
||||
- [`evidence/2026-11-20-table-cm-a08.md`](evidence/2026-11-20-table-cm-a08.md)
|
||||
— ADR-0032 table logical value, stdlib select/filter/sort/append/set/join/
|
||||
group, input immutability and T0 storage boundary.
|
||||
- [`evidence/2026-11-20-provider-abi-cm-a09.md`](evidence/2026-11-20-provider-abi-cm-a09.md)
|
||||
— ADR-0033 in-process ResourceProvider v1, grants/mode checks, mock provider
|
||||
read/write and registration rejection cases.
|
||||
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
|
||||
— G0..G12 matrix and RC decision.
|
||||
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
# ADR-0033: In-process ResourceProvider v1 (CM-A09 first slice)
|
||||
|
||||
Status: **Accepted (in-process slice only)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A11, P3/P5), ADR-0025, ADR-0026, ADR-0031
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-provider-abi-cm-a09.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0025/0026 hard-code two resource kinds (`stdio/stdout`, project `file`) in
|
||||
the host. CM-A09 requires a versioned, discoverable provider contract so a new
|
||||
device/OS endpoint can be added **without** changing the expression parser,
|
||||
`Value` model, runtime dispatch or `HostOp` enum. The RFC warns against a
|
||||
second shadow plugin ecosystem: providers should reuse the same host/ABI
|
||||
infrastructure where possible.
|
||||
|
||||
The full C ABI/Worker resource protocol is a larger artifact (C structs, wire
|
||||
buffers, ownership, process isolation). This ADR freezes the **in-process
|
||||
trusted provider** slice first; a Worker/C ABI binding can implement the same
|
||||
contract later. In-process providers are explicitly **not** a sandbox.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Provider contract v1
|
||||
|
||||
```rust
|
||||
pub trait ResourceProvider: Send {
|
||||
fn name(&self) -> &str; // stable grant name
|
||||
fn protocol_version(&self) -> u32 { 1 } // must be 1 in this slice
|
||||
fn kinds(&self) -> &[&str]; // resource kinds it serves
|
||||
fn open(&mut self, kind: &str, descriptor: &str, rights: &[String],
|
||||
span: Option<Span>) -> Result<u64, RuntimeError>; // provider token
|
||||
fn read(&mut self, token: u64, max_bytes: usize,
|
||||
span: Option<Span>) -> Result<ResourceRead, RuntimeError>;
|
||||
fn write(&mut self, token: u64, bytes: &[u8],
|
||||
span: Option<Span>) -> Result<usize, RuntimeError>;
|
||||
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError>;
|
||||
}
|
||||
```
|
||||
|
||||
* `ResourceRead { bytes, eof }` is the same host type used by built-in streams.
|
||||
* The **host** still issues the opaque language `ResourceHandle`; the provider
|
||||
token is private to the provider and never enters a `Value`, wire message or
|
||||
trace.
|
||||
* Bytes cross the provider boundary as copied slices for the duration of a
|
||||
call; no provider may retain a `Memory` pointer across calls or after a
|
||||
`grow`.
|
||||
* `open` returns a provider token; `read/write/close` must reject unknown,
|
||||
stale or closed tokens with a structured error. The host checks the language
|
||||
handle before dispatching.
|
||||
|
||||
### 2. Registration
|
||||
|
||||
`StdHost::register_resource_provider(Box<dyn ResourceProvider>)`:
|
||||
|
||||
* rejects `protocol_version != 1`;
|
||||
* rejects an empty `kinds()` list;
|
||||
* rejects a provider name or kind that is empty, `stdio`, `file`, or already
|
||||
registered (no silent shadowing);
|
||||
* stores the provider under each declared kind and its stable name for grants.
|
||||
|
||||
Registration is a host/embedding API, not a language import. Importing a
|
||||
`.mq.md` package never registers or authorizes a provider. Installation and
|
||||
authorization remain separate (INV-06).
|
||||
|
||||
### 3. Authorization and modes
|
||||
|
||||
* `Capabilities` gains `resource_providers: Vec<String>` and
|
||||
`allow_resource_provider(name)`; a matching name is required at
|
||||
`resource.open` time.
|
||||
* `project` only. `pure` and `strict` refuse provider kinds even when
|
||||
registered; strict remains fail-closed (CM-A11). Existing stdio/file rules
|
||||
are unchanged.
|
||||
* A provider's own rights/descriptor validation is not a substitute for the
|
||||
grant; denial is `permission.denied` with a diagnostic.
|
||||
* In-process provider code shares the host address space and is **not**
|
||||
sandboxed. Only explicitly trusted providers may be registered; the API
|
||||
documentation and ADR state this.
|
||||
|
||||
### 4. Runtime surface
|
||||
|
||||
No new `Value`, `HostOp`, Markdown node or grammar. A language program still
|
||||
writes:
|
||||
|
||||
```text
|
||||
resource.open("<kind>", "<descriptor>", ["read"])
|
||||
```
|
||||
|
||||
The existing `resource.read/write/close` and `Result` shapes are reused. The
|
||||
provider name is used only for the grant check and trace metadata; the language
|
||||
does not need to know it beyond the grant.
|
||||
|
||||
### 5. What remains for CM-A09
|
||||
|
||||
* C ABI / Worker resource transport (buffer handoff, ownership, crash isolation,
|
||||
version negotiation through the existing plugin ABI).
|
||||
* Provider discovery/installation from packages, signatures and platform
|
||||
support matrices.
|
||||
* Network/socket/device providers themselves.
|
||||
* Strict-mode provider verification.
|
||||
|
||||
This ADR is a contract for trusted in-process providers, not a second plugin
|
||||
ecosystem.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A new endpoint can be implemented as a `ResourceProvider` and registered
|
||||
without touching parser/runtime/`HostOp`; language code uses the same
|
||||
`resource.*` calls.
|
||||
* The provider token cannot be forged into a language handle; only the host
|
||||
creates handles.
|
||||
* In-process providers can crash/block the host; they are trusted. Worker C ABI
|
||||
is still needed for untrusted extensions.
|
||||
* `Capabilities` grows one grant list; CLI exposure is deferred because there
|
||||
is no language-level provider installer yet.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-provider-abi-cm-a09.md` records a mock read/write
|
||||
provider registered without parser/runtime changes, grant/pure/strict denial,
|
||||
protocol/kind conflict registration errors and the full workspace gate.
|
||||
@@ -29,7 +29,8 @@ implementation status lives in `docs/STATUS.md`; frozen behavior is in
|
||||
| CM-A07 Compat-B print routing | [0030](../adr/0030-print-compat-b-routing.md) | Accepted |
|
||||
| CM-A07 Compat-C remove legacy print formatting | [0031](../adr/0031-print-compat-c-remove-host-formatting.md) | Accepted |
|
||||
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
|
||||
| Table physical storage (T1/T2/T3), generic providers, strict file/network resources | — | Not decided; do not implement by implication |
|
||||
| CM-A09 in-process ResourceProvider v1 | [0033](../adr/0033-in-process-resource-provider-v1.md) | Accepted (trusted in-process slice) |
|
||||
| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication |
|
||||
|
||||
## What a future phase must not do
|
||||
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
# RFC-CM CM-A09 evidence: in-process ResourceProvider v1
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a8e62a5` plus this step's changes.
|
||||
Decision record: [ADR-0033](../adr/0033-in-process-resource-provider-v1.md)
|
||||
Related: [resource streams](2026-11-20-resource-stream-p3.md), [STATUS §5](../STATUS.md).
|
||||
Scope: trusted in-process provider contract only; C/Worker transport remains open.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
423 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Contract
|
||||
|
||||
`ResourceProvider` v1:
|
||||
|
||||
```text
|
||||
name() -> stable grant name
|
||||
protocol_version() -> 1
|
||||
kinds() -> resource kinds
|
||||
open(kind, descriptor, rights) -> provider token
|
||||
read(token, max_bytes) -> {bytes, eof}
|
||||
write(token, bytes) -> count
|
||||
close(token)
|
||||
```
|
||||
|
||||
The host still issues the language `ResourceHandle`; the provider token never
|
||||
enters a `Value`, wire message or trace. Bytes are copied for the call duration;
|
||||
providers must not retain Memory pointers.
|
||||
|
||||
## Registration and authorization
|
||||
|
||||
* `StdHost::register_resource_provider` rejects protocol != 1, empty kinds,
|
||||
reserved `stdio`/`file` names/kinds and duplicate kind registration.
|
||||
* `Capabilities.resource_providers` + `allow_resource_provider(name)` add the
|
||||
grant; `resource.open` requires the provider name to be granted.
|
||||
* Project mode only. Pure and strict refuse provider kinds; registration grants
|
||||
nothing by itself.
|
||||
* In-process providers are trusted and **not** sandboxed; the ADR states this
|
||||
explicitly.
|
||||
|
||||
## Tests
|
||||
|
||||
`crates/marqdo-host/tests/resource_streams.rs` (12 total; 3 provider tests):
|
||||
|
||||
```text
|
||||
registered_provider_streams_work_with_a_grant_in_project_mode
|
||||
mock sensor: read ab/cdef with eof, write 1, close, second close resource.closed
|
||||
|
||||
provider_requires_a_name_grant_and_project_mode
|
||||
no grant -> permission.denied
|
||||
pure -> permission.denied
|
||||
strict -> permission.denied
|
||||
|
||||
provider_registration_rejects_versions_reserved_names_and_conflicts
|
||||
version 2, reserved stdio kind, duplicate kind -> resource.unsupported
|
||||
```
|
||||
|
||||
No parser, `Value`, runtime dispatch or `HostOp` change was required; the mock
|
||||
provider is registered through a host API and the language sees the existing
|
||||
`resource.*` calls.
|
||||
|
||||
## Not done
|
||||
|
||||
* C ABI / Worker resource transport: buffer handoff, ownership, crash/timeout
|
||||
isolation and version negotiation through the plugin ABI.
|
||||
* Provider package installation, signatures, discovery and platform matrix.
|
||||
* Network/socket/TCP/HTTP, clock, random, process, directory, device providers.
|
||||
* Strict-mode provider verification.
|
||||
* No external `marqdo-stdlib` repository publication.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-host --test resource_streams
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
Reference in new issue
Block a user