feat(host): add in-process ResourceProvider v1 (CM-A09 slice)

- Add ResourceProvider v1 trait with name/protocol/kinds/open/read/write/close;
  host still issues the opaque language Resource handle, provider tokens stay
  private.
- Add StdHost::register_resource_provider with protocol/version, reserved-name
  and duplicate-kind checks; add Capabilities.allow_resource_provider grants.
- Provider kinds are project-mode only; pure/strict refuse them; in-process
  providers are trusted and not a sandbox (ADR-0033).
- Reuse resource.* and Result shapes; no parser/Value/runtime/HostOp changes.
- Add mock read/write provider and rejection tests, docs/spec/status/evidence.

Gate: fmt, clippy -D warnings, 423 tests passed.
This commit is contained in:
cflmy committed 2026-11-20 22:38:16 +08:00
1 parent a8e62a5ba9
commit 1be4662049
13 files changed
+640 -18

No files matched your search

+6
View File
@@ -125,6 +125,12 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
before module loading, a `std:marqdo-stdlib@<version>#<digest>` package
instance id, `project.stdlib` in Analysis v2, and test coverage for
`std:` resolution, tamper rejection and frozen-binary hash equality.
- CM-A09 in-process ResourceProvider v1 (ADR-0033): `ResourceProvider` trait,
`StdHost::register_resource_provider`, per-kind registration with protocol
version/conflict checks and `Capabilities.allow_resource_provider`; provider
streams reuse `resource.*` and are project-mode only. A mock read/write
provider test runs without parser/runtime/`HostOp` changes. C/Worker provider
transport remains open.
- CM-A08 table logical/stdlib slice (ADR-0032): `stdlib/data/table.mq.md`
adds select/filter/sort/append/set-cell/join/group as value-returning
`.mq.md` functions; `Value::Table` semantics and GFM AST laziness unchanged;
+4
View File
@@ -105,6 +105,10 @@ executes anything.
`.mq.md` print chain is the only execution path for `打印`/`print` on an official install (ADR-0031),
covering all 15 value types; `HostOp::Print` and the legacy formatter are
removed (ADR-0027/0028/0029/0030/0031).
* **In-process ResourceProvider v1 (RFC-CM CM-A09 first slice)** — trusted
providers register extra resource kinds, project-mode + name-grant gated,
using the existing `resource.*`/`Result` ABI; no parser/runtime/`HostOp`
change (ADR-0033).
* **Table stdlib algorithms (RFC-CM CM-A08)** — `stdlib/data/table.mq.md`
select/filter/sort/append/set/join/group over the logical `Table` value;
value-returning, input-immutable, with GFM AST laziness unchanged (ADR-0032).
+30
View File
@@ -323,6 +323,8 @@ pub struct Capabilities {
pub foreign: bool,
/// Allowed plugin names.
pub plugins: Vec<String>,
/// Allowed resource-provider names (ADR-0033).
pub resource_providers: Vec<String>,
}
impl Capabilities {
@@ -337,6 +339,7 @@ impl Capabilities {
process: false,
foreign: false,
plugins: Vec::new(),
resource_providers: Vec::new(),
}
}
@@ -371,6 +374,33 @@ impl Capabilities {
self
}
/// Grant a registered resource provider by its stable name (ADR-0033).
pub fn allow_resource_provider(mut self, name: impl Into<String>) -> Self {
self.resource_providers.push(name.into());
self
}
/// Check a resource-provider grant.
pub fn check_resource_provider(
&self,
name: &str,
span: Option<Span>,
) -> Result<(), RuntimeError> {
if self
.resource_providers
.iter()
.any(|allowed| allowed == name)
{
Ok(())
} else {
Err(denied(
span,
format!("resource provider `{name}` was not granted"),
format!("grant it with Capabilities::allow_resource_provider({name:?})"),
))
}
}
/// Whether a project-relative path matches a granted pattern.
///
/// Patterns and paths are normalised by stripping a leading `./` so that
+101 -6
View File
@@ -20,7 +20,7 @@ use crate::context::ExecutionContext;
use crate::fs::ProjectFs;
use crate::handles::{contains_plugin_handle, PluginHandleTable};
use crate::plugin_worker::WorkerPlugin;
use crate::resource::{ResourceRead, ResourceTable};
use crate::resource::{ResourceProvider, ResourceRead, ResourceTable, SharedResourceProvider};
use crate::trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState};
use marqdo_sandbox::{PlatformCapabilities, SandboxMode};
@@ -202,6 +202,8 @@ pub struct StdHost {
pub handles: PluginHandleTable,
/// Host-owned resource/stream table (ADR-0025).
resources: ResourceTable,
/// Registered in-process providers keyed by resource kind (ADR-0033).
resource_providers: BTreeMap<String, (String, SharedResourceProvider)>,
semantic_events: Vec<SemanticEvent>,
next_event_seq: u64,
trace_state: TraceState,
@@ -263,6 +265,7 @@ impl StdHost {
broker_audit: Arc::new(Mutex::new(Vec::new())),
handles: PluginHandleTable::default(),
resources: ResourceTable::new(),
resource_providers: BTreeMap::new(),
semantic_events: Vec::new(),
next_event_seq: 0,
trace_state: TraceState::default(),
@@ -565,6 +568,72 @@ impl StdHost {
Ok(Value::Unit)
}
/// Register an explicitly trusted in-process provider (ADR-0033).
///
/// In-process providers share the host address space and are not a
/// sandbox; only trusted adapters may be registered. Registration is not a
/// language operation and does not grant access by itself.
pub fn register_resource_provider(
&mut self,
provider: Box<dyn ResourceProvider>,
) -> Result<(), RuntimeError> {
if provider.protocol_version() != 1 {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"resource provider `{}` uses unsupported protocol version {}",
provider.name(),
provider.protocol_version()
),
));
}
let name = provider.name().to_string();
if name.is_empty() || name == "stdio" || name == "file" {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider name `{name}` is reserved or empty"),
));
}
let kinds = provider
.kinds()
.iter()
.map(|kind| kind.to_string())
.collect::<Vec<_>>();
if kinds.is_empty() {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider `{name}` declares no kinds"),
));
}
for kind in &kinds {
if kind.is_empty() || kind == "stdio" || kind == "file" {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider `{name}` declares reserved kind `{kind}`"),
));
}
if let Some((existing, _)) = self.resource_providers.get(kind) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"resource kind `{kind}` is already registered by provider `{existing}`"
),
));
}
}
let provider: SharedResourceProvider = std::sync::Arc::new(std::sync::Mutex::new(provider));
for kind in kinds {
self.resource_providers
.insert(kind, (name.clone(), std::sync::Arc::clone(&provider)));
}
Ok(())
}
fn record_resource_effect(
&mut self,
op: &str,
@@ -686,11 +755,37 @@ impl StdHost {
"project file streams support exactly [read] or [write] in this build",
))
}
other => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
format!("resource kind `{other}` is not implemented in this build"),
)),
other => {
if self.mode != SandboxMode::Project {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"resource providers are available only in project mode",
));
}
let Some((provider_name, provider)) = self.resource_providers.get(other).cloned()
else {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
format!("resource kind `{other}` is not registered"),
));
};
self.capabilities
.check_resource_provider(&provider_name, ctx.span)?;
let handle = self
.resources
.open_provider(provider, other, descriptor, rights, ctx.span)?;
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind={other} provider={provider_name}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
}
}
}
+1 -1
View File
@@ -22,5 +22,5 @@ pub use fs::ProjectFs;
pub use handles::{contains_plugin_handle, PluginHandleTable};
pub use host::{arity, expect_text, Host, HostOp, StdHost};
pub use marqdo_sandbox::{PlatformCapabilities, SandboxMode, StrictIsolation};
pub use resource::{ResourceRead, ResourceTable};
pub use resource::{ResourceProvider, ResourceRead, ResourceTable, SharedResourceProvider};
pub use trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState, TraceTruncation};
+106 -8
View File
@@ -6,12 +6,55 @@
use std::collections::BTreeMap;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError};
static NEXT_RUN_ID: AtomicU64 = AtomicU64::new(1);
/// A versioned trusted provider of additional resource kinds (ADR-0033).
///
/// In-process providers share the host address space and are **not** a
/// sandbox; only explicitly trusted code may be registered.
pub trait ResourceProvider: Send {
/// Stable grant name.
fn name(&self) -> &str;
/// Protocol version; must be 1 in this build.
fn protocol_version(&self) -> u32 {
1
}
/// Resource kinds served by this provider.
fn kinds(&self) -> &[&str];
/// Open one descriptor, returning a provider-private token.
fn open(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
span: Option<Span>,
) -> Result<u64, RuntimeError>;
/// Read up to `max_bytes` from a token.
fn read(
&mut self,
token: u64,
max_bytes: usize,
span: Option<Span>,
) -> Result<ResourceRead, RuntimeError>;
/// Write bytes to a token; return the accepted count.
fn write(
&mut self,
token: u64,
bytes: &[u8],
span: Option<Span>,
) -> Result<usize, RuntimeError>;
/// Close a token.
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError>;
}
/// Shared provider handle registered on the host.
pub type SharedResourceProvider = Arc<Mutex<Box<dyn ResourceProvider>>>;
/// Bytes returned by one stream read.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ResourceRead {
@@ -36,6 +79,12 @@ enum Stream {
bytes: Vec<u8>,
closed: bool,
},
Provider {
kind: String,
provider: SharedResourceProvider,
token: u64,
closed: bool,
},
}
struct Slot {
@@ -49,6 +98,7 @@ impl Slot {
Stream::Stdout { closed } => *closed,
Stream::File { closed, .. } => *closed,
Stream::FileWriter { closed, .. } => *closed,
Stream::Provider { closed, .. } => *closed,
}
}
}
@@ -105,6 +155,27 @@ impl ResourceTable {
})
}
/// Open a provider token and wrap it in a host-issued handle.
pub fn open_provider(
&mut self,
provider: SharedResourceProvider,
kind: &str,
descriptor: &str,
rights: &[String],
span: Option<Span>,
) -> Result<ResourceHandle, RuntimeError> {
let token = {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.open(kind, descriptor, rights, span)?
};
Ok(self.insert(Stream::Provider {
kind: kind.to_string(),
provider,
token,
closed: false,
}))
}
fn insert(&mut self, stream: Stream) -> ResourceHandle {
let id = self.next_resource;
self.next_resource += 1;
@@ -195,6 +266,12 @@ impl ResourceTable {
span,
"a file write stream does not support resource.read",
)),
Stream::Provider {
provider, token, ..
} => {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.read(*token, max_bytes, span)
}
}
}
@@ -249,6 +326,12 @@ impl ResourceTable {
buffer.extend_from_slice(bytes);
Ok(bytes.len())
}
Stream::Provider {
provider, token, ..
} => {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.write(*token, bytes, span)
}
}
}
@@ -270,6 +353,16 @@ impl ResourceTable {
Stream::Stdout { closed } => *closed = true,
Stream::File { closed, .. } => *closed = true,
Stream::FileWriter { closed, .. } => *closed = true,
Stream::Provider {
provider,
token,
closed,
..
} => {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.close(*token, span)?;
*closed = true;
}
}
Ok(())
}
@@ -301,19 +394,24 @@ impl ResourceTable {
}
/// Descriptor kind for diagnostics/trace, if the handle is valid.
pub fn kind(
&self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<&'static str, RuntimeError> {
pub fn kind(&self, handle: ResourceHandle, span: Option<Span>) -> Result<String, RuntimeError> {
let slot = self.slot(handle, span)?;
Ok(match slot.stream {
Stream::Stdout { .. } => "stdio",
Stream::File { .. } | Stream::FileWriter { .. } => "file",
Ok(match &slot.stream {
Stream::Stdout { .. } => "stdio".to_string(),
Stream::File { .. } | Stream::FileWriter { .. } => "file".to_string(),
Stream::Provider { kind, .. } => kind.clone(),
})
}
}
fn provider_lock_error(span: Option<Span>) -> RuntimeError {
RuntimeError::new(
ErrorCategory::HostError,
span,
"resource provider lock is poisoned",
)
}
fn unknown_handle(span: Option<Span>) -> RuntimeError {
RuntimeError::new(
ErrorCategory::ResourceFault,
@@ -215,3 +215,163 @@ fn traversal_and_unsupported_rights_are_refused() {
.unwrap_err();
assert_eq!(unknown_kind.category, ErrorCategory::ResourceUnsupported);
}
struct MockProvider {
data: Vec<u8>,
position: usize,
written: Vec<u8>,
version: u32,
kinds: Vec<&'static str>,
}
impl MockProvider {
fn new(data: &[u8], version: u32, kinds: Vec<&'static str>) -> Self {
Self {
data: data.to_vec(),
position: 0,
written: Vec::new(),
version,
kinds,
}
}
}
impl marqdo_host::ResourceProvider for MockProvider {
fn name(&self) -> &str {
"mock"
}
fn protocol_version(&self) -> u32 {
self.version
}
fn kinds(&self) -> &[&str] {
&self.kinds
}
fn open(
&mut self,
_kind: &str,
_descriptor: &str,
_rights: &[String],
_span: Option<marqdo_span::Span>,
) -> Result<u64, marqdo_value::RuntimeError> {
self.position = 0;
Ok(7)
}
fn read(
&mut self,
token: u64,
max_bytes: usize,
_span: Option<marqdo_span::Span>,
) -> Result<marqdo_host::ResourceRead, marqdo_value::RuntimeError> {
assert_eq!(token, 7);
let start = self.position;
let end = start.saturating_add(max_bytes).min(self.data.len());
self.position = end;
Ok(marqdo_host::ResourceRead {
bytes: self.data[start..end].to_vec(),
eof: end >= self.data.len(),
})
}
fn write(
&mut self,
token: u64,
bytes: &[u8],
_span: Option<marqdo_span::Span>,
) -> Result<usize, marqdo_value::RuntimeError> {
assert_eq!(token, 7);
self.written.extend_from_slice(bytes);
Ok(bytes.len())
}
fn close(
&mut self,
token: u64,
_span: Option<marqdo_span::Span>,
) -> Result<(), marqdo_value::RuntimeError> {
assert_eq!(token, 7);
Ok(())
}
}
#[test]
fn registered_provider_streams_work_with_a_grant_in_project_mode() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("mock"),
dir.path(),
);
host.register_resource_provider(Box::new(MockProvider::new(
b"abcdef",
1,
vec!["mock.sensor"],
)))
.unwrap();
let ctx = context();
let handle = handle(
host.resource_open("mock.sensor", "sensor-1", &["read".to_string()], &ctx)
.unwrap(),
);
let first = host.resource_read(handle, 2, &ctx).unwrap();
assert_eq!(first.bytes, b"ab");
assert!(!first.eof);
let rest = host.resource_read(handle, 16, &ctx).unwrap();
assert_eq!(rest.bytes, b"cdef");
assert!(rest.eof);
assert_eq!(host.resource_write(handle, b"x", &ctx).unwrap(), 1);
host.resource_close(handle, &ctx).unwrap();
assert_eq!(
host.resource_close(handle, &ctx).unwrap_err().code(),
"resource.closed"
);
}
#[test]
fn provider_requires_a_name_grant_and_project_mode() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
denied
.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
.unwrap();
let error = denied
.resource_open("mock.sensor", "s", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
let mut host = StdHost::new(
mode,
Capabilities::pure().allow_resource_provider("mock"),
dir.path(),
);
host.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
.unwrap();
let error = host
.resource_open("mock.sensor", "s", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
}
#[test]
fn provider_registration_rejects_versions_reserved_names_and_conflicts() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_resource_provider("mock"),
dir.path(),
);
let bad_version = host
.register_resource_provider(Box::new(MockProvider::new(b"x", 2, vec!["mock.sensor"])))
.unwrap_err();
assert_eq!(bad_version.category, ErrorCategory::ResourceUnsupported);
let reserved = host
.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["stdio"])))
.unwrap_err();
assert_eq!(reserved.category, ErrorCategory::ResourceUnsupported);
host.register_resource_provider(Box::new(MockProvider::new(b"x", 1, vec!["mock.sensor"])))
.unwrap();
let conflict = host
.register_resource_provider(Box::new(MockProvider::new(b"y", 1, vec!["mock.sensor"])))
.unwrap_err();
assert_eq!(conflict.category, ErrorCategory::ResourceUnsupported);
}
+8 -1
View File
@@ -135,6 +135,10 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
PluginHandle `plugin/export` are read-only observation members (host id
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
- In-process ResourceProvider v1 (CM-A09 first slice, ADR-0033): trusted
providers register additional resource kinds without parser/runtime/HostOp
changes; project-mode + name grant required; pure/strict refuse provider
kinds; C/Worker transport remains open.
- Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md`
provides select/filter/sort/append/set/join/group over the logical
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
@@ -191,7 +195,7 @@ behavior. This table records only what is in this tree now.
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: publication as a separate external stdlib repository. The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). C01–C06/C11/C12 each have `.mq.md` evidence, including JSON floats and a JSON formatter. |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Still missing: directories, network, generic provider registry, Table storage, strict file/network resources. |
| P4 Table logical/physical migration | **PARTIAL** | ADR-0032/CM-A08: `Value::Table` logical semantics unchanged (T0 `Vec<Vec<Value>>`); `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group as value-returning `.mq.md` functions with input immutability. No T1/T2/T3 physical storage, column store or Memory-backed table. |
| P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) |
| P5 third-party Provider / TCP / HTTP | **PARTIAL** | ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, network/socket/TCP/HTTP, clock/random/process/device providers, or provider package installation. |
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
The P1 memory domain is pure computation: it needs no grant and is available in
@@ -229,6 +233,9 @@ user-written allocator is explicitly **not** claimed.
- [`evidence/2026-11-20-table-cm-a08.md`](evidence/2026-11-20-table-cm-a08.md)
— ADR-0032 table logical value, stdlib select/filter/sort/append/set/join/
group, input immutability and T0 storage boundary.
- [`evidence/2026-11-20-provider-abi-cm-a09.md`](evidence/2026-11-20-provider-abi-cm-a09.md)
— ADR-0033 in-process ResourceProvider v1, grants/mode checks, mock provider
read/write and registration rejection cases.
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
— G0..G12 matrix and RC decision.
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
@@ -0,0 +1,123 @@
# ADR-0033: In-process ResourceProvider v1 (CM-A09 first slice)
Status: **Accepted (in-process slice only)**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A11, P3/P5), ADR-0025, ADR-0026, ADR-0031
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-provider-abi-cm-a09.md`
## Context
ADR-0025/0026 hard-code two resource kinds (`stdio/stdout`, project `file`) in
the host. CM-A09 requires a versioned, discoverable provider contract so a new
device/OS endpoint can be added **without** changing the expression parser,
`Value` model, runtime dispatch or `HostOp` enum. The RFC warns against a
second shadow plugin ecosystem: providers should reuse the same host/ABI
infrastructure where possible.
The full C ABI/Worker resource protocol is a larger artifact (C structs, wire
buffers, ownership, process isolation). This ADR freezes the **in-process
trusted provider** slice first; a Worker/C ABI binding can implement the same
contract later. In-process providers are explicitly **not** a sandbox.
## Decision
### 1. Provider contract v1
```rust
pub trait ResourceProvider: Send {
fn name(&self) -> &str; // stable grant name
fn protocol_version(&self) -> u32 { 1 } // must be 1 in this slice
fn kinds(&self) -> &[&str]; // resource kinds it serves
fn open(&mut self, kind: &str, descriptor: &str, rights: &[String],
span: Option<Span>) -> Result<u64, RuntimeError>; // provider token
fn read(&mut self, token: u64, max_bytes: usize,
span: Option<Span>) -> Result<ResourceRead, RuntimeError>;
fn write(&mut self, token: u64, bytes: &[u8],
span: Option<Span>) -> Result<usize, RuntimeError>;
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError>;
}
```
* `ResourceRead { bytes, eof }` is the same host type used by built-in streams.
* The **host** still issues the opaque language `ResourceHandle`; the provider
token is private to the provider and never enters a `Value`, wire message or
trace.
* Bytes cross the provider boundary as copied slices for the duration of a
call; no provider may retain a `Memory` pointer across calls or after a
`grow`.
* `open` returns a provider token; `read/write/close` must reject unknown,
stale or closed tokens with a structured error. The host checks the language
handle before dispatching.
### 2. Registration
`StdHost::register_resource_provider(Box<dyn ResourceProvider>)`:
* rejects `protocol_version != 1`;
* rejects an empty `kinds()` list;
* rejects a provider name or kind that is empty, `stdio`, `file`, or already
registered (no silent shadowing);
* stores the provider under each declared kind and its stable name for grants.
Registration is a host/embedding API, not a language import. Importing a
`.mq.md` package never registers or authorizes a provider. Installation and
authorization remain separate (INV-06).
### 3. Authorization and modes
* `Capabilities` gains `resource_providers: Vec<String>` and
`allow_resource_provider(name)`; a matching name is required at
`resource.open` time.
* `project` only. `pure` and `strict` refuse provider kinds even when
registered; strict remains fail-closed (CM-A11). Existing stdio/file rules
are unchanged.
* A provider's own rights/descriptor validation is not a substitute for the
grant; denial is `permission.denied` with a diagnostic.
* In-process provider code shares the host address space and is **not**
sandboxed. Only explicitly trusted providers may be registered; the API
documentation and ADR state this.
### 4. Runtime surface
No new `Value`, `HostOp`, Markdown node or grammar. A language program still
writes:
```text
resource.open("<kind>", "<descriptor>", ["read"])
```
The existing `resource.read/write/close` and `Result` shapes are reused. The
provider name is used only for the grant check and trace metadata; the language
does not need to know it beyond the grant.
### 5. What remains for CM-A09
* C ABI / Worker resource transport (buffer handoff, ownership, crash isolation,
version negotiation through the existing plugin ABI).
* Provider discovery/installation from packages, signatures and platform
support matrices.
* Network/socket/device providers themselves.
* Strict-mode provider verification.
This ADR is a contract for trusted in-process providers, not a second plugin
ecosystem.
## Consequences
* A new endpoint can be implemented as a `ResourceProvider` and registered
without touching parser/runtime/`HostOp`; language code uses the same
`resource.*` calls.
* The provider token cannot be forged into a language handle; only the host
creates handles.
* In-process providers can crash/block the host; they are trusted. Worker C ABI
is still needed for untrusted extensions.
* `Capabilities` grows one grant list; CLI exposure is deferred because there
is no language-level provider installer yet.
## Verification
`docs/evidence/2026-11-20-provider-abi-cm-a09.md` records a mock read/write
provider registered without parser/runtime changes, grant/pure/strict denial,
protocol/kind conflict registration errors and the full workspace gate.
+2 -1
View File
@@ -29,7 +29,8 @@ implementation status lives in `docs/STATUS.md`; frozen behavior is in
| CM-A07 Compat-B print routing | [0030](../adr/0030-print-compat-b-routing.md) | Accepted |
| CM-A07 Compat-C remove legacy print formatting | [0031](../adr/0031-print-compat-c-remove-host-formatting.md) | Accepted |
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
| Table physical storage (T1/T2/T3), generic providers, strict file/network resources | — | Not decided; do not implement by implication |
| CM-A09 in-process ResourceProvider v1 | [0033](../adr/0033-in-process-resource-provider-v1.md) | Accepted (trusted in-process slice) |
| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication |
## What a future phase must not do
@@ -0,0 +1,90 @@
# RFC-CM CM-A09 evidence: in-process ResourceProvider v1
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `a8e62a5` plus this step's changes.
Decision record: [ADR-0033](../adr/0033-in-process-resource-provider-v1.md)
Related: [resource streams](2026-11-20-resource-stream-p3.md), [STATUS §5](../STATUS.md).
Scope: trusted in-process provider contract only; C/Worker transport remains open.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
423 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Contract
`ResourceProvider` v1:
```text
name() -> stable grant name
protocol_version() -> 1
kinds() -> resource kinds
open(kind, descriptor, rights) -> provider token
read(token, max_bytes) -> {bytes, eof}
write(token, bytes) -> count
close(token)
```
The host still issues the language `ResourceHandle`; the provider token never
enters a `Value`, wire message or trace. Bytes are copied for the call duration;
providers must not retain Memory pointers.
## Registration and authorization
* `StdHost::register_resource_provider` rejects protocol != 1, empty kinds,
reserved `stdio`/`file` names/kinds and duplicate kind registration.
* `Capabilities.resource_providers` + `allow_resource_provider(name)` add the
grant; `resource.open` requires the provider name to be granted.
* Project mode only. Pure and strict refuse provider kinds; registration grants
nothing by itself.
* In-process providers are trusted and **not** sandboxed; the ADR states this
explicitly.
## Tests
`crates/marqdo-host/tests/resource_streams.rs` (12 total; 3 provider tests):
```text
registered_provider_streams_work_with_a_grant_in_project_mode
mock sensor: read ab/cdef with eof, write 1, close, second close resource.closed
provider_requires_a_name_grant_and_project_mode
no grant -> permission.denied
pure -> permission.denied
strict -> permission.denied
provider_registration_rejects_versions_reserved_names_and_conflicts
version 2, reserved stdio kind, duplicate kind -> resource.unsupported
```
No parser, `Value`, runtime dispatch or `HostOp` change was required; the mock
provider is registered through a host API and the language sees the existing
`resource.*` calls.
## Not done
* C ABI / Worker resource transport: buffer handoff, ownership, crash/timeout
isolation and version negotiation through the plugin ABI.
* Provider package installation, signatures, discovery and platform matrix.
* Network/socket/TCP/HTTP, clock, random, process, directory, device providers.
* Strict-mode provider verification.
* No external `marqdo-stdlib` repository publication.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-host --test resource_streams
cargo test --workspace --locked
```
+2 -1
View File
@@ -27,7 +27,7 @@ Normative implementation:
| Mode | Intended guarantee | Actual status in this build |
|---|---|---|
| `pure` | No external I/O; memory computation and budgets only. | **Enforced.** `Capabilities::pure()` = `console`, `math` only; every FS/foreign/plugin grant flag makes the CLI exit 2 with "pure mode prohibits external I/O grants". |
| `project` | Operator-approved project resources under `CapabilityHost` + bounded resources. | **Enforced for the implemented operations.** Defaults equal `pure`; `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin` add grants. `resource.open` supports stdout write, granted project-file read and granted buffered project-file write (ADR-0025/0026); network/process remain unsupported. Sandboxing is cooperative (in-process checks), not OS isolation. |
| `project` | Operator-approved project resources under `CapabilityHost` + bounded resources. | **Enforced for the implemented operations.** Defaults equal `pure`; `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin` add grants. `resource.open` supports stdout write, granted project-file read/write (ADR-0025/0026) and granted in-process provider kinds (ADR-0033); network/process remain unsupported. Sandboxing is cooperative (in-process checks), not OS isolation; in-process providers are trusted, not sandboxed. |
| `strict` | Untrusted documents under real OS isolation. | **Linux: enforced when the runtime probe passes (ADR-0021).** `resource.open` is **refused** in strict except for the existing `stdio/stdout` console channel (console grant); file/network/process resources remain refused until a provider passes the isolation lane (CM-A11, ADR-0030). `PlatformCapabilities::detect()` runs a real Bubblewrap smoke test; success reports `Available { mechanism }`, failure reports `Unavailable { reason }`. Strict L1 supports the interpreter, console and native Math only; `--allow-read/--allow-write/--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file` are rejected with exit 2. macOS/Windows and Linux hosts without the mechanisms remain `Unavailable`; there is **no fallback** to an unrestricted process. |
`marqdo inspect <file> --platform` prints the detection report, for example:
@@ -63,6 +63,7 @@ set for one run:
| `process: bool` | `false` | required together with `foreign` for `foreign.run`/`foreign.call` |
| `foreign: bool` | `false` | foreign kernels |
| `plugins: Vec<String>` | empty | `plugin.invoke` and `StdHost::load_plugin` |
| `resource_providers: Vec<String>` | empty | registered in-process `ResourceProvider` kinds; project mode only (ADR-0033) |
Requests versus grants:
+7
View File
@@ -171,6 +171,13 @@ by:
* fixture 55 additionally has a second-free detection path: freeing an
already-free block returns `{ok:false, code:"alloc.double_free"}`.
RFC-CM CM-A09 in-process provider v1 (ADR-0033) is pinned by the three
`registered_provider_streams_*` / `provider_*` tests in
`crates/marqdo-host/tests/resource_streams.rs`: a mock provider is registered
and read/written through `resource.*` with a project-mode name grant; missing
grant, pure/strict, bad protocol version, reserved kinds and duplicate kinds
are rejected.
RFC-CM CM-A08 table stdlib (ADR-0032) is pinned by
`crates/cli/tests/stdlib_table.rs`, which runs `stdlib/tests/table.mq.md` and
checks select/filter/sort/append/set/join/group results plus original-table