feat(host): add trusted C ABI ResourceProvider v1
- Add the frozen marqdo_resource_provider.h ABI (version + open/read/write/ close/last_error) and a libloading-based host adapter with version/symbol checks and callsite-attached host errors (ADR-0036). - Add StdHost::register_c_resource_provider using the same kind/name validation and project-mode name grant as the in-process/worker providers. - Compile a C shared-library test provider; cover round-trip and fail-closed missing-library behavior. Trusted native code, not a sandbox; plugin-Worker binding remains open. - Update ADRs/spec/status/README/changelog and evidence. Gate: fmt, clippy -D warnings, 435 tests passed.
This commit is contained in:
1 parent
01f6fe46a7
commit
b22a89e7b4
15 files changed
+577
-2
No files matched your search
@@ -136,6 +136,10 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
|
||||
JSON report; no bytecode/JIT/native backend is introduced.
|
||||
- P4 cross-storage table equivalence: fixture 72 compares row-store Table
|
||||
sort/filter output with a `.mq.md` columnar store cell-by-cell.
|
||||
- CM-A09 C ABI provider v1 (ADR-0036): trusted shared libraries implement
|
||||
open/read/write/close/version/last_error; host loads them with version and
|
||||
symbol checks, project-mode + name grant only. Plugin-Worker binding remains
|
||||
open.
|
||||
- CM-A09 worker provider transport (ADR-0035): host-managed helper process
|
||||
speaking framed JSON for resource open/read/write/close; trusted-only,
|
||||
project-mode + name grant; C ABI binding still open.
|
||||
|
||||
Generated
+1
@@ -394,6 +394,7 @@ name = "marqdo-host"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"glob",
|
||||
"libloading",
|
||||
"marqdo-foreign",
|
||||
"marqdo-math",
|
||||
"marqdo-plugin-abi",
|
||||
|
||||
@@ -111,6 +111,9 @@ executes anything.
|
||||
* **Blocking TCP endpoint (RFC-CM P5 first slice)** — `resource.open("tcp",
|
||||
"<ip>:<port>", ["read","write"])` with exact IP grants, project mode and
|
||||
5-second timeouts; DNS/TLS/UDP/server sockets excluded (ADR-0034).
|
||||
* **C ABI ResourceProvider v1 (RFC-CM CM-A09)** — trusted native shared
|
||||
libraries expose resource open/read/write/close through a frozen header;
|
||||
project + name grant; plugin-Worker binding remains open (ADR-0036).
|
||||
* **Worker ResourceProvider transport v1 (RFC-CM CM-A09)** — trusted helper
|
||||
process with framed JSON resource operations; project + name grant; C ABI
|
||||
binding remains open (ADR-0035).
|
||||
|
||||
@@ -9,6 +9,7 @@ repository.workspace = true
|
||||
authors.workspace = true
|
||||
|
||||
[dependencies]
|
||||
libloading = { workspace = true }
|
||||
marqdo-plugin-abi = { workspace = true }
|
||||
glob = { workspace = true }
|
||||
marqdo-span = { workspace = true }
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
/* Marqdo ResourceProvider C ABI v1 (ADR-0036).
|
||||
*
|
||||
* This ABI is for trusted native adapters, not a sandbox. The host loads the
|
||||
* shared library, checks marqdo_resource_provider_abi_version() == 1 and calls
|
||||
* these symbols for language-level resource.* operations.
|
||||
*
|
||||
* Conventions:
|
||||
* - return 0 on success, nonzero on failure;
|
||||
* - failure text is available from marqdo_resource_last_error();
|
||||
* - rights is a comma-separated list (e.g. "read,write");
|
||||
* - provider tokens are private to the adapter and must not be exposed to
|
||||
* language Values, wire frames or trace output.
|
||||
*/
|
||||
#ifndef MARQDO_RESOURCE_PROVIDER_H
|
||||
#define MARQDO_RESOURCE_PROVIDER_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
#define MARQDO_RESOURCE_PROVIDER_ABI_VERSION 1u
|
||||
|
||||
uint32_t marqdo_resource_provider_abi_version(void);
|
||||
|
||||
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
|
||||
const char *rights_csv, uint64_t *out_token);
|
||||
|
||||
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
|
||||
uint32_t capacity, uint32_t *out_read,
|
||||
int32_t *out_eof);
|
||||
|
||||
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
|
||||
uint32_t length, uint32_t *out_written);
|
||||
|
||||
int32_t marqdo_resource_close(uint64_t token);
|
||||
|
||||
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity);
|
||||
|
||||
#endif /* MARQDO_RESOURCE_PROVIDER_H */
|
||||
@@ -21,6 +21,7 @@ use crate::context::ExecutionContext;
|
||||
use crate::fs::ProjectFs;
|
||||
use crate::handles::{contains_plugin_handle, PluginHandleTable};
|
||||
use crate::plugin_worker::WorkerPlugin;
|
||||
use crate::provider_c::CResourceProvider;
|
||||
use crate::provider_worker::WorkerResourceProvider;
|
||||
use crate::resource::{ResourceProvider, ResourceRead, ResourceTable, SharedResourceProvider};
|
||||
use crate::trace::{EffectTrace, SemanticEvent, TraceBudget, TraceEntry, TraceState};
|
||||
@@ -582,6 +583,25 @@ impl StdHost {
|
||||
self.register_provider_checked(provider)
|
||||
}
|
||||
|
||||
/// Register a trusted C ABI resource provider (ADR-0036).
|
||||
///
|
||||
/// The shared library runs in the host address space and is **trusted**, not
|
||||
/// sandboxed. Registration still requires the provider-name grant at
|
||||
/// `resource.open` and is project-mode only.
|
||||
pub fn register_c_resource_provider(
|
||||
&mut self,
|
||||
name: &str,
|
||||
kinds: &[&str],
|
||||
library: &std::path::Path,
|
||||
) -> Result<(), RuntimeError> {
|
||||
let kinds = kinds
|
||||
.iter()
|
||||
.map(|kind| (*kind).to_string())
|
||||
.collect::<Vec<_>>();
|
||||
let provider = CResourceProvider::open(name, &kinds, library)?;
|
||||
self.register_provider_checked(Box::new(provider))
|
||||
}
|
||||
|
||||
/// Register a trusted worker-process resource provider (ADR-0035).
|
||||
///
|
||||
/// The helper speaks framed JSON over stdin/stdout and is **not** an OS
|
||||
|
||||
@@ -12,6 +12,7 @@ pub mod handles;
|
||||
pub mod host;
|
||||
pub mod kernels;
|
||||
mod plugin_worker;
|
||||
mod provider_c;
|
||||
mod provider_worker;
|
||||
pub mod resource;
|
||||
pub mod trace;
|
||||
|
||||
@@ -0,0 +1,214 @@
|
||||
//! C ABI resource provider v1 (ADR-0036).
|
||||
//!
|
||||
//! The library is trusted native code. Loading it shares the host address space
|
||||
//! and is explicitly not a sandbox; only explicitly trusted adapters may be
|
||||
//! registered.
|
||||
|
||||
use std::ffi::{c_char, CString};
|
||||
use std::path::Path;
|
||||
|
||||
use libloading::Library;
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{ErrorCategory, RuntimeError};
|
||||
|
||||
use crate::resource::{ResourceProvider, ResourceRead};
|
||||
|
||||
const ABI_VERSION: u32 = 1;
|
||||
const ERROR_MAX: usize = 1024;
|
||||
|
||||
type AbiVersionFn = unsafe extern "C" fn() -> u32;
|
||||
type OpenFn = unsafe extern "C" fn(*const c_char, *const c_char, *const c_char, *mut u64) -> i32;
|
||||
type ReadFn = unsafe extern "C" fn(u64, *mut u8, u32, *mut u32, *mut i32) -> i32;
|
||||
type WriteFn = unsafe extern "C" fn(u64, *const u8, u32, *mut u32) -> i32;
|
||||
type CloseFn = unsafe extern "C" fn(u64) -> i32;
|
||||
type LastErrorFn = unsafe extern "C" fn(*mut c_char, u32) -> i32;
|
||||
|
||||
pub(crate) struct CResourceProvider {
|
||||
name: String,
|
||||
kinds: Vec<String>,
|
||||
library: Library,
|
||||
}
|
||||
|
||||
impl CResourceProvider {
|
||||
pub(crate) fn open(name: &str, kinds: &[String], path: &Path) -> Result<Self, RuntimeError> {
|
||||
let library = unsafe { Library::new(path) }.map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::KernelUnavailable,
|
||||
None,
|
||||
format!(
|
||||
"cannot load C resource provider `{name}` ({}): {error}",
|
||||
path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let version = {
|
||||
let symbol: libloading::Symbol<AbiVersionFn> = unsafe {
|
||||
library
|
||||
.get(b"marqdo_resource_provider_abi_version\0")
|
||||
.map_err(|error| {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!("C resource provider `{name}` is missing abi_version: {error}"),
|
||||
)
|
||||
})?
|
||||
};
|
||||
unsafe { symbol() }
|
||||
};
|
||||
if version != ABI_VERSION {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::ResourceUnsupported,
|
||||
None,
|
||||
format!(
|
||||
"C resource provider `{name}` uses ABI version {version}; host requires {ABI_VERSION}"
|
||||
),
|
||||
));
|
||||
}
|
||||
Ok(Self {
|
||||
name: name.to_string(),
|
||||
kinds: kinds.to_vec(),
|
||||
library,
|
||||
})
|
||||
}
|
||||
|
||||
fn open_fn(&self) -> Result<libloading::Symbol<'_, OpenFn>, RuntimeError> {
|
||||
unsafe { self.library.get(b"marqdo_resource_open\0") }
|
||||
.map_err(|error| provider_error(&self.name, &format!("missing open: {error}")))
|
||||
}
|
||||
fn read_fn(&self) -> Result<libloading::Symbol<'_, ReadFn>, RuntimeError> {
|
||||
unsafe { self.library.get(b"marqdo_resource_read\0") }
|
||||
.map_err(|error| provider_error(&self.name, &format!("missing read: {error}")))
|
||||
}
|
||||
fn write_fn(&self) -> Result<libloading::Symbol<'_, WriteFn>, RuntimeError> {
|
||||
unsafe { self.library.get(b"marqdo_resource_write\0") }
|
||||
.map_err(|error| provider_error(&self.name, &format!("missing write: {error}")))
|
||||
}
|
||||
fn close_fn(&self) -> Result<libloading::Symbol<'_, CloseFn>, RuntimeError> {
|
||||
unsafe { self.library.get(b"marqdo_resource_close\0") }
|
||||
.map_err(|error| provider_error(&self.name, &format!("missing close: {error}")))
|
||||
}
|
||||
|
||||
fn last_error(&self) -> String {
|
||||
let Ok(symbol) = (unsafe {
|
||||
self.library
|
||||
.get::<LastErrorFn>(b"marqdo_resource_last_error\0")
|
||||
}) else {
|
||||
return format!("C resource provider `{}` failed", self.name);
|
||||
};
|
||||
let mut buffer = vec![0i8; ERROR_MAX];
|
||||
let code = unsafe { symbol(buffer.as_mut_ptr(), ERROR_MAX as u32) };
|
||||
if code != 0 {
|
||||
return format!("C resource provider `{}` failed", self.name);
|
||||
}
|
||||
let bytes = buffer
|
||||
.iter()
|
||||
.map(|value| *value as u8)
|
||||
.take_while(|value| *value != 0)
|
||||
.collect::<Vec<_>>();
|
||||
String::from_utf8_lossy(&bytes).into_owned()
|
||||
}
|
||||
|
||||
fn failure(&self, span: Option<Span>) -> RuntimeError {
|
||||
let message = self.last_error();
|
||||
RuntimeError::new(ErrorCategory::HostError, span, message)
|
||||
}
|
||||
}
|
||||
|
||||
impl ResourceProvider for CResourceProvider {
|
||||
fn name(&self) -> &str {
|
||||
&self.name
|
||||
}
|
||||
fn kinds(&self) -> &[&str] {
|
||||
&[]
|
||||
}
|
||||
fn kind_strings(&self) -> Vec<String> {
|
||||
self.kinds.clone()
|
||||
}
|
||||
fn open(
|
||||
&mut self,
|
||||
kind: &str,
|
||||
descriptor: &str,
|
||||
rights: &[String],
|
||||
span: Option<Span>,
|
||||
) -> Result<u64, RuntimeError> {
|
||||
let kind = CString::new(kind).map_err(|_| provider_error(&self.name, "kind has NUL"))?;
|
||||
let descriptor = CString::new(descriptor)
|
||||
.map_err(|_| provider_error(&self.name, "descriptor has NUL"))?;
|
||||
let rights = CString::new(rights.join(","))
|
||||
.map_err(|_| provider_error(&self.name, "rights have NUL"))?;
|
||||
let mut token = 0u64;
|
||||
let code = unsafe {
|
||||
(self.open_fn()?)(
|
||||
kind.as_ptr(),
|
||||
descriptor.as_ptr(),
|
||||
rights.as_ptr(),
|
||||
&mut token,
|
||||
)
|
||||
};
|
||||
if code != 0 {
|
||||
return Err(self.failure(span));
|
||||
}
|
||||
Ok(token)
|
||||
}
|
||||
fn read(
|
||||
&mut self,
|
||||
token: u64,
|
||||
max_bytes: usize,
|
||||
span: Option<Span>,
|
||||
) -> Result<ResourceRead, RuntimeError> {
|
||||
let capacity = u32::try_from(max_bytes).unwrap_or(u32::MAX);
|
||||
let mut buffer = vec![0u8; capacity as usize];
|
||||
let mut read = 0u32;
|
||||
let mut eof = 0i32;
|
||||
let code =
|
||||
unsafe { (self.read_fn()?)(token, buffer.as_mut_ptr(), capacity, &mut read, &mut eof) };
|
||||
if code != 0 {
|
||||
return Err(self.failure(span));
|
||||
}
|
||||
buffer.truncate(read as usize);
|
||||
Ok(ResourceRead {
|
||||
bytes: buffer,
|
||||
eof: eof != 0,
|
||||
})
|
||||
}
|
||||
fn write(
|
||||
&mut self,
|
||||
token: u64,
|
||||
bytes: &[u8],
|
||||
span: Option<Span>,
|
||||
) -> Result<usize, RuntimeError> {
|
||||
let length = u32::try_from(bytes.len())
|
||||
.map_err(|_| provider_error(&self.name, "write size exceeds the C ABI u32 limit"))?;
|
||||
let mut written = 0u32;
|
||||
let code = unsafe { (self.write_fn()?)(token, bytes.as_ptr(), length, &mut written) };
|
||||
if code != 0 {
|
||||
return Err(self.failure(span));
|
||||
}
|
||||
Ok(written as usize)
|
||||
}
|
||||
fn close(&mut self, token: u64, span: Option<Span>) -> Result<(), RuntimeError> {
|
||||
let code = unsafe { (self.close_fn()?)(token) };
|
||||
if code != 0 {
|
||||
return Err(self.failure(span));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for CResourceProvider {
|
||||
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
formatter
|
||||
.debug_struct("CResourceProvider")
|
||||
.field("name", &self.name)
|
||||
.field("kinds", &self.kinds)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
fn provider_error(name: &str, message: &str) -> RuntimeError {
|
||||
RuntimeError::new(
|
||||
ErrorCategory::HostError,
|
||||
None,
|
||||
format!("C resource provider `{name}`: {message}"),
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
#![cfg(unix)]
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
use marqdo_host::{Capabilities, Host, StdHost};
|
||||
use marqdo_sandbox::SandboxMode;
|
||||
use marqdo_value::ErrorCategory;
|
||||
|
||||
fn context() -> marqdo_host::ExecutionContext {
|
||||
marqdo_host::ExecutionContext::new("test".to_string(), None)
|
||||
}
|
||||
|
||||
fn handle(value: marqdo_value::Value) -> marqdo_value::ResourceHandle {
|
||||
match value {
|
||||
marqdo_value::Value::Resource(handle) => handle,
|
||||
other => panic!("expected resource, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn cc_available() -> bool {
|
||||
Command::new("cc")
|
||||
.arg("--version")
|
||||
.output()
|
||||
.map(|output| output.status.success())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
fn build_provider(dir: &Path) -> PathBuf {
|
||||
let source = dir.join("provider.c");
|
||||
std::fs::write(
|
||||
&source,
|
||||
r#"
|
||||
#include <stdint.h>
|
||||
#include <stddef.h>
|
||||
#include <string.h>
|
||||
|
||||
static const char DATA[] = "c-abi";
|
||||
static size_t position = 0;
|
||||
static uint64_t next_token = 1;
|
||||
|
||||
uint32_t marqdo_resource_provider_abi_version(void) { return 1u; }
|
||||
|
||||
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
|
||||
const char *rights_csv, uint64_t *out_token) {
|
||||
(void)kind; (void)descriptor; (void)rights_csv;
|
||||
position = 0;
|
||||
*out_token = next_token++;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
|
||||
uint32_t capacity, uint32_t *out_read,
|
||||
int32_t *out_eof) {
|
||||
(void)token;
|
||||
size_t remaining = sizeof(DATA) - 1 - position;
|
||||
size_t count = remaining < capacity ? remaining : capacity;
|
||||
memcpy(buffer, DATA + position, count);
|
||||
position += count;
|
||||
*out_read = (uint32_t)count;
|
||||
*out_eof = position >= sizeof(DATA) - 1 ? 1 : 0;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
|
||||
uint32_t length, uint32_t *out_written) {
|
||||
(void)token; (void)bytes;
|
||||
*out_written = length;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t marqdo_resource_close(uint64_t token) { (void)token; return 0; }
|
||||
|
||||
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity) {
|
||||
if (capacity > 0) buffer[0] = 0;
|
||||
return 0;
|
||||
}
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
let extension = if cfg!(target_os = "macos") {
|
||||
"dylib"
|
||||
} else {
|
||||
"so"
|
||||
};
|
||||
let library = dir.join(format!("libecho_provider.{extension}"));
|
||||
let status = Command::new("cc")
|
||||
.args(["-shared", "-fPIC", "-O2", "-o"])
|
||||
.arg(&library)
|
||||
.arg(&source)
|
||||
.status()
|
||||
.expect("run cc");
|
||||
assert!(status.success(), "C provider build failed");
|
||||
library
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn c_abi_provider_round_trips() {
|
||||
if !cc_available() {
|
||||
eprintln!("skipping C provider test: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let library = build_provider(dir.path());
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("cabi"),
|
||||
project.path(),
|
||||
);
|
||||
host.register_c_resource_provider("cabi", &["cabi.sensor"], &library)
|
||||
.unwrap();
|
||||
let ctx = context();
|
||||
let handle = handle(
|
||||
host.resource_open("cabi.sensor", "sensor-1", &["read".to_string()], &ctx)
|
||||
.unwrap(),
|
||||
);
|
||||
let first = host.resource_read(handle, 2, &ctx).unwrap();
|
||||
assert_eq!(first.bytes, b"c-");
|
||||
assert!(!first.eof);
|
||||
let rest = host.resource_read(handle, 16, &ctx).unwrap();
|
||||
assert_eq!(rest.bytes, b"abi");
|
||||
assert!(rest.eof);
|
||||
assert_eq!(host.resource_write(handle, b"xyz", &ctx).unwrap(), 3);
|
||||
host.resource_close(handle, &ctx).unwrap();
|
||||
assert_eq!(
|
||||
host.resource_close(handle, &ctx).unwrap_err().code(),
|
||||
"resource.closed"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn c_abi_provider_loading_fails_closed() {
|
||||
let project = tempfile::tempdir().unwrap();
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::project().allow_resource_provider("cabi"),
|
||||
project.path(),
|
||||
);
|
||||
let error = host
|
||||
.register_c_resource_provider(
|
||||
"cabi",
|
||||
&["cabi.sensor"],
|
||||
Path::new("/definitely/missing/marqdo-provider.so"),
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.category, ErrorCategory::KernelUnavailable);
|
||||
}
|
||||
+7
-1
@@ -136,6 +136,9 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
PluginHandle `plugin/export` are read-only observation members (host id
|
||||
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
|
||||
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
|
||||
- C ABI ResourceProvider v1 (CM-A09, ADR-0036): trusted shared libraries expose
|
||||
open/read/write/close through the frozen header; version/load failures are
|
||||
fail-closed; project-mode + name grant only. Plugin-Worker binding remains open.
|
||||
- Worker resource provider transport (CM-A09, ADR-0035): host-managed helper
|
||||
process with framed JSON, ping/open/read/write/close; project-mode + name
|
||||
grant; trusted, not a sandbox; C ABI binding still open.
|
||||
@@ -215,7 +218,7 @@ behavior. This table records only what is in this tree now.
|
||||
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PASS (local pure lane); external stdlib repo publication pending** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), an Int matrix multiply (70), and a sparse-matrix multiply (73); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Local gate PASS: `crates/cli/tests/p2_closure.rs` runs 21 C01–C06/C11/C12 fixtures plus the stdlib table self-test in `--mode pure` (no foreign/plugin/import grants), scans sources for foreign/plugin/import shortcuts and compares value/stdout to goldens; evidence [`p2-closure-gate`](evidence/2026-11-20-p2-closure-gate.md). The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). Remaining outside C01–C12: publication as a separate external stdlib repository, and no remote CI URL. |
|
||||
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Providers are registered through ADR-0033 (in-process) and ADR-0035 (worker helper). Still missing: directories, C ABI provider binding, provider packages, strict provider support. |
|
||||
| P4 Table logical/physical migration | **PASS (local gate); T1/T2/T3 future** | ADR-0032/CM-A08: `Value::Table` semantics and GFM AST laziness unchanged; `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group/`转GFM` as value-returning `.mq.md` functions with input immutability; zero Rust table algorithms were added. Cross-storage equivalence is pinned by fixture 72 (row Table vs `.mq.md` columnar store, identical headers/rows/cells). The optional T1/T2/T3 physical-storage/column-store migration was not required by the gate and is not implemented. |
|
||||
| P5 third-party Provider / TCP / HTTP | **PASS (local gate); external network/CI and C-Worker transport open** | C10: `stdlib/net/http.mq.md` + fixture 71 parse HTTP/1.1 status/headers/chunked bodies and build requests; `http_over_tcp.rs` runs a loopback HTTP exchange wholly in `.mq.md` over `resource.open("tcp",...)`. ADR-0034 IP-literal blocking TCP endpoint (exact IP grant, project-only, 5s timeouts) and ADR-0033 in-process `ResourceProvider` v1 (registration, protocol/kind checks, name grant, mock read/write provider) and ADR-0035 host-managed worker provider transport (framed JSON helper, ping/open/read/write/close, trusted-only) are tested. External DNS/TLS/UDP, host HTTP, provider packages and the C ABI provider binding remain out of scope. |
|
||||
| P5 third-party Provider / TCP / HTTP | **PASS (local gate); external network/CI and C-Worker transport open** | C10: `stdlib/net/http.mq.md` + fixture 71 parse HTTP/1.1 status/headers/chunked bodies and build requests; `http_over_tcp.rs` runs a loopback HTTP exchange wholly in `.mq.md` over `resource.open("tcp",...)`. ADR-0034 IP-literal blocking TCP endpoint (exact IP grant, project-only, 5s timeouts) and ADR-0033 in-process `ResourceProvider` v1 (registration, protocol/kind checks, name grant, mock read/write provider) ADR-0035 host-managed worker provider transport (framed JSON helper, ping/open/read/write/close, trusted-only) and ADR-0036 trusted C ABI provider (shared-library symbols, version/load checks) are tested. External DNS/TLS/UDP, host HTTP, provider packages and the C ABI provider binding remain out of scope. |
|
||||
| P6 performance / native backends | **BASELINE ONLY** | reproducible profile harness `tools/perf/bench.py` and [`p6-profile`](evidence/2026-11-20-p6-profile.md) for 7 workloads; no bytecode/JIT/columnar numeric backend or equivalence/optimization gate yet |
|
||||
|
||||
The P1 memory domain is pure computation: it needs no grant and is available in
|
||||
@@ -261,6 +264,9 @@ user-written allocator is explicitly **not** claimed.
|
||||
echo tests.
|
||||
- [`evidence/2026-11-20-table-cross-storage.md`](evidence/2026-11-20-table-cross-storage.md)
|
||||
— P4 row-store Table vs `.mq.md` columnar store equivalence for sort/filter.
|
||||
- [`evidence/2026-11-20-c-provider.md`](evidence/2026-11-20-c-provider.md)
|
||||
— ADR-0036 C ABI ResourceProvider v1, compiled C adapter round-trip and
|
||||
fail-closed load/version checks.
|
||||
- [`evidence/2026-11-20-worker-provider.md`](evidence/2026-11-20-worker-provider.md)
|
||||
— ADR-0035 trusted worker resource-provider transport, framed-JSON helper
|
||||
round-trip and grant/mode/error tests.
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
# ADR-0036: C ABI ResourceProvider v1
|
||||
|
||||
Status: **Accepted (trusted native adapter; plugin-Worker binding still open)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A11, P5), ADR-0033, ADR-0035
|
||||
Header: `crates/marqdo-host/include/marqdo_resource_provider.h`
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-c-provider.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0033 provides an in-process provider trait and ADR-0035 a host-managed
|
||||
helper process with framed JSON. CM-A09 also needs a **C ABI** so native
|
||||
adapters can expose resource endpoints without linking Rust or changing the
|
||||
runtime. This ADR freezes the first C ABI version for trusted native adapters.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Version and symbols
|
||||
|
||||
```c
|
||||
#define MARQDO_RESOURCE_PROVIDER_ABI_VERSION 1u
|
||||
|
||||
uint32_t marqdo_resource_provider_abi_version(void);
|
||||
int32_t marqdo_resource_open(const char *kind, const char *descriptor,
|
||||
const char *rights_csv, uint64_t *out_token);
|
||||
int32_t marqdo_resource_read(uint64_t token, uint8_t *buffer,
|
||||
uint32_t capacity, uint32_t *out_read,
|
||||
int32_t *out_eof);
|
||||
int32_t marqdo_resource_write(uint64_t token, const uint8_t *bytes,
|
||||
uint32_t length, uint32_t *out_written);
|
||||
int32_t marqdo_resource_close(uint64_t token);
|
||||
int32_t marqdo_resource_last_error(char *buffer, uint32_t capacity);
|
||||
```
|
||||
|
||||
* The host loads the library, requires ABI version 1 and resolves the symbols
|
||||
at registration; missing symbols/version mismatch fail closed with
|
||||
`resource.unsupported`/`kernel.unavailable`.
|
||||
* Return `0` on success, nonzero on failure; failure text comes from
|
||||
`last_error` and is surfaced as a structured host error with the callsite
|
||||
span.
|
||||
* `rights_csv` is a comma-separated list. Provider tokens are private to the
|
||||
adapter; the host still issues the language-visible `ResourceHandle`.
|
||||
* Buffers are borrowed for the call only; the adapter must not retain host
|
||||
pointers. The host caps frame/buffer sizes by the existing resource limits.
|
||||
|
||||
### 2. Registration and authorization
|
||||
|
||||
* `StdHost::register_c_resource_provider(name, kinds, library_path)` is a host
|
||||
embedding API; it uses the same name/kind validation as ADR-0033 and the same
|
||||
provider-name grant at `resource.open`.
|
||||
* Project mode only; pure/strict refuse provider kinds. Strict stays
|
||||
fail-closed (CM-A11).
|
||||
* The C library runs in the host address space with host privileges. It is
|
||||
**trusted**, not sandboxed; the untrusted-plugin path remains the separate
|
||||
plugin Worker transport. Registration is not reachable from `.mq.md`.
|
||||
|
||||
### 3. Remaining CM-A09 work
|
||||
|
||||
* Plugin-Worker binding for `ResourceProvider` through the existing plugin ABI
|
||||
(buffer handoff/ownership/version negotiation).
|
||||
* Provider package installation, signatures and discovery.
|
||||
* Untrusted-provider isolation and strict-mode provider support.
|
||||
* ABI version negotiation beyond rejecting non-v1 libraries (future ADR).
|
||||
|
||||
## Consequences
|
||||
|
||||
* Native adapters can expose new resource kinds through `resource.*` without
|
||||
parser/`Value`/runtime/`HostOp` changes.
|
||||
* ABI mistakes are fail-closed at registration or surfaced as host errors; no
|
||||
fallback transport is attempted.
|
||||
* The header is small and stable enough for a future SDK/package workflow.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-c-provider.md` records a compiled C test provider
|
||||
(`cc -shared`), version/load checks, open/read/write/close round-trip, missing
|
||||
library fail-closed and the full workspace gate.
|
||||
@@ -31,6 +31,8 @@ implementation status lives in `docs/STATUS.md`; frozen behavior is in
|
||||
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
|
||||
| CM-A09 in-process ResourceProvider v1 | [0033](../adr/0033-in-process-resource-provider-v1.md) | Accepted (trusted in-process slice) |
|
||||
| P5 blocking TCP resource endpoint v1 | [0034](../adr/0034-tcp-resource-endpoint.md) | Accepted (IP-literal, no DNS/TLS) |
|
||||
| CM-A09 worker resource-provider transport v1 | [0035](../adr/0035-worker-resource-provider-transport.md) | Accepted (trusted helper) |
|
||||
| CM-A09 C ABI ResourceProvider v1 | [0036](../adr/0036-c-abi-resource-provider.md) | Accepted (trusted native adapter) |
|
||||
| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication |
|
||||
|
||||
## What a future phase must not do
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
# RFC-CM CM-A09 evidence: C ABI ResourceProvider v1
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `12be02e` plus this step's changes.
|
||||
Decision record: [ADR-0036](../adr/0036-c-abi-resource-provider.md)
|
||||
Header: [`marqdo_resource_provider.h`](../../crates/marqdo-host/include/marqdo_resource_provider.h)
|
||||
Related: [in-process provider](2026-11-20-provider-abi-cm-a09.md),
|
||||
[worker provider](2026-11-20-worker-provider.md), [STATUS §5](../STATUS.md).
|
||||
Scope: trusted native C ABI adapter; plugin-Worker binding still open.
|
||||
|
||||
## ABI
|
||||
|
||||
```c
|
||||
marqdo_resource_provider_abi_version() -> uint32_t (must be 1)
|
||||
marqdo_resource_open(kind, descriptor, rights_csv, *token) -> i32
|
||||
marqdo_resource_read(token, buffer, capacity, *read, *eof) -> i32
|
||||
marqdo_resource_write(token, bytes, length, *written) -> i32
|
||||
marqdo_resource_close(token) -> i32
|
||||
marqdo_resource_last_error(buffer, capacity) -> i32
|
||||
```
|
||||
|
||||
The host resolves these symbols at registration, rejects non-v1 libraries and
|
||||
missing symbols, attaches callsite spans to errors, and never lets a provider
|
||||
token enter a language `Value`, wire frame or trace.
|
||||
|
||||
## Test
|
||||
|
||||
`crates/marqdo-host/tests/c_provider.rs` (`#[cfg(unix)]`):
|
||||
|
||||
1. builds a C shared library with `cc -shared -fPIC` (skips if `cc` is absent);
|
||||
2. registers it as provider `cabi` for kind `cabi.sensor` with a project-mode
|
||||
grant;
|
||||
3. `resource.open/read/write/close` round-trips `c-`/`abi`, `write=3`, second
|
||||
close `resource.closed`;
|
||||
4. a missing library path fails with `kernel.unavailable`.
|
||||
|
||||
No Rust runtime/`HostOp`/parser change was needed; language code still sees the
|
||||
same `resource.*` surface.
|
||||
|
||||
## Security boundary
|
||||
|
||||
The C library is loaded in-process with host privileges. It is trusted native
|
||||
code and explicitly **not** a sandbox. Pure/strict refuse provider kinds; strict
|
||||
remains fail-closed. Untrusted plugin loading remains the separate plugin Worker
|
||||
transport.
|
||||
|
||||
## Not done
|
||||
|
||||
* Plugin-Worker resource-provider binding through the existing plugin ABI.
|
||||
* Provider package installation/signatures/discovery.
|
||||
* ABI version negotiation beyond v1 rejection.
|
||||
* Untrusted-provider isolation or strict-mode providers.
|
||||
* Windows shared-library build/lane.
|
||||
+1
-1
@@ -63,7 +63,7 @@ set for one run:
|
||||
| `process: bool` | `false` | required together with `foreign` for `foreign.run`/`foreign.call` |
|
||||
| `foreign: bool` | `false` | foreign kernels |
|
||||
| `plugins: Vec<String>` | empty | `plugin.invoke` and `StdHost::load_plugin` |
|
||||
| `resource_providers: Vec<String>` | empty | registered in-process/worker `ResourceProvider` kinds; project mode only; helpers are trusted, not sandboxed (ADR-0033/0035) |
|
||||
| `resource_providers: Vec<String>` | empty | registered in-process/worker/C-ABI `ResourceProvider` kinds; project mode only; helper processes and native libraries are trusted, not sandboxed (ADR-0033/0035/0036) |
|
||||
|
||||
Requests versus grants:
|
||||
|
||||
|
||||
@@ -177,6 +177,11 @@ grant/pure/strict/DNS refusal) and
|
||||
`crates/cli/tests/resource_streams.rs::tcp_stream_round_trips_through_mqmd_with_a_network_grant`
|
||||
(`--allow-net 127.0.0.1`).
|
||||
|
||||
RFC-CM CM-A09 C ABI provider (ADR-0036) is pinned by
|
||||
`crates/marqdo-host/tests/c_provider.rs`, which compiles a small C shared
|
||||
library, registers it, round-trips `resource.*` and checks missing-library
|
||||
fail-closed behavior.
|
||||
|
||||
RFC-CM CM-A09 worker provider transport (ADR-0035) is pinned by
|
||||
`crates/marqdo-host/tests/worker_provider.rs`: a Python helper is spawned,
|
||||
pinged, opened/read/written/closed through framed JSON; missing grant, pure,
|
||||
|
||||
Reference in new issue
Block a user