feat(ext): single callable binding, provider v2 CLI, capability handlers, stdlib aliases

Implements RFC-EXT-0001 PR-00..PR-07 first pass:
- CallSiteKey/CallBindingTable + one frontend binding pass; runtime dispatch by identity
- prepare composition root shared by check/run/inspect/Analysis; exact plugin reconciliation
- provider.json v2 --provider/--allow-provider worker CLI with fail-closed verification
- capability handler registry with real fs.read and audit
- verified stdlib public aliases replace print_override/AST scan
- docs: specs, ADR-0054/0055/0057/0058, README, CI/release gates, evidence

PR-04 extension selection/lock v4 and final frozen-binary evidence remain.
This commit is contained in:
cflmy committed 2026-11-22 00:17:36 +08:00
1 parent 9aa536c2c9
commit 1f6fd644a0
54 files changed
+8710 -828

No files matched your search

+128
View File
@@ -163,6 +163,25 @@ rules are [`RFC-LANG-0002`](docs/architecture/RFC-LANG-0002-unified-semantics.md
* **Provider package manifest v1 (RFC-CM CM-A09)** — local `marqdo.provider`
manifest installs a trusted C ABI provider; no registry/signature/CLI install
(ADR-0037).
* **Third-party Resource Provider v2 through the normal CLI (RFC-EXT-0001
PR-03)** — a `provider.json` v2 bundle with a targeted, sha256-verified
`c-abi-1` artifact is selected with repeatable `--provider` and authorized by
the exact `--allow-provider` name on `run`/`call`; the existing
`marqdo-provider-worker` transport is spawned only after every manifest,
artifact, grant and duplicate-kind check passes. `check`/`inspect` never load
providers, `pure`/`strict` refuse them, and the worker transport is
linux/x86_64-only in this release (ADR-0055,
[`spec/resource-provider-v2.md`](spec/resource-provider-v2.md)).
* **Single callable binding through the frozen composition root
(RFC-EXT-0001 PR-01/02)** — one `CallableRegistry` and one binding pass feed
`check`/`run`/`call`/`inspect`/Analysis; the runtime dispatches by
`CallSiteKey` and never re-resolves a name or guesses a plugin. Static plugin
descriptors must reconcile one-to-one with the loaded exports (ADR-0054,
[`spec/callable-resolution.md`](spec/callable-resolution.md)).
* **Official stdlib public aliases (RFC-EXT-0001 PR-06)** — `打印`/`print` are
verified `stdlib.json` metadata bound to `io.mq.md::写值`; the
`print_override`/AST-scan/runtime special case is removed with byte-identical
output (ADR-0058).
* **C ABI ResourceProvider v1 (RFC-CM CM-A09)** — trusted native shared
libraries expose resource open/read/write/close through a frozen header;
project + name grant; plugin-Worker binding remains open (ADR-0036).
@@ -358,6 +377,115 @@ marqdo benchmark run .
marqdo tool remove benchmark
```
### Extending Marqdo without modifying or rebuilding the CLI
RFC-EXT-0001 defines five extension kinds. Each one is discovered only through
an explicit manifest/flag, is versioned, and is authorized separately from
being visible. A physically separate repository exercising all five against a
frozen binary is `marqdo-extension-conformance`
(`https://git.cflmy.cn/cflmy/marqdo-extension-conformance`).
| Kind | Static discovery | Execution | Grant |
|---|---|---|---|
| `.mq.md` package | `marqdo.yaml` dependency + `marqdo.lock` | Marqdo interpreter | project mode |
| Callable plugin | `plugin.yaml` ABI v1 exports | worker (default) or trusted native | `--plugin` + `--allow-plugin NAME` |
| Foreign kernel | `kernel.json` + pinned executable digest | one-shot child process | `--kernel` + `--allow-foreign` |
| Resource provider | `provider.json` schema v2 | `marqdo-provider-worker` + C ABI | `--provider` + `--allow-provider NAME` |
| CLI Tool | `tool.json` install record | independent CLI process | tool installation, no project grants |
```bash
# 1. Pure Markdown library from a pinned Git dependency
marqdo lock && marqdo sync
marqdo check consumer/index.mq.md
marqdo run consumer/index.mq.md
# 2. Callable ABI plugin (worker transport by default)
marqdo run plugin/demo.mq.md \
--plugin plugin/build --allow-plugin demo.ext
# 3. Foreign kernel with a language label the CLI has never heard of
marqdo run kernel/demo.mq.md \
--kernel kernel/build/kernel.json --allow-foreign
# 4. Resource provider v2 (see the next section)
marqdo run provider/demo.mq.md \
--provider provider/build/provider.json --allow-provider demo.sensor
# 5. Independent CLI Tool consuming only public Analysis v2
marqdo tool install tool/tool.json
marqdo inspect-ext tool/demo.mq.md
```
#### One callable resolution, visible in `check` and `run`
Calls are bound statically once per prepared program (ADR-0054, normative
details in [`spec/callable-resolution.md`](spec/callable-resolution.md)):
* `f(...)`: local callable value → official alias/reserved builtin → current
namespace → canonical name → unique local name;
* `alias.f(...)`: explicit import alias → builtin dotted name → exact plugin
export;
* `some_value.f(...)`: dynamic language value only.
`check` and `run` consume the same binding; an unknown export, wrong arguments
or a duplicate export fails before execution. An import alias that collides
with a builtin/plugin dotted namespace (for example importing a module as
`math`) is rejected with `ext.import_namespace_conflict`; rename it. A plugin
manifest is the static contract: `run` verifies the loaded artifact and its
exports/signatures match the manifest exactly
(`ext.plugin_binding_mismatch` otherwise) and never synthesises a signature.
#### Official stdlib aliases, no print magic
`打印`/`print` are declared in the verified `stdlib/stdlib.json`
(`public_aliases`, covered by `aliases_digest`) and bound as ordinary
`Marqdo` calls to `io.mq.md::写值` (ADR-0058). There is no
`Program.print_override`, no AST-wide print scan and no runtime print branch;
output bytes, permissions and Trace behavior are unchanged, and a program that
never uses the alias does not load the stdlib module graph. Third-party
packages cannot claim `打印`/`print` or `std.*` names.
### Third-party resource providers (`provider.json` v2)
An external repository adds a new resource kind (for example `demo.sensor`)
with a `provider.json` schema-version-2 bundle and a targeted `c-abi-1` shared
library. No CLI rebuild and no core change are needed: select the manifest with
repeatable `--provider` and authorize it by its exact manifest `name` with
`--allow-provider` (also repeatable). A `--provider` without a matching
`--allow-provider` is `permission.denied` and the helper is never started;
wildcards (`*`) are not accepted.
```bash
marqdo run index.mq.md \
--provider vendor/demo-sensor/provider.json \
--allow-provider demo.sensor
marqdo call index.mq.md main \
--provider vendor/demo-sensor/provider.json \
--allow-provider demo.sensor
```
The document still uses the ordinary resource surface, e.g.
`resource.open("demo.sensor", "unit-1", ["read"])`; provider kinds do not become
new callables.
* `check`/`inspect` do not accept these options and never load a provider.
* `--mode pure` and `--mode strict` reject `--provider`/`--allow-provider` in
preflight; strict L1 stays fail-closed and does not silently switch to
`project`.
* Everything is verified before any worker starts: `schema_version: 2`,
`abi: c-abi-1`, `transport: worker`, strict SemVer, non-reserved kinds, the
current target artifact's bundle containment and sha256, exact grants and
duplicate provider names/kinds.
* The helper `marqdo-provider-worker` must ship next to the `marqdo` executable
for the same target (no `PATH` fallback); a missing helper is
`ext.worker_unavailable`. The worker transport is linux/x86_64-only in this
release; other platforms fail with `ext.platform_unsupported`.
* The helper is a trusted process, not an OS sandbox: it runs with the user's
privileges. `sha256` is an integrity check, not publisher authentication.
See [`spec/resource-provider-v2.md`](spec/resource-provider-v2.md) and
[ADR-0055](docs/adr/0055-resource-provider-cli-v2.md).
## References
* [CommonMark 0.31.2](https://spec.commonmark.org/0.31.2/) ·
+23 -31
View File
@@ -536,7 +536,7 @@ fn expr_calls(
) {
match expr {
Expr::Call { callee, args, span } => {
let (state, target) = resolve_callee(project, module, callee);
let (state, target) = resolve_callee(project, module, function, *span);
calls.push(json!({
"function": function.id.to_string(),
"span": span_json(*span),
@@ -572,42 +572,34 @@ fn expr_calls(
}
}
/// Project the authoritative call binding into the Analysis `state`/`target`
/// fields.
///
/// Analysis is a read-only projection of the same prepared binding the runtime
/// consumes (RFC-EXT-0001 §4/§9): there is no second name-resolution algorithm
/// here. `state` keeps its v1 vocabulary (`function`, `builtin`, `plugin`,
/// `unresolved`); official stdlib aliases such as `打印` therefore appear as
/// `function` targets because that is the implementation that actually runs.
fn resolve_callee(
project: &LoadedProject,
module: &Module,
callee: &Expr,
_module: &Module,
function: &Function,
span: Span,
) -> (&'static str, Value) {
match callee {
Expr::Ident(ident) => {
if marqdo_ir::builtins::find(&ident.name).is_some() {
("builtin", json!(ident.name))
} else if let Some(function) = module.function(&ident.name) {
("function", json!(function.id.to_string()))
} else {
("unresolved", Value::Null)
}
let key = marqdo_ir::call_binding::CallSiteKey::new(function.id.clone(), span);
match project.program.call_bindings.get(&key) {
Some(marqdo_ir::call_binding::BoundTarget::Builtin { canonical_name }) => {
("builtin", json!(canonical_name))
}
Expr::Member { base, name, .. } => {
if let Expr::Ident(alias) = base.as_ref() {
let qualified = format!("{}.{}", alias.name, name);
if marqdo_ir::builtins::find(&qualified).is_some() {
return ("builtin", json!(qualified));
}
if let Some(target_module) = project.program.resolve_import(&module.id, &alias.name)
{
if let Some(target) = project.program.module(target_module) {
if let Some(function) = target.function(name) {
return ("function", json!(function.id.to_string()));
}
}
}
if project.program.external_callable(&qualified).is_some() {
return ("plugin", json!(qualified));
}
}
Some(marqdo_ir::call_binding::BoundTarget::Marqdo { function }) => {
("function", json!(function.to_string()))
}
Some(marqdo_ir::call_binding::BoundTarget::AbiPlugin { export, .. }) => {
("plugin", json!(export))
}
Some(marqdo_ir::call_binding::BoundTarget::DynamicValue) | None => {
("unresolved", Value::Null)
}
_ => ("unresolved", Value::Null),
}
}
+26 -132
View File
@@ -27,148 +27,42 @@ pub use sandbox::SandboxSettings;
pub use tool::{fmt, plugin_inspect};
use crate::args;
use crate::loader::{self, FileDiagnostic, LoadedProject};
use crate::extension_descriptors::{self, StaticPluginSet};
use crate::loader::{FileDiagnostic, LoadedProject};
use crate::output;
use crate::project_prepare::{self, PrepareOptions, PreparedProgram};
fn load_project(file: Option<&Path>, json: bool) -> Result<LoadedProject, i32> {
let file = file.unwrap_or_else(|| Path::new("index.mq.md"));
match loader::load(file) {
Ok(project) => Ok(project),
/// Prepare one document through the single composition-root step.
///
/// `check`/`run`/`call`/`inspect` must all use this function so they share the
/// same call bindings and diagnostics (RFC-EXT-0001 §4.4, §18.2).
pub(super) fn prepare_project(
file: Option<&Path>,
json: bool,
plugins: &[PathBuf],
) -> Result<PreparedProgram, i32> {
let options = PrepareOptions {
plugins: plugins.to_vec(),
};
match project_prepare::prepare(file, &options) {
Ok(prepared) => Ok(prepared),
Err(error) => {
if json {
output::print_json(&json!({"success": false, "error": error}));
output::print_json(&json!({
"success": false,
"error": {
"code": error.code,
"message": error.message,
}
}));
} else {
eprintln!("error: {error}");
}
if error.contains("marqdo.lock") || error.contains("lockfile") {
Err(1)
} else {
Err(2)
eprintln!("error[{}]: {}", error.code, error.message);
}
Err(error.exit_code)
}
}
}
/// Add plugin export descriptors from manifests without loading a library.
fn add_static_plugin_descriptors(
project: &mut LoadedProject,
paths: &[PathBuf],
) -> Result<(), String> {
for path in paths {
let manifest_path = if path.is_dir() {
path.join("plugin.yaml")
} else {
path.clone()
};
let manifest = marqdo_plugin_abi::PluginManifest::load(&manifest_path)
.map_err(|error| format!("{}: {}", error.code, error.message))?;
manifest
.validate()
.map_err(|error| format!("{}: {}", error.code, error.message))?;
let digest = manifest
.resolve_artifact(&marqdo_plugin_abi::HostTarget::current())
.map_err(|error| format!("{}: {}", error.code, error.message))?
.sha256
.clone();
for export in &manifest.plugin.exports {
if project
.program
.external_callables
.iter()
.any(|callable| callable.public_name == export.name)
{
return Err(format!(
"plugin export `{}` is declared more than once",
export.name
));
}
let signature = plugin_signature(&export.signature).map_err(|error| {
format!(
"plugin export `{}` has an invalid signature: {error}",
export.name
)
})?;
project
.program
.external_callables
.push(marqdo_ir::callable::CallableDescriptor {
id: marqdo_ir::callable::CallableId(format!(
"plugin:{}:{}",
manifest.plugin.name, export.name
)),
public_name: export.name.clone(),
owner_package: manifest.plugin.name.clone(),
signature,
effects_requested: manifest.plugin.requires.clone(),
origin: marqdo_ir::callable::CallableOrigin::AbiPlugin {
package: manifest.plugin.name.clone(),
export: export.name.clone(),
},
implementation_digest: Some(digest.clone()),
source_span: None,
});
}
}
if paths.is_empty() {
return Ok(());
}
let existing: Vec<(Option<marqdo_span::Span>, String)> = project
.diagnostics
.iter()
.map(|file| {
(
file.diagnostic.span,
format!("{}:{}", file.module_id, file.diagnostic.message),
)
})
.collect();
for (module_id, diagnostic) in marqdo_frontend::calls::validate_program(&project.program) {
let key = format!("{}:{}", module_id, diagnostic.message);
if existing.contains(&(diagnostic.span, key)) {
continue;
}
if let Some(file) = project.files.get(&module_id) {
project.diagnostics.push(FileDiagnostic {
module_id,
path: file.path.clone(),
source: file.source.clone(),
diagnostic,
});
}
}
Ok(())
}
/// Convert a plugin manifest signature (`($a: Int) -> Int`) to the language
/// parameter list, dropping the display-only return annotation and `$` sigils.
fn plugin_signature(raw: &str) -> Result<marqdo_ir::callable::CallableSignature, String> {
let before_return = raw.split("->").next().unwrap_or(raw).trim();
let open = before_return
.find('(')
.ok_or_else(|| "missing `(`".to_string())?;
let close = before_return
.rfind(')')
.ok_or_else(|| "missing `)`".to_string())?;
if close < open {
return Err("malformed parameter list".to_string());
}
let params = before_return[open + 1..close].replace('$', "");
let parsed =
marqdo_frontend::signature::parse(&format!("({params})"), marqdo_span::Span::default());
if parsed.has_errors() {
return Err(parsed
.diagnostics
.iter()
.map(|diagnostic| diagnostic.message.clone())
.collect::<Vec<_>>()
.join("; "));
}
Ok(marqdo_ir::callable::CallableSignature {
params: parsed.params,
kind: marqdo_ir::SignatureKind::Explicit,
})
}
/// `marqdo check`.
fn validate_refs(project: &LoadedProject) -> Vec<FileDiagnostic> {
let mut diagnostics = Vec::new();
+3 -10
View File
@@ -9,18 +9,11 @@ pub fn check(
plugins: &[PathBuf],
emit: Option<&Path>,
) -> i32 {
let mut project = match load_project(file, json) {
Ok(project) => project,
let prepared = match prepare_project(file, json, plugins) {
Ok(prepared) => prepared,
Err(code) => return code,
};
if let Err(error) = add_static_plugin_descriptors(&mut project, plugins) {
if json {
output::print_json(&json!({"success": false, "error": error}));
} else {
eprintln!("error: {error}");
}
return 2;
}
let mut project = prepared.loaded;
if refs {
let ref_diagnostics = validate_refs(&project);
project.diagnostics.extend(ref_diagnostics);
+3 -2
View File
@@ -20,10 +20,11 @@ pub struct InspectFlags {
}
pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool) -> i32 {
let project = match load_project(file, json) {
Ok(project) => project,
let prepared = match prepare_project(file, json, &[]) {
Ok(prepared) => prepared,
Err(code) => return code,
};
let project = prepared.loaded;
let entry = project.entry_file();
let module = &entry.analysis.module;
let selected_any =
+253 -42
View File
@@ -4,6 +4,7 @@
//! runtime helpers, but keeps the run session out of the command table.
use super::*;
use crate::provider_manifest;
use marqdo_sandbox::StrictChildState;
pub fn execute(
@@ -15,18 +16,12 @@ pub fn execute(
trace: bool,
) -> i32 {
let command_name = if function.is_some() { "call" } else { "run" };
let mut project = match load_project(file, json) {
Ok(project) => project,
let prepared = match prepare_project(file, json, &settings.plugins) {
Ok(prepared) => prepared,
Err(code) => return code,
};
if let Err(error) = add_static_plugin_descriptors(&mut project, &settings.plugins) {
if json {
output::print_json(&json!({"success": false, "error": error}));
} else {
eprintln!("error: {error}");
}
return 2;
}
let project = prepared.loaded;
let static_plugins = prepared.plugins;
if project.has_errors() {
if json {
output::print_json(&json!({
@@ -136,10 +131,80 @@ pub fn execute(
return 2;
}
}
for plugin_path in &settings.plugins {
if let Err(message) =
load_plugin_into_host(&mut host, plugin_path, settings.trust_native_plugin)
// T6: verify every selected artifact/path/hash before starting a worker or
// loading native code. Static path/signature checks happened in `prepare`.
//
// Provider selection is host assembly, not callable semantics: `check`
// never sees it. Parse and verify every `--provider` manifest, resolve the
// current target artifact, check the exact grant and reject duplicate
// names/kinds *before* any provider or plugin worker process is spawned
// (RFC-EXT-0001 §5.4.6/§18.2 T6–T7).
let providers = match provider_manifest::read_provider_manifests(&settings.providers) {
Ok(providers) => providers,
Err(error) => {
emit_prepare_error(json, command_name, &error);
return error.exit_code;
}
};
// `--provider` alone never authorizes; the exact `--allow-provider NAME`
// is required before the helper process may be started. An ungranted
// provider fails closed instead of being silently ignored.
for provider in &providers {
if !settings
.allow_provider
.iter()
.any(|grant| grant == &provider.name)
{
let error = extension_descriptors::PrepareError::load(
"permission.denied",
format!(
"resource provider `{}` was selected with --provider but is not granted; \
add --allow-provider {}",
provider.name, provider.name
),
);
emit_prepare_error(json, command_name, &error);
return error.exit_code;
}
}
// A grant that matches no selected provider is a deterministic option
// error: providers are never discovered from PATH, the working directory or
// the network.
for grant in &settings.allow_provider {
if !providers.iter().any(|provider| &provider.name == grant) {
let error = extension_descriptors::PrepareError::option(
"ext.provider_grant_unmatched",
format!(
"--allow-provider `{grant}` matches no selected --provider manifest; \
providers are never discovered automatically"
),
);
emit_prepare_error(json, command_name, &error);
return error.exit_code;
}
}
for plugin in &static_plugins.plugins {
if let Err(error) = extension_descriptors::verify_bundle_regular_file(
&plugin.base_dir,
&plugin.artifact_relative,
) {
emit_prepare_error(json, command_name, &error);
return error.exit_code;
}
let artifact = plugin.artifact_path();
if let Err(error) = extension_descriptors::verify_sha256(&artifact, &plugin.artifact_sha256)
{
emit_prepare_error(json, command_name, &error);
return error.exit_code;
}
}
// T7: start only selected, granted and verified extensions.
for plugin in &static_plugins.plugins {
if let Err(message) = load_plugin_into_host(
&mut host,
&plugin.manifest_path,
settings.trust_native_plugin,
) {
if json {
output::print_json(&json!({"success": false, "error": message}));
} else {
@@ -148,36 +213,24 @@ pub fn execute(
return 1;
}
}
// Expose the complete export names loaded from trusted manifests so the
// runtime resolves exactly those names instead of probing namespaces.
for (name, _signature, digest, package) in host.plugin_exports() {
if project
.program
.external_callables
.iter()
.any(|callable| callable.public_name == name)
{
continue;
// T8: exact one-to-one reconciliation with the static descriptors. Never
// synthesise a missing signature (`params: Vec::new()`) or guess an export.
if let Err(message) = reconcile_plugin_exports(&static_plugins, &host) {
emit_error(json, command_name, "ext.plugin_binding_mismatch", &message);
return 1;
}
// T7b: start only granted and verified provider helpers. Every manifest,
// artifact, target, grant and duplicate-kind check above already passed, so
// no worker is spawned for a provider that failed verification.
for provider in &providers {
if let Err(error) = provider_manifest::load_resource_provider_into_host(
&mut host,
provider,
&settings.allow_provider,
) {
emit_prepare_error(json, command_name, &error);
return error.exit_code;
}
project
.program
.external_callables
.push(marqdo_ir::callable::CallableDescriptor {
id: marqdo_ir::callable::CallableId(format!("plugin:{package}:{name}")),
public_name: name.clone(),
owner_package: package.clone(),
signature: marqdo_ir::callable::CallableSignature {
params: Vec::new(),
kind: marqdo_ir::SignatureKind::Explicit,
},
effects_requested: vec!["host".to_string()],
origin: marqdo_ir::callable::CallableOrigin::AbiPlugin {
package,
export: name,
},
implementation_digest: Some(digest),
source_span: None,
});
}
let function_name = function.unwrap_or("main").to_string();
let entry_module = &project.program.modules[&project.entry];
@@ -292,6 +345,80 @@ pub fn execute(
}
}
fn emit_prepare_error(
json: bool,
command_name: &str,
error: &crate::extension_descriptors::PrepareError,
) {
if json {
output::print_json(&json!({
"command": command_name,
"success": false,
"error": {"code": error.code, "message": error.message},
}));
} else {
eprintln!("error[{}]: {}", error.code, error.message);
}
}
/// Compare the Host's actual plugin exports with the static manifest set.
///
/// Names, raw signatures, owners and verified artifact digests must match
/// exactly; extra, missing or altered exports fail closed with
/// `ext.plugin_binding_mismatch` and the main function never runs.
fn reconcile_plugin_exports(
static_plugins: &StaticPluginSet,
host: &StdHost,
) -> Result<(), String> {
let mut expected: std::collections::BTreeMap<String, (String, String, String)> =
std::collections::BTreeMap::new();
for plugin in &static_plugins.plugins {
for export in &plugin.exports {
expected.insert(
export.name.clone(),
(
export.signature.clone(),
plugin.artifact_sha256.clone(),
plugin.package.clone(),
),
);
}
}
let loaded = host.plugin_exports();
if loaded.len() != expected.len() {
return Err(format!(
"static plugin descriptors declare {} export(s) but the runtime loaded {}; \
refusing to execute",
expected.len(),
loaded.len()
));
}
for (name, signature, digest, package) in loaded {
let Some((expected_signature, expected_digest, expected_package)) = expected.get(&name)
else {
return Err(format!(
"runtime export `{name}` was not declared by any selected plugin manifest"
));
};
if &signature != expected_signature {
return Err(format!(
"export `{name}` signature mismatch: manifest `{expected_signature}`, runtime `{signature}`"
));
}
if &digest != expected_digest {
return Err(format!(
"export `{name}` artifact digest mismatch: manifest `{expected_digest}`, runtime `{digest}`"
));
}
if &package != expected_package {
return Err(format!(
"export `{name}` owner mismatch: manifest `{expected_package}`, runtime `{package}`"
));
}
}
Ok(())
}
/// Read piped stdin once at the composition boundary.
///
/// A terminal stdin is not read, so `marqdo run` never blocks waiting for an
@@ -310,3 +437,87 @@ fn read_stdin() -> Vec<u8> {
}
bytes
}
#[cfg(test)]
mod tests {
use super::*;
fn empty_host() -> StdHost {
StdHost::new(
SandboxMode::Project,
marqdo_host::Capabilities::pure(),
std::env::temp_dir(),
)
}
fn static_plugin_with_export() -> StaticPluginSet {
let descriptor = marqdo_ir::callable::CallableDescriptor {
id: marqdo_ir::callable::CallableId("plugin:demo.ext:solver.add".into()),
public_name: "solver.add".into(),
owner_package: "demo.ext".into(),
signature: marqdo_ir::callable::CallableSignature {
params: vec![
marqdo_ir::Param {
name: "a".into(),
span: Span::default(),
type_annotation: None,
default: None,
},
marqdo_ir::Param {
name: "b".into(),
span: Span::default(),
type_annotation: None,
default: None,
},
],
kind: marqdo_ir::SignatureKind::Explicit,
},
effects_requested: vec!["math.compute".into()],
origin: marqdo_ir::callable::CallableOrigin::AbiPlugin {
package: "demo.ext".into(),
export: "solver.add".into(),
},
implementation_digest: Some("a".repeat(64)),
source_span: None,
};
StaticPluginSet {
plugins: vec![crate::extension_descriptors::StaticPlugin {
manifest_path: PathBuf::from("/fixtures/demo/plugin.yaml"),
base_dir: PathBuf::from("/fixtures/demo"),
package: "demo.ext".into(),
version: "0.1.0".into(),
kind: marqdo_plugin_abi::PluginKind::Worker,
manifest_sha256: format!("sha256:{}", "b".repeat(64)),
artifact_target: marqdo_plugin_abi::HostTarget::current()
.triple()
.to_string(),
artifact_sha256: "a".repeat(64),
artifact_relative: "build/plugin.so".into(),
exports: vec![crate::extension_descriptors::StaticPluginExport {
name: "solver.add".into(),
signature: "($a: Int, $b: Int) -> Int".into(),
descriptor,
}],
requires: Vec::new(),
}],
}
}
#[test]
fn reconcile_accepts_matching_empty_sets() {
assert!(
reconcile_plugin_exports(&StaticPluginSet::default(), &empty_host()).is_ok(),
"no selected plugin and no loaded plugin must reconcile"
);
}
#[test]
fn reconcile_rejects_missing_runtime_exports() {
let error = reconcile_plugin_exports(&static_plugin_with_export(), &empty_host())
.expect_err("a declared export that the runtime did not load must fail closed");
assert!(
error.contains("declare 1 export(s)") || error.contains("not declared"),
"{error}"
);
}
}
+45 -5
View File
@@ -43,6 +43,10 @@ pub struct SandboxSettings {
pub timeout: Option<u64>,
/// Explicitly trusted plugin manifests to load.
pub plugins: Vec<PathBuf>,
/// Explicitly selected resource provider v2 manifests (RFC-EXT-0001 §5.4).
pub providers: Vec<PathBuf>,
/// Exact resource provider name grants (declaration is not a grant).
pub allow_provider: Vec<String>,
/// Explicit trust decision for in-process native plugin loading.
pub trust_native_plugin: bool,
/// Optional Trace v2 output path.
@@ -68,8 +72,8 @@ impl SandboxSettings {
/// Reject strict-mode requests that this L1 isolation level does not
/// support. Strict currently covers pure Marqdo and native Math only;
/// granting Foreign, ABI plugins, tools or trace files would require
/// extending the supervisor contract first.
/// granting Foreign, ABI plugins, resource providers, tools or trace files
/// would require extending the supervisor contract first.
pub fn strict_preflight(&self) -> Result<(), String> {
let requested = self.allow_foreign
|| !self.allow_read.is_empty()
@@ -80,6 +84,8 @@ impl SandboxSettings {
|| self.allow_random
|| !self.allow_plugin.is_empty()
|| !self.plugins.is_empty()
|| !self.providers.is_empty()
|| !self.allow_provider.is_empty()
|| self.trust_native_plugin
|| !self.kernels.is_empty()
|| self.trace_file.is_some();
@@ -87,8 +93,9 @@ impl SandboxSettings {
return Err(
"strict mode currently supports pure Marqdo and native Math only; \
--allow-read/--allow-write/--allow-env/--allow-process/--allow-random/\
--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file are
rejected. Use --mode project for those capabilities."
--allow-foreign/--allow-plugin/--plugin/--provider/--allow-provider/\
--trust-native-plugin/--kernel/--trace-file are rejected. \
Use --mode project for those capabilities."
.to_string(),
);
}
@@ -108,9 +115,13 @@ impl SandboxSettings {
|| self.allow_random
|| self.allow_foreign
|| !self.allow_plugin.is_empty()
|| !self.providers.is_empty()
|| !self.allow_provider.is_empty()
{
return Err(
"pure mode prohibits external I/O grants; use --mode project".to_string(),
"pure mode prohibits external I/O grants and extension providers; \
use --mode project"
.to_string(),
);
}
Ok(Capabilities::pure())
@@ -141,6 +152,10 @@ impl SandboxSettings {
for plugin in &self.allow_plugin {
capabilities = capabilities.allow_plugin(plugin.clone());
}
for provider in &self.allow_provider {
validate_provider_grant(provider)?;
capabilities = capabilities.allow_resource_provider(provider.clone());
}
Ok(capabilities)
}
SandboxMode::Strict => {
@@ -150,3 +165,28 @@ impl SandboxSettings {
}
}
}
/// Reject non-exact `--allow-provider` grants.
///
/// Provider grants are exact stable names (RFC-EXT-0001 §5.2): there is no `*`
/// wildcard, no prefix matching and no implicit discovery, so a grant either
/// names a selected manifest or the run fails closed.
fn validate_provider_grant(name: &str) -> Result<(), String> {
if name.is_empty() {
return Err("--allow-provider requires a non-empty provider name".to_string());
}
if name
.chars()
.any(|character| matches!(character, '*' | '?' | '[' | ']'))
{
return Err(format!(
"--allow-provider `{name}`: wildcards are not supported; grants are exact provider names"
));
}
if name.chars().any(char::is_control) {
return Err(format!(
"--allow-provider `{name}`: provider names must not contain control characters"
));
}
Ok(())
}
+414
View File
@@ -0,0 +1,414 @@
//! Static extension descriptors for the CLI composition root
//! (RFC-EXT-0001 §4.4/§6, ADR-0054/0056).
//!
//! This module only *reads* manifests. It never `dlopen`s a library, never
//! spawns a worker, never opens a network connection and never executes a
//! side-effect script. The descriptors produced here feed the single
//! [`marqdo_ir::callable::CallableRegistry`] and binding pass; `run` later
//! verifies the real artifacts against these recorded identities.
use std::collections::BTreeSet;
use std::path::{Component, Path, PathBuf};
use marqdo_ir::callable::{CallableDescriptor, CallableId, CallableOrigin, CallableSignature};
use marqdo_ir::SignatureKind;
use marqdo_plugin_abi::{HostTarget, PluginKind, PluginManifest};
use marqdo_span::Span;
use sha2::{Digest, Sha256};
/// A structured prepare-stage failure.
#[derive(Debug, Clone)]
pub struct PrepareError {
/// Stable diagnostic code (`ext.*` unless reusing an existing event code).
pub code: String,
/// Human-readable message; never contains secrets or binary content.
pub message: String,
/// CLI exit code (2 for manifest/option errors, 1 for load/verify errors).
pub exit_code: i32,
}
impl PrepareError {
/// Construct a manifest/option error (exit code 2).
pub fn option(code: impl Into<String>, message: impl Into<String>) -> Self {
Self {
code: code.into(),
message: message.into(),
exit_code: 2,
}
}
/// Construct a loading/verification error (exit code 1).
pub fn load(code: impl Into<String>, message: impl Into<String>) -> Self {
Self {
code: code.into(),
message: message.into(),
exit_code: 1,
}
}
}
/// One statically declared export.
#[derive(Debug, Clone)]
pub struct StaticPluginExport {
/// Complete dotted export name.
pub name: String,
/// Raw manifest signature string, compared exactly at run time.
pub signature: String,
/// Parsed static descriptor used by the binding pass.
pub descriptor: CallableDescriptor,
}
/// One statically read plugin package.
///
/// The identity fields are recorded for `inspect --extensions`, evidence and
/// reconciliation; they are not consulted for name resolution (the binding
/// table is).
#[allow(dead_code)]
#[derive(Debug, Clone)]
pub struct StaticPlugin {
/// Canonical manifest path.
pub manifest_path: PathBuf,
/// Package directory (artifact root).
pub base_dir: PathBuf,
/// Declared package name.
pub package: String,
/// Declared package version.
pub version: String,
/// Deployment kind (`native`, `worker`, `wasm`).
pub kind: PluginKind,
/// `sha256:<hex>` of the manifest file bytes.
pub manifest_sha256: String,
/// Target triple selected for this host.
pub artifact_target: String,
/// Declared (not yet verified) artifact sha256, lowercase hex.
pub artifact_sha256: String,
/// Bundle-relative artifact path.
pub artifact_relative: String,
/// Exports in manifest order.
pub exports: Vec<StaticPluginExport>,
/// Declared capability requests (metadata only; never grants).
pub requires: Vec<String>,
}
impl StaticPlugin {
/// Absolute artifact path inside the package bundle.
pub fn artifact_path(&self) -> PathBuf {
self.base_dir.join(&self.artifact_relative)
}
}
/// Static plugin selection read from explicit CLI `--plugin` flags.
#[derive(Debug, Clone, Default)]
pub struct StaticPluginSet {
/// Packages in deterministic manifest order.
pub plugins: Vec<StaticPlugin>,
}
impl StaticPluginSet {
/// All export descriptors in deterministic order.
pub fn descriptors(&self) -> Vec<CallableDescriptor> {
let mut descriptors = Vec::new();
for plugin in &self.plugins {
for export in &plugin.exports {
descriptors.push(export.descriptor.clone());
}
}
descriptors
}
}
/// Read and validate all explicitly selected `--plugin` manifests.
///
/// No binary is opened or executed. Duplicate package identities, duplicate
/// full export names, reserved names and malformed signatures are rejected
/// here, before any binding is produced.
pub fn read_plugin_descriptors(paths: &[PathBuf]) -> Result<StaticPluginSet, PrepareError> {
let mut set = StaticPluginSet::default();
let mut export_names: BTreeSet<String> = BTreeSet::new();
let mut packages: BTreeSet<(String, String)> = BTreeSet::new();
for path in paths {
let manifest_path = if path.is_dir() {
path.join("plugin.yaml")
} else {
path.clone()
};
let manifest_path = manifest_path.canonicalize().map_err(|error| {
PrepareError::option(
"ext.manifest_unreadable",
format!(
"cannot open plugin manifest `{}`: {error}",
manifest_path.display()
),
)
})?;
let base_dir = manifest_path
.parent()
.map(Path::to_path_buf)
.unwrap_or_else(|| PathBuf::from("."));
let base_dir = base_dir.canonicalize().unwrap_or(base_dir);
let manifest_bytes = std::fs::read(&manifest_path).map_err(|error| {
PrepareError::option(
"ext.manifest_unreadable",
format!(
"cannot read plugin manifest `{}`: {error}",
manifest_path.display()
),
)
})?;
let manifest = PluginManifest::load(&manifest_path)
.map_err(|error| PrepareError::option(error.code.clone(), error.message.clone()))?;
manifest
.validate()
.map_err(|error| PrepareError::option(error.code.clone(), error.message.clone()))?;
let host = HostTarget::current();
let artifact = manifest.resolve_artifact(&host).map_err(|error| {
PrepareError::option("ext.no_matching_artifact", error.message.clone())
})?;
let artifact_relative = artifact.file.clone();
ensure_bundle_relative(&artifact_relative).map_err(|error| {
PrepareError::option(
"ext.artifact_integrity",
format!(
"plugin `{}` artifact path `{artifact_relative}` is invalid: {error}",
manifest.plugin.name
),
)
})?;
let manifest_sha256 = format!("sha256:{}", hex(&Sha256::digest(&manifest_bytes)));
let identity = (manifest.plugin.name.clone(), manifest_sha256.clone());
if !packages.insert(identity.clone()) {
// The exact same package declaration twice is idempotent.
continue;
}
if set
.plugins
.iter()
.any(|plugin| plugin.package == identity.0)
{
return Err(PrepareError::option(
"ext.duplicate_callable",
format!(
"plugin package `{}` is selected with two different manifests; \
explicit aliases are not implemented, so this is rejected",
identity.0
),
));
}
let mut exports = Vec::new();
for export in &manifest.plugin.exports {
if !export_names.insert(export.name.clone()) {
return Err(PrepareError::option(
"ext.duplicate_callable",
format!(
"plugin export `{}` is declared by more than one selected package",
export.name
),
));
}
let signature = plugin_signature(&export.signature).map_err(|error| {
PrepareError::option(
"ext.plugin_invalid_signature",
format!(
"plugin export `{}` has an invalid signature: {error}",
export.name
),
)
})?;
let descriptor = CallableDescriptor {
id: CallableId(format!("plugin:{}:{}", manifest.plugin.name, export.name)),
public_name: export.name.clone(),
owner_package: manifest.plugin.name.clone(),
signature,
effects_requested: manifest.plugin.requires.clone(),
origin: CallableOrigin::AbiPlugin {
package: manifest.plugin.name.clone(),
export: export.name.clone(),
},
implementation_digest: Some(artifact.sha256.clone()),
source_span: None,
};
exports.push(StaticPluginExport {
name: export.name.clone(),
signature: export.signature.clone(),
descriptor,
});
}
set.plugins.push(StaticPlugin {
manifest_path,
base_dir,
package: manifest.plugin.name.clone(),
version: manifest.plugin.version.clone(),
kind: manifest.plugin.kind,
manifest_sha256,
artifact_target: host.triple().to_string(),
artifact_sha256: artifact.sha256.clone(),
artifact_relative,
exports,
requires: manifest.plugin.requires.clone(),
});
}
set.plugins.sort_by(|left, right| {
(&left.package, &left.manifest_sha256).cmp(&(&right.package, &right.manifest_sha256))
});
Ok(set)
}
/// Convert a plugin manifest signature (`($a: Int, $b: Int) -> Int`) to the
/// language parameter list.
///
/// ABI v1 transports named arguments as a map, so positional arguments are a
/// static-validated convenience: the binding pass normalises them to the
/// manifest order. Default values are rejected until the ABI wire semantics for
/// them are versioned (RFC-EXT-0001 §4.5.3); silently injecting defaults at run
/// time is forbidden.
pub fn plugin_signature(raw: &str) -> Result<CallableSignature, String> {
let before_return = raw.split("->").next().unwrap_or(raw).trim();
let open = before_return
.find('(')
.ok_or_else(|| "missing `(`".to_string())?;
let close = before_return
.rfind(')')
.ok_or_else(|| "missing `)`".to_string())?;
if close < open {
return Err("malformed parameter list".to_string());
}
let params = before_return[open + 1..close].replace('$', "");
let parsed = marqdo_frontend::signature::parse(&format!("({params})"), Span::default());
if parsed.has_errors() {
return Err(parsed
.diagnostics
.iter()
.map(|diagnostic| diagnostic.message.clone())
.collect::<Vec<_>>()
.join("; "));
}
if parsed.params.iter().any(|param| param.default.is_some()) {
return Err(
"ABI v1 plugin signatures must not declare default values; make parameters required"
.to_string(),
);
}
Ok(CallableSignature {
params: parsed.params,
kind: SignatureKind::Explicit,
})
}
/// Reject absolute paths, `..` components and empty paths.
pub fn ensure_bundle_relative(path: &str) -> Result<(), String> {
let mut normalised = PathBuf::new();
for component in Path::new(path).components() {
match component {
Component::Normal(part) => normalised.push(part),
Component::CurDir => {}
Component::ParentDir | Component::RootDir | Component::Prefix(_) => {
return Err("must be a relative path inside the package".to_string())
}
}
}
if normalised.as_os_str().is_empty() {
return Err("must not be empty".to_string());
}
Ok(())
}
/// Verify a bundle file is a regular file below `bundle_root`, travelling
/// through symlinks only if the final canonical target stays inside the bundle.
pub fn verify_bundle_regular_file(
bundle_root: &Path,
relative: &str,
) -> Result<PathBuf, PrepareError> {
ensure_bundle_relative(relative).map_err(|error| {
PrepareError::load(
"ext.artifact_integrity",
format!("artifact path `{relative}` is invalid: {error}"),
)
})?;
let root = bundle_root.canonicalize().map_err(|error| {
PrepareError::load(
"ext.artifact_integrity",
format!(
"cannot resolve package root `{}`: {error}",
bundle_root.display()
),
)
})?;
let candidate = root.join(relative);
let canonical = candidate.canonicalize().map_err(|error| {
PrepareError::load(
"ext.artifact_integrity",
format!("cannot open artifact `{}`: {error}", candidate.display()),
)
})?;
if !canonical.starts_with(&root) {
return Err(PrepareError::load(
"ext.artifact_integrity",
format!(
"artifact `{relative}` resolves outside its package bundle (`{}`)",
canonical.display()
),
));
}
let metadata = std::fs::metadata(&canonical).map_err(|error| {
PrepareError::load(
"ext.artifact_integrity",
format!("cannot stat artifact `{}`: {error}", canonical.display()),
)
})?;
if !metadata.is_file() {
return Err(PrepareError::load(
"ext.artifact_integrity",
format!("artifact `{}` is not a regular file", canonical.display()),
));
}
Ok(canonical)
}
/// Verify a file's lowercase sha256 against a declared value.
pub fn verify_sha256(path: &Path, declared: &str) -> Result<(), PrepareError> {
let bytes = std::fs::read(path).map_err(|error| {
PrepareError::load(
"ext.artifact_integrity",
format!("cannot read artifact `{}`: {error}", path.display()),
)
})?;
let actual = hex(&Sha256::digest(&bytes));
if actual != declared.to_ascii_lowercase() {
return Err(PrepareError::load(
"ext.artifact_integrity",
format!(
"artifact `{}` sha256 mismatch: declared {declared}, actual {actual}",
path.display()
),
));
}
Ok(())
}
fn hex(bytes: &[u8]) -> String {
let mut out = String::with_capacity(bytes.len() * 2);
for byte in bytes {
out.push_str(&format!("{byte:02x}"));
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn plugin_signature_rejects_defaults_and_parses_positional() {
assert!(plugin_signature("($a: Int, $b: Int) -> Int").is_ok());
let error = plugin_signature("($a: Int = 1) -> Int").unwrap_err();
assert!(error.contains("default"), "{error}");
}
#[test]
fn bundle_relative_paths_reject_traversal() {
assert!(ensure_bundle_relative("lib/demo.so").is_ok());
assert!(ensure_bundle_relative("../demo.so").is_err());
assert!(ensure_bundle_relative("/etc/passwd").is_err());
assert!(ensure_bundle_relative("").is_err());
}
}
+171 -72
View File
@@ -14,7 +14,8 @@ use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use marqdo_expression::Expr;
use marqdo_frontend::{analyze_source, AnalysisResult};
use marqdo_frontend::AnalysisResult;
use marqdo_ir::callable::OfficialAlias;
use marqdo_ir::{ModuleId, PlaceTarget, Program, ResolvedImport, Stmt};
use marqdo_package::{
discover_root, LockFile, Manifest, PackageResolver, ResolvedPackage, LOCK_FILE, MANIFEST_FILE,
@@ -157,6 +158,11 @@ pub struct LoadedProject {
pub diagnostics: Vec<FileDiagnostic>,
/// Verified bundled stdlib identity, when one was available.
pub stdlib: Option<StdlibInfo>,
/// Official stdlib public aliases referenced by this program (ADR-0058).
///
/// These are ordinary `Marqdo` bindings; the runtime has no print special
/// case and no `Program.print_override`.
pub official_aliases: Vec<marqdo_ir::callable::OfficialAlias>,
}
impl LoadedProject {
@@ -280,32 +286,11 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
let entry_id = loader.load_module(&entry_module.package, &entry_module.relative_path)?;
loader.program.main_module = Some(entry_id.clone());
// Compat-B (ADR-0030): only programs that actually call 打印/print need the
// stdlib print module; injecting it for every program would change the
// module graph, analysis output and startup cost for no benefit.
if program_uses_print(&loader.program) {
if let Some(stdlib) = loader.stdlib.clone() {
let package = stdlib.package();
let io = loader.load_module(&package, "io.mq.md")?;
let Some(module) = loader.program.module(&io) else {
return Err("stdlib print module was not loaded".to_string());
};
let Some(function) = module.function("写值") else {
return Err("stdlib print module has no `写值` function".to_string());
};
loader.program.print_override = Some(function.id.clone());
}
}
for (module_id, diagnostic) in marqdo_frontend::calls::validate_program(&loader.program) {
if let Some(file) = loader.files.get(&module_id) {
loader.diagnostics.push(FileDiagnostic {
module_id,
path: file.path.clone(),
source: file.source.clone(),
diagnostic,
});
}
}
// Official stdlib public aliases (ADR-0058): the verified metadata names
// the target module/function; only aliases the program actually references
// load their module, and the runtime binds them as ordinary Marqdo calls.
let official_aliases = loader.load_official_aliases()?;
let stdlib_info = loader.stdlib.as_ref().map(|bundle| StdlibInfo {
version: bundle.version().to_string(),
digest: bundle.digest().to_string(),
@@ -328,6 +313,7 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
packages: loader.packages,
diagnostics: loader.diagnostics,
stdlib: stdlib_info,
official_aliases,
})
}
@@ -378,7 +364,7 @@ impl Loader {
.or_insert_with(|| package_root.clone());
let source = std::fs::read_to_string(&path)
.map_err(|error| format!("cannot read `{}`: {error}", path.display()))?;
let analysis = analyze_source(&module_id.to_string(), &source);
let analysis = marqdo_frontend::analyze_source_for_program(&module_id.to_string(), &source);
let imports: Vec<(String, String, marqdo_span::Span)> = analysis
.document
.frontmatter()
@@ -516,57 +502,170 @@ impl Loader {
}
Ok(module_id)
}
}
/// Whether the loaded program calls the legacy `打印`/`print` builtin.
fn program_uses_print(program: &Program) -> bool {
program.modules.values().any(|module| {
module
.functions
.iter()
.any(|function| stmts_use_print(&function.body))
})
}
fn stmts_use_print(stmts: &[Stmt]) -> bool {
stmts.iter().any(|stmt| match stmt {
Stmt::Assign { target, value, .. } => {
let target_uses = match target {
PlaceTarget::Local { .. } => false,
PlaceTarget::Index { base, index, .. } => {
expr_uses_print(base) || expr_uses_print(index)
}
/// Resolve the official stdlib public aliases referenced by the program.
///
/// Only the verified `stdlib.json` metadata can declare an official alias;
/// a third-party package cannot claim `打印`/`print` or `std.*`. Modules are
/// loaded only when the program references one of the alias names, so an
/// ordinary program does not grow its module graph or startup cost.
fn load_official_aliases(&mut self) -> Result<Vec<OfficialAlias>, String> {
let Some(stdlib) = self.stdlib.clone() else {
return Ok(Vec::new());
};
if stdlib.public_aliases().is_empty() {
return Ok(Vec::new());
}
let referenced = referenced_identifiers(&self.program);
let mut aliases = Vec::new();
for alias in stdlib.public_aliases() {
if !alias
.public_names
.iter()
.any(|name| referenced.contains(name))
{
continue;
}
let resolved = stdlib.resolve(&alias.target_module)?;
let module_id = self.load_module(&resolved.package, &resolved.relative_path)?;
let Some(module) = self.program.module(&module_id) else {
return Err(format!(
"stdlib public alias target module `{}` was not loaded",
alias.target_module
));
};
target_uses || expr_uses_print(value)
let Some(function) = module.function(&alias.target_function) else {
return Err(format!(
"stdlib public alias `{}` target `{}::{}` does not exist",
alias.public_names.join("/"),
alias.target_module,
alias.target_function
));
};
if let Some(expected) = &alias.expected_signature {
let parsed =
marqdo_frontend::signature::parse(expected, marqdo_span::Span::default());
if parsed.has_errors() {
return Err(format!(
"stdlib public alias `{}` has an invalid expected_signature `{expected}`",
alias.public_names.join("/")
));
}
let expected_names: Vec<&str> = parsed
.params
.iter()
.map(|param| param.name.as_str())
.collect();
let actual_names: Vec<&str> = function
.params
.iter()
.map(|param| param.name.as_str())
.collect();
if expected_names != actual_names
|| function.signature != marqdo_ir::SignatureKind::Explicit
{
return Err(format!(
"stdlib public alias `{}` expected signature `{expected}` does not match \
target `{}::{}` (params {:?}, kind {:?})",
alias.public_names.join("/"),
alias.target_module,
alias.target_function,
actual_names,
function.signature
));
}
}
aliases.push(OfficialAlias {
public_names: alias.public_names.clone(),
function: function.id.clone(),
signature: marqdo_ir::callable::CallableSignature {
params: function.params.clone(),
kind: function.signature,
},
owner_package: module_id.instance.clone(),
effects_requested: alias.effects.clone(),
});
}
Stmt::Eval { expr, .. } | Stmt::Return { value: expr, .. } => expr_uses_print(expr),
Stmt::Branch { arms, .. } => arms
.iter()
.any(|arm| expr_uses_print(&arm.condition) || stmts_use_print(&arm.body)),
Stmt::Loop {
condition, body, ..
} => expr_uses_print(condition) || stmts_use_print(body),
})
Ok(aliases)
}
}
/// Collect every identifier referenced by a program.
///
/// This is a conservative "is this alias used at all?" scan; it only decides
/// whether an official alias target module should be loaded. It does not decide
/// call targets (that is the single binding pass).
fn referenced_identifiers(program: &Program) -> std::collections::BTreeSet<String> {
let mut names = std::collections::BTreeSet::new();
for module in program.modules.values() {
for function in &module.functions {
for param in &function.params {
if let Some(default) = &param.default {
collect_identifiers_expr(default, &mut names);
}
}
collect_identifiers_stmts(&function.body, &mut names);
}
}
names
}
fn collect_identifiers_stmts(stmts: &[Stmt], names: &mut std::collections::BTreeSet<String>) {
for stmt in stmts {
match stmt {
Stmt::Assign { target, value, .. } => {
if let PlaceTarget::Index { base, index, .. } = target {
collect_identifiers_expr(base, names);
collect_identifiers_expr(index, names);
}
collect_identifiers_expr(value, names);
}
Stmt::Eval { expr, .. } | Stmt::Return { value: expr, .. } => {
collect_identifiers_expr(expr, names)
}
Stmt::Branch { arms, .. } => {
for arm in arms {
collect_identifiers_expr(&arm.condition, names);
collect_identifiers_stmts(&arm.body, names);
}
}
Stmt::Loop {
condition, body, ..
} => {
collect_identifiers_expr(condition, names);
collect_identifiers_stmts(body, names);
}
}
}
}
fn expr_uses_print(expr: &Expr) -> bool {
fn collect_identifiers_expr(expr: &Expr, names: &mut std::collections::BTreeSet<String>) {
match expr {
Expr::Ident(ident) => {
names.insert(ident.name.clone());
}
Expr::List(items) => items
.iter()
.for_each(|item| collect_identifiers_expr(item, names)),
Expr::Record(entries) => entries
.iter()
.for_each(|entry| collect_identifiers_expr(&entry.value, names)),
Expr::Unary { operand, .. } => collect_identifiers_expr(operand, names),
Expr::Binary { left, right, .. } => {
collect_identifiers_expr(left, names);
collect_identifiers_expr(right, names);
}
Expr::Member { base, .. } => collect_identifiers_expr(base, names),
Expr::Index { base, index, .. } => {
collect_identifiers_expr(base, names);
collect_identifiers_expr(index, names);
}
Expr::Call { callee, args, .. } => {
matches!(callee.as_ref(), Expr::Ident(ident) if matches!(ident.name.as_str(), "打印" | "print"))
|| expr_uses_print(callee)
|| args.iter().any(|arg| expr_uses_print(&arg.value))
collect_identifiers_expr(callee, names);
for arg in args {
collect_identifiers_expr(&arg.value, names);
}
}
Expr::List(items) => items.iter().any(expr_uses_print),
Expr::Record(entries) => entries.iter().any(|entry| expr_uses_print(&entry.value)),
Expr::Unary { operand, .. } => expr_uses_print(operand),
Expr::Binary { left, right, .. } => expr_uses_print(left) || expr_uses_print(right),
Expr::Member { base, .. } => expr_uses_print(base),
Expr::Index { base, index, .. } => expr_uses_print(base) || expr_uses_print(index),
Expr::Int(_)
| Expr::Float(_)
| Expr::Text(_)
| Expr::Bool(_)
| Expr::None
| Expr::Ident(_) => false,
Expr::Int(_) | Expr::Float(_) | Expr::Text(_) | Expr::Bool(_) | Expr::None => {}
}
}
+15
View File
@@ -7,8 +7,11 @@
mod analysis;
mod args;
mod commands;
mod extension_descriptors;
mod loader;
mod output;
mod project_prepare;
mod provider_manifest;
mod stdlib;
mod tool;
mod trace;
@@ -128,6 +131,16 @@ struct SandboxArgs {
/// Explicitly trusted plugin manifest/directory to load (repeatable).
#[arg(long = "plugin", value_name = "PATH")]
plugin: Vec<PathBuf>,
/// Explicitly selected resource provider v2 manifest (repeatable).
///
/// Selecting a provider never authorizes it; `--allow-provider NAME` is
/// required as well. `check`/`inspect` do not load providers.
#[arg(long = "provider", value_name = "PATH")]
provider: Vec<PathBuf>,
/// Grant one selected resource provider by its exact manifest name
/// (repeatable; wildcards are not supported).
#[arg(long = "allow-provider", value_name = "NAME")]
allow_provider: Vec<String>,
/// Write a versioned Trace v2 file for this run.
#[arg(long = "trace-file", value_name = "PATH")]
trace_file: Option<PathBuf>,
@@ -164,6 +177,8 @@ impl SandboxArgs {
max_region_memory: self.max_region_memory,
timeout: self.timeout,
plugins: self.plugin.clone(),
providers: self.provider.clone(),
allow_provider: self.allow_provider.clone(),
trust_native_plugin: self.trust_native_plugin,
trace_file: self.trace_file.clone(),
trace_max_events: self.trace_max_events,
+138
View File
@@ -0,0 +1,138 @@
//! The single `prepare` composition-root step (RFC-EXT-0001 §4.4, §18.2).
//!
//! Order of operations (T1–T5 of the RFC):
//!
//! 1. load the Markdown/package graph (read only, no execution);
//! 2. read static extension descriptors (never `dlopen`/spawn);
//! 3. build the unique [`CallableRegistry`] and reject static conflicts;
//! 4. run the single program-wide call binding pass;
//! 5. produce a [`PreparedProgram`] consumed by `check`, `run`, `call`,
//! `inspect` and Analysis.
//!
//! `run` may only assemble the Host and start workers after this returns.
use std::path::PathBuf;
use marqdo_ir::callable::CallableRegistry;
use crate::extension_descriptors::{read_plugin_descriptors, PrepareError, StaticPluginSet};
use crate::loader::{self, FileDiagnostic, LoadedProject};
/// Options that influence static preparation.
#[derive(Debug, Clone, Default)]
pub struct PrepareOptions {
/// Explicit `--plugin` manifests (repeatable, no implicit discovery).
pub plugins: Vec<PathBuf>,
}
/// A statically prepared program: everything `check`/`run`/`inspect` share.
pub struct PreparedProgram {
/// Loaded project graph with authoritative diagnostics.
pub loaded: LoadedProject,
/// Static plugin descriptors (identity only; no binary loaded).
pub plugins: StaticPluginSet,
/// Unique callable registry used by the binding pass.
///
/// Kept on the prepared value so later consumers (`inspect --extensions`,
/// evidence) can report the same static table without rebuilding it.
#[allow(dead_code)]
pub registry: CallableRegistry,
}
/// Prepare one document without executing any project or extension code.
pub fn prepare(
file: Option<&std::path::Path>,
options: &PrepareOptions,
) -> Result<PreparedProgram, PrepareError> {
// T1: static module/package loading. No call diagnostics are produced by
// the loader; the single binding pass below owns them.
let mut loaded = loader::load(file.unwrap_or_else(|| std::path::Path::new("index.mq.md")))
.map_err(|message| default_load_error(&message))?;
// T2: extension descriptors are read from manifests only.
let plugins = read_plugin_descriptors(&options.plugins)?;
// T3: unique static table. Duplicate exports/builtin shadowing are rejected
// before any call site can be bound.
for descriptor in plugins.descriptors() {
if loaded
.program
.external_callable(&descriptor.public_name)
.is_some()
{
return Err(PrepareError::option(
"ext.duplicate_callable",
format!(
"plugin export `{}` is declared more than once",
descriptor.public_name
),
));
}
if marqdo_ir::builtins::find(&descriptor.public_name).is_some() {
return Err(PrepareError::option(
"ext.duplicate_callable",
format!(
"plugin export `{}` conflicts with a builtin; plugins may not shadow builtins",
descriptor.public_name
),
));
}
loaded.program.external_callables.push(descriptor);
}
let mut registry = CallableRegistry::try_from_program(&loaded.program).map_err(|conflict| {
PrepareError::option(
"ext.duplicate_callable",
format!("static callable table conflict: {conflict}"),
)
})?;
// Official stdlib public aliases (ADR-0058) are ordinary Marqdo bindings.
for alias in &loaded.official_aliases {
registry
.insert_official_alias(alias.clone())
.map_err(|error| {
PrepareError::option(
"ext.duplicate_callable",
format!("stdlib public alias: {error}"),
)
})?;
}
// T4: exactly one call binding pass for the whole program.
let bound = marqdo_frontend::calls::bind_program_calls(&loaded.program, &registry);
for (module_id, diagnostic) in bound.diagnostics {
let Some(file) = loaded.files.get(&module_id) else {
continue;
};
let (path, source) = (file.path.clone(), file.source.clone());
let duplicate = loaded.diagnostics.iter().any(|existing| {
existing.module_id == module_id
&& existing.diagnostic.code == diagnostic.code
&& existing.diagnostic.span == diagnostic.span
&& existing.diagnostic.message == diagnostic.message
});
if !duplicate {
loaded.diagnostics.push(FileDiagnostic {
module_id,
path,
source,
diagnostic,
});
}
}
loaded.program.set_call_bindings(bound.bindings);
Ok(PreparedProgram {
loaded,
plugins,
registry,
})
}
fn default_load_error(message: &str) -> PrepareError {
if message.contains("marqdo.lock") || message.contains("lockfile") {
PrepareError::load("project.lock", message)
} else {
PrepareError::option("project.load", message)
}
}
+952
View File
@@ -0,0 +1,952 @@
//! Strict third-party Resource Provider v2 manifests for the CLI composition
//! root (RFC-EXT-0001 §5.3, ADR-0055).
//!
//! This module *reads* `provider.json` schema version 2 and verifies the
//! selected artifact. It never `dlopen`s a library, never spawns a helper and
//! never authorizes a provider: parsing a manifest is not a grant. The v1
//! `marqdo.provider` manifest stays a host-embedding path
//! (`StdHost::register_provider_package`, ADR-0037) and is rejected here with
//! `ext.unsupported_provider_schema`; there is no silent migration.
//!
//! The CLI only accepts the `c-abi-1` + `worker` transport combination and
//! resolves the helper as a sibling of the running `marqdo` executable; PATH
//! lookup is never used. Integrity hashes are content-integrity checks, not
//! publisher authentication.
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Path, PathBuf};
use marqdo_host::StdHost;
use marqdo_plugin_abi::{HostTarget, TargetTriple};
use serde::Deserialize;
use sha2::{Digest, Sha256};
use crate::extension_descriptors::{
ensure_bundle_relative, verify_bundle_regular_file, verify_sha256, PrepareError,
};
/// Provider manifest file name used by the CLI (`--provider` accepts this file).
pub const PROVIDER_MANIFEST_FILE: &str = "provider.json";
/// Provider manifest schema name.
pub const PROVIDER_SCHEMA: &str = "marqdo.provider";
/// Provider manifest schema version accepted by the CLI.
pub const PROVIDER_SCHEMA_VERSION: u64 = 2;
/// Provider ABI accepted by the CLI (ADR-0036/0038).
pub const PROVIDER_ABI: &str = "c-abi-1";
/// Provider transport accepted by the CLI (ADR-0035/0038).
pub const PROVIDER_TRANSPORT: &str = "worker";
/// Helper executable shipped next to the `marqdo` binary (ADR-0038).
pub const PROVIDER_WORKER_BINARY: &str = "marqdo-provider-worker";
/// Upper bounds for a single provider manifest; these bound parser work only.
pub const MAX_PROVIDER_NAME_BYTES: usize = 128;
pub const MAX_PROVIDER_KINDS: usize = 64;
pub const MAX_PROVIDER_ARTIFACTS: usize = 32;
pub const MAX_PROVIDER_PERMISSIONS: usize = 64;
/// Maximum selected artifact size (256 MiB).
pub const MAX_PROVIDER_ARTIFACT_BYTES: u64 = 256 * 1024 * 1024;
/// Maximum manifest size (1 MiB); a manifest is metadata, not an artifact.
pub const MAX_PROVIDER_MANIFEST_BYTES: u64 = 1024 * 1024;
/// Core resource kinds owned by the host; third parties may never override
/// them (RFC-EXT-0001 §5.3.1). `stdio`/`file`/`tcp`/`tls`/`udp`/`tcp-listener`
/// are dispatched by `marqdo-host` before the provider registry is consulted.
pub const RESERVED_RESOURCE_KINDS: &[&str] =
&["stdio", "file", "tcp", "tls", "udp", "tcp-listener"];
/// Raw `provider.json` schema version 2 document.
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct ProviderManifest {
schema: String,
schema_version: u64,
name: String,
version: String,
abi: String,
transport: String,
kinds: Vec<String>,
artifacts: Vec<ManifestArtifact>,
#[serde(default)]
permissions_requested: Vec<String>,
/// Free-form, non-executable metadata. Ignored for execution and never used
/// to select artifacts or grant permissions.
#[serde(rename = "metadata", default)]
_metadata: Option<serde_json::Value>,
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct ManifestArtifact {
target: String,
path: String,
sha256: String,
}
/// A manifest whose selected artifact has been resolved and verified.
///
/// Producing one requires reading the manifest, checking the schema rules,
/// selecting the artifact for the current host target and verifying the
/// content hash. It still does not grant access and no worker has been started.
#[derive(Debug, Clone)]
pub struct VerifiedProvider {
/// Stable grant name (`--allow-provider NAME` must match exactly).
pub name: String,
/// Strict SemVer version declared by the manifest.
pub version: String,
/// Non-reserved resource kinds served by this provider.
pub kinds: Vec<String>,
/// Canonical manifest path.
pub manifest_path: PathBuf,
/// Canonical bundle directory (manifest parent).
pub base_dir: PathBuf,
/// `sha256:<hex>` of the manifest bytes at verification time.
pub manifest_digest: String,
/// Selected transport (`worker` in v2).
pub transport: String,
/// Canonical path of the selected, hash-verified artifact.
pub selected_artifact_path: PathBuf,
/// Bundle-relative artifact path exactly as declared.
pub selected_artifact_relative: String,
/// Lowercase hex sha256 declared by the manifest.
pub selected_artifact_sha256: String,
/// Target triple the selected artifact was built for.
pub selected_target: String,
}
impl VerifiedProvider {
/// Re-verify the manifest bytes and the selected artifact immediately
/// before the provider worker is spawned.
///
/// `run` must never trust a hash it checked earlier: another process can
/// replace the manifest or the artifact between selection and use.
pub fn reverify(&self) -> Result<(), PrepareError> {
let bytes = std::fs::read(&self.manifest_path).map_err(|error| {
PrepareError::load(
"ext.artifact_integrity",
format!(
"provider manifest `{}` could not be re-read before use: {error}",
self.manifest_path.display()
),
)
})?;
if bytes.len() as u64 > MAX_PROVIDER_MANIFEST_BYTES {
return Err(PrepareError::load(
"ext.artifact_integrity",
format!(
"provider manifest `{}` grew beyond the {MAX_PROVIDER_MANIFEST_BYTES}-byte limit before use",
self.manifest_path.display()
),
));
}
let digest = manifest_digest(&bytes);
if digest != self.manifest_digest {
return Err(PrepareError::load(
"ext.artifact_integrity",
format!(
"provider manifest `{}` changed after verification (expected {}, actual {}); refusing to start its worker",
self.manifest_path.display(),
self.manifest_digest,
digest
),
));
}
let artifact =
verify_bundle_regular_file(&self.base_dir, &self.selected_artifact_relative)?;
if artifact != self.selected_artifact_path {
return Err(PrepareError::load(
"ext.artifact_integrity",
format!(
"provider `{}` artifact `{}` now resolves to `{}` instead of the verified `{}`",
self.name,
self.selected_artifact_relative,
artifact.display(),
self.selected_artifact_path.display()
),
));
}
check_artifact_size(&artifact)?;
verify_sha256(&artifact, &self.selected_artifact_sha256)?;
Ok(())
}
}
/// Read, validate and verify every explicitly selected `--provider` manifest.
///
/// No worker is spawned and no artifact is executed. Duplicate provider names
/// and duplicate resource kinds across providers are rejected deterministically
/// (the returned list is sorted by provider name) so a later registration can
/// never overwrite an earlier one.
pub fn read_provider_manifests(paths: &[PathBuf]) -> Result<Vec<VerifiedProvider>, PrepareError> {
let mut manifest_paths: BTreeSet<PathBuf> = BTreeSet::new();
for path in paths {
if path.is_dir() {
return Err(PrepareError::option(
"ext.manifest_unreadable",
format!(
"--provider `{}` must name a `{PROVIDER_MANIFEST_FILE}` file, not a directory",
path.display()
),
));
}
let canonical = path.canonicalize().map_err(|error| {
PrepareError::option(
"ext.manifest_unreadable",
format!(
"cannot open provider manifest `{}`: {error}",
path.display()
),
)
})?;
manifest_paths.insert(canonical);
}
let mut providers = Vec::with_capacity(manifest_paths.len());
for path in manifest_paths {
providers.push(read_provider_manifest(&path)?);
}
// Deterministic conflict resolution independent of CLI argument order.
providers.sort_by(|left, right| {
(&left.name, &left.manifest_digest).cmp(&(&right.name, &right.manifest_digest))
});
let mut unique: Vec<VerifiedProvider> = Vec::with_capacity(providers.len());
for provider in providers {
if let Some(existing) = unique.iter().find(|item| item.name == provider.name) {
if existing.manifest_digest != provider.manifest_digest {
return Err(PrepareError::option(
"ext.duplicate_provider",
format!(
"provider name `{}` is selected with two different manifests (`{}` and `{}`); \
explicit aliases are not implemented, so this is rejected",
provider.name,
existing.manifest_path.display(),
provider.manifest_path.display()
),
));
}
// The exact same declaration selected twice is idempotent.
continue;
}
unique.push(provider);
}
let mut claimed_kinds: BTreeMap<String, String> = BTreeMap::new();
for provider in &unique {
for kind in &provider.kinds {
if let Some(owner) = claimed_kinds.get(kind) {
return Err(PrepareError::option(
"ext.duplicate_provider_kind",
format!(
"resource kind `{kind}` is declared by both provider `{owner}` and provider `{}`; \
duplicate kinds are rejected before any worker starts",
provider.name
),
));
}
claimed_kinds.insert(kind.clone(), provider.name.clone());
}
}
Ok(unique)
}
fn read_provider_manifest(canonical_path: &Path) -> Result<VerifiedProvider, PrepareError> {
let metadata = std::fs::metadata(canonical_path).map_err(|error| {
PrepareError::option(
"ext.manifest_unreadable",
format!(
"cannot stat provider manifest `{}`: {error}",
canonical_path.display()
),
)
})?;
if metadata.len() > MAX_PROVIDER_MANIFEST_BYTES {
return Err(PrepareError::option(
"ext.manifest_invalid",
format!(
"provider manifest `{}` is {} bytes, larger than the {MAX_PROVIDER_MANIFEST_BYTES}-byte limit",
canonical_path.display(),
metadata.len()
),
));
}
let bytes = std::fs::read(canonical_path).map_err(|error| {
PrepareError::option(
"ext.manifest_unreadable",
format!(
"cannot read provider manifest `{}`: {error}",
canonical_path.display()
),
)
})?;
let digest = manifest_digest(&bytes);
let value: serde_json::Value = serde_json::from_slice(&bytes).map_err(|error| {
PrepareError::option(
"ext.manifest_invalid",
format!(
"provider manifest `{}` is not valid UTF-8 JSON: {error}",
canonical_path.display()
),
)
})?;
// Schema/version gate runs before structural parsing so a v1 (or future)
// manifest gets the stable `ext.unsupported_provider_schema` code instead
// of a generic field error.
let schema = value.get("schema").and_then(serde_json::Value::as_str);
if schema != Some(PROVIDER_SCHEMA) {
return Err(PrepareError::option(
"ext.unsupported_provider_schema",
format!(
"provider manifest `{}` does not declare `schema: \"{PROVIDER_SCHEMA}\"`; \
the CLI accepts schema version {PROVIDER_SCHEMA_VERSION} only",
canonical_path.display()
),
));
}
let schema_version = value
.get("schema_version")
.and_then(serde_json::Value::as_u64);
if schema_version != Some(PROVIDER_SCHEMA_VERSION) {
return Err(PrepareError::option(
"ext.unsupported_provider_schema",
format!(
"provider manifest `{}` declares schema_version {}; the CLI accepts {PROVIDER_SCHEMA_VERSION} only \
(v1 and future versions are rejected, there is no silent migration)",
canonical_path.display(),
schema_version
.map(|version| version.to_string())
.unwrap_or_else(|| "<missing>".to_string())
),
));
}
let manifest: ProviderManifest = serde_json::from_value(value).map_err(|error| {
PrepareError::option(
"ext.manifest_invalid",
format!(
"provider manifest `{}` is not a valid {PROVIDER_SCHEMA} v{PROVIDER_SCHEMA_VERSION} document: {error}",
canonical_path.display()
),
)
})?;
if manifest.schema != PROVIDER_SCHEMA || manifest.schema_version != PROVIDER_SCHEMA_VERSION {
// Unreachable after the Value-level gate; kept fail-closed.
return Err(PrepareError::option(
"ext.unsupported_provider_schema",
format!(
"provider manifest `{}` uses an unsupported schema",
canonical_path.display()
),
));
}
validate_manifest(&manifest, canonical_path)?;
let base_dir = canonical_path
.parent()
.map(Path::to_path_buf)
.unwrap_or_else(|| PathBuf::from("."));
let selected_target = HostTarget::current().triple().to_string();
let artifact = manifest
.artifacts
.iter()
.find(|artifact| artifact.target == selected_target)
.ok_or_else(|| {
PrepareError::option(
"ext.no_matching_artifact",
format!(
"provider `{}` has no artifact for the current target `{selected_target}`; \
other platforms are never substituted",
manifest.name
),
)
})?;
let artifact_relative = normalise_artifact_path(&manifest.name, &artifact.path)?;
let artifact_path = verify_bundle_regular_file(&base_dir, &artifact_relative)?;
check_artifact_size(&artifact_path)?;
verify_sha256(&artifact_path, &artifact.sha256)?;
Ok(VerifiedProvider {
name: manifest.name,
version: manifest.version,
kinds: manifest.kinds,
manifest_path: canonical_path.to_path_buf(),
base_dir,
manifest_digest: digest,
transport: manifest.transport,
selected_artifact_path: artifact_path,
selected_artifact_relative: artifact_relative,
selected_artifact_sha256: artifact.sha256.to_ascii_lowercase(),
selected_target,
})
}
fn validate_manifest(
manifest: &ProviderManifest,
manifest_path: &Path,
) -> Result<(), PrepareError> {
let invalid = |message: String| {
PrepareError::option(
"ext.manifest_invalid",
format!("provider manifest `{}`: {message}", manifest_path.display()),
)
};
validate_provider_name(&manifest.name)
.map_err(|error| invalid(format!("invalid provider name: {error}")))?;
if RESERVED_RESOURCE_KINDS.contains(&manifest.name.as_str()) {
return Err(invalid(format!(
"provider name `{}` is a reserved core resource kind",
manifest.name
)));
}
semver::Version::parse(&manifest.version).map_err(|error| {
invalid(format!(
"version `{}` is not strict SemVer: {error}",
manifest.version
))
})?;
if manifest.abi != PROVIDER_ABI {
return Err(PrepareError::option(
"ext.unsupported_provider_schema",
format!(
"provider `{}` declares abi `{}`; schema v{PROVIDER_SCHEMA_VERSION} accepts `{PROVIDER_ABI}` only",
manifest.name, manifest.abi
),
));
}
if manifest.transport != PROVIDER_TRANSPORT {
return Err(PrepareError::option(
"ext.unsupported_provider_schema",
format!(
"provider `{}` declares transport `{}`; schema v{PROVIDER_SCHEMA_VERSION} accepts `{PROVIDER_TRANSPORT}` only",
manifest.name, manifest.transport
),
));
}
if manifest.kinds.is_empty() {
return Err(invalid(
"kinds must declare at least one resource kind".to_string(),
));
}
if manifest.kinds.len() > MAX_PROVIDER_KINDS {
return Err(invalid(format!(
"kinds declares more than {MAX_PROVIDER_KINDS} entries"
)));
}
let mut kinds = BTreeSet::new();
for kind in &manifest.kinds {
validate_provider_name(kind)
.map_err(|error| invalid(format!("invalid resource kind `{kind}`: {error}")))?;
if RESERVED_RESOURCE_KINDS.contains(&kind.as_str()) {
return Err(invalid(format!(
"resource kind `{kind}` is reserved by the host and cannot be overridden by a provider"
)));
}
if !kinds.insert(kind.clone()) {
return Err(invalid(format!(
"resource kind `{kind}` is declared more than once"
)));
}
}
if manifest.artifacts.is_empty() {
return Err(invalid(
"artifacts must declare at least one target artifact".to_string(),
));
}
if manifest.artifacts.len() > MAX_PROVIDER_ARTIFACTS {
return Err(invalid(format!(
"artifacts declares more than {MAX_PROVIDER_ARTIFACTS} entries"
)));
}
let mut targets = BTreeSet::new();
for artifact in &manifest.artifacts {
let triple = TargetTriple::parse(&artifact.target).map_err(|error| {
invalid(format!(
"artifact target `{}` is not a target triple: {error}",
artifact.target
))
})?;
if !targets.insert(triple.to_string()) {
return Err(invalid(format!(
"artifact target `{triple}` is declared more than once"
)));
}
if !is_lowercase_sha256(&artifact.sha256) {
return Err(invalid(format!(
"artifact for target `{:?}` must declare a 64-character lowercase hex sha256",
artifact.target
)));
}
}
if manifest.permissions_requested.len() > MAX_PROVIDER_PERMISSIONS {
return Err(invalid(format!(
"permissions_requested declares more than {MAX_PROVIDER_PERMISSIONS} entries"
)));
}
let mut permissions = BTreeSet::new();
for permission in &manifest.permissions_requested {
if !is_known_capability(permission) && !is_vendor_capability(permission) {
return Err(invalid(format!(
"permissions_requested entry `{permission}` is not a known capability or a dotted vendor capability"
)));
}
if !permissions.insert(permission.clone()) {
return Err(invalid(format!(
"permission `{permission}` is declared more than once"
)));
}
}
// `metadata` is intentionally not validated: it has no execution meaning.
Ok(())
}
/// Validate a dotted provider/kind name using the plugin name grammar
/// (ASCII lowercase segment start, then lowercase/digits/`-`/`_`).
pub fn validate_provider_name(name: &str) -> Result<(), String> {
if name.is_empty() {
return Err("must not be empty".to_string());
}
if name.len() > MAX_PROVIDER_NAME_BYTES {
return Err(format!(
"must not be longer than {MAX_PROVIDER_NAME_BYTES} bytes"
));
}
for segment in name.split('.') {
if segment.is_empty() {
return Err(format!("`{name}` contains an empty segment"));
}
let mut bytes = segment.bytes();
let first = bytes.next().unwrap_or(b'a');
if !first.is_ascii_lowercase() {
return Err(format!(
"`{name}` must start every `.`-separated segment with a lowercase ASCII letter"
));
}
if !bytes.all(|byte| {
byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'-' || byte == b'_'
}) {
return Err(format!(
"`{name}` may only contain lowercase ASCII letters, digits, `-` and `_`"
));
}
}
Ok(())
}
/// Reject absolute paths and `..` traversal in declared artifact paths.
fn normalise_artifact_path(provider: &str, path: &str) -> Result<String, PrepareError> {
ensure_bundle_relative(path).map_err(|error| {
PrepareError::load(
"ext.artifact_integrity",
format!("provider `{provider}` artifact path `{path}` is invalid: {error}"),
)
})?;
Ok(path.to_string())
}
fn check_artifact_size(path: &Path) -> Result<(), PrepareError> {
let metadata = std::fs::metadata(path).map_err(|error| {
PrepareError::load(
"ext.artifact_integrity",
format!(
"cannot stat provider artifact `{}`: {error}",
path.display()
),
)
})?;
if metadata.len() > MAX_PROVIDER_ARTIFACT_BYTES {
return Err(PrepareError::load(
"ext.artifact_integrity",
format!(
"provider artifact `{}` is {} bytes, larger than the {MAX_PROVIDER_ARTIFACT_BYTES}-byte limit",
path.display(),
metadata.len()
),
));
}
Ok(())
}
fn is_lowercase_sha256(value: &str) -> bool {
value.len() == 64
&& value
.bytes()
.all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte))
}
fn is_known_capability(name: &str) -> bool {
marqdo_plugin_abi::KNOWN_CAPABILITIES.contains(&name)
}
/// A dotted vendor capability proposal (validation only, never a grant).
fn is_vendor_capability(name: &str) -> bool {
let segments: Vec<&str> = name.split('.').collect();
if segments.len() < 3 {
return false;
}
if matches!(
segments[0],
"console" | "math" | "fs" | "net" | "process" | "log" | "clock" | "random"
) {
return false;
}
segments.iter().all(|segment| {
!segment.is_empty()
&& segment.bytes().enumerate().all(|(index, byte)| {
byte.is_ascii_lowercase() || byte.is_ascii_digit() || (byte == b'-' && index > 0)
})
})
}
fn manifest_digest(bytes: &[u8]) -> String {
let mut hasher = Sha256::new();
hasher.update(bytes);
let digest = hasher.finalize();
let mut hex = String::with_capacity(digest.len() * 2);
for byte in digest {
hex.push_str(&format!("{byte:02x}"));
}
format!("sha256:{hex}")
}
/// Whether the provider `worker` transport is supported on this build.
///
/// RFC-EXT-0001 §5.6 makes Linux x86_64 the mandatory lane. Other targets stay
/// compilable but must fail closed with `ext.platform_unsupported` instead of
/// pretending a helper works.
pub fn provider_worker_platform_supported() -> bool {
cfg!(all(target_os = "linux", target_arch = "x86_64"))
}
/// Resolve the provider helper shipped next to the running `marqdo` executable.
///
/// Resolution is *only* the sibling file: there is no PATH lookup and no
/// directory search (RFC-EXT-0001 §5.4.5). A missing helper is
/// `ext.worker_unavailable`, never a fallback to in-process native loading.
pub fn provider_worker_executable() -> Result<PathBuf, PrepareError> {
if !provider_worker_platform_supported() {
return Err(PrepareError::load(
"ext.platform_unsupported",
format!(
"the resource provider worker transport is supported on linux/x86_64 only in this release; \
this build targets {}/{}",
std::env::consts::OS,
std::env::consts::ARCH
),
));
}
let executable = std::env::current_exe().map_err(|error| {
PrepareError::load(
"ext.worker_unavailable",
format!(
"cannot locate the marqdo executable to find `{PROVIDER_WORKER_BINARY}`: {error}"
),
)
})?;
let directory = executable.parent().ok_or_else(|| {
PrepareError::load(
"ext.worker_unavailable",
format!(
"the marqdo executable `{}` has no parent directory",
executable.display()
),
)
})?;
let helper = directory.join(format!(
"{PROVIDER_WORKER_BINARY}{}",
std::env::consts::EXE_SUFFIX
));
match std::fs::metadata(&helper) {
Ok(metadata) if metadata.is_file() => Ok(helper),
_ => Err(PrepareError::load(
"ext.worker_unavailable",
format!(
"the resource provider worker helper `{}` is missing; it must be shipped next to the \
marqdo executable (PATH lookup is not allowed)",
helper.display()
),
)),
}
}
/// Check the exact `--allow-provider` grant, re-verify the manifest and
/// artifact, then register the existing worker resource-provider transport.
///
/// The grant check is repeated here (it was already checked during pre-spawn
/// verification) so no caller can register an ungranted provider. The provider
/// is always registered through
/// [`StdHost::register_worker_resource_provider`]; this path never falls back
/// to the trusted in-process C ABI or a native library.
pub fn load_resource_provider_into_host(
host: &mut StdHost,
provider: &VerifiedProvider,
allow_provider: &[String],
) -> Result<(), PrepareError> {
if !allow_provider.iter().any(|grant| grant == &provider.name) {
return Err(PrepareError::load(
"permission.denied",
format!(
"resource provider `{}` was selected but not granted; add --allow-provider {}",
provider.name, provider.name
),
));
}
if provider.transport != PROVIDER_TRANSPORT {
return Err(PrepareError::option(
"ext.unsupported_provider_schema",
format!(
"provider `{}` transport `{}` cannot be loaded by the CLI",
provider.name, provider.transport
),
));
}
// Re-verify immediately before use; never trust an earlier hash.
provider.reverify()?;
let worker = provider_worker_executable()?;
let kinds: Vec<&str> = provider.kinds.iter().map(String::as_str).collect();
let args = vec![provider
.selected_artifact_path
.to_string_lossy()
.into_owned()];
host.register_worker_resource_provider(&provider.name, &kinds, &worker, &args)
.map_err(|error| {
PrepareError::load(
"ext.worker_unavailable",
format!(
"cannot start the worker for resource provider `{}` v{} (target {}) using `{}`: {}",
provider.name,
provider.version,
provider.selected_target,
worker.display(),
error.message
),
)
})
}
#[cfg(test)]
mod tests {
use super::*;
fn tempdir(tag: &str) -> PathBuf {
let path = std::env::temp_dir().join(format!(
"marqdo-provider-manifest-{}-{tag}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|duration| duration.as_nanos())
.unwrap_or(0)
));
std::fs::create_dir_all(&path).unwrap();
path
}
fn write_manifest(dir: &Path, manifest: &str) -> PathBuf {
let path = dir.join(PROVIDER_MANIFEST_FILE);
std::fs::write(&path, manifest).unwrap();
path
}
fn artifact(dir: &Path, name: &str, bytes: &[u8]) -> (String, String) {
let path = dir.join(name);
std::fs::write(&path, bytes).unwrap();
let mut hasher = Sha256::new();
hasher.update(bytes);
let mut hex = String::new();
for byte in hasher.finalize() {
hex.push_str(&format!("{byte:02x}"));
}
(name.to_string(), hex)
}
fn manifest_text(name: &str, kinds: &[&str], path: &str, sha: &str) -> String {
let kinds = kinds
.iter()
.map(|kind| format!("\"{kind}\""))
.collect::<Vec<_>>()
.join(", ");
format!(
r#"{{
"schema": "marqdo.provider",
"schema_version": 2,
"name": "{name}",
"version": "0.1.0",
"abi": "c-abi-1",
"transport": "worker",
"kinds": [{kinds}],
"artifacts": [
{{"target": "{}", "path": "{path}", "sha256": "{sha}"}}
],
"permissions_requested": []
}}"#,
HostTarget::current().triple()
)
}
#[test]
fn valid_v2_manifest_verifies_artifact() {
let dir = tempdir("valid");
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
let manifest = write_manifest(
&dir,
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha),
);
let providers = read_provider_manifests(&[manifest]).unwrap();
assert_eq!(providers.len(), 1);
assert_eq!(providers[0].name, "demo.sensor");
assert_eq!(
providers[0].selected_target,
HostTarget::current().triple().to_string()
);
assert!(providers[0].manifest_digest.starts_with("sha256:"));
providers[0].reverify().unwrap();
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn v1_manifest_is_rejected_without_migration() {
let dir = tempdir("v1");
let manifest = write_manifest(
&dir,
r#"{"schema":"marqdo.provider","schema_version":1,"name":"cabi","kinds":["cabi.sensor"],"abi":"c-abi-1","library":{"linux":"libcabi.so"}}"#,
);
let error = read_provider_manifests(&[manifest]).unwrap_err();
assert_eq!(error.code, "ext.unsupported_provider_schema");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn missing_target_is_no_matching_artifact() {
let dir = tempdir("target");
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
let manifest = write_manifest(
&dir,
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha).replace(
&HostTarget::current().triple().to_string(),
"aarch64-unknown-linux-gnu",
),
);
let error = read_provider_manifests(&[manifest]).unwrap_err();
assert_eq!(error.code, "ext.no_matching_artifact");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn reserved_kind_is_rejected() {
let dir = tempdir("reserved");
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
let manifest = write_manifest(&dir, &manifest_text("demo.sensor", &["file"], &path, &sha));
let error = read_provider_manifests(&[manifest]).unwrap_err();
assert_eq!(error.code, "ext.manifest_invalid");
assert!(error.message.contains("reserved"), "{}", error.message);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn traversal_and_digest_mismatch_are_artifact_integrity() {
let dir = tempdir("integrity");
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
let traversal = write_manifest(
&dir,
&manifest_text("demo.sensor", &["demo.sensor"], "../libdemo.so", &sha),
);
let error = read_provider_manifests(&[traversal]).unwrap_err();
assert_eq!(error.code, "ext.artifact_integrity");
let mismatch = write_manifest(
&dir,
&manifest_text("demo.sensor", &["demo.sensor"], &path, &"0".repeat(64)),
);
let error = read_provider_manifests(&[mismatch]).unwrap_err();
assert_eq!(error.code, "ext.artifact_integrity");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn duplicate_kinds_across_providers_are_rejected() {
let dir = tempdir("duplicate-kind");
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
let first = write_manifest(
&dir,
&manifest_text("demo.one", &["demo.sensor"], &path, &sha),
);
let second_dir = dir.join("second");
std::fs::create_dir_all(&second_dir).unwrap();
std::fs::copy(dir.join(&path), second_dir.join(&path)).unwrap();
let second = write_manifest(
&second_dir,
&manifest_text("demo.two", &["demo.sensor"], &path, &sha),
);
let error = read_provider_manifests(&[first, second]).unwrap_err();
assert_eq!(error.code, "ext.duplicate_provider_kind");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn duplicate_provider_name_with_different_manifests_is_rejected() {
let dir = tempdir("duplicate-name");
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
let first = write_manifest(
&dir,
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha),
);
let second_dir = dir.join("second");
std::fs::create_dir_all(&second_dir).unwrap();
std::fs::copy(dir.join(&path), second_dir.join(&path)).unwrap();
// Same name, different version => different manifest digest.
let second = write_manifest(
&second_dir,
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha)
.replace("\"version\": \"0.1.0\"", "\"version\": \"0.2.0\""),
);
let error = read_provider_manifests(&[first.clone(), second]).unwrap_err();
assert_eq!(error.code, "ext.duplicate_provider");
// Selecting the exact same manifest twice is idempotent.
let providers = read_provider_manifests(&[first.clone(), first]).unwrap();
assert_eq!(providers.len(), 1);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn unknown_fields_and_bad_names_are_manifest_invalid() {
let dir = tempdir("invalid");
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
let unknown = write_manifest(
&dir,
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha).replace(
"\"permissions_requested\": []",
"\"permissions_requested\": [], \"exec\": true",
),
);
let error = read_provider_manifests(&[unknown]).unwrap_err();
assert_eq!(error.code, "ext.manifest_invalid");
let bad_name = write_manifest(
&dir,
&manifest_text("Demo.Sensor", &["demo.sensor"], &path, &sha),
);
let error = read_provider_manifests(&[bad_name]).unwrap_err();
assert_eq!(error.code, "ext.manifest_invalid");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn worker_helper_resolution_only_uses_the_executable_sibling() {
// The real helper is installed next to the CLI binary by the
// integration fixture; this test only asserts the platform/absence
// contract and that a path is never derived from PATH.
if provider_worker_platform_supported() {
let helper = provider_worker_executable();
// In a unit-test binary `current_exe()` is the test binary, so the
// sibling helper is normally absent: the error must be structured.
match helper {
Ok(path) => assert!(path.is_file()),
Err(error) => assert_eq!(error.code, "ext.worker_unavailable"),
}
} else {
let error = provider_worker_executable().unwrap_err();
assert_eq!(error.code, "ext.platform_unsupported");
}
}
}
+209
View File
@@ -13,6 +13,25 @@ use sha2::{Digest, Sha256};
/// Environment override used by tests and relocatable installations.
pub const STDLIB_DIR_ENV: &str = "MARQDO_STDLIB_DIR";
/// One official bundled-stdlib public alias (ADR-0058).
///
/// The alias table lives in the verified `stdlib.json` metadata, not in Rust
/// source: `打印`/`print` are ordinary `Marqdo` bindings to `io.mq.md::写值`
/// and the runtime no longer special-cases print names.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PublicAlias {
/// Public names that resolve to the target function.
pub public_names: Vec<String>,
/// Package-relative target module (for example `io.mq.md`).
pub target_module: String,
/// Canonical/local target function name.
pub target_function: String,
/// Declared parameter names, used to verify the target signature.
pub expected_signature: Option<String>,
/// Conservative effect request summary.
pub effects: Vec<String>,
}
/// Verified bundled standard library.
#[derive(Debug, Clone)]
pub struct StdlibBundle {
@@ -20,6 +39,7 @@ pub struct StdlibBundle {
name: String,
version: String,
digest: String,
aliases: Vec<PublicAlias>,
}
impl StdlibBundle {
@@ -81,14 +101,42 @@ impl StdlibBundle {
"stdlib digest mismatch: declared {declared}, actual {actual}"
));
}
let aliases = parse_public_aliases(&manifest, &manifest_path)?;
let declared_alias_digest = manifest
.get("aliases_digest")
.and_then(serde_json::Value::as_str);
let actual_alias_digest = aliases_digest(&aliases);
match (declared_alias_digest, aliases.is_empty()) {
(Some(declared), _) => {
if declared != actual_alias_digest {
return Err(format!(
"stdlib public-alias metadata digest mismatch: declared {declared}, \
actual {actual_alias_digest}"
));
}
}
(None, false) => {
return Err(format!(
"`{}` declares public_aliases without aliases_digest",
manifest_path.display()
));
}
(None, true) => {}
}
Ok(Self {
root,
name,
version,
digest: actual,
aliases,
})
}
/// Verified official public aliases.
pub fn public_aliases(&self) -> &[PublicAlias] {
&self.aliases
}
/// Bundle root.
pub fn root(&self) -> &Path {
&self.root
@@ -227,6 +275,167 @@ fn hex(bytes: &[u8]) -> String {
out
}
fn parse_public_aliases(
manifest: &serde_json::Value,
manifest_path: &Path,
) -> Result<Vec<PublicAlias>, String> {
let Some(entries) = manifest.get("public_aliases") else {
return Ok(Vec::new());
};
let entries = entries.as_array().ok_or_else(|| {
format!(
"`{}` public_aliases must be an array",
manifest_path.display()
)
})?;
let mut aliases = Vec::new();
let mut seen_names = std::collections::BTreeSet::new();
for entry in entries {
let public_names = entry
.get("public_names")
.and_then(serde_json::Value::as_array)
.ok_or_else(|| {
format!(
"`{}` public alias has no public_names array",
manifest_path.display()
)
})?
.iter()
.map(|name| {
name.as_str().map(ToString::to_string).ok_or_else(|| {
format!(
"`{}` public alias name must be a string",
manifest_path.display()
)
})
})
.collect::<Result<Vec<_>, _>>()?;
if public_names.is_empty() {
return Err(format!(
"`{}` public alias declares no names",
manifest_path.display()
));
}
for name in &public_names {
if name.is_empty() {
return Err(format!(
"`{}` public alias name must not be empty",
manifest_path.display()
));
}
if !seen_names.insert(name.clone()) {
return Err(format!(
"`{}` public alias name `{name}` is declared more than once",
manifest_path.display()
));
}
}
let target_module = entry
.get("target_module")
.and_then(serde_json::Value::as_str)
.ok_or_else(|| {
format!(
"`{}` public alias has no target_module",
manifest_path.display()
)
})?
.to_string();
validate_bundle_relative(&target_module).map_err(|error| {
format!(
"`{}` public alias target_module `{target_module}` is invalid: {error}",
manifest_path.display()
)
})?;
let target_function = entry
.get("target_function")
.and_then(serde_json::Value::as_str)
.ok_or_else(|| {
format!(
"`{}` public alias has no target_function",
manifest_path.display()
)
})?
.to_string();
if target_function.is_empty() {
return Err(format!(
"`{}` public alias target_function must not be empty",
manifest_path.display()
));
}
let expected_signature = entry
.get("expected_signature")
.and_then(serde_json::Value::as_str)
.map(ToString::to_string);
let effects = entry
.get("effects")
.and_then(serde_json::Value::as_array)
.map(|effects| {
effects
.iter()
.filter_map(serde_json::Value::as_str)
.map(ToString::to_string)
.collect()
})
.unwrap_or_default();
aliases.push(PublicAlias {
public_names,
target_module,
target_function,
expected_signature,
effects,
});
}
aliases.sort_by(|left, right| {
(&left.target_module, &left.target_function)
.cmp(&(&right.target_module, &right.target_function))
});
Ok(aliases)
}
fn validate_bundle_relative(path: &str) -> Result<(), String> {
let mut normalised = PathBuf::new();
for component in Path::new(path).components() {
match component {
Component::Normal(part) => normalised.push(part),
Component::CurDir => {}
Component::ParentDir | Component::RootDir | Component::Prefix(_) => {
return Err("must be a relative path inside the bundle".to_string())
}
}
}
if normalised.as_os_str().is_empty() {
return Err("must not be empty".to_string());
}
Ok(())
}
/// Canonical digest over the official public-alias table.
///
/// Format (identical to `tools/pack-stdlib.py`): for every alias sorted by
/// `(target_module, target_function)`: module, NUL, function, NUL, then every
/// public name sorted, NUL. A fixed binary format avoids JSON formatting drift.
pub fn aliases_digest(aliases: &[PublicAlias]) -> String {
let mut sorted = aliases.to_vec();
sorted.sort_by(|left, right| {
(&left.target_module, &left.target_function)
.cmp(&(&right.target_module, &right.target_function))
});
let mut hasher = Sha256::new();
for alias in &sorted {
hasher.update(alias.target_module.as_bytes());
hasher.update([0]);
hasher.update(alias.target_function.as_bytes());
hasher.update([0]);
let mut names = alias.public_names.clone();
names.sort();
for name in names {
hasher.update(name.as_bytes());
hasher.update([0]);
}
}
format!("sha256:{}", hex(&hasher.finalize()))
}
#[cfg(test)]
mod tests {
use super::*;
+122
View File
@@ -272,3 +272,125 @@ fn walk_rs(root: &Path) -> Vec<PathBuf> {
}
files
}
/// RFC-EXT-0001 §4/§14: the runtime must dispatch by the prepared binding and
/// never re-resolve a name or special-case a print function.
#[test]
fn callable_binding_is_the_only_static_resolution_path() {
let root = repo_root();
let loader = read(&root.join("crates/cli/src/loader.rs"));
let interpreter = read(&root.join("crates/marqdo-runtime/src/interpreter.rs"));
let run = read(&root.join("crates/cli/src/commands/run.rs"));
let frontend = read(&root.join("crates/marqdo-frontend/src/calls.rs"));
assert!(
!loader.contains("program_uses_print"),
"loader must not scan for per-function names (ADR-0058)"
);
assert!(
!interpreter.contains("print_override"),
"the runtime must not special-case print (ADR-0054/0058)"
);
assert!(
!interpreter.contains("external_callables.is_empty"),
"the runtime must not fall back to plugin guessing (ADR-0054)"
);
assert!(
interpreter.contains("call_bindings.get"),
"the runtime must dispatch by the prepared call binding table"
);
assert!(
!interpreter.contains("builtins::find(&qualified)"),
"a qualified name must not be re-resolved at run time"
);
assert!(
!run.contains("external_callables")
|| !run.contains("external_callables")
|| !run.contains("push(marqdo_ir::callable::CallableDescriptor"),
"run must not synthesise plugin descriptors (ADR-0054/§4.5)"
);
assert!(
frontend.contains("bind_program_calls"),
"the frontend must expose the single binding entry point"
);
}
/// RFC-EXT-0001 §14.2/S4: no business-domain builtin or host-op branch may be
/// added for an ecosystem feature. These must be implemented as `.mq.md`,
/// plugin exports or provider streams.
#[test]
fn no_business_domain_builtins_or_host_ops() {
let root = repo_root();
let builtins = read(&root.join("crates/marqdo-ir/src/builtins.rs"));
let host = read(&root.join("crates/marqdo-host/src/host.rs"));
for forbidden in ["quantum.", "ai.chat", "vendor.", "demo.sensor", "marqdo."] {
assert!(
!builtins.contains(forbidden),
"business builtin namespace `{forbidden}` must not exist"
);
}
assert!(
!host.contains("\"demo.sensor\""),
"provider kinds must be data, not host-op string branches"
);
}
/// RFC-EXT-0001 §8: official stdlib public aliases are verified metadata, not
/// hard-coded runtime names. Recompute the canonical metadata digest.
#[test]
fn official_stdlib_public_aliases_are_verified_metadata() {
let root = repo_root().join("stdlib");
let manifest: serde_json::Value =
serde_json::from_slice(&std::fs::read(root.join("stdlib.json")).unwrap()).unwrap();
let aliases = manifest["public_aliases"]
.as_array()
.expect("stdlib.json must declare public_aliases");
assert!(!aliases.is_empty());
let mut hasher = Sha256::new();
let mut entries: Vec<&serde_json::Value> = aliases.iter().collect();
entries.sort_by_key(|alias| {
(
alias["target_module"].as_str().unwrap_or("").to_string(),
alias["target_function"].as_str().unwrap_or("").to_string(),
)
});
for alias in &entries {
hasher.update(alias["target_module"].as_str().unwrap().as_bytes());
hasher.update([0]);
hasher.update(alias["target_function"].as_str().unwrap().as_bytes());
hasher.update([0]);
let mut names: Vec<&str> = alias["public_names"]
.as_array()
.unwrap()
.iter()
.filter_map(serde_json::Value::as_str)
.collect();
names.sort();
for name in names {
hasher.update(name.as_bytes());
hasher.update([0]);
}
}
let actual = format!("sha256:{}", hex(&hasher.finalize()));
assert_eq!(
manifest["aliases_digest"], actual,
"stdlib.json aliases_digest must cover the public alias table"
);
for alias in aliases {
let target_module = alias["target_module"].as_str().unwrap();
let target_function = alias["target_function"].as_str().unwrap();
let module = read(&root.join(target_module));
assert!(
module.contains(&format!("## {target_function}")),
"public alias target `{target_module}::{target_function}` must exist"
);
}
}
fn hex(bytes: &[u8]) -> String {
let mut out = String::with_capacity(bytes.len() * 2);
for byte in bytes {
out.push_str(&format!("{byte:02x}"));
}
out
}
+653
View File
@@ -0,0 +1,653 @@
//! Black-box callable resolution / binding conformance (RFC-EXT-0001 §4.6).
//!
//! Every test drives the real `marqdo` binary through the public CLI. The
//! point is to prove that `check` and `run` share one static resolution and
//! that the runtime never guesses a plugin from a dotted string prefix.
#![cfg(unix)]
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
use std::sync::atomic::{AtomicU32, Ordering};
use marqdo_plugin_abi::{sha256_file, HostTarget};
static COUNTER: AtomicU32 = AtomicU32::new(0);
struct TempDir(PathBuf);
impl TempDir {
fn new(tag: &str) -> Self {
let id = COUNTER.fetch_add(1, Ordering::Relaxed);
let path = std::env::temp_dir().join(format!(
"marqdo-callable-test-{}-{id}-{tag}",
std::process::id()
));
std::fs::create_dir_all(&path).unwrap();
Self(path)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn repo() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
fn cli(args: &[&str], dir: &Path) -> Output {
Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(args)
.current_dir(dir)
.output()
.unwrap()
}
fn write(path: &Path, text: &str) {
std::fs::write(path, text).unwrap();
}
fn library_name() -> &'static str {
if cfg!(target_os = "macos") {
"solver.dylib"
} else {
"solver.so"
}
}
fn shared_flags() -> &'static [&'static str] {
if cfg!(target_os = "macos") {
&["-dynamiclib", "-fPIC"]
} else {
&["-shared", "-fPIC"]
}
}
fn have_cc() -> bool {
Command::new("cc")
.arg("--version")
.output()
.map(|output| output.status.success())
.unwrap_or(false)
}
fn compile_plugin(dir: &Path, extra: &[&str]) -> PathBuf {
let output = dir.join(library_name());
let status = Command::new("cc")
.args(["-std=c11", "-O2"])
.args(shared_flags())
.arg("-I")
.arg(repo().join("include"))
.args(extra)
.arg(repo().join("examples/plugin-demo/solver.c"))
.arg("-o")
.arg(&output)
.status()
.expect("cc");
assert!(status.success(), "cc failed");
output
}
fn write_worker_manifest(dir: &Path, artifact: &Path, exports: &str) {
let sha = sha256_file(artifact).unwrap();
write(
&dir.join("plugin.yaml"),
&format!(
r#"plugin:
name: demo.solver
version: 0.1.0
abi: "^1.0"
kind: worker
exports:
{exports} requires:
- math.compute
artifacts:
- target: {}
file: {}
sha256: "{sha}"
"#,
HostTarget::current().triple(),
artifact.file_name().unwrap().to_string_lossy()
),
);
}
fn add_export(name: &str, signature: &str) -> String {
format!(" - name: {name}\n signature: \"{signature}\"\n")
}
fn stderr(output: &Output) -> String {
String::from_utf8_lossy(&output.stderr).to_string()
}
fn stdout(output: &Output) -> String {
String::from_utf8_lossy(&output.stdout).to_string()
}
// CALL-01
#[test]
fn import_alias_conflicting_with_builtin_namespace_is_rejected() {
let dir = TempDir::new("call01");
write(
&dir.path().join("math.mq.md"),
"## @exec eval\n\n`params(a: Int)`\n\n`return a + 100`\n",
);
write(
&dir.path().join("index.mq.md"),
"---\nimport:\n math: \"./math.mq.md\"\n---\n\n## @exec main\n\n`return math.eval(a=1)`\n",
);
let check = cli(&["check", "index.mq.md"], dir.path());
assert_eq!(check.status.code(), Some(1), "{}", stderr(&check));
assert!(
stderr(&check).contains("ext.import_namespace_conflict"),
"{}",
stderr(&check)
);
let run = cli(&["run", "index.mq.md"], dir.path());
assert_eq!(run.status.code(), Some(1));
assert!(stdout(&run).is_empty());
assert!(
stderr(&run).contains("ext.import_namespace_conflict"),
"{}",
stderr(&run)
);
}
// CALL-02
#[test]
fn dotted_builtin_call_binds_without_an_import() {
let output = cli(&["check", "examples/math.mq.md"], &repo());
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
let run = cli(&["call", "examples/math.mq.md", "计算圆面积"], &repo());
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
}
// CALL-03
#[test]
fn renamed_import_alias_binds_the_module_function() {
let dir = TempDir::new("call03");
write(
&dir.path().join("math.mq.md"),
"## @exec eval\n\n`params(a: Int)`\n\n`return a + 100`\n",
);
write(
&dir.path().join("index.mq.md"),
"---\nimport:\n mymath: \"./math.mq.md\"\n---\n\n## @exec main\n\n`return mymath.eval(a=1)`\n",
);
let check = cli(&["check", "index.mq.md"], dir.path());
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
let run = cli(&["run", "index.mq.md", "--json"], dir.path());
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
let json: serde_json::Value = serde_json::from_str(&stdout(&run)).unwrap();
assert_eq!(
json["value"],
serde_json::json!({"type": "int", "decimal": "101"})
);
}
// CALL-04 / CALL-05
#[test]
fn unknown_plugin_export_is_a_static_symbol_error() {
let dir = TempDir::new("call04");
write(
&dir.path().join("index.mq.md"),
"## @exec main\n\n`return solver.add(a=1, b=2)`\n",
);
let check = cli(&["check", "index.mq.md"], dir.path());
assert_eq!(check.status.code(), Some(1));
assert!(
stderr(&check).contains("symbol.unknown"),
"{}",
stderr(&check)
);
let run = cli(&["run", "index.mq.md"], dir.path());
assert_eq!(run.status.code(), Some(1));
assert!(stderr(&run).contains("symbol.unknown"), "{}", stderr(&run));
assert!(stdout(&run).is_empty());
// CALL-05: the diagnostic does not change when an unrelated plugin is
// loaded, because the static export table is still the only source.
if !have_cc() {
return;
}
let plugin_dir = TempDir::new("call05-plugin");
let artifact = compile_plugin(plugin_dir.path(), &[]);
write_worker_manifest(
plugin_dir.path(),
&artifact,
&add_export("solver.add", "($a: Int, $b: Int) -> Int"),
);
write(
&dir.path().join("index.mq.md"),
"## @exec main\n\n`return solver.subtract(a=1, b=2)`\n",
);
let with_plugin = cli(
&[
"run",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
"--allow-plugin",
"demo.solver",
],
dir.path(),
);
assert_eq!(with_plugin.status.code(), Some(1));
assert!(
stderr(&with_plugin).contains("symbol.unknown"),
"{}",
stderr(&with_plugin)
);
}
// CALL-06
#[test]
fn plugin_with_two_prefixes_binds_both_exports() {
if !have_cc() {
eprintln!("skipping: cc is unavailable");
return;
}
let dir = TempDir::new("call06");
let plugin_dir = TempDir::new("call06-plugin");
let artifact = compile_plugin(plugin_dir.path(), &["-DMQ_SOLVER_MULTI_EXPORT"]);
write_worker_manifest(
plugin_dir.path(),
&artifact,
&(add_export("solver.add", "($a: Int, $b: Int) -> Int")
+ &add_export("solver.neg", "($a: Int) -> Int")),
);
write(
&dir.path().join("index.mq.md"),
"## @exec main\n\n`和 = solver.add(20, b=22)`\n\n`相反数 = solver.neg(7)`\n\n`打印(和)`\n\n`打印(相反数)`\n",
);
let output = cli(
&[
"run",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
"--allow-plugin",
"demo.solver",
],
dir.path(),
);
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
assert_eq!(stdout(&output), "42\n-7\n");
}
// CALL-07
#[test]
fn duplicate_complete_export_across_plugins_is_rejected_at_prepare() {
if !have_cc() {
eprintln!("skipping: cc is unavailable");
return;
}
let dir = TempDir::new("call07");
write(
&dir.path().join("index.mq.md"),
"## @exec main\n\n`return solver.add(a=1, b=2)`\n",
);
let mut plugins = Vec::new();
for tag in ["a", "b"] {
let plugin_dir = TempDir::new(tag);
let artifact = compile_plugin(plugin_dir.path(), &[]);
let sha = sha256_file(&artifact).unwrap();
write(
&plugin_dir.path().join("plugin.yaml"),
&format!(
r#"plugin:
name: demo.{tag}.solver
version: 0.1.0
abi: "^1.0"
kind: worker
exports:
- name: solver.add
signature: "($a: Int, $b: Int) -> Int"
artifacts:
- target: {}
file: {}
sha256: "{sha}"
"#,
HostTarget::current().triple(),
artifact.file_name().unwrap().to_string_lossy()
),
);
plugins.push(plugin_dir);
}
let check = cli(
&[
"check",
"index.mq.md",
"--plugin",
plugins[0].path().to_str().unwrap(),
"--plugin",
plugins[1].path().to_str().unwrap(),
],
dir.path(),
);
assert_eq!(check.status.code(), Some(2), "{}", stderr(&check));
assert!(
stderr(&check).contains("ext.duplicate_callable"),
"{}",
stderr(&check)
);
}
// CALL-08
#[test]
fn bad_plugin_arguments_fail_in_check_and_run_identically() {
if !have_cc() {
eprintln!("skipping: cc is unavailable");
return;
}
let dir = TempDir::new("call08");
let plugin_dir = TempDir::new("call08-plugin");
let artifact = compile_plugin(plugin_dir.path(), &[]);
write_worker_manifest(
plugin_dir.path(),
&artifact,
&add_export("solver.add", "($a: Int, $b: Int) -> Int"),
);
for (tag, call) in [
("missing", "solver.add(a=1)"),
("unknown", "solver.add(a=1, b=2, c=3)"),
("duplicate", "solver.add(a=1, a=2, b=3)"),
("too-many-positional", "solver.add(1, 2, 3)"),
] {
write(
&dir.path().join("index.mq.md"),
&format!("## @exec main\n\n`return {call}`\n"),
);
let check = cli(
&[
"check",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
],
dir.path(),
);
assert_eq!(check.status.code(), Some(1), "{tag}: {}", stderr(&check));
let expected = if tag == "duplicate" {
// The expression grammar rejects duplicate named arguments before
// call binding; either way the failure is static.
"sem.syntax"
} else {
"call.arity"
};
assert!(
stderr(&check).contains(expected),
"{tag}: {}",
stderr(&check)
);
let run = cli(
&[
"run",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
"--allow-plugin",
"demo.solver",
],
dir.path(),
);
assert_eq!(run.status.code(), Some(1), "{tag}: {}", stderr(&run));
assert!(stdout(&run).is_empty(), "{tag}");
// `check` appends a summary line and `run` a refusal line, but the
// structured call.arity diagnostic itself must be identical.
let arity = |text: &str| {
text.lines()
.filter(|line| line.contains("call.arity") || line.contains("sem.syntax"))
.map(ToString::to_string)
.collect::<Vec<_>>()
};
assert!(!arity(&stderr(&check)).is_empty(), "{tag}");
assert_eq!(
arity(&stderr(&check)),
arity(&stderr(&run)),
"{tag}: check != run"
);
}
}
// CALL-09
#[test]
fn changed_artifact_digest_is_rejected_before_main_runs() {
if !have_cc() {
eprintln!("skipping: cc is unavailable");
return;
}
let dir = TempDir::new("call09");
let plugin_dir = TempDir::new("call09-plugin");
let artifact = compile_plugin(plugin_dir.path(), &[]);
write_worker_manifest(
plugin_dir.path(),
&artifact,
&add_export("solver.add", "($a: Int, $b: Int) -> Int"),
);
write(
&dir.path().join("index.mq.md"),
"## @exec main\n\n`打印(\"must-not-run\")`\n\n`return solver.add(a=1, b=2)`\n",
);
// Static check succeeds: it never opens the artifact.
let check = cli(
&[
"check",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
],
dir.path(),
);
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
// Corrupt the artifact bytes; run must reject before the entry function.
let mut bytes = std::fs::read(&artifact).unwrap();
bytes.push(0);
std::fs::write(&artifact, bytes).unwrap();
let run = cli(
&[
"run",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
"--allow-plugin",
"demo.solver",
],
dir.path(),
);
assert_eq!(run.status.code(), Some(1), "{}", stderr(&run));
assert!(
stderr(&run).contains("ext.artifact_integrity"),
"{}",
stderr(&run)
);
assert!(stdout(&run).is_empty(), "{}", stdout(&run));
}
// CALL-11
#[test]
fn local_function_reference_is_a_dynamic_language_call() {
let dir = TempDir::new("call11");
write(
&dir.path().join("index.mq.md"),
"## @exec 平方\n\n`params(n: Int)`\n\n`return n * n`\n\n## @exec 主\n\n`f = ref(\"#平方\")`\n\n`return f(n=4)`\n",
);
let run = cli(&["call", "index.mq.md", "主", "--json"], dir.path());
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
let json: serde_json::Value = serde_json::from_str(&stdout(&run)).unwrap();
assert_eq!(
json["value"],
serde_json::json!({"type": "int", "decimal": "16"})
);
}
// CALL-12
#[test]
fn calling_a_text_value_is_not_callable_and_never_reaches_the_host() {
let dir = TempDir::new("call12");
write(
&dir.path().join("index.mq.md"),
"## @exec main\n\n`f = \"text\"`\n\n`return f()`\n",
);
let check = cli(&["check", "index.mq.md"], dir.path());
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
let run = cli(&["run", "index.mq.md"], dir.path());
assert_eq!(run.status.code(), Some(1), "{}", stderr(&run));
assert!(
stderr(&run).contains("call.not_callable") || stderr(&run).contains("not_callable"),
"{}",
stderr(&run)
);
}
// CALL-13
#[test]
fn explicit_import_aliases_bind_same_named_functions_independently() {
let dir = TempDir::new("call13");
write(
&dir.path().join("a.mq.md"),
"## @exec value\n\n`params()`\n\n`return 10`\n",
);
write(
&dir.path().join("b.mq.md"),
"## @exec value\n\n`params()`\n\n`return 20`\n",
);
write(
&dir.path().join("index.mq.md"),
"---\nimport:\n aa: \"./a.mq.md\"\n bb: \"./b.mq.md\"\n---\n\n## @exec main\n\n`return aa.value() + bb.value()`\n",
);
let run = cli(&["run", "index.mq.md", "--json"], dir.path());
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
let json: serde_json::Value = serde_json::from_str(&stdout(&run)).unwrap();
assert_eq!(
json["value"],
serde_json::json!({"type": "int", "decimal": "30"})
);
}
// CALL-14
#[test]
fn check_never_executes_a_plugin_artifact_or_side_effect_script() {
let dir = TempDir::new("call14");
let plugin_dir = TempDir::new("call14-plugin");
let marker = plugin_dir.path().join("executed.marker");
let artifact = plugin_dir.path().join(library_name());
write(
&artifact,
&format!("#!/bin/sh\ntouch '{}'\nexit 0\n", marker.display()),
);
let sha = sha256_file(&artifact).unwrap();
write(
&plugin_dir.path().join("plugin.yaml"),
&format!(
r#"plugin:
name: demo.script
version: 0.1.0
abi: "^1.0"
kind: worker
exports:
- name: demo.run
signature: "() -> Int"
artifacts:
- target: {}
file: {}
sha256: "{sha}"
"#,
HostTarget::current().triple(),
artifact.file_name().unwrap().to_string_lossy()
),
);
write(
&dir.path().join("index.mq.md"),
"## @exec main\n\n`return demo.run()`\n",
);
let check = cli(
&[
"check",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
],
dir.path(),
);
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
assert!(!marker.exists(), "check executed the artifact");
}
// CALL-15
#[test]
fn check_emit_and_run_share_the_same_static_binding_source() {
if !have_cc() {
eprintln!("skipping: cc is unavailable");
return;
}
let dir = TempDir::new("call15");
let plugin_dir = TempDir::new("call15-plugin");
let artifact = compile_plugin(plugin_dir.path(), &[]);
write_worker_manifest(
plugin_dir.path(),
&artifact,
&add_export("solver.add", "($a: Int, $b: Int) -> Int"),
);
write(
&dir.path().join("index.mq.md"),
"## @exec main\n\n`return solver.add(a=1, b=2)`\n",
);
let analysis_path = dir.path().join("analysis.json");
let check = cli(
&[
"check",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
"--emit",
analysis_path.to_str().unwrap(),
],
dir.path(),
);
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
let analysis: serde_json::Value =
serde_json::from_str(&std::fs::read_to_string(&analysis_path).unwrap()).unwrap();
let functions = analysis["modules"]
.as_array()
.and_then(|modules| modules.first())
.and_then(|module| module["functions"].as_array())
.unwrap_or_else(|| panic!("analysis has no module functions: {analysis}"));
assert!(!functions.is_empty());
let run = cli(
&[
"run",
"index.mq.md",
"--plugin",
plugin_dir.path().to_str().unwrap(),
"--allow-plugin",
"demo.solver",
"--json",
],
dir.path(),
);
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
// Without the descriptor the same call is statically unknown: run/check
// consume the same prepared binding, not a runtime name probe.
let without = cli(&["check", "index.mq.md"], dir.path());
assert_eq!(without.status.code(), Some(1));
assert!(stderr(&without).contains("symbol.unknown"));
}
// CALL-16
#[test]
fn print_alias_keeps_the_existing_output_bytes() {
let run = cli(&["run", "examples/hello.mq.md"], &repo());
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
assert_eq!(stdout(&run), "你好,Marqdo\n");
}
+80 -18
View File
@@ -166,8 +166,8 @@ fn semantic_goldens_match_end_to_end_execution() {
let mut program = marqdo_ir::Program::default();
program.insert(analysis.module.clone());
program.main_module = Some(module_id.clone());
install_stdlib_print(&mut program);
install_entry_stdlib_imports(&mut program, &module_id, &analysis);
bind_calls_with_stdlib_aliases(&mut program);
let mut capabilities = Capabilities::pure();
if grant_foreign {
@@ -284,26 +284,88 @@ fn normalise_relative(path: &Path) -> String {
.join("/")
}
/// Load the bundled stdlib print modules for direct-runtime fixtures.
/// Bind direct-runtime fixtures exactly like the CLI composition root.
///
/// The CLI loader normally injects these for Compat-B. The semantic harness
/// builds a Program directly, so it mirrors the same resolution here; without
/// it, fixtures that call `打印` would hit the Compat-C "no bundle" error.
fn install_stdlib_print(program: &mut Program) {
/// The semantic harness builds a `Program` directly, so it mirrors the
/// production path: load the official stdlib public-alias targets, build the
/// unique registry, run the single binding pass and install its table. There is
/// no `print_override` and no runtime name guessing.
fn bind_calls_with_stdlib_aliases(program: &mut Program) {
let root = repo_root().join("stdlib");
if !root.join("stdlib.json").is_file() {
return;
}
let mut loaded = std::collections::BTreeSet::new();
let io = match load_stdlib_module(&root, "io.mq.md", program, &mut loaded) {
Ok(id) => id,
Err(error) => panic!("cannot load bundled stdlib: {error}"),
};
let function = program
.module(&io)
.and_then(|module| module.function("写值"))
.unwrap_or_else(|| panic!("stdlib io has no 写值 function"));
program.print_override = Some(function.id.clone());
let mut official = Vec::new();
if root.join("stdlib.json").is_file() {
let text = fs::read_to_string(root.join("stdlib.json"))
.unwrap_or_else(|error| panic!("cannot read stdlib.json: {error}"));
let manifest: serde_json::Value = serde_json::from_str(&text)
.unwrap_or_else(|error| panic!("invalid stdlib.json: {error}"));
if let Some(aliases) = manifest
.get("public_aliases")
.and_then(serde_json::Value::as_array)
{
for entry in aliases {
let public_names: Vec<String> = entry
.get("public_names")
.and_then(serde_json::Value::as_array)
.unwrap_or_else(|| panic!("public alias has no public_names"))
.iter()
.filter_map(serde_json::Value::as_str)
.map(ToString::to_string)
.collect();
let target_module = entry
.get("target_module")
.and_then(serde_json::Value::as_str)
.unwrap_or_else(|| panic!("public alias has no target_module"));
let target_function = entry
.get("target_function")
.and_then(serde_json::Value::as_str)
.unwrap_or_else(|| panic!("public alias has no target_function"));
let module_id = load_stdlib_module(&root, target_module, program, &mut loaded)
.unwrap_or_else(|error| panic!("cannot load stdlib alias module: {error}"));
let function = program
.module(&module_id)
.and_then(|module| module.function(target_function))
.unwrap_or_else(|| {
panic!("stdlib alias target `{target_function}` is missing")
});
let effects = entry
.get("effects")
.and_then(serde_json::Value::as_array)
.map(|effects| {
effects
.iter()
.filter_map(serde_json::Value::as_str)
.map(ToString::to_string)
.collect()
})
.unwrap_or_default();
official.push(marqdo_ir::callable::OfficialAlias {
public_names,
function: function.id.clone(),
signature: marqdo_ir::callable::CallableSignature {
params: function.params.clone(),
kind: function.signature,
},
owner_package: module_id.instance.clone(),
effects_requested: effects,
});
}
}
}
let mut registry = program.callable_registry();
for alias in official {
registry
.insert_official_alias(alias)
.unwrap_or_else(|error| panic!("cannot register stdlib alias: {error}"));
}
let bound = marqdo_frontend::calls::bind_program_calls(program, &registry);
if !bound.diagnostics.is_empty() {
panic!(
"fixture failed static call binding: {:#?}",
bound.diagnostics
);
}
program.set_call_bindings(bound.bindings);
}
fn load_stdlib_module(
+13
View File
@@ -224,6 +224,7 @@ fn runtime_negative_goldens() {
let mut program = marqdo_ir::Program::default();
program.insert(analysis.module.clone());
program.main_module = Some(module_id.clone());
bind_program_calls(&mut program);
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::pure(),
@@ -284,3 +285,15 @@ fn runtime_negative_goldens() {
}
let _ = fs::remove_dir_all(&project_root);
}
/// Bind direct-runtime fixtures exactly like the CLI composition root.
fn bind_program_calls(program: &mut marqdo_ir::Program) {
let registry = program.callable_registry();
let bound = marqdo_frontend::calls::bind_program_calls(program, &registry);
assert!(
bound.diagnostics.is_empty(),
"fixture failed static call binding: {:#?}",
bound.diagnostics
);
program.set_call_bindings(bound.bindings);
}
+8
View File
@@ -395,6 +395,14 @@ fn narration_removal_preserves_execution() {
let mut program = marqdo_ir::Program::default();
program.insert(analysis.module.clone());
program.main_module = Some(module_id.clone());
let registry = program.callable_registry();
let bound = marqdo_frontend::calls::bind_program_calls(&program, &registry);
assert!(
bound.diagnostics.is_empty(),
"property fixture failed binding: {:#?}",
bound.diagnostics
);
program.set_call_bindings(bound.bindings);
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::pure(),
File diff suppressed because it is too large. Load diff
+260
View File
@@ -0,0 +1,260 @@
//! Official stdlib public alias routing (RFC-EXT-0001 §8, ADR-0058).
//!
//! `打印`/`print` are ordinary `Marqdo` bindings to `io.mq.md::写值` declared
//! in the verified `stdlib.json` metadata. There is no `print_override`, no
//! runtime name special case and no hard-coded per-function scan.
#![cfg(unix)]
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
use std::sync::atomic::{AtomicU32, Ordering};
static COUNTER: AtomicU32 = AtomicU32::new(0);
struct TempDir(PathBuf);
impl TempDir {
fn new(tag: &str) -> Self {
let id = COUNTER.fetch_add(1, Ordering::Relaxed);
let path = std::env::temp_dir().join(format!(
"marqdo-stdlib-alias-test-{}-{id}-{tag}",
std::process::id()
));
std::fs::create_dir_all(&path).unwrap();
Self(path)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn repo() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
fn cli(args: &[&str], dir: &Path, env: &[(&str, &Path)]) -> Output {
let mut command = Command::new(env!("CARGO_BIN_EXE_marqdo"));
command.args(args).current_dir(dir);
for (key, value) in env {
command.env(key, value);
}
command.output().unwrap()
}
fn stderr(output: &Output) -> String {
String::from_utf8_lossy(&output.stderr).to_string()
}
fn stdout(output: &Output) -> String {
String::from_utf8_lossy(&output.stdout).to_string()
}
fn copy_stdlib(target: &Path) {
let source = repo().join("stdlib");
std::fs::create_dir_all(target).unwrap();
for entry in std::fs::read_dir(&source).unwrap() {
let entry = entry.unwrap();
let path = entry.path();
if path.is_dir() {
copy_dir(&path, &target.join(entry.file_name()));
} else {
std::fs::copy(&path, target.join(entry.file_name())).unwrap();
}
}
}
fn copy_dir(source: &Path, target: &Path) {
std::fs::create_dir_all(target).unwrap();
for entry in std::fs::read_dir(source).unwrap() {
let entry = entry.unwrap();
let path = entry.path();
if path.is_dir() {
copy_dir(&path, &target.join(entry.file_name()));
} else {
std::fs::copy(&path, target.join(entry.file_name())).unwrap();
}
}
}
#[test]
fn print_and_its_alias_are_bound_to_the_official_module() {
let dir = TempDir::new("print");
std::fs::write(
dir.path().join("index.mq.md"),
"## @exec main\n\n`打印(1)`\n\n`print(\"two\")`\n\n`return None`\n",
)
.unwrap();
let output = cli(&["run", "index.mq.md"], dir.path(), &[]);
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
assert_eq!(stdout(&output), "1\ntwo\n");
}
#[test]
fn program_without_print_does_not_load_the_stdlib_module_graph() {
let dir = TempDir::new("no-print");
std::fs::write(
dir.path().join("index.mq.md"),
"## @exec main\n\n`params()`\n\n`return 1 + 2`\n",
)
.unwrap();
let analysis_path = dir.path().join("factorial.json");
let output = cli(
&[
"check",
"index.mq.md",
"--emit",
analysis_path.to_str().unwrap(),
],
dir.path(),
&[],
);
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
let analysis: serde_json::Value =
serde_json::from_str(&std::fs::read_to_string(&analysis_path).unwrap()).unwrap();
let modules = analysis["modules"].as_array().unwrap();
assert!(
modules
.iter()
.all(|module| !module["id"].as_str().unwrap_or("").starts_with("std:")),
"a program without print must not load the stdlib: {analysis}"
);
}
#[test]
fn print_program_loads_exactly_the_declared_alias_target() {
let dir = TempDir::new("with-print");
std::fs::write(
dir.path().join("index.mq.md"),
"## @exec main\n\n`打印(\"x\")`\n\n`return None`\n",
)
.unwrap();
let analysis_path = dir.path().join("hello.json");
let output = cli(
&[
"check",
"index.mq.md",
"--emit",
analysis_path.to_str().unwrap(),
],
dir.path(),
&[],
);
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
let analysis: serde_json::Value =
serde_json::from_str(&std::fs::read_to_string(&analysis_path).unwrap()).unwrap();
let ids: Vec<&str> = analysis["modules"]
.as_array()
.unwrap()
.iter()
.filter_map(|module| module["id"].as_str())
.collect();
assert!(
ids.iter().any(|id| id.ends_with("::io.mq.md")),
"print must load io.mq.md: {ids:?}"
);
assert!(
ids.iter().any(|id| id.ends_with("::format.mq.md")),
"io.mq.md imports format.mq.md: {ids:?}"
);
}
#[test]
fn tampered_public_alias_metadata_digest_is_rejected() {
let dir = TempDir::new("alias-digest");
let stdlib = dir.path().join("stdlib");
copy_stdlib(&stdlib);
let manifest_path = stdlib.join("stdlib.json");
let text = std::fs::read_to_string(&manifest_path).unwrap();
let mut manifest: serde_json::Value = serde_json::from_str(&text).unwrap();
manifest["aliases_digest"] = serde_json::json!("sha256:".to_string() + &"0".repeat(64));
std::fs::write(
&manifest_path,
serde_json::to_string_pretty(&manifest).unwrap() + "\n",
)
.unwrap();
let doc = dir.path().join("index.mq.md");
std::fs::write(&doc, "## @exec main\n\n`打印(1)`\n\n`return None`\n").unwrap();
let output = cli(
&["run", doc.to_str().unwrap()],
dir.path(),
&[("MARQDO_STDLIB_DIR", &stdlib)],
);
assert!(!output.status.success(), "{}", stderr(&output));
assert!(
stderr(&output).contains("public-alias metadata digest mismatch"),
"{}",
stderr(&output)
);
}
#[test]
fn tampered_alias_target_signature_is_rejected() {
let dir = TempDir::new("alias-target");
let stdlib = dir.path().join("stdlib");
copy_stdlib(&stdlib);
let manifest_path = stdlib.join("stdlib.json");
let text = std::fs::read_to_string(&manifest_path).unwrap();
let mut manifest: serde_json::Value = serde_json::from_str(&text).unwrap();
manifest["public_aliases"][0]["expected_signature"] = serde_json::json!("params(值, 额外)");
// Recompute the declared metadata digest so only the target check can fail.
let canonical = manifest["public_aliases"][0].clone();
let module = canonical["target_module"].as_str().unwrap();
let function = canonical["target_function"].as_str().unwrap();
let mut names: Vec<&str> = canonical["public_names"]
.as_array()
.unwrap()
.iter()
.filter_map(serde_json::Value::as_str)
.collect();
names.sort();
use sha2::{Digest, Sha256};
let mut hasher = Sha256::new();
hasher.update(module.as_bytes());
hasher.update([0]);
hasher.update(function.as_bytes());
hasher.update([0]);
for name in names {
hasher.update(name.as_bytes());
hasher.update([0]);
}
let digest = format!("sha256:{}", hex(&hasher.finalize()));
manifest["aliases_digest"] = serde_json::json!(digest);
std::fs::write(
&manifest_path,
serde_json::to_string_pretty(&manifest).unwrap() + "\n",
)
.unwrap();
let doc = dir.path().join("index.mq.md");
std::fs::write(&doc, "## @exec main\n\n`打印(1)`\n\n`return None`\n").unwrap();
let output = cli(
&["run", doc.to_str().unwrap()],
dir.path(),
&[("MARQDO_STDLIB_DIR", &stdlib)],
);
assert!(!output.status.success(), "{}", stderr(&output));
assert!(
stderr(&output).contains("expected signature") || stderr(&output).contains("alias"),
"{}",
stderr(&output)
);
}
fn hex(bytes: &[u8]) -> String {
let mut out = String::with_capacity(bytes.len() * 2);
for byte in bytes {
out.push_str(&format!("{byte:02x}"));
}
out
}
+26 -1
View File
@@ -39,7 +39,30 @@ impl AnalysisResult {
/// Analyze one Markdown source as a Marqdo module.
///
/// `module_id` is the canonical project-relative identity (e.g. `index.mq.md`).
///
/// This standalone entry point also runs the module-local call check so that a
/// single document can be validated without a program graph. The CLI
/// composition root uses [`analyze_source_for_program`] instead: it defers all
/// call resolution to the single program-wide binding pass (ADR-0054).
pub fn analyze_source(module_id: &str, source: &str) -> AnalysisResult {
analyze_source_inner(module_id, source, true)
}
/// Analyze one Markdown source for the composition root.
///
/// Identical to [`analyze_source`] except that it does not run the provisional
/// module-local call check; the caller binds calls exactly once over the whole
/// program (including extension descriptors) and therefore owns the only
/// authoritative call diagnostics.
pub fn analyze_source_for_program(module_id: &str, source: &str) -> AnalysisResult {
analyze_source_inner(module_id, source, false)
}
fn analyze_source_inner(
module_id: &str,
source: &str,
validate_local_calls: bool,
) -> AnalysisResult {
let id = ModuleId::from(module_id);
let document = marqdo_markdown::parse(source);
let units = units::build_units(&document, &id);
@@ -161,7 +184,9 @@ pub fn analyze_source(module_id: &str, source: &str) -> AnalysisResult {
resources,
anchors: units.anchors.clone(),
};
diagnostics.extend(calls::validate_module(&module));
if validate_local_calls {
diagnostics.extend(calls::validate_module(&module));
}
// One stable order for every consumer: source position first, then the
// stable code, as required by `spec/mapping.md` §8. Diagnostics without a
+498 -248
View File
@@ -1,33 +1,51 @@
//! Static call validation.
//! Static call resolution and binding (RFC-EXT-0001 §4, ADR-0054).
//!
//! Calls are resolved after all functions have signatures (including inferred
//! ones), so arity/name mistakes are compile errors with source spans instead
//! of surprises at runtime. Type errors stay runtime concerns: v1 has no static
//! type inference beyond literal checks.
//! This module owns the *single* resolution algorithm used by `check`, `run`,
//! `inspect` and Analysis: it walks every statically visible call site exactly
//! once, resolves it against [`marqdo_ir::callable::CallableRegistry`] and
//! records a [`marqdo_ir::call_binding::BoundTarget`] in one
//! [`marqdo_ir::call_binding::CallBindingTable`]. The runtime consumes that
//! table and never re-resolves a name.
//!
//! Resolution priority is explicit and order-independent:
//!
//! | call form | rule |
//! |---|---|
//! | `f(...)` | local variable (dynamic) → official alias/plugin-fixed builtin → module function (current namespace first, then canonical, then unique local) |
//! | `alias.f(...)` | explicit import alias → builtin dotted prefix → exact plugin export |
//! | `some_value.f(...)` | dynamic language value only |
//!
//! Import aliases that collide with a builtin dotted namespace or a visible
//! plugin namespace are rejected with `ext.import_namespace_conflict`; declared
//! plugin exports that collide with each other or with builtins are rejected
//! when the registry is built. The previous "try builtin, then guess plugin at
//! runtime" behaviour is gone.
//!
//! `validate_module` keeps the standalone frontend API (a single document with
//! no program graph) for AST/conformance tests: unresolved imports are
//! *deferred* there because only the composition root can bind a whole program.
use std::collections::{BTreeMap, BTreeSet};
use marqdo_expression::{Arg, Expr, Ident};
use marqdo_ir::{builtins, Module, ModuleId, Param, PlaceTarget, Program, SignatureKind, Stmt};
use marqdo_ir::builtins;
use marqdo_ir::call_binding::{BoundTarget, CallBindingTable, CallSiteKey};
use marqdo_ir::callable::{CallableDescriptor, CallableRegistry};
use marqdo_ir::{
Function, FunctionId, Module, ModuleId, PlaceTarget, Program, SignatureKind, Stmt,
};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
/// Signature information needed to validate a call.
#[derive(Debug, Clone)]
pub struct CallableSignature {
/// Parameter list.
pub params: Vec<Param>,
/// Explicit signatures allow positional arguments; inferred do not.
pub kind: SignatureKind,
}
/// Build the callable signature table for a module.
pub fn signature_table(module: &Module) -> BTreeMap<String, CallableSignature> {
/// Build the callable signature table for one module (diagnostics helper).
pub fn signature_table(
module: &Module,
) -> BTreeMap<String, marqdo_ir::callable::CallableSignature> {
let mut table = BTreeMap::new();
for function in &module.functions {
table.insert(
function.id.name.clone(),
CallableSignature {
marqdo_ir::callable::CallableSignature {
params: function.params.clone(),
kind: function.signature,
},
@@ -36,10 +54,107 @@ pub fn signature_table(module: &Module) -> BTreeMap<String, CallableSignature> {
table
}
/// Validate every call in every function body of the module.
/// Result of binding a whole program.
#[derive(Debug, Clone, Default)]
pub struct BoundProgram {
/// One deterministic binding per static call site.
pub bindings: CallBindingTable,
/// Call-resolution diagnostics, sorted later by the caller.
pub diagnostics: Vec<(ModuleId, Diagnostic)>,
}
/// Validate every call in a single module (standalone frontend entry).
///
/// Imports and plugin exports cannot be resolved without a program graph, so
/// those call sites are deferred to the composition-root binding pass instead
/// of producing false errors or a second priority order.
pub fn validate_module(module: &Module) -> Vec<Diagnostic> {
let table = signature_table(module);
let mut program = Program::default();
program.modules.insert(module.id.clone(), module.clone());
program.main_module = Some(module.id.clone());
let registry = CallableRegistry::try_from_program(&program)
.expect("a single module cannot contain duplicate external declarations");
bind_with(module, &program, &registry, true)
.diagnostics
.into_iter()
.map(|(_, diagnostic)| diagnostic)
.collect()
}
/// Validate every call in a loaded program (tests and standalone frontend API).
///
/// The CLI composition root uses [`bind_program_calls`] and keeps the binding
/// table; this wrapper keeps the historical API for frontend tests.
pub fn validate_program(program: &Program) -> Vec<(ModuleId, Diagnostic)> {
let registry = match CallableRegistry::try_from_program(program) {
Ok(registry) => registry,
Err(_) => CallableRegistry::from_program(program),
};
bind_program_calls_with(program, &registry, false).diagnostics
}
/// Resolve every call site in `program` against `registry` exactly once.
///
/// This is the only binding entry point used by the composition root. The
/// returned table must be installed on [`Program::set_call_bindings`] before
/// execution.
pub fn bind_program_calls(program: &Program, registry: &CallableRegistry) -> BoundProgram {
bind_program_calls_with(program, registry, false)
}
fn bind_program_calls_with(
program: &Program,
registry: &CallableRegistry,
standalone: bool,
) -> BoundProgram {
let mut result = BoundProgram::default();
for module in program.modules.values() {
let mut bound = bind_with(module, program, registry, standalone);
// Conflict diagnostics are produced once per module import, before
// any call site; keep them in front of call diagnostics.
result.diagnostics.append(&mut bound.diagnostics);
for (key, target) in bound.bindings.iter() {
result.bindings.insert(key.clone(), target.clone());
}
}
result
}
fn bind_with(
module: &Module,
program: &Program,
registry: &CallableRegistry,
standalone: bool,
) -> BoundProgram {
let mut conflicted = BTreeSet::new();
let mut diagnostics = Vec::new();
// Reject import aliases that collide with a fixed builtin dotted namespace
// or a visible plugin namespace. `alias.f(...)` must have one target.
for import in &module.imports {
let builtin_conflict = registry.has_builtin_dotted_prefix(&import.alias);
let plugin_conflict = registry.has_external_dotted_prefix(&import.alias);
if builtin_conflict || plugin_conflict {
conflicted.insert(import.alias.clone());
let owner = if builtin_conflict {
"a builtin dotted namespace"
} else {
"a declared plugin export namespace"
};
diagnostics.push((
module.id.clone(),
Diagnostic::error(
codes::EXT_IMPORT_NAMESPACE_CONFLICT,
Some(import.span),
format!("import alias `{}` conflicts with {owner}", import.alias),
)
.with_hint(format!(
"rename the import alias, for example `{}_lib`, so `{}.…` has one meaning",
import.alias, import.alias
)),
));
}
}
let mut bound = BoundProgram::default();
for function in &module.functions {
let mut locals: BTreeSet<String> = function
.params
@@ -47,73 +162,50 @@ pub fn validate_module(module: &Module) -> Vec<Diagnostic> {
.map(|param| param.name.clone())
.collect();
collect_assigned(&function.body, &mut locals);
let mut check = CallChecker {
current_namespace: function.namespace.clone(),
let mut binder = Binder {
module,
program,
registry,
function_id: function.id.clone(),
namespace: function.namespace.clone(),
locals: &locals,
table: &table,
imports: module
.imports
.iter()
.map(|import| import.alias.clone())
.collect(),
module_id: module.id.clone(),
program: None,
cross_only: false,
standalone,
conflicted: &conflicted,
bindings: CallBindingTable::default(),
diagnostics: Vec::new(),
};
check.stmts(&function.body);
diagnostics.extend(check.diagnostics);
}
diagnostics
}
/// Validate cross-module and qualified-builtin calls once a whole program is
/// loaded. Local calls are already validated by [`validate_module`].
pub fn validate_program(program: &Program) -> Vec<(ModuleId, Diagnostic)> {
let mut results = Vec::new();
for module in program.modules.values() {
let table = signature_table(module);
for function in &module.functions {
let mut locals: BTreeSet<String> = function
.params
.iter()
.map(|param| param.name.clone())
.collect();
collect_assigned(&function.body, &mut locals);
let mut check = CallChecker {
current_namespace: function.namespace.clone(),
locals: &locals,
table: &table,
imports: module
.imports
.iter()
.map(|import| import.alias.clone())
.collect(),
module_id: module.id.clone(),
program: Some(program),
cross_only: true,
diagnostics: Vec::new(),
};
check.stmts(&function.body);
results.extend(
check
.diagnostics
.into_iter()
.map(|diagnostic| (module.id.clone(), diagnostic)),
);
for param in &function.params {
// Signature defaults are compile-time expressions evaluated by the
// runtime in the callee's own frame; bind their calls with the
// callee identity so the runtime lookup matches.
if let Some(default) = &param.default {
binder.expr(default);
}
}
binder.stmts(&function.body);
for (key, target) in binder.bindings.iter() {
bound.bindings.insert(key.clone(), target.clone());
}
diagnostics.extend(
binder
.diagnostics
.into_iter()
.map(|diagnostic| (module.id.clone(), diagnostic)),
);
}
BoundProgram {
bindings: bound.bindings,
diagnostics,
}
results
}
fn collect_assigned(stmts: &[Stmt], out: &mut BTreeSet<String>) {
for stmt in stmts {
match stmt {
Stmt::Assign {
target: PlaceTarget::Local { name, .. },
..
} => {
out.insert(name.clone());
Stmt::Assign { target, .. } => {
if let PlaceTarget::Local { name, .. } = target {
out.insert(name.clone());
}
}
Stmt::Branch { arms, .. } => {
for arm in arms {
@@ -121,23 +213,25 @@ fn collect_assigned(stmts: &[Stmt], out: &mut BTreeSet<String>) {
}
}
Stmt::Loop { body, .. } => collect_assigned(body, out),
_ => {}
Stmt::Eval { .. } | Stmt::Return { .. } => {}
}
}
}
struct CallChecker<'a> {
current_namespace: Option<String>,
struct Binder<'a> {
module: &'a Module,
program: &'a Program,
registry: &'a CallableRegistry,
function_id: FunctionId,
namespace: Option<String>,
locals: &'a BTreeSet<String>,
table: &'a BTreeMap<String, CallableSignature>,
imports: BTreeSet<String>,
module_id: ModuleId,
program: Option<&'a Program>,
cross_only: bool,
standalone: bool,
conflicted: &'a BTreeSet<String>,
bindings: CallBindingTable,
diagnostics: Vec<Diagnostic>,
}
impl<'a> CallChecker<'a> {
impl<'a> Binder<'a> {
fn stmts(&mut self, stmts: &[Stmt]) {
for stmt in stmts {
match stmt {
@@ -192,205 +286,309 @@ impl<'a> CallChecker<'a> {
}
}
fn bind(&mut self, span: Span, target: BoundTarget) {
let key = CallSiteKey::new(self.function_id.clone(), span);
self.bindings.insert(key, target);
}
fn diagnostic(&mut self, code: &str, span: Span, message: String, hint: Option<String>) {
let mut diagnostic = Diagnostic::error(code, Some(span), message);
if let Some(hint) = hint {
diagnostic = diagnostic.with_hint(hint);
}
self.diagnostics.push(diagnostic);
}
fn call(&mut self, callee: &Expr, args: &[Arg], span: Span) {
match callee {
Expr::Ident(Ident { name, .. }) => {
if self.cross_only {
Expr::Ident(Ident {
name,
span: name_span,
}) => {
if self.locals.contains(name) {
self.bind(span, BoundTarget::DynamicValue);
return;
}
if let Some(target) = self.registry.alias_target(name).cloned() {
if let Some(descriptor) = self.registry.builtin(name) {
self.validate_signature(
&descriptor.signature,
args,
name,
span,
*name_span,
);
}
self.bind(span, BoundTarget::Marqdo { function: target });
return;
}
if let Some(builtin) = builtins::find(name) {
if let Err(message) = builtins::validate_args(builtin, args) {
self.diagnostics.push(
Diagnostic::error(codes::CALL_ARITY, Some(span), message)
.with_hint("check the builtin parameter list"),
self.diagnostic(
codes::CALL_ARITY,
span,
message,
Some("check the builtin parameter list".to_string()),
);
}
self.bind(
span,
BoundTarget::Builtin {
canonical_name: builtin.name.to_string(),
},
);
return;
}
let name_span = match callee {
Expr::Ident(Ident { span, .. }) => *span,
_ => span,
};
if let Some(signature) = self.resolve(name) {
self.validate_against(name, signature, args, span, name_span);
return;
match self.resolve_module_function(name) {
Resolution::One(function) => {
if let Some(target) = self.program.function(&function) {
let signature = marqdo_ir::callable::CallableSignature {
params: target.params.clone(),
kind: target.signature,
};
self.validate_signature(&signature, args, name, span, *name_span);
}
self.bind(span, BoundTarget::Marqdo { function });
}
Resolution::Ambiguous => {
self.diagnostic(
codes::SYMBOL_UNKNOWN,
*name_span,
format!("ambiguous function name `{name}`"),
Some(
"qualify it with its namespace, for example `namespace.函数(...)`"
.to_string(),
),
);
}
Resolution::None => {
if self.standalone
&& self
.module
.imports
.iter()
.any(|import| import.alias == *name)
{
// A local may hold import-related values in the
// standalone frontend API; defer.
return;
}
self.diagnostic(
codes::SYMBOL_UNKNOWN,
*name_span,
format!("unknown function `{name}`"),
Some(
"define it as a level-2 heading, call a builtin, or load the \
plugin that declares it"
.to_string(),
),
);
}
}
if self.locals.contains(name) || self.imports.contains(name) {
// A local may hold a FunctionRef value; resolved at runtime.
return;
}
self.diagnostics.push(
Diagnostic::error(
codes::SYMBOL_UNKNOWN,
Some(name_span),
format!("unknown function `{name}`"),
)
.with_hint("define it as a level-2 heading or call a builtin"),
);
}
Expr::Member { base, name, .. } => {
if let Expr::Ident(Ident { name: alias, .. }) = base.as_ref() {
let qualified = format!("{}.{name}", alias);
if let Some(builtin) = builtins::find(&qualified) {
if let Err(message) = builtins::validate_args(builtin, args) {
self.diagnostics.push(
Diagnostic::error(codes::CALL_ARITY, Some(span), message)
.with_hint("check the builtin parameter list"),
if let Expr::Ident(Ident {
name: alias,
span: alias_span,
}) = base.as_ref()
{
if self.locals.contains(alias) {
self.bind(span, BoundTarget::DynamicValue);
return;
}
if self.conflicted.contains(alias) {
// The import conflict diagnostic already tells the user
// that `alias.…` has no single meaning; do not cascade.
return;
}
if let Some(target_module) =
self.program.resolve_import(&self.module.id, alias).cloned()
{
if let Some(function) = self.registry.function_id(&target_module, name) {
if let Some(target) = self.program.function(&function) {
let signature = marqdo_ir::callable::CallableSignature {
params: target.params.clone(),
kind: target.signature,
};
self.validate_signature(
&signature,
args,
&format!("{alias}.{name}"),
span,
span,
);
}
self.bind(span, BoundTarget::Marqdo { function });
} else {
self.diagnostic(
codes::SYMBOL_UNKNOWN,
span,
format!("module `{alias}` has no function `{name}`"),
Some(format!(
"check `inspect --signature` on the imported module `{alias}`"
)),
);
}
return;
}
if self.imports.contains(alias) {
if let Some(program) = self.program {
if let Some(target_module) =
program.resolve_import(&self.module_id, alias)
{
if let Some(target) = program.module(target_module) {
if let Some(function) = target.function(name) {
let signature = CallableSignature {
params: function.params.clone(),
kind: function.signature,
};
self.validate_against(
&qualified, &signature, args, span, span,
);
return;
}
self.diagnostics.push(
Diagnostic::error(
codes::SYMBOL_UNKNOWN,
Some(span),
format!("module `{alias}` has no function `{name}`"),
)
.with_hint(
"check `inspect --signature` on the imported module",
),
);
return;
let qualified = format!("{alias}.{name}");
if let Some(builtin) = builtins::find(&qualified) {
if let Err(message) = builtins::validate_args(builtin, args) {
self.diagnostic(
codes::CALL_ARITY,
span,
message,
Some("check the builtin parameter list".to_string()),
);
}
self.bind(
span,
BoundTarget::Builtin {
canonical_name: builtin.name.to_string(),
},
);
return;
}
if let Some(descriptor) = self.registry.external(&qualified) {
self.validate_signature(
&descriptor.signature,
args,
&qualified,
span,
span,
);
let param_names = descriptor
.signature
.params
.iter()
.map(|param| param.name.clone())
.collect();
let target = match &descriptor.origin {
marqdo_ir::callable::CallableOrigin::AbiPlugin { package, export } => {
BoundTarget::AbiPlugin {
package: package.clone(),
export: export.clone(),
digest: descriptor
.implementation_digest
.clone()
.unwrap_or_default(),
param_names,
}
}
}
_ => BoundTarget::DynamicValue,
};
self.bind(span, target);
return;
}
// A declared ABI export is a static descriptor: validate
// its signature here, without loading the shared library.
if let Some(program) = self.program {
if let Some(descriptor) = program.external_callable(&qualified) {
if let Err(message) =
descriptor.signature.validate_args(args, &qualified)
{
self.diagnostics.push(
Diagnostic::error(codes::CALL_ARITY, Some(span), message)
.with_hint("check the plugin manifest export signature"),
);
}
return;
}
if self.standalone {
// Without a program graph an import or plugin export
// cannot be distinguished statically; defer instead of
// emitting a false error.
return;
}
self.diagnostic(
codes::SYMBOL_UNKNOWN,
*alias_span,
format!("unknown function or plugin export `{qualified}`"),
Some(
"declare the export in a plugin manifest and pass `--plugin` for \
`run`, or fix the module import alias"
.to_string(),
),
);
return;
}
self.bind(span, BoundTarget::DynamicValue);
self.expr(base);
}
other => self.expr(other),
}
}
fn resolve(&self, name: &str) -> Option<&'a CallableSignature> {
if let Some(namespace) = &self.current_namespace {
if let Some(found) = self.table.get(&format!("{namespace}.{name}")) {
return Some(found);
other => {
self.bind(span, BoundTarget::DynamicValue);
self.expr(other);
}
}
if let Some(found) = self.table.get(name) {
return Some(found);
}
// Unique local name across namespaces.
let mut matches = self
.table
.iter()
.filter(|(key, _)| key.ends_with(&format!(".{name}")))
.map(|(_, signature)| signature);
let first = matches.next()?;
if matches.next().is_some() {
None
} else {
Some(first)
}
}
fn validate_against(
fn validate_signature(
&mut self,
name: &str,
signature: &CallableSignature,
signature: &marqdo_ir::callable::CallableSignature,
args: &[Arg],
display: &str,
call_span: Span,
name_span: Span,
) {
if signature.kind == SignatureKind::Inferred {
if let Some(positional) = args.iter().find(|arg| arg.name.is_none()) {
self.diagnostics.push(
Diagnostic::error(
codes::CALL_ARITY,
Some(positional.span),
format!(
"`{name}` has an inferred signature and only accepts named arguments"
),
)
.with_hint("pass arguments as `name = value`"),
);
if let Err(message) = signature.validate_args(args, display) {
// Positional overflow and missing arguments point at the call or
// the function name respectively; keep messages deterministic.
let span = if message.contains("missing required") {
name_span
} else {
call_span
};
self.diagnostic(
codes::CALL_ARITY,
span,
message,
Some("check `inspect --signature` or the plugin manifest signature".to_string()),
);
}
}
fn resolve_module_function(&self, name: &str) -> Resolution {
if let Some(namespace) = &self.namespace {
let canonical = format!("{namespace}.{name}");
if let Some(function) = self
.module
.functions
.iter()
.find(|function| function.id.name == canonical)
{
return Resolution::One(function.id.clone());
}
}
let mut supplied: BTreeSet<&str> = BTreeSet::new();
let mut positional = 0usize;
for arg in args {
match &arg.name {
Some(argument_name) => {
if !signature
.params
.iter()
.any(|param| param.name == *argument_name)
{
self.diagnostics.push(
Diagnostic::error(
codes::CALL_ARITY,
Some(arg.span),
format!("`{name}` has no parameter named `{argument_name}`"),
)
.with_hint("check `inspect --signature`"),
);
}
supplied.insert(argument_name.as_str());
}
None => {
if let Some(param) = signature.params.get(positional) {
supplied.insert(param.name.as_str());
positional += 1;
} else {
self.diagnostics.push(Diagnostic::error(
codes::CALL_ARITY,
Some(call_span),
format!(
"`{name}` accepts at most {} positional argument(s)",
signature.params.len()
),
));
}
}
}
if let Some(function) = self
.module
.functions
.iter()
.find(|function| function.id.name == name)
{
return Resolution::One(function.id.clone());
}
for param in &signature.params {
if param.default.is_none() && !supplied.contains(param.name.as_str()) {
self.diagnostics.push(
Diagnostic::error(
codes::CALL_ARITY,
Some(name_span),
format!("`{name}` is missing required argument `{}`", param.name),
)
.with_hint("pass it by name, e.g. `x = value`"),
);
}
let mut matches = self
.module
.functions
.iter()
.filter(|function| function.local_name == name);
match (matches.next(), matches.next()) {
(Some(function), None) => Resolution::One(function.id.clone()),
(None, _) => Resolution::None,
(Some(_), Some(_)) => Resolution::Ambiguous,
}
}
}
enum Resolution {
One(FunctionId),
Ambiguous,
None,
}
/// Whether a descriptor is a statically declared ABI plugin export.
pub fn is_abi_export(descriptor: &CallableDescriptor) -> bool {
matches!(
descriptor.origin,
marqdo_ir::callable::CallableOrigin::AbiPlugin { .. }
)
}
/// Whether a function signature requires explicit named/positional checking.
pub fn _signature_kind(signature: &marqdo_ir::callable::CallableSignature) -> SignatureKind {
signature.kind
}
/// Helper used by tests and by `inspect` to render a function body trace.
#[allow(dead_code)]
fn _unused(_: &Function, _: &Program) {}
#[cfg(test)]
mod tests {
use super::*;
@@ -494,11 +692,36 @@ mod tests {
package: "demo".into(),
export: "solver.add".into(),
},
implementation_digest: None,
implementation_digest: Some("a".repeat(64)),
source_span: None,
});
let diagnostics = validate_program(&program);
assert!(diagnostics.iter().any(|(_, d)| d.code == codes::CALL_ARITY));
let registry = CallableRegistry::try_from_program(&program).unwrap();
let bound = bind_program_calls(&program, &registry);
assert!(bound
.diagnostics
.iter()
.any(|(_, d)| d.code == codes::CALL_ARITY));
// A valid positional/named mix binds to the exact export identity.
let module = make_module(vec![function(
"main",
&[],
SignatureKind::Explicit,
vec![return_call("solver.add(1, b=2)")],
)]);
program.modules.insert(module.id.clone(), module);
let registry = CallableRegistry::try_from_program(&program).unwrap();
let bound = bind_program_calls(&program, &registry);
assert!(bound.diagnostics.is_empty(), "{:?}", bound.diagnostics);
let target = bound.bindings.iter().next().unwrap().1;
match target {
BoundTarget::AbiPlugin {
package, export, ..
} => {
assert_eq!(package, "demo");
assert_eq!(export, "solver.add");
}
other => panic!("unexpected target {other:?}"),
}
}
#[test]
@@ -556,4 +779,31 @@ mod tests {
.iter()
.any(|d| d.code == codes::CALL_ARITY));
}
#[test]
fn import_alias_conflicting_with_builtin_namespace_is_rejected() {
let module = Module {
id: ModuleId::from("index.mq.md"),
units: vec![],
functions: vec![function(
"main",
&[],
SignatureKind::Explicit,
vec![return_call("math.eval(x=1)")],
)],
imports: vec![marqdo_ir::ImportDecl {
alias: "math".into(),
target: "./math.mq.md".into(),
span: Span::new(0, 1),
}],
resources: vec![],
anchors: std::collections::BTreeMap::new(),
};
let diagnostics = validate_module(&module);
assert!(diagnostics
.iter()
.any(|d| d.code == codes::EXT_IMPORT_NAMESPACE_CONFLICT));
// Standalone mode defers unresolved imports, but the composition root
// run is covered by the CLI black-box CALL-01 test.
}
}
+1 -1
View File
@@ -14,5 +14,5 @@ pub mod resources;
pub mod signature;
pub mod units;
pub use analyze::{analyze_source, AnalysisResult};
pub use analyze::{analyze_source, analyze_source_for_program, AnalysisResult};
pub use units::{build_units, github_slug, FunctionScope, HeadingInfo, UnitIndex};
+434 -84
View File
@@ -1,12 +1,19 @@
//! Capability model.
//! Capability model and the trusted capability-handler registry.
//!
//! A project manifest can *request* capabilities, but requests are not grants.
//! Effective permissions are the host default for the sandbox mode intersected
//! with what the operator/CLI explicitly granted. Every host operation checks
//! against this set at the point of execution.
//!
//! Executable vendor/core capabilities additionally need an explicitly
//! registered trusted [`CapabilityHandler`] (ADR-0057, RFC-EXT-0001 §7.2).
//! Being authorized is never the same as being executable: a request without a
//! handler fails with `capability.unavailable` instead of returning an empty
//! successful reply.
use std::collections::BTreeSet;
use std::collections::{BTreeMap, BTreeSet};
use std::sync::{Arc, Mutex};
use std::time::Instant;
use glob::Pattern;
use marqdo_plugin_abi::abi::MAX_CAPABILITY_REPLY_BYTES;
@@ -14,8 +21,33 @@ use marqdo_plugin_abi::PluginError;
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, RuntimeError};
use crate::budget::Budget;
use crate::fs::ProjectFs;
/// Core capability names understood by [`CapabilityBroker::decide`].
///
/// This is the closed set of host primitives (ADR-0011). A trusted embedder
/// may install a [`CapabilityHandler`] for one of these names; until it does,
/// the capability is authorized by policy but *unavailable* at execution.
pub const CORE_CAPABILITIES: [&str; 10] = [
"console",
"math.compute",
"fs.read",
"fs.write",
"net.request",
"process",
"clock",
"random",
"log",
"artifact.record",
];
/// Capability namespace prefixes reserved for the host itself.
///
/// Handler registration rejects these prefixes so a vendor package can never
/// shadow a host-internal operation.
pub const RESERVED_HANDLER_PREFIXES: [&str; 2] = ["marqdo.", "host."];
/// One audited capability request from a plugin.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CapabilityAudit {
@@ -27,19 +59,143 @@ pub struct CapabilityAudit {
pub decision: &'static str,
/// Human-readable reason.
pub reason: String,
/// Wall-clock duration of the decision/execution in microseconds.
pub duration_micros: u64,
}
/// Broker for plugin host-capability requests (ADR-0011).
/// One capability request handed to an installed [`CapabilityHandler`].
#[derive(Debug, Clone, Copy)]
pub struct CapabilityRequest<'a> {
/// Trusted package identity supplied by the host, never by plugin wire data.
pub package_instance: &'a str,
/// Capability being requested.
pub capability: &'a str,
/// Raw request payload (JSON for the predefined capabilities).
pub payload: &'a [u8],
/// Run/session trace id used to correlate audit entries with a run.
pub trace_id: &'a str,
}
/// Trusted, explicitly registered executor for one capability.
///
/// Core capabilities are checked against the effective grant set. Vendor
/// capabilities are accepted only when a trusted host policy registered the
/// exact name; a declaration in a manifest is never enough.
#[derive(Debug, Clone)]
/// RFC-EXT-0001 §7.2 names the policy parameter `EffectiveGrant` and the budget
/// `ResourceBudget`; this build deliberately reuses the existing
/// [`Capabilities`] grant set and [`Budget`] instead of introducing a second
/// permission system (the RFC explicitly allows that substitution).
///
/// A handler is only reachable after [`CapabilityBroker::decide`] authorizes
/// the request, but it must still verify its own scope (project-relative paths,
/// globs, per-file limits) and must charge work to `budget` where applicable.
/// Registration is host-only: there is no wire path from a plugin to this
/// registry.
pub trait CapabilityHandler: Send {
/// Stable capability name this handler serves.
fn name(&self) -> &str;
/// Execute one already-authorized request.
fn execute(
&mut self,
request: &CapabilityRequest<'_>,
capabilities: &Capabilities,
budget: &mut Budget,
) -> Result<Vec<u8>, PluginError>;
}
/// Built-in handler for granted `fs.read` requests.
///
/// Reads go through [`ProjectFs`] (root confinement, traversal rejection,
/// regular-file/UTF-8/8 MiB limits) and the grant is checked **again** against
/// the normalized project-relative path at the execution point, so a raw
/// payload cannot win the glob check with a path the filesystem later
/// normalizes differently.
pub struct FsReadHandler {
fs: ProjectFs,
}
impl FsReadHandler {
/// Install the built-in handler over one project filesystem view.
pub fn new(fs: ProjectFs) -> Self {
Self { fs }
}
}
impl CapabilityHandler for FsReadHandler {
fn name(&self) -> &str {
"fs.read"
}
fn execute(
&mut self,
request: &CapabilityRequest<'_>,
capabilities: &Capabilities,
_budget: &mut Budget,
) -> Result<Vec<u8>, PluginError> {
// Reply-size limits are enforced by the broker, so this handler keeps
// the historical per-read behavior and only needs the grant re-check.
let raw_path = request_path(request.payload)?;
let (_, relative) = self.fs.resolve(&raw_path, None).map_err(map_runtime)?;
capabilities
.check_read(&relative, None)
.map_err(map_runtime)?;
let contents = self.fs.read(&raw_path, None).map_err(map_runtime)?;
Ok(contents.into_bytes())
}
}
struct HandlerEntry {
handler: Box<dyn CapabilityHandler>,
/// Exact trusted package binding. `None` means the handler is authorized
/// by the capability grant alone (for example project-scoped `fs.read`).
package: Option<String>,
}
/// Broker for plugin host-capability requests (ADR-0011, ADR-0057).
///
/// The broker keeps two independent sets: trusted vendor *policy* entries and
/// installed *handlers*. Execution is
///
/// ```text
/// decide (grant + policy + payload shape)
/// -> handler installed? (else capability.unavailable)
/// -> package-bound handler owns the claimed package? (else permission.denied)
/// -> handler executes under the broker budget and reply limit
/// -> audit { package, capability, granted|denied|unavailable, reason, duration }
/// ```
///
/// A capability that is merely authorized never executes: there is no
/// empty-success path.
pub struct CapabilityBroker {
capabilities: Capabilities,
vendor_policy: BTreeSet<String>,
handlers: BTreeMap<String, HandlerEntry>,
reply_limit: usize,
budget: Budget,
trace_id: String,
audit: Arc<Mutex<Vec<CapabilityAudit>>>,
fs: Option<ProjectFs>,
}
impl std::fmt::Debug for CapabilityBroker {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let bound: Vec<(&str, &str)> = self
.handlers
.iter()
.filter_map(|(name, entry)| {
entry
.package
.as_deref()
.map(|package| (name.as_str(), package))
})
.collect();
formatter
.debug_struct("CapabilityBroker")
.field("capabilities", &self.capabilities)
.field("vendor_policy", &self.vendor_policy)
.field("handlers", &self.handlers.keys().collect::<Vec<_>>())
.field("package_bound_handlers", &bound)
.field("reply_limit", &self.reply_limit)
.field("budget", &self.budget)
.field("trace_id", &self.trace_id)
.finish()
}
}
impl CapabilityBroker {
@@ -53,19 +209,116 @@ impl CapabilityBroker {
Self {
capabilities,
vendor_policy: BTreeSet::new(),
handlers: BTreeMap::new(),
reply_limit: MAX_CAPABILITY_REPLY_BYTES,
budget: Budget::default(),
trace_id: String::new(),
audit,
fs: None,
}
}
/// Attach the project filesystem used to execute granted `fs.read`
/// requests. Without it, `fs.read` is authorized but unavailable.
/// Attach the project filesystem and install the trusted built-in
/// `fs.read` handler.
///
/// Without an attached filesystem, granted `fs.read` requests stay
/// authorized but have no handler, so execution returns
/// `capability.unavailable` (never an empty success).
///
/// # Panics
///
/// Panics if a handler for `fs.read` was already registered on this broker,
/// which would make the built-in handler ambiguous.
pub fn with_fs(mut self, fs: ProjectFs) -> Self {
self.fs = Some(fs);
self.register_handler(FsReadHandler::new(fs))
.expect("a fresh broker has no fs.read handler to conflict with");
self
}
/// Override the maximum reply size a handler may return.
pub fn with_reply_limit(mut self, bytes: usize) -> Self {
self.reply_limit = bytes;
self
}
/// Attach the per-run budget that handlers charge work to.
pub fn with_budget(mut self, budget: Budget) -> Self {
self.budget = budget;
self
}
/// Set the run/session trace id surfaced to handlers.
pub fn with_trace_id(mut self, trace_id: impl Into<String>) -> Self {
self.trace_id = trace_id.into();
self
}
/// Remaining broker budget (diagnostics and tests).
pub fn budget(&self) -> &Budget {
&self.budget
}
/// Install a trusted handler reachable by any package that holds a grant.
///
/// Registration is a host-only operation; a plugin manifest cannot reach
/// this API. Duplicate, reserved and empty/invalid names are rejected.
pub fn register_handler<H>(&mut self, handler: H) -> Result<(), PluginError>
where
H: CapabilityHandler + 'static,
{
self.install_handler(handler, None)
}
/// Install a trusted handler bound to one exact trusted package identity.
///
/// A request that claims any other package is `permission.denied`, even
/// when the capability grant and vendor policy match.
pub fn register_package_handler<H>(
&mut self,
package: impl Into<String>,
handler: H,
) -> Result<(), PluginError>
where
H: CapabilityHandler + 'static,
{
self.install_handler(handler, Some(package.into()))
}
fn install_handler<H>(&mut self, handler: H, package: Option<String>) -> Result<(), PluginError>
where
H: CapabilityHandler + 'static,
{
let name = handler.name().to_string();
validate_handler_name(&name)?;
if self.handlers.contains_key(&name) {
return Err(PluginError::capability(
"capability.handler_conflict",
format!("a capability handler is already registered for `{name}`"),
));
}
self.handlers.insert(
name,
HandlerEntry {
handler: Box::new(handler),
package,
},
);
Ok(())
}
/// Names with an installed trusted handler, in deterministic order.
pub fn handler_names(&self) -> Vec<&str> {
self.handlers.keys().map(String::as_str).collect()
}
/// Whether a trusted executing handler is installed for `capability`.
pub fn has_handler(&self, capability: &str) -> bool {
self.handlers.contains_key(capability)
}
/// Register a trusted vendor capability policy entry.
///
/// This is policy only: execution additionally requires an installed
/// handler. Registering a vendor name does not install anything.
pub fn allow_vendor(&mut self, capability: impl Into<String>) {
self.vendor_policy.insert(capability.into());
}
@@ -76,103 +329,132 @@ impl CapabilityBroker {
}
/// Authorize one request, returning reply bytes on success.
///
/// This is the grant decision only; it does not prove that an executor is
/// installed. Call [`Self::execute`] to actually run a capability.
pub fn authorize(
&mut self,
package: &str,
capability: &str,
request: &[u8],
) -> Result<Vec<u8>, PluginError> {
let started = Instant::now();
let result = self.decide(package, capability, request);
let (decision, reason) = match &result {
Ok(()) => ("granted", String::new()),
Err(error) => ("denied", format!("{}: {}", error.code, error.message)),
};
if let Ok(mut audit) = self.audit.lock() {
audit.push(CapabilityAudit {
package: package.to_string(),
capability: capability.to_string(),
decision,
reason,
});
}
self.record(package, capability, decision, &reason, started);
result.map(|()| Vec::new())
}
/// Authorize and execute one capability request.
/// Authorize and execute one capability request through its trusted
/// handler.
///
/// Only `fs.read` has a real implementation in this build. Other
/// capabilities return `capability.unavailable` instead of pretending an
/// empty reply means success (ADR-0011, spec §12.2).
/// The chain is fixed: `decide` → handler installed → package binding →
/// handler execution under the broker budget and reply limit → audit.
/// Anything without an installed handler is `capability.unavailable`; it
/// is never reported as an empty success (ADR-0011, ADR-0057).
pub fn execute(
&mut self,
package: &str,
capability: &str,
request: &[u8],
) -> Result<Vec<u8>, PluginError> {
let started = Instant::now();
// 1. Authorization: effective grants, vendor policy, payload shape.
if let Err(error) = self.decide(package, capability, request) {
let reason = format!("{}: {}", error.code, error.message);
self.record(package, capability, "denied", &reason);
self.record(package, capability, "denied", &reason, started);
return Err(error);
}
let result = match capability {
"fs.read" => {
let path = request_path(request)?;
let Some(fs) = &self.fs else {
return self.unavailable(
package,
capability,
"no project filesystem is attached",
);
};
match fs.read(&path, None) {
Ok(contents) => {
if contents.len() > MAX_CAPABILITY_REPLY_BYTES {
return self.unavailable(
package,
capability,
"the file exceeds the capability reply limit",
);
}
Ok(contents.into_bytes())
}
Err(error) => {
let error = map_runtime(error);
let reason = format!("{}: {}", error.code, error.message);
self.record(package, capability, "denied", &reason);
Err(error)
}
}
// 2. A trusted executor must exist; a package-bound executor must own
// the trusted package identity the host is claiming.
let bound_package = match self.handlers.get(capability) {
Some(entry) => entry.package.clone(),
None => {
let message = "this capability has no executing handler in this build";
self.record(package, capability, "unavailable", message, started);
return Err(PluginError::capability(
"capability.unavailable",
message.to_string(),
));
}
_ => self.unavailable(
package,
capability,
"this capability has no executing handler in this build",
),
};
if result.is_ok() {
self.record(package, capability, "granted", "");
if let Some(bound) = bound_package {
if bound != package {
let error = plugin_denied(&format!(
"capability `{capability}` is bound to another trusted package"
));
let reason = format!("{}: {}", error.code, error.message);
self.record(package, capability, "denied", &reason, started);
return Err(error);
}
}
// 3. Execute under the broker budget and reply limit.
let broker_request = CapabilityRequest {
package_instance: package,
capability,
payload: request,
trace_id: &self.trace_id,
};
let reply_limit = self.reply_limit;
let outcome = self
.handlers
.get_mut(capability)
.expect("handler presence was checked above")
.handler
.execute(&broker_request, &self.capabilities, &mut self.budget);
match outcome {
Ok(contents) => {
if contents.len() > reply_limit {
let error = PluginError::capability(
"resource.exhausted",
format!(
"capability reply is {} bytes; the limit is {reply_limit} bytes",
contents.len()
),
);
let reason = format!("{}: {}", error.code, error.message);
self.record(package, capability, "denied", &reason, started);
return Err(error);
}
self.record(package, capability, "granted", "", started);
Ok(contents)
}
Err(error) => {
let reason = format!("{}: {}", error.code, error.message);
let decision = if error.code == "capability.unavailable" {
"unavailable"
} else {
"denied"
};
self.record(package, capability, decision, &reason, started);
Err(error)
}
}
result
}
fn unavailable<T>(
fn record(
&mut self,
package: &str,
capability: &str,
message: &str,
) -> Result<T, PluginError> {
let error = PluginError::capability("capability.unavailable", message.to_string());
self.record(package, capability, "unavailable", message);
Err(error)
}
fn record(&mut self, package: &str, capability: &str, decision: &'static str, reason: &str) {
decision: &'static str,
reason: &str,
started: Instant,
) {
let elapsed = started.elapsed();
let duration_micros = u64::try_from(elapsed.as_micros()).unwrap_or(u64::MAX);
if let Ok(mut audit) = self.audit.lock() {
audit.push(CapabilityAudit {
package: package.to_string(),
capability: capability.to_string(),
decision,
reason: reason.to_string(),
duration_micros,
});
}
}
@@ -214,17 +496,14 @@ impl CapabilityBroker {
.then_some(())
.ok_or_else(|| plugin_denied("process capability was not granted")),
other => {
if is_vendor_capability(other)
&& (self.vendor_policy.contains(other) || self.vendor_policy.contains("*"))
{
Ok(())
} else if is_vendor_capability(other) {
Err(PluginError::capability(
"capability.unavailable",
format!(
"vendor capability `{other}` has no trusted handler for package `{package}`"
),
))
if is_vendor_capability(other) {
if self.vendor_policy.contains(other) || self.vendor_policy.contains("*") {
Ok(())
} else {
Err(plugin_denied(&format!(
"vendor capability `{other}` has no trusted policy entry for package `{package}`"
)))
}
} else {
Err(PluginError::capability(
"capability.unknown",
@@ -304,6 +583,38 @@ pub fn is_vendor_capability(name: &str) -> bool {
})
}
/// Validate a trusted handler name before registration.
///
/// Registration uses [`CapabilityHandler::name`] as the registry key, so there
/// is no second name argument that could disagree with the handler's declared
/// capability (the RFC's "declared protocol mismatch" rejection).
fn validate_handler_name(name: &str) -> Result<(), PluginError> {
if name.is_empty() || name.trim() != name {
return Err(PluginError::capability(
"capability.handler_invalid",
"capability handler names must be non-empty and must not contain surrounding whitespace",
));
}
if RESERVED_HANDLER_PREFIXES
.iter()
.any(|prefix| name.starts_with(prefix))
{
return Err(PluginError::capability(
"capability.handler_reserved",
format!("capability handler name `{name}` uses a reserved host namespace"),
));
}
if !CORE_CAPABILITIES.contains(&name) && !is_vendor_capability(name) {
return Err(PluginError::capability(
"capability.handler_invalid",
format!(
"`{name}` is not a registrable capability name: expected a core capability or a vendor-namespaced name"
),
));
}
Ok(())
}
/// Effective capability grants for one run.
#[derive(Debug, Clone, Default)]
pub struct Capabilities {
@@ -594,20 +905,56 @@ mod tests {
.unwrap_err();
assert_eq!(denied.code, "permission.denied");
assert!(broker.authorize("demo.solver", "fs.read", b"{}").is_err());
let unavailable = broker
// Declaring a vendor capability is not a grant: without a trusted
// policy entry authorization is denied (RFC-EXT-0001 CAP-01).
let vendor_denied = broker
.authorize("demo.solver", "vendor.device.read", b"{}")
.unwrap_err();
assert_eq!(unavailable.code, "capability.unavailable");
assert_eq!(vendor_denied.code, "permission.denied");
broker.allow_vendor("vendor.device.read");
assert!(broker
.authorize("demo.solver", "vendor.device.read", b"{}")
.is_ok());
// Policy is not a handler: authorize() succeeds but execution still
// needs an executor and reports `capability.unavailable`.
assert!(!broker.has_handler("vendor.device.read"));
assert_eq!(
broker
.execute("demo.solver", "vendor.device.read", b"{}")
.unwrap_err()
.code,
"capability.unavailable"
);
assert!(!is_vendor_capability("vendor"));
assert!(!is_vendor_capability("fs.read.extra"));
let audit = broker.audit();
let audit = audit.lock().unwrap();
assert_eq!(audit.len(), 5);
assert_eq!(audit.len(), 6);
assert!(audit.iter().any(|entry| entry.decision == "denied"));
assert!(audit.iter().any(|entry| entry.decision == "unavailable"));
assert!(audit.iter().all(|entry| entry.duration_micros <= 5_000_000));
}
#[test]
fn every_core_capability_is_understood_by_decide() {
for name in CORE_CAPABILITIES {
let broker = CapabilityBroker::new(Capabilities::project());
let payload: &[u8] = match name {
"fs.read" => br#"{"path":"data/a.csv"}"#,
"fs.write" => br#"{"path":"data/a.csv","contents":"x"}"#,
"net.request" => br#"{"host":"127.0.0.1"}"#,
_ => b"{}",
};
let error = broker
.decide("demo.solver", name, payload)
.err()
.map(|error| error.code);
assert_ne!(
error.as_deref(),
Some("capability.unknown"),
"core capability `{name}` must be handled by decide"
);
}
}
#[test]
@@ -638,6 +985,9 @@ mod execute_tests {
.allow_read("data/**")
.allow_write("data/**");
let mut broker = CapabilityBroker::new(capabilities).with_fs(fs);
assert!(broker.has_handler("fs.read"));
assert_eq!(broker.handler_names(), vec!["fs.read"]);
assert_eq!(broker.reply_limit, MAX_CAPABILITY_REPLY_BYTES);
let contents = broker
.execute("demo.solver", "fs.read", br#"{"path":"data/allowed.txt"}"#)
+4 -1
View File
@@ -19,7 +19,10 @@ pub mod resource;
pub mod trace;
pub use budget::{Budget, DEFAULT_FUEL, DEFAULT_OUTPUT_BYTES, DEFAULT_STACK_LIMIT};
pub use capability::Capabilities;
pub use capability::{
Capabilities, CapabilityAudit, CapabilityBroker, CapabilityHandler, CapabilityRequest,
FsReadHandler, CORE_CAPABILITIES, RESERVED_HANDLER_PREFIXES,
};
pub use context::ExecutionContext;
pub use fs::ProjectFs;
pub use handles::{contains_plugin_handle, PluginHandleTable};
@@ -0,0 +1,581 @@
//! Trusted capability-handler registry (RFC-EXT-0001 §7, ADR-0057).
//!
//! These are end-to-end tests over the public `marqdo-host` API. They cover the
//! CAP-01..CAP-12 checks listed in RFC-EXT-0001 §7.5: policy vs handler,
//! package binding, registration validation, real `fs.read` scope enforcement,
//! reply budgets, forged/replayed handles, audit hygiene, resource-stream
//! regression, project-vs-strict labeling and the "no handler is never a
//! silent OK" rule.
use std::sync::atomic::{AtomicU32, Ordering};
use std::time::{Duration, Instant};
use marqdo_host::{
contains_plugin_handle, Budget, Capabilities, CapabilityBroker, CapabilityHandler,
CapabilityRequest, PluginHandleTable, ProjectFs, ResourceTable, SandboxMode, StdHost,
CORE_CAPABILITIES,
};
use marqdo_plugin_abi::PluginError;
use marqdo_value::{RuntimeError, Value};
static COUNTER: AtomicU32 = AtomicU32::new(0);
struct TempDir(std::path::PathBuf);
impl TempDir {
fn new() -> Self {
let id = COUNTER.fetch_add(1, Ordering::Relaxed);
let path = std::env::temp_dir().join(format!(
"marqdo-capability-handlers-{}-{id}",
std::process::id()
));
std::fs::create_dir_all(&path).unwrap();
Self(path)
}
fn path(&self) -> &std::path::Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
/// A trusted test handler that returns canned bytes.
struct FixedHandler {
name: &'static str,
reply: Vec<u8>,
}
impl CapabilityHandler for FixedHandler {
fn name(&self) -> &str {
self.name
}
fn execute(
&mut self,
_request: &CapabilityRequest<'_>,
_capabilities: &Capabilities,
_budget: &mut Budget,
) -> Result<Vec<u8>, PluginError> {
Ok(self.reply.clone())
}
}
/// A trusted test handler that proves the request identity and budget reach it.
struct EchoHandler {
name: &'static str,
fuel: u64,
}
impl CapabilityHandler for EchoHandler {
fn name(&self) -> &str {
self.name
}
fn execute(
&mut self,
request: &CapabilityRequest<'_>,
_capabilities: &Capabilities,
budget: &mut Budget,
) -> Result<Vec<u8>, PluginError> {
budget
.consume_fuel(self.fuel, None)
.map_err(map_runtime_error)?;
Ok(format!(
"{}|{}|{}",
request.package_instance, request.capability, request.trace_id
)
.into_bytes())
}
}
fn map_runtime_error(error: RuntimeError) -> PluginError {
PluginError::capability(error.code(), error.message)
}
fn audit_entries(broker: &CapabilityBroker) -> Vec<marqdo_host::CapabilityAudit> {
let audit = broker.audit();
let guard = audit.lock().expect("audit lock");
guard.clone()
}
/// CAP-01: a declared vendor capability without a trusted policy entry is
/// denied, even when a handler happens to be installed.
#[test]
fn cap_01_vendor_capability_without_policy_is_denied() {
let mut broker = CapabilityBroker::new(Capabilities::project());
broker
.register_handler(FixedHandler {
name: "vendor.device.read",
reply: b"device".to_vec(),
})
.unwrap();
let error = broker
.execute("demo.solver", "vendor.device.read", b"{}")
.unwrap_err();
assert_eq!(error.code, "permission.denied", "{error:?}");
assert!(broker
.authorize("demo.solver", "vendor.device.read", b"{}")
.is_err());
let audit = audit_entries(&broker);
assert_eq!(audit.len(), 2);
assert!(audit.iter().all(|entry| entry.decision == "denied"));
assert!(audit
.iter()
.all(|entry| entry.capability == "vendor.device.read"));
}
/// CAP-02: authorized but no handler installed is `capability.unavailable`;
/// `authorize` alone does not execute and cannot fake an empty success.
#[test]
fn cap_02_authorized_capability_without_handler_is_unavailable() {
let mut broker = CapabilityBroker::new(Capabilities::project().allow_write("out/**"));
broker.allow_vendor("vendor.device.read");
let unavailable = broker
.execute("demo.solver", "vendor.device.read", b"{}")
.unwrap_err();
assert_eq!(
unavailable.code, "capability.unavailable",
"{unavailable:?}"
);
// Policy is a grant, not an executor: authorize() succeeds, execute() still
// refuses because no trusted handler was registered.
assert!(broker
.authorize(
"demo.solver",
"fs.write",
br#"{"path":"out/result.txt","contents":"x"}"#,
)
.is_ok());
let write = broker
.execute(
"demo.solver",
"fs.write",
br#"{"path":"out/result.txt","contents":"x"}"#,
)
.unwrap_err();
assert_eq!(write.code, "capability.unavailable", "{write:?}");
let audit = audit_entries(&broker);
assert!(audit
.iter()
.any(|entry| entry.decision == "unavailable" && entry.capability == "fs.write"));
assert!(audit
.iter()
.any(|entry| entry.decision == "granted" && entry.capability == "fs.write"));
}
/// CAP-03: a handler bound to one trusted package rejects cross-package
/// impersonation, while the bound package still executes and the request
/// identity/trace id reach the handler.
#[test]
fn cap_03_package_bound_handler_rejects_impersonation() {
let mut broker = CapabilityBroker::new(Capabilities::project()).with_trace_id("trace-9");
broker.allow_vendor("vendor.device.read");
broker
.register_package_handler(
"demo.solver",
EchoHandler {
name: "vendor.device.read",
fuel: 0,
},
)
.unwrap();
let spoofed = broker
.execute("attacker.solver", "vendor.device.read", b"{}")
.unwrap_err();
assert_eq!(spoofed.code, "permission.denied", "{spoofed:?}");
let reply = broker
.execute("demo.solver", "vendor.device.read", b"{}")
.unwrap();
assert_eq!(
String::from_utf8(reply).unwrap(),
"demo.solver|vendor.device.read|trace-9"
);
let audit = audit_entries(&broker);
assert!(audit
.iter()
.any(|entry| entry.decision == "denied" && entry.package == "attacker.solver"));
assert!(audit
.iter()
.any(|entry| entry.decision == "granted" && entry.package == "demo.solver"));
}
/// CAP-04: duplicate, reserved, empty and malformed handler names are rejected
/// and never installed.
#[test]
fn cap_04_registration_rejects_duplicate_reserved_and_empty_names() {
let mut broker = CapabilityBroker::new(Capabilities::project());
let rejected = |broker: &mut CapabilityBroker, name: &'static str, expected: &str| {
let error = broker
.register_handler(FixedHandler {
name,
reply: Vec::new(),
})
.unwrap_err();
assert_eq!(error.code, expected, "name `{name}`: {error:?}");
assert!(!broker.has_handler(name));
};
rejected(&mut broker, "", "capability.handler_invalid");
rejected(&mut broker, "host.terminal", "capability.handler_reserved");
rejected(
&mut broker,
"marqdo.internal.read",
"capability.handler_reserved",
);
rejected(
&mut broker,
"not a capability",
"capability.handler_invalid",
);
rejected(&mut broker, "vendor..read", "capability.handler_invalid");
broker
.register_handler(FixedHandler {
name: "vendor.device.read",
reply: b"device".to_vec(),
})
.unwrap();
let duplicate = broker
.register_handler(FixedHandler {
name: "vendor.device.read",
reply: b"other".to_vec(),
})
.unwrap_err();
assert_eq!(
duplicate.code, "capability.handler_conflict",
"{duplicate:?}"
);
// The trusted built-in fs.read handler cannot be replaced either.
let dir = TempDir::new();
let mut with_fs =
CapabilityBroker::new(Capabilities::project()).with_fs(ProjectFs::new(dir.path()));
assert!(with_fs.has_handler("fs.read"));
let conflict = with_fs
.register_handler(FixedHandler {
name: "fs.read",
reply: Vec::new(),
})
.unwrap_err();
assert_eq!(conflict.code, "capability.handler_conflict", "{conflict:?}");
}
/// CAP-05: the real `fs.read` handler enforces ProjectFs root/scope limits; a
/// permissive glob grant cannot be used to read outside the project root or to
/// traverse with `..`.
#[test]
fn cap_05_fs_read_handler_enforces_scope() {
let dir = TempDir::new();
std::fs::create_dir_all(dir.path().join("data")).unwrap();
std::fs::write(dir.path().join("data/allowed.txt"), "allowed").unwrap();
std::fs::create_dir_all(dir.path().join("private")).unwrap();
std::fs::write(dir.path().join("private/secret.txt"), "secret").unwrap();
let mut broker = CapabilityBroker::new(Capabilities::project().allow_read("data/**"))
.with_fs(ProjectFs::new(dir.path()));
let allowed = broker
.execute("demo.solver", "fs.read", br#"{"path":"data/allowed.txt"}"#)
.unwrap();
assert_eq!(String::from_utf8(allowed).unwrap(), "allowed");
for out_of_scope in [
br#"{"path":"private/secret.txt"}"#.as_slice(),
br#"{"path":"../escape.txt"}"#.as_slice(),
br#"{"path":"/etc/passwd"}"#.as_slice(),
] {
let error = broker
.execute("demo.solver", "fs.read", out_of_scope)
.unwrap_err();
assert_eq!(
error.code, "permission.denied",
"{out_of_scope:?}: {error:?}"
);
}
// Even a `**` grant only authorizes project-relative paths: traversal and
// absolute paths are rejected by ProjectFs::resolve at the handler.
let mut permissive = CapabilityBroker::new(Capabilities::project().allow_read("**"))
.with_fs(ProjectFs::new(dir.path()));
for rejected in [
br#"{"path":"data/../private/secret.txt"}"#.as_slice(),
br#"{"path":"/etc/passwd"}"#.as_slice(),
] {
let error = permissive
.execute("demo.solver", "fs.read", rejected)
.unwrap_err();
assert_eq!(error.code, "permission.denied", "{rejected:?}: {error:?}");
}
}
/// CAP-06: a handler reply over the broker budget is a resource limit, and the
/// handler receives (and charges) the broker budget.
#[test]
fn cap_06_handler_reply_over_budget_is_resource_limit() {
let mut broker = CapabilityBroker::new(Capabilities::project())
.with_reply_limit(64)
.with_budget(Budget {
fuel: 10,
..Budget::default()
});
broker.allow_vendor("vendor.device.big");
broker.allow_vendor("vendor.device.fuel");
broker
.register_handler(FixedHandler {
name: "vendor.device.big",
reply: vec![b'x'; 65],
})
.unwrap();
broker
.register_handler(EchoHandler {
name: "vendor.device.fuel",
fuel: 4,
})
.unwrap();
let oversized = broker
.execute("demo.solver", "vendor.device.big", b"{}")
.unwrap_err();
assert_eq!(oversized.code, "resource.exhausted", "{oversized:?}");
let reply = broker
.execute("demo.solver", "vendor.device.fuel", b"{}")
.unwrap();
assert_eq!(
String::from_utf8(reply).unwrap(),
"demo.solver|vendor.device.fuel|"
);
assert_eq!(broker.budget().remaining_fuel(), 6);
let audit = audit_entries(&broker);
assert!(audit.iter().all(|entry| entry.duration_micros <= 5_000_000));
assert!(audit
.iter()
.any(|entry| entry.capability == "vendor.device.big" && entry.decision == "denied"));
}
/// CAP-07: forged plugin handles are rejected by the host table, and a handle
/// hidden inside an untrusted extension value is detected.
#[test]
fn cap_07_forged_handles_are_rejected() {
let mut table = PluginHandleTable::default();
let handle = table.mint("demo.capability", "capability.read", "demo.capability", 1);
table
.validate(&handle, "demo.capability", 1)
.expect("host-minted handle is valid");
let mut forged = handle.clone();
forged.id = handle.id.wrapping_add(1000);
let error = table
.validate(&forged, "demo.capability", 1)
.expect_err("forged id must be rejected");
assert!(
error.message.contains("plugin_handle.untrusted"),
"{error:?}"
);
let wrong_owner = table
.validate(&handle, "other.package", 1)
.expect_err("cross-owner handle must be rejected");
assert!(
wrong_owner.message.contains("plugin_handle.untrusted"),
"{wrong_owner:?}"
);
let untrusted_value = Value::Map(
[("handle".to_string(), Value::PluginHandle(handle.clone()))]
.into_iter()
.collect(),
);
assert!(contains_plugin_handle(&untrusted_value));
}
/// CAP-08: a handle from an old run or a cleared (unloaded) run is rejected on
/// replay.
#[test]
fn cap_08_replayed_handle_after_run_end_is_rejected() {
let mut table = PluginHandleTable::default();
let handle = table.mint("demo.capability", "capability.read", "demo.capability", 7);
table
.validate(&handle, "demo.capability", 7)
.expect("current run accepts its own handle");
let new_run = table
.validate(&handle, "demo.capability", 8)
.expect_err("a new run must not accept the old handle");
assert!(
new_run.message.contains("plugin_handle.untrusted"),
"{new_run:?}"
);
table.clear_run(7);
assert!(table.is_empty());
let replayed = table
.validate(&handle, "demo.capability", 7)
.expect_err("a cleared run must reject the replayed handle");
assert!(
replayed.message.contains("plugin_handle.untrusted"),
"{replayed:?}"
);
}
/// CAP-09: the capability audit records decisions and durations but never file
/// contents or request secrets.
#[test]
fn cap_09_audit_contains_no_file_contents_or_secrets() {
const SECRET: &str = "TOP-SECRET-PAYLOAD";
const API_KEY: &str = "sk-live-not-audited";
let dir = TempDir::new();
std::fs::create_dir_all(dir.path().join("data")).unwrap();
std::fs::write(dir.path().join("data/secret.txt"), SECRET).unwrap();
let mut broker = CapabilityBroker::new(Capabilities::project().allow_read("data/**"))
.with_fs(ProjectFs::new(dir.path()));
let request = format!(r#"{{"path":"data/secret.txt","api_key":"{API_KEY}"}}"#);
let contents = broker
.execute("demo.solver", "fs.read", request.as_bytes())
.unwrap();
assert_eq!(String::from_utf8(contents).unwrap(), SECRET);
let denied = broker
.execute("demo.solver", "fs.read", br#"{"path":"private/key.txt"}"#)
.unwrap_err();
assert_eq!(denied.code, "permission.denied");
let audit_debug = format!("{:?}", audit_entries(&broker));
assert!(!audit_debug.contains(SECRET), "{audit_debug}");
assert!(!audit_debug.contains(API_KEY), "{audit_debug}");
let audit = audit_entries(&broker);
assert!(audit
.iter()
.any(|entry| entry.decision == "granted" && entry.capability == "fs.read"));
assert!(audit
.iter()
.any(|entry| entry.decision == "denied" && entry.capability == "fs.read"));
assert!(audit.iter().all(|entry| entry.duration_micros <= 5_000_000));
}
/// CAP-10: low-level resource open/read/write/close stays on a sane performance
/// path. This is a catastrophic-regression guard with a deliberately generous
/// bound, not a benchmark (CI timeouts remain authoritative).
#[test]
fn cap_10_resource_stream_operations_do_not_regress_catastrophically() {
const CHUNK: usize = 256 * 1024;
const ITERATIONS: usize = 32;
let payload = vec![b'x'; CHUNK];
let mut table = ResourceTable::default();
let started = Instant::now();
for _ in 0..ITERATIONS {
let reader = table.open_file("data/blob.bin", payload.clone());
let read = table.read(reader, payload.len(), None).unwrap();
assert_eq!(read.bytes.len(), CHUNK);
assert!(read.eof);
table.close(reader, None).unwrap();
let writer = table.open_file_writer("out/blob.bin");
assert_eq!(table.write(writer, &payload, None).unwrap(), CHUNK);
table.close(writer, None).unwrap();
}
let elapsed = started.elapsed();
// 32 x (256 KiB read + 256 KiB write) is ~16 MiB of in-memory copying.
assert!(
elapsed < Duration::from_secs(5),
"resource open/read/write/close took {elapsed:?} for {ITERATIONS} iterations"
);
}
/// CAP-11: the default `project` mode is not an OS-strict mode and grants
/// nothing beyond `pure` by default.
#[test]
fn cap_11_default_project_mode_is_not_os_strict() {
let dir = TempDir::new();
let host = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
assert_eq!(host.mode, SandboxMode::Project);
assert_ne!(host.mode, SandboxMode::Strict);
assert_eq!(host.mode.as_str(), "project");
// Project defaults equal pure defaults: no implicit fs/process/plugin grant.
assert!(host.capabilities.fs_read.is_empty());
assert!(host.capabilities.fs_write.is_empty());
assert!(!host.capabilities.process);
assert!(!host.capabilities.foreign);
assert!(host.capabilities.plugins.is_empty());
// Strict isolation availability is a property of `strict`; project mode is
// cooperative in-process checks, whether or not the platform could isolate.
assert_eq!(
SandboxMode::parse(host.mode.as_str()),
Some(SandboxMode::Project)
);
}
/// CAP-12: core capabilities without an executing handler are always
/// `capability.unavailable`, never a silent empty success.
#[test]
fn cap_12_core_capabilities_without_handler_are_unavailable() {
let dir = TempDir::new();
std::fs::create_dir_all(dir.path().join("data")).unwrap();
std::fs::write(dir.path().join("data/ok.txt"), "ok").unwrap();
let mut capabilities = Capabilities::project()
.allow_read("data/**")
.allow_write("data/**")
.allow_net("127.0.0.1")
.allow_process("/bin/true");
capabilities.process = true;
let mut broker = CapabilityBroker::new(capabilities).with_fs(ProjectFs::new(dir.path()));
// The one migrated capability has a real handler.
let read = broker
.execute("demo.solver", "fs.read", br#"{"path":"data/ok.txt"}"#)
.unwrap();
assert_eq!(String::from_utf8(read).unwrap(), "ok");
// Every other core capability is authorized by its grant but has no
// executor in this build.
for name in CORE_CAPABILITIES {
if name == "fs.read" {
continue;
}
let payload: &[u8] = match name {
"fs.write" => br#"{"path":"data/x","contents":"x"}"#,
"net.request" => br#"{"host":"127.0.0.1"}"#,
_ => b"{}",
};
let error = broker.execute("demo.solver", name, payload).unwrap_err();
assert_eq!(
error.code, "capability.unavailable",
"core capability `{name}` must not silently succeed: {error:?}"
);
assert!(!broker.has_handler(name));
}
let unknown = broker
.execute("demo.solver", "teleport", b"{}")
.unwrap_err();
assert_eq!(unknown.code, "capability.unknown", "{unknown:?}");
let audit = audit_entries(&broker);
assert!(
audit
.iter()
.filter(|entry| entry.decision == "unavailable")
.count()
>= CORE_CAPABILITIES.len() - 1,
"{audit:?}"
);
}
+201
View File
@@ -0,0 +1,201 @@
//! Resolved static call bindings (RFC-EXT-0001 §4.2, ADR-0054).
//!
//! The frontend resolves every statically visible call site exactly once and
//! records the decision here. The runtime only dispatches by this table: it
//! never re-resolves a qualified name and never guesses a plugin from a string
//! prefix. The table is internal IR (not part of Analysis/Trace v2 wire
//! contracts) and is deliberately skipped when a [`crate::Program`] is
//! serialized.
//!
//! Identity rules:
//!
//! * a [`CallSiteKey`] is `(module, function, byte span)` and is therefore
//! independent of disk paths, registration order and hash-map iteration;
//! * a bound target names the *resolved implementation*, including the plugin
//! package, complete export name and verified artifact digest for ABI calls;
//! * [`BoundTarget::DynamicValue`] is the only non-static path and may only
//! call language values (for example a `FunctionRef`); it is never a back
//! door to an unknown ABI export.
use std::collections::BTreeMap;
use std::fmt;
use serde::{Deserialize, Serialize};
use crate::{FunctionId, ModuleId};
/// Stable identity of one static call site.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
pub struct CallSiteKey {
/// Module containing the call.
pub module: ModuleId,
/// Function containing the call.
pub function: FunctionId,
/// First byte offset of the call expression.
pub span_start: usize,
/// End byte offset (exclusive) of the call expression.
pub span_end: usize,
}
impl CallSiteKey {
/// Construct a key from a function identity and an expression span.
pub fn new(function: FunctionId, span: marqdo_span::Span) -> Self {
Self {
module: function.module.clone(),
function,
span_start: span.start,
span_end: span.end,
}
}
}
impl fmt::Display for CallSiteKey {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"{}@{}..{}",
self.function, self.span_start, self.span_end
)
}
}
/// Resolved implementation target for a call site.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum BoundTarget {
/// A fixed language builtin, by canonical builtin name.
Builtin {
/// Canonical name (for example `math.eval`), not a user alias.
canonical_name: String,
},
/// A function defined by a Markdown heading.
Marqdo {
/// Exact function identity.
function: FunctionId,
},
/// A statically declared ABI plugin export.
AbiPlugin {
/// Plugin package that owns the export.
package: String,
/// Complete dotted export name.
export: String,
/// Verified artifact digest from the selected manifest artifact.
digest: String,
/// Declared parameter names in manifest order; used to normalise
/// positional arguments deterministically for the ABI v1 map wire.
param_names: Vec<String>,
},
/// The callee is a language value resolved at runtime (local variable or
/// an expression result). Only language callables such as `FunctionRef`
/// are valid; this path cannot invoke an ABI plugin.
DynamicValue,
}
impl BoundTarget {
/// Whether this binding names a statically known implementation.
pub fn is_static(&self) -> bool {
!matches!(self, BoundTarget::DynamicValue)
}
}
/// One deterministic binding table for a prepared program.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct CallBindingTable {
sites: BTreeMap<CallSiteKey, BoundTarget>,
}
impl CallBindingTable {
/// Insert a binding. Returns `false` when the site was already bound to a
/// different target; re-binding the same target is idempotent.
pub fn insert(&mut self, key: CallSiteKey, target: BoundTarget) -> bool {
match self.sites.get(&key) {
Some(existing) => existing == &target,
None => {
self.sites.insert(key, target);
true
}
}
}
/// Look up the bound target for a call site.
pub fn get(&self, key: &CallSiteKey) -> Option<&BoundTarget> {
self.sites.get(key)
}
/// Number of bound call sites.
pub fn len(&self) -> usize {
self.sites.len()
}
/// Whether no call site is bound.
pub fn is_empty(&self) -> bool {
self.sites.is_empty()
}
/// Iterate bindings deterministically (call site order).
pub fn iter(&self) -> impl Iterator<Item = (&CallSiteKey, &BoundTarget)> {
self.sites.iter()
}
/// Number of statically bound (non-dynamic) call sites.
pub fn static_count(&self) -> usize {
self.sites
.values()
.filter(|target| target.is_static())
.count()
}
/// Number of dynamically resolved call sites.
pub fn dynamic_count(&self) -> usize {
self.sites.len() - self.static_count()
}
}
#[cfg(test)]
mod tests {
use super::*;
use marqdo_span::Span;
fn function(name: &str) -> FunctionId {
FunctionId {
module: ModuleId::root_default("index.mq.md"),
name: name.to_string(),
}
}
fn key(name: &str, span: Span) -> CallSiteKey {
CallSiteKey::new(function(name), span)
}
#[test]
fn keys_are_deterministic_and_span_based() {
let left = key("f", Span::new(10, 20));
let right = key("f", Span::new(10, 20));
let other = key("f", Span::new(21, 30));
assert_eq!(left, right);
assert_ne!(left, other);
assert_eq!(left.to_string(), "root:default::index.mq.md::f@10..20");
}
#[test]
fn insert_is_idempotent_but_rejects_conflicting_targets() {
let mut table = CallBindingTable::default();
let site = key("f", Span::new(1, 2));
assert!(table.insert(
site.clone(),
BoundTarget::Builtin {
canonical_name: "长度".into()
}
));
assert!(table.insert(
site.clone(),
BoundTarget::Builtin {
canonical_name: "长度".into()
}
));
assert!(!table.insert(site, BoundTarget::DynamicValue));
assert_eq!(table.len(), 1);
assert_eq!(table.static_count(), 1);
assert_eq!(table.dynamic_count(), 0);
}
}
+148 -8
View File
@@ -10,7 +10,7 @@ use std::fmt;
use serde::{Deserialize, Serialize};
use crate::{builtins, Function, Module, ModuleId, Param, Program, SignatureKind};
use crate::{builtins, Function, FunctionId, Module, ModuleId, Param, Program, SignatureKind};
use marqdo_span::Span;
/// Globally unique callable identity.
@@ -138,6 +138,42 @@ pub struct CallableRegistry {
builtins: BTreeMap<String, CallableDescriptor>,
functions: BTreeMap<ModuleId, BTreeMap<String, CallableDescriptor>>,
external: BTreeMap<String, CallableDescriptor>,
/// Official stdlib public aliases (for example `打印`/`print`) mapped to
/// the bundled Markdown function that implements them. Only the verified
/// stdlib loader may populate this table; third-party packages cannot.
official_aliases: BTreeMap<String, FunctionId>,
}
/// One official bundled-stdlib public alias.
#[derive(Debug, Clone, PartialEq)]
pub struct OfficialAlias {
/// Public names that resolve to the same target (for example both
/// `打印` and `print`).
pub public_names: Vec<String>,
/// Bundled Marqdo function implementing the alias.
pub function: FunctionId,
/// Effective signature; the loader must have checked it against the
/// verified stdlib metadata before registration.
pub signature: CallableSignature,
/// Owner package instance (the verified stdlib instance).
pub owner_package: String,
/// Conservative effect request summary for Trace/Analysis.
pub effects_requested: Vec<String>,
}
/// A deterministic static-table conflict.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CallableConflict {
/// Colliding public name.
pub public_name: String,
/// Human-readable reason; never used as a machine code.
pub reason: String,
}
impl fmt::Display for CallableConflict {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "`{}`: {}", self.public_name, self.reason)
}
}
impl CallableRegistry {
@@ -150,21 +186,46 @@ impl CallableRegistry {
}
/// Build a registry for all modules and declared external descriptors.
pub fn from_program(program: &Program) -> Self {
///
/// Duplicate external names are rejected rather than silently keeping the
/// first entry: a static table must never depend on registration order.
pub fn try_from_program(program: &Program) -> Result<Self, CallableConflict> {
let mut registry = Self::default();
registry.add_builtins();
for module in program.modules.values() {
registry.add_module(module);
}
for descriptor in &program.external_callables {
// Conflicting declarations are reported by the loader; keep the
// first deterministic entry here rather than overwriting silently.
registry
.external
.entry(descriptor.public_name.clone())
.or_insert_with(|| descriptor.clone());
.insert_external(descriptor.clone())
.map_err(|reason| CallableConflict {
public_name: descriptor.public_name.clone(),
reason,
})?;
}
registry
Ok(registry)
}
/// Build a registry for all modules and declared external descriptors.
///
/// Prefer [`CallableRegistry::try_from_program`] on the CLI composition
/// path; this compatibility wrapper keeps the first declaration for
/// already-validated callers (tests and embedders).
pub fn from_program(program: &Program) -> Self {
Self::try_from_program(program).unwrap_or_else(|_| {
let mut registry = Self::default();
registry.add_builtins();
for module in program.modules.values() {
registry.add_module(module);
}
for descriptor in &program.external_callables {
registry
.external
.entry(descriptor.public_name.clone())
.or_insert_with(|| descriptor.clone());
}
registry
})
}
/// Look up a builtin by canonical name or alias.
@@ -188,6 +249,85 @@ impl CallableRegistry {
self.external.values()
}
/// Register one official bundled-stdlib public alias.
///
/// The loader calls this only after verifying the bundled stdlib digest and
/// the metadata digest, so a third-party package can never claim `打印`,
/// `print` or any other official public name through this path.
pub fn insert_official_alias(&mut self, alias: OfficialAlias) -> Result<(), String> {
if alias.public_names.is_empty() {
return Err("official alias must declare at least one public name".to_string());
}
if alias.function.module.instance.is_empty() || alias.function.name.is_empty() {
return Err("official alias target function identity must not be empty".to_string());
}
for name in &alias.public_names {
if name.is_empty() {
return Err("official alias public name must not be empty".to_string());
}
if self.external.contains_key(name) {
return Err(format!(
"official alias `{name}` conflicts with a declared plugin export"
));
}
let descriptor = CallableDescriptor {
id: CallableId(format!("marqdo:{}", alias.function)),
public_name: name.clone(),
owner_package: alias.owner_package.clone(),
signature: alias.signature.clone(),
effects_requested: alias.effects_requested.clone(),
origin: CallableOrigin::Marqdo,
implementation_digest: None,
source_span: None,
};
// Official aliases intentionally replace the reserved placeholder
// builtin (for example `打印`); no third-party path can do this.
self.builtins.insert(name.clone(), descriptor);
self.official_aliases
.insert(name.clone(), alias.function.clone());
}
Ok(())
}
/// Target Marqdo function of an official alias, if `name` is one.
pub fn alias_target(&self, name: &str) -> Option<&FunctionId> {
self.official_aliases.get(name)
}
/// Iterate official aliases deterministically (public name, target).
pub fn official_aliases(&self) -> impl Iterator<Item = (&String, &FunctionId)> {
self.official_aliases.iter()
}
/// Whether a dotted builtin namespace starts with `prefix.`.
pub fn has_builtin_dotted_prefix(&self, prefix: &str) -> bool {
self.builtins.keys().any(|name| {
name.strip_prefix(prefix)
.is_some_and(|rest| rest.starts_with('.'))
})
}
/// Whether a declared plugin export starts with `prefix.`.
pub fn has_external_dotted_prefix(&self, prefix: &str) -> bool {
self.external.keys().any(|name| {
name.strip_prefix(prefix)
.is_some_and(|rest| rest.starts_with('.'))
})
}
/// Resolve a module function to its exact identity.
pub fn function_id(&self, module: &ModuleId, name: &str) -> Option<FunctionId> {
let descriptor = self.function(module, name)?;
if matches!(descriptor.origin, CallableOrigin::Marqdo) {
Some(FunctionId {
module: module.clone(),
name: descriptor.public_name.clone(),
})
} else {
None
}
}
/// Insert an external descriptor, rejecting duplicate public names.
pub fn insert_external(&mut self, descriptor: CallableDescriptor) -> Result<(), String> {
if self.builtins.contains_key(&descriptor.public_name) {
+21 -4
View File
@@ -17,6 +17,7 @@ use marqdo_span::Span;
use serde::{Deserialize, Serialize};
pub mod builtins;
pub mod call_binding;
pub mod callable;
/// Canonical logical identity of a module: package instance plus a path
@@ -476,10 +477,13 @@ pub struct Program {
/// metadata only: `check` never loads a library.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub external_callables: Vec<callable::CallableDescriptor>,
/// Compat-B/C `打印`/`print` destination (ADR-0030/0031). When set, the
/// runtime calls this stdlib function; there is no legacy HostOp path.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub print_override: Option<FunctionId>,
/// Resolved static call bindings (RFC-EXT-0001 §4.2, ADR-0054).
///
/// Internal IR: deliberately skipped on the wire so Analysis/Trace v2
/// consumers are unaffected. A prepared program always carries a complete
/// table; the runtime never re-resolves names.
#[serde(skip)]
pub call_bindings: call_binding::CallBindingTable,
}
impl Program {
@@ -517,6 +521,19 @@ impl Program {
pub fn callable_registry(&self) -> callable::CallableRegistry {
callable::CallableRegistry::from_program(self)
}
/// Resolved static call binding for one call site, if any.
pub fn call_binding(
&self,
key: &call_binding::CallSiteKey,
) -> Option<&call_binding::BoundTarget> {
self.call_bindings.get(key)
}
/// Install the prepared binding table.
pub fn set_call_bindings(&mut self, bindings: call_binding::CallBindingTable) {
self.call_bindings = bindings;
}
}
#[cfg(test)]
+79 -125
View File
@@ -54,7 +54,6 @@ pub struct RunOutcome {
/// Execution frame for one function call.
struct Frame {
function: FunctionId,
namespace: Option<String>,
locals: BTreeMap<String, Value>,
}
@@ -179,7 +178,6 @@ impl<'a> Interpreter<'a> {
};
let mut frame = Frame {
function: function.id.clone(),
namespace: function.namespace.clone(),
locals,
};
self.depth += 1;
@@ -277,7 +275,6 @@ impl<'a> Interpreter<'a> {
// an empty frame so they cannot reach caller state.
let mut empty = Frame {
function: function.id.clone(),
namespace: function.namespace.clone(),
locals: BTreeMap::new(),
};
let value = self.eval_expr(default, &mut empty)?;
@@ -616,113 +613,96 @@ impl<'a> Interpreter<'a> {
span: Span,
frame: &mut Frame,
) -> Result<Value, RuntimeError> {
let caller_module = frame.function.module.clone();
match callee {
Expr::Ident(ident) => {
if let Some(value) = frame.locals.get(&ident.name).cloned() {
let args = self.eval_args(args, frame)?;
return self.call_value(program, value, args, Some(span), Some(&caller_module));
}
if matches!(ident.name.as_str(), "打印" | "print") {
if let Some(target) = program.print_override.clone() {
let args = self.eval_args(args, frame)?;
let Some(function) = program.function(&target) else {
return Err(RuntimeError::new(
ErrorCategory::UnknownSymbol,
Some(span),
format!("print override `{target}` does not exist"),
));
};
return self.call_function(
program,
function,
args,
Some(span),
Some(&caller_module),
);
}
}
if let Some(builtin) = builtins::find(&ident.name) {
let args = self.eval_args(args, frame)?;
return self.call_builtin(builtin, args, Some(span), frame);
}
let args = self.eval_args(args, frame)?;
let (module, namespace) = (frame.function.module.clone(), frame.namespace.clone());
let Some(function) =
resolve_function(program, &module, namespace.as_deref(), &ident.name)
else {
// ADR-0054 / RFC-EXT-0001 §4: the frontend resolved this call site
// exactly once. Dispatch by the recorded identity; never re-resolve a
// name, never infer a plugin from a dotted prefix.
let key = marqdo_ir::call_binding::CallSiteKey::new(frame.function.clone(), span);
let Some(target) = program.call_bindings.get(&key).cloned() else {
return Err(RuntimeError::new(
ErrorCategory::Internal,
Some(span),
format!(
"call site `{key}` has no static binding; run a prepared program (ADR-0054)"
),
));
};
match target {
marqdo_ir::call_binding::BoundTarget::Builtin { canonical_name } => {
let Some(builtin) = builtins::find(&canonical_name) else {
return Err(RuntimeError::new(
ErrorCategory::UnknownSymbol,
ErrorCategory::Internal,
Some(span),
format!("unknown function `{}`", ident.name),
format!("bound builtin `{canonical_name}` is not registered"),
));
};
self.call_function(program, function, args, Some(span), Some(&caller_module))
let args = self.eval_args(args, frame)?;
self.call_builtin(builtin, args, Some(span), frame)
}
Expr::Member { base, name, .. } => {
if let Expr::Ident(alias) = base.as_ref() {
if let Some(target_module) =
program.resolve_import(&frame.function.module, &alias.name)
{
let args = self.eval_args(args, frame)?;
let Some(function) = resolve_function(program, target_module, None, name)
else {
return Err(RuntimeError::new(
ErrorCategory::UnknownSymbol,
Some(span),
format!("unknown function `{name}` in module `{target_module}`"),
));
};
return self.call_function(
program,
function,
args,
Some(span),
Some(&caller_module),
);
}
let qualified = format!("{}.{name}", alias.name);
if let Some(builtin) = builtins::find(&qualified) {
let args = self.eval_args(args, frame)?;
return self.call_builtin(builtin, args, Some(span), frame);
}
if !frame.locals.contains_key(&alias.name) {
// A declared external descriptor is the only static
// source of plugin exports. Calls without a descriptor
// are still accepted for compatibility only when no
// plugin descriptors exist at all; this fallback is
// removed once the CLI always supplies descriptors.
let qualified = format!("{}.{}", alias.name, name);
let declared = program.external_callable(&qualified).is_some();
if !declared && !program.external_callables.is_empty() {
return Err(RuntimeError::new(
ErrorCategory::UnknownSymbol,
Some(span),
format!("unknown plugin export `{qualified}`"),
));
}
// Plugin calls take named arguments only in this build.
let evaluated = self.eval_args(args, frame)?;
let mut named = std::collections::BTreeMap::new();
for argument in evaluated {
let Some(argument_name) = argument.name else {
marqdo_ir::call_binding::BoundTarget::Marqdo { function } => {
let Some(target_function) = program.function(&function) else {
return Err(RuntimeError::new(
ErrorCategory::Internal,
Some(span),
format!("bound function `{function}` does not exist"),
));
};
let args = self.eval_args(args, frame)?;
self.call_function(
program,
target_function,
args,
Some(span),
Some(&frame.function.module.clone()),
)
}
marqdo_ir::call_binding::BoundTarget::AbiPlugin {
package: _,
export,
digest: _,
param_names,
} => {
// ABI v1 transports named arguments as a map. Normalise
// positional arguments deterministically against the static
// manifest order; static validation already rejected unknown,
// duplicate or missing arguments.
let evaluated = self.eval_args(args, frame)?;
let mut named = std::collections::BTreeMap::new();
let mut position = 0usize;
for argument in evaluated {
let name = match argument.name {
Some(name) => name,
None => {
let Some(name) = param_names.get(position).cloned() else {
return Err(RuntimeError::new(
ErrorCategory::Type,
Some(argument.span),
"plugin calls require named arguments",
ErrorCategory::Internal,
Some(span),
format!(
"positional argument {} was not normalised for `{export}`",
position + 1
),
));
};
named.insert(argument_name, argument.value);
position += 1;
name
}
let module = frame.function.module.clone();
return self.host_invoke(
HostOp::PluginInvoke,
vec![Value::Text(qualified), Value::Map(named)],
};
if named.insert(name.clone(), argument.value).is_some() {
return Err(RuntimeError::new(
ErrorCategory::Arity,
Some(span),
&module,
);
format!("duplicate argument `{name}` for `{export}`"),
));
}
}
let module = frame.function.module.clone();
self.host_invoke(
HostOp::PluginInvoke,
vec![Value::Text(export), Value::Map(named)],
Some(span),
&module,
)
}
marqdo_ir::call_binding::BoundTarget::DynamicValue => {
let callee_value = self.eval_expr(callee, frame)?;
let args = self.eval_args(args, frame)?;
self.call_value(
@@ -730,18 +710,7 @@ impl<'a> Interpreter<'a> {
callee_value,
args,
Some(span),
Some(&caller_module),
)
}
other => {
let callee_value = self.eval_expr(other, frame)?;
let args = self.eval_args(args, frame)?;
self.call_value(
program,
callee_value,
args,
Some(span),
Some(&caller_module),
Some(&frame.function.module.clone()),
)
}
}
@@ -935,21 +904,6 @@ impl<'a> Interpreter<'a> {
}
}
fn resolve_function<'p>(
program: &'p Program,
module: &ModuleId,
namespace: Option<&str>,
name: &str,
) -> Option<&'p Function> {
let module = program.module(module)?;
if let Some(namespace) = namespace {
if let Some(found) = module.namespaced_function(namespace, name) {
return Some(found);
}
}
module.function(name)
}
/// Bind already-evaluated arguments to a builtin parameter list.
fn bind_builtin_args(
builtin: &'static builtins::Builtin,
@@ -19,6 +19,14 @@ fn build(source: &str) -> Program {
..Program::default()
};
program.insert(result.module);
let registry = program.callable_registry();
let bound = marqdo_frontend::calls::bind_program_calls(&program, &registry);
assert!(
bound.diagnostics.is_empty(),
"unexpected binding diagnostics: {:#?}",
bound.diagnostics
);
program.set_call_bindings(bound.bindings);
program
}
+28
View File
@@ -81,6 +81,34 @@ pub mod codes {
pub const PROJECT_LOCK: &str = "project.lock";
/// Dependency resolution failed.
pub const PROJECT_DEPENDENCY: &str = "project.dependency";
/// A statically visible callable export is declared more than once
/// (RFC-EXT-0001 §16.1).
pub const EXT_DUPLICATE_CALLABLE: &str = "ext.duplicate_callable";
/// An import alias collides with a builtin or plugin dotted namespace.
pub const EXT_IMPORT_NAMESPACE_CONFLICT: &str = "ext.import_namespace_conflict";
/// A loaded plugin's exports/signature/owner/artifact digest do not exactly
/// match the statically prepared descriptors.
pub const EXT_PLUGIN_BINDING_MISMATCH: &str = "ext.plugin_binding_mismatch";
/// Two resource providers try to register the same non-reserved kind.
pub const EXT_DUPLICATE_PROVIDER_KIND: &str = "ext.duplicate_provider_kind";
/// A legacy/future provider manifest is passed to the v2-only CLI entry.
pub const EXT_UNSUPPORTED_PROVIDER_SCHEMA: &str = "ext.unsupported_provider_schema";
/// No provider artifact matches the current host target triple.
pub const EXT_NO_MATCHING_ARTIFACT: &str = "ext.no_matching_artifact";
/// A provider artifact differs from its declared sha256, escapes its
/// bundle, is not a regular file or is an unallowed symlink.
pub const EXT_ARTIFACT_INTEGRITY: &str = "ext.artifact_integrity";
/// The matching provider worker/helper program is missing or cannot start.
pub const EXT_WORKER_UNAVAILABLE: &str = "ext.worker_unavailable";
/// The current platform explicitly does not support the requested
/// extension transport.
pub const EXT_PLATFORM_UNSUPPORTED: &str = "ext.platform_unsupported";
/// A locked extension identity does not match the actual manifest or
/// artifact on disk.
pub const EXT_EXTENSION_LOCK_MISMATCH: &str = "ext.extension_lock_mismatch";
/// An extension is explicitly selected but is not part of the project lock
/// (development-only use; release mode rejects it).
pub const EXT_UNLOCKED_EXTENSION: &str = "ext.unlocked_extension";
/// Internal invariant violation; never expected in normal operation.
pub const INTERNAL: &str = "internal.error";
}
@@ -0,0 +1,125 @@
# ADR-0054: One static callable binding, one runtime dispatch
- Status: Accepted (2026-11-21)
- Supersedes the resolution model of ADR-0009/ADR-0010 where they conflict
- Implements: RFC-EXT-0001 §4 (PR-01/PR-02)
## Context
The tree had four independent ways to decide what a call means:
1. `marqdo-frontend/src/calls.rs` checked a Member call's builtin name before
its import alias;
2. `marqdo-runtime/src/interpreter.rs` checked the import alias before the
builtin for the same expression;
3. the runtime treated an undeclared `alias.name` as a plugin invocation when
`program.external_callables` was empty ("try it at run time");
4. `marqdo-cli` synthesised an empty-parameter descriptor (`params: Vec::new()`)
for exports that were loaded but not statically described.
Consequences observed while building RFC-EXT-0001's fixtures:
* `check` could accept a call that `run` dispatched to a different
implementation (or vice versa);
* an unknown symbol failed only at run time, with a Host side effect already
attempted;
* an import alias that happened to equal a builtin dotted namespace (`math`)
silently changed the meaning of `math.eval(...)` depending on the pipeline;
* plugin arity/name errors could pass static checking and only fail (or
succeed with different arguments) at the ABI boundary.
The language contract (RFC-LANG-0002) already says calls are expressions and
that `check` and `run` must agree; the implementation had no such guarantee.
## Decision
1. **One registry.** `marqdo_ir::callable::CallableRegistry` is the only static
table. It contains builtins (including official bundled-stdlib public
aliases), module functions keyed by canonical name plus unique local name,
and exact plugin exports (`namespace.function`). Duplicate declarations are
rejected when the registry is built; order never decides the answer.
2. **One binding pass.** `marqdo_frontend::calls::bind_program_calls(program,
registry)` walks every statically visible call expression exactly once and
produces `CallBindingTable` plus diagnostics. It replaces the split
`validate_module`/`validate_program` priority logic on the CLI path (the
standalone frontend keeps a wrapper for single-document tests). The result
is installed on `Program::call_bindings`.
3. **Deterministic identity.** A call site is
`(module id, function id, byte span)` (`CallSiteKey`), never a disk path,
registration order or hash-map iteration. A bound target is one of:
* `Builtin { canonical_name }` — fixed core function;
* `Marqdo { function }` — exact `FunctionId` (module functions and official
stdlib aliases such as `打印`/`print` → `io.mq.md::写值`);
* `AbiPlugin { package, export, digest, param_names }` — complete export
name plus the statically verified artifact digest and manifest argument
order;
* `DynamicValue` — a language value (for example a `FunctionRef`); this is
the **only** non-static path and cannot reach an ABI export by name.
4. **Runtime dispatches by identity.** `eval_call` looks up the site's binding
and calls the recorded target. It never calls `builtins::find(qualified)`,
never inspects `external_callables.is_empty()` and never guesses a plugin.
A missing binding is an internal error (`internal.error`), not a reason to
re-resolve.
5. **Prepare finalises once.** `marqdo-cli` has one composition step
(`project_prepare::prepare`): load sources → read extension descriptors
(no `dlopen`/spawn) → build the registry → bind calls once. `check`,
`run`/`call`, `inspect` and Analysis consume that one prepared program.
Static plugin descriptors are inserted before binding, not appended after a
first validation.
6. **Exact run-time reconciliation.** `run` verifies each selected plugin
artifact, loads it, then compares the Host's actual exports (name, raw
signature, artifact digest, owner package) with the static descriptors
one-to-one. Any extra, missing or altered export fails with
`ext.plugin_binding_mismatch` before the entry function runs. No descriptor
is synthesised at run time.
7. **Print is not special.** `Program.print_override` and the runtime
`打印`/`print` branch are removed. The builtin placeholder remains only for
embedders without the official bundle and fails with
`kernel.unavailable`; the official bundle declares aliases in verified
metadata (ADR-0058).
## Priority matrix (normative)
See `spec/callable-resolution.md` for the complete matrix and diagnostics. In
short:
* `f(...)`: local variable → official alias/reserved builtin → current
namespace → canonical name → unique local name → otherwise
`symbol.unknown` (ambiguity is an error, never a first-match);
* `alias.f(...)`: explicit import alias → builtin dotted namespace → exact
plugin export → otherwise `symbol.unknown`;
* `some_value.f(...)`: dynamic language dispatch only.
Import aliases that collide with a builtin or plugin dotted namespace are
rejected at prepare with `ext.import_namespace_conflict`; duplicate plugin
exports are rejected with `ext.duplicate_callable`.
## Consequences
* `check` and `run` now share binding identity; CALL-01…CALL-16 in
`crates/cli/tests/callable_resolution.rs` pin the matrix.
* 0.x breaking change: a project importing a module under an alias that equals
a builtin namespace (for example `math`) must rename the alias. There is no
`--legacy` switch and no dual semantics.
* Analysis v2's `calls[].state/target` are now projected from the same binding
(official aliases appear as `function` targets). Analysis v2 schema and
fields are otherwise unchanged; private `CallBindingTable` is not serialized.
* Runtime no longer performs per-call name resolution, so the dynamic path is
limited to language values. A text or map value called as a function is
`call.not_callable` and never becomes a Host plugin request.
* Plugin manifests must be complete before binding: a manifest with a missing
or malformed signature is a prepare error; ABI v1 default values are
rejected until the wire semantics are versioned.
## Rejected alternatives
* **Pick a fixed precedence and keep two implementations.** Rejected: the RFC
requires one static answer, and duplicated precedence is exactly the defect.
* **Build the registry lazily in the runtime.** Rejected: `check` would still
not prove the binding, and runtime loading would change static resolution.
* **Keep the empty-`external_callables` fallback for "developer convenience".**
Rejected: it turns a compile error into a Host call attempt and breaks
`check`'s no-execution guarantee.
* **Expose `CallableId`/`CallBindingTable` in Analysis v2.** Rejected:
private IR must not become a public wire contract without a version bump.
EOF
+196
View File
@@ -0,0 +1,196 @@
# ADR-0055: Resource Provider v2 through the normal CLI
Status: **Accepted**
Date: 2026-11-21
Relates to: RFC-EXT-0001 §5/§16/§18, ADR-0033, ADR-0034, ADR-0035, ADR-0036,
ADR-0037, ADR-0038, ADR-0021 (strict L1), ADR-0052 (single Markdown semantics)
Specs: `spec/resource-provider-v2.md`
Implementation: `crates/cli/src/provider_manifest.rs`,
`crates/cli/src/commands/sandbox.rs`, `crates/cli/src/commands/run.rs`,
`crates/cli/src/main.rs`, `crates/cli/tests/provider_cli.rs`
## Context
The host already implements versioned resource-provider registration
(`register_resource_provider`, `register_worker_resource_provider`,
`register_c_resource_provider`, `register_provider_package`) and
`resource.open` dispatches unknown kinds through that registry with a provider
name grant (ADR-0033/0035/0036/0037/0038). What was missing was a **public
composition-root entry point**: `main.rs`/`SandboxSettings`/`run.rs` had no
`--provider`/`--allow-provider`, so an external repository could not use a new
resource kind on a frozen `marqdo` binary without patching and rebuilding the
CLI (RFC-EXT-0001 EXT-07).
The existing `marqdo.provider` manifest (`schema_version: 1`, ADR-0037) is a
host-embedding installation format: `name/kinds/abi/library`, no `version`, no
targeted artifact digest and no transport. It cannot support portable,
integrity-checked CLI loading. RFC-EXT-0001 §5.3 asks for a v2 document rather
than silently overloading v1. It also requires that selecting an extension is
not authorizing it, that `pure`/`strict` keep failing closed, and that no new
`HostOp`/builtin/business branch is introduced.
This ADR records the PR-03 decision on the current branch.
## Decision
### 1. CLI surface and ordering
`--provider PATH` (repeatable, a `provider.json` file) and
`--allow-provider NAME` (repeatable, exact name) are added to `SandboxArgs`, so
they are shared by `run` and `call` only. `check`/`inspect` do not declare them
and never read a provider manifest. `SandboxSettings` carries
`providers: Vec<PathBuf>` and `allow_provider: Vec<String>`; the `project`
capability set calls the existing `Capabilities::allow_resource_provider(name)`
for each grant.
Run ordering follows RFC-EXT-0001 §18.2: `prepare` (no provider work) →
diagnostics → mode/strict preflight → capabilities → host assembly and budgets
→ kernels → **all provider manifests/targets/artifacts/grants/duplicate kinds
verified and plugin artifacts verified** → plugin workers loaded and exports
reconciled → provider helpers spawned → entry-function check → execute. No
worker (plugin or provider) starts before every T6/T6b verification passes, and
the main function never runs after a provider failure.
`pure` and `strict` reject both flags in preflight (`cli.invalid_grants` /
`strict.unsupported_grant`, exit 2) without silently switching to `project`.
`--allow-provider` rejects wildcards (`*`, `?`, `[`, `]`) and empty names; a
grant that matches no selected manifest is `ext.provider_grant_unmatched`
(exit 2). A `--provider` selected without its exact grant is
`permission.denied` (exit 1) before the helper is spawned.
### 2. Strict v2 manifest parser
New `crates/cli/src/provider_manifest.rs` owns the document, separate from the
plugin `plugin.yaml` parser:
* accepts exactly `schema = "marqdo.provider"`, `schema_version = 2`,
`abi = "c-abi-1"`, `transport = "worker"`; other versions/ABIs/transports are
`ext.unsupported_provider_schema` (no silent migration, no v1 fallback from
the CLI);
* strict dotted lowercase name/kind grammar, strict SemVer, non-empty unique
non-reserved kinds, bounded counts, known/vendor `permissions_requested`
declarations only, optional non-executable `metadata`;
* unknown top-level/artifact fields are rejected (`ext.manifest_invalid`);
* artifacts need unique parseable targets and 64-lowercase-hex sha256;
the current `HostTarget` artifact is selected (`ext.no_matching_artifact`
when absent), resolved inside the manifest bundle (absolute/`..`/symlink
escape/non-regular/size limit/sha mismatch → `ext.artifact_integrity`);
* duplicate names with different manifest digests (`ext.duplicate_provider`)
and duplicate kinds across providers (`ext.duplicate_provider_kind`) are
deterministic because paths are canonicalised, sorted and de-duplicated
before conflict detection.
`VerifiedProvider` is the static `{name, kinds, manifest_path, base_dir,
manifest_digest, transport, selected_artifact_path/relative/sha256/target,
version}` record. It is still declaration/verification only: no `dlopen`, no
spawn, no grant.
### 3. Activation reuses the existing host API
`load_resource_provider_into_host(host, verified, allow_provider)` repeats the
exact grant check, re-verifies the manifest digest and the artifact
(regular file, containment, size, sha256) immediately before use, resolves the
helper, and calls **the existing**
`StdHost::register_worker_resource_provider(name, kinds, helper, [artifact])`.
It never falls back to `register_c_resource_provider`,
`register_provider_package`, in-process native loading or another provider.
`resource.open` still re-checks the provider name grant in the Host. Provider
registration keeps using `register_provider_checked` for name/kind/duplicate
validation, so no second provider registry is introduced.
### 4. Helper resolution and platform claim
The helper is resolved only as a sibling of the running `marqdo` executable:
`<exe dir>/marqdo-provider-worker[.exe]`. `PATH`, the current directory,
environment overrides and downloads are not consulted. Missing/non-regular
helper, spawn failure or failed `ping` → `ext.worker_unavailable` (exit 1).
The `worker` transport is claimed only on **linux/x86_64**; every other target
returns `ext.platform_unsupported` (exit 1) instead of pretending success.
Release artifacts must ship the helper next to the CLI for the same target
triple. macOS/Windows stay buildable and explicitly unsupported until they have
a native fixture and release evidence (RFC-EXT-0001 §5.6).
### 5. No new host op or language surface
There is no new `HostOp`, builtin, `resource_request` string branch, `Value`
variant or Markdown syntax. A provider kind is still an argument to
`resource.open("<kind>", descriptor, rights)`; the only change is the CLI
composition root that selects, verifies and authorizes the existing host
transport. Provider tokens stay inside the helper; the language sees only the
opaque Host `Resource` handle. Worker errors are structured `RuntimeError`s
surfaced as CLI errors — the CLI never panics on a helper crash/timeout.
### 6. Tests
`crates/cli/tests/provider_cli.rs` drives the real CLI binary with a real C
`c-abi-1` sensor library and the real `marqdo-provider-worker` helper built by
cargo and installed next to the CLI (the same layout a release tarball uses;
production resolution is not weakened for tests). Coverage: PROV-01/02/03/04/
05/06/07/08/09/10/13, plus `check` refusing the flag. Marker files written by
the C library's ABI-version hook prove whether a worker actually started, so
"no worker on unauthorized/invalid input" is evidence, not an assertion about
code paths.
## Consequences
* A third party can add a new resource kind to a frozen `marqdo` binary through
`provider.json` v2 + `--provider` + `--allow-provider`, without a core
rebuild, new builtin or new CLI command.
* v1 `marqdo.provider` remains a host-embedding format; CLI users get a clear
`ext.unsupported_provider_schema` instead of a silently unverified load.
* Integrity hashing raises the cost of accidental corruption/tampering but is
not a trust root; a compromised or malicious helper still runs with the
user's privileges. Documentation states this explicitly.
* `pure`/`strict` semantics and the existing resource/provider host contracts
are unchanged (strict still L1-only and fail-closed).
* The provider helper becomes a release artifact that must be packaged next to
the CLI; a missing helper is a first-class, deterministic error.
## Alternatives considered
1. **Reuse `plugin.yaml`/plugin ABI directly for providers.** Rejected: a
resource provider's contract is open/read/write/close with provider-private
tokens and stream kinds, not callable exports; sharing the manifest/ABI
parser would conflate two independent protocols (RFC-EXT-0001 §1.3).
2. **Extend v1 in place with optional `version`/`artifacts`.** Rejected:
v1 is host-embedding installation metadata and is already accepted by
`register_provider_package`; silently changing its meaning or accepting a
half-populated v1 would break the frozen ADR-0037 contract. v2 is explicit.
3. **Allow in-process C ABI loading when the worker helper is missing.**
Rejected: it would silently lower the process boundary and RFC-EXT-0001
§5.2/§5.4 forbids falling back to Native. `ext.worker_unavailable` is the
correct fail-closed result.
4. **PATH lookup for `marqdo-provider-worker`.** Rejected: `PATH` is
attacker-controlled state and would make the executed artifact
non-deterministic; the sibling rule binds the helper to the release.
5. **Register providers lazily at the first `resource.open`.** Rejected for
this slice: authorization and integrity must be decided before the main
function executes, and the existing host API is synchronous spawn-on-register.
6. **Accept a `*` provider grant for developer convenience.** Rejected: grants
are exact names (RFC-EXT-0001 §5.2); wildcards would let a typo authorize an
arbitrary selected provider.
## Deviations recorded
* `ext.manifest_invalid`, `ext.duplicate_provider` and
`ext.provider_grant_unmatched` are **additive** `ext.*` codes beyond the list
frozen in RFC-EXT-0001 §18.1. The frozen list has no generic malformed-v2
code, no duplicate-provider-name code and no unmatched-grant code; reusing
`ext.unsupported_provider_schema`/`ext.duplicate_provider_kind` for these
would report the wrong reason. They follow the same `ext.` naming and exit
code contract and are frozen by this ADR.
* `ext.artifact_integrity` failures exit 1 (load/verify), while static
manifest-selection failures exit 2, matching RFC-EXT-0001's exit-code
guidance; plugin artifact checking already uses the same split.
* Only linux/x86_64 is claimed for the provider `worker` transport in this
release (RFC-EXT-0001 §5.6 permits this explicit platform limit; Windows and
macOS return `ext.platform_unsupported`).
## Verification
`crates/cli/tests/provider_cli.rs` (12 real tests, linux/x86_64) plus
`crates/cli/src/provider_manifest.rs` unit tests pass; `spec/security.md`,
`spec/resource-provider-v2.md` and the README provider section document the
same contract. Evidence is recorded in the PR-03 task report (commands, exit
codes, per-PROV matrix).
@@ -0,0 +1,229 @@
# ADR-0057: Trusted capability-handler registration
Status: **Accepted**
Date: 2026-11-21
Implements: RFC-EXT-0001 §7 (PR-05)
Extends: ADR-0011 (capability broker)
Scope: `crates/marqdo-host/src/capability.rs` (host-internal registry)
## Context
ADR-0011 introduced `CapabilityBroker` with two separate ideas but only one
implementation path:
* `decide()` answered "is this capability granted?" from `Capabilities` and a
trusted vendor-policy allow-list; and
* `execute()` special-cased the single real core implementation (`fs.read`)
inline, returning `capability.unavailable` for everything else.
That shape is why adding a capability looked like a one-line allow-list edit,
and it made "declared in policy" and "can actually execute" the same object.
RFC-EXT-0001 §7.1/§7.2 requires the chain:
```text
manifest requests capability
-> operator/CLI grants it
-> broker asks: is a trusted handler installed?
-> handler re-checks owner/grant/scope/budget
-> handler executes
-> audit { identity, effect, decision, duration, trace_id }
```
A name in an allow-list must never be enough to execute, and a capability with
no executor must never look like an empty success.
## Decision
### 1. Two independent registries
`CapabilityBroker` keeps:
* `vendor_policy: BTreeSet<String>` — names the trusted host *policy* accepts
(from `allow_vendor`); and
* `handlers: BTreeMap<String, HandlerEntry>` — names that have an explicitly
registered trusted executor.
`HandlerEntry` additionally carries an optional exact trusted package binding
(`None` = authorized by the capability grant alone).
The two sets cannot substitute for each other:
* vendor capability without a policy entry -> `permission.denied`
(declaration is not a grant; CAP-01);
* vendor policy entry without a handler -> `capability.unavailable` (CAP-02);
* core capability granted by `Capabilities` but without a handler ->
`capability.unavailable` (CAP-12).
### 2. Handler interface
```rust
pub struct CapabilityRequest<'a> {
pub package_instance: &'a str, // trusted identity from the host, not wire data
pub capability: &'a str,
pub payload: &'a [u8],
pub trace_id: &'a str,
}
pub trait CapabilityHandler: Send {
fn name(&self) -> &str;
fn execute(
&mut self,
request: &CapabilityRequest<'_>,
capabilities: &Capabilities,
budget: &mut Budget,
) -> Result<Vec<u8>, PluginError>;
}
```
The registry key is `handler.name()`, so the handler's declared capability can
never disagree with the name it is registered under (the RFC's "declared
protocol mismatch" rejection cannot occur by construction).
Registration is host-only; there is no wire path from a plugin manifest to this
API. `register_handler` installs a handler usable by any package holding a
grant; `register_package_handler(package, handler)` binds it to one exact
trusted package identity and denies every other claimed package (CAP-03, the
"trust the package the host loaded, not a self-reported name" rule of §7.4).
### 3. Registration validation
`register_handler` / `register_package_handler` reject:
| condition | stable code |
|---|---|
| empty name, surrounding whitespace, or not a core/vendor name | `capability.handler_invalid` |
| name under a reserved host prefix (`marqdo.`, `host.`) | `capability.handler_reserved` |
| name already registered (including the built-in `fs.read`) | `capability.handler_conflict` |
A rejected handler is never installed. The component that constructs the
broker is the trust boundary: only trusted host/embedder code can register, so
"untrusted source" is rejected by the absence of an API rather than a flag.
### 4. Fixed execution chain and audit
`execute()` is one function with one order:
1. `decide()` — `Capabilities`, vendor policy, request payload shape;
2. handler installed? — otherwise audit `unavailable`, return
`capability.unavailable`;
3. package-bound handler owns the claimed package? — otherwise audit `denied`,
return `permission.denied`;
4. handler executes under the broker `Budget` and the broker reply limit;
5. audit one entry with `package`, `capability`, `decision`
(`granted`/`denied`/`unavailable`), reason and `duration_micros`.
`authorize()` remains the grant decision only (`granted`/`denied`); it never
executes and therefore cannot demonstrate an executor. This keeps the
authorization and execution questions separate and auditable.
The reply is bounded by `reply_limit` (default
`MAX_CAPABILITY_REPLY_BYTES`) and oversize replies are `resource.exhausted`
(CAP-06). Handlers also receive `&mut Budget`, so an embedder can wire the
per-run budget and handlers can charge fuel/output (tested). Response payloads
are never written to the audit log, so file contents and request secrets stay
out of it (CAP-09).
### 5. `fs.read` becomes the first standard handler
The former inline `fs.read` match arm moved into `FsReadHandler`, installed by
`CapabilityBroker::with_fs(fs)`. It:
* resolves the path with `ProjectFs::resolve` (absolute paths, `..`, empty
paths rejected) **before** I/O;
* re-checks `Capabilities::check_read` against the normalized
project-relative path at the execution point, so the grant is enforced on the
same path the filesystem uses;
* reads through `ProjectFs` (regular file, UTF-8, project root, 8 MiB limit).
`with_fs` still installs exactly one handler; `capability_broker.rs` and
`host.rs` call sites are unchanged. Every other core capability and every
vendor capability without a registered handler keeps returning
`capability.unavailable`; nothing returns an empty `Ok` as a stand-in for
"implemented".
## RFC deviations (explicit)
1. **`EffectiveGrant` -> `Capabilities`; `ResourceBudget` -> `Budget`.** The
RFC (§7.2) says these are design-interface names and the implementation must
reuse the existing grant and limit structures instead of building a second
permission system; that is exactly what happened.
2. **`CapabilityError` -> `PluginError`.** The capability bridge across the
plugin/worker ABI already uses `marqdo_plugin_abi::PluginError`; reusing it
avoids a second error taxonomy. Codes remain the stable contract.
3. **`trace_id` is plumbed through `CapabilityRequest` and settable with
`with_trace_id`, but the host wiring is a follow-up.** `host.rs` does not set
it yet, so audit entries from the current CLI plugin path have an empty
trace id. Wiring it requires the run/session id from `ExecutionContext`,
which is outside this PR's write scope.
4. **`resource.exhausted` preserves the broker code but not the runtime
category.** `PluginError` kind `Capability` maps to
`RuntimeError` category `PermissionDenied` in `host.rs::map_plugin_error`.
The stable code is `resource.exhausted` (what CAP-06 asserts at the broker
boundary); teaching the host mapping about a resource-limit plugin code is a
follow-up because `host.rs` is outside this PR's scope.
5. **`CapabilityBroker` is no longer `Clone`** because it owns `Box<dyn
CapabilityHandler>`. No in-tree caller cloned it; a future embedder that
needs a clone should share the broker behind the same lock instead, which is
also what the one-call-at-a-time ABI requires.
## Alternatives
1. Keep the closed match arm and add capabilities to it. Rejected: every
vendor operation would need a core release and the broker would still have
no separation between "granted" and "executable".
2. Treat a vendor policy entry as proof of an executor. Rejected: that is the
exact "declaration becomes privilege" failure the RFC forbids; CAP-02 locks
it out.
3. Register handlers by `(name, package)` strings only and trust the caller's
package argument. Rejected: a wire-supplied package name must not authorize
anything; package-bound registration pins the identity the host selected at
load time.
4. Give every core capability an empty handler so `execute()` always returns
`Ok`. Rejected: it converts "not implemented" into a silent success, the
failure mode RFC §7.1 calls out.
## Compatibility
* Existing `Capabilities`, `with_fs`, `with_audit`, `allow_vendor`,
`authorize`, `execute` and `capability_audit()` call sites are unchanged;
`host.rs` needed no edit.
* One observable change: a vendor capability without a trusted policy entry is
now `permission.denied` instead of `capability.unavailable`. This matches
ADR-0011 ("a missing grant is `permission.denied`") and RFC CAP-01.
* `CapabilityAudit` gained `duration_micros`; all fields remain public and the
existing tests that read `package`/`capability`/`decision` compile unchanged.
* `CapabilityBroker` lost `Clone` (see deviation 5).
* No ABI, wire, manifest or CLI change.
## Security
* Registration is a trusted-host operation with no manifest/wire entry point;
a plugin cannot install, replace or enumerate handlers.
* Package-bound handlers reject cross-package impersonation even when policy
and grant match.
* `fs.read` keeps `ProjectFs` root confinement, glob scope and size limits; the
grant is re-checked on the normalized path at the execution point.
* Audit records decisions and durations but never payload contents.
* This remains a cooperative in-process API contract, not an OS sandbox: a
native plugin can still issue direct syscalls. The registry narrows what the
host will *do on request*; it does not confine plugin memory or syscalls.
## Testing
`crates/marqdo-host/tests/capability_handlers.rs` covers CAP-01..CAP-12:
policy-vs-handler separation, package binding, registration rejection, real
`fs.read` scope enforcement, reply budget and handler budget charging, forged
and replayed handles, audit hygiene, resource-stream regression smoke, project
mode not being OS-strict, and "no handler is never a silent success".
Unit tests in `capability.rs` keep the `CORE_CAPABILITIES` list in sync with
`decide()` and assert that the built-in `fs.read` handler is installed by
`with_fs`.
## Migration & rollout
`with_fs` installs the migrated `fs.read` handler automatically, so existing
plugin loads keep working. Future core capabilities migrate by adding a handler
and registering it in the trusted composition root; until then they remain
`capability.unavailable`. Vendor operations that need a non-streaming API
should first use the existing `resource.open/read/write/close` stream path
(RFC-EXT-0001 §7.3) instead of registering a new host handler.
+105
View File
@@ -0,0 +1,105 @@
# ADR-0058: Official stdlib public aliases replace print magic
- Status: Accepted (2026-11-21)
- Implements: RFC-EXT-0001 §8 (PR-06)
- Related: ADR-0028/0030/0031/0039, ADR-0054
## Context
`打印`/`print` were implemented twice:
1. `crates/cli/src/loader.rs::program_uses_print` walked the entire AST, and if
any function called `打印`/`print` it injected the bundled
`stdlib/io.mq.md` module and set `Program.print_override` to its `写值`
function;
2. `crates/marqdo-runtime/src/interpreter.rs` contained a `打印`/`print`
special case that redirected to the override, while the builtin table also
contained a placeholder `打印` entry.
Copying this pattern to the next official high-level function (formatting,
console input, …) would add one AST scanner and one runtime branch per name.
The RFC forbids that ("为标准库每个高级函数复制 print_override 模式"), while
also requiring byte-identical output and no new Host op.
## Decision
1. **Verified metadata.** The bundled `stdlib/stdlib.json` declares
```json
"public_aliases": [
{
"public_names": ["print", "打印"],
"target_module": "io.mq.md",
"target_function": "写值",
"expected_signature": "params(值)",
"effects": ["host"]
}
],
"aliases_digest": "sha256:<64 lowercase hex>"
```
`aliases_digest` is the SHA-256 of the canonical binary encoding: for each
alias sorted by `(target_module, target_function)`, the module bytes, NUL,
the function bytes, NUL, then every public name in sorted order followed by
NUL. The same encoding is implemented by `tools/pack-stdlib.py` and
`crates/cli/src/stdlib.rs::aliases_digest`; `stdlib.json` cannot be edited
without updating the digest, and the loader verifies both digests before any
module loads. Metadata is integrity-checked, not signed: a SHA-256 proves
content equality, not publisher identity.
2. **Lazy, generic loading.** The loader reads the metadata, decides whether the
program references any public alias name (a generic identifier scan, not a
per-function hard code), and only then loads the declared target module.
The target signature is checked against `expected_signature`; a mismatch is
a load error. A program without `打印`/`print` keeps its previous module
graph and startup cost.
3. **Ordinary Marqdo binding.** For each used alias the loader produces an
`OfficialAlias { public_names, function, signature, owner_package,
effects_requested }`; `project_prepare` registers it in the
`CallableRegistry`. The binding pass targets
`Marqdo { function: <stdlib instance>::io.mq.md::写值 }`. The runtime calls
an ordinary Markdown function; it has no print branch.
4. **Reserved names stay reserved.** Official aliases may replace the builtin
placeholder for the same public name. Third-party registry insertion rejects
builtin *and* official-alias names (`ext.duplicate_callable`), so no plugin
or imported module can claim `打印`/`print`, `std.*` or `marqdo.*`.
`plugin.yaml` still rejects reserved export prefixes at manifest validation.
5. **No Host print operation.** Output continues to flow through
`io.写值` → `format.显示` → UTF-8 bytes → `Memory` →
`resource.write(stdout)`. There is no `HostOp::Print` and no second
formatter. Unbundled embedders that call the placeholder builtin receive
`kernel.unavailable`.
6. **Removed code.** `program_uses_print`, `stmts_use_print`,
`expr_uses_print`, `Program.print_override`, the runtime redirect branch and
the whole-AST print scanner are deleted. A structural test
(`architecture_boundaries.rs::callable_binding_is_the_only_static_resolution_path`)
fails if they reappear.
## Consequences
* `打印` and `print` are now ordinary calls in Analysis/Trace: the call target
is the stdlib function, `state` is `function`, and no synthetic `builtin`
target is reported for the alias.
* Output must stay byte-identical. `crates/cli/tests/stdlib_alias_routing.rs`
plus the existing print/format/Trace goldens pin this; a diff is a blocking
regression, not an acceptable "new design".
* A tampered alias table (metadata digest or expected signature) is rejected
before module loading; `stdlib.json` and the `.mq.md` tree remain separately
digest-pinned, and `tools/pack-stdlib.py` refuses to build an inconsistent
bundle.
* The generic metadata path is the only place a future official high-level name
needs to be declared locally. Any *new language syntax* still requires a
language RFC; aliases cannot change `@exec`/`@knowledge` semantics.
## Rejected alternatives
* **Keep `print_override` and add a second alias for another function.**
Rejected: it multiplies hard-coded scans/branches and violates RFC-EXT-0001
§8 and §14.10.
* **Move the print formatter into the Host as a `Print` op.** Rejected: it
would reintroduce an un-audited output path and duplicate the stdlib
formatter; output is a resource write.
* **Let third-party packages register `print`/`打印` aliases.** Rejected:
official public names are a language surface, not an extension registry.
* **Put the alias table in Rust `match` arms instead of metadata.** Rejected:
the table must be versioned with the verified bundle so a stale/mixed bundle
cannot run a different function.
+33 -5
View File
@@ -15,6 +15,15 @@ replace `spec/` or a recorded test log.
> bundled `stdlib/` implement it. The workspace gate is green at `9c1723d`;
> rollout evidence and historical banners are indexed in
> [`../migration/README.md`](../migration/README.md).
>
> **Extensibility closure (RFC-EXT-0001).** The current tree implements the
> single static callable binding (ADR-0054), official stdlib public aliases
> replacing print magic (ADR-0058), third-party resource providers through the
> normal CLI (ADR-0055), and the capability-handler registry (ADR-0057). The
> extension selection/lock v4 composition and the frozen external-binary
> evidence are tracked as remaining PR-04/PR-08 work, not claimed here. The
> normative resolution contract is
> [`spec/callable-resolution.md`](../../spec/callable-resolution.md).
## 1. Four state dimensions
@@ -25,7 +34,7 @@ platform".
| Dimension | Current state | Source |
|---|---|---|
| Implemented | Rust workspace `0.1.0`: RFC-LANG-0002 Markdown/frontend/IR, v2 statement parser, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, migrated bundled stdlib, CLI composition root | `Cargo.toml`, `crates/`, `stdlib/` |
| Tested locally | `cargo test --workspace --locked`: **507 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline; fmt and clippy clean | `docs/STATUS.md` §0, `docs/evidence/2026-11-21-semantics-v2-green.md` |
| Tested locally | `cargo test --workspace --locked`: **584 passed, 0 failed** across 66 targets on Linux x86_64 in the RFC-EXT-0001 tree; fmt and clippy clean. Historical baseline: 507 passed at `9c1723d`. | `docs/STATUS.md`, `docs/evidence/2026-11-22-ext-implementation.md` |
| Bundled stdlib released | In-repo bundle `marqdo-stdlib@0.1.0`, SHA-256 digest verified before loading; it is **not** an external registry package | `stdlib/stdlib.json`, `stdlib/README.md` |
| Multi-platform verified | Linux local gate is the only executed lane here; Windows/macOS native lanes are **UNVERIFIED**, and the GitHub security lane remains a **VM-only** gate | `docs/STATUS.md` §1, §4 |
@@ -53,6 +62,12 @@ OS / external services / devices
CLI is only the composition root: parse arguments, load the project,
assemble Host + Runtime, route output, exit with a status.
`project_prepare::prepare` is the single finalisation step: it loads sources,
reads static extension descriptors (never `dlopen`/spawn), builds the unique
`CallableRegistry`, binds every call site once and hands the same
`PreparedProgram` to `check`/`run`/`call`/`inspect`/Analysis. The runtime
dispatches only by the resulting `CallBindingTable`; it never re-resolves a
name or guesses a plugin.
Callable Plugin / Foreign kernel / Tool Protocol are separate extension
categories; they do not enter the standard library or the language IR.
```
@@ -112,7 +127,10 @@ review RFC, not a new crate per provider.
| network endpoints | `tcp`/`udp`/`tcp-listener`/`tls` open + `resource.accept`; exact host grant, DNS after grant, TLS certificate validation | implemented (project mode) | `resource_streams` |
| `std:net/http` | byte-exact HTTP/1.1 parse/build; chunk sizes are byte counts; text wrapper compatible | implemented (pure `.mq.md`) | `stdlib_http` |
| HTTP/1.1 | `stdlib/net/http.mq.md`: status/header/chunked-byte parse, request build | implemented over an already-authorized TCP stream; not a DNS/TLS client | `stdlib_http`, `http_over_tcp`, `stdlib_product` |
| print | bundled `io.mq.md`/`console.mq.md` through `resource.write`; legacy `HostOp::Print` removed | implemented (Compat-C) | `print_routing`, `stdlib_product` |
| print / official stdlib aliases | `打印`/`print` are verified `stdlib.json` public aliases bound to `io.mq.md::写值`; ordinary Marqdo calls, no `print_override`, no runtime branch, no `HostOp::Print` | implemented (ADR-0058) | `stdlib_alias_routing`, `print_routing`, `stdlib_product` |
| callable resolution | one registry + one `bind_program_calls` pass; `check`/`run`/`inspect`/Analysis share `CallSiteKey` bindings; plugin exports reconcile exactly with static manifests; no runtime name fallback | implemented (ADR-0054) | `callable_resolution`, `plugin_worker`, `analysis_trace` |
| resource provider v2 CLI | `--provider`/`--allow-provider` select and authorize a `provider.json` schema-2 `c-abi-1` worker artifact; manifest/artifact/grants/duplicate kinds verified before spawn; sibling helper only | implemented linux/x86_64; other platforms fail closed (ADR-0055) | `provider_cli` |
| capability handlers | trusted handler registry executes granted capabilities under budget/audit; missing handler is `capability.unavailable`; `fs.read` is the first real handler | implemented (ADR-0057) | `capability_handlers` |
| strict mode | only the pure/Math execution path and the built-in stdout console channel; project/foreign/plugin/tool/trace/file/network grants are rejected | implemented L1; L2 not implemented | `tests/security`, `resource_streams`, `stdlib_product` |
## 4. Extension path classification
@@ -120,9 +138,10 @@ review RFC, not a new crate per provider.
| Extension path | Responsibility | State |
|---|---|---|
| Tool Protocol | external `marqdo` CLI tools, inspect/verify/IDE commands; versioned manifest and install/remove | implemented v1; not a standard-I/O replacement inside a user program |
| Callable Plugin ABI | native callable functions and controlled experimental acceleration | implemented v1 (native + worker); not an ordinary JSON/CSV formatter |
| Resource Provider | new resource kinds/devices/network/file endpoints | implemented v1 in-process, worker and trusted C ABI; not one provider per high-level algorithm |
| Foreign kernel | explicit cross-language compatibility for Python/Node one-shot execution | implemented; not a Marqdo implementation of common algorithms |
| Callable Plugin ABI | native callable functions and controlled experimental acceleration | implemented v1 (native + worker); static manifest is the contract and run reconciles exports exactly; not an ordinary JSON/CSV formatter |
| Resource Provider | new resource kinds/devices/network/file endpoints | v2 CLI entry implemented for trusted worker transport on linux/x86_64 (ADR-0055); v1 in-process/trusted C ABI remain embedder APIs; not one provider per high-level algorithm |
| Foreign kernel | explicit cross-language compatibility for Python/Node one-shot execution | implemented; registry accepts an explicitly declared unknown language; not a Marqdo implementation of common algorithms |
| Official stdlib public alias | language-surface aliases such as `打印`/`print` declared in verified bundle metadata | implemented (ADR-0058); third parties cannot claim these names |
| Shared registry/policy/budget/audit/trace | package/adapter identity, descriptor/version checks, per-request grants, timeouts, error classes and trace metadata | housed inside `marqdo-host`; do not invent a shared binary ABI for function calls and resource streams |
## 5. Gap and closure backlog
@@ -146,6 +165,9 @@ that a phase is complete.
| P3 | No concurrency tasks, atomics or cancellable long-running scheduling | open; separate ADR |
| P3 | No native optimizer/JIT or columnar numeric backend | open; benchmark-first |
| P3 | Windows Native/Worker ABI is unsupported; macOS/Windows CI is unverified | open |
| P0 | RFC-EXT-0001 PR-04: `marqdo.yaml.extensions` + lock v4 extension selection, CLI composition-root wiring, `inspect --extensions` | **open / in progress** |
| P0 | RFC-EXT-0001 PR-08: frozen release binary, external conformance rerun, SHA-bound evidence manifest, final docs/status convergence | **open / in progress** |
| P1 | Provider worker transport only linux/x86_64; macOS/Windows provider lanes fail closed | open; separate platform work |
## 6. Documentation governance
@@ -184,3 +206,9 @@ that a phase is complete.
- [ADR-0052: byte-exact HTTP framing](../adr/0051-byte-exact-http-framing.md) — accepted byte-list HTTP/1.1 parser/builder.
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
- [RFC-EXT-0001](RFC-EXT-0001.md) — extensibility closure task specification (implementation status tracked here and in STATUS).
- [ADR-0054: single callable binding](../adr/0054-callable-binding-single-source.md) — one registry/binding pass, runtime dispatch by identity.
- [ADR-0055: resource provider CLI v2](../adr/0055-resource-provider-cli-v2.md) — provider v2 manifest/worker CLI closure and trust boundary.
- [ADR-0057: capability handler registration](../adr/0057-capability-handler-registration.md) — declared capability vs installed handler vs audit.
- [ADR-0058: stdlib public aliases](../adr/0058-stdlib-public-alias.md) — verified metadata replaces print magic.
- [`spec/callable-resolution.md`](../../spec/callable-resolution.md) and [`spec/resource-provider-v2.md`](../../spec/resource-provider-v2.md) — normative extension boundaries.
+42
View File
@@ -72,3 +72,45 @@ cargo build --release --locked -p marqdo-cli
The ignored `.secrets/dev.env` may contain a Woodpecker API token for local
queries/triggers. The token is not needed by the pipeline itself and must never
be committed.
## External extension evidence (RFC-EXT-0001 PR-07/PR-08)
A release candidate must also prove that a **frozen** CLI binary can use
extensions from physically separate repositories without modifying or
rebuilding this workspace. This is a new necessary condition; it does **not**
replace the security/platform/performance gates above.
Required inputs:
* a release build of this commit: `cargo build --release --locked -p marqdo-cli`;
* the provider helper shipped next to it for the same target:
`marqdo-provider-worker` (the provider transport resolves only that sibling;
no `PATH` fallback);
* a clean checkout of the external fixture repository
`https://git.cflmy.cn/cflmy/marqdo-extension-conformance`.
Generation and validation:
```bash
CLI="$(realpath target/release/marqdo)"
python3 tools/ci/verify_extension_contracts.py \
--cli "$CLI" \
--fixtures /path/to/marqdo-extension-conformance
python3 tools/ci/verify_extension_contracts.py \
--validate docs/evidence/extension/<40-hex-source-SHA>/manifest.json
```
The generator hashes the CLI before and after the external suite, records the
source SHA and fixture repo SHA, and refuses to write evidence for a changed
binary, a dirty fixture checkout, a `FAIL`, or an unexplained
`UNSUPPORTED`/`SKIPPED` result. The external suite itself
(`tests/run_all.py` in that repository) uses only `$MARQDO_BIN` (absolute),
never Cargo, and covers: pinned-Git `.mq.md` package, worker ABI plugin,
unknown foreign kernel, provider v2 worker, and an independent Analysis/Trace
consumer.
The schema is [`evidence/extension/schema.json`](evidence/extension/schema.json);
the generator is `tools/ci/verify_extension_contracts.py`. A manifest whose
`tested_sha` is not the release SHA is invalid. Platform-specific results must
be reported as `UNSUPPORTED` (for example provider worker on non-linux/x86_64);
a Linux PASS never substitutes for a macOS/Windows lane.
@@ -0,0 +1,195 @@
# RFC-EXT-0001 implementation evidence (P0 core + PR-03/05/06/07 first pass)
Status: **partial, honest record.** PR-00/01/02/03/05/06 are implemented and
tested on Linux x86_64 in this tree; PR-04 (extension selection/lock v4) and the
final frozen-release evidence are still in progress. This document is not a
release claim. The authoritative executed status is regenerated in
`docs/STATUS.md` and `docs/evidence/extension/<SHA>/manifest.json`.
## 1. Identity and environment
| Field | Value |
|---|---|
| Trees | `/root/work/marqdo-cli` (branch `feat/extensible-ecosystem`) and separate `/root/work/marqdo-extension-conformance` |
| Baseline | `595279fe38048b52b96d9ff1e7d57a867ec5c4c6`; green baseline in `docs/evidence/ext-baseline-595279f.md` |
| Rust | `rustc 1.100.0`, `cargo 1.100.0` |
| OS / arch | Linux x86_64 |
| Commands | logged under `.cache/ext/` (gitignored) |
No commit SHA is claimed for the implementation yet: work is uncommitted on the
feature branch, so this file cannot be a same-SHA release artifact.
## 2. Executed workspace gate (current tree)
```text
cargo fmt --all -- --check PASS (exit 0)
cargo clippy --workspace --all-targets -- -D warnings PASS (exit 0)
cargo test --workspace --locked PASS (exit 0)
66 test targets: 584 passed, 0 failed, 0 ignored
```
Targeted new suites:
```text
cargo test -p marqdo-cli --test callable_resolution 14 passed
cargo test -p marqdo-cli --test stdlib_alias_routing 5 passed
cargo test -p marqdo-cli --test provider_cli 12 passed
cargo test -p marqdo-cli --test analysis_trace 7 passed
cargo test -p marqdo-cli --test plugin_worker 4 passed
cargo test -p marqdo-cli --test print_routing 2 passed
cargo test -p marqdo-cli --test architecture_boundaries 9 passed
cargo test -p marqdo-host (includes capability_handlers 12)
```
## 3. What was implemented (source paths)
### PR-01/02 — one static binding, bound runtime dispatch
* `crates/marqdo-ir/src/call_binding.rs` — `CallSiteKey`, `BoundTarget`
(`Builtin`/`Marqdo`/`AbiPlugin`/`DynamicValue`), deterministic
`CallBindingTable`, unit tests.
* `crates/marqdo-ir/src/callable.rs` — `try_from_program` (rejects duplicate
exports instead of first-wins), `OfficialAlias` registration, dotted-prefix
conflict helpers, `function_id`.
* `crates/marqdo-frontend/src/calls.rs` — the single `bind_program_calls` pass,
RFC priority matrix, import namespace conflicts, exact plugin signature
validation; `validate_module` is a standalone-frontend wrapper.
* `crates/marqdo-frontend/src/analyze.rs` — `analyze_source_for_program` defers
call checking to the composition root; standalone `analyze_source` keeps its
single-document checks for the conformance corpora.
* `crates/marqdo-runtime/src/interpreter.rs` — dispatch by `CallSiteKey`; the
unknown-plugin fallback, `print_override` branch and dead `resolve_function`
are removed; bound ABI calls normalise positional arguments to manifest order.
* `crates/cli/src/extension_descriptors.rs` — static plugin manifest reading
(no `dlopen`/spawn), signature parsing (defaults rejected), bundle-relative
path checks, artifact sha helpers.
* `crates/cli/src/project_prepare.rs` — one `prepare()`: load → descriptors →
registry → bind once → `PreparedProgram`; diagnostics are appended once.
* `crates/cli/src/commands/{check,run,inspect}.rs` + `loader.rs` — all consume
`prepare_project`; `run` verifies artifacts before load and reconciles the
Host exports one-to-one (`ext.plugin_binding_mismatch` on any difference).
* `crates/cli/src/analysis.rs` — Analysis `calls[].state/target` are projected
from the same binding table (no second resolver).
* `crates/cli/tests/callable_resolution.rs` — CALL-01/02/03/04/05/06/07/08/09/
11/12/13/14/15/16 as black-box CLI tests (CALL-10 has unit coverage in
`commands/run.rs` for missing/extra runtime exports).
Normative docs: `spec/callable-resolution.md`, `docs/adr/0054-*`,
`spec/plugin-abi.md` §12, `spec/analysis-v2.md` calls note.
### PR-03 — resource provider v2 through the normal CLI
* `crates/cli/src/main.rs` — repeatable `--provider PATH` and
`--allow-provider NAME` on `run`/`call` only.
* `crates/cli/src/commands/sandbox.rs` — grants map to
`allow_resource_provider`; `pure`/`strict` preflight refuse the flags;
wildcards rejected.
* `crates/cli/src/provider_manifest.rs` — strict schema v2 parser/verifier,
target selection, bundle containment, sha256, reserved-kind rejection,
duplicate kind/provider checks, sibling `marqdo-provider-worker` resolution
(no `PATH` fallback; `ext.worker_unavailable`/`ext.platform_unsupported`).
* `crates/cli/src/commands/run.rs` — verify all manifests/artifacts/grants and
duplicate kinds before spawning; spawn only granted+verified providers; no
native fallback; no new `HostOp`/`resource_request` branch.
* Tests: `crates/cli/tests/provider_cli.rs` (real C sensor + real worker
helper), 12 tests; `spec/resource-provider-v2.md`, ADR-0055, README section.
Provider evidence recorded by the PR-03 implementer:
| ID | Result | Evidence summary |
|---|---|---|
| PROV-01 | PASS | exit 0, `count=5`, `eof=true`, one worker/open/close marker |
| PROV-02 | PASS | exit 1 `permission.denied`; helper never started |
| PROV-03 | PASS | exit 2 `ext.provider_grant_unmatched`; no worker |
| PROV-04 | PASS | exit 2 `ext.duplicate_provider_kind`; no worker |
| PROV-05 | PASS | one provider, two kinds, counts `[5,2]`, both closed |
| PROV-06 | PASS | `stdio/file/tcp/tls/udp/tcp-listener` rejected `ext.manifest_invalid` |
| PROV-07 | PASS | sha mismatch / symlink escape / `..` → `ext.artifact_integrity` before spawn |
| PROV-08 | PASS | `ext.no_matching_artifact` |
| PROV-09 | PASS | v1 → `ext.unsupported_provider_schema`, no migration |
| PROV-10 | PASS | pure `cli.invalid_grants`, strict `strict.unsupported_grant`, `*` rejected |
| PROV-13 | PASS | sibling helper missing → `ext.worker_unavailable`, PATH helper unused |
Platform: provider worker transport is claimed **linux/x86_64 only**; macOS and
Windows are explicitly `UNSUPPORTED`/fail-closed (`ext.platform_unsupported`),
not claimed supported. PROV-11/12/14/15 belong to host-protocol and PR-07
external closure and are not claimed by the PR-03 CLI matrix.
### PR-05 — capability handler registry
`crates/marqdo-host/src/capability.rs` now has a trusted handler directory,
`fs.read` as a real handler, decide → handler-installed → execute under budget →
audit with duration; missing handler stays `capability.unavailable`, duplicate/
reserved/empty registrations are rejected. `crates/marqdo-host/tests/
capability_handlers.rs` has 12 CAP tests (CAP-01…CAP-12); ADR-0057 and
`spec/security.md` updated. `cargo test -p marqdo-host` PASS.
### PR-06 — official stdlib public aliases
`stdlib/stdlib.json` declares `public_aliases` (`print`/`打印` →
`io.mq.md::写值`) and `aliases_digest`; `crates/cli/src/stdlib.rs` verifies the
metadata digest and expected target signature; `loader.rs` loads the target
module only when the alias name is referenced and produces `OfficialAlias`
records; `project_prepare` registers them and the binding pass targets the
ordinary `Marqdo` function. `program_uses_print`, `Program.print_override`, the
runtime print branch and the AST scanner are deleted; the placeholder builtin
still fails with `kernel.unavailable` for unbundled embedders. Tests:
`crates/cli/tests/stdlib_alias_routing.rs` (5) + existing print/format/Trace
goldens; the stdlib fixture aliases that collided with builtin namespaces
(`table`/`text`/`fs`) were renamed and the bundle digest regenerated.
`tools/pack-stdlib.py` verifies `aliases_digest` before packaging.
ADR-0058 documents the decision.
### PR-07 (first pass) — independent external repository
`/root/work/marqdo-extension-conformance` is a separate Git repository (own
history, remote `https://git.cflmy.cn/cflmy/marqdo-extension-conformance`,
pushed; commit `b729afc` at the time of this record). It contains:
* `pure-lib/` + `consumer/` with a pinned full-SHA Git dependency;
* `plugin/` — real C ABI v1 worker plugin, three exports under two prefixes;
* `kernel/` — `mocklang` provider with a language the CLI does not know;
* `provider/` — `provider.json` v2 + real C ABI sensor library;
* `tool/` — stdlib-only Python consumer of Analysis v2 / Trace v2;
* `tests/run_all.py`, `tests/verify_frozen_cli.py`, `CONTRACT.md`,
`.github/workflows/conformance.yml`.
Provisional run against the current development binary (debug build, not a
release candidate):
```text
[PASS] plugin_worker, callable_registry, foreign_kernel,
resource_provider_cli, package_pinned_git,
tool_analysis_consumer, negative_security
(all categories PASS; frozen CLI sha256 recorded in evidence/results.json)
```
This is **not** final PR-07/PR-08 evidence: the binary was not a `--release`
candidate, and `tools/ci/verify_extension_contracts.py` has not yet emitted a
`docs/evidence/extension/<SHA>/manifest.json`. The external suite will be rerun
against the frozen release binary after PR-04 lands.
## 4. Known gaps / deviations (not hidden)
1. **PR-04 not landed yet.** `marqdo.yaml.extensions` and lock v4 are still in
progress; the CLI composition root for extension selection is not wired.
2. **Provider platform claims are Linux x86_64 only**; macOS/Windows are
explicitly unsupported for the worker transport. The RFC's mac/Windows
provider lanes are not claimed.
3. **Additive provider error codes** `ext.manifest_invalid`,
`ext.duplicate_provider`, `ext.provider_grant_unmatched` were added because
the RFC's frozen code list has no precise code for malformed v2, duplicate
provider names, or an unmatched `--allow-provider`; documented in
`spec/resource-provider-v2.md` and ADR-0055.
4. **`--allow-provider` without a selected manifest** aborts with
`permission.denied` before spawning rather than silently ignoring the grant
(needed to satisfy PROV-02 with the existing spawn-on-register Host API).
5. **Analysis v2 `calls[].state/target` semantics changed for official aliases**
(print now projects as a `function` target). This is a projection of
existing fields, documented in `spec/analysis-v2.md`/ADR-0054/ADR-0058;
there is no v3 bump and the private binding table is not exposed.
6. **CALL-10's host-side mismatches** are unit-tested for missing/extra exports;
a live TOCTOU signature/digest mismatch is covered by artifact-tamper tests
plus exact reconciliation, not by swapping a manifest mid-run.
7. Full `--release` build, frozen external rerun, SHA-bound evidence manifest,
docs/STATUS/CURRENT/COMPATIBILITY/CHANGELOG finalisation remain for PR-08.
+82
View File
@@ -0,0 +1,82 @@
# RFC-EXT-0001 baseline audit — `595279f`
Status: **baseline evidence recorded before implementation.** This is not a
release claim and none of the new RFC-EXT-0001 behaviour existed at this SHA.
## 1. Identity and environment
| Field | Value |
|---|---|
| Repository | `cflmy/marqdo-cli` (private remote `https://git.cflmy.cn/cflmy/marqdo-cli`) |
| Baseline commit | `595279fe38048b52b96d9ff1e7d57a867ec5c4c6` (`master`, 2026-11-21) |
| Working branch | `feat/extensible-ecosystem` |
| Rust | `rustc 1.100.0 (0ef1bc386 2026-11-09)` |
| Cargo | `cargo 1.100.0 (3acc79c0c 2026-11-05)` |
| OS / arch | Linux `x86_64` (Ubuntu container) |
| Local homes | `RUSTUP_HOME=.rustup-home`, `CARGO_HOME=.cargo-home` |
| Command log | `.cache/ext/baseline.log` (local, gitignored) |
## 2. Executed gate (same SHA, before any RFC-EXT-0001 edit)
```text
cargo fmt --all -- --check exit 0 PASS
cargo clippy --workspace --all-targets -- -D warnings exit 0 PASS
cargo test --workspace --locked exit 0 PASS
62 test targets: 521 passed, 0 failed, 2 ignored
```
The RFC §2.2 static inventory commands were also run and their findings are
recorded in §3. A green baseline is only the starting point, not evidence for
any new feature.
## 3. Source facts at the baseline (RFC §2.1 refresh)
Each row was checked against the actual tree, not against the RFC prose. None of
the rows was `already_closed` at `595279f`.
| ID | Observed at `595279f` | Impact | RFC treatment |
|---|---|---|---|
| EXT-01 | `marqdo-frontend/src/units.rs` `SemanticMode` is `Exec`/`Knowledge` only; unknown `@xxx` is `sem.invalid_mode_marker` | third parties cannot add an execution mode (intended closure) | keep; metadata is inert Markdown |
| EXT-02 | `frontend/src/calls.rs` checks Member builtin before import; `runtime/src/interpreter.rs` checks import before builtin | check and run can bind different targets | PR-01/02 single binding |
| EXT-03 | `ir/src/callable.rs` has `CallableRegistry` but calls still use `builtins::find` and runtime fallbacks | registry was not the single truth | PR-01/02 |
| EXT-04 | interpreter tolerated undeclared `alias.name` as a plugin call when `external_callables` was empty | static-unknown symbol failed only at run time | PR-02 remove fallback |
| EXT-05 | `cli/src/commands/run.rs` synthesised `params: Vec::new()` descriptors for loaded exports | manifest/runtime signature could differ silently | PR-02 exact reconciliation |
| EXT-06 | `loader.rs` validated the program before the CLI appended static plugin descriptors, then validated again | duplicated diagnostics and ordering hazards | PR-01 single `prepare` finalize |
| EXT-07 | `host/src/host/providers.rs` has Rust/C/Worker provider registration; CLI had no `--provider` | providers were only reachable by trusted embedders | PR-03 |
| EXT-08 | `host/src/capability.rs::execute` implemented only `fs.read`; other names returned unavailable | declaring a capability did not mean executing it | PR-05 handler registry |
| EXT-09 | `host/src/host.rs::resource_request` has per-string built-in operations | adding a non-streaming operation required core edits | PR-03/05; keep core operations |
| EXT-10 | `cli/src/loader.rs::program_uses_print` injected `stdlib/io.mq.md` and set `Program.print_override`; runtime special-cased `打印/print` | migrating another stdlib function by copying this pattern would hard-code every name | PR-06 public aliases |
| EXT-11 | `foreign/src/lib.rs::KernelRegistry` supports unknown languages; old `normalize_language` only knows Python/Node | registry is the execution path, legacy helper is display-only | keep registry as the only query |
| EXT-12 | Plugin ABI v1 Native/Worker implemented; Wasm unsupported; Tool Protocol tested | do not rebuild plugin/tool platform | keep + add identity reconciliation |
| EXT-13 | `package/src/lib.rs` supports path + pinned `git` + package instances; `stdlib/` is a verified bundle | real `.mq.md` source decoupling exists | keep + external black-box test |
| EXT-14 | strict L1 permits only limited pure computation and stdout and rejects Worker/Foreign/Provider | project mode is not an OS sandbox | keep fail-closed; strict L2 is a separate RFC |
## 4. Issues reproduced while recording this baseline
They were reproduced while writing the RFC acceptance fixtures; all are fixed by
the PR series recorded later in `docs/STATUS.md` and
`docs/evidence/extension/<SHA>/manifest.json`:
1. **Duplicate resolution (EXT-02/03/04).** A fixture with an import alias that
matched a builtin dotted namespace (`import: {math: ./math.mq.md}` plus
`math.eval(...)`) had no deterministic binding rule; the frontend checked the
builtin while the runtime checked the import first.
2. **Runtime plugin guessing (EXT-04).** `solver.add(...)` without `--plugin`
reached the runtime and attempted `PluginInvoke` instead of failing during
`check` with `symbol.unknown`.
3. **Synthesised plugin signatures (EXT-05).** `run` could execute a plugin
export whose static descriptor had an empty parameter list, so an arity or
position/name mismatch was only discovered (or not) at the ABI boundary.
4. **Print magic (EXT-10).** `打印`/`print` were routed by a whole-AST
`program_uses_print` scan plus a runtime name special case; there was no
generic de-magicking path for a second official stdlib function.
5. **Provider CLI gap (EXT-07).** There was no `--provider`/`--allow-provider`;
an external `provider.json` could not be used through the shipped binary.
## 5. Not claimed by this baseline
* No `--provider`/`--allow-provider` existed; strict L2 remains unimplemented.
* No frozen-binary external repository evidence existed.
* No `docs/evidence/extension/<SHA>/manifest.json` existed.
* G3 remote security and G4 performance status is unchanged by this document;
see `docs/STATUS.md` and `docs/releasing.md`.
+32
View File
@@ -0,0 +1,32 @@
# External-extension evidence
RFC-EXT-0001 §10.2/§11.3 requires SHA-bound proof that an **unchanged** CLI
binary can use extensions from physically separate Git repositories. This
directory holds that evidence.
* `schema.json` — JSON Schema for the evidence manifest.
* `<40-hex-source-SHA>/manifest.json` — generated by
`tools/ci/verify_extension_contracts.py`; never hand-written.
Generation (from a clean fixture checkout and a release build of this repo):
```bash
cargo build --release --locked -p marqdo-cli
CLI="$(realpath target/release/marqdo)"
python3 tools/ci/verify_extension_contracts.py \
--cli "$CLI" \
--fixtures /path/to/marqdo-extension-conformance
python3 tools/ci/verify_extension_contracts.py \
--validate docs/evidence/extension/<SHA>/manifest.json
```
The generator refuses to write evidence when:
* the CLI binary hash changes between the start and end of the suite;
* the fixture repository has uncommitted changes;
* a required result is missing or `FAIL`;
* `UNSUPPORTED`/`SKIPPED` is recorded without `--allow-unverified` and a report
entry explaining the platform limitation.
A local or container run is **not** a substitute for the release gates in
`docs/releasing.md` (G3/G4), and no tag is created by this evidence.
+63
View File
@@ -0,0 +1,63 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://marqdo.dev/schemas/extension-evidence-v1.json",
"title": "Marqdo external-extension evidence manifest",
"description": "SHA-bound evidence that independent repositories can use the frozen CLI without modifying or recompiling marqdo-cli (RFC-EXT-0001 §11.3). Generated by tools/ci/verify_extension_contracts.py; never hand-edited.",
"type": "object",
"additionalProperties": false,
"required": [
"schema",
"schema_version",
"tested_sha",
"cli_binary_sha256",
"fixture_repo_sha",
"os",
"arch",
"results",
"commands",
"failures",
"unverified"
],
"properties": {
"schema": { "const": "marqdo.extension-evidence" },
"schema_version": { "const": 1 },
"tested_sha": {
"type": "string",
"pattern": "^[0-9a-f]{40}$",
"description": "Exact marqdo-cli source commit the frozen binary was built from."
},
"cli_binary_sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"fixture_repo_sha": {
"type": "string",
"pattern": "^[0-9a-f]{40}$"
},
"os": { "type": "string", "enum": ["linux", "macos", "windows"] },
"arch": { "type": "string" },
"results": {
"type": "object",
"additionalProperties": {
"type": "string",
"enum": ["PASS", "FAIL", "UNSUPPORTED", "SKIPPED"]
},
"required": [
"callable_registry",
"plugin_worker",
"foreign_kernel",
"resource_provider_cli",
"package_pinned_git",
"tool_analysis_consumer",
"negative_security"
]
},
"commands": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"failures": { "type": "array", "items": { "type": "string" } },
"unverified": { "type": "array", "items": { "type": "string" } }
}
}
+30
View File
@@ -129,3 +129,33 @@ self-hosted G-C URL. See
delete the failing test.
- If the security gate regresses: set strict back to `Unavailable`, publish a
security note, and do not claim isolation until the lane is green again.
## 7. Extension-ecosystem gate (RFC-EXT-0001)
Before a tag, in addition to §3 and §4:
1. `cargo build --release --locked -p marqdo-cli` and build
`-p marqdo-provider-worker`; ship both binaries in the release archive for
the same target. A provider request without the sibling helper must fail
with `ext.worker_unavailable` — never fall back to `PATH`.
2. Run the external frozen-binary suite from
`https://git.cflmy.cn/cflmy/marqdo-extension-conformance` against the exact
release binaries and generate
`docs/evidence/extension/<40-hex-source-SHA>/manifest.json` with
`tools/ci/verify_extension_contracts.py`. The manifest must validate against
`docs/evidence/extension/schema.json`, its `tested_sha` must equal the
tagged commit, and every mandatory result must be `PASS` on the claimed
platform. `UNSUPPORTED` is allowed only for a platform explicitly excluded
from the support matrix in `COMPATIBILITY.md`.
3. `check`/`run`/`inspect`/Analysis share one prepared call binding
(`spec/callable-resolution.md`); `check` never loads an extension binary;
plugin exports reconcile exactly with static manifests; official stdlib
aliases are verified metadata with byte-identical print output.
4. Existing gates remain mandatory and unchanged: G3 strict isolation evidence,
G4 measured performance on the fixed node, native platform lanes, schema
consumers and `tools/ci/release_evidence.py`. The extension evidence is an
additional gate; it never substitutes for them, and local/container smoke is
not a release claim.
5. If any extension gate is missing or mismatched, the release decision is
**DO NOT RELEASE**. Tagging and asset publication still require explicit
owner approval.
+7
View File
@@ -95,6 +95,13 @@ a separate statement. `imports`, `references` and `calls` remain static facts; a
Markdown reference is not a call. `effects` remains `["unknown"]` (a
conservative summary, not a purity proof).
`calls[]` is projected from the single prepared call binding
(`spec/callable-resolution.md` §6): `state` remains
`function`/`builtin`/`plugin`/`unresolved`, and official stdlib aliases such as
`打印` are reported as `function` targets because that is the implementation
that runs. The private `CallBindingTable` is not serialized; Analysis v2's
schema and version are unchanged.
An `assign` node keeps the v1 string `target` for a local rebinding. Addressable
memory adds one additive target form (ADR-0022); consumers that only understand
local assignments can ignore an object target:
+281
View File
@@ -0,0 +1,281 @@
# Marqdo callable resolution and binding
Status: **normative for the current implementation** (ADR-0054/ADR-0058,
RFC-EXT-0001 §4). This document describes the single static resolution used by
`check`, `run`, `call`, `inspect` and Analysis. There is no second priority
order and no runtime name guessing.
## 1. Identity
A call is an expression (`Expr::Call`). Each statically visible call site has a
deterministic key:
```text
CallSiteKey = (module id, function id, span.start, span.end)
```
The key is independent of the checkout path, package cache location,
registration order and map iteration. For a signature-default expression the
owning function is the function that declares the default, because the runtime
evaluates defaults in that function's empty frame.
A binding is one of:
| Bound target | Meaning |
|---|---|
| `Builtin { canonical_name }` | fixed language builtin, addressed by its canonical name (never by a user alias) |
| `Marqdo { function }` | exact `FunctionId` of a Markdown-defined function, including an official bundled-stdlib public alias |
| `AbiPlugin { package, export, digest, param_names }` | exact plugin export; `digest` is the selected, verified artifact SHA-256 and `param_names` is the manifest order used to normalise positional arguments |
| `DynamicValue` | the callee is a language value (for example a `FunctionRef`); resolved at run time from the value, never by name |
`CallBindingTable` is private IR. It is not serialized into Analysis or Trace.
The public `calls[].state/target` projection is derived from it (see §6).
## 2. Resolution matrix
### 2.1 Unqualified `f(...)`
In this exact order; the first applicable step decides:
1. **Local variable.** If `f` is a parameter or has been assigned anywhere in
the function body, the site binds `DynamicValue`. At run time the value must
be callable (`FunctionRef` in this release); a `Text`/`Map`/etc. value is
`call.not_callable`. This is a language variable rule, not an escape hatch to
a plugin.
2. **Official alias or reserved builtin.** The registry's builtin table contains
fixed builtins and, when the verified bundle is present and the program uses
them, official stdlib aliases. An official alias wins over the placeholder
builtin of the same name. Third parties cannot insert names here.
3. **Current namespace function.** If the enclosing level-1 namespace `N`
declares `N.f`, that function is chosen.
4. **Canonical function name.** A function whose canonical name is exactly `f`.
5. **Unique local name.** A function whose `local_name` is `f` in exactly one
namespace. If two namespaces define the same local name and none of steps
1–4 applies, the call is an error (`symbol.unknown`, message reports the
ambiguity); it is never "first in document order".
6. Otherwise `symbol.unknown`.
### 2.2 Qualified `alias.f(...)`
1. **Local variable named `alias`.** If `alias` is a local value, the site binds
`DynamicValue`; `alias.f` is ordinary language member access. This cannot
become a plugin call.
2. **Explicit import.** If the module declares `import: { alias: <target> }`,
the call binds only a function of that imported module (canonical name first,
then unique local name). A missing function is `symbol.unknown`; it never
falls back to a builtin or plugin with the same dotted spelling.
3. **Builtin dotted namespace.** Otherwise, an exact builtin
`alias.f` (for example `math.eval`) binds `Builtin`.
4. **Exact plugin export.** Otherwise, if a selected static plugin declares the
complete name `alias.f`, the site binds that export. The manifest signature
is validated here; no library is loaded.
5. Otherwise `symbol.unknown`.
`alias` in step 1/2 must be exactly an explicit import or a local value; there
is no implicit namespace search across packages.
### 2.3 Other callees
`some_value.f(...)`, `f_list[i](...)`, `(expr)(...)` and any other non-Ident,
non-member-of-Ident callee bind `DynamicValue`. Only language callables can be
invoked at run time.
### 2.4 Import namespace conflicts
Before any call is bound, each module's import aliases are checked against:
* builtin dotted namespaces (`alias.` must not prefix any builtin name), and
* visible plugin dotted namespaces (`alias.` must not prefix any selected
plugin export).
A collision is `ext.import_namespace_conflict` at the front-matter import span,
with a rename hint. Example: importing `./math.mq.md` as `math` conflicts with
the builtin namespace `math.`; rename the alias to `mymath`. This is a
deliberate 0.x breaking change with no compatibility switch (ADR-0054).
### 2.5 Duplicate callables
The static table rejects:
* the same complete plugin export declared by two selected packages
(`ext.duplicate_callable` at prepare);
* a plugin export that equals a builtin name or an official alias
(`ext.duplicate_callable`);
* two different manifests for the same plugin package identity in one selection
(`ext.duplicate_callable`); the exact same package identity is idempotent;
* a malformed/ambiguous static table entry (`ext.duplicate_callable`).
Order of `--plugin` flags never decides which declaration wins.
## 3. Argument checking
### 3.1 Markdown functions and builtins
`CallableSignature::validate_args` enforces:
* named arguments must exist exactly once; unknown names are `call.arity`;
* positional arguments require an explicit language signature and map to the
declared parameter order; overflow is `call.arity`;
* required parameters without defaults must be supplied; missing is
`call.arity`;
* inferred signatures accept named arguments only (`call.arity` otherwise).
Messages are deterministic and identical between `check` and `run`.
### 3.2 ABI plugin exports
* The static manifest signature is authoritative. Unknown, duplicate, missing
or excess arguments are rejected at prepare/bind time (`call.arity`); the
plugin is never spawned for a statically invalid call.
* ABI v1 transports arguments as a named map. Positional arguments are
statically validated and deterministically normalised to the manifest's
declared parameter order at run time. Example: `solver.add(1, b=2)` becomes
`{a:1,b:2}`.
* Manifest signature default values are rejected in ABI v1 until the wire
semantics are versioned; no default is injected at run time.
* A plugin that is declared but not selected (`--plugin` missing) is
`symbol.unknown`; a selected plugin without an exact export descriptor is
`symbol.unknown`. There is no runtime fallback.
## 4. Run-time reconciliation
`run`/`call` performs, in order:
1. static prepare (sources, manifests, signatures, binding) — no binary loaded;
2. verify each selected artifact path, target and SHA-256;
3. verify the `--allow-plugin` grant;
4. load exactly the selected artifacts;
5. compare the Host's actual exports with the static descriptors one-to-one
(`name`, raw signature, artifact digest, owner package).
A mismatch in step 5 is `ext.plugin_binding_mismatch`, reported before the
entry function runs. `run` must never synthesise a missing descriptor or an
empty signature.
## 5. `check` versus `run`
`check` and `run` consume the same `PreparedProgram` binding table. Therefore:
* the same call site has the same `BoundTarget` in both;
* a statically invalid call fails `check` and `run` with the same diagnostic
code and message; `run` refuses before any Host side effect;
* `check --plugin` reads manifests and signatures only: it never `dlopen`s a
library, spawns a worker or executes a side-effect script; a missing plugin
binary is not an error for `check`;
* `--emit` Analysis is derived from the same binding (no second resolver).
## 6. Analysis projection
`calls[]` keeps the v2 fields `{function, span, callee, state, target}`.
`state` is one of `function`, `builtin`, `plugin`, `unresolved`; it is now
derived from the binding table:
| Bound target | `state` | `target` |
|---|---|---|
| `Builtin` | `builtin` | canonical builtin name |
| `Marqdo` | `function` | exact `FunctionId` string |
| `AbiPlugin` | `plugin` | complete export name |
| `DynamicValue` / unbound | `unresolved` | `null` |
Official stdlib aliases (for example `打印`) therefore project as `function`
targets, because that is the implementation that runs. This is a projection of
the existing fields, not a new public schema. The private `CallBindingTable` is
never exposed; a future explicit `resolved_call_target` field would require an
Analysis version bump.
## 7. Diagnostics
| Code | Stage | Trigger |
|---|---|---|
| `symbol.unknown` | prepare/bind | no target for the call; ambiguity; unknown imported function; undeclared plugin export |
| `call.arity` | prepare/bind | bad argument name/number/duplication, invalid positional use |
| `ext.import_namespace_conflict` | prepare | import alias collides with a builtin/plugin dotted namespace |
| `ext.duplicate_callable` | prepare | duplicate complete export, builtin shadowing, conflicting package manifests |
| `ext.plugin_binding_mismatch` | run init | loaded exports/signatures/owner/digest differ from static descriptors |
| `ext.no_matching_artifact` | manifest read | plugin/provider has no artifact for the current target |
| `ext.artifact_integrity` | manifest read / run init | sha256 mismatch, path traversal, non-regular file, symlink escape |
| `call.not_callable` | run | dynamic value is not callable |
| `internal.error` | run | a call site has no binding (a prepared program must always be complete) |
Existing codes are reused rather than duplicated; `ext.capability_handler_unavailable`
is internal-only — the public wire code remains `capability.unavailable`.
## 8. Examples
### 8.1 Imported function, no conflict
```markdown
---
import:
mymath: "./math.mq.md"
---
## @exec main
`return mymath.eval(a=1)`
```
Binds `Marqdo { function: <package instance>::math.mq.md::eval }` in both
`check` and `run`.
### 8.2 Builtin without an import
```markdown
## @exec main
`return math.eval(ref("#公式"), {"r": 2}, target="A")`
```
Binds `Builtin { canonical_name: "math.eval" }`.
### 8.3 Import namespace conflict
```markdown
---
import:
math: "./math.mq.md"
---
## @exec main
`return math.eval(a=1)`
```
`check` fails with `ext.import_namespace_conflict`; `run` refuses before
executing anything. Rename the alias.
### 8.4 Plugin export
```markdown
## @exec main
`return solver.add(1, b=2)`
```
* without `--plugin solver/plugin.yaml` or with a manifest that does not
declare `solver.add`: `symbol.unknown` (no Host fallback);
* with the manifest: binds `AbiPlugin { package: "demo.solver", export:
"solver.add", digest: "<sha256>", param_names: ["a","b"] }`;
* with a changed artifact byte: `ext.artifact_integrity` before the entry
function;
* with a missing `b`: `call.arity` at prepare.
### 8.5 Dynamic `FunctionRef`
```markdown
## @exec 平方
`params(n: Int)`
`return n * n`
## @exec main
`f = ref("#平方")`
`return f(n=4)`
```
The call site binds `DynamicValue`; the runtime calls the `FunctionRef` value.
A `Text` value in the same position fails `call.not_callable` and never reaches
the Host.
+30
View File
@@ -459,3 +459,33 @@ when no C compiler is available.
| Positional plugin arguments | Not supported; named only. |
| Thread-safe / concurrent invocation | Not supported (single-threaded v1 contract). |
| Old `marqdo_abi.h` v2 compatibility | Not implemented and not claimed; a separate adapter would be required. |
## 12. Static descriptor / run-time reconciliation (RFC-EXT-0001 §4.5)
The CLI never derives a callable signature from a loaded library. The
**manifest is the static contract** and the runtime is checked against it:
1. `check --plugin <manifest>` reads and validates the manifest, including every
export name and signature; it does not load the artifact and does not need
the artifact file to exist.
2. `run`/`call` first verifies the selected artifact path, host target and
SHA-256, then checks the plugin grant, then loads exactly the selected
artifacts.
3. After loading, the Host's actual exports are compared one-to-one with the
static descriptors: complete name, raw signature string, owner package and
verified artifact digest. Extra, missing, renamed or differently-signed
exports fail with `ext.plugin_binding_mismatch` before the entry function
runs.
4. The binding pass validates call arguments against the manifest signature.
ABI v1 transports a named map, so positional arguments are statically
validated and deterministically normalised to manifest order
(`solver.add(1, b=2)` → `{a:1,b:2}`). Unknown, duplicate, missing or excess
arguments are `call.arity` before any process is started.
5. Manifest signatures with default values are rejected in ABI v1 until the
default-value wire semantics are versioned. The host never injects a default.
6. Duplicate complete export names across selected packages, or exports that
shadow a builtin/official name, are `ext.duplicate_callable` at prepare.
Selection order never decides which declaration wins.
None of this changes ABI v1's C layout or wire format; it closes the static
side of the contract described by `spec/callable-resolution.md`.
+287
View File
@@ -0,0 +1,287 @@
# Marqdo Resource Provider v2 (`provider.json`)
Status: **frozen provider manifest schema version 2** for the `marqdo` CLI
composition root. Implementation: `crates/cli/src/provider_manifest.rs`;
prerequisite host transports: ADR-0033/0035/0036/0037/0038; CLI decision:
ADR-0055. The worker helper is `marqdo-provider-worker`
(`crates/marqdo-provider-worker`, ADR-0038).
> **A trusted helper is not a sandbox.** The provider `worker` transport runs a
> separate process with the invoking user's OS privileges. It isolates a crash
> or protocol failure from the interpreter; it does **not** restrict file
> system, network, process or memory access. `sha256` values are content
> *integrity* checks, not publisher authentication. `strict` L1 refuses every
> provider (fail-closed) until a real isolation adapter exists.
## 1. CLI surface
```text
marqdo run FILE [--provider PATH/provider.json ...] [--allow-provider NAME ...]
marqdo call FILE FUNCTION [--arg NAME=VALUE ...] \
[--provider PATH/provider.json ...] [--allow-provider NAME ...]
marqdo check FILE # no --provider/--allow-provider options exist
```
* `--provider PATH` is repeatable. `PATH` names a `provider.json` file
(directories are rejected). Only explicitly passed manifests are considered;
there is no directory scan, package-manager discovery, registry download or
network fetch.
* `--allow-provider NAME` is repeatable and matches the manifest `name`
**exactly**. There is no `*`/glob wildcard, prefix match or implicit
authorization. Selecting a provider with `--provider` never grants it.
* `check`/`inspect` do not accept these options and never read a provider
manifest, start a helper or call provider code. Provider kinds are not new
callables; they are values passed to `resource.open(kind, ...)`.
* `--mode pure` and `--mode strict` reject `--provider`/`--allow-provider` in
preflight (exit 2, no helper spawn, no silent downgrade to `project`).
`pure` reports `cli.invalid_grants`; `strict` reports
`strict.unsupported_grant`.
### 1.1 Ordering
`run`/`call` follow RFC-EXT-0001 §18.2:
1. T1–T5: load the Markdown graph and prepare call bindings (unchanged;
providers are not part of `prepare`).
2. T6: parse all `--provider` manifests, validate schema/name/version/ABI/
transport/kinds/artifact declarations, select and verify the current-target
artifact (regular file, inside the bundle, size bound, sha256), reject
duplicate provider names and duplicate kinds across providers, and check
the exact `--allow-provider` grants.
3. T6b: verify all selected plugin manifests/artifacts (existing behavior).
4. T7: start plugin workers and reconcile exports, then spawn one
`marqdo-provider-worker` per granted+verified provider.
5. T9: execute the entry function; every `resource.open` re-checks the provider
name grant in the Host.
No worker process (plugin or provider) is started before every T6/T6b check
passes. A manifest, target, integrity, grant or duplicate-kind failure leaves
the main function unexecuted.
An `--allow-provider NAME` that matches no selected manifest is a deterministic
option error (`ext.provider_grant_unmatched`, exit 2) rather than silent
discovery fan-out.
## 2. Manifest file
* File name convention: `provider.json`. It must be a UTF-8 JSON object of at
most 1 MiB and is read from an explicit path, never from `PATH`.
* The CLI accepts **schema version 2 only**. A `schema_version: 1` manifest is
rejected with `ext.unsupported_provider_schema`; there is no silent
migration and no v1/v2 auto-detection. (v1 remains usable only through the
host embedding API `StdHost::register_provider_package`, ADR-0037.)
* Unknown top-level fields are rejected inside schema v2. Extensible,
non-executable data must live in the `metadata` object, which has no
execution meaning. A future schema version must be a new document, not an
in-place extension of v2.
```json
{
"schema": "marqdo.provider",
"schema_version": 2,
"name": "demo.sensor",
"version": "0.1.0",
"abi": "c-abi-1",
"transport": "worker",
"kinds": ["demo.sensor"],
"artifacts": [
{
"target": "x86_64-unknown-linux-gnu",
"path": "lib/libdemo_sensor.so",
"sha256": "<64-lowercase-hex-sha256>"
}
],
"permissions_requested": []
}
```
### 2.1 Fields
| Field | Required | Rules |
|---|---|---|
| `schema` | yes | Exactly `"marqdo.provider"`. |
| `schema_version` | yes | Exactly `2` (integer). |
| `name` | yes | Stable grant name. Non-empty dotted name: each `.`-separated segment starts with a lowercase ASCII letter and continues with lowercase ASCII letters, digits, `-` or `_`; at most 128 bytes. A name equal to a reserved core resource kind is rejected. |
| `version` | yes | Strict SemVer (`semver::Version`, three numeric components). |
| `abi` | yes | Exactly `"c-abi-1"` (ABI v1, ADR-0036/0038). Anything else is `ext.unsupported_provider_schema`. |
| `transport` | yes | Exactly `"worker"` (framed-JSON helper, ADR-0035/0038). Anything else is `ext.unsupported_provider_schema`. |
| `kinds` | yes | Non-empty, at most 64 unique dotted names with the same grammar as `name`. Core kinds are reserved (see §2.2). |
| `artifacts` | yes | Non-empty, at most 32 entries with unique target triples. Each entry has `target`, `path`, `sha256`; unknown entry fields are rejected. |
| `permissions_requested` | no | Declarations only, at most 64 unique strings: a known host capability (`marqdo_plugin_abi::KNOWN_CAPABILITIES`) or a dotted vendor capability (at least three segments, no core namespace prefix). Declarations never grant anything. |
| `metadata` | no | Arbitrary JSON. Ignored for execution; never used to select artifacts or grant permissions. |
`artifacts[].target` must parse as a target triple. `artifacts[].path` is
bundle-relative: absolute paths, `..` components and other escapes are
rejected. `artifacts[].sha256` must be 64 lowercase hexadecimal characters.
`artifacts[].path` is resolved against the manifest's own directory (the
bundle root); a symlink is followed only when the canonical target stays inside
that root, and the final target must be a regular file no larger than 256 MiB.
### 2.2 Reserved core kinds
`stdio`, `file`, `tcp`, `tls`, `udp` and `tcp-listener` are implemented by
`marqdo-host` before the provider registry is consulted. A provider may not
declare them (nor use one as its `name`); registration can never shadow them.
Reserved core kinds are rejected with `ext.manifest_invalid`.
### 2.3 Artifact selection
The only artifact used is the one whose `target` equals
`marqdo_plugin_abi::HostTarget::current().triple()`. A missing current target
is `ext.no_matching_artifact` (exit 2); another platform's artifact or a
`default` fallback is never substituted.
## 3. Identity and integrity
* The manifest content digest is `sha256:<hex>` over the manifest bytes; the
selected artifact digest is the declared lowercase `sha256`. Both are
re-verified immediately before the helper is spawned, and again when the
registration helper is called, so a file replaced between preparation and
activation fails closed.
* These hashes prove the manifest/artifact pair is byte-identical to what was
selected. They do **not** prove who published it, that it is trustworthy or
that it may be granted. Publisher trust needs a signature/trust-root layer
that is out of scope for schema v2.
* No version of this schema stores a secret, environment value or shell
fragment. Manifests are never concatenated into a shell command.
### 3.1 Duplicate identity
* The same provider `name` selected with two different manifest digests is
`ext.duplicate_provider`; the exact same manifest selected twice is
idempotent.
* One non-reserved `kind` declared by two providers is
`ext.duplicate_provider_kind`. Registration never overwrites a previous
provider ("last wins" is forbidden).
* Error selection is deterministic: manifests are canonicalised, de-duplicated
and sorted before conflicts are reported, independent of CLI argument order.
* One provider may serve several non-reserved kinds; all of them are
registered together and must be usable.
## 4. Worker helper resolution
The CLI resolves the helper **only** as a sibling of the running `marqdo`
executable:
```text
<directory of the marqdo executable>/marqdo-provider-worker[.exe]
```
* No `PATH` lookup, no current-directory search, no environment override and no
download fallback. A helper found on `PATH` is ignored.
* A missing sibling (or a non-regular file) is `ext.worker_unavailable`
(exit 1). A helper that fails to start or fails the protocol ping is also
`ext.worker_unavailable`; there is no fallback to in-process/native loading.
* Release archives must ship the `marqdo` binary and the
`marqdo-provider-worker` helper built for the same target triple.
* The `worker` transport is supported only on **linux/x86_64** in this release.
Other platforms compile but fail closed with `ext.platform_unsupported`
(exit 1). Windows and macOS are explicitly unsupported until they have a
native fixture and release evidence; Linux success is not a platform claim.
## 5. Worker protocol v1 and token lifecycle
The helper is `marqdo-provider-worker <selected-artifact-path>`. It loads
exactly one `c-abi-1` shared library, verifies
`marqdo_resource_provider_abi_version() == 1`, and speaks the ADR-0035 framed
protocol over stdin/stdout: 4-byte big-endian length + JSON, 1 MiB maximum per
frame.
```text
{"op":"ping"} -> {"ok":true} (handshake)
{"op":"open","kind","descriptor","rights"} -> {"ok":true,"token":u64}
{"op":"read","token","max_bytes"} -> {"ok":true,"bytes":[u8...],"eof":bool}
{"op":"write","token","bytes":[u8...]} -> {"ok":true,"count":u64}
{"op":"close","token"} -> {"ok":true}
{"op":"shutdown"} -> process exit
{"ok":false,"code","message"} -> structured failure
```
Token lifecycle:
1. `ping` must succeed before the host registers the provider. A failed
handshake aborts the run (`ext.worker_unavailable`).
2. `open` creates a provider-private `u64` token in the helper. The language
never sees it: the Host issues the opaque language `Resource` handle that
binds run id + provider name + token. Provider tokens cannot be forged into
language values or wire messages.
3. `read`/`write`/`close` are dispatched by the Host with the token it stored.
A read returns at most `max_bytes`; bytes cross the boundary by copy.
4. `close` closes exactly one token. Repeated or unknown tokens must fail with
a structured error; the Host never panics.
5. When the run ends (normal, error or timeout) every open token is closed and
the helper receives `shutdown`; the Host kills and reaps the child on drop.
A helper crash/protocol violation becomes a structured `host.error` for the
offending operation (CLI exit 1) and is never retried against another
provider or an in-process transport.
6. Provider error codes map to existing categories: `permission.denied`,
`resource.closed`, `resource.exhausted`, `resource.unsupported`, otherwise
`host.error`.
## 6. Authorization
* Registration, manifest parsing and artifact verification are not
authorization. Every `resource.open("<kind>", ...)` still requires:
project mode, the provider name in `Capabilities.resource_providers`
(populated only by exact `--allow-provider NAME`), and the Host's provider
name check. `permissions_requested` declarations do not affect this.
* `pure` and `strict` never reach provider kinds. `project` is the only mode
that can use a third-party provider.
* Unauthorized selection fails closed before the worker is spawned:
`--provider` without a matching `--allow-provider` is `permission.denied`
(exit 1) and the helper is never started.
## 7. Error codes and exit codes
Manifest/CLI option failures exit 2; load/verification/spawn failures exit 1;
`permission.denied` exits 1. With `--json` the CLI emits exactly one JSON
document with a `code` (nested under `error.code` for these structured
errors).
| Code | Stage | Trigger |
|---|---|---|
| `ext.manifest_unreadable` | manifest read (2) | missing/unreadable path, directory instead of `provider.json` |
| `ext.unsupported_provider_schema` | manifest read (2) | `schema` mismatch, `schema_version != 2`, unsupported `abi`/`transport` |
| `ext.manifest_invalid` | manifest read (2) | invalid UTF-8/JSON, unknown field, bad name/kind/version/permission, reserved kind, duplicate kind/target inside one manifest |
| `ext.no_matching_artifact` | manifest read (2) | no artifact for the current host target |
| `ext.duplicate_provider` | manifest read (2) | same provider name with two different manifest digests |
| `ext.duplicate_provider_kind` | manifest read (2) | two providers declare the same non-reserved kind |
| `ext.provider_grant_unmatched` | grant check (2) | `--allow-provider` names no selected manifest |
| `ext.artifact_integrity` | artifact verify / pre-spawn re-verify (1) | sha256 mismatch, absolute/`..`/symlink escape, non-regular file, size limit, manifest changed after verification |
| `permission.denied` | grant check (1) | `--provider` selected without its exact `--allow-provider` grant |
| `ext.worker_unavailable` | spawn (1) | sibling helper missing/not a regular file, spawn failure, ping failure |
| `ext.platform_unsupported` | spawn (1) | provider `worker` transport on a non-linux/x86_64 build |
| `cli.invalid_grants` | CLI preflight (2) | wildcard/empty `--allow-provider`, provider flags in `pure` mode |
| `strict.unsupported_grant` | strict preflight (2) | provider flags in `strict` mode |
`ext.manifest_invalid`, `ext.duplicate_provider` and
`ext.provider_grant_unmatched` are additive codes: RFC-EXT-0001 §18.1 froze the
provider-specific codes it could name, but a generic malformed-v2-manifest
code, a duplicate-provider-name code and an unmatched-grant code are required
to keep failure reasons distinct. They follow the same `ext.` naming and exit
code contract and are stable.
## 8. Test matrix (PR-03)
`crates/cli/tests/provider_cli.rs` builds a real C `c-abi-1` sensor provider,
runs the real `marqdo-provider-worker` next to the real CLI binary, and covers:
| ID | Evidence |
|---|---|
| PROV-01 | granted v2 worker + `resource.open/read/close("demo.sensor")` returns count 5, EOF, one close, `resource.open` in the host trace |
| PROV-02 | `--provider` without grant → `permission.denied`; marker proves the helper was never started |
| PROV-03 | `--allow-provider` without `--provider` → `ext.provider_grant_unmatched`, no worker |
| PROV-04 | same kind from two providers → `ext.duplicate_provider_kind` before any worker |
| PROV-05 | one provider with two non-reserved kinds → both open/read/close, one helper |
| PROV-06 | `stdio`/`file`/`tcp`/`tls`/`udp`/`tcp-listener` kinds rejected, no worker |
| PROV-07 | digest mismatch, symlink escape, `..` traversal → `ext.artifact_integrity`, no worker |
| PROV-08 | no current-target artifact → `ext.no_matching_artifact` |
| PROV-09 | `schema_version: 1` → `ext.unsupported_provider_schema` |
| PROV-10 | `pure`/`strict` + provider flags → preflight refusal, no worker; wildcard grant rejected |
| PROV-13 | relocated CLI without sibling helper (helper present on `PATH`) → `ext.worker_unavailable`, PATH is never used |
PROV-11 (worker crash/timeout/size-limit), PROV-12 (closed/stale/forged token),
PROV-14 (concurrent lifecycle) and PROV-15 (frozen-binary external repository)
are covered by the host worker/protocol tests or belong to the PR-07 external
conformance repository; they are not part of this CLI slice's manifest matrix.
+71 -12
View File
@@ -9,6 +9,7 @@ Normative implementation:
|---|---|
| Sandbox modes, strict refusal | `crates/marqdo-sandbox/src/lib.rs` |
| Capability sets and checks | `crates/marqdo-host/src/capability.rs` |
| Trusted capability handlers | `crates/marqdo-host/src/capability.rs` (`CapabilityBroker`, `CapabilityHandler`, `FsReadHandler`; ADR-0057) |
| Host dispatch and per-op checks | `crates/marqdo-host/src/host.rs` |
| Project-rooted filesystem | `crates/marqdo-host/src/fs.rs` |
| Budgets | `crates/marqdo-host/src/budget.rs` |
@@ -175,18 +176,72 @@ before reading. Full symlink/race hardening is an open P4 item.
* plugin invoke → the **complete export name** must be registered, then
`capabilities.check_plugin(package)` on **every call**; loading also checks
the same grant once;
* plugin host-capability requests → `CapabilityBroker` checks the package and
effective grants and **executes** granted `fs.read` through `ProjectFs`,
returning the real file bytes. Other core capabilities (`fs.write`,
`net.request`, `process`, console/math/log/...), and vendor capabilities
without a trusted executing handler, return `capability.unavailable`; they
never pretend an empty reply is success. Unknown/vendor names without a
trusted policy entry become `capability.unavailable`/`permission.denied`;
* plugin host-capability requests → `CapabilityBroker` runs a fixed chain:
`decide` (package, effective grants, trusted vendor policy, payload shape) →
a trusted `CapabilityHandler` must be installed for the exact capability name
→ a package-bound handler must own the trusted package identity → the handler
executes under the broker budget and reply limit → one audit entry. Granted
`fs.read` executes through `FsReadHandler`, which resolves the path with
`ProjectFs` and re-checks the read grant on the normalized project-relative
path before returning the real file bytes. Every capability without an
installed handler — other core capabilities (`fs.write`, `net.request`,
`process`, console/math/log/clock/random/artifact.record) and vendor
capabilities without a trusted executor — returns `capability.unavailable`;
it never pretends an empty reply is success. A vendor name with no trusted
policy entry is `permission.denied` (ADR-0011, ADR-0057);
* print → output budget only.
Static analysis may warn, but it never replaces these runtime checks. There is
no host bypass for tests: tests use the same `StdHost`/`Capabilities` types.
### 2.3 Executable capabilities need an installed trusted handler
`CapabilityBroker` (`crates/marqdo-host/src/capability.rs`, ADR-0057) separates
*being authorized* from *being executable*:
| Question | Registry | Failure |
|---|---|---|
| Is this capability granted for this trusted package? | `Capabilities` / `vendor_policy` | `permission.denied` |
| Is a trusted executor installed for this exact name? | `handlers: BTreeMap<String, HandlerEntry>` | `capability.unavailable` |
| Does a package-bound handler own the claimed package? | `HandlerEntry.package` | `permission.denied` |
Execution order is fixed: `decide` → handler installed → package binding →
handler execution under the broker `Budget` and reply limit → audit. A name in
`vendor_policy` is policy only; it does not install anything, and it cannot
turn into an executor. A core capability granted by `Capabilities` but missing
a handler is `capability.unavailable`, never an empty `Ok`.
* **Registration is host-only.** `register_handler(handler)` /
`register_package_handler(package, handler)` are called by the trusted host /
embedder composition root; no manifest, plugin or wire message can reach the
registry. The registry key is `handler.name()`, so a handler cannot be
registered under a name it does not declare.
* **Rejected names** (stable codes): empty/whitespace/malformed names are
`capability.handler_invalid`; names under the reserved host prefixes
`marqdo.` / `host.` are `capability.handler_reserved`; an already registered
name (including the built-in `fs.read`) is `capability.handler_conflict`. A
rejected handler is not installed.
* **`fs.read`** is the first migrated standard handler (`FsReadHandler`),
installed by `CapabilityBroker::with_fs(fs)`. It keeps the `ProjectFs` root,
traversal/absolute-path rejection, regular-file/UTF-8/8 MiB limits and the
glob grant, and it re-checks the grant against the normalized path at the
execution point. The broker reply limit (`MAX_CAPABILITY_REPLY_BYTES` by
default) bounds the returned bytes; an oversize reply is
`resource.exhausted`.
* **Audit**: every decision records package, capability,
`granted`/`denied`/`unavailable`, a reason and `duration_micros`. Payloads
and file contents are never recorded, so ordinary secrets do not appear in
the audit log.
* **Budgets**: handlers receive `&mut Budget` and can charge fuel/output; the
broker also enforces the reply byte limit. Native handlers still run
in-process and are trusted code, not a sandbox.
Known gaps in this build: the broker `trace_id` is settable
(`with_trace_id`) but `host.rs` does not supply it yet; and a broker-level
`resource.exhausted` reply keeps its stable code but the host plugin-error
mapping currently collapses `PluginError` kind `capability` to the
`permission.denied` runtime category (changing that mapping is a follow-up).
## 3. Resource budgets
`Budget` (`crates/marqdo-host/src/budget.rs`) is per run:
@@ -338,11 +393,15 @@ Neither command executes project code. Specifically:
4. Write-path symlink TOCTOU (see §2.1).
5. Plugin **worker crash isolation** and the worker capability bridge are
implemented (not usable in strict L1): both native and worker plugins can
reach the broker's
`fs.read` execution when granted. Other core capabilities and vendor
handlers remain unavailable. Broker execution is a cooperative API
contract, not an OS sandbox; a worker and a native plugin can still issue
direct syscalls.
reach the broker's `fs.read` handler when granted (ADR-0057). Only
`fs.read` has a registered handler; every other core capability and every
vendor capability without an explicitly registered trusted handler remains
`capability.unavailable`, and there is no API to register one from a plugin,
manifest or CLI flag. Broker execution is a cooperative API contract, not an
OS sandbox; a worker and a native plugin can still issue direct syscalls.
Follow-ups: wire the run `trace_id` into the broker audit and teach the host
plugin-error mapping to report `resource.exhausted` as a resource category
rather than `permission.denied`.
6. `plugin_handle` wire values are structurally reconstructable, but host
authority comes only from the `PluginHandleTable` owner/run/generation
check; untrusted plugin/foreign output containing a handle is rejected
+16 -1
View File
@@ -1,5 +1,20 @@
{
"name": "marqdo-stdlib",
"version": "0.1.0",
"digest": "sha256:b94f1a48ca6619689584aca3af7befac64ac9fdb7ea7c0a9ccb491df8df226b8"
"digest": "sha256:5407d3665ba2fe105820d0f1af5efaafa10c277a69eb819c3225719813c8c713",
"public_aliases": [
{
"public_names": [
"print",
"打印"
],
"target_module": "io.mq.md",
"target_function": "写值",
"expected_signature": "params(值)",
"effects": [
"host"
]
}
],
"aliases_digest": "sha256:5f513c0fdbb3c1e9829be3c9fa23a2d1e89cb8510073fb5eb1e8e2f5265a83bb"
}
+15 -15
View File
@@ -1,38 +1,38 @@
---
import:
fs: "std:fs/file"
fs_lib: "std:fs/file"
---
# @exec fs
## main
**创建 = fs.创建目录("out")**
**创建 = fs_lib.创建目录("out")**
**写入 = fs.写文本("out/fs-a.txt", "你好\n")**
**写入 = fs_lib.写文本("out/fs-a.txt", "你好\n")**
**存在 = fs.存在("out/fs-a.txt")**
**存在 = fs_lib.存在("out/fs-a.txt")**
**元数据 = fs.元数据("out/fs-a.txt")**
**元数据 = fs_lib.元数据("out/fs-a.txt")**
**列表 = fs.列出目录("out")**
**列表 = fs_lib.列出目录("out")**
**是文件 = fs.是文件("out/fs-a.txt")**
**是文件 = fs_lib.是文件("out/fs-a.txt")**
**是目录 = fs.是目录("out")**
**是目录 = fs_lib.是目录("out")**
**重命名 = fs.重命名("out/fs-a.txt", "out/fs-b.txt")**
**重命名 = fs_lib.重命名("out/fs-a.txt", "out/fs-b.txt")**
**读取 = fs.读文本("out/fs-b.txt")**
**读取 = fs_lib.读文本("out/fs-b.txt")**
**追加 = fs.追加文本("out/fs-b.txt", "世界\n")**
**追加 = fs_lib.追加文本("out/fs-b.txt", "世界\n")**
**片段 = fs.读取片段("out/fs-b.txt", 0, 100)**
**片段 = fs_lib.读取片段("out/fs-b.txt", 0, 100)**
**遍历 = fs.遍历("out")**
**遍历 = fs_lib.遍历("out")**
**删除 = fs.删除("out/fs-b.txt")**
**删除 = fs_lib.删除("out/fs-b.txt")**
**删除目录 = fs.删除("out")**
**删除目录 = fs_lib.删除("out")**
`return [创建.ok, 写入.value.count, 存在.value, 元数据.value.kind, 元数据.value.size, 列表.value, 是文件.value, 是目录.value, 重命名.ok, 读取.value, 追加.value, 片段.value, 遍历.value, 删除.ok, 删除目录.ok]`
+9 -9
View File
@@ -1,6 +1,6 @@
---
import:
table: "std:data/table"
table_lib: "std:data/table"
---
# @exec table
@@ -9,22 +9,22 @@ import:
**数据 = 表(["名称", "分数"], [["甲", 87], ["乙", 92], ["丙", 71]])**
**选择 = table.选择列(数据, 1)**
**选择 = table_lib.选择列(数据, 1)**
**过滤 = table.过滤数值大于(数据, 1, 80)**
**过滤 = table_lib.过滤数值大于(数据, 1, 80)**
**排序 = table.排序(数据, 1)**
**排序 = table_lib.排序(数据, 1)**
**追加 = table.追加行(数据, ["丁", 65])**
**追加 = table_lib.追加行(数据, ["丁", 65])**
**设置 = table.设置单元格(数据, 0, 1, 99)**
**设置 = table_lib.设置单元格(数据, 0, 1, 99)**
**左 = 表(["名称", "分数"], [["甲", 87], ["乙", 92]])**
**右 = 表(["名称", "等级"], [["乙", "A"], ["甲", "B"]])**
**连接 = table.连接(左, 右, 0, 0)**
**连接 = table_lib.连接(左, 右, 0, 0)**
**分组 = table.分组(表(["类别", "值"], [["a", 1], ["b", 2], ["a", 3]]), 0, 1)**
**分组 = table_lib.分组(表(["类别", "值"], [["a", 1], ["b", 2], ["a", 3]]), 0, 1)**
`return [选择, 过滤, 排序, 追加, 设置, 连接, 分组, 数据, table.转GFM(数据)]`
`return [选择, 过滤, 排序, 追加, 设置, 连接, 分组, 数据, table_lib.转GFM(数据)]`
+2 -2
View File
@@ -1,6 +1,6 @@
---
import:
text: "std:text"
text_lib: "std:text"
---
# @exec text
@@ -9,6 +9,6 @@ import:
**内存 = memory.new(1, 1)**
**长度 = text.utf8编码("é", 内存, 0)**
**长度 = text_lib.utf8编码("é", 内存, 0)**
`return [长度, 内存[0], 内存[1]]`
+245
View File
@@ -0,0 +1,245 @@
#!/usr/bin/env python3
"""Frozen-CLI external-extension evidence generator and validator.
RFC-EXT-0001 §10.2/§11.3: a release candidate must prove that independent
repositories work against an *unchanged* CLI binary. This script:
1. hashes the CLI binary before and after the external suite and refuses to
emit PASS if the hash moved;
2. records the exact marqdo-cli source SHA and the external fixture repo SHA;
3. runs the fixture repository's own black-box suite with ``MARQDO_BIN`` set to
the frozen binary (the suite itself must not run cargo or read the CLI
source tree);
4. validates/normalises the per-case results into the repository evidence
schema (``docs/evidence/extension/schema.json``);
5. writes ``docs/evidence/extension/<tested_sha>/manifest.json``.
Usage:
python3 tools/ci/verify_extension_contracts.py \
--cli target/release/marqdo \
--fixtures ../marqdo-extension-conformance \
--out docs/evidence/extension/<sha>/manifest.json
python3 tools/ci/verify_extension_contracts.py --validate manifest.json
Only Python's standard library is used. No network access is performed by this
script; the external suite documents its own offline cache behaviour.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import platform
import subprocess
import sys
import tempfile
from pathlib import Path
REPO = Path(__file__).resolve().parents[2]
SCHEMA = REPO / "docs" / "evidence" / "extension" / "schema.json"
REQUIRED_RESULTS = [
"callable_registry",
"plugin_worker",
"foreign_kernel",
"resource_provider_cli",
"package_pinned_git",
"tool_analysis_consumer",
"negative_security",
]
ALLOWED = {"PASS", "FAIL", "UNSUPPORTED", "SKIPPED"}
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def git_sha(directory: Path) -> str:
output = subprocess.run(
["git", "-C", str(directory), "rev-parse", "HEAD"],
check=True,
capture_output=True,
text=True,
)
return output.stdout.strip()
def os_name() -> str:
name = platform.system().lower()
if name.startswith("linux"):
return "linux"
if name == "darwin":
return "macos"
if name.startswith("windows"):
return "windows"
raise SystemExit(f"unsupported OS for extension evidence: {name}")
def validate_manifest(manifest: dict) -> list[str]:
problems: list[str] = []
if manifest.get("schema") != "marqdo.extension-evidence":
problems.append("schema must be marqdo.extension-evidence")
if manifest.get("schema_version") != 1:
problems.append("schema_version must be 1")
for field in ["tested_sha", "fixture_repo_sha"]:
value = manifest.get(field, "")
if not (isinstance(value, str) and len(value) == 40 and all(c in "0123456789abcdef" for c in value)):
problems.append(f"{field} must be a 40-character lowercase git SHA")
cli = manifest.get("cli_binary_sha256", "")
if not (isinstance(cli, str) and len(cli) == 64 and all(c in "0123456789abcdef" for c in cli)):
problems.append("cli_binary_sha256 must be a 64-character lowercase sha256")
if manifest.get("os") not in {"linux", "macos", "windows"}:
problems.append("os must be linux, macos or windows")
results = manifest.get("results")
if not isinstance(results, dict):
problems.append("results must be an object")
else:
for name in REQUIRED_RESULTS:
if results.get(name) not in ALLOWED:
problems.append(f"results.{name} must be one of {sorted(ALLOWED)}")
for key, value in results.items():
if key not in REQUIRED_RESULTS:
problems.append(f"unknown result key {key}")
if value not in ALLOWED:
problems.append(f"result {key}={value} is not allowed")
commands = manifest.get("commands")
if not isinstance(commands, list) or not commands or not all(isinstance(c, str) for c in commands):
problems.append("commands must be a non-empty array of strings")
for field in ["failures", "unverified"]:
if not isinstance(manifest.get(field), list):
problems.append(f"{field} must be an array")
if not problems:
for name, value in (manifest.get("results") or {}).items():
if value in {"FAIL"}:
problems.append(f"result {name} is FAIL")
return problems
def run_suite(fixtures: Path, cli: Path) -> tuple[bool, dict, list[str]]:
"""Run the external suite and return (ok, results, commands)."""
results: dict[str, str] = {}
commands: list[str] = []
env = dict(os.environ)
env["MARQDO_BIN"] = str(cli)
env["MARQDO_FROZEN_CLI"] = str(cli)
runner = fixtures / "tests" / "run_all.py"
if not runner.is_file():
return False, results, commands
commands.append(f"python3 {runner} (cwd={fixtures})")
completed = subprocess.run(
[sys.executable, str(runner)],
cwd=fixtures,
env=env,
capture_output=True,
text=True,
)
print(completed.stdout, end="")
print(completed.stderr, end="", file=sys.stderr)
results_path = fixtures / "evidence" / "results.json"
if results_path.is_file():
with results_path.open("r", encoding="utf-8") as handle:
payload = json.load(handle)
for key, value in (payload.get("results") or {}).items():
results[key] = str(value).upper()
return completed.returncode == 0, results, commands
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--cli", type=Path, help="Absolute/relative frozen marqdo binary")
parser.add_argument("--fixtures", type=Path, help="External fixture repository root")
parser.add_argument("--source-repo", type=Path, default=REPO, help="marqdo-cli checkout")
parser.add_argument("--out", type=Path, help="Evidence manifest output path")
parser.add_argument("--validate", type=Path, help="Only validate an existing manifest")
parser.add_argument("--allow-unverified", action="store_true", help="Record UNSUPPORTED/SKIPPED without failing")
args = parser.parse_args()
if args.validate:
manifest = json.loads(args.validate.read_text(encoding="utf-8"))
problems = validate_manifest(manifest)
if problems:
for problem in problems:
print(f"INVALID: {problem}", file=sys.stderr)
return 1
print(f"valid extension evidence: {args.validate}")
return 0
if not args.cli or not args.fixtures:
parser.error("--cli and --fixtures are required unless --validate is used")
cli = args.cli.resolve()
fixtures = args.fixtures.resolve()
if not cli.is_file():
print(f"error: CLI binary not found: {cli}", file=sys.stderr)
return 2
if not (fixtures / ".git").exists():
print(f"error: fixture path is not a git repository: {fixtures}", file=sys.stderr)
return 2
if subprocess.run(
["git", "-C", str(fixtures), "status", "--porcelain"],
capture_output=True,
text=True,
check=True,
).stdout.strip():
print("error: fixture repository has uncommitted changes", file=sys.stderr)
return 2
cli_before = sha256_file(cli)
tested_sha = git_sha(args.source_repo.resolve())
fixture_sha = git_sha(fixtures)
ok, results, commands = run_suite(fixtures, cli)
cli_after = sha256_file(cli)
if cli_before != cli_after:
print("error: frozen CLI hash changed during the external suite", file=sys.stderr)
return 1
failures = [name for name, value in results.items() if value == "FAIL"]
unverified = [
name for name, value in results.items() if value in {"UNSUPPORTED", "SKIPPED"}
]
missing = [name for name in REQUIRED_RESULTS if name not in results]
if missing:
failures.extend(f"missing:{name}" for name in missing)
manifest = {
"schema": "marqdo.extension-evidence",
"schema_version": 1,
"tested_sha": tested_sha,
"cli_binary_sha256": cli_before,
"fixture_repo_sha": fixture_sha,
"os": os_name(),
"arch": platform.machine(),
"results": {name: results.get(name, "SKIPPED") for name in REQUIRED_RESULTS},
"commands": commands,
"failures": failures,
"unverified": unverified,
}
problems = validate_manifest(manifest)
if unverified and not args.allow_unverified:
problems.append(
"UNSUPPORTED/SKIPPED results require --allow-unverified and an explicit report entry"
)
if not ok and not failures:
failures.append("external suite exited non-zero without structured results")
problems.append("external suite failed")
if problems:
for problem in problems:
print(f"FAIL: {problem}", file=sys.stderr)
print(json.dumps(manifest, indent=2))
return 1
out = args.out or (REPO / "docs" / "evidence" / "extension" / tested_sha / "manifest.json")
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8")
print(f"wrote extension evidence: {out}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+20
View File
@@ -18,6 +18,22 @@ STDLIB = ROOT / "stdlib"
DIST = ROOT / "dist"
def aliases_digest(manifest: dict) -> str:
digest = hashlib.sha256()
for alias in sorted(
manifest.get("public_aliases", []),
key=lambda a: (a["target_module"], a["target_function"]),
):
digest.update(alias["target_module"].encode())
digest.update(b"\0")
digest.update(alias["target_function"].encode())
digest.update(b"\0")
for name in sorted(alias["public_names"]):
digest.update(name.encode())
digest.update(b"\0")
return f"sha256:{digest.hexdigest()}"
def tree_digest() -> str:
digest = hashlib.sha256()
for path in sorted(
@@ -37,6 +53,9 @@ def main() -> int:
digest = tree_digest()
if manifest["digest"] != f"sha256:{digest}":
raise SystemExit("stdlib.json digest does not match the tree")
alias_digest = aliases_digest(manifest)
if manifest.get("aliases_digest") != alias_digest:
raise SystemExit("stdlib.json aliases_digest does not match public_aliases")
files = sorted(
[p for p in STDLIB.rglob("*") if p.is_file() and p.name != "stdlib.json"]
@@ -65,6 +84,7 @@ def main() -> int:
"name": manifest["name"],
"version": version,
"tree_digest": f"sha256:{digest}",
"aliases_digest": alias_digest,
"archive": archive.name,
"archive_sha256": f"sha256:{archive_sha}",
"files": [p.relative_to(STDLIB).as_posix() for p in files],