feat(ext): single callable binding, provider v2 CLI, capability handlers, stdlib aliases
Implements RFC-EXT-0001 PR-00..PR-07 first pass: - CallSiteKey/CallBindingTable + one frontend binding pass; runtime dispatch by identity - prepare composition root shared by check/run/inspect/Analysis; exact plugin reconciliation - provider.json v2 --provider/--allow-provider worker CLI with fail-closed verification - capability handler registry with real fs.read and audit - verified stdlib public aliases replace print_override/AST scan - docs: specs, ADR-0054/0055/0057/0058, README, CI/release gates, evidence PR-04 extension selection/lock v4 and final frozen-binary evidence remain.
This commit is contained in:
1 parent
9aa536c2c9
commit
1f6fd644a0
54 files changed
+8710
-828
No files matched your search
@@ -163,6 +163,25 @@ rules are [`RFC-LANG-0002`](docs/architecture/RFC-LANG-0002-unified-semantics.md
|
||||
* **Provider package manifest v1 (RFC-CM CM-A09)** — local `marqdo.provider`
|
||||
manifest installs a trusted C ABI provider; no registry/signature/CLI install
|
||||
(ADR-0037).
|
||||
* **Third-party Resource Provider v2 through the normal CLI (RFC-EXT-0001
|
||||
PR-03)** — a `provider.json` v2 bundle with a targeted, sha256-verified
|
||||
`c-abi-1` artifact is selected with repeatable `--provider` and authorized by
|
||||
the exact `--allow-provider` name on `run`/`call`; the existing
|
||||
`marqdo-provider-worker` transport is spawned only after every manifest,
|
||||
artifact, grant and duplicate-kind check passes. `check`/`inspect` never load
|
||||
providers, `pure`/`strict` refuse them, and the worker transport is
|
||||
linux/x86_64-only in this release (ADR-0055,
|
||||
[`spec/resource-provider-v2.md`](spec/resource-provider-v2.md)).
|
||||
* **Single callable binding through the frozen composition root
|
||||
(RFC-EXT-0001 PR-01/02)** — one `CallableRegistry` and one binding pass feed
|
||||
`check`/`run`/`call`/`inspect`/Analysis; the runtime dispatches by
|
||||
`CallSiteKey` and never re-resolves a name or guesses a plugin. Static plugin
|
||||
descriptors must reconcile one-to-one with the loaded exports (ADR-0054,
|
||||
[`spec/callable-resolution.md`](spec/callable-resolution.md)).
|
||||
* **Official stdlib public aliases (RFC-EXT-0001 PR-06)** — `打印`/`print` are
|
||||
verified `stdlib.json` metadata bound to `io.mq.md::写值`; the
|
||||
`print_override`/AST-scan/runtime special case is removed with byte-identical
|
||||
output (ADR-0058).
|
||||
* **C ABI ResourceProvider v1 (RFC-CM CM-A09)** — trusted native shared
|
||||
libraries expose resource open/read/write/close through a frozen header;
|
||||
project + name grant; plugin-Worker binding remains open (ADR-0036).
|
||||
@@ -358,6 +377,115 @@ marqdo benchmark run .
|
||||
marqdo tool remove benchmark
|
||||
```
|
||||
|
||||
### Extending Marqdo without modifying or rebuilding the CLI
|
||||
|
||||
RFC-EXT-0001 defines five extension kinds. Each one is discovered only through
|
||||
an explicit manifest/flag, is versioned, and is authorized separately from
|
||||
being visible. A physically separate repository exercising all five against a
|
||||
frozen binary is `marqdo-extension-conformance`
|
||||
(`https://git.cflmy.cn/cflmy/marqdo-extension-conformance`).
|
||||
|
||||
| Kind | Static discovery | Execution | Grant |
|
||||
|---|---|---|---|
|
||||
| `.mq.md` package | `marqdo.yaml` dependency + `marqdo.lock` | Marqdo interpreter | project mode |
|
||||
| Callable plugin | `plugin.yaml` ABI v1 exports | worker (default) or trusted native | `--plugin` + `--allow-plugin NAME` |
|
||||
| Foreign kernel | `kernel.json` + pinned executable digest | one-shot child process | `--kernel` + `--allow-foreign` |
|
||||
| Resource provider | `provider.json` schema v2 | `marqdo-provider-worker` + C ABI | `--provider` + `--allow-provider NAME` |
|
||||
| CLI Tool | `tool.json` install record | independent CLI process | tool installation, no project grants |
|
||||
|
||||
```bash
|
||||
# 1. Pure Markdown library from a pinned Git dependency
|
||||
marqdo lock && marqdo sync
|
||||
marqdo check consumer/index.mq.md
|
||||
marqdo run consumer/index.mq.md
|
||||
|
||||
# 2. Callable ABI plugin (worker transport by default)
|
||||
marqdo run plugin/demo.mq.md \
|
||||
--plugin plugin/build --allow-plugin demo.ext
|
||||
|
||||
# 3. Foreign kernel with a language label the CLI has never heard of
|
||||
marqdo run kernel/demo.mq.md \
|
||||
--kernel kernel/build/kernel.json --allow-foreign
|
||||
|
||||
# 4. Resource provider v2 (see the next section)
|
||||
marqdo run provider/demo.mq.md \
|
||||
--provider provider/build/provider.json --allow-provider demo.sensor
|
||||
|
||||
# 5. Independent CLI Tool consuming only public Analysis v2
|
||||
marqdo tool install tool/tool.json
|
||||
marqdo inspect-ext tool/demo.mq.md
|
||||
```
|
||||
|
||||
#### One callable resolution, visible in `check` and `run`
|
||||
|
||||
Calls are bound statically once per prepared program (ADR-0054, normative
|
||||
details in [`spec/callable-resolution.md`](spec/callable-resolution.md)):
|
||||
|
||||
* `f(...)`: local callable value → official alias/reserved builtin → current
|
||||
namespace → canonical name → unique local name;
|
||||
* `alias.f(...)`: explicit import alias → builtin dotted name → exact plugin
|
||||
export;
|
||||
* `some_value.f(...)`: dynamic language value only.
|
||||
|
||||
`check` and `run` consume the same binding; an unknown export, wrong arguments
|
||||
or a duplicate export fails before execution. An import alias that collides
|
||||
with a builtin/plugin dotted namespace (for example importing a module as
|
||||
`math`) is rejected with `ext.import_namespace_conflict`; rename it. A plugin
|
||||
manifest is the static contract: `run` verifies the loaded artifact and its
|
||||
exports/signatures match the manifest exactly
|
||||
(`ext.plugin_binding_mismatch` otherwise) and never synthesises a signature.
|
||||
|
||||
#### Official stdlib aliases, no print magic
|
||||
|
||||
`打印`/`print` are declared in the verified `stdlib/stdlib.json`
|
||||
(`public_aliases`, covered by `aliases_digest`) and bound as ordinary
|
||||
`Marqdo` calls to `io.mq.md::写值` (ADR-0058). There is no
|
||||
`Program.print_override`, no AST-wide print scan and no runtime print branch;
|
||||
output bytes, permissions and Trace behavior are unchanged, and a program that
|
||||
never uses the alias does not load the stdlib module graph. Third-party
|
||||
packages cannot claim `打印`/`print` or `std.*` names.
|
||||
|
||||
### Third-party resource providers (`provider.json` v2)
|
||||
|
||||
An external repository adds a new resource kind (for example `demo.sensor`)
|
||||
with a `provider.json` schema-version-2 bundle and a targeted `c-abi-1` shared
|
||||
library. No CLI rebuild and no core change are needed: select the manifest with
|
||||
repeatable `--provider` and authorize it by its exact manifest `name` with
|
||||
`--allow-provider` (also repeatable). A `--provider` without a matching
|
||||
`--allow-provider` is `permission.denied` and the helper is never started;
|
||||
wildcards (`*`) are not accepted.
|
||||
|
||||
```bash
|
||||
marqdo run index.mq.md \
|
||||
--provider vendor/demo-sensor/provider.json \
|
||||
--allow-provider demo.sensor
|
||||
|
||||
marqdo call index.mq.md main \
|
||||
--provider vendor/demo-sensor/provider.json \
|
||||
--allow-provider demo.sensor
|
||||
```
|
||||
|
||||
The document still uses the ordinary resource surface, e.g.
|
||||
`resource.open("demo.sensor", "unit-1", ["read"])`; provider kinds do not become
|
||||
new callables.
|
||||
|
||||
* `check`/`inspect` do not accept these options and never load a provider.
|
||||
* `--mode pure` and `--mode strict` reject `--provider`/`--allow-provider` in
|
||||
preflight; strict L1 stays fail-closed and does not silently switch to
|
||||
`project`.
|
||||
* Everything is verified before any worker starts: `schema_version: 2`,
|
||||
`abi: c-abi-1`, `transport: worker`, strict SemVer, non-reserved kinds, the
|
||||
current target artifact's bundle containment and sha256, exact grants and
|
||||
duplicate provider names/kinds.
|
||||
* The helper `marqdo-provider-worker` must ship next to the `marqdo` executable
|
||||
for the same target (no `PATH` fallback); a missing helper is
|
||||
`ext.worker_unavailable`. The worker transport is linux/x86_64-only in this
|
||||
release; other platforms fail with `ext.platform_unsupported`.
|
||||
* The helper is a trusted process, not an OS sandbox: it runs with the user's
|
||||
privileges. `sha256` is an integrity check, not publisher authentication.
|
||||
See [`spec/resource-provider-v2.md`](spec/resource-provider-v2.md) and
|
||||
[ADR-0055](docs/adr/0055-resource-provider-cli-v2.md).
|
||||
|
||||
## References
|
||||
|
||||
* [CommonMark 0.31.2](https://spec.commonmark.org/0.31.2/) ·
|
||||
|
||||
+23
-31
@@ -536,7 +536,7 @@ fn expr_calls(
|
||||
) {
|
||||
match expr {
|
||||
Expr::Call { callee, args, span } => {
|
||||
let (state, target) = resolve_callee(project, module, callee);
|
||||
let (state, target) = resolve_callee(project, module, function, *span);
|
||||
calls.push(json!({
|
||||
"function": function.id.to_string(),
|
||||
"span": span_json(*span),
|
||||
@@ -572,42 +572,34 @@ fn expr_calls(
|
||||
}
|
||||
}
|
||||
|
||||
/// Project the authoritative call binding into the Analysis `state`/`target`
|
||||
/// fields.
|
||||
///
|
||||
/// Analysis is a read-only projection of the same prepared binding the runtime
|
||||
/// consumes (RFC-EXT-0001 §4/§9): there is no second name-resolution algorithm
|
||||
/// here. `state` keeps its v1 vocabulary (`function`, `builtin`, `plugin`,
|
||||
/// `unresolved`); official stdlib aliases such as `打印` therefore appear as
|
||||
/// `function` targets because that is the implementation that actually runs.
|
||||
fn resolve_callee(
|
||||
project: &LoadedProject,
|
||||
module: &Module,
|
||||
callee: &Expr,
|
||||
_module: &Module,
|
||||
function: &Function,
|
||||
span: Span,
|
||||
) -> (&'static str, Value) {
|
||||
match callee {
|
||||
Expr::Ident(ident) => {
|
||||
if marqdo_ir::builtins::find(&ident.name).is_some() {
|
||||
("builtin", json!(ident.name))
|
||||
} else if let Some(function) = module.function(&ident.name) {
|
||||
("function", json!(function.id.to_string()))
|
||||
} else {
|
||||
("unresolved", Value::Null)
|
||||
}
|
||||
let key = marqdo_ir::call_binding::CallSiteKey::new(function.id.clone(), span);
|
||||
match project.program.call_bindings.get(&key) {
|
||||
Some(marqdo_ir::call_binding::BoundTarget::Builtin { canonical_name }) => {
|
||||
("builtin", json!(canonical_name))
|
||||
}
|
||||
Expr::Member { base, name, .. } => {
|
||||
if let Expr::Ident(alias) = base.as_ref() {
|
||||
let qualified = format!("{}.{}", alias.name, name);
|
||||
if marqdo_ir::builtins::find(&qualified).is_some() {
|
||||
return ("builtin", json!(qualified));
|
||||
}
|
||||
if let Some(target_module) = project.program.resolve_import(&module.id, &alias.name)
|
||||
{
|
||||
if let Some(target) = project.program.module(target_module) {
|
||||
if let Some(function) = target.function(name) {
|
||||
return ("function", json!(function.id.to_string()));
|
||||
}
|
||||
}
|
||||
}
|
||||
if project.program.external_callable(&qualified).is_some() {
|
||||
return ("plugin", json!(qualified));
|
||||
}
|
||||
}
|
||||
Some(marqdo_ir::call_binding::BoundTarget::Marqdo { function }) => {
|
||||
("function", json!(function.to_string()))
|
||||
}
|
||||
Some(marqdo_ir::call_binding::BoundTarget::AbiPlugin { export, .. }) => {
|
||||
("plugin", json!(export))
|
||||
}
|
||||
Some(marqdo_ir::call_binding::BoundTarget::DynamicValue) | None => {
|
||||
("unresolved", Value::Null)
|
||||
}
|
||||
_ => ("unresolved", Value::Null),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+26
-132
@@ -27,148 +27,42 @@ pub use sandbox::SandboxSettings;
|
||||
pub use tool::{fmt, plugin_inspect};
|
||||
|
||||
use crate::args;
|
||||
use crate::loader::{self, FileDiagnostic, LoadedProject};
|
||||
use crate::extension_descriptors::{self, StaticPluginSet};
|
||||
use crate::loader::{FileDiagnostic, LoadedProject};
|
||||
use crate::output;
|
||||
use crate::project_prepare::{self, PrepareOptions, PreparedProgram};
|
||||
|
||||
fn load_project(file: Option<&Path>, json: bool) -> Result<LoadedProject, i32> {
|
||||
let file = file.unwrap_or_else(|| Path::new("index.mq.md"));
|
||||
match loader::load(file) {
|
||||
Ok(project) => Ok(project),
|
||||
/// Prepare one document through the single composition-root step.
|
||||
///
|
||||
/// `check`/`run`/`call`/`inspect` must all use this function so they share the
|
||||
/// same call bindings and diagnostics (RFC-EXT-0001 §4.4, §18.2).
|
||||
pub(super) fn prepare_project(
|
||||
file: Option<&Path>,
|
||||
json: bool,
|
||||
plugins: &[PathBuf],
|
||||
) -> Result<PreparedProgram, i32> {
|
||||
let options = PrepareOptions {
|
||||
plugins: plugins.to_vec(),
|
||||
};
|
||||
match project_prepare::prepare(file, &options) {
|
||||
Ok(prepared) => Ok(prepared),
|
||||
Err(error) => {
|
||||
if json {
|
||||
output::print_json(&json!({"success": false, "error": error}));
|
||||
output::print_json(&json!({
|
||||
"success": false,
|
||||
"error": {
|
||||
"code": error.code,
|
||||
"message": error.message,
|
||||
}
|
||||
}));
|
||||
} else {
|
||||
eprintln!("error: {error}");
|
||||
}
|
||||
if error.contains("marqdo.lock") || error.contains("lockfile") {
|
||||
Err(1)
|
||||
} else {
|
||||
Err(2)
|
||||
eprintln!("error[{}]: {}", error.code, error.message);
|
||||
}
|
||||
Err(error.exit_code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Add plugin export descriptors from manifests without loading a library.
|
||||
fn add_static_plugin_descriptors(
|
||||
project: &mut LoadedProject,
|
||||
paths: &[PathBuf],
|
||||
) -> Result<(), String> {
|
||||
for path in paths {
|
||||
let manifest_path = if path.is_dir() {
|
||||
path.join("plugin.yaml")
|
||||
} else {
|
||||
path.clone()
|
||||
};
|
||||
let manifest = marqdo_plugin_abi::PluginManifest::load(&manifest_path)
|
||||
.map_err(|error| format!("{}: {}", error.code, error.message))?;
|
||||
manifest
|
||||
.validate()
|
||||
.map_err(|error| format!("{}: {}", error.code, error.message))?;
|
||||
let digest = manifest
|
||||
.resolve_artifact(&marqdo_plugin_abi::HostTarget::current())
|
||||
.map_err(|error| format!("{}: {}", error.code, error.message))?
|
||||
.sha256
|
||||
.clone();
|
||||
for export in &manifest.plugin.exports {
|
||||
if project
|
||||
.program
|
||||
.external_callables
|
||||
.iter()
|
||||
.any(|callable| callable.public_name == export.name)
|
||||
{
|
||||
return Err(format!(
|
||||
"plugin export `{}` is declared more than once",
|
||||
export.name
|
||||
));
|
||||
}
|
||||
let signature = plugin_signature(&export.signature).map_err(|error| {
|
||||
format!(
|
||||
"plugin export `{}` has an invalid signature: {error}",
|
||||
export.name
|
||||
)
|
||||
})?;
|
||||
project
|
||||
.program
|
||||
.external_callables
|
||||
.push(marqdo_ir::callable::CallableDescriptor {
|
||||
id: marqdo_ir::callable::CallableId(format!(
|
||||
"plugin:{}:{}",
|
||||
manifest.plugin.name, export.name
|
||||
)),
|
||||
public_name: export.name.clone(),
|
||||
owner_package: manifest.plugin.name.clone(),
|
||||
signature,
|
||||
effects_requested: manifest.plugin.requires.clone(),
|
||||
origin: marqdo_ir::callable::CallableOrigin::AbiPlugin {
|
||||
package: manifest.plugin.name.clone(),
|
||||
export: export.name.clone(),
|
||||
},
|
||||
implementation_digest: Some(digest.clone()),
|
||||
source_span: None,
|
||||
});
|
||||
}
|
||||
}
|
||||
if paths.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
let existing: Vec<(Option<marqdo_span::Span>, String)> = project
|
||||
.diagnostics
|
||||
.iter()
|
||||
.map(|file| {
|
||||
(
|
||||
file.diagnostic.span,
|
||||
format!("{}:{}", file.module_id, file.diagnostic.message),
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
for (module_id, diagnostic) in marqdo_frontend::calls::validate_program(&project.program) {
|
||||
let key = format!("{}:{}", module_id, diagnostic.message);
|
||||
if existing.contains(&(diagnostic.span, key)) {
|
||||
continue;
|
||||
}
|
||||
if let Some(file) = project.files.get(&module_id) {
|
||||
project.diagnostics.push(FileDiagnostic {
|
||||
module_id,
|
||||
path: file.path.clone(),
|
||||
source: file.source.clone(),
|
||||
diagnostic,
|
||||
});
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Convert a plugin manifest signature (`($a: Int) -> Int`) to the language
|
||||
/// parameter list, dropping the display-only return annotation and `$` sigils.
|
||||
fn plugin_signature(raw: &str) -> Result<marqdo_ir::callable::CallableSignature, String> {
|
||||
let before_return = raw.split("->").next().unwrap_or(raw).trim();
|
||||
let open = before_return
|
||||
.find('(')
|
||||
.ok_or_else(|| "missing `(`".to_string())?;
|
||||
let close = before_return
|
||||
.rfind(')')
|
||||
.ok_or_else(|| "missing `)`".to_string())?;
|
||||
if close < open {
|
||||
return Err("malformed parameter list".to_string());
|
||||
}
|
||||
let params = before_return[open + 1..close].replace('$', "");
|
||||
let parsed =
|
||||
marqdo_frontend::signature::parse(&format!("({params})"), marqdo_span::Span::default());
|
||||
if parsed.has_errors() {
|
||||
return Err(parsed
|
||||
.diagnostics
|
||||
.iter()
|
||||
.map(|diagnostic| diagnostic.message.clone())
|
||||
.collect::<Vec<_>>()
|
||||
.join("; "));
|
||||
}
|
||||
Ok(marqdo_ir::callable::CallableSignature {
|
||||
params: parsed.params,
|
||||
kind: marqdo_ir::SignatureKind::Explicit,
|
||||
})
|
||||
}
|
||||
|
||||
/// `marqdo check`.
|
||||
fn validate_refs(project: &LoadedProject) -> Vec<FileDiagnostic> {
|
||||
let mut diagnostics = Vec::new();
|
||||
|
||||
@@ -9,18 +9,11 @@ pub fn check(
|
||||
plugins: &[PathBuf],
|
||||
emit: Option<&Path>,
|
||||
) -> i32 {
|
||||
let mut project = match load_project(file, json) {
|
||||
Ok(project) => project,
|
||||
let prepared = match prepare_project(file, json, plugins) {
|
||||
Ok(prepared) => prepared,
|
||||
Err(code) => return code,
|
||||
};
|
||||
if let Err(error) = add_static_plugin_descriptors(&mut project, plugins) {
|
||||
if json {
|
||||
output::print_json(&json!({"success": false, "error": error}));
|
||||
} else {
|
||||
eprintln!("error: {error}");
|
||||
}
|
||||
return 2;
|
||||
}
|
||||
let mut project = prepared.loaded;
|
||||
if refs {
|
||||
let ref_diagnostics = validate_refs(&project);
|
||||
project.diagnostics.extend(ref_diagnostics);
|
||||
|
||||
@@ -20,10 +20,11 @@ pub struct InspectFlags {
|
||||
}
|
||||
|
||||
pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool) -> i32 {
|
||||
let project = match load_project(file, json) {
|
||||
Ok(project) => project,
|
||||
let prepared = match prepare_project(file, json, &[]) {
|
||||
Ok(prepared) => prepared,
|
||||
Err(code) => return code,
|
||||
};
|
||||
let project = prepared.loaded;
|
||||
let entry = project.entry_file();
|
||||
let module = &entry.analysis.module;
|
||||
let selected_any =
|
||||
|
||||
+253
-42
@@ -4,6 +4,7 @@
|
||||
//! runtime helpers, but keeps the run session out of the command table.
|
||||
|
||||
use super::*;
|
||||
use crate::provider_manifest;
|
||||
use marqdo_sandbox::StrictChildState;
|
||||
|
||||
pub fn execute(
|
||||
@@ -15,18 +16,12 @@ pub fn execute(
|
||||
trace: bool,
|
||||
) -> i32 {
|
||||
let command_name = if function.is_some() { "call" } else { "run" };
|
||||
let mut project = match load_project(file, json) {
|
||||
Ok(project) => project,
|
||||
let prepared = match prepare_project(file, json, &settings.plugins) {
|
||||
Ok(prepared) => prepared,
|
||||
Err(code) => return code,
|
||||
};
|
||||
if let Err(error) = add_static_plugin_descriptors(&mut project, &settings.plugins) {
|
||||
if json {
|
||||
output::print_json(&json!({"success": false, "error": error}));
|
||||
} else {
|
||||
eprintln!("error: {error}");
|
||||
}
|
||||
return 2;
|
||||
}
|
||||
let project = prepared.loaded;
|
||||
let static_plugins = prepared.plugins;
|
||||
if project.has_errors() {
|
||||
if json {
|
||||
output::print_json(&json!({
|
||||
@@ -136,10 +131,80 @@ pub fn execute(
|
||||
return 2;
|
||||
}
|
||||
}
|
||||
for plugin_path in &settings.plugins {
|
||||
if let Err(message) =
|
||||
load_plugin_into_host(&mut host, plugin_path, settings.trust_native_plugin)
|
||||
// T6: verify every selected artifact/path/hash before starting a worker or
|
||||
// loading native code. Static path/signature checks happened in `prepare`.
|
||||
//
|
||||
// Provider selection is host assembly, not callable semantics: `check`
|
||||
// never sees it. Parse and verify every `--provider` manifest, resolve the
|
||||
// current target artifact, check the exact grant and reject duplicate
|
||||
// names/kinds *before* any provider or plugin worker process is spawned
|
||||
// (RFC-EXT-0001 §5.4.6/§18.2 T6–T7).
|
||||
let providers = match provider_manifest::read_provider_manifests(&settings.providers) {
|
||||
Ok(providers) => providers,
|
||||
Err(error) => {
|
||||
emit_prepare_error(json, command_name, &error);
|
||||
return error.exit_code;
|
||||
}
|
||||
};
|
||||
// `--provider` alone never authorizes; the exact `--allow-provider NAME`
|
||||
// is required before the helper process may be started. An ungranted
|
||||
// provider fails closed instead of being silently ignored.
|
||||
for provider in &providers {
|
||||
if !settings
|
||||
.allow_provider
|
||||
.iter()
|
||||
.any(|grant| grant == &provider.name)
|
||||
{
|
||||
let error = extension_descriptors::PrepareError::load(
|
||||
"permission.denied",
|
||||
format!(
|
||||
"resource provider `{}` was selected with --provider but is not granted; \
|
||||
add --allow-provider {}",
|
||||
provider.name, provider.name
|
||||
),
|
||||
);
|
||||
emit_prepare_error(json, command_name, &error);
|
||||
return error.exit_code;
|
||||
}
|
||||
}
|
||||
// A grant that matches no selected provider is a deterministic option
|
||||
// error: providers are never discovered from PATH, the working directory or
|
||||
// the network.
|
||||
for grant in &settings.allow_provider {
|
||||
if !providers.iter().any(|provider| &provider.name == grant) {
|
||||
let error = extension_descriptors::PrepareError::option(
|
||||
"ext.provider_grant_unmatched",
|
||||
format!(
|
||||
"--allow-provider `{grant}` matches no selected --provider manifest; \
|
||||
providers are never discovered automatically"
|
||||
),
|
||||
);
|
||||
emit_prepare_error(json, command_name, &error);
|
||||
return error.exit_code;
|
||||
}
|
||||
}
|
||||
for plugin in &static_plugins.plugins {
|
||||
if let Err(error) = extension_descriptors::verify_bundle_regular_file(
|
||||
&plugin.base_dir,
|
||||
&plugin.artifact_relative,
|
||||
) {
|
||||
emit_prepare_error(json, command_name, &error);
|
||||
return error.exit_code;
|
||||
}
|
||||
let artifact = plugin.artifact_path();
|
||||
if let Err(error) = extension_descriptors::verify_sha256(&artifact, &plugin.artifact_sha256)
|
||||
{
|
||||
emit_prepare_error(json, command_name, &error);
|
||||
return error.exit_code;
|
||||
}
|
||||
}
|
||||
// T7: start only selected, granted and verified extensions.
|
||||
for plugin in &static_plugins.plugins {
|
||||
if let Err(message) = load_plugin_into_host(
|
||||
&mut host,
|
||||
&plugin.manifest_path,
|
||||
settings.trust_native_plugin,
|
||||
) {
|
||||
if json {
|
||||
output::print_json(&json!({"success": false, "error": message}));
|
||||
} else {
|
||||
@@ -148,36 +213,24 @@ pub fn execute(
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
// Expose the complete export names loaded from trusted manifests so the
|
||||
// runtime resolves exactly those names instead of probing namespaces.
|
||||
for (name, _signature, digest, package) in host.plugin_exports() {
|
||||
if project
|
||||
.program
|
||||
.external_callables
|
||||
.iter()
|
||||
.any(|callable| callable.public_name == name)
|
||||
{
|
||||
continue;
|
||||
// T8: exact one-to-one reconciliation with the static descriptors. Never
|
||||
// synthesise a missing signature (`params: Vec::new()`) or guess an export.
|
||||
if let Err(message) = reconcile_plugin_exports(&static_plugins, &host) {
|
||||
emit_error(json, command_name, "ext.plugin_binding_mismatch", &message);
|
||||
return 1;
|
||||
}
|
||||
// T7b: start only granted and verified provider helpers. Every manifest,
|
||||
// artifact, target, grant and duplicate-kind check above already passed, so
|
||||
// no worker is spawned for a provider that failed verification.
|
||||
for provider in &providers {
|
||||
if let Err(error) = provider_manifest::load_resource_provider_into_host(
|
||||
&mut host,
|
||||
provider,
|
||||
&settings.allow_provider,
|
||||
) {
|
||||
emit_prepare_error(json, command_name, &error);
|
||||
return error.exit_code;
|
||||
}
|
||||
project
|
||||
.program
|
||||
.external_callables
|
||||
.push(marqdo_ir::callable::CallableDescriptor {
|
||||
id: marqdo_ir::callable::CallableId(format!("plugin:{package}:{name}")),
|
||||
public_name: name.clone(),
|
||||
owner_package: package.clone(),
|
||||
signature: marqdo_ir::callable::CallableSignature {
|
||||
params: Vec::new(),
|
||||
kind: marqdo_ir::SignatureKind::Explicit,
|
||||
},
|
||||
effects_requested: vec!["host".to_string()],
|
||||
origin: marqdo_ir::callable::CallableOrigin::AbiPlugin {
|
||||
package,
|
||||
export: name,
|
||||
},
|
||||
implementation_digest: Some(digest),
|
||||
source_span: None,
|
||||
});
|
||||
}
|
||||
let function_name = function.unwrap_or("main").to_string();
|
||||
let entry_module = &project.program.modules[&project.entry];
|
||||
@@ -292,6 +345,80 @@ pub fn execute(
|
||||
}
|
||||
}
|
||||
|
||||
fn emit_prepare_error(
|
||||
json: bool,
|
||||
command_name: &str,
|
||||
error: &crate::extension_descriptors::PrepareError,
|
||||
) {
|
||||
if json {
|
||||
output::print_json(&json!({
|
||||
"command": command_name,
|
||||
"success": false,
|
||||
"error": {"code": error.code, "message": error.message},
|
||||
}));
|
||||
} else {
|
||||
eprintln!("error[{}]: {}", error.code, error.message);
|
||||
}
|
||||
}
|
||||
|
||||
/// Compare the Host's actual plugin exports with the static manifest set.
|
||||
///
|
||||
/// Names, raw signatures, owners and verified artifact digests must match
|
||||
/// exactly; extra, missing or altered exports fail closed with
|
||||
/// `ext.plugin_binding_mismatch` and the main function never runs.
|
||||
fn reconcile_plugin_exports(
|
||||
static_plugins: &StaticPluginSet,
|
||||
host: &StdHost,
|
||||
) -> Result<(), String> {
|
||||
let mut expected: std::collections::BTreeMap<String, (String, String, String)> =
|
||||
std::collections::BTreeMap::new();
|
||||
for plugin in &static_plugins.plugins {
|
||||
for export in &plugin.exports {
|
||||
expected.insert(
|
||||
export.name.clone(),
|
||||
(
|
||||
export.signature.clone(),
|
||||
plugin.artifact_sha256.clone(),
|
||||
plugin.package.clone(),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
let loaded = host.plugin_exports();
|
||||
if loaded.len() != expected.len() {
|
||||
return Err(format!(
|
||||
"static plugin descriptors declare {} export(s) but the runtime loaded {}; \
|
||||
refusing to execute",
|
||||
expected.len(),
|
||||
loaded.len()
|
||||
));
|
||||
}
|
||||
for (name, signature, digest, package) in loaded {
|
||||
let Some((expected_signature, expected_digest, expected_package)) = expected.get(&name)
|
||||
else {
|
||||
return Err(format!(
|
||||
"runtime export `{name}` was not declared by any selected plugin manifest"
|
||||
));
|
||||
};
|
||||
if &signature != expected_signature {
|
||||
return Err(format!(
|
||||
"export `{name}` signature mismatch: manifest `{expected_signature}`, runtime `{signature}`"
|
||||
));
|
||||
}
|
||||
if &digest != expected_digest {
|
||||
return Err(format!(
|
||||
"export `{name}` artifact digest mismatch: manifest `{expected_digest}`, runtime `{digest}`"
|
||||
));
|
||||
}
|
||||
if &package != expected_package {
|
||||
return Err(format!(
|
||||
"export `{name}` owner mismatch: manifest `{expected_package}`, runtime `{package}`"
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Read piped stdin once at the composition boundary.
|
||||
///
|
||||
/// A terminal stdin is not read, so `marqdo run` never blocks waiting for an
|
||||
@@ -310,3 +437,87 @@ fn read_stdin() -> Vec<u8> {
|
||||
}
|
||||
bytes
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn empty_host() -> StdHost {
|
||||
StdHost::new(
|
||||
SandboxMode::Project,
|
||||
marqdo_host::Capabilities::pure(),
|
||||
std::env::temp_dir(),
|
||||
)
|
||||
}
|
||||
|
||||
fn static_plugin_with_export() -> StaticPluginSet {
|
||||
let descriptor = marqdo_ir::callable::CallableDescriptor {
|
||||
id: marqdo_ir::callable::CallableId("plugin:demo.ext:solver.add".into()),
|
||||
public_name: "solver.add".into(),
|
||||
owner_package: "demo.ext".into(),
|
||||
signature: marqdo_ir::callable::CallableSignature {
|
||||
params: vec![
|
||||
marqdo_ir::Param {
|
||||
name: "a".into(),
|
||||
span: Span::default(),
|
||||
type_annotation: None,
|
||||
default: None,
|
||||
},
|
||||
marqdo_ir::Param {
|
||||
name: "b".into(),
|
||||
span: Span::default(),
|
||||
type_annotation: None,
|
||||
default: None,
|
||||
},
|
||||
],
|
||||
kind: marqdo_ir::SignatureKind::Explicit,
|
||||
},
|
||||
effects_requested: vec!["math.compute".into()],
|
||||
origin: marqdo_ir::callable::CallableOrigin::AbiPlugin {
|
||||
package: "demo.ext".into(),
|
||||
export: "solver.add".into(),
|
||||
},
|
||||
implementation_digest: Some("a".repeat(64)),
|
||||
source_span: None,
|
||||
};
|
||||
StaticPluginSet {
|
||||
plugins: vec![crate::extension_descriptors::StaticPlugin {
|
||||
manifest_path: PathBuf::from("/fixtures/demo/plugin.yaml"),
|
||||
base_dir: PathBuf::from("/fixtures/demo"),
|
||||
package: "demo.ext".into(),
|
||||
version: "0.1.0".into(),
|
||||
kind: marqdo_plugin_abi::PluginKind::Worker,
|
||||
manifest_sha256: format!("sha256:{}", "b".repeat(64)),
|
||||
artifact_target: marqdo_plugin_abi::HostTarget::current()
|
||||
.triple()
|
||||
.to_string(),
|
||||
artifact_sha256: "a".repeat(64),
|
||||
artifact_relative: "build/plugin.so".into(),
|
||||
exports: vec![crate::extension_descriptors::StaticPluginExport {
|
||||
name: "solver.add".into(),
|
||||
signature: "($a: Int, $b: Int) -> Int".into(),
|
||||
descriptor,
|
||||
}],
|
||||
requires: Vec::new(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reconcile_accepts_matching_empty_sets() {
|
||||
assert!(
|
||||
reconcile_plugin_exports(&StaticPluginSet::default(), &empty_host()).is_ok(),
|
||||
"no selected plugin and no loaded plugin must reconcile"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reconcile_rejects_missing_runtime_exports() {
|
||||
let error = reconcile_plugin_exports(&static_plugin_with_export(), &empty_host())
|
||||
.expect_err("a declared export that the runtime did not load must fail closed");
|
||||
assert!(
|
||||
error.contains("declare 1 export(s)") || error.contains("not declared"),
|
||||
"{error}"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -43,6 +43,10 @@ pub struct SandboxSettings {
|
||||
pub timeout: Option<u64>,
|
||||
/// Explicitly trusted plugin manifests to load.
|
||||
pub plugins: Vec<PathBuf>,
|
||||
/// Explicitly selected resource provider v2 manifests (RFC-EXT-0001 §5.4).
|
||||
pub providers: Vec<PathBuf>,
|
||||
/// Exact resource provider name grants (declaration is not a grant).
|
||||
pub allow_provider: Vec<String>,
|
||||
/// Explicit trust decision for in-process native plugin loading.
|
||||
pub trust_native_plugin: bool,
|
||||
/// Optional Trace v2 output path.
|
||||
@@ -68,8 +72,8 @@ impl SandboxSettings {
|
||||
|
||||
/// Reject strict-mode requests that this L1 isolation level does not
|
||||
/// support. Strict currently covers pure Marqdo and native Math only;
|
||||
/// granting Foreign, ABI plugins, tools or trace files would require
|
||||
/// extending the supervisor contract first.
|
||||
/// granting Foreign, ABI plugins, resource providers, tools or trace files
|
||||
/// would require extending the supervisor contract first.
|
||||
pub fn strict_preflight(&self) -> Result<(), String> {
|
||||
let requested = self.allow_foreign
|
||||
|| !self.allow_read.is_empty()
|
||||
@@ -80,6 +84,8 @@ impl SandboxSettings {
|
||||
|| self.allow_random
|
||||
|| !self.allow_plugin.is_empty()
|
||||
|| !self.plugins.is_empty()
|
||||
|| !self.providers.is_empty()
|
||||
|| !self.allow_provider.is_empty()
|
||||
|| self.trust_native_plugin
|
||||
|| !self.kernels.is_empty()
|
||||
|| self.trace_file.is_some();
|
||||
@@ -87,8 +93,9 @@ impl SandboxSettings {
|
||||
return Err(
|
||||
"strict mode currently supports pure Marqdo and native Math only; \
|
||||
--allow-read/--allow-write/--allow-env/--allow-process/--allow-random/\
|
||||
--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file are
|
||||
rejected. Use --mode project for those capabilities."
|
||||
--allow-foreign/--allow-plugin/--plugin/--provider/--allow-provider/\
|
||||
--trust-native-plugin/--kernel/--trace-file are rejected. \
|
||||
Use --mode project for those capabilities."
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
@@ -108,9 +115,13 @@ impl SandboxSettings {
|
||||
|| self.allow_random
|
||||
|| self.allow_foreign
|
||||
|| !self.allow_plugin.is_empty()
|
||||
|| !self.providers.is_empty()
|
||||
|| !self.allow_provider.is_empty()
|
||||
{
|
||||
return Err(
|
||||
"pure mode prohibits external I/O grants; use --mode project".to_string(),
|
||||
"pure mode prohibits external I/O grants and extension providers; \
|
||||
use --mode project"
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
Ok(Capabilities::pure())
|
||||
@@ -141,6 +152,10 @@ impl SandboxSettings {
|
||||
for plugin in &self.allow_plugin {
|
||||
capabilities = capabilities.allow_plugin(plugin.clone());
|
||||
}
|
||||
for provider in &self.allow_provider {
|
||||
validate_provider_grant(provider)?;
|
||||
capabilities = capabilities.allow_resource_provider(provider.clone());
|
||||
}
|
||||
Ok(capabilities)
|
||||
}
|
||||
SandboxMode::Strict => {
|
||||
@@ -150,3 +165,28 @@ impl SandboxSettings {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Reject non-exact `--allow-provider` grants.
|
||||
///
|
||||
/// Provider grants are exact stable names (RFC-EXT-0001 §5.2): there is no `*`
|
||||
/// wildcard, no prefix matching and no implicit discovery, so a grant either
|
||||
/// names a selected manifest or the run fails closed.
|
||||
fn validate_provider_grant(name: &str) -> Result<(), String> {
|
||||
if name.is_empty() {
|
||||
return Err("--allow-provider requires a non-empty provider name".to_string());
|
||||
}
|
||||
if name
|
||||
.chars()
|
||||
.any(|character| matches!(character, '*' | '?' | '[' | ']'))
|
||||
{
|
||||
return Err(format!(
|
||||
"--allow-provider `{name}`: wildcards are not supported; grants are exact provider names"
|
||||
));
|
||||
}
|
||||
if name.chars().any(char::is_control) {
|
||||
return Err(format!(
|
||||
"--allow-provider `{name}`: provider names must not contain control characters"
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,414 @@
|
||||
//! Static extension descriptors for the CLI composition root
|
||||
//! (RFC-EXT-0001 §4.4/§6, ADR-0054/0056).
|
||||
//!
|
||||
//! This module only *reads* manifests. It never `dlopen`s a library, never
|
||||
//! spawns a worker, never opens a network connection and never executes a
|
||||
//! side-effect script. The descriptors produced here feed the single
|
||||
//! [`marqdo_ir::callable::CallableRegistry`] and binding pass; `run` later
|
||||
//! verifies the real artifacts against these recorded identities.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
|
||||
use marqdo_ir::callable::{CallableDescriptor, CallableId, CallableOrigin, CallableSignature};
|
||||
use marqdo_ir::SignatureKind;
|
||||
use marqdo_plugin_abi::{HostTarget, PluginKind, PluginManifest};
|
||||
use marqdo_span::Span;
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
/// A structured prepare-stage failure.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PrepareError {
|
||||
/// Stable diagnostic code (`ext.*` unless reusing an existing event code).
|
||||
pub code: String,
|
||||
/// Human-readable message; never contains secrets or binary content.
|
||||
pub message: String,
|
||||
/// CLI exit code (2 for manifest/option errors, 1 for load/verify errors).
|
||||
pub exit_code: i32,
|
||||
}
|
||||
|
||||
impl PrepareError {
|
||||
/// Construct a manifest/option error (exit code 2).
|
||||
pub fn option(code: impl Into<String>, message: impl Into<String>) -> Self {
|
||||
Self {
|
||||
code: code.into(),
|
||||
message: message.into(),
|
||||
exit_code: 2,
|
||||
}
|
||||
}
|
||||
|
||||
/// Construct a loading/verification error (exit code 1).
|
||||
pub fn load(code: impl Into<String>, message: impl Into<String>) -> Self {
|
||||
Self {
|
||||
code: code.into(),
|
||||
message: message.into(),
|
||||
exit_code: 1,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One statically declared export.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct StaticPluginExport {
|
||||
/// Complete dotted export name.
|
||||
pub name: String,
|
||||
/// Raw manifest signature string, compared exactly at run time.
|
||||
pub signature: String,
|
||||
/// Parsed static descriptor used by the binding pass.
|
||||
pub descriptor: CallableDescriptor,
|
||||
}
|
||||
|
||||
/// One statically read plugin package.
|
||||
///
|
||||
/// The identity fields are recorded for `inspect --extensions`, evidence and
|
||||
/// reconciliation; they are not consulted for name resolution (the binding
|
||||
/// table is).
|
||||
#[allow(dead_code)]
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct StaticPlugin {
|
||||
/// Canonical manifest path.
|
||||
pub manifest_path: PathBuf,
|
||||
/// Package directory (artifact root).
|
||||
pub base_dir: PathBuf,
|
||||
/// Declared package name.
|
||||
pub package: String,
|
||||
/// Declared package version.
|
||||
pub version: String,
|
||||
/// Deployment kind (`native`, `worker`, `wasm`).
|
||||
pub kind: PluginKind,
|
||||
/// `sha256:<hex>` of the manifest file bytes.
|
||||
pub manifest_sha256: String,
|
||||
/// Target triple selected for this host.
|
||||
pub artifact_target: String,
|
||||
/// Declared (not yet verified) artifact sha256, lowercase hex.
|
||||
pub artifact_sha256: String,
|
||||
/// Bundle-relative artifact path.
|
||||
pub artifact_relative: String,
|
||||
/// Exports in manifest order.
|
||||
pub exports: Vec<StaticPluginExport>,
|
||||
/// Declared capability requests (metadata only; never grants).
|
||||
pub requires: Vec<String>,
|
||||
}
|
||||
|
||||
impl StaticPlugin {
|
||||
/// Absolute artifact path inside the package bundle.
|
||||
pub fn artifact_path(&self) -> PathBuf {
|
||||
self.base_dir.join(&self.artifact_relative)
|
||||
}
|
||||
}
|
||||
|
||||
/// Static plugin selection read from explicit CLI `--plugin` flags.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct StaticPluginSet {
|
||||
/// Packages in deterministic manifest order.
|
||||
pub plugins: Vec<StaticPlugin>,
|
||||
}
|
||||
|
||||
impl StaticPluginSet {
|
||||
/// All export descriptors in deterministic order.
|
||||
pub fn descriptors(&self) -> Vec<CallableDescriptor> {
|
||||
let mut descriptors = Vec::new();
|
||||
for plugin in &self.plugins {
|
||||
for export in &plugin.exports {
|
||||
descriptors.push(export.descriptor.clone());
|
||||
}
|
||||
}
|
||||
descriptors
|
||||
}
|
||||
}
|
||||
|
||||
/// Read and validate all explicitly selected `--plugin` manifests.
|
||||
///
|
||||
/// No binary is opened or executed. Duplicate package identities, duplicate
|
||||
/// full export names, reserved names and malformed signatures are rejected
|
||||
/// here, before any binding is produced.
|
||||
pub fn read_plugin_descriptors(paths: &[PathBuf]) -> Result<StaticPluginSet, PrepareError> {
|
||||
let mut set = StaticPluginSet::default();
|
||||
let mut export_names: BTreeSet<String> = BTreeSet::new();
|
||||
let mut packages: BTreeSet<(String, String)> = BTreeSet::new();
|
||||
for path in paths {
|
||||
let manifest_path = if path.is_dir() {
|
||||
path.join("plugin.yaml")
|
||||
} else {
|
||||
path.clone()
|
||||
};
|
||||
let manifest_path = manifest_path.canonicalize().map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.manifest_unreadable",
|
||||
format!(
|
||||
"cannot open plugin manifest `{}`: {error}",
|
||||
manifest_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let base_dir = manifest_path
|
||||
.parent()
|
||||
.map(Path::to_path_buf)
|
||||
.unwrap_or_else(|| PathBuf::from("."));
|
||||
let base_dir = base_dir.canonicalize().unwrap_or(base_dir);
|
||||
let manifest_bytes = std::fs::read(&manifest_path).map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.manifest_unreadable",
|
||||
format!(
|
||||
"cannot read plugin manifest `{}`: {error}",
|
||||
manifest_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let manifest = PluginManifest::load(&manifest_path)
|
||||
.map_err(|error| PrepareError::option(error.code.clone(), error.message.clone()))?;
|
||||
manifest
|
||||
.validate()
|
||||
.map_err(|error| PrepareError::option(error.code.clone(), error.message.clone()))?;
|
||||
let host = HostTarget::current();
|
||||
let artifact = manifest.resolve_artifact(&host).map_err(|error| {
|
||||
PrepareError::option("ext.no_matching_artifact", error.message.clone())
|
||||
})?;
|
||||
let artifact_relative = artifact.file.clone();
|
||||
ensure_bundle_relative(&artifact_relative).map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"plugin `{}` artifact path `{artifact_relative}` is invalid: {error}",
|
||||
manifest.plugin.name
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let manifest_sha256 = format!("sha256:{}", hex(&Sha256::digest(&manifest_bytes)));
|
||||
let identity = (manifest.plugin.name.clone(), manifest_sha256.clone());
|
||||
if !packages.insert(identity.clone()) {
|
||||
// The exact same package declaration twice is idempotent.
|
||||
continue;
|
||||
}
|
||||
if set
|
||||
.plugins
|
||||
.iter()
|
||||
.any(|plugin| plugin.package == identity.0)
|
||||
{
|
||||
return Err(PrepareError::option(
|
||||
"ext.duplicate_callable",
|
||||
format!(
|
||||
"plugin package `{}` is selected with two different manifests; \
|
||||
explicit aliases are not implemented, so this is rejected",
|
||||
identity.0
|
||||
),
|
||||
));
|
||||
}
|
||||
let mut exports = Vec::new();
|
||||
for export in &manifest.plugin.exports {
|
||||
if !export_names.insert(export.name.clone()) {
|
||||
return Err(PrepareError::option(
|
||||
"ext.duplicate_callable",
|
||||
format!(
|
||||
"plugin export `{}` is declared by more than one selected package",
|
||||
export.name
|
||||
),
|
||||
));
|
||||
}
|
||||
let signature = plugin_signature(&export.signature).map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.plugin_invalid_signature",
|
||||
format!(
|
||||
"plugin export `{}` has an invalid signature: {error}",
|
||||
export.name
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let descriptor = CallableDescriptor {
|
||||
id: CallableId(format!("plugin:{}:{}", manifest.plugin.name, export.name)),
|
||||
public_name: export.name.clone(),
|
||||
owner_package: manifest.plugin.name.clone(),
|
||||
signature,
|
||||
effects_requested: manifest.plugin.requires.clone(),
|
||||
origin: CallableOrigin::AbiPlugin {
|
||||
package: manifest.plugin.name.clone(),
|
||||
export: export.name.clone(),
|
||||
},
|
||||
implementation_digest: Some(artifact.sha256.clone()),
|
||||
source_span: None,
|
||||
};
|
||||
exports.push(StaticPluginExport {
|
||||
name: export.name.clone(),
|
||||
signature: export.signature.clone(),
|
||||
descriptor,
|
||||
});
|
||||
}
|
||||
set.plugins.push(StaticPlugin {
|
||||
manifest_path,
|
||||
base_dir,
|
||||
package: manifest.plugin.name.clone(),
|
||||
version: manifest.plugin.version.clone(),
|
||||
kind: manifest.plugin.kind,
|
||||
manifest_sha256,
|
||||
artifact_target: host.triple().to_string(),
|
||||
artifact_sha256: artifact.sha256.clone(),
|
||||
artifact_relative,
|
||||
exports,
|
||||
requires: manifest.plugin.requires.clone(),
|
||||
});
|
||||
}
|
||||
set.plugins.sort_by(|left, right| {
|
||||
(&left.package, &left.manifest_sha256).cmp(&(&right.package, &right.manifest_sha256))
|
||||
});
|
||||
Ok(set)
|
||||
}
|
||||
|
||||
/// Convert a plugin manifest signature (`($a: Int, $b: Int) -> Int`) to the
|
||||
/// language parameter list.
|
||||
///
|
||||
/// ABI v1 transports named arguments as a map, so positional arguments are a
|
||||
/// static-validated convenience: the binding pass normalises them to the
|
||||
/// manifest order. Default values are rejected until the ABI wire semantics for
|
||||
/// them are versioned (RFC-EXT-0001 §4.5.3); silently injecting defaults at run
|
||||
/// time is forbidden.
|
||||
pub fn plugin_signature(raw: &str) -> Result<CallableSignature, String> {
|
||||
let before_return = raw.split("->").next().unwrap_or(raw).trim();
|
||||
let open = before_return
|
||||
.find('(')
|
||||
.ok_or_else(|| "missing `(`".to_string())?;
|
||||
let close = before_return
|
||||
.rfind(')')
|
||||
.ok_or_else(|| "missing `)`".to_string())?;
|
||||
if close < open {
|
||||
return Err("malformed parameter list".to_string());
|
||||
}
|
||||
let params = before_return[open + 1..close].replace('$', "");
|
||||
let parsed = marqdo_frontend::signature::parse(&format!("({params})"), Span::default());
|
||||
if parsed.has_errors() {
|
||||
return Err(parsed
|
||||
.diagnostics
|
||||
.iter()
|
||||
.map(|diagnostic| diagnostic.message.clone())
|
||||
.collect::<Vec<_>>()
|
||||
.join("; "));
|
||||
}
|
||||
if parsed.params.iter().any(|param| param.default.is_some()) {
|
||||
return Err(
|
||||
"ABI v1 plugin signatures must not declare default values; make parameters required"
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
Ok(CallableSignature {
|
||||
params: parsed.params,
|
||||
kind: SignatureKind::Explicit,
|
||||
})
|
||||
}
|
||||
|
||||
/// Reject absolute paths, `..` components and empty paths.
|
||||
pub fn ensure_bundle_relative(path: &str) -> Result<(), String> {
|
||||
let mut normalised = PathBuf::new();
|
||||
for component in Path::new(path).components() {
|
||||
match component {
|
||||
Component::Normal(part) => normalised.push(part),
|
||||
Component::CurDir => {}
|
||||
Component::ParentDir | Component::RootDir | Component::Prefix(_) => {
|
||||
return Err("must be a relative path inside the package".to_string())
|
||||
}
|
||||
}
|
||||
}
|
||||
if normalised.as_os_str().is_empty() {
|
||||
return Err("must not be empty".to_string());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Verify a bundle file is a regular file below `bundle_root`, travelling
|
||||
/// through symlinks only if the final canonical target stays inside the bundle.
|
||||
pub fn verify_bundle_regular_file(
|
||||
bundle_root: &Path,
|
||||
relative: &str,
|
||||
) -> Result<PathBuf, PrepareError> {
|
||||
ensure_bundle_relative(relative).map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!("artifact path `{relative}` is invalid: {error}"),
|
||||
)
|
||||
})?;
|
||||
let root = bundle_root.canonicalize().map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"cannot resolve package root `{}`: {error}",
|
||||
bundle_root.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let candidate = root.join(relative);
|
||||
let canonical = candidate.canonicalize().map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!("cannot open artifact `{}`: {error}", candidate.display()),
|
||||
)
|
||||
})?;
|
||||
if !canonical.starts_with(&root) {
|
||||
return Err(PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"artifact `{relative}` resolves outside its package bundle (`{}`)",
|
||||
canonical.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
let metadata = std::fs::metadata(&canonical).map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!("cannot stat artifact `{}`: {error}", canonical.display()),
|
||||
)
|
||||
})?;
|
||||
if !metadata.is_file() {
|
||||
return Err(PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!("artifact `{}` is not a regular file", canonical.display()),
|
||||
));
|
||||
}
|
||||
Ok(canonical)
|
||||
}
|
||||
|
||||
/// Verify a file's lowercase sha256 against a declared value.
|
||||
pub fn verify_sha256(path: &Path, declared: &str) -> Result<(), PrepareError> {
|
||||
let bytes = std::fs::read(path).map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!("cannot read artifact `{}`: {error}", path.display()),
|
||||
)
|
||||
})?;
|
||||
let actual = hex(&Sha256::digest(&bytes));
|
||||
if actual != declared.to_ascii_lowercase() {
|
||||
return Err(PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"artifact `{}` sha256 mismatch: declared {declared}, actual {actual}",
|
||||
path.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn hex(bytes: &[u8]) -> String {
|
||||
let mut out = String::with_capacity(bytes.len() * 2);
|
||||
for byte in bytes {
|
||||
out.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn plugin_signature_rejects_defaults_and_parses_positional() {
|
||||
assert!(plugin_signature("($a: Int, $b: Int) -> Int").is_ok());
|
||||
let error = plugin_signature("($a: Int = 1) -> Int").unwrap_err();
|
||||
assert!(error.contains("default"), "{error}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bundle_relative_paths_reject_traversal() {
|
||||
assert!(ensure_bundle_relative("lib/demo.so").is_ok());
|
||||
assert!(ensure_bundle_relative("../demo.so").is_err());
|
||||
assert!(ensure_bundle_relative("/etc/passwd").is_err());
|
||||
assert!(ensure_bundle_relative("").is_err());
|
||||
}
|
||||
}
|
||||
+171
-72
@@ -14,7 +14,8 @@ use std::collections::BTreeMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use marqdo_expression::Expr;
|
||||
use marqdo_frontend::{analyze_source, AnalysisResult};
|
||||
use marqdo_frontend::AnalysisResult;
|
||||
use marqdo_ir::callable::OfficialAlias;
|
||||
use marqdo_ir::{ModuleId, PlaceTarget, Program, ResolvedImport, Stmt};
|
||||
use marqdo_package::{
|
||||
discover_root, LockFile, Manifest, PackageResolver, ResolvedPackage, LOCK_FILE, MANIFEST_FILE,
|
||||
@@ -157,6 +158,11 @@ pub struct LoadedProject {
|
||||
pub diagnostics: Vec<FileDiagnostic>,
|
||||
/// Verified bundled stdlib identity, when one was available.
|
||||
pub stdlib: Option<StdlibInfo>,
|
||||
/// Official stdlib public aliases referenced by this program (ADR-0058).
|
||||
///
|
||||
/// These are ordinary `Marqdo` bindings; the runtime has no print special
|
||||
/// case and no `Program.print_override`.
|
||||
pub official_aliases: Vec<marqdo_ir::callable::OfficialAlias>,
|
||||
}
|
||||
|
||||
impl LoadedProject {
|
||||
@@ -280,32 +286,11 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
|
||||
let entry_id = loader.load_module(&entry_module.package, &entry_module.relative_path)?;
|
||||
loader.program.main_module = Some(entry_id.clone());
|
||||
|
||||
// Compat-B (ADR-0030): only programs that actually call 打印/print need the
|
||||
// stdlib print module; injecting it for every program would change the
|
||||
// module graph, analysis output and startup cost for no benefit.
|
||||
if program_uses_print(&loader.program) {
|
||||
if let Some(stdlib) = loader.stdlib.clone() {
|
||||
let package = stdlib.package();
|
||||
let io = loader.load_module(&package, "io.mq.md")?;
|
||||
let Some(module) = loader.program.module(&io) else {
|
||||
return Err("stdlib print module was not loaded".to_string());
|
||||
};
|
||||
let Some(function) = module.function("写值") else {
|
||||
return Err("stdlib print module has no `写值` function".to_string());
|
||||
};
|
||||
loader.program.print_override = Some(function.id.clone());
|
||||
}
|
||||
}
|
||||
for (module_id, diagnostic) in marqdo_frontend::calls::validate_program(&loader.program) {
|
||||
if let Some(file) = loader.files.get(&module_id) {
|
||||
loader.diagnostics.push(FileDiagnostic {
|
||||
module_id,
|
||||
path: file.path.clone(),
|
||||
source: file.source.clone(),
|
||||
diagnostic,
|
||||
});
|
||||
}
|
||||
}
|
||||
// Official stdlib public aliases (ADR-0058): the verified metadata names
|
||||
// the target module/function; only aliases the program actually references
|
||||
// load their module, and the runtime binds them as ordinary Marqdo calls.
|
||||
let official_aliases = loader.load_official_aliases()?;
|
||||
|
||||
let stdlib_info = loader.stdlib.as_ref().map(|bundle| StdlibInfo {
|
||||
version: bundle.version().to_string(),
|
||||
digest: bundle.digest().to_string(),
|
||||
@@ -328,6 +313,7 @@ pub fn load(entry: &Path) -> Result<LoadedProject, String> {
|
||||
packages: loader.packages,
|
||||
diagnostics: loader.diagnostics,
|
||||
stdlib: stdlib_info,
|
||||
official_aliases,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -378,7 +364,7 @@ impl Loader {
|
||||
.or_insert_with(|| package_root.clone());
|
||||
let source = std::fs::read_to_string(&path)
|
||||
.map_err(|error| format!("cannot read `{}`: {error}", path.display()))?;
|
||||
let analysis = analyze_source(&module_id.to_string(), &source);
|
||||
let analysis = marqdo_frontend::analyze_source_for_program(&module_id.to_string(), &source);
|
||||
let imports: Vec<(String, String, marqdo_span::Span)> = analysis
|
||||
.document
|
||||
.frontmatter()
|
||||
@@ -516,57 +502,170 @@ impl Loader {
|
||||
}
|
||||
Ok(module_id)
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the loaded program calls the legacy `打印`/`print` builtin.
|
||||
fn program_uses_print(program: &Program) -> bool {
|
||||
program.modules.values().any(|module| {
|
||||
module
|
||||
.functions
|
||||
.iter()
|
||||
.any(|function| stmts_use_print(&function.body))
|
||||
})
|
||||
}
|
||||
|
||||
fn stmts_use_print(stmts: &[Stmt]) -> bool {
|
||||
stmts.iter().any(|stmt| match stmt {
|
||||
Stmt::Assign { target, value, .. } => {
|
||||
let target_uses = match target {
|
||||
PlaceTarget::Local { .. } => false,
|
||||
PlaceTarget::Index { base, index, .. } => {
|
||||
expr_uses_print(base) || expr_uses_print(index)
|
||||
}
|
||||
/// Resolve the official stdlib public aliases referenced by the program.
|
||||
///
|
||||
/// Only the verified `stdlib.json` metadata can declare an official alias;
|
||||
/// a third-party package cannot claim `打印`/`print` or `std.*`. Modules are
|
||||
/// loaded only when the program references one of the alias names, so an
|
||||
/// ordinary program does not grow its module graph or startup cost.
|
||||
fn load_official_aliases(&mut self) -> Result<Vec<OfficialAlias>, String> {
|
||||
let Some(stdlib) = self.stdlib.clone() else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
if stdlib.public_aliases().is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let referenced = referenced_identifiers(&self.program);
|
||||
let mut aliases = Vec::new();
|
||||
for alias in stdlib.public_aliases() {
|
||||
if !alias
|
||||
.public_names
|
||||
.iter()
|
||||
.any(|name| referenced.contains(name))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let resolved = stdlib.resolve(&alias.target_module)?;
|
||||
let module_id = self.load_module(&resolved.package, &resolved.relative_path)?;
|
||||
let Some(module) = self.program.module(&module_id) else {
|
||||
return Err(format!(
|
||||
"stdlib public alias target module `{}` was not loaded",
|
||||
alias.target_module
|
||||
));
|
||||
};
|
||||
target_uses || expr_uses_print(value)
|
||||
let Some(function) = module.function(&alias.target_function) else {
|
||||
return Err(format!(
|
||||
"stdlib public alias `{}` target `{}::{}` does not exist",
|
||||
alias.public_names.join("/"),
|
||||
alias.target_module,
|
||||
alias.target_function
|
||||
));
|
||||
};
|
||||
if let Some(expected) = &alias.expected_signature {
|
||||
let parsed =
|
||||
marqdo_frontend::signature::parse(expected, marqdo_span::Span::default());
|
||||
if parsed.has_errors() {
|
||||
return Err(format!(
|
||||
"stdlib public alias `{}` has an invalid expected_signature `{expected}`",
|
||||
alias.public_names.join("/")
|
||||
));
|
||||
}
|
||||
let expected_names: Vec<&str> = parsed
|
||||
.params
|
||||
.iter()
|
||||
.map(|param| param.name.as_str())
|
||||
.collect();
|
||||
let actual_names: Vec<&str> = function
|
||||
.params
|
||||
.iter()
|
||||
.map(|param| param.name.as_str())
|
||||
.collect();
|
||||
if expected_names != actual_names
|
||||
|| function.signature != marqdo_ir::SignatureKind::Explicit
|
||||
{
|
||||
return Err(format!(
|
||||
"stdlib public alias `{}` expected signature `{expected}` does not match \
|
||||
target `{}::{}` (params {:?}, kind {:?})",
|
||||
alias.public_names.join("/"),
|
||||
alias.target_module,
|
||||
alias.target_function,
|
||||
actual_names,
|
||||
function.signature
|
||||
));
|
||||
}
|
||||
}
|
||||
aliases.push(OfficialAlias {
|
||||
public_names: alias.public_names.clone(),
|
||||
function: function.id.clone(),
|
||||
signature: marqdo_ir::callable::CallableSignature {
|
||||
params: function.params.clone(),
|
||||
kind: function.signature,
|
||||
},
|
||||
owner_package: module_id.instance.clone(),
|
||||
effects_requested: alias.effects.clone(),
|
||||
});
|
||||
}
|
||||
Stmt::Eval { expr, .. } | Stmt::Return { value: expr, .. } => expr_uses_print(expr),
|
||||
Stmt::Branch { arms, .. } => arms
|
||||
.iter()
|
||||
.any(|arm| expr_uses_print(&arm.condition) || stmts_use_print(&arm.body)),
|
||||
Stmt::Loop {
|
||||
condition, body, ..
|
||||
} => expr_uses_print(condition) || stmts_use_print(body),
|
||||
})
|
||||
Ok(aliases)
|
||||
}
|
||||
}
|
||||
|
||||
/// Collect every identifier referenced by a program.
|
||||
///
|
||||
/// This is a conservative "is this alias used at all?" scan; it only decides
|
||||
/// whether an official alias target module should be loaded. It does not decide
|
||||
/// call targets (that is the single binding pass).
|
||||
fn referenced_identifiers(program: &Program) -> std::collections::BTreeSet<String> {
|
||||
let mut names = std::collections::BTreeSet::new();
|
||||
for module in program.modules.values() {
|
||||
for function in &module.functions {
|
||||
for param in &function.params {
|
||||
if let Some(default) = ¶m.default {
|
||||
collect_identifiers_expr(default, &mut names);
|
||||
}
|
||||
}
|
||||
collect_identifiers_stmts(&function.body, &mut names);
|
||||
}
|
||||
}
|
||||
names
|
||||
}
|
||||
|
||||
fn collect_identifiers_stmts(stmts: &[Stmt], names: &mut std::collections::BTreeSet<String>) {
|
||||
for stmt in stmts {
|
||||
match stmt {
|
||||
Stmt::Assign { target, value, .. } => {
|
||||
if let PlaceTarget::Index { base, index, .. } = target {
|
||||
collect_identifiers_expr(base, names);
|
||||
collect_identifiers_expr(index, names);
|
||||
}
|
||||
collect_identifiers_expr(value, names);
|
||||
}
|
||||
Stmt::Eval { expr, .. } | Stmt::Return { value: expr, .. } => {
|
||||
collect_identifiers_expr(expr, names)
|
||||
}
|
||||
Stmt::Branch { arms, .. } => {
|
||||
for arm in arms {
|
||||
collect_identifiers_expr(&arm.condition, names);
|
||||
collect_identifiers_stmts(&arm.body, names);
|
||||
}
|
||||
}
|
||||
Stmt::Loop {
|
||||
condition, body, ..
|
||||
} => {
|
||||
collect_identifiers_expr(condition, names);
|
||||
collect_identifiers_stmts(body, names);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn expr_uses_print(expr: &Expr) -> bool {
|
||||
fn collect_identifiers_expr(expr: &Expr, names: &mut std::collections::BTreeSet<String>) {
|
||||
match expr {
|
||||
Expr::Ident(ident) => {
|
||||
names.insert(ident.name.clone());
|
||||
}
|
||||
Expr::List(items) => items
|
||||
.iter()
|
||||
.for_each(|item| collect_identifiers_expr(item, names)),
|
||||
Expr::Record(entries) => entries
|
||||
.iter()
|
||||
.for_each(|entry| collect_identifiers_expr(&entry.value, names)),
|
||||
Expr::Unary { operand, .. } => collect_identifiers_expr(operand, names),
|
||||
Expr::Binary { left, right, .. } => {
|
||||
collect_identifiers_expr(left, names);
|
||||
collect_identifiers_expr(right, names);
|
||||
}
|
||||
Expr::Member { base, .. } => collect_identifiers_expr(base, names),
|
||||
Expr::Index { base, index, .. } => {
|
||||
collect_identifiers_expr(base, names);
|
||||
collect_identifiers_expr(index, names);
|
||||
}
|
||||
Expr::Call { callee, args, .. } => {
|
||||
matches!(callee.as_ref(), Expr::Ident(ident) if matches!(ident.name.as_str(), "打印" | "print"))
|
||||
|| expr_uses_print(callee)
|
||||
|| args.iter().any(|arg| expr_uses_print(&arg.value))
|
||||
collect_identifiers_expr(callee, names);
|
||||
for arg in args {
|
||||
collect_identifiers_expr(&arg.value, names);
|
||||
}
|
||||
}
|
||||
Expr::List(items) => items.iter().any(expr_uses_print),
|
||||
Expr::Record(entries) => entries.iter().any(|entry| expr_uses_print(&entry.value)),
|
||||
Expr::Unary { operand, .. } => expr_uses_print(operand),
|
||||
Expr::Binary { left, right, .. } => expr_uses_print(left) || expr_uses_print(right),
|
||||
Expr::Member { base, .. } => expr_uses_print(base),
|
||||
Expr::Index { base, index, .. } => expr_uses_print(base) || expr_uses_print(index),
|
||||
Expr::Int(_)
|
||||
| Expr::Float(_)
|
||||
| Expr::Text(_)
|
||||
| Expr::Bool(_)
|
||||
| Expr::None
|
||||
| Expr::Ident(_) => false,
|
||||
Expr::Int(_) | Expr::Float(_) | Expr::Text(_) | Expr::Bool(_) | Expr::None => {}
|
||||
}
|
||||
}
|
||||
@@ -7,8 +7,11 @@
|
||||
mod analysis;
|
||||
mod args;
|
||||
mod commands;
|
||||
mod extension_descriptors;
|
||||
mod loader;
|
||||
mod output;
|
||||
mod project_prepare;
|
||||
mod provider_manifest;
|
||||
mod stdlib;
|
||||
mod tool;
|
||||
mod trace;
|
||||
@@ -128,6 +131,16 @@ struct SandboxArgs {
|
||||
/// Explicitly trusted plugin manifest/directory to load (repeatable).
|
||||
#[arg(long = "plugin", value_name = "PATH")]
|
||||
plugin: Vec<PathBuf>,
|
||||
/// Explicitly selected resource provider v2 manifest (repeatable).
|
||||
///
|
||||
/// Selecting a provider never authorizes it; `--allow-provider NAME` is
|
||||
/// required as well. `check`/`inspect` do not load providers.
|
||||
#[arg(long = "provider", value_name = "PATH")]
|
||||
provider: Vec<PathBuf>,
|
||||
/// Grant one selected resource provider by its exact manifest name
|
||||
/// (repeatable; wildcards are not supported).
|
||||
#[arg(long = "allow-provider", value_name = "NAME")]
|
||||
allow_provider: Vec<String>,
|
||||
/// Write a versioned Trace v2 file for this run.
|
||||
#[arg(long = "trace-file", value_name = "PATH")]
|
||||
trace_file: Option<PathBuf>,
|
||||
@@ -164,6 +177,8 @@ impl SandboxArgs {
|
||||
max_region_memory: self.max_region_memory,
|
||||
timeout: self.timeout,
|
||||
plugins: self.plugin.clone(),
|
||||
providers: self.provider.clone(),
|
||||
allow_provider: self.allow_provider.clone(),
|
||||
trust_native_plugin: self.trust_native_plugin,
|
||||
trace_file: self.trace_file.clone(),
|
||||
trace_max_events: self.trace_max_events,
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
//! The single `prepare` composition-root step (RFC-EXT-0001 §4.4, §18.2).
|
||||
//!
|
||||
//! Order of operations (T1–T5 of the RFC):
|
||||
//!
|
||||
//! 1. load the Markdown/package graph (read only, no execution);
|
||||
//! 2. read static extension descriptors (never `dlopen`/spawn);
|
||||
//! 3. build the unique [`CallableRegistry`] and reject static conflicts;
|
||||
//! 4. run the single program-wide call binding pass;
|
||||
//! 5. produce a [`PreparedProgram`] consumed by `check`, `run`, `call`,
|
||||
//! `inspect` and Analysis.
|
||||
//!
|
||||
//! `run` may only assemble the Host and start workers after this returns.
|
||||
|
||||
use std::path::PathBuf;
|
||||
|
||||
use marqdo_ir::callable::CallableRegistry;
|
||||
|
||||
use crate::extension_descriptors::{read_plugin_descriptors, PrepareError, StaticPluginSet};
|
||||
use crate::loader::{self, FileDiagnostic, LoadedProject};
|
||||
|
||||
/// Options that influence static preparation.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct PrepareOptions {
|
||||
/// Explicit `--plugin` manifests (repeatable, no implicit discovery).
|
||||
pub plugins: Vec<PathBuf>,
|
||||
}
|
||||
|
||||
/// A statically prepared program: everything `check`/`run`/`inspect` share.
|
||||
pub struct PreparedProgram {
|
||||
/// Loaded project graph with authoritative diagnostics.
|
||||
pub loaded: LoadedProject,
|
||||
/// Static plugin descriptors (identity only; no binary loaded).
|
||||
pub plugins: StaticPluginSet,
|
||||
/// Unique callable registry used by the binding pass.
|
||||
///
|
||||
/// Kept on the prepared value so later consumers (`inspect --extensions`,
|
||||
/// evidence) can report the same static table without rebuilding it.
|
||||
#[allow(dead_code)]
|
||||
pub registry: CallableRegistry,
|
||||
}
|
||||
|
||||
/// Prepare one document without executing any project or extension code.
|
||||
pub fn prepare(
|
||||
file: Option<&std::path::Path>,
|
||||
options: &PrepareOptions,
|
||||
) -> Result<PreparedProgram, PrepareError> {
|
||||
// T1: static module/package loading. No call diagnostics are produced by
|
||||
// the loader; the single binding pass below owns them.
|
||||
let mut loaded = loader::load(file.unwrap_or_else(|| std::path::Path::new("index.mq.md")))
|
||||
.map_err(|message| default_load_error(&message))?;
|
||||
|
||||
// T2: extension descriptors are read from manifests only.
|
||||
let plugins = read_plugin_descriptors(&options.plugins)?;
|
||||
|
||||
// T3: unique static table. Duplicate exports/builtin shadowing are rejected
|
||||
// before any call site can be bound.
|
||||
for descriptor in plugins.descriptors() {
|
||||
if loaded
|
||||
.program
|
||||
.external_callable(&descriptor.public_name)
|
||||
.is_some()
|
||||
{
|
||||
return Err(PrepareError::option(
|
||||
"ext.duplicate_callable",
|
||||
format!(
|
||||
"plugin export `{}` is declared more than once",
|
||||
descriptor.public_name
|
||||
),
|
||||
));
|
||||
}
|
||||
if marqdo_ir::builtins::find(&descriptor.public_name).is_some() {
|
||||
return Err(PrepareError::option(
|
||||
"ext.duplicate_callable",
|
||||
format!(
|
||||
"plugin export `{}` conflicts with a builtin; plugins may not shadow builtins",
|
||||
descriptor.public_name
|
||||
),
|
||||
));
|
||||
}
|
||||
loaded.program.external_callables.push(descriptor);
|
||||
}
|
||||
let mut registry = CallableRegistry::try_from_program(&loaded.program).map_err(|conflict| {
|
||||
PrepareError::option(
|
||||
"ext.duplicate_callable",
|
||||
format!("static callable table conflict: {conflict}"),
|
||||
)
|
||||
})?;
|
||||
|
||||
// Official stdlib public aliases (ADR-0058) are ordinary Marqdo bindings.
|
||||
for alias in &loaded.official_aliases {
|
||||
registry
|
||||
.insert_official_alias(alias.clone())
|
||||
.map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.duplicate_callable",
|
||||
format!("stdlib public alias: {error}"),
|
||||
)
|
||||
})?;
|
||||
}
|
||||
|
||||
// T4: exactly one call binding pass for the whole program.
|
||||
let bound = marqdo_frontend::calls::bind_program_calls(&loaded.program, ®istry);
|
||||
for (module_id, diagnostic) in bound.diagnostics {
|
||||
let Some(file) = loaded.files.get(&module_id) else {
|
||||
continue;
|
||||
};
|
||||
let (path, source) = (file.path.clone(), file.source.clone());
|
||||
let duplicate = loaded.diagnostics.iter().any(|existing| {
|
||||
existing.module_id == module_id
|
||||
&& existing.diagnostic.code == diagnostic.code
|
||||
&& existing.diagnostic.span == diagnostic.span
|
||||
&& existing.diagnostic.message == diagnostic.message
|
||||
});
|
||||
if !duplicate {
|
||||
loaded.diagnostics.push(FileDiagnostic {
|
||||
module_id,
|
||||
path,
|
||||
source,
|
||||
diagnostic,
|
||||
});
|
||||
}
|
||||
}
|
||||
loaded.program.set_call_bindings(bound.bindings);
|
||||
|
||||
Ok(PreparedProgram {
|
||||
loaded,
|
||||
plugins,
|
||||
registry,
|
||||
})
|
||||
}
|
||||
|
||||
fn default_load_error(message: &str) -> PrepareError {
|
||||
if message.contains("marqdo.lock") || message.contains("lockfile") {
|
||||
PrepareError::load("project.lock", message)
|
||||
} else {
|
||||
PrepareError::option("project.load", message)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,952 @@
|
||||
//! Strict third-party Resource Provider v2 manifests for the CLI composition
|
||||
//! root (RFC-EXT-0001 §5.3, ADR-0055).
|
||||
//!
|
||||
//! This module *reads* `provider.json` schema version 2 and verifies the
|
||||
//! selected artifact. It never `dlopen`s a library, never spawns a helper and
|
||||
//! never authorizes a provider: parsing a manifest is not a grant. The v1
|
||||
//! `marqdo.provider` manifest stays a host-embedding path
|
||||
//! (`StdHost::register_provider_package`, ADR-0037) and is rejected here with
|
||||
//! `ext.unsupported_provider_schema`; there is no silent migration.
|
||||
//!
|
||||
//! The CLI only accepts the `c-abi-1` + `worker` transport combination and
|
||||
//! resolves the helper as a sibling of the running `marqdo` executable; PATH
|
||||
//! lookup is never used. Integrity hashes are content-integrity checks, not
|
||||
//! publisher authentication.
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use marqdo_host::StdHost;
|
||||
use marqdo_plugin_abi::{HostTarget, TargetTriple};
|
||||
use serde::Deserialize;
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
use crate::extension_descriptors::{
|
||||
ensure_bundle_relative, verify_bundle_regular_file, verify_sha256, PrepareError,
|
||||
};
|
||||
|
||||
/// Provider manifest file name used by the CLI (`--provider` accepts this file).
|
||||
pub const PROVIDER_MANIFEST_FILE: &str = "provider.json";
|
||||
|
||||
/// Provider manifest schema name.
|
||||
pub const PROVIDER_SCHEMA: &str = "marqdo.provider";
|
||||
/// Provider manifest schema version accepted by the CLI.
|
||||
pub const PROVIDER_SCHEMA_VERSION: u64 = 2;
|
||||
/// Provider ABI accepted by the CLI (ADR-0036/0038).
|
||||
pub const PROVIDER_ABI: &str = "c-abi-1";
|
||||
/// Provider transport accepted by the CLI (ADR-0035/0038).
|
||||
pub const PROVIDER_TRANSPORT: &str = "worker";
|
||||
/// Helper executable shipped next to the `marqdo` binary (ADR-0038).
|
||||
pub const PROVIDER_WORKER_BINARY: &str = "marqdo-provider-worker";
|
||||
|
||||
/// Upper bounds for a single provider manifest; these bound parser work only.
|
||||
pub const MAX_PROVIDER_NAME_BYTES: usize = 128;
|
||||
pub const MAX_PROVIDER_KINDS: usize = 64;
|
||||
pub const MAX_PROVIDER_ARTIFACTS: usize = 32;
|
||||
pub const MAX_PROVIDER_PERMISSIONS: usize = 64;
|
||||
/// Maximum selected artifact size (256 MiB).
|
||||
pub const MAX_PROVIDER_ARTIFACT_BYTES: u64 = 256 * 1024 * 1024;
|
||||
/// Maximum manifest size (1 MiB); a manifest is metadata, not an artifact.
|
||||
pub const MAX_PROVIDER_MANIFEST_BYTES: u64 = 1024 * 1024;
|
||||
|
||||
/// Core resource kinds owned by the host; third parties may never override
|
||||
/// them (RFC-EXT-0001 §5.3.1). `stdio`/`file`/`tcp`/`tls`/`udp`/`tcp-listener`
|
||||
/// are dispatched by `marqdo-host` before the provider registry is consulted.
|
||||
pub const RESERVED_RESOURCE_KINDS: &[&str] =
|
||||
&["stdio", "file", "tcp", "tls", "udp", "tcp-listener"];
|
||||
|
||||
/// Raw `provider.json` schema version 2 document.
|
||||
#[derive(Debug, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct ProviderManifest {
|
||||
schema: String,
|
||||
schema_version: u64,
|
||||
name: String,
|
||||
version: String,
|
||||
abi: String,
|
||||
transport: String,
|
||||
kinds: Vec<String>,
|
||||
artifacts: Vec<ManifestArtifact>,
|
||||
#[serde(default)]
|
||||
permissions_requested: Vec<String>,
|
||||
/// Free-form, non-executable metadata. Ignored for execution and never used
|
||||
/// to select artifacts or grant permissions.
|
||||
#[serde(rename = "metadata", default)]
|
||||
_metadata: Option<serde_json::Value>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct ManifestArtifact {
|
||||
target: String,
|
||||
path: String,
|
||||
sha256: String,
|
||||
}
|
||||
|
||||
/// A manifest whose selected artifact has been resolved and verified.
|
||||
///
|
||||
/// Producing one requires reading the manifest, checking the schema rules,
|
||||
/// selecting the artifact for the current host target and verifying the
|
||||
/// content hash. It still does not grant access and no worker has been started.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct VerifiedProvider {
|
||||
/// Stable grant name (`--allow-provider NAME` must match exactly).
|
||||
pub name: String,
|
||||
/// Strict SemVer version declared by the manifest.
|
||||
pub version: String,
|
||||
/// Non-reserved resource kinds served by this provider.
|
||||
pub kinds: Vec<String>,
|
||||
/// Canonical manifest path.
|
||||
pub manifest_path: PathBuf,
|
||||
/// Canonical bundle directory (manifest parent).
|
||||
pub base_dir: PathBuf,
|
||||
/// `sha256:<hex>` of the manifest bytes at verification time.
|
||||
pub manifest_digest: String,
|
||||
/// Selected transport (`worker` in v2).
|
||||
pub transport: String,
|
||||
/// Canonical path of the selected, hash-verified artifact.
|
||||
pub selected_artifact_path: PathBuf,
|
||||
/// Bundle-relative artifact path exactly as declared.
|
||||
pub selected_artifact_relative: String,
|
||||
/// Lowercase hex sha256 declared by the manifest.
|
||||
pub selected_artifact_sha256: String,
|
||||
/// Target triple the selected artifact was built for.
|
||||
pub selected_target: String,
|
||||
}
|
||||
|
||||
impl VerifiedProvider {
|
||||
/// Re-verify the manifest bytes and the selected artifact immediately
|
||||
/// before the provider worker is spawned.
|
||||
///
|
||||
/// `run` must never trust a hash it checked earlier: another process can
|
||||
/// replace the manifest or the artifact between selection and use.
|
||||
pub fn reverify(&self) -> Result<(), PrepareError> {
|
||||
let bytes = std::fs::read(&self.manifest_path).map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"provider manifest `{}` could not be re-read before use: {error}",
|
||||
self.manifest_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
if bytes.len() as u64 > MAX_PROVIDER_MANIFEST_BYTES {
|
||||
return Err(PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"provider manifest `{}` grew beyond the {MAX_PROVIDER_MANIFEST_BYTES}-byte limit before use",
|
||||
self.manifest_path.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
let digest = manifest_digest(&bytes);
|
||||
if digest != self.manifest_digest {
|
||||
return Err(PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"provider manifest `{}` changed after verification (expected {}, actual {}); refusing to start its worker",
|
||||
self.manifest_path.display(),
|
||||
self.manifest_digest,
|
||||
digest
|
||||
),
|
||||
));
|
||||
}
|
||||
let artifact =
|
||||
verify_bundle_regular_file(&self.base_dir, &self.selected_artifact_relative)?;
|
||||
if artifact != self.selected_artifact_path {
|
||||
return Err(PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"provider `{}` artifact `{}` now resolves to `{}` instead of the verified `{}`",
|
||||
self.name,
|
||||
self.selected_artifact_relative,
|
||||
artifact.display(),
|
||||
self.selected_artifact_path.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
check_artifact_size(&artifact)?;
|
||||
verify_sha256(&artifact, &self.selected_artifact_sha256)?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Read, validate and verify every explicitly selected `--provider` manifest.
|
||||
///
|
||||
/// No worker is spawned and no artifact is executed. Duplicate provider names
|
||||
/// and duplicate resource kinds across providers are rejected deterministically
|
||||
/// (the returned list is sorted by provider name) so a later registration can
|
||||
/// never overwrite an earlier one.
|
||||
pub fn read_provider_manifests(paths: &[PathBuf]) -> Result<Vec<VerifiedProvider>, PrepareError> {
|
||||
let mut manifest_paths: BTreeSet<PathBuf> = BTreeSet::new();
|
||||
for path in paths {
|
||||
if path.is_dir() {
|
||||
return Err(PrepareError::option(
|
||||
"ext.manifest_unreadable",
|
||||
format!(
|
||||
"--provider `{}` must name a `{PROVIDER_MANIFEST_FILE}` file, not a directory",
|
||||
path.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
let canonical = path.canonicalize().map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.manifest_unreadable",
|
||||
format!(
|
||||
"cannot open provider manifest `{}`: {error}",
|
||||
path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
manifest_paths.insert(canonical);
|
||||
}
|
||||
|
||||
let mut providers = Vec::with_capacity(manifest_paths.len());
|
||||
for path in manifest_paths {
|
||||
providers.push(read_provider_manifest(&path)?);
|
||||
}
|
||||
|
||||
// Deterministic conflict resolution independent of CLI argument order.
|
||||
providers.sort_by(|left, right| {
|
||||
(&left.name, &left.manifest_digest).cmp(&(&right.name, &right.manifest_digest))
|
||||
});
|
||||
let mut unique: Vec<VerifiedProvider> = Vec::with_capacity(providers.len());
|
||||
for provider in providers {
|
||||
if let Some(existing) = unique.iter().find(|item| item.name == provider.name) {
|
||||
if existing.manifest_digest != provider.manifest_digest {
|
||||
return Err(PrepareError::option(
|
||||
"ext.duplicate_provider",
|
||||
format!(
|
||||
"provider name `{}` is selected with two different manifests (`{}` and `{}`); \
|
||||
explicit aliases are not implemented, so this is rejected",
|
||||
provider.name,
|
||||
existing.manifest_path.display(),
|
||||
provider.manifest_path.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
// The exact same declaration selected twice is idempotent.
|
||||
continue;
|
||||
}
|
||||
unique.push(provider);
|
||||
}
|
||||
|
||||
let mut claimed_kinds: BTreeMap<String, String> = BTreeMap::new();
|
||||
for provider in &unique {
|
||||
for kind in &provider.kinds {
|
||||
if let Some(owner) = claimed_kinds.get(kind) {
|
||||
return Err(PrepareError::option(
|
||||
"ext.duplicate_provider_kind",
|
||||
format!(
|
||||
"resource kind `{kind}` is declared by both provider `{owner}` and provider `{}`; \
|
||||
duplicate kinds are rejected before any worker starts",
|
||||
provider.name
|
||||
),
|
||||
));
|
||||
}
|
||||
claimed_kinds.insert(kind.clone(), provider.name.clone());
|
||||
}
|
||||
}
|
||||
|
||||
Ok(unique)
|
||||
}
|
||||
|
||||
fn read_provider_manifest(canonical_path: &Path) -> Result<VerifiedProvider, PrepareError> {
|
||||
let metadata = std::fs::metadata(canonical_path).map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.manifest_unreadable",
|
||||
format!(
|
||||
"cannot stat provider manifest `{}`: {error}",
|
||||
canonical_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
if metadata.len() > MAX_PROVIDER_MANIFEST_BYTES {
|
||||
return Err(PrepareError::option(
|
||||
"ext.manifest_invalid",
|
||||
format!(
|
||||
"provider manifest `{}` is {} bytes, larger than the {MAX_PROVIDER_MANIFEST_BYTES}-byte limit",
|
||||
canonical_path.display(),
|
||||
metadata.len()
|
||||
),
|
||||
));
|
||||
}
|
||||
let bytes = std::fs::read(canonical_path).map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.manifest_unreadable",
|
||||
format!(
|
||||
"cannot read provider manifest `{}`: {error}",
|
||||
canonical_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let digest = manifest_digest(&bytes);
|
||||
let value: serde_json::Value = serde_json::from_slice(&bytes).map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.manifest_invalid",
|
||||
format!(
|
||||
"provider manifest `{}` is not valid UTF-8 JSON: {error}",
|
||||
canonical_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
|
||||
// Schema/version gate runs before structural parsing so a v1 (or future)
|
||||
// manifest gets the stable `ext.unsupported_provider_schema` code instead
|
||||
// of a generic field error.
|
||||
let schema = value.get("schema").and_then(serde_json::Value::as_str);
|
||||
if schema != Some(PROVIDER_SCHEMA) {
|
||||
return Err(PrepareError::option(
|
||||
"ext.unsupported_provider_schema",
|
||||
format!(
|
||||
"provider manifest `{}` does not declare `schema: \"{PROVIDER_SCHEMA}\"`; \
|
||||
the CLI accepts schema version {PROVIDER_SCHEMA_VERSION} only",
|
||||
canonical_path.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
let schema_version = value
|
||||
.get("schema_version")
|
||||
.and_then(serde_json::Value::as_u64);
|
||||
if schema_version != Some(PROVIDER_SCHEMA_VERSION) {
|
||||
return Err(PrepareError::option(
|
||||
"ext.unsupported_provider_schema",
|
||||
format!(
|
||||
"provider manifest `{}` declares schema_version {}; the CLI accepts {PROVIDER_SCHEMA_VERSION} only \
|
||||
(v1 and future versions are rejected, there is no silent migration)",
|
||||
canonical_path.display(),
|
||||
schema_version
|
||||
.map(|version| version.to_string())
|
||||
.unwrap_or_else(|| "<missing>".to_string())
|
||||
),
|
||||
));
|
||||
}
|
||||
|
||||
let manifest: ProviderManifest = serde_json::from_value(value).map_err(|error| {
|
||||
PrepareError::option(
|
||||
"ext.manifest_invalid",
|
||||
format!(
|
||||
"provider manifest `{}` is not a valid {PROVIDER_SCHEMA} v{PROVIDER_SCHEMA_VERSION} document: {error}",
|
||||
canonical_path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
if manifest.schema != PROVIDER_SCHEMA || manifest.schema_version != PROVIDER_SCHEMA_VERSION {
|
||||
// Unreachable after the Value-level gate; kept fail-closed.
|
||||
return Err(PrepareError::option(
|
||||
"ext.unsupported_provider_schema",
|
||||
format!(
|
||||
"provider manifest `{}` uses an unsupported schema",
|
||||
canonical_path.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
validate_manifest(&manifest, canonical_path)?;
|
||||
|
||||
let base_dir = canonical_path
|
||||
.parent()
|
||||
.map(Path::to_path_buf)
|
||||
.unwrap_or_else(|| PathBuf::from("."));
|
||||
let selected_target = HostTarget::current().triple().to_string();
|
||||
let artifact = manifest
|
||||
.artifacts
|
||||
.iter()
|
||||
.find(|artifact| artifact.target == selected_target)
|
||||
.ok_or_else(|| {
|
||||
PrepareError::option(
|
||||
"ext.no_matching_artifact",
|
||||
format!(
|
||||
"provider `{}` has no artifact for the current target `{selected_target}`; \
|
||||
other platforms are never substituted",
|
||||
manifest.name
|
||||
),
|
||||
)
|
||||
})?;
|
||||
|
||||
let artifact_relative = normalise_artifact_path(&manifest.name, &artifact.path)?;
|
||||
let artifact_path = verify_bundle_regular_file(&base_dir, &artifact_relative)?;
|
||||
check_artifact_size(&artifact_path)?;
|
||||
verify_sha256(&artifact_path, &artifact.sha256)?;
|
||||
|
||||
Ok(VerifiedProvider {
|
||||
name: manifest.name,
|
||||
version: manifest.version,
|
||||
kinds: manifest.kinds,
|
||||
manifest_path: canonical_path.to_path_buf(),
|
||||
base_dir,
|
||||
manifest_digest: digest,
|
||||
transport: manifest.transport,
|
||||
selected_artifact_path: artifact_path,
|
||||
selected_artifact_relative: artifact_relative,
|
||||
selected_artifact_sha256: artifact.sha256.to_ascii_lowercase(),
|
||||
selected_target,
|
||||
})
|
||||
}
|
||||
|
||||
fn validate_manifest(
|
||||
manifest: &ProviderManifest,
|
||||
manifest_path: &Path,
|
||||
) -> Result<(), PrepareError> {
|
||||
let invalid = |message: String| {
|
||||
PrepareError::option(
|
||||
"ext.manifest_invalid",
|
||||
format!("provider manifest `{}`: {message}", manifest_path.display()),
|
||||
)
|
||||
};
|
||||
|
||||
validate_provider_name(&manifest.name)
|
||||
.map_err(|error| invalid(format!("invalid provider name: {error}")))?;
|
||||
if RESERVED_RESOURCE_KINDS.contains(&manifest.name.as_str()) {
|
||||
return Err(invalid(format!(
|
||||
"provider name `{}` is a reserved core resource kind",
|
||||
manifest.name
|
||||
)));
|
||||
}
|
||||
semver::Version::parse(&manifest.version).map_err(|error| {
|
||||
invalid(format!(
|
||||
"version `{}` is not strict SemVer: {error}",
|
||||
manifest.version
|
||||
))
|
||||
})?;
|
||||
if manifest.abi != PROVIDER_ABI {
|
||||
return Err(PrepareError::option(
|
||||
"ext.unsupported_provider_schema",
|
||||
format!(
|
||||
"provider `{}` declares abi `{}`; schema v{PROVIDER_SCHEMA_VERSION} accepts `{PROVIDER_ABI}` only",
|
||||
manifest.name, manifest.abi
|
||||
),
|
||||
));
|
||||
}
|
||||
if manifest.transport != PROVIDER_TRANSPORT {
|
||||
return Err(PrepareError::option(
|
||||
"ext.unsupported_provider_schema",
|
||||
format!(
|
||||
"provider `{}` declares transport `{}`; schema v{PROVIDER_SCHEMA_VERSION} accepts `{PROVIDER_TRANSPORT}` only",
|
||||
manifest.name, manifest.transport
|
||||
),
|
||||
));
|
||||
}
|
||||
if manifest.kinds.is_empty() {
|
||||
return Err(invalid(
|
||||
"kinds must declare at least one resource kind".to_string(),
|
||||
));
|
||||
}
|
||||
if manifest.kinds.len() > MAX_PROVIDER_KINDS {
|
||||
return Err(invalid(format!(
|
||||
"kinds declares more than {MAX_PROVIDER_KINDS} entries"
|
||||
)));
|
||||
}
|
||||
let mut kinds = BTreeSet::new();
|
||||
for kind in &manifest.kinds {
|
||||
validate_provider_name(kind)
|
||||
.map_err(|error| invalid(format!("invalid resource kind `{kind}`: {error}")))?;
|
||||
if RESERVED_RESOURCE_KINDS.contains(&kind.as_str()) {
|
||||
return Err(invalid(format!(
|
||||
"resource kind `{kind}` is reserved by the host and cannot be overridden by a provider"
|
||||
)));
|
||||
}
|
||||
if !kinds.insert(kind.clone()) {
|
||||
return Err(invalid(format!(
|
||||
"resource kind `{kind}` is declared more than once"
|
||||
)));
|
||||
}
|
||||
}
|
||||
if manifest.artifacts.is_empty() {
|
||||
return Err(invalid(
|
||||
"artifacts must declare at least one target artifact".to_string(),
|
||||
));
|
||||
}
|
||||
if manifest.artifacts.len() > MAX_PROVIDER_ARTIFACTS {
|
||||
return Err(invalid(format!(
|
||||
"artifacts declares more than {MAX_PROVIDER_ARTIFACTS} entries"
|
||||
)));
|
||||
}
|
||||
let mut targets = BTreeSet::new();
|
||||
for artifact in &manifest.artifacts {
|
||||
let triple = TargetTriple::parse(&artifact.target).map_err(|error| {
|
||||
invalid(format!(
|
||||
"artifact target `{}` is not a target triple: {error}",
|
||||
artifact.target
|
||||
))
|
||||
})?;
|
||||
if !targets.insert(triple.to_string()) {
|
||||
return Err(invalid(format!(
|
||||
"artifact target `{triple}` is declared more than once"
|
||||
)));
|
||||
}
|
||||
if !is_lowercase_sha256(&artifact.sha256) {
|
||||
return Err(invalid(format!(
|
||||
"artifact for target `{:?}` must declare a 64-character lowercase hex sha256",
|
||||
artifact.target
|
||||
)));
|
||||
}
|
||||
}
|
||||
if manifest.permissions_requested.len() > MAX_PROVIDER_PERMISSIONS {
|
||||
return Err(invalid(format!(
|
||||
"permissions_requested declares more than {MAX_PROVIDER_PERMISSIONS} entries"
|
||||
)));
|
||||
}
|
||||
let mut permissions = BTreeSet::new();
|
||||
for permission in &manifest.permissions_requested {
|
||||
if !is_known_capability(permission) && !is_vendor_capability(permission) {
|
||||
return Err(invalid(format!(
|
||||
"permissions_requested entry `{permission}` is not a known capability or a dotted vendor capability"
|
||||
)));
|
||||
}
|
||||
if !permissions.insert(permission.clone()) {
|
||||
return Err(invalid(format!(
|
||||
"permission `{permission}` is declared more than once"
|
||||
)));
|
||||
}
|
||||
}
|
||||
// `metadata` is intentionally not validated: it has no execution meaning.
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Validate a dotted provider/kind name using the plugin name grammar
|
||||
/// (ASCII lowercase segment start, then lowercase/digits/`-`/`_`).
|
||||
pub fn validate_provider_name(name: &str) -> Result<(), String> {
|
||||
if name.is_empty() {
|
||||
return Err("must not be empty".to_string());
|
||||
}
|
||||
if name.len() > MAX_PROVIDER_NAME_BYTES {
|
||||
return Err(format!(
|
||||
"must not be longer than {MAX_PROVIDER_NAME_BYTES} bytes"
|
||||
));
|
||||
}
|
||||
for segment in name.split('.') {
|
||||
if segment.is_empty() {
|
||||
return Err(format!("`{name}` contains an empty segment"));
|
||||
}
|
||||
let mut bytes = segment.bytes();
|
||||
let first = bytes.next().unwrap_or(b'a');
|
||||
if !first.is_ascii_lowercase() {
|
||||
return Err(format!(
|
||||
"`{name}` must start every `.`-separated segment with a lowercase ASCII letter"
|
||||
));
|
||||
}
|
||||
if !bytes.all(|byte| {
|
||||
byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'-' || byte == b'_'
|
||||
}) {
|
||||
return Err(format!(
|
||||
"`{name}` may only contain lowercase ASCII letters, digits, `-` and `_`"
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Reject absolute paths and `..` traversal in declared artifact paths.
|
||||
fn normalise_artifact_path(provider: &str, path: &str) -> Result<String, PrepareError> {
|
||||
ensure_bundle_relative(path).map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!("provider `{provider}` artifact path `{path}` is invalid: {error}"),
|
||||
)
|
||||
})?;
|
||||
Ok(path.to_string())
|
||||
}
|
||||
|
||||
fn check_artifact_size(path: &Path) -> Result<(), PrepareError> {
|
||||
let metadata = std::fs::metadata(path).map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"cannot stat provider artifact `{}`: {error}",
|
||||
path.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
if metadata.len() > MAX_PROVIDER_ARTIFACT_BYTES {
|
||||
return Err(PrepareError::load(
|
||||
"ext.artifact_integrity",
|
||||
format!(
|
||||
"provider artifact `{}` is {} bytes, larger than the {MAX_PROVIDER_ARTIFACT_BYTES}-byte limit",
|
||||
path.display(),
|
||||
metadata.len()
|
||||
),
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn is_lowercase_sha256(value: &str) -> bool {
|
||||
value.len() == 64
|
||||
&& value
|
||||
.bytes()
|
||||
.all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte))
|
||||
}
|
||||
|
||||
fn is_known_capability(name: &str) -> bool {
|
||||
marqdo_plugin_abi::KNOWN_CAPABILITIES.contains(&name)
|
||||
}
|
||||
|
||||
/// A dotted vendor capability proposal (validation only, never a grant).
|
||||
fn is_vendor_capability(name: &str) -> bool {
|
||||
let segments: Vec<&str> = name.split('.').collect();
|
||||
if segments.len() < 3 {
|
||||
return false;
|
||||
}
|
||||
if matches!(
|
||||
segments[0],
|
||||
"console" | "math" | "fs" | "net" | "process" | "log" | "clock" | "random"
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
segments.iter().all(|segment| {
|
||||
!segment.is_empty()
|
||||
&& segment.bytes().enumerate().all(|(index, byte)| {
|
||||
byte.is_ascii_lowercase() || byte.is_ascii_digit() || (byte == b'-' && index > 0)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
fn manifest_digest(bytes: &[u8]) -> String {
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(bytes);
|
||||
let digest = hasher.finalize();
|
||||
let mut hex = String::with_capacity(digest.len() * 2);
|
||||
for byte in digest {
|
||||
hex.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
format!("sha256:{hex}")
|
||||
}
|
||||
|
||||
/// Whether the provider `worker` transport is supported on this build.
|
||||
///
|
||||
/// RFC-EXT-0001 §5.6 makes Linux x86_64 the mandatory lane. Other targets stay
|
||||
/// compilable but must fail closed with `ext.platform_unsupported` instead of
|
||||
/// pretending a helper works.
|
||||
pub fn provider_worker_platform_supported() -> bool {
|
||||
cfg!(all(target_os = "linux", target_arch = "x86_64"))
|
||||
}
|
||||
|
||||
/// Resolve the provider helper shipped next to the running `marqdo` executable.
|
||||
///
|
||||
/// Resolution is *only* the sibling file: there is no PATH lookup and no
|
||||
/// directory search (RFC-EXT-0001 §5.4.5). A missing helper is
|
||||
/// `ext.worker_unavailable`, never a fallback to in-process native loading.
|
||||
pub fn provider_worker_executable() -> Result<PathBuf, PrepareError> {
|
||||
if !provider_worker_platform_supported() {
|
||||
return Err(PrepareError::load(
|
||||
"ext.platform_unsupported",
|
||||
format!(
|
||||
"the resource provider worker transport is supported on linux/x86_64 only in this release; \
|
||||
this build targets {}/{}",
|
||||
std::env::consts::OS,
|
||||
std::env::consts::ARCH
|
||||
),
|
||||
));
|
||||
}
|
||||
let executable = std::env::current_exe().map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.worker_unavailable",
|
||||
format!(
|
||||
"cannot locate the marqdo executable to find `{PROVIDER_WORKER_BINARY}`: {error}"
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let directory = executable.parent().ok_or_else(|| {
|
||||
PrepareError::load(
|
||||
"ext.worker_unavailable",
|
||||
format!(
|
||||
"the marqdo executable `{}` has no parent directory",
|
||||
executable.display()
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let helper = directory.join(format!(
|
||||
"{PROVIDER_WORKER_BINARY}{}",
|
||||
std::env::consts::EXE_SUFFIX
|
||||
));
|
||||
match std::fs::metadata(&helper) {
|
||||
Ok(metadata) if metadata.is_file() => Ok(helper),
|
||||
_ => Err(PrepareError::load(
|
||||
"ext.worker_unavailable",
|
||||
format!(
|
||||
"the resource provider worker helper `{}` is missing; it must be shipped next to the \
|
||||
marqdo executable (PATH lookup is not allowed)",
|
||||
helper.display()
|
||||
),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Check the exact `--allow-provider` grant, re-verify the manifest and
|
||||
/// artifact, then register the existing worker resource-provider transport.
|
||||
///
|
||||
/// The grant check is repeated here (it was already checked during pre-spawn
|
||||
/// verification) so no caller can register an ungranted provider. The provider
|
||||
/// is always registered through
|
||||
/// [`StdHost::register_worker_resource_provider`]; this path never falls back
|
||||
/// to the trusted in-process C ABI or a native library.
|
||||
pub fn load_resource_provider_into_host(
|
||||
host: &mut StdHost,
|
||||
provider: &VerifiedProvider,
|
||||
allow_provider: &[String],
|
||||
) -> Result<(), PrepareError> {
|
||||
if !allow_provider.iter().any(|grant| grant == &provider.name) {
|
||||
return Err(PrepareError::load(
|
||||
"permission.denied",
|
||||
format!(
|
||||
"resource provider `{}` was selected but not granted; add --allow-provider {}",
|
||||
provider.name, provider.name
|
||||
),
|
||||
));
|
||||
}
|
||||
if provider.transport != PROVIDER_TRANSPORT {
|
||||
return Err(PrepareError::option(
|
||||
"ext.unsupported_provider_schema",
|
||||
format!(
|
||||
"provider `{}` transport `{}` cannot be loaded by the CLI",
|
||||
provider.name, provider.transport
|
||||
),
|
||||
));
|
||||
}
|
||||
// Re-verify immediately before use; never trust an earlier hash.
|
||||
provider.reverify()?;
|
||||
let worker = provider_worker_executable()?;
|
||||
let kinds: Vec<&str> = provider.kinds.iter().map(String::as_str).collect();
|
||||
let args = vec![provider
|
||||
.selected_artifact_path
|
||||
.to_string_lossy()
|
||||
.into_owned()];
|
||||
host.register_worker_resource_provider(&provider.name, &kinds, &worker, &args)
|
||||
.map_err(|error| {
|
||||
PrepareError::load(
|
||||
"ext.worker_unavailable",
|
||||
format!(
|
||||
"cannot start the worker for resource provider `{}` v{} (target {}) using `{}`: {}",
|
||||
provider.name,
|
||||
provider.version,
|
||||
provider.selected_target,
|
||||
worker.display(),
|
||||
error.message
|
||||
),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn tempdir(tag: &str) -> PathBuf {
|
||||
let path = std::env::temp_dir().join(format!(
|
||||
"marqdo-provider-manifest-{}-{tag}-{}",
|
||||
std::process::id(),
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|duration| duration.as_nanos())
|
||||
.unwrap_or(0)
|
||||
));
|
||||
std::fs::create_dir_all(&path).unwrap();
|
||||
path
|
||||
}
|
||||
|
||||
fn write_manifest(dir: &Path, manifest: &str) -> PathBuf {
|
||||
let path = dir.join(PROVIDER_MANIFEST_FILE);
|
||||
std::fs::write(&path, manifest).unwrap();
|
||||
path
|
||||
}
|
||||
|
||||
fn artifact(dir: &Path, name: &str, bytes: &[u8]) -> (String, String) {
|
||||
let path = dir.join(name);
|
||||
std::fs::write(&path, bytes).unwrap();
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(bytes);
|
||||
let mut hex = String::new();
|
||||
for byte in hasher.finalize() {
|
||||
hex.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
(name.to_string(), hex)
|
||||
}
|
||||
|
||||
fn manifest_text(name: &str, kinds: &[&str], path: &str, sha: &str) -> String {
|
||||
let kinds = kinds
|
||||
.iter()
|
||||
.map(|kind| format!("\"{kind}\""))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
format!(
|
||||
r#"{{
|
||||
"schema": "marqdo.provider",
|
||||
"schema_version": 2,
|
||||
"name": "{name}",
|
||||
"version": "0.1.0",
|
||||
"abi": "c-abi-1",
|
||||
"transport": "worker",
|
||||
"kinds": [{kinds}],
|
||||
"artifacts": [
|
||||
{{"target": "{}", "path": "{path}", "sha256": "{sha}"}}
|
||||
],
|
||||
"permissions_requested": []
|
||||
}}"#,
|
||||
HostTarget::current().triple()
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn valid_v2_manifest_verifies_artifact() {
|
||||
let dir = tempdir("valid");
|
||||
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
|
||||
let manifest = write_manifest(
|
||||
&dir,
|
||||
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha),
|
||||
);
|
||||
let providers = read_provider_manifests(&[manifest]).unwrap();
|
||||
assert_eq!(providers.len(), 1);
|
||||
assert_eq!(providers[0].name, "demo.sensor");
|
||||
assert_eq!(
|
||||
providers[0].selected_target,
|
||||
HostTarget::current().triple().to_string()
|
||||
);
|
||||
assert!(providers[0].manifest_digest.starts_with("sha256:"));
|
||||
providers[0].reverify().unwrap();
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn v1_manifest_is_rejected_without_migration() {
|
||||
let dir = tempdir("v1");
|
||||
let manifest = write_manifest(
|
||||
&dir,
|
||||
r#"{"schema":"marqdo.provider","schema_version":1,"name":"cabi","kinds":["cabi.sensor"],"abi":"c-abi-1","library":{"linux":"libcabi.so"}}"#,
|
||||
);
|
||||
let error = read_provider_manifests(&[manifest]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.unsupported_provider_schema");
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_target_is_no_matching_artifact() {
|
||||
let dir = tempdir("target");
|
||||
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
|
||||
let manifest = write_manifest(
|
||||
&dir,
|
||||
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha).replace(
|
||||
&HostTarget::current().triple().to_string(),
|
||||
"aarch64-unknown-linux-gnu",
|
||||
),
|
||||
);
|
||||
let error = read_provider_manifests(&[manifest]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.no_matching_artifact");
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reserved_kind_is_rejected() {
|
||||
let dir = tempdir("reserved");
|
||||
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
|
||||
let manifest = write_manifest(&dir, &manifest_text("demo.sensor", &["file"], &path, &sha));
|
||||
let error = read_provider_manifests(&[manifest]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.manifest_invalid");
|
||||
assert!(error.message.contains("reserved"), "{}", error.message);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn traversal_and_digest_mismatch_are_artifact_integrity() {
|
||||
let dir = tempdir("integrity");
|
||||
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
|
||||
let traversal = write_manifest(
|
||||
&dir,
|
||||
&manifest_text("demo.sensor", &["demo.sensor"], "../libdemo.so", &sha),
|
||||
);
|
||||
let error = read_provider_manifests(&[traversal]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.artifact_integrity");
|
||||
let mismatch = write_manifest(
|
||||
&dir,
|
||||
&manifest_text("demo.sensor", &["demo.sensor"], &path, &"0".repeat(64)),
|
||||
);
|
||||
let error = read_provider_manifests(&[mismatch]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.artifact_integrity");
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn duplicate_kinds_across_providers_are_rejected() {
|
||||
let dir = tempdir("duplicate-kind");
|
||||
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
|
||||
let first = write_manifest(
|
||||
&dir,
|
||||
&manifest_text("demo.one", &["demo.sensor"], &path, &sha),
|
||||
);
|
||||
let second_dir = dir.join("second");
|
||||
std::fs::create_dir_all(&second_dir).unwrap();
|
||||
std::fs::copy(dir.join(&path), second_dir.join(&path)).unwrap();
|
||||
let second = write_manifest(
|
||||
&second_dir,
|
||||
&manifest_text("demo.two", &["demo.sensor"], &path, &sha),
|
||||
);
|
||||
let error = read_provider_manifests(&[first, second]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.duplicate_provider_kind");
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn duplicate_provider_name_with_different_manifests_is_rejected() {
|
||||
let dir = tempdir("duplicate-name");
|
||||
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
|
||||
let first = write_manifest(
|
||||
&dir,
|
||||
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha),
|
||||
);
|
||||
let second_dir = dir.join("second");
|
||||
std::fs::create_dir_all(&second_dir).unwrap();
|
||||
std::fs::copy(dir.join(&path), second_dir.join(&path)).unwrap();
|
||||
// Same name, different version => different manifest digest.
|
||||
let second = write_manifest(
|
||||
&second_dir,
|
||||
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha)
|
||||
.replace("\"version\": \"0.1.0\"", "\"version\": \"0.2.0\""),
|
||||
);
|
||||
let error = read_provider_manifests(&[first.clone(), second]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.duplicate_provider");
|
||||
// Selecting the exact same manifest twice is idempotent.
|
||||
let providers = read_provider_manifests(&[first.clone(), first]).unwrap();
|
||||
assert_eq!(providers.len(), 1);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_fields_and_bad_names_are_manifest_invalid() {
|
||||
let dir = tempdir("invalid");
|
||||
let (path, sha) = artifact(&dir, "libdemo.so", b"provider-bytes");
|
||||
let unknown = write_manifest(
|
||||
&dir,
|
||||
&manifest_text("demo.sensor", &["demo.sensor"], &path, &sha).replace(
|
||||
"\"permissions_requested\": []",
|
||||
"\"permissions_requested\": [], \"exec\": true",
|
||||
),
|
||||
);
|
||||
let error = read_provider_manifests(&[unknown]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.manifest_invalid");
|
||||
let bad_name = write_manifest(
|
||||
&dir,
|
||||
&manifest_text("Demo.Sensor", &["demo.sensor"], &path, &sha),
|
||||
);
|
||||
let error = read_provider_manifests(&[bad_name]).unwrap_err();
|
||||
assert_eq!(error.code, "ext.manifest_invalid");
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn worker_helper_resolution_only_uses_the_executable_sibling() {
|
||||
// The real helper is installed next to the CLI binary by the
|
||||
// integration fixture; this test only asserts the platform/absence
|
||||
// contract and that a path is never derived from PATH.
|
||||
if provider_worker_platform_supported() {
|
||||
let helper = provider_worker_executable();
|
||||
// In a unit-test binary `current_exe()` is the test binary, so the
|
||||
// sibling helper is normally absent: the error must be structured.
|
||||
match helper {
|
||||
Ok(path) => assert!(path.is_file()),
|
||||
Err(error) => assert_eq!(error.code, "ext.worker_unavailable"),
|
||||
}
|
||||
} else {
|
||||
let error = provider_worker_executable().unwrap_err();
|
||||
assert_eq!(error.code, "ext.platform_unsupported");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -13,6 +13,25 @@ use sha2::{Digest, Sha256};
|
||||
/// Environment override used by tests and relocatable installations.
|
||||
pub const STDLIB_DIR_ENV: &str = "MARQDO_STDLIB_DIR";
|
||||
|
||||
/// One official bundled-stdlib public alias (ADR-0058).
|
||||
///
|
||||
/// The alias table lives in the verified `stdlib.json` metadata, not in Rust
|
||||
/// source: `打印`/`print` are ordinary `Marqdo` bindings to `io.mq.md::写值`
|
||||
/// and the runtime no longer special-cases print names.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct PublicAlias {
|
||||
/// Public names that resolve to the target function.
|
||||
pub public_names: Vec<String>,
|
||||
/// Package-relative target module (for example `io.mq.md`).
|
||||
pub target_module: String,
|
||||
/// Canonical/local target function name.
|
||||
pub target_function: String,
|
||||
/// Declared parameter names, used to verify the target signature.
|
||||
pub expected_signature: Option<String>,
|
||||
/// Conservative effect request summary.
|
||||
pub effects: Vec<String>,
|
||||
}
|
||||
|
||||
/// Verified bundled standard library.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct StdlibBundle {
|
||||
@@ -20,6 +39,7 @@ pub struct StdlibBundle {
|
||||
name: String,
|
||||
version: String,
|
||||
digest: String,
|
||||
aliases: Vec<PublicAlias>,
|
||||
}
|
||||
|
||||
impl StdlibBundle {
|
||||
@@ -81,14 +101,42 @@ impl StdlibBundle {
|
||||
"stdlib digest mismatch: declared {declared}, actual {actual}"
|
||||
));
|
||||
}
|
||||
let aliases = parse_public_aliases(&manifest, &manifest_path)?;
|
||||
let declared_alias_digest = manifest
|
||||
.get("aliases_digest")
|
||||
.and_then(serde_json::Value::as_str);
|
||||
let actual_alias_digest = aliases_digest(&aliases);
|
||||
match (declared_alias_digest, aliases.is_empty()) {
|
||||
(Some(declared), _) => {
|
||||
if declared != actual_alias_digest {
|
||||
return Err(format!(
|
||||
"stdlib public-alias metadata digest mismatch: declared {declared}, \
|
||||
actual {actual_alias_digest}"
|
||||
));
|
||||
}
|
||||
}
|
||||
(None, false) => {
|
||||
return Err(format!(
|
||||
"`{}` declares public_aliases without aliases_digest",
|
||||
manifest_path.display()
|
||||
));
|
||||
}
|
||||
(None, true) => {}
|
||||
}
|
||||
Ok(Self {
|
||||
root,
|
||||
name,
|
||||
version,
|
||||
digest: actual,
|
||||
aliases,
|
||||
})
|
||||
}
|
||||
|
||||
/// Verified official public aliases.
|
||||
pub fn public_aliases(&self) -> &[PublicAlias] {
|
||||
&self.aliases
|
||||
}
|
||||
|
||||
/// Bundle root.
|
||||
pub fn root(&self) -> &Path {
|
||||
&self.root
|
||||
@@ -227,6 +275,167 @@ fn hex(bytes: &[u8]) -> String {
|
||||
out
|
||||
}
|
||||
|
||||
fn parse_public_aliases(
|
||||
manifest: &serde_json::Value,
|
||||
manifest_path: &Path,
|
||||
) -> Result<Vec<PublicAlias>, String> {
|
||||
let Some(entries) = manifest.get("public_aliases") else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
let entries = entries.as_array().ok_or_else(|| {
|
||||
format!(
|
||||
"`{}` public_aliases must be an array",
|
||||
manifest_path.display()
|
||||
)
|
||||
})?;
|
||||
let mut aliases = Vec::new();
|
||||
let mut seen_names = std::collections::BTreeSet::new();
|
||||
for entry in entries {
|
||||
let public_names = entry
|
||||
.get("public_names")
|
||||
.and_then(serde_json::Value::as_array)
|
||||
.ok_or_else(|| {
|
||||
format!(
|
||||
"`{}` public alias has no public_names array",
|
||||
manifest_path.display()
|
||||
)
|
||||
})?
|
||||
.iter()
|
||||
.map(|name| {
|
||||
name.as_str().map(ToString::to_string).ok_or_else(|| {
|
||||
format!(
|
||||
"`{}` public alias name must be a string",
|
||||
manifest_path.display()
|
||||
)
|
||||
})
|
||||
})
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
if public_names.is_empty() {
|
||||
return Err(format!(
|
||||
"`{}` public alias declares no names",
|
||||
manifest_path.display()
|
||||
));
|
||||
}
|
||||
for name in &public_names {
|
||||
if name.is_empty() {
|
||||
return Err(format!(
|
||||
"`{}` public alias name must not be empty",
|
||||
manifest_path.display()
|
||||
));
|
||||
}
|
||||
if !seen_names.insert(name.clone()) {
|
||||
return Err(format!(
|
||||
"`{}` public alias name `{name}` is declared more than once",
|
||||
manifest_path.display()
|
||||
));
|
||||
}
|
||||
}
|
||||
let target_module = entry
|
||||
.get("target_module")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.ok_or_else(|| {
|
||||
format!(
|
||||
"`{}` public alias has no target_module",
|
||||
manifest_path.display()
|
||||
)
|
||||
})?
|
||||
.to_string();
|
||||
validate_bundle_relative(&target_module).map_err(|error| {
|
||||
format!(
|
||||
"`{}` public alias target_module `{target_module}` is invalid: {error}",
|
||||
manifest_path.display()
|
||||
)
|
||||
})?;
|
||||
let target_function = entry
|
||||
.get("target_function")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.ok_or_else(|| {
|
||||
format!(
|
||||
"`{}` public alias has no target_function",
|
||||
manifest_path.display()
|
||||
)
|
||||
})?
|
||||
.to_string();
|
||||
if target_function.is_empty() {
|
||||
return Err(format!(
|
||||
"`{}` public alias target_function must not be empty",
|
||||
manifest_path.display()
|
||||
));
|
||||
}
|
||||
let expected_signature = entry
|
||||
.get("expected_signature")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.map(ToString::to_string);
|
||||
let effects = entry
|
||||
.get("effects")
|
||||
.and_then(serde_json::Value::as_array)
|
||||
.map(|effects| {
|
||||
effects
|
||||
.iter()
|
||||
.filter_map(serde_json::Value::as_str)
|
||||
.map(ToString::to_string)
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
aliases.push(PublicAlias {
|
||||
public_names,
|
||||
target_module,
|
||||
target_function,
|
||||
expected_signature,
|
||||
effects,
|
||||
});
|
||||
}
|
||||
aliases.sort_by(|left, right| {
|
||||
(&left.target_module, &left.target_function)
|
||||
.cmp(&(&right.target_module, &right.target_function))
|
||||
});
|
||||
Ok(aliases)
|
||||
}
|
||||
|
||||
fn validate_bundle_relative(path: &str) -> Result<(), String> {
|
||||
let mut normalised = PathBuf::new();
|
||||
for component in Path::new(path).components() {
|
||||
match component {
|
||||
Component::Normal(part) => normalised.push(part),
|
||||
Component::CurDir => {}
|
||||
Component::ParentDir | Component::RootDir | Component::Prefix(_) => {
|
||||
return Err("must be a relative path inside the bundle".to_string())
|
||||
}
|
||||
}
|
||||
}
|
||||
if normalised.as_os_str().is_empty() {
|
||||
return Err("must not be empty".to_string());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Canonical digest over the official public-alias table.
|
||||
///
|
||||
/// Format (identical to `tools/pack-stdlib.py`): for every alias sorted by
|
||||
/// `(target_module, target_function)`: module, NUL, function, NUL, then every
|
||||
/// public name sorted, NUL. A fixed binary format avoids JSON formatting drift.
|
||||
pub fn aliases_digest(aliases: &[PublicAlias]) -> String {
|
||||
let mut sorted = aliases.to_vec();
|
||||
sorted.sort_by(|left, right| {
|
||||
(&left.target_module, &left.target_function)
|
||||
.cmp(&(&right.target_module, &right.target_function))
|
||||
});
|
||||
let mut hasher = Sha256::new();
|
||||
for alias in &sorted {
|
||||
hasher.update(alias.target_module.as_bytes());
|
||||
hasher.update([0]);
|
||||
hasher.update(alias.target_function.as_bytes());
|
||||
hasher.update([0]);
|
||||
let mut names = alias.public_names.clone();
|
||||
names.sort();
|
||||
for name in names {
|
||||
hasher.update(name.as_bytes());
|
||||
hasher.update([0]);
|
||||
}
|
||||
}
|
||||
format!("sha256:{}", hex(&hasher.finalize()))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
@@ -272,3 +272,125 @@ fn walk_rs(root: &Path) -> Vec<PathBuf> {
|
||||
}
|
||||
files
|
||||
}
|
||||
|
||||
/// RFC-EXT-0001 §4/§14: the runtime must dispatch by the prepared binding and
|
||||
/// never re-resolve a name or special-case a print function.
|
||||
#[test]
|
||||
fn callable_binding_is_the_only_static_resolution_path() {
|
||||
let root = repo_root();
|
||||
let loader = read(&root.join("crates/cli/src/loader.rs"));
|
||||
let interpreter = read(&root.join("crates/marqdo-runtime/src/interpreter.rs"));
|
||||
let run = read(&root.join("crates/cli/src/commands/run.rs"));
|
||||
let frontend = read(&root.join("crates/marqdo-frontend/src/calls.rs"));
|
||||
|
||||
assert!(
|
||||
!loader.contains("program_uses_print"),
|
||||
"loader must not scan for per-function names (ADR-0058)"
|
||||
);
|
||||
assert!(
|
||||
!interpreter.contains("print_override"),
|
||||
"the runtime must not special-case print (ADR-0054/0058)"
|
||||
);
|
||||
assert!(
|
||||
!interpreter.contains("external_callables.is_empty"),
|
||||
"the runtime must not fall back to plugin guessing (ADR-0054)"
|
||||
);
|
||||
assert!(
|
||||
interpreter.contains("call_bindings.get"),
|
||||
"the runtime must dispatch by the prepared call binding table"
|
||||
);
|
||||
assert!(
|
||||
!interpreter.contains("builtins::find(&qualified)"),
|
||||
"a qualified name must not be re-resolved at run time"
|
||||
);
|
||||
assert!(
|
||||
!run.contains("external_callables")
|
||||
|| !run.contains("external_callables")
|
||||
|| !run.contains("push(marqdo_ir::callable::CallableDescriptor"),
|
||||
"run must not synthesise plugin descriptors (ADR-0054/§4.5)"
|
||||
);
|
||||
assert!(
|
||||
frontend.contains("bind_program_calls"),
|
||||
"the frontend must expose the single binding entry point"
|
||||
);
|
||||
}
|
||||
|
||||
/// RFC-EXT-0001 §14.2/S4: no business-domain builtin or host-op branch may be
|
||||
/// added for an ecosystem feature. These must be implemented as `.mq.md`,
|
||||
/// plugin exports or provider streams.
|
||||
#[test]
|
||||
fn no_business_domain_builtins_or_host_ops() {
|
||||
let root = repo_root();
|
||||
let builtins = read(&root.join("crates/marqdo-ir/src/builtins.rs"));
|
||||
let host = read(&root.join("crates/marqdo-host/src/host.rs"));
|
||||
for forbidden in ["quantum.", "ai.chat", "vendor.", "demo.sensor", "marqdo."] {
|
||||
assert!(
|
||||
!builtins.contains(forbidden),
|
||||
"business builtin namespace `{forbidden}` must not exist"
|
||||
);
|
||||
}
|
||||
assert!(
|
||||
!host.contains("\"demo.sensor\""),
|
||||
"provider kinds must be data, not host-op string branches"
|
||||
);
|
||||
}
|
||||
|
||||
/// RFC-EXT-0001 §8: official stdlib public aliases are verified metadata, not
|
||||
/// hard-coded runtime names. Recompute the canonical metadata digest.
|
||||
#[test]
|
||||
fn official_stdlib_public_aliases_are_verified_metadata() {
|
||||
let root = repo_root().join("stdlib");
|
||||
let manifest: serde_json::Value =
|
||||
serde_json::from_slice(&std::fs::read(root.join("stdlib.json")).unwrap()).unwrap();
|
||||
let aliases = manifest["public_aliases"]
|
||||
.as_array()
|
||||
.expect("stdlib.json must declare public_aliases");
|
||||
assert!(!aliases.is_empty());
|
||||
let mut hasher = Sha256::new();
|
||||
let mut entries: Vec<&serde_json::Value> = aliases.iter().collect();
|
||||
entries.sort_by_key(|alias| {
|
||||
(
|
||||
alias["target_module"].as_str().unwrap_or("").to_string(),
|
||||
alias["target_function"].as_str().unwrap_or("").to_string(),
|
||||
)
|
||||
});
|
||||
for alias in &entries {
|
||||
hasher.update(alias["target_module"].as_str().unwrap().as_bytes());
|
||||
hasher.update([0]);
|
||||
hasher.update(alias["target_function"].as_str().unwrap().as_bytes());
|
||||
hasher.update([0]);
|
||||
let mut names: Vec<&str> = alias["public_names"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter_map(serde_json::Value::as_str)
|
||||
.collect();
|
||||
names.sort();
|
||||
for name in names {
|
||||
hasher.update(name.as_bytes());
|
||||
hasher.update([0]);
|
||||
}
|
||||
}
|
||||
let actual = format!("sha256:{}", hex(&hasher.finalize()));
|
||||
assert_eq!(
|
||||
manifest["aliases_digest"], actual,
|
||||
"stdlib.json aliases_digest must cover the public alias table"
|
||||
);
|
||||
for alias in aliases {
|
||||
let target_module = alias["target_module"].as_str().unwrap();
|
||||
let target_function = alias["target_function"].as_str().unwrap();
|
||||
let module = read(&root.join(target_module));
|
||||
assert!(
|
||||
module.contains(&format!("## {target_function}")),
|
||||
"public alias target `{target_module}::{target_function}` must exist"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn hex(bytes: &[u8]) -> String {
|
||||
let mut out = String::with_capacity(bytes.len() * 2);
|
||||
for byte in bytes {
|
||||
out.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
out
|
||||
}
|
||||
@@ -0,0 +1,653 @@
|
||||
//! Black-box callable resolution / binding conformance (RFC-EXT-0001 §4.6).
|
||||
//!
|
||||
//! Every test drives the real `marqdo` binary through the public CLI. The
|
||||
//! point is to prove that `check` and `run` share one static resolution and
|
||||
//! that the runtime never guesses a plugin from a dotted string prefix.
|
||||
#![cfg(unix)]
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Output};
|
||||
use std::sync::atomic::{AtomicU32, Ordering};
|
||||
|
||||
use marqdo_plugin_abi::{sha256_file, HostTarget};
|
||||
|
||||
static COUNTER: AtomicU32 = AtomicU32::new(0);
|
||||
|
||||
struct TempDir(PathBuf);
|
||||
|
||||
impl TempDir {
|
||||
fn new(tag: &str) -> Self {
|
||||
let id = COUNTER.fetch_add(1, Ordering::Relaxed);
|
||||
let path = std::env::temp_dir().join(format!(
|
||||
"marqdo-callable-test-{}-{id}-{tag}",
|
||||
std::process::id()
|
||||
));
|
||||
std::fs::create_dir_all(&path).unwrap();
|
||||
Self(path)
|
||||
}
|
||||
|
||||
fn path(&self) -> &Path {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TempDir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn repo() -> PathBuf {
|
||||
Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.unwrap()
|
||||
.parent()
|
||||
.unwrap()
|
||||
.to_path_buf()
|
||||
}
|
||||
|
||||
fn cli(args: &[&str], dir: &Path) -> Output {
|
||||
Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(args)
|
||||
.current_dir(dir)
|
||||
.output()
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
fn write(path: &Path, text: &str) {
|
||||
std::fs::write(path, text).unwrap();
|
||||
}
|
||||
|
||||
fn library_name() -> &'static str {
|
||||
if cfg!(target_os = "macos") {
|
||||
"solver.dylib"
|
||||
} else {
|
||||
"solver.so"
|
||||
}
|
||||
}
|
||||
|
||||
fn shared_flags() -> &'static [&'static str] {
|
||||
if cfg!(target_os = "macos") {
|
||||
&["-dynamiclib", "-fPIC"]
|
||||
} else {
|
||||
&["-shared", "-fPIC"]
|
||||
}
|
||||
}
|
||||
|
||||
fn have_cc() -> bool {
|
||||
Command::new("cc")
|
||||
.arg("--version")
|
||||
.output()
|
||||
.map(|output| output.status.success())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
fn compile_plugin(dir: &Path, extra: &[&str]) -> PathBuf {
|
||||
let output = dir.join(library_name());
|
||||
let status = Command::new("cc")
|
||||
.args(["-std=c11", "-O2"])
|
||||
.args(shared_flags())
|
||||
.arg("-I")
|
||||
.arg(repo().join("include"))
|
||||
.args(extra)
|
||||
.arg(repo().join("examples/plugin-demo/solver.c"))
|
||||
.arg("-o")
|
||||
.arg(&output)
|
||||
.status()
|
||||
.expect("cc");
|
||||
assert!(status.success(), "cc failed");
|
||||
output
|
||||
}
|
||||
|
||||
fn write_worker_manifest(dir: &Path, artifact: &Path, exports: &str) {
|
||||
let sha = sha256_file(artifact).unwrap();
|
||||
write(
|
||||
&dir.join("plugin.yaml"),
|
||||
&format!(
|
||||
r#"plugin:
|
||||
name: demo.solver
|
||||
version: 0.1.0
|
||||
abi: "^1.0"
|
||||
kind: worker
|
||||
exports:
|
||||
{exports} requires:
|
||||
- math.compute
|
||||
artifacts:
|
||||
- target: {}
|
||||
file: {}
|
||||
sha256: "{sha}"
|
||||
"#,
|
||||
HostTarget::current().triple(),
|
||||
artifact.file_name().unwrap().to_string_lossy()
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
fn add_export(name: &str, signature: &str) -> String {
|
||||
format!(" - name: {name}\n signature: \"{signature}\"\n")
|
||||
}
|
||||
|
||||
fn stderr(output: &Output) -> String {
|
||||
String::from_utf8_lossy(&output.stderr).to_string()
|
||||
}
|
||||
|
||||
fn stdout(output: &Output) -> String {
|
||||
String::from_utf8_lossy(&output.stdout).to_string()
|
||||
}
|
||||
|
||||
// CALL-01
|
||||
#[test]
|
||||
fn import_alias_conflicting_with_builtin_namespace_is_rejected() {
|
||||
let dir = TempDir::new("call01");
|
||||
write(
|
||||
&dir.path().join("math.mq.md"),
|
||||
"## @exec eval\n\n`params(a: Int)`\n\n`return a + 100`\n",
|
||||
);
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"---\nimport:\n math: \"./math.mq.md\"\n---\n\n## @exec main\n\n`return math.eval(a=1)`\n",
|
||||
);
|
||||
let check = cli(&["check", "index.mq.md"], dir.path());
|
||||
assert_eq!(check.status.code(), Some(1), "{}", stderr(&check));
|
||||
assert!(
|
||||
stderr(&check).contains("ext.import_namespace_conflict"),
|
||||
"{}",
|
||||
stderr(&check)
|
||||
);
|
||||
let run = cli(&["run", "index.mq.md"], dir.path());
|
||||
assert_eq!(run.status.code(), Some(1));
|
||||
assert!(stdout(&run).is_empty());
|
||||
assert!(
|
||||
stderr(&run).contains("ext.import_namespace_conflict"),
|
||||
"{}",
|
||||
stderr(&run)
|
||||
);
|
||||
}
|
||||
|
||||
// CALL-02
|
||||
#[test]
|
||||
fn dotted_builtin_call_binds_without_an_import() {
|
||||
let output = cli(&["check", "examples/math.mq.md"], &repo());
|
||||
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
|
||||
let run = cli(&["call", "examples/math.mq.md", "计算圆面积"], &repo());
|
||||
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
|
||||
}
|
||||
|
||||
// CALL-03
|
||||
#[test]
|
||||
fn renamed_import_alias_binds_the_module_function() {
|
||||
let dir = TempDir::new("call03");
|
||||
write(
|
||||
&dir.path().join("math.mq.md"),
|
||||
"## @exec eval\n\n`params(a: Int)`\n\n`return a + 100`\n",
|
||||
);
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"---\nimport:\n mymath: \"./math.mq.md\"\n---\n\n## @exec main\n\n`return mymath.eval(a=1)`\n",
|
||||
);
|
||||
let check = cli(&["check", "index.mq.md"], dir.path());
|
||||
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
|
||||
let run = cli(&["run", "index.mq.md", "--json"], dir.path());
|
||||
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
|
||||
let json: serde_json::Value = serde_json::from_str(&stdout(&run)).unwrap();
|
||||
assert_eq!(
|
||||
json["value"],
|
||||
serde_json::json!({"type": "int", "decimal": "101"})
|
||||
);
|
||||
}
|
||||
|
||||
// CALL-04 / CALL-05
|
||||
#[test]
|
||||
fn unknown_plugin_export_is_a_static_symbol_error() {
|
||||
let dir = TempDir::new("call04");
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`return solver.add(a=1, b=2)`\n",
|
||||
);
|
||||
let check = cli(&["check", "index.mq.md"], dir.path());
|
||||
assert_eq!(check.status.code(), Some(1));
|
||||
assert!(
|
||||
stderr(&check).contains("symbol.unknown"),
|
||||
"{}",
|
||||
stderr(&check)
|
||||
);
|
||||
let run = cli(&["run", "index.mq.md"], dir.path());
|
||||
assert_eq!(run.status.code(), Some(1));
|
||||
assert!(stderr(&run).contains("symbol.unknown"), "{}", stderr(&run));
|
||||
assert!(stdout(&run).is_empty());
|
||||
// CALL-05: the diagnostic does not change when an unrelated plugin is
|
||||
// loaded, because the static export table is still the only source.
|
||||
if !have_cc() {
|
||||
return;
|
||||
}
|
||||
let plugin_dir = TempDir::new("call05-plugin");
|
||||
let artifact = compile_plugin(plugin_dir.path(), &[]);
|
||||
write_worker_manifest(
|
||||
plugin_dir.path(),
|
||||
&artifact,
|
||||
&add_export("solver.add", "($a: Int, $b: Int) -> Int"),
|
||||
);
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`return solver.subtract(a=1, b=2)`\n",
|
||||
);
|
||||
let with_plugin = cli(
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
"--allow-plugin",
|
||||
"demo.solver",
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(with_plugin.status.code(), Some(1));
|
||||
assert!(
|
||||
stderr(&with_plugin).contains("symbol.unknown"),
|
||||
"{}",
|
||||
stderr(&with_plugin)
|
||||
);
|
||||
}
|
||||
|
||||
// CALL-06
|
||||
#[test]
|
||||
fn plugin_with_two_prefixes_binds_both_exports() {
|
||||
if !have_cc() {
|
||||
eprintln!("skipping: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = TempDir::new("call06");
|
||||
let plugin_dir = TempDir::new("call06-plugin");
|
||||
let artifact = compile_plugin(plugin_dir.path(), &["-DMQ_SOLVER_MULTI_EXPORT"]);
|
||||
write_worker_manifest(
|
||||
plugin_dir.path(),
|
||||
&artifact,
|
||||
&(add_export("solver.add", "($a: Int, $b: Int) -> Int")
|
||||
+ &add_export("solver.neg", "($a: Int) -> Int")),
|
||||
);
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`和 = solver.add(20, b=22)`\n\n`相反数 = solver.neg(7)`\n\n`打印(和)`\n\n`打印(相反数)`\n",
|
||||
);
|
||||
let output = cli(
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
"--allow-plugin",
|
||||
"demo.solver",
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
|
||||
assert_eq!(stdout(&output), "42\n-7\n");
|
||||
}
|
||||
|
||||
// CALL-07
|
||||
#[test]
|
||||
fn duplicate_complete_export_across_plugins_is_rejected_at_prepare() {
|
||||
if !have_cc() {
|
||||
eprintln!("skipping: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = TempDir::new("call07");
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`return solver.add(a=1, b=2)`\n",
|
||||
);
|
||||
let mut plugins = Vec::new();
|
||||
for tag in ["a", "b"] {
|
||||
let plugin_dir = TempDir::new(tag);
|
||||
let artifact = compile_plugin(plugin_dir.path(), &[]);
|
||||
let sha = sha256_file(&artifact).unwrap();
|
||||
write(
|
||||
&plugin_dir.path().join("plugin.yaml"),
|
||||
&format!(
|
||||
r#"plugin:
|
||||
name: demo.{tag}.solver
|
||||
version: 0.1.0
|
||||
abi: "^1.0"
|
||||
kind: worker
|
||||
exports:
|
||||
- name: solver.add
|
||||
signature: "($a: Int, $b: Int) -> Int"
|
||||
artifacts:
|
||||
- target: {}
|
||||
file: {}
|
||||
sha256: "{sha}"
|
||||
"#,
|
||||
HostTarget::current().triple(),
|
||||
artifact.file_name().unwrap().to_string_lossy()
|
||||
),
|
||||
);
|
||||
plugins.push(plugin_dir);
|
||||
}
|
||||
let check = cli(
|
||||
&[
|
||||
"check",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugins[0].path().to_str().unwrap(),
|
||||
"--plugin",
|
||||
plugins[1].path().to_str().unwrap(),
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(check.status.code(), Some(2), "{}", stderr(&check));
|
||||
assert!(
|
||||
stderr(&check).contains("ext.duplicate_callable"),
|
||||
"{}",
|
||||
stderr(&check)
|
||||
);
|
||||
}
|
||||
|
||||
// CALL-08
|
||||
#[test]
|
||||
fn bad_plugin_arguments_fail_in_check_and_run_identically() {
|
||||
if !have_cc() {
|
||||
eprintln!("skipping: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = TempDir::new("call08");
|
||||
let plugin_dir = TempDir::new("call08-plugin");
|
||||
let artifact = compile_plugin(plugin_dir.path(), &[]);
|
||||
write_worker_manifest(
|
||||
plugin_dir.path(),
|
||||
&artifact,
|
||||
&add_export("solver.add", "($a: Int, $b: Int) -> Int"),
|
||||
);
|
||||
for (tag, call) in [
|
||||
("missing", "solver.add(a=1)"),
|
||||
("unknown", "solver.add(a=1, b=2, c=3)"),
|
||||
("duplicate", "solver.add(a=1, a=2, b=3)"),
|
||||
("too-many-positional", "solver.add(1, 2, 3)"),
|
||||
] {
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
&format!("## @exec main\n\n`return {call}`\n"),
|
||||
);
|
||||
let check = cli(
|
||||
&[
|
||||
"check",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(check.status.code(), Some(1), "{tag}: {}", stderr(&check));
|
||||
let expected = if tag == "duplicate" {
|
||||
// The expression grammar rejects duplicate named arguments before
|
||||
// call binding; either way the failure is static.
|
||||
"sem.syntax"
|
||||
} else {
|
||||
"call.arity"
|
||||
};
|
||||
assert!(
|
||||
stderr(&check).contains(expected),
|
||||
"{tag}: {}",
|
||||
stderr(&check)
|
||||
);
|
||||
let run = cli(
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
"--allow-plugin",
|
||||
"demo.solver",
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(run.status.code(), Some(1), "{tag}: {}", stderr(&run));
|
||||
assert!(stdout(&run).is_empty(), "{tag}");
|
||||
// `check` appends a summary line and `run` a refusal line, but the
|
||||
// structured call.arity diagnostic itself must be identical.
|
||||
let arity = |text: &str| {
|
||||
text.lines()
|
||||
.filter(|line| line.contains("call.arity") || line.contains("sem.syntax"))
|
||||
.map(ToString::to_string)
|
||||
.collect::<Vec<_>>()
|
||||
};
|
||||
assert!(!arity(&stderr(&check)).is_empty(), "{tag}");
|
||||
assert_eq!(
|
||||
arity(&stderr(&check)),
|
||||
arity(&stderr(&run)),
|
||||
"{tag}: check != run"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// CALL-09
|
||||
#[test]
|
||||
fn changed_artifact_digest_is_rejected_before_main_runs() {
|
||||
if !have_cc() {
|
||||
eprintln!("skipping: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = TempDir::new("call09");
|
||||
let plugin_dir = TempDir::new("call09-plugin");
|
||||
let artifact = compile_plugin(plugin_dir.path(), &[]);
|
||||
write_worker_manifest(
|
||||
plugin_dir.path(),
|
||||
&artifact,
|
||||
&add_export("solver.add", "($a: Int, $b: Int) -> Int"),
|
||||
);
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`打印(\"must-not-run\")`\n\n`return solver.add(a=1, b=2)`\n",
|
||||
);
|
||||
// Static check succeeds: it never opens the artifact.
|
||||
let check = cli(
|
||||
&[
|
||||
"check",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
|
||||
// Corrupt the artifact bytes; run must reject before the entry function.
|
||||
let mut bytes = std::fs::read(&artifact).unwrap();
|
||||
bytes.push(0);
|
||||
std::fs::write(&artifact, bytes).unwrap();
|
||||
let run = cli(
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
"--allow-plugin",
|
||||
"demo.solver",
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(run.status.code(), Some(1), "{}", stderr(&run));
|
||||
assert!(
|
||||
stderr(&run).contains("ext.artifact_integrity"),
|
||||
"{}",
|
||||
stderr(&run)
|
||||
);
|
||||
assert!(stdout(&run).is_empty(), "{}", stdout(&run));
|
||||
}
|
||||
|
||||
// CALL-11
|
||||
#[test]
|
||||
fn local_function_reference_is_a_dynamic_language_call() {
|
||||
let dir = TempDir::new("call11");
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec 平方\n\n`params(n: Int)`\n\n`return n * n`\n\n## @exec 主\n\n`f = ref(\"#平方\")`\n\n`return f(n=4)`\n",
|
||||
);
|
||||
let run = cli(&["call", "index.mq.md", "主", "--json"], dir.path());
|
||||
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
|
||||
let json: serde_json::Value = serde_json::from_str(&stdout(&run)).unwrap();
|
||||
assert_eq!(
|
||||
json["value"],
|
||||
serde_json::json!({"type": "int", "decimal": "16"})
|
||||
);
|
||||
}
|
||||
|
||||
// CALL-12
|
||||
#[test]
|
||||
fn calling_a_text_value_is_not_callable_and_never_reaches_the_host() {
|
||||
let dir = TempDir::new("call12");
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`f = \"text\"`\n\n`return f()`\n",
|
||||
);
|
||||
let check = cli(&["check", "index.mq.md"], dir.path());
|
||||
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
|
||||
let run = cli(&["run", "index.mq.md"], dir.path());
|
||||
assert_eq!(run.status.code(), Some(1), "{}", stderr(&run));
|
||||
assert!(
|
||||
stderr(&run).contains("call.not_callable") || stderr(&run).contains("not_callable"),
|
||||
"{}",
|
||||
stderr(&run)
|
||||
);
|
||||
}
|
||||
|
||||
// CALL-13
|
||||
#[test]
|
||||
fn explicit_import_aliases_bind_same_named_functions_independently() {
|
||||
let dir = TempDir::new("call13");
|
||||
write(
|
||||
&dir.path().join("a.mq.md"),
|
||||
"## @exec value\n\n`params()`\n\n`return 10`\n",
|
||||
);
|
||||
write(
|
||||
&dir.path().join("b.mq.md"),
|
||||
"## @exec value\n\n`params()`\n\n`return 20`\n",
|
||||
);
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"---\nimport:\n aa: \"./a.mq.md\"\n bb: \"./b.mq.md\"\n---\n\n## @exec main\n\n`return aa.value() + bb.value()`\n",
|
||||
);
|
||||
let run = cli(&["run", "index.mq.md", "--json"], dir.path());
|
||||
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
|
||||
let json: serde_json::Value = serde_json::from_str(&stdout(&run)).unwrap();
|
||||
assert_eq!(
|
||||
json["value"],
|
||||
serde_json::json!({"type": "int", "decimal": "30"})
|
||||
);
|
||||
}
|
||||
|
||||
// CALL-14
|
||||
#[test]
|
||||
fn check_never_executes_a_plugin_artifact_or_side_effect_script() {
|
||||
let dir = TempDir::new("call14");
|
||||
let plugin_dir = TempDir::new("call14-plugin");
|
||||
let marker = plugin_dir.path().join("executed.marker");
|
||||
let artifact = plugin_dir.path().join(library_name());
|
||||
write(
|
||||
&artifact,
|
||||
&format!("#!/bin/sh\ntouch '{}'\nexit 0\n", marker.display()),
|
||||
);
|
||||
let sha = sha256_file(&artifact).unwrap();
|
||||
write(
|
||||
&plugin_dir.path().join("plugin.yaml"),
|
||||
&format!(
|
||||
r#"plugin:
|
||||
name: demo.script
|
||||
version: 0.1.0
|
||||
abi: "^1.0"
|
||||
kind: worker
|
||||
exports:
|
||||
- name: demo.run
|
||||
signature: "() -> Int"
|
||||
artifacts:
|
||||
- target: {}
|
||||
file: {}
|
||||
sha256: "{sha}"
|
||||
"#,
|
||||
HostTarget::current().triple(),
|
||||
artifact.file_name().unwrap().to_string_lossy()
|
||||
),
|
||||
);
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`return demo.run()`\n",
|
||||
);
|
||||
let check = cli(
|
||||
&[
|
||||
"check",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
|
||||
assert!(!marker.exists(), "check executed the artifact");
|
||||
}
|
||||
|
||||
// CALL-15
|
||||
#[test]
|
||||
fn check_emit_and_run_share_the_same_static_binding_source() {
|
||||
if !have_cc() {
|
||||
eprintln!("skipping: cc is unavailable");
|
||||
return;
|
||||
}
|
||||
let dir = TempDir::new("call15");
|
||||
let plugin_dir = TempDir::new("call15-plugin");
|
||||
let artifact = compile_plugin(plugin_dir.path(), &[]);
|
||||
write_worker_manifest(
|
||||
plugin_dir.path(),
|
||||
&artifact,
|
||||
&add_export("solver.add", "($a: Int, $b: Int) -> Int"),
|
||||
);
|
||||
write(
|
||||
&dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`return solver.add(a=1, b=2)`\n",
|
||||
);
|
||||
let analysis_path = dir.path().join("analysis.json");
|
||||
let check = cli(
|
||||
&[
|
||||
"check",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
"--emit",
|
||||
analysis_path.to_str().unwrap(),
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(check.status.code(), Some(0), "{}", stderr(&check));
|
||||
let analysis: serde_json::Value =
|
||||
serde_json::from_str(&std::fs::read_to_string(&analysis_path).unwrap()).unwrap();
|
||||
let functions = analysis["modules"]
|
||||
.as_array()
|
||||
.and_then(|modules| modules.first())
|
||||
.and_then(|module| module["functions"].as_array())
|
||||
.unwrap_or_else(|| panic!("analysis has no module functions: {analysis}"));
|
||||
assert!(!functions.is_empty());
|
||||
let run = cli(
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--plugin",
|
||||
plugin_dir.path().to_str().unwrap(),
|
||||
"--allow-plugin",
|
||||
"demo.solver",
|
||||
"--json",
|
||||
],
|
||||
dir.path(),
|
||||
);
|
||||
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
|
||||
// Without the descriptor the same call is statically unknown: run/check
|
||||
// consume the same prepared binding, not a runtime name probe.
|
||||
let without = cli(&["check", "index.mq.md"], dir.path());
|
||||
assert_eq!(without.status.code(), Some(1));
|
||||
assert!(stderr(&without).contains("symbol.unknown"));
|
||||
}
|
||||
|
||||
// CALL-16
|
||||
#[test]
|
||||
fn print_alias_keeps_the_existing_output_bytes() {
|
||||
let run = cli(&["run", "examples/hello.mq.md"], &repo());
|
||||
assert_eq!(run.status.code(), Some(0), "{}", stderr(&run));
|
||||
assert_eq!(stdout(&run), "你好,Marqdo\n");
|
||||
}
|
||||
@@ -166,8 +166,8 @@ fn semantic_goldens_match_end_to_end_execution() {
|
||||
let mut program = marqdo_ir::Program::default();
|
||||
program.insert(analysis.module.clone());
|
||||
program.main_module = Some(module_id.clone());
|
||||
install_stdlib_print(&mut program);
|
||||
install_entry_stdlib_imports(&mut program, &module_id, &analysis);
|
||||
bind_calls_with_stdlib_aliases(&mut program);
|
||||
|
||||
let mut capabilities = Capabilities::pure();
|
||||
if grant_foreign {
|
||||
@@ -284,26 +284,88 @@ fn normalise_relative(path: &Path) -> String {
|
||||
.join("/")
|
||||
}
|
||||
|
||||
/// Load the bundled stdlib print modules for direct-runtime fixtures.
|
||||
/// Bind direct-runtime fixtures exactly like the CLI composition root.
|
||||
///
|
||||
/// The CLI loader normally injects these for Compat-B. The semantic harness
|
||||
/// builds a Program directly, so it mirrors the same resolution here; without
|
||||
/// it, fixtures that call `打印` would hit the Compat-C "no bundle" error.
|
||||
fn install_stdlib_print(program: &mut Program) {
|
||||
/// The semantic harness builds a `Program` directly, so it mirrors the
|
||||
/// production path: load the official stdlib public-alias targets, build the
|
||||
/// unique registry, run the single binding pass and install its table. There is
|
||||
/// no `print_override` and no runtime name guessing.
|
||||
fn bind_calls_with_stdlib_aliases(program: &mut Program) {
|
||||
let root = repo_root().join("stdlib");
|
||||
if !root.join("stdlib.json").is_file() {
|
||||
return;
|
||||
}
|
||||
let mut loaded = std::collections::BTreeSet::new();
|
||||
let io = match load_stdlib_module(&root, "io.mq.md", program, &mut loaded) {
|
||||
Ok(id) => id,
|
||||
Err(error) => panic!("cannot load bundled stdlib: {error}"),
|
||||
};
|
||||
let function = program
|
||||
.module(&io)
|
||||
.and_then(|module| module.function("写值"))
|
||||
.unwrap_or_else(|| panic!("stdlib io has no 写值 function"));
|
||||
program.print_override = Some(function.id.clone());
|
||||
let mut official = Vec::new();
|
||||
if root.join("stdlib.json").is_file() {
|
||||
let text = fs::read_to_string(root.join("stdlib.json"))
|
||||
.unwrap_or_else(|error| panic!("cannot read stdlib.json: {error}"));
|
||||
let manifest: serde_json::Value = serde_json::from_str(&text)
|
||||
.unwrap_or_else(|error| panic!("invalid stdlib.json: {error}"));
|
||||
if let Some(aliases) = manifest
|
||||
.get("public_aliases")
|
||||
.and_then(serde_json::Value::as_array)
|
||||
{
|
||||
for entry in aliases {
|
||||
let public_names: Vec<String> = entry
|
||||
.get("public_names")
|
||||
.and_then(serde_json::Value::as_array)
|
||||
.unwrap_or_else(|| panic!("public alias has no public_names"))
|
||||
.iter()
|
||||
.filter_map(serde_json::Value::as_str)
|
||||
.map(ToString::to_string)
|
||||
.collect();
|
||||
let target_module = entry
|
||||
.get("target_module")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.unwrap_or_else(|| panic!("public alias has no target_module"));
|
||||
let target_function = entry
|
||||
.get("target_function")
|
||||
.and_then(serde_json::Value::as_str)
|
||||
.unwrap_or_else(|| panic!("public alias has no target_function"));
|
||||
let module_id = load_stdlib_module(&root, target_module, program, &mut loaded)
|
||||
.unwrap_or_else(|error| panic!("cannot load stdlib alias module: {error}"));
|
||||
let function = program
|
||||
.module(&module_id)
|
||||
.and_then(|module| module.function(target_function))
|
||||
.unwrap_or_else(|| {
|
||||
panic!("stdlib alias target `{target_function}` is missing")
|
||||
});
|
||||
let effects = entry
|
||||
.get("effects")
|
||||
.and_then(serde_json::Value::as_array)
|
||||
.map(|effects| {
|
||||
effects
|
||||
.iter()
|
||||
.filter_map(serde_json::Value::as_str)
|
||||
.map(ToString::to_string)
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
official.push(marqdo_ir::callable::OfficialAlias {
|
||||
public_names,
|
||||
function: function.id.clone(),
|
||||
signature: marqdo_ir::callable::CallableSignature {
|
||||
params: function.params.clone(),
|
||||
kind: function.signature,
|
||||
},
|
||||
owner_package: module_id.instance.clone(),
|
||||
effects_requested: effects,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut registry = program.callable_registry();
|
||||
for alias in official {
|
||||
registry
|
||||
.insert_official_alias(alias)
|
||||
.unwrap_or_else(|error| panic!("cannot register stdlib alias: {error}"));
|
||||
}
|
||||
let bound = marqdo_frontend::calls::bind_program_calls(program, ®istry);
|
||||
if !bound.diagnostics.is_empty() {
|
||||
panic!(
|
||||
"fixture failed static call binding: {:#?}",
|
||||
bound.diagnostics
|
||||
);
|
||||
}
|
||||
program.set_call_bindings(bound.bindings);
|
||||
}
|
||||
|
||||
fn load_stdlib_module(
|
||||
|
||||
@@ -224,6 +224,7 @@ fn runtime_negative_goldens() {
|
||||
let mut program = marqdo_ir::Program::default();
|
||||
program.insert(analysis.module.clone());
|
||||
program.main_module = Some(module_id.clone());
|
||||
bind_program_calls(&mut program);
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::pure(),
|
||||
@@ -284,3 +285,15 @@ fn runtime_negative_goldens() {
|
||||
}
|
||||
let _ = fs::remove_dir_all(&project_root);
|
||||
}
|
||||
|
||||
/// Bind direct-runtime fixtures exactly like the CLI composition root.
|
||||
fn bind_program_calls(program: &mut marqdo_ir::Program) {
|
||||
let registry = program.callable_registry();
|
||||
let bound = marqdo_frontend::calls::bind_program_calls(program, ®istry);
|
||||
assert!(
|
||||
bound.diagnostics.is_empty(),
|
||||
"fixture failed static call binding: {:#?}",
|
||||
bound.diagnostics
|
||||
);
|
||||
program.set_call_bindings(bound.bindings);
|
||||
}
|
||||
@@ -395,6 +395,14 @@ fn narration_removal_preserves_execution() {
|
||||
let mut program = marqdo_ir::Program::default();
|
||||
program.insert(analysis.module.clone());
|
||||
program.main_module = Some(module_id.clone());
|
||||
let registry = program.callable_registry();
|
||||
let bound = marqdo_frontend::calls::bind_program_calls(&program, ®istry);
|
||||
assert!(
|
||||
bound.diagnostics.is_empty(),
|
||||
"property fixture failed binding: {:#?}",
|
||||
bound.diagnostics
|
||||
);
|
||||
program.set_call_bindings(bound.bindings);
|
||||
let mut host = StdHost::new(
|
||||
SandboxMode::Project,
|
||||
Capabilities::pure(),
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,260 @@
|
||||
//! Official stdlib public alias routing (RFC-EXT-0001 §8, ADR-0058).
|
||||
//!
|
||||
//! `打印`/`print` are ordinary `Marqdo` bindings to `io.mq.md::写值` declared
|
||||
//! in the verified `stdlib.json` metadata. There is no `print_override`, no
|
||||
//! runtime name special case and no hard-coded per-function scan.
|
||||
#![cfg(unix)]
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Output};
|
||||
use std::sync::atomic::{AtomicU32, Ordering};
|
||||
|
||||
static COUNTER: AtomicU32 = AtomicU32::new(0);
|
||||
|
||||
struct TempDir(PathBuf);
|
||||
|
||||
impl TempDir {
|
||||
fn new(tag: &str) -> Self {
|
||||
let id = COUNTER.fetch_add(1, Ordering::Relaxed);
|
||||
let path = std::env::temp_dir().join(format!(
|
||||
"marqdo-stdlib-alias-test-{}-{id}-{tag}",
|
||||
std::process::id()
|
||||
));
|
||||
std::fs::create_dir_all(&path).unwrap();
|
||||
Self(path)
|
||||
}
|
||||
|
||||
fn path(&self) -> &Path {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TempDir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn repo() -> PathBuf {
|
||||
Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.unwrap()
|
||||
.parent()
|
||||
.unwrap()
|
||||
.to_path_buf()
|
||||
}
|
||||
|
||||
fn cli(args: &[&str], dir: &Path, env: &[(&str, &Path)]) -> Output {
|
||||
let mut command = Command::new(env!("CARGO_BIN_EXE_marqdo"));
|
||||
command.args(args).current_dir(dir);
|
||||
for (key, value) in env {
|
||||
command.env(key, value);
|
||||
}
|
||||
command.output().unwrap()
|
||||
}
|
||||
|
||||
fn stderr(output: &Output) -> String {
|
||||
String::from_utf8_lossy(&output.stderr).to_string()
|
||||
}
|
||||
|
||||
fn stdout(output: &Output) -> String {
|
||||
String::from_utf8_lossy(&output.stdout).to_string()
|
||||
}
|
||||
|
||||
fn copy_stdlib(target: &Path) {
|
||||
let source = repo().join("stdlib");
|
||||
std::fs::create_dir_all(target).unwrap();
|
||||
for entry in std::fs::read_dir(&source).unwrap() {
|
||||
let entry = entry.unwrap();
|
||||
let path = entry.path();
|
||||
if path.is_dir() {
|
||||
copy_dir(&path, &target.join(entry.file_name()));
|
||||
} else {
|
||||
std::fs::copy(&path, target.join(entry.file_name())).unwrap();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn copy_dir(source: &Path, target: &Path) {
|
||||
std::fs::create_dir_all(target).unwrap();
|
||||
for entry in std::fs::read_dir(source).unwrap() {
|
||||
let entry = entry.unwrap();
|
||||
let path = entry.path();
|
||||
if path.is_dir() {
|
||||
copy_dir(&path, &target.join(entry.file_name()));
|
||||
} else {
|
||||
std::fs::copy(&path, target.join(entry.file_name())).unwrap();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn print_and_its_alias_are_bound_to_the_official_module() {
|
||||
let dir = TempDir::new("print");
|
||||
std::fs::write(
|
||||
dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`打印(1)`\n\n`print(\"two\")`\n\n`return None`\n",
|
||||
)
|
||||
.unwrap();
|
||||
let output = cli(&["run", "index.mq.md"], dir.path(), &[]);
|
||||
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
|
||||
assert_eq!(stdout(&output), "1\ntwo\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn program_without_print_does_not_load_the_stdlib_module_graph() {
|
||||
let dir = TempDir::new("no-print");
|
||||
std::fs::write(
|
||||
dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`params()`\n\n`return 1 + 2`\n",
|
||||
)
|
||||
.unwrap();
|
||||
let analysis_path = dir.path().join("factorial.json");
|
||||
let output = cli(
|
||||
&[
|
||||
"check",
|
||||
"index.mq.md",
|
||||
"--emit",
|
||||
analysis_path.to_str().unwrap(),
|
||||
],
|
||||
dir.path(),
|
||||
&[],
|
||||
);
|
||||
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
|
||||
let analysis: serde_json::Value =
|
||||
serde_json::from_str(&std::fs::read_to_string(&analysis_path).unwrap()).unwrap();
|
||||
let modules = analysis["modules"].as_array().unwrap();
|
||||
assert!(
|
||||
modules
|
||||
.iter()
|
||||
.all(|module| !module["id"].as_str().unwrap_or("").starts_with("std:")),
|
||||
"a program without print must not load the stdlib: {analysis}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn print_program_loads_exactly_the_declared_alias_target() {
|
||||
let dir = TempDir::new("with-print");
|
||||
std::fs::write(
|
||||
dir.path().join("index.mq.md"),
|
||||
"## @exec main\n\n`打印(\"x\")`\n\n`return None`\n",
|
||||
)
|
||||
.unwrap();
|
||||
let analysis_path = dir.path().join("hello.json");
|
||||
let output = cli(
|
||||
&[
|
||||
"check",
|
||||
"index.mq.md",
|
||||
"--emit",
|
||||
analysis_path.to_str().unwrap(),
|
||||
],
|
||||
dir.path(),
|
||||
&[],
|
||||
);
|
||||
assert_eq!(output.status.code(), Some(0), "{}", stderr(&output));
|
||||
let analysis: serde_json::Value =
|
||||
serde_json::from_str(&std::fs::read_to_string(&analysis_path).unwrap()).unwrap();
|
||||
let ids: Vec<&str> = analysis["modules"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter_map(|module| module["id"].as_str())
|
||||
.collect();
|
||||
assert!(
|
||||
ids.iter().any(|id| id.ends_with("::io.mq.md")),
|
||||
"print must load io.mq.md: {ids:?}"
|
||||
);
|
||||
assert!(
|
||||
ids.iter().any(|id| id.ends_with("::format.mq.md")),
|
||||
"io.mq.md imports format.mq.md: {ids:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tampered_public_alias_metadata_digest_is_rejected() {
|
||||
let dir = TempDir::new("alias-digest");
|
||||
let stdlib = dir.path().join("stdlib");
|
||||
copy_stdlib(&stdlib);
|
||||
let manifest_path = stdlib.join("stdlib.json");
|
||||
let text = std::fs::read_to_string(&manifest_path).unwrap();
|
||||
let mut manifest: serde_json::Value = serde_json::from_str(&text).unwrap();
|
||||
manifest["aliases_digest"] = serde_json::json!("sha256:".to_string() + &"0".repeat(64));
|
||||
std::fs::write(
|
||||
&manifest_path,
|
||||
serde_json::to_string_pretty(&manifest).unwrap() + "\n",
|
||||
)
|
||||
.unwrap();
|
||||
let doc = dir.path().join("index.mq.md");
|
||||
std::fs::write(&doc, "## @exec main\n\n`打印(1)`\n\n`return None`\n").unwrap();
|
||||
let output = cli(
|
||||
&["run", doc.to_str().unwrap()],
|
||||
dir.path(),
|
||||
&[("MARQDO_STDLIB_DIR", &stdlib)],
|
||||
);
|
||||
assert!(!output.status.success(), "{}", stderr(&output));
|
||||
assert!(
|
||||
stderr(&output).contains("public-alias metadata digest mismatch"),
|
||||
"{}",
|
||||
stderr(&output)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tampered_alias_target_signature_is_rejected() {
|
||||
let dir = TempDir::new("alias-target");
|
||||
let stdlib = dir.path().join("stdlib");
|
||||
copy_stdlib(&stdlib);
|
||||
let manifest_path = stdlib.join("stdlib.json");
|
||||
let text = std::fs::read_to_string(&manifest_path).unwrap();
|
||||
let mut manifest: serde_json::Value = serde_json::from_str(&text).unwrap();
|
||||
manifest["public_aliases"][0]["expected_signature"] = serde_json::json!("params(值, 额外)");
|
||||
// Recompute the declared metadata digest so only the target check can fail.
|
||||
let canonical = manifest["public_aliases"][0].clone();
|
||||
let module = canonical["target_module"].as_str().unwrap();
|
||||
let function = canonical["target_function"].as_str().unwrap();
|
||||
let mut names: Vec<&str> = canonical["public_names"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter_map(serde_json::Value::as_str)
|
||||
.collect();
|
||||
names.sort();
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(module.as_bytes());
|
||||
hasher.update([0]);
|
||||
hasher.update(function.as_bytes());
|
||||
hasher.update([0]);
|
||||
for name in names {
|
||||
hasher.update(name.as_bytes());
|
||||
hasher.update([0]);
|
||||
}
|
||||
let digest = format!("sha256:{}", hex(&hasher.finalize()));
|
||||
manifest["aliases_digest"] = serde_json::json!(digest);
|
||||
std::fs::write(
|
||||
&manifest_path,
|
||||
serde_json::to_string_pretty(&manifest).unwrap() + "\n",
|
||||
)
|
||||
.unwrap();
|
||||
let doc = dir.path().join("index.mq.md");
|
||||
std::fs::write(&doc, "## @exec main\n\n`打印(1)`\n\n`return None`\n").unwrap();
|
||||
let output = cli(
|
||||
&["run", doc.to_str().unwrap()],
|
||||
dir.path(),
|
||||
&[("MARQDO_STDLIB_DIR", &stdlib)],
|
||||
);
|
||||
assert!(!output.status.success(), "{}", stderr(&output));
|
||||
assert!(
|
||||
stderr(&output).contains("expected signature") || stderr(&output).contains("alias"),
|
||||
"{}",
|
||||
stderr(&output)
|
||||
);
|
||||
}
|
||||
|
||||
fn hex(bytes: &[u8]) -> String {
|
||||
let mut out = String::with_capacity(bytes.len() * 2);
|
||||
for byte in bytes {
|
||||
out.push_str(&format!("{byte:02x}"));
|
||||
}
|
||||
out
|
||||
}
|
||||
@@ -39,7 +39,30 @@ impl AnalysisResult {
|
||||
/// Analyze one Markdown source as a Marqdo module.
|
||||
///
|
||||
/// `module_id` is the canonical project-relative identity (e.g. `index.mq.md`).
|
||||
///
|
||||
/// This standalone entry point also runs the module-local call check so that a
|
||||
/// single document can be validated without a program graph. The CLI
|
||||
/// composition root uses [`analyze_source_for_program`] instead: it defers all
|
||||
/// call resolution to the single program-wide binding pass (ADR-0054).
|
||||
pub fn analyze_source(module_id: &str, source: &str) -> AnalysisResult {
|
||||
analyze_source_inner(module_id, source, true)
|
||||
}
|
||||
|
||||
/// Analyze one Markdown source for the composition root.
|
||||
///
|
||||
/// Identical to [`analyze_source`] except that it does not run the provisional
|
||||
/// module-local call check; the caller binds calls exactly once over the whole
|
||||
/// program (including extension descriptors) and therefore owns the only
|
||||
/// authoritative call diagnostics.
|
||||
pub fn analyze_source_for_program(module_id: &str, source: &str) -> AnalysisResult {
|
||||
analyze_source_inner(module_id, source, false)
|
||||
}
|
||||
|
||||
fn analyze_source_inner(
|
||||
module_id: &str,
|
||||
source: &str,
|
||||
validate_local_calls: bool,
|
||||
) -> AnalysisResult {
|
||||
let id = ModuleId::from(module_id);
|
||||
let document = marqdo_markdown::parse(source);
|
||||
let units = units::build_units(&document, &id);
|
||||
@@ -161,7 +184,9 @@ pub fn analyze_source(module_id: &str, source: &str) -> AnalysisResult {
|
||||
resources,
|
||||
anchors: units.anchors.clone(),
|
||||
};
|
||||
diagnostics.extend(calls::validate_module(&module));
|
||||
if validate_local_calls {
|
||||
diagnostics.extend(calls::validate_module(&module));
|
||||
}
|
||||
|
||||
// One stable order for every consumer: source position first, then the
|
||||
// stable code, as required by `spec/mapping.md` §8. Diagnostics without a
|
||||
|
||||
+498
-248
@@ -1,33 +1,51 @@
|
||||
//! Static call validation.
|
||||
//! Static call resolution and binding (RFC-EXT-0001 §4, ADR-0054).
|
||||
//!
|
||||
//! Calls are resolved after all functions have signatures (including inferred
|
||||
//! ones), so arity/name mistakes are compile errors with source spans instead
|
||||
//! of surprises at runtime. Type errors stay runtime concerns: v1 has no static
|
||||
//! type inference beyond literal checks.
|
||||
//! This module owns the *single* resolution algorithm used by `check`, `run`,
|
||||
//! `inspect` and Analysis: it walks every statically visible call site exactly
|
||||
//! once, resolves it against [`marqdo_ir::callable::CallableRegistry`] and
|
||||
//! records a [`marqdo_ir::call_binding::BoundTarget`] in one
|
||||
//! [`marqdo_ir::call_binding::CallBindingTable`]. The runtime consumes that
|
||||
//! table and never re-resolves a name.
|
||||
//!
|
||||
//! Resolution priority is explicit and order-independent:
|
||||
//!
|
||||
//! | call form | rule |
|
||||
//! |---|---|
|
||||
//! | `f(...)` | local variable (dynamic) → official alias/plugin-fixed builtin → module function (current namespace first, then canonical, then unique local) |
|
||||
//! | `alias.f(...)` | explicit import alias → builtin dotted prefix → exact plugin export |
|
||||
//! | `some_value.f(...)` | dynamic language value only |
|
||||
//!
|
||||
//! Import aliases that collide with a builtin dotted namespace or a visible
|
||||
//! plugin namespace are rejected with `ext.import_namespace_conflict`; declared
|
||||
//! plugin exports that collide with each other or with builtins are rejected
|
||||
//! when the registry is built. The previous "try builtin, then guess plugin at
|
||||
//! runtime" behaviour is gone.
|
||||
//!
|
||||
//! `validate_module` keeps the standalone frontend API (a single document with
|
||||
//! no program graph) for AST/conformance tests: unresolved imports are
|
||||
//! *deferred* there because only the composition root can bind a whole program.
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use marqdo_expression::{Arg, Expr, Ident};
|
||||
use marqdo_ir::{builtins, Module, ModuleId, Param, PlaceTarget, Program, SignatureKind, Stmt};
|
||||
use marqdo_ir::builtins;
|
||||
use marqdo_ir::call_binding::{BoundTarget, CallBindingTable, CallSiteKey};
|
||||
use marqdo_ir::callable::{CallableDescriptor, CallableRegistry};
|
||||
use marqdo_ir::{
|
||||
Function, FunctionId, Module, ModuleId, PlaceTarget, Program, SignatureKind, Stmt,
|
||||
};
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{codes, Diagnostic};
|
||||
|
||||
/// Signature information needed to validate a call.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CallableSignature {
|
||||
/// Parameter list.
|
||||
pub params: Vec<Param>,
|
||||
/// Explicit signatures allow positional arguments; inferred do not.
|
||||
pub kind: SignatureKind,
|
||||
}
|
||||
|
||||
/// Build the callable signature table for a module.
|
||||
pub fn signature_table(module: &Module) -> BTreeMap<String, CallableSignature> {
|
||||
/// Build the callable signature table for one module (diagnostics helper).
|
||||
pub fn signature_table(
|
||||
module: &Module,
|
||||
) -> BTreeMap<String, marqdo_ir::callable::CallableSignature> {
|
||||
let mut table = BTreeMap::new();
|
||||
for function in &module.functions {
|
||||
table.insert(
|
||||
function.id.name.clone(),
|
||||
CallableSignature {
|
||||
marqdo_ir::callable::CallableSignature {
|
||||
params: function.params.clone(),
|
||||
kind: function.signature,
|
||||
},
|
||||
@@ -36,10 +54,107 @@ pub fn signature_table(module: &Module) -> BTreeMap<String, CallableSignature> {
|
||||
table
|
||||
}
|
||||
|
||||
/// Validate every call in every function body of the module.
|
||||
/// Result of binding a whole program.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct BoundProgram {
|
||||
/// One deterministic binding per static call site.
|
||||
pub bindings: CallBindingTable,
|
||||
/// Call-resolution diagnostics, sorted later by the caller.
|
||||
pub diagnostics: Vec<(ModuleId, Diagnostic)>,
|
||||
}
|
||||
|
||||
/// Validate every call in a single module (standalone frontend entry).
|
||||
///
|
||||
/// Imports and plugin exports cannot be resolved without a program graph, so
|
||||
/// those call sites are deferred to the composition-root binding pass instead
|
||||
/// of producing false errors or a second priority order.
|
||||
pub fn validate_module(module: &Module) -> Vec<Diagnostic> {
|
||||
let table = signature_table(module);
|
||||
let mut program = Program::default();
|
||||
program.modules.insert(module.id.clone(), module.clone());
|
||||
program.main_module = Some(module.id.clone());
|
||||
let registry = CallableRegistry::try_from_program(&program)
|
||||
.expect("a single module cannot contain duplicate external declarations");
|
||||
bind_with(module, &program, ®istry, true)
|
||||
.diagnostics
|
||||
.into_iter()
|
||||
.map(|(_, diagnostic)| diagnostic)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Validate every call in a loaded program (tests and standalone frontend API).
|
||||
///
|
||||
/// The CLI composition root uses [`bind_program_calls`] and keeps the binding
|
||||
/// table; this wrapper keeps the historical API for frontend tests.
|
||||
pub fn validate_program(program: &Program) -> Vec<(ModuleId, Diagnostic)> {
|
||||
let registry = match CallableRegistry::try_from_program(program) {
|
||||
Ok(registry) => registry,
|
||||
Err(_) => CallableRegistry::from_program(program),
|
||||
};
|
||||
bind_program_calls_with(program, ®istry, false).diagnostics
|
||||
}
|
||||
|
||||
/// Resolve every call site in `program` against `registry` exactly once.
|
||||
///
|
||||
/// This is the only binding entry point used by the composition root. The
|
||||
/// returned table must be installed on [`Program::set_call_bindings`] before
|
||||
/// execution.
|
||||
pub fn bind_program_calls(program: &Program, registry: &CallableRegistry) -> BoundProgram {
|
||||
bind_program_calls_with(program, registry, false)
|
||||
}
|
||||
|
||||
fn bind_program_calls_with(
|
||||
program: &Program,
|
||||
registry: &CallableRegistry,
|
||||
standalone: bool,
|
||||
) -> BoundProgram {
|
||||
let mut result = BoundProgram::default();
|
||||
for module in program.modules.values() {
|
||||
let mut bound = bind_with(module, program, registry, standalone);
|
||||
// Conflict diagnostics are produced once per module import, before
|
||||
// any call site; keep them in front of call diagnostics.
|
||||
result.diagnostics.append(&mut bound.diagnostics);
|
||||
for (key, target) in bound.bindings.iter() {
|
||||
result.bindings.insert(key.clone(), target.clone());
|
||||
}
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
fn bind_with(
|
||||
module: &Module,
|
||||
program: &Program,
|
||||
registry: &CallableRegistry,
|
||||
standalone: bool,
|
||||
) -> BoundProgram {
|
||||
let mut conflicted = BTreeSet::new();
|
||||
let mut diagnostics = Vec::new();
|
||||
// Reject import aliases that collide with a fixed builtin dotted namespace
|
||||
// or a visible plugin namespace. `alias.f(...)` must have one target.
|
||||
for import in &module.imports {
|
||||
let builtin_conflict = registry.has_builtin_dotted_prefix(&import.alias);
|
||||
let plugin_conflict = registry.has_external_dotted_prefix(&import.alias);
|
||||
if builtin_conflict || plugin_conflict {
|
||||
conflicted.insert(import.alias.clone());
|
||||
let owner = if builtin_conflict {
|
||||
"a builtin dotted namespace"
|
||||
} else {
|
||||
"a declared plugin export namespace"
|
||||
};
|
||||
diagnostics.push((
|
||||
module.id.clone(),
|
||||
Diagnostic::error(
|
||||
codes::EXT_IMPORT_NAMESPACE_CONFLICT,
|
||||
Some(import.span),
|
||||
format!("import alias `{}` conflicts with {owner}", import.alias),
|
||||
)
|
||||
.with_hint(format!(
|
||||
"rename the import alias, for example `{}_lib`, so `{}.…` has one meaning",
|
||||
import.alias, import.alias
|
||||
)),
|
||||
));
|
||||
}
|
||||
}
|
||||
let mut bound = BoundProgram::default();
|
||||
for function in &module.functions {
|
||||
let mut locals: BTreeSet<String> = function
|
||||
.params
|
||||
@@ -47,73 +162,50 @@ pub fn validate_module(module: &Module) -> Vec<Diagnostic> {
|
||||
.map(|param| param.name.clone())
|
||||
.collect();
|
||||
collect_assigned(&function.body, &mut locals);
|
||||
let mut check = CallChecker {
|
||||
current_namespace: function.namespace.clone(),
|
||||
let mut binder = Binder {
|
||||
module,
|
||||
program,
|
||||
registry,
|
||||
function_id: function.id.clone(),
|
||||
namespace: function.namespace.clone(),
|
||||
locals: &locals,
|
||||
table: &table,
|
||||
imports: module
|
||||
.imports
|
||||
.iter()
|
||||
.map(|import| import.alias.clone())
|
||||
.collect(),
|
||||
module_id: module.id.clone(),
|
||||
program: None,
|
||||
cross_only: false,
|
||||
standalone,
|
||||
conflicted: &conflicted,
|
||||
bindings: CallBindingTable::default(),
|
||||
diagnostics: Vec::new(),
|
||||
};
|
||||
check.stmts(&function.body);
|
||||
diagnostics.extend(check.diagnostics);
|
||||
}
|
||||
diagnostics
|
||||
}
|
||||
|
||||
/// Validate cross-module and qualified-builtin calls once a whole program is
|
||||
/// loaded. Local calls are already validated by [`validate_module`].
|
||||
pub fn validate_program(program: &Program) -> Vec<(ModuleId, Diagnostic)> {
|
||||
let mut results = Vec::new();
|
||||
for module in program.modules.values() {
|
||||
let table = signature_table(module);
|
||||
for function in &module.functions {
|
||||
let mut locals: BTreeSet<String> = function
|
||||
.params
|
||||
.iter()
|
||||
.map(|param| param.name.clone())
|
||||
.collect();
|
||||
collect_assigned(&function.body, &mut locals);
|
||||
let mut check = CallChecker {
|
||||
current_namespace: function.namespace.clone(),
|
||||
locals: &locals,
|
||||
table: &table,
|
||||
imports: module
|
||||
.imports
|
||||
.iter()
|
||||
.map(|import| import.alias.clone())
|
||||
.collect(),
|
||||
module_id: module.id.clone(),
|
||||
program: Some(program),
|
||||
cross_only: true,
|
||||
diagnostics: Vec::new(),
|
||||
};
|
||||
check.stmts(&function.body);
|
||||
results.extend(
|
||||
check
|
||||
.diagnostics
|
||||
.into_iter()
|
||||
.map(|diagnostic| (module.id.clone(), diagnostic)),
|
||||
);
|
||||
for param in &function.params {
|
||||
// Signature defaults are compile-time expressions evaluated by the
|
||||
// runtime in the callee's own frame; bind their calls with the
|
||||
// callee identity so the runtime lookup matches.
|
||||
if let Some(default) = ¶m.default {
|
||||
binder.expr(default);
|
||||
}
|
||||
}
|
||||
binder.stmts(&function.body);
|
||||
for (key, target) in binder.bindings.iter() {
|
||||
bound.bindings.insert(key.clone(), target.clone());
|
||||
}
|
||||
diagnostics.extend(
|
||||
binder
|
||||
.diagnostics
|
||||
.into_iter()
|
||||
.map(|diagnostic| (module.id.clone(), diagnostic)),
|
||||
);
|
||||
}
|
||||
BoundProgram {
|
||||
bindings: bound.bindings,
|
||||
diagnostics,
|
||||
}
|
||||
results
|
||||
}
|
||||
|
||||
fn collect_assigned(stmts: &[Stmt], out: &mut BTreeSet<String>) {
|
||||
for stmt in stmts {
|
||||
match stmt {
|
||||
Stmt::Assign {
|
||||
target: PlaceTarget::Local { name, .. },
|
||||
..
|
||||
} => {
|
||||
out.insert(name.clone());
|
||||
Stmt::Assign { target, .. } => {
|
||||
if let PlaceTarget::Local { name, .. } = target {
|
||||
out.insert(name.clone());
|
||||
}
|
||||
}
|
||||
Stmt::Branch { arms, .. } => {
|
||||
for arm in arms {
|
||||
@@ -121,23 +213,25 @@ fn collect_assigned(stmts: &[Stmt], out: &mut BTreeSet<String>) {
|
||||
}
|
||||
}
|
||||
Stmt::Loop { body, .. } => collect_assigned(body, out),
|
||||
_ => {}
|
||||
Stmt::Eval { .. } | Stmt::Return { .. } => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct CallChecker<'a> {
|
||||
current_namespace: Option<String>,
|
||||
struct Binder<'a> {
|
||||
module: &'a Module,
|
||||
program: &'a Program,
|
||||
registry: &'a CallableRegistry,
|
||||
function_id: FunctionId,
|
||||
namespace: Option<String>,
|
||||
locals: &'a BTreeSet<String>,
|
||||
table: &'a BTreeMap<String, CallableSignature>,
|
||||
imports: BTreeSet<String>,
|
||||
module_id: ModuleId,
|
||||
program: Option<&'a Program>,
|
||||
cross_only: bool,
|
||||
standalone: bool,
|
||||
conflicted: &'a BTreeSet<String>,
|
||||
bindings: CallBindingTable,
|
||||
diagnostics: Vec<Diagnostic>,
|
||||
}
|
||||
|
||||
impl<'a> CallChecker<'a> {
|
||||
impl<'a> Binder<'a> {
|
||||
fn stmts(&mut self, stmts: &[Stmt]) {
|
||||
for stmt in stmts {
|
||||
match stmt {
|
||||
@@ -192,205 +286,309 @@ impl<'a> CallChecker<'a> {
|
||||
}
|
||||
}
|
||||
|
||||
fn bind(&mut self, span: Span, target: BoundTarget) {
|
||||
let key = CallSiteKey::new(self.function_id.clone(), span);
|
||||
self.bindings.insert(key, target);
|
||||
}
|
||||
|
||||
fn diagnostic(&mut self, code: &str, span: Span, message: String, hint: Option<String>) {
|
||||
let mut diagnostic = Diagnostic::error(code, Some(span), message);
|
||||
if let Some(hint) = hint {
|
||||
diagnostic = diagnostic.with_hint(hint);
|
||||
}
|
||||
self.diagnostics.push(diagnostic);
|
||||
}
|
||||
|
||||
fn call(&mut self, callee: &Expr, args: &[Arg], span: Span) {
|
||||
match callee {
|
||||
Expr::Ident(Ident { name, .. }) => {
|
||||
if self.cross_only {
|
||||
Expr::Ident(Ident {
|
||||
name,
|
||||
span: name_span,
|
||||
}) => {
|
||||
if self.locals.contains(name) {
|
||||
self.bind(span, BoundTarget::DynamicValue);
|
||||
return;
|
||||
}
|
||||
if let Some(target) = self.registry.alias_target(name).cloned() {
|
||||
if let Some(descriptor) = self.registry.builtin(name) {
|
||||
self.validate_signature(
|
||||
&descriptor.signature,
|
||||
args,
|
||||
name,
|
||||
span,
|
||||
*name_span,
|
||||
);
|
||||
}
|
||||
self.bind(span, BoundTarget::Marqdo { function: target });
|
||||
return;
|
||||
}
|
||||
if let Some(builtin) = builtins::find(name) {
|
||||
if let Err(message) = builtins::validate_args(builtin, args) {
|
||||
self.diagnostics.push(
|
||||
Diagnostic::error(codes::CALL_ARITY, Some(span), message)
|
||||
.with_hint("check the builtin parameter list"),
|
||||
self.diagnostic(
|
||||
codes::CALL_ARITY,
|
||||
span,
|
||||
message,
|
||||
Some("check the builtin parameter list".to_string()),
|
||||
);
|
||||
}
|
||||
self.bind(
|
||||
span,
|
||||
BoundTarget::Builtin {
|
||||
canonical_name: builtin.name.to_string(),
|
||||
},
|
||||
);
|
||||
return;
|
||||
}
|
||||
let name_span = match callee {
|
||||
Expr::Ident(Ident { span, .. }) => *span,
|
||||
_ => span,
|
||||
};
|
||||
if let Some(signature) = self.resolve(name) {
|
||||
self.validate_against(name, signature, args, span, name_span);
|
||||
return;
|
||||
match self.resolve_module_function(name) {
|
||||
Resolution::One(function) => {
|
||||
if let Some(target) = self.program.function(&function) {
|
||||
let signature = marqdo_ir::callable::CallableSignature {
|
||||
params: target.params.clone(),
|
||||
kind: target.signature,
|
||||
};
|
||||
self.validate_signature(&signature, args, name, span, *name_span);
|
||||
}
|
||||
self.bind(span, BoundTarget::Marqdo { function });
|
||||
}
|
||||
Resolution::Ambiguous => {
|
||||
self.diagnostic(
|
||||
codes::SYMBOL_UNKNOWN,
|
||||
*name_span,
|
||||
format!("ambiguous function name `{name}`"),
|
||||
Some(
|
||||
"qualify it with its namespace, for example `namespace.函数(...)`"
|
||||
.to_string(),
|
||||
),
|
||||
);
|
||||
}
|
||||
Resolution::None => {
|
||||
if self.standalone
|
||||
&& self
|
||||
.module
|
||||
.imports
|
||||
.iter()
|
||||
.any(|import| import.alias == *name)
|
||||
{
|
||||
// A local may hold import-related values in the
|
||||
// standalone frontend API; defer.
|
||||
return;
|
||||
}
|
||||
self.diagnostic(
|
||||
codes::SYMBOL_UNKNOWN,
|
||||
*name_span,
|
||||
format!("unknown function `{name}`"),
|
||||
Some(
|
||||
"define it as a level-2 heading, call a builtin, or load the \
|
||||
plugin that declares it"
|
||||
.to_string(),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
if self.locals.contains(name) || self.imports.contains(name) {
|
||||
// A local may hold a FunctionRef value; resolved at runtime.
|
||||
return;
|
||||
}
|
||||
self.diagnostics.push(
|
||||
Diagnostic::error(
|
||||
codes::SYMBOL_UNKNOWN,
|
||||
Some(name_span),
|
||||
format!("unknown function `{name}`"),
|
||||
)
|
||||
.with_hint("define it as a level-2 heading or call a builtin"),
|
||||
);
|
||||
}
|
||||
Expr::Member { base, name, .. } => {
|
||||
if let Expr::Ident(Ident { name: alias, .. }) = base.as_ref() {
|
||||
let qualified = format!("{}.{name}", alias);
|
||||
if let Some(builtin) = builtins::find(&qualified) {
|
||||
if let Err(message) = builtins::validate_args(builtin, args) {
|
||||
self.diagnostics.push(
|
||||
Diagnostic::error(codes::CALL_ARITY, Some(span), message)
|
||||
.with_hint("check the builtin parameter list"),
|
||||
if let Expr::Ident(Ident {
|
||||
name: alias,
|
||||
span: alias_span,
|
||||
}) = base.as_ref()
|
||||
{
|
||||
if self.locals.contains(alias) {
|
||||
self.bind(span, BoundTarget::DynamicValue);
|
||||
return;
|
||||
}
|
||||
if self.conflicted.contains(alias) {
|
||||
// The import conflict diagnostic already tells the user
|
||||
// that `alias.…` has no single meaning; do not cascade.
|
||||
return;
|
||||
}
|
||||
if let Some(target_module) =
|
||||
self.program.resolve_import(&self.module.id, alias).cloned()
|
||||
{
|
||||
if let Some(function) = self.registry.function_id(&target_module, name) {
|
||||
if let Some(target) = self.program.function(&function) {
|
||||
let signature = marqdo_ir::callable::CallableSignature {
|
||||
params: target.params.clone(),
|
||||
kind: target.signature,
|
||||
};
|
||||
self.validate_signature(
|
||||
&signature,
|
||||
args,
|
||||
&format!("{alias}.{name}"),
|
||||
span,
|
||||
span,
|
||||
);
|
||||
}
|
||||
self.bind(span, BoundTarget::Marqdo { function });
|
||||
} else {
|
||||
self.diagnostic(
|
||||
codes::SYMBOL_UNKNOWN,
|
||||
span,
|
||||
format!("module `{alias}` has no function `{name}`"),
|
||||
Some(format!(
|
||||
"check `inspect --signature` on the imported module `{alias}`"
|
||||
)),
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if self.imports.contains(alias) {
|
||||
if let Some(program) = self.program {
|
||||
if let Some(target_module) =
|
||||
program.resolve_import(&self.module_id, alias)
|
||||
{
|
||||
if let Some(target) = program.module(target_module) {
|
||||
if let Some(function) = target.function(name) {
|
||||
let signature = CallableSignature {
|
||||
params: function.params.clone(),
|
||||
kind: function.signature,
|
||||
};
|
||||
self.validate_against(
|
||||
&qualified, &signature, args, span, span,
|
||||
);
|
||||
return;
|
||||
}
|
||||
self.diagnostics.push(
|
||||
Diagnostic::error(
|
||||
codes::SYMBOL_UNKNOWN,
|
||||
Some(span),
|
||||
format!("module `{alias}` has no function `{name}`"),
|
||||
)
|
||||
.with_hint(
|
||||
"check `inspect --signature` on the imported module",
|
||||
),
|
||||
);
|
||||
return;
|
||||
let qualified = format!("{alias}.{name}");
|
||||
if let Some(builtin) = builtins::find(&qualified) {
|
||||
if let Err(message) = builtins::validate_args(builtin, args) {
|
||||
self.diagnostic(
|
||||
codes::CALL_ARITY,
|
||||
span,
|
||||
message,
|
||||
Some("check the builtin parameter list".to_string()),
|
||||
);
|
||||
}
|
||||
self.bind(
|
||||
span,
|
||||
BoundTarget::Builtin {
|
||||
canonical_name: builtin.name.to_string(),
|
||||
},
|
||||
);
|
||||
return;
|
||||
}
|
||||
if let Some(descriptor) = self.registry.external(&qualified) {
|
||||
self.validate_signature(
|
||||
&descriptor.signature,
|
||||
args,
|
||||
&qualified,
|
||||
span,
|
||||
span,
|
||||
);
|
||||
let param_names = descriptor
|
||||
.signature
|
||||
.params
|
||||
.iter()
|
||||
.map(|param| param.name.clone())
|
||||
.collect();
|
||||
let target = match &descriptor.origin {
|
||||
marqdo_ir::callable::CallableOrigin::AbiPlugin { package, export } => {
|
||||
BoundTarget::AbiPlugin {
|
||||
package: package.clone(),
|
||||
export: export.clone(),
|
||||
digest: descriptor
|
||||
.implementation_digest
|
||||
.clone()
|
||||
.unwrap_or_default(),
|
||||
param_names,
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => BoundTarget::DynamicValue,
|
||||
};
|
||||
self.bind(span, target);
|
||||
return;
|
||||
}
|
||||
// A declared ABI export is a static descriptor: validate
|
||||
// its signature here, without loading the shared library.
|
||||
if let Some(program) = self.program {
|
||||
if let Some(descriptor) = program.external_callable(&qualified) {
|
||||
if let Err(message) =
|
||||
descriptor.signature.validate_args(args, &qualified)
|
||||
{
|
||||
self.diagnostics.push(
|
||||
Diagnostic::error(codes::CALL_ARITY, Some(span), message)
|
||||
.with_hint("check the plugin manifest export signature"),
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if self.standalone {
|
||||
// Without a program graph an import or plugin export
|
||||
// cannot be distinguished statically; defer instead of
|
||||
// emitting a false error.
|
||||
return;
|
||||
}
|
||||
self.diagnostic(
|
||||
codes::SYMBOL_UNKNOWN,
|
||||
*alias_span,
|
||||
format!("unknown function or plugin export `{qualified}`"),
|
||||
Some(
|
||||
"declare the export in a plugin manifest and pass `--plugin` for \
|
||||
`run`, or fix the module import alias"
|
||||
.to_string(),
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
self.bind(span, BoundTarget::DynamicValue);
|
||||
self.expr(base);
|
||||
}
|
||||
other => self.expr(other),
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve(&self, name: &str) -> Option<&'a CallableSignature> {
|
||||
if let Some(namespace) = &self.current_namespace {
|
||||
if let Some(found) = self.table.get(&format!("{namespace}.{name}")) {
|
||||
return Some(found);
|
||||
other => {
|
||||
self.bind(span, BoundTarget::DynamicValue);
|
||||
self.expr(other);
|
||||
}
|
||||
}
|
||||
if let Some(found) = self.table.get(name) {
|
||||
return Some(found);
|
||||
}
|
||||
// Unique local name across namespaces.
|
||||
let mut matches = self
|
||||
.table
|
||||
.iter()
|
||||
.filter(|(key, _)| key.ends_with(&format!(".{name}")))
|
||||
.map(|(_, signature)| signature);
|
||||
let first = matches.next()?;
|
||||
if matches.next().is_some() {
|
||||
None
|
||||
} else {
|
||||
Some(first)
|
||||
}
|
||||
}
|
||||
|
||||
fn validate_against(
|
||||
fn validate_signature(
|
||||
&mut self,
|
||||
name: &str,
|
||||
signature: &CallableSignature,
|
||||
signature: &marqdo_ir::callable::CallableSignature,
|
||||
args: &[Arg],
|
||||
display: &str,
|
||||
call_span: Span,
|
||||
name_span: Span,
|
||||
) {
|
||||
if signature.kind == SignatureKind::Inferred {
|
||||
if let Some(positional) = args.iter().find(|arg| arg.name.is_none()) {
|
||||
self.diagnostics.push(
|
||||
Diagnostic::error(
|
||||
codes::CALL_ARITY,
|
||||
Some(positional.span),
|
||||
format!(
|
||||
"`{name}` has an inferred signature and only accepts named arguments"
|
||||
),
|
||||
)
|
||||
.with_hint("pass arguments as `name = value`"),
|
||||
);
|
||||
if let Err(message) = signature.validate_args(args, display) {
|
||||
// Positional overflow and missing arguments point at the call or
|
||||
// the function name respectively; keep messages deterministic.
|
||||
let span = if message.contains("missing required") {
|
||||
name_span
|
||||
} else {
|
||||
call_span
|
||||
};
|
||||
self.diagnostic(
|
||||
codes::CALL_ARITY,
|
||||
span,
|
||||
message,
|
||||
Some("check `inspect --signature` or the plugin manifest signature".to_string()),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_module_function(&self, name: &str) -> Resolution {
|
||||
if let Some(namespace) = &self.namespace {
|
||||
let canonical = format!("{namespace}.{name}");
|
||||
if let Some(function) = self
|
||||
.module
|
||||
.functions
|
||||
.iter()
|
||||
.find(|function| function.id.name == canonical)
|
||||
{
|
||||
return Resolution::One(function.id.clone());
|
||||
}
|
||||
}
|
||||
let mut supplied: BTreeSet<&str> = BTreeSet::new();
|
||||
let mut positional = 0usize;
|
||||
for arg in args {
|
||||
match &arg.name {
|
||||
Some(argument_name) => {
|
||||
if !signature
|
||||
.params
|
||||
.iter()
|
||||
.any(|param| param.name == *argument_name)
|
||||
{
|
||||
self.diagnostics.push(
|
||||
Diagnostic::error(
|
||||
codes::CALL_ARITY,
|
||||
Some(arg.span),
|
||||
format!("`{name}` has no parameter named `{argument_name}`"),
|
||||
)
|
||||
.with_hint("check `inspect --signature`"),
|
||||
);
|
||||
}
|
||||
supplied.insert(argument_name.as_str());
|
||||
}
|
||||
None => {
|
||||
if let Some(param) = signature.params.get(positional) {
|
||||
supplied.insert(param.name.as_str());
|
||||
positional += 1;
|
||||
} else {
|
||||
self.diagnostics.push(Diagnostic::error(
|
||||
codes::CALL_ARITY,
|
||||
Some(call_span),
|
||||
format!(
|
||||
"`{name}` accepts at most {} positional argument(s)",
|
||||
signature.params.len()
|
||||
),
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(function) = self
|
||||
.module
|
||||
.functions
|
||||
.iter()
|
||||
.find(|function| function.id.name == name)
|
||||
{
|
||||
return Resolution::One(function.id.clone());
|
||||
}
|
||||
for param in &signature.params {
|
||||
if param.default.is_none() && !supplied.contains(param.name.as_str()) {
|
||||
self.diagnostics.push(
|
||||
Diagnostic::error(
|
||||
codes::CALL_ARITY,
|
||||
Some(name_span),
|
||||
format!("`{name}` is missing required argument `{}`", param.name),
|
||||
)
|
||||
.with_hint("pass it by name, e.g. `x = value`"),
|
||||
);
|
||||
}
|
||||
let mut matches = self
|
||||
.module
|
||||
.functions
|
||||
.iter()
|
||||
.filter(|function| function.local_name == name);
|
||||
match (matches.next(), matches.next()) {
|
||||
(Some(function), None) => Resolution::One(function.id.clone()),
|
||||
(None, _) => Resolution::None,
|
||||
(Some(_), Some(_)) => Resolution::Ambiguous,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
enum Resolution {
|
||||
One(FunctionId),
|
||||
Ambiguous,
|
||||
None,
|
||||
}
|
||||
|
||||
/// Whether a descriptor is a statically declared ABI plugin export.
|
||||
pub fn is_abi_export(descriptor: &CallableDescriptor) -> bool {
|
||||
matches!(
|
||||
descriptor.origin,
|
||||
marqdo_ir::callable::CallableOrigin::AbiPlugin { .. }
|
||||
)
|
||||
}
|
||||
|
||||
/// Whether a function signature requires explicit named/positional checking.
|
||||
pub fn _signature_kind(signature: &marqdo_ir::callable::CallableSignature) -> SignatureKind {
|
||||
signature.kind
|
||||
}
|
||||
|
||||
/// Helper used by tests and by `inspect` to render a function body trace.
|
||||
#[allow(dead_code)]
|
||||
fn _unused(_: &Function, _: &Program) {}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -494,11 +692,36 @@ mod tests {
|
||||
package: "demo".into(),
|
||||
export: "solver.add".into(),
|
||||
},
|
||||
implementation_digest: None,
|
||||
implementation_digest: Some("a".repeat(64)),
|
||||
source_span: None,
|
||||
});
|
||||
let diagnostics = validate_program(&program);
|
||||
assert!(diagnostics.iter().any(|(_, d)| d.code == codes::CALL_ARITY));
|
||||
let registry = CallableRegistry::try_from_program(&program).unwrap();
|
||||
let bound = bind_program_calls(&program, ®istry);
|
||||
assert!(bound
|
||||
.diagnostics
|
||||
.iter()
|
||||
.any(|(_, d)| d.code == codes::CALL_ARITY));
|
||||
// A valid positional/named mix binds to the exact export identity.
|
||||
let module = make_module(vec![function(
|
||||
"main",
|
||||
&[],
|
||||
SignatureKind::Explicit,
|
||||
vec![return_call("solver.add(1, b=2)")],
|
||||
)]);
|
||||
program.modules.insert(module.id.clone(), module);
|
||||
let registry = CallableRegistry::try_from_program(&program).unwrap();
|
||||
let bound = bind_program_calls(&program, ®istry);
|
||||
assert!(bound.diagnostics.is_empty(), "{:?}", bound.diagnostics);
|
||||
let target = bound.bindings.iter().next().unwrap().1;
|
||||
match target {
|
||||
BoundTarget::AbiPlugin {
|
||||
package, export, ..
|
||||
} => {
|
||||
assert_eq!(package, "demo");
|
||||
assert_eq!(export, "solver.add");
|
||||
}
|
||||
other => panic!("unexpected target {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -556,4 +779,31 @@ mod tests {
|
||||
.iter()
|
||||
.any(|d| d.code == codes::CALL_ARITY));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn import_alias_conflicting_with_builtin_namespace_is_rejected() {
|
||||
let module = Module {
|
||||
id: ModuleId::from("index.mq.md"),
|
||||
units: vec![],
|
||||
functions: vec![function(
|
||||
"main",
|
||||
&[],
|
||||
SignatureKind::Explicit,
|
||||
vec![return_call("math.eval(x=1)")],
|
||||
)],
|
||||
imports: vec![marqdo_ir::ImportDecl {
|
||||
alias: "math".into(),
|
||||
target: "./math.mq.md".into(),
|
||||
span: Span::new(0, 1),
|
||||
}],
|
||||
resources: vec![],
|
||||
anchors: std::collections::BTreeMap::new(),
|
||||
};
|
||||
let diagnostics = validate_module(&module);
|
||||
assert!(diagnostics
|
||||
.iter()
|
||||
.any(|d| d.code == codes::EXT_IMPORT_NAMESPACE_CONFLICT));
|
||||
// Standalone mode defers unresolved imports, but the composition root
|
||||
// run is covered by the CLI black-box CALL-01 test.
|
||||
}
|
||||
}
|
||||
@@ -14,5 +14,5 @@ pub mod resources;
|
||||
pub mod signature;
|
||||
pub mod units;
|
||||
|
||||
pub use analyze::{analyze_source, AnalysisResult};
|
||||
pub use analyze::{analyze_source, analyze_source_for_program, AnalysisResult};
|
||||
pub use units::{build_units, github_slug, FunctionScope, HeadingInfo, UnitIndex};
|
||||
@@ -1,12 +1,19 @@
|
||||
//! Capability model.
|
||||
//! Capability model and the trusted capability-handler registry.
|
||||
//!
|
||||
//! A project manifest can *request* capabilities, but requests are not grants.
|
||||
//! Effective permissions are the host default for the sandbox mode intersected
|
||||
//! with what the operator/CLI explicitly granted. Every host operation checks
|
||||
//! against this set at the point of execution.
|
||||
//!
|
||||
//! Executable vendor/core capabilities additionally need an explicitly
|
||||
//! registered trusted [`CapabilityHandler`] (ADR-0057, RFC-EXT-0001 §7.2).
|
||||
//! Being authorized is never the same as being executable: a request without a
|
||||
//! handler fails with `capability.unavailable` instead of returning an empty
|
||||
//! successful reply.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::Instant;
|
||||
|
||||
use glob::Pattern;
|
||||
use marqdo_plugin_abi::abi::MAX_CAPABILITY_REPLY_BYTES;
|
||||
@@ -14,8 +21,33 @@ use marqdo_plugin_abi::PluginError;
|
||||
use marqdo_span::Span;
|
||||
use marqdo_value::{ErrorCategory, RuntimeError};
|
||||
|
||||
use crate::budget::Budget;
|
||||
use crate::fs::ProjectFs;
|
||||
|
||||
/// Core capability names understood by [`CapabilityBroker::decide`].
|
||||
///
|
||||
/// This is the closed set of host primitives (ADR-0011). A trusted embedder
|
||||
/// may install a [`CapabilityHandler`] for one of these names; until it does,
|
||||
/// the capability is authorized by policy but *unavailable* at execution.
|
||||
pub const CORE_CAPABILITIES: [&str; 10] = [
|
||||
"console",
|
||||
"math.compute",
|
||||
"fs.read",
|
||||
"fs.write",
|
||||
"net.request",
|
||||
"process",
|
||||
"clock",
|
||||
"random",
|
||||
"log",
|
||||
"artifact.record",
|
||||
];
|
||||
|
||||
/// Capability namespace prefixes reserved for the host itself.
|
||||
///
|
||||
/// Handler registration rejects these prefixes so a vendor package can never
|
||||
/// shadow a host-internal operation.
|
||||
pub const RESERVED_HANDLER_PREFIXES: [&str; 2] = ["marqdo.", "host."];
|
||||
|
||||
/// One audited capability request from a plugin.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct CapabilityAudit {
|
||||
@@ -27,19 +59,143 @@ pub struct CapabilityAudit {
|
||||
pub decision: &'static str,
|
||||
/// Human-readable reason.
|
||||
pub reason: String,
|
||||
/// Wall-clock duration of the decision/execution in microseconds.
|
||||
pub duration_micros: u64,
|
||||
}
|
||||
|
||||
/// Broker for plugin host-capability requests (ADR-0011).
|
||||
/// One capability request handed to an installed [`CapabilityHandler`].
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct CapabilityRequest<'a> {
|
||||
/// Trusted package identity supplied by the host, never by plugin wire data.
|
||||
pub package_instance: &'a str,
|
||||
/// Capability being requested.
|
||||
pub capability: &'a str,
|
||||
/// Raw request payload (JSON for the predefined capabilities).
|
||||
pub payload: &'a [u8],
|
||||
/// Run/session trace id used to correlate audit entries with a run.
|
||||
pub trace_id: &'a str,
|
||||
}
|
||||
|
||||
/// Trusted, explicitly registered executor for one capability.
|
||||
///
|
||||
/// Core capabilities are checked against the effective grant set. Vendor
|
||||
/// capabilities are accepted only when a trusted host policy registered the
|
||||
/// exact name; a declaration in a manifest is never enough.
|
||||
#[derive(Debug, Clone)]
|
||||
/// RFC-EXT-0001 §7.2 names the policy parameter `EffectiveGrant` and the budget
|
||||
/// `ResourceBudget`; this build deliberately reuses the existing
|
||||
/// [`Capabilities`] grant set and [`Budget`] instead of introducing a second
|
||||
/// permission system (the RFC explicitly allows that substitution).
|
||||
///
|
||||
/// A handler is only reachable after [`CapabilityBroker::decide`] authorizes
|
||||
/// the request, but it must still verify its own scope (project-relative paths,
|
||||
/// globs, per-file limits) and must charge work to `budget` where applicable.
|
||||
/// Registration is host-only: there is no wire path from a plugin to this
|
||||
/// registry.
|
||||
pub trait CapabilityHandler: Send {
|
||||
/// Stable capability name this handler serves.
|
||||
fn name(&self) -> &str;
|
||||
/// Execute one already-authorized request.
|
||||
fn execute(
|
||||
&mut self,
|
||||
request: &CapabilityRequest<'_>,
|
||||
capabilities: &Capabilities,
|
||||
budget: &mut Budget,
|
||||
) -> Result<Vec<u8>, PluginError>;
|
||||
}
|
||||
|
||||
/// Built-in handler for granted `fs.read` requests.
|
||||
///
|
||||
/// Reads go through [`ProjectFs`] (root confinement, traversal rejection,
|
||||
/// regular-file/UTF-8/8 MiB limits) and the grant is checked **again** against
|
||||
/// the normalized project-relative path at the execution point, so a raw
|
||||
/// payload cannot win the glob check with a path the filesystem later
|
||||
/// normalizes differently.
|
||||
pub struct FsReadHandler {
|
||||
fs: ProjectFs,
|
||||
}
|
||||
|
||||
impl FsReadHandler {
|
||||
/// Install the built-in handler over one project filesystem view.
|
||||
pub fn new(fs: ProjectFs) -> Self {
|
||||
Self { fs }
|
||||
}
|
||||
}
|
||||
|
||||
impl CapabilityHandler for FsReadHandler {
|
||||
fn name(&self) -> &str {
|
||||
"fs.read"
|
||||
}
|
||||
|
||||
fn execute(
|
||||
&mut self,
|
||||
request: &CapabilityRequest<'_>,
|
||||
capabilities: &Capabilities,
|
||||
_budget: &mut Budget,
|
||||
) -> Result<Vec<u8>, PluginError> {
|
||||
// Reply-size limits are enforced by the broker, so this handler keeps
|
||||
// the historical per-read behavior and only needs the grant re-check.
|
||||
let raw_path = request_path(request.payload)?;
|
||||
let (_, relative) = self.fs.resolve(&raw_path, None).map_err(map_runtime)?;
|
||||
capabilities
|
||||
.check_read(&relative, None)
|
||||
.map_err(map_runtime)?;
|
||||
let contents = self.fs.read(&raw_path, None).map_err(map_runtime)?;
|
||||
Ok(contents.into_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
struct HandlerEntry {
|
||||
handler: Box<dyn CapabilityHandler>,
|
||||
/// Exact trusted package binding. `None` means the handler is authorized
|
||||
/// by the capability grant alone (for example project-scoped `fs.read`).
|
||||
package: Option<String>,
|
||||
}
|
||||
|
||||
/// Broker for plugin host-capability requests (ADR-0011, ADR-0057).
|
||||
///
|
||||
/// The broker keeps two independent sets: trusted vendor *policy* entries and
|
||||
/// installed *handlers*. Execution is
|
||||
///
|
||||
/// ```text
|
||||
/// decide (grant + policy + payload shape)
|
||||
/// -> handler installed? (else capability.unavailable)
|
||||
/// -> package-bound handler owns the claimed package? (else permission.denied)
|
||||
/// -> handler executes under the broker budget and reply limit
|
||||
/// -> audit { package, capability, granted|denied|unavailable, reason, duration }
|
||||
/// ```
|
||||
///
|
||||
/// A capability that is merely authorized never executes: there is no
|
||||
/// empty-success path.
|
||||
pub struct CapabilityBroker {
|
||||
capabilities: Capabilities,
|
||||
vendor_policy: BTreeSet<String>,
|
||||
handlers: BTreeMap<String, HandlerEntry>,
|
||||
reply_limit: usize,
|
||||
budget: Budget,
|
||||
trace_id: String,
|
||||
audit: Arc<Mutex<Vec<CapabilityAudit>>>,
|
||||
fs: Option<ProjectFs>,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for CapabilityBroker {
|
||||
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
let bound: Vec<(&str, &str)> = self
|
||||
.handlers
|
||||
.iter()
|
||||
.filter_map(|(name, entry)| {
|
||||
entry
|
||||
.package
|
||||
.as_deref()
|
||||
.map(|package| (name.as_str(), package))
|
||||
})
|
||||
.collect();
|
||||
formatter
|
||||
.debug_struct("CapabilityBroker")
|
||||
.field("capabilities", &self.capabilities)
|
||||
.field("vendor_policy", &self.vendor_policy)
|
||||
.field("handlers", &self.handlers.keys().collect::<Vec<_>>())
|
||||
.field("package_bound_handlers", &bound)
|
||||
.field("reply_limit", &self.reply_limit)
|
||||
.field("budget", &self.budget)
|
||||
.field("trace_id", &self.trace_id)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
impl CapabilityBroker {
|
||||
@@ -53,19 +209,116 @@ impl CapabilityBroker {
|
||||
Self {
|
||||
capabilities,
|
||||
vendor_policy: BTreeSet::new(),
|
||||
handlers: BTreeMap::new(),
|
||||
reply_limit: MAX_CAPABILITY_REPLY_BYTES,
|
||||
budget: Budget::default(),
|
||||
trace_id: String::new(),
|
||||
audit,
|
||||
fs: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Attach the project filesystem used to execute granted `fs.read`
|
||||
/// requests. Without it, `fs.read` is authorized but unavailable.
|
||||
/// Attach the project filesystem and install the trusted built-in
|
||||
/// `fs.read` handler.
|
||||
///
|
||||
/// Without an attached filesystem, granted `fs.read` requests stay
|
||||
/// authorized but have no handler, so execution returns
|
||||
/// `capability.unavailable` (never an empty success).
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics if a handler for `fs.read` was already registered on this broker,
|
||||
/// which would make the built-in handler ambiguous.
|
||||
pub fn with_fs(mut self, fs: ProjectFs) -> Self {
|
||||
self.fs = Some(fs);
|
||||
self.register_handler(FsReadHandler::new(fs))
|
||||
.expect("a fresh broker has no fs.read handler to conflict with");
|
||||
self
|
||||
}
|
||||
|
||||
/// Override the maximum reply size a handler may return.
|
||||
pub fn with_reply_limit(mut self, bytes: usize) -> Self {
|
||||
self.reply_limit = bytes;
|
||||
self
|
||||
}
|
||||
|
||||
/// Attach the per-run budget that handlers charge work to.
|
||||
pub fn with_budget(mut self, budget: Budget) -> Self {
|
||||
self.budget = budget;
|
||||
self
|
||||
}
|
||||
|
||||
/// Set the run/session trace id surfaced to handlers.
|
||||
pub fn with_trace_id(mut self, trace_id: impl Into<String>) -> Self {
|
||||
self.trace_id = trace_id.into();
|
||||
self
|
||||
}
|
||||
|
||||
/// Remaining broker budget (diagnostics and tests).
|
||||
pub fn budget(&self) -> &Budget {
|
||||
&self.budget
|
||||
}
|
||||
|
||||
/// Install a trusted handler reachable by any package that holds a grant.
|
||||
///
|
||||
/// Registration is a host-only operation; a plugin manifest cannot reach
|
||||
/// this API. Duplicate, reserved and empty/invalid names are rejected.
|
||||
pub fn register_handler<H>(&mut self, handler: H) -> Result<(), PluginError>
|
||||
where
|
||||
H: CapabilityHandler + 'static,
|
||||
{
|
||||
self.install_handler(handler, None)
|
||||
}
|
||||
|
||||
/// Install a trusted handler bound to one exact trusted package identity.
|
||||
///
|
||||
/// A request that claims any other package is `permission.denied`, even
|
||||
/// when the capability grant and vendor policy match.
|
||||
pub fn register_package_handler<H>(
|
||||
&mut self,
|
||||
package: impl Into<String>,
|
||||
handler: H,
|
||||
) -> Result<(), PluginError>
|
||||
where
|
||||
H: CapabilityHandler + 'static,
|
||||
{
|
||||
self.install_handler(handler, Some(package.into()))
|
||||
}
|
||||
|
||||
fn install_handler<H>(&mut self, handler: H, package: Option<String>) -> Result<(), PluginError>
|
||||
where
|
||||
H: CapabilityHandler + 'static,
|
||||
{
|
||||
let name = handler.name().to_string();
|
||||
validate_handler_name(&name)?;
|
||||
if self.handlers.contains_key(&name) {
|
||||
return Err(PluginError::capability(
|
||||
"capability.handler_conflict",
|
||||
format!("a capability handler is already registered for `{name}`"),
|
||||
));
|
||||
}
|
||||
self.handlers.insert(
|
||||
name,
|
||||
HandlerEntry {
|
||||
handler: Box::new(handler),
|
||||
package,
|
||||
},
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Names with an installed trusted handler, in deterministic order.
|
||||
pub fn handler_names(&self) -> Vec<&str> {
|
||||
self.handlers.keys().map(String::as_str).collect()
|
||||
}
|
||||
|
||||
/// Whether a trusted executing handler is installed for `capability`.
|
||||
pub fn has_handler(&self, capability: &str) -> bool {
|
||||
self.handlers.contains_key(capability)
|
||||
}
|
||||
|
||||
/// Register a trusted vendor capability policy entry.
|
||||
///
|
||||
/// This is policy only: execution additionally requires an installed
|
||||
/// handler. Registering a vendor name does not install anything.
|
||||
pub fn allow_vendor(&mut self, capability: impl Into<String>) {
|
||||
self.vendor_policy.insert(capability.into());
|
||||
}
|
||||
@@ -76,103 +329,132 @@ impl CapabilityBroker {
|
||||
}
|
||||
|
||||
/// Authorize one request, returning reply bytes on success.
|
||||
///
|
||||
/// This is the grant decision only; it does not prove that an executor is
|
||||
/// installed. Call [`Self::execute`] to actually run a capability.
|
||||
pub fn authorize(
|
||||
&mut self,
|
||||
package: &str,
|
||||
capability: &str,
|
||||
request: &[u8],
|
||||
) -> Result<Vec<u8>, PluginError> {
|
||||
let started = Instant::now();
|
||||
let result = self.decide(package, capability, request);
|
||||
let (decision, reason) = match &result {
|
||||
Ok(()) => ("granted", String::new()),
|
||||
Err(error) => ("denied", format!("{}: {}", error.code, error.message)),
|
||||
};
|
||||
if let Ok(mut audit) = self.audit.lock() {
|
||||
audit.push(CapabilityAudit {
|
||||
package: package.to_string(),
|
||||
capability: capability.to_string(),
|
||||
decision,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
self.record(package, capability, decision, &reason, started);
|
||||
result.map(|()| Vec::new())
|
||||
}
|
||||
|
||||
/// Authorize and execute one capability request.
|
||||
/// Authorize and execute one capability request through its trusted
|
||||
/// handler.
|
||||
///
|
||||
/// Only `fs.read` has a real implementation in this build. Other
|
||||
/// capabilities return `capability.unavailable` instead of pretending an
|
||||
/// empty reply means success (ADR-0011, spec §12.2).
|
||||
/// The chain is fixed: `decide` → handler installed → package binding →
|
||||
/// handler execution under the broker budget and reply limit → audit.
|
||||
/// Anything without an installed handler is `capability.unavailable`; it
|
||||
/// is never reported as an empty success (ADR-0011, ADR-0057).
|
||||
pub fn execute(
|
||||
&mut self,
|
||||
package: &str,
|
||||
capability: &str,
|
||||
request: &[u8],
|
||||
) -> Result<Vec<u8>, PluginError> {
|
||||
let started = Instant::now();
|
||||
|
||||
// 1. Authorization: effective grants, vendor policy, payload shape.
|
||||
if let Err(error) = self.decide(package, capability, request) {
|
||||
let reason = format!("{}: {}", error.code, error.message);
|
||||
self.record(package, capability, "denied", &reason);
|
||||
self.record(package, capability, "denied", &reason, started);
|
||||
return Err(error);
|
||||
}
|
||||
let result = match capability {
|
||||
"fs.read" => {
|
||||
let path = request_path(request)?;
|
||||
let Some(fs) = &self.fs else {
|
||||
return self.unavailable(
|
||||
package,
|
||||
capability,
|
||||
"no project filesystem is attached",
|
||||
);
|
||||
};
|
||||
match fs.read(&path, None) {
|
||||
Ok(contents) => {
|
||||
if contents.len() > MAX_CAPABILITY_REPLY_BYTES {
|
||||
return self.unavailable(
|
||||
package,
|
||||
capability,
|
||||
"the file exceeds the capability reply limit",
|
||||
);
|
||||
}
|
||||
Ok(contents.into_bytes())
|
||||
}
|
||||
Err(error) => {
|
||||
let error = map_runtime(error);
|
||||
let reason = format!("{}: {}", error.code, error.message);
|
||||
self.record(package, capability, "denied", &reason);
|
||||
Err(error)
|
||||
}
|
||||
}
|
||||
|
||||
// 2. A trusted executor must exist; a package-bound executor must own
|
||||
// the trusted package identity the host is claiming.
|
||||
let bound_package = match self.handlers.get(capability) {
|
||||
Some(entry) => entry.package.clone(),
|
||||
None => {
|
||||
let message = "this capability has no executing handler in this build";
|
||||
self.record(package, capability, "unavailable", message, started);
|
||||
return Err(PluginError::capability(
|
||||
"capability.unavailable",
|
||||
message.to_string(),
|
||||
));
|
||||
}
|
||||
_ => self.unavailable(
|
||||
package,
|
||||
capability,
|
||||
"this capability has no executing handler in this build",
|
||||
),
|
||||
};
|
||||
if result.is_ok() {
|
||||
self.record(package, capability, "granted", "");
|
||||
if let Some(bound) = bound_package {
|
||||
if bound != package {
|
||||
let error = plugin_denied(&format!(
|
||||
"capability `{capability}` is bound to another trusted package"
|
||||
));
|
||||
let reason = format!("{}: {}", error.code, error.message);
|
||||
self.record(package, capability, "denied", &reason, started);
|
||||
return Err(error);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Execute under the broker budget and reply limit.
|
||||
let broker_request = CapabilityRequest {
|
||||
package_instance: package,
|
||||
capability,
|
||||
payload: request,
|
||||
trace_id: &self.trace_id,
|
||||
};
|
||||
let reply_limit = self.reply_limit;
|
||||
let outcome = self
|
||||
.handlers
|
||||
.get_mut(capability)
|
||||
.expect("handler presence was checked above")
|
||||
.handler
|
||||
.execute(&broker_request, &self.capabilities, &mut self.budget);
|
||||
match outcome {
|
||||
Ok(contents) => {
|
||||
if contents.len() > reply_limit {
|
||||
let error = PluginError::capability(
|
||||
"resource.exhausted",
|
||||
format!(
|
||||
"capability reply is {} bytes; the limit is {reply_limit} bytes",
|
||||
contents.len()
|
||||
),
|
||||
);
|
||||
let reason = format!("{}: {}", error.code, error.message);
|
||||
self.record(package, capability, "denied", &reason, started);
|
||||
return Err(error);
|
||||
}
|
||||
self.record(package, capability, "granted", "", started);
|
||||
Ok(contents)
|
||||
}
|
||||
Err(error) => {
|
||||
let reason = format!("{}: {}", error.code, error.message);
|
||||
let decision = if error.code == "capability.unavailable" {
|
||||
"unavailable"
|
||||
} else {
|
||||
"denied"
|
||||
};
|
||||
self.record(package, capability, decision, &reason, started);
|
||||
Err(error)
|
||||
}
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
fn unavailable<T>(
|
||||
fn record(
|
||||
&mut self,
|
||||
package: &str,
|
||||
capability: &str,
|
||||
message: &str,
|
||||
) -> Result<T, PluginError> {
|
||||
let error = PluginError::capability("capability.unavailable", message.to_string());
|
||||
self.record(package, capability, "unavailable", message);
|
||||
Err(error)
|
||||
}
|
||||
|
||||
fn record(&mut self, package: &str, capability: &str, decision: &'static str, reason: &str) {
|
||||
decision: &'static str,
|
||||
reason: &str,
|
||||
started: Instant,
|
||||
) {
|
||||
let elapsed = started.elapsed();
|
||||
let duration_micros = u64::try_from(elapsed.as_micros()).unwrap_or(u64::MAX);
|
||||
if let Ok(mut audit) = self.audit.lock() {
|
||||
audit.push(CapabilityAudit {
|
||||
package: package.to_string(),
|
||||
capability: capability.to_string(),
|
||||
decision,
|
||||
reason: reason.to_string(),
|
||||
duration_micros,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -214,17 +496,14 @@ impl CapabilityBroker {
|
||||
.then_some(())
|
||||
.ok_or_else(|| plugin_denied("process capability was not granted")),
|
||||
other => {
|
||||
if is_vendor_capability(other)
|
||||
&& (self.vendor_policy.contains(other) || self.vendor_policy.contains("*"))
|
||||
{
|
||||
Ok(())
|
||||
} else if is_vendor_capability(other) {
|
||||
Err(PluginError::capability(
|
||||
"capability.unavailable",
|
||||
format!(
|
||||
"vendor capability `{other}` has no trusted handler for package `{package}`"
|
||||
),
|
||||
))
|
||||
if is_vendor_capability(other) {
|
||||
if self.vendor_policy.contains(other) || self.vendor_policy.contains("*") {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(plugin_denied(&format!(
|
||||
"vendor capability `{other}` has no trusted policy entry for package `{package}`"
|
||||
)))
|
||||
}
|
||||
} else {
|
||||
Err(PluginError::capability(
|
||||
"capability.unknown",
|
||||
@@ -304,6 +583,38 @@ pub fn is_vendor_capability(name: &str) -> bool {
|
||||
})
|
||||
}
|
||||
|
||||
/// Validate a trusted handler name before registration.
|
||||
///
|
||||
/// Registration uses [`CapabilityHandler::name`] as the registry key, so there
|
||||
/// is no second name argument that could disagree with the handler's declared
|
||||
/// capability (the RFC's "declared protocol mismatch" rejection).
|
||||
fn validate_handler_name(name: &str) -> Result<(), PluginError> {
|
||||
if name.is_empty() || name.trim() != name {
|
||||
return Err(PluginError::capability(
|
||||
"capability.handler_invalid",
|
||||
"capability handler names must be non-empty and must not contain surrounding whitespace",
|
||||
));
|
||||
}
|
||||
if RESERVED_HANDLER_PREFIXES
|
||||
.iter()
|
||||
.any(|prefix| name.starts_with(prefix))
|
||||
{
|
||||
return Err(PluginError::capability(
|
||||
"capability.handler_reserved",
|
||||
format!("capability handler name `{name}` uses a reserved host namespace"),
|
||||
));
|
||||
}
|
||||
if !CORE_CAPABILITIES.contains(&name) && !is_vendor_capability(name) {
|
||||
return Err(PluginError::capability(
|
||||
"capability.handler_invalid",
|
||||
format!(
|
||||
"`{name}` is not a registrable capability name: expected a core capability or a vendor-namespaced name"
|
||||
),
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Effective capability grants for one run.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct Capabilities {
|
||||
@@ -594,20 +905,56 @@ mod tests {
|
||||
.unwrap_err();
|
||||
assert_eq!(denied.code, "permission.denied");
|
||||
assert!(broker.authorize("demo.solver", "fs.read", b"{}").is_err());
|
||||
let unavailable = broker
|
||||
// Declaring a vendor capability is not a grant: without a trusted
|
||||
// policy entry authorization is denied (RFC-EXT-0001 CAP-01).
|
||||
let vendor_denied = broker
|
||||
.authorize("demo.solver", "vendor.device.read", b"{}")
|
||||
.unwrap_err();
|
||||
assert_eq!(unavailable.code, "capability.unavailable");
|
||||
assert_eq!(vendor_denied.code, "permission.denied");
|
||||
broker.allow_vendor("vendor.device.read");
|
||||
assert!(broker
|
||||
.authorize("demo.solver", "vendor.device.read", b"{}")
|
||||
.is_ok());
|
||||
// Policy is not a handler: authorize() succeeds but execution still
|
||||
// needs an executor and reports `capability.unavailable`.
|
||||
assert!(!broker.has_handler("vendor.device.read"));
|
||||
assert_eq!(
|
||||
broker
|
||||
.execute("demo.solver", "vendor.device.read", b"{}")
|
||||
.unwrap_err()
|
||||
.code,
|
||||
"capability.unavailable"
|
||||
);
|
||||
assert!(!is_vendor_capability("vendor"));
|
||||
assert!(!is_vendor_capability("fs.read.extra"));
|
||||
let audit = broker.audit();
|
||||
let audit = audit.lock().unwrap();
|
||||
assert_eq!(audit.len(), 5);
|
||||
assert_eq!(audit.len(), 6);
|
||||
assert!(audit.iter().any(|entry| entry.decision == "denied"));
|
||||
assert!(audit.iter().any(|entry| entry.decision == "unavailable"));
|
||||
assert!(audit.iter().all(|entry| entry.duration_micros <= 5_000_000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_core_capability_is_understood_by_decide() {
|
||||
for name in CORE_CAPABILITIES {
|
||||
let broker = CapabilityBroker::new(Capabilities::project());
|
||||
let payload: &[u8] = match name {
|
||||
"fs.read" => br#"{"path":"data/a.csv"}"#,
|
||||
"fs.write" => br#"{"path":"data/a.csv","contents":"x"}"#,
|
||||
"net.request" => br#"{"host":"127.0.0.1"}"#,
|
||||
_ => b"{}",
|
||||
};
|
||||
let error = broker
|
||||
.decide("demo.solver", name, payload)
|
||||
.err()
|
||||
.map(|error| error.code);
|
||||
assert_ne!(
|
||||
error.as_deref(),
|
||||
Some("capability.unknown"),
|
||||
"core capability `{name}` must be handled by decide"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -638,6 +985,9 @@ mod execute_tests {
|
||||
.allow_read("data/**")
|
||||
.allow_write("data/**");
|
||||
let mut broker = CapabilityBroker::new(capabilities).with_fs(fs);
|
||||
assert!(broker.has_handler("fs.read"));
|
||||
assert_eq!(broker.handler_names(), vec!["fs.read"]);
|
||||
assert_eq!(broker.reply_limit, MAX_CAPABILITY_REPLY_BYTES);
|
||||
|
||||
let contents = broker
|
||||
.execute("demo.solver", "fs.read", br#"{"path":"data/allowed.txt"}"#)
|
||||
|
||||
@@ -19,7 +19,10 @@ pub mod resource;
|
||||
pub mod trace;
|
||||
|
||||
pub use budget::{Budget, DEFAULT_FUEL, DEFAULT_OUTPUT_BYTES, DEFAULT_STACK_LIMIT};
|
||||
pub use capability::Capabilities;
|
||||
pub use capability::{
|
||||
Capabilities, CapabilityAudit, CapabilityBroker, CapabilityHandler, CapabilityRequest,
|
||||
FsReadHandler, CORE_CAPABILITIES, RESERVED_HANDLER_PREFIXES,
|
||||
};
|
||||
pub use context::ExecutionContext;
|
||||
pub use fs::ProjectFs;
|
||||
pub use handles::{contains_plugin_handle, PluginHandleTable};
|
||||
|
||||
@@ -0,0 +1,581 @@
|
||||
//! Trusted capability-handler registry (RFC-EXT-0001 §7, ADR-0057).
|
||||
//!
|
||||
//! These are end-to-end tests over the public `marqdo-host` API. They cover the
|
||||
//! CAP-01..CAP-12 checks listed in RFC-EXT-0001 §7.5: policy vs handler,
|
||||
//! package binding, registration validation, real `fs.read` scope enforcement,
|
||||
//! reply budgets, forged/replayed handles, audit hygiene, resource-stream
|
||||
//! regression, project-vs-strict labeling and the "no handler is never a
|
||||
//! silent OK" rule.
|
||||
|
||||
use std::sync::atomic::{AtomicU32, Ordering};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use marqdo_host::{
|
||||
contains_plugin_handle, Budget, Capabilities, CapabilityBroker, CapabilityHandler,
|
||||
CapabilityRequest, PluginHandleTable, ProjectFs, ResourceTable, SandboxMode, StdHost,
|
||||
CORE_CAPABILITIES,
|
||||
};
|
||||
use marqdo_plugin_abi::PluginError;
|
||||
use marqdo_value::{RuntimeError, Value};
|
||||
|
||||
static COUNTER: AtomicU32 = AtomicU32::new(0);
|
||||
|
||||
struct TempDir(std::path::PathBuf);
|
||||
|
||||
impl TempDir {
|
||||
fn new() -> Self {
|
||||
let id = COUNTER.fetch_add(1, Ordering::Relaxed);
|
||||
let path = std::env::temp_dir().join(format!(
|
||||
"marqdo-capability-handlers-{}-{id}",
|
||||
std::process::id()
|
||||
));
|
||||
std::fs::create_dir_all(&path).unwrap();
|
||||
Self(path)
|
||||
}
|
||||
|
||||
fn path(&self) -> &std::path::Path {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TempDir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
/// A trusted test handler that returns canned bytes.
|
||||
struct FixedHandler {
|
||||
name: &'static str,
|
||||
reply: Vec<u8>,
|
||||
}
|
||||
|
||||
impl CapabilityHandler for FixedHandler {
|
||||
fn name(&self) -> &str {
|
||||
self.name
|
||||
}
|
||||
|
||||
fn execute(
|
||||
&mut self,
|
||||
_request: &CapabilityRequest<'_>,
|
||||
_capabilities: &Capabilities,
|
||||
_budget: &mut Budget,
|
||||
) -> Result<Vec<u8>, PluginError> {
|
||||
Ok(self.reply.clone())
|
||||
}
|
||||
}
|
||||
|
||||
/// A trusted test handler that proves the request identity and budget reach it.
|
||||
struct EchoHandler {
|
||||
name: &'static str,
|
||||
fuel: u64,
|
||||
}
|
||||
|
||||
impl CapabilityHandler for EchoHandler {
|
||||
fn name(&self) -> &str {
|
||||
self.name
|
||||
}
|
||||
|
||||
fn execute(
|
||||
&mut self,
|
||||
request: &CapabilityRequest<'_>,
|
||||
_capabilities: &Capabilities,
|
||||
budget: &mut Budget,
|
||||
) -> Result<Vec<u8>, PluginError> {
|
||||
budget
|
||||
.consume_fuel(self.fuel, None)
|
||||
.map_err(map_runtime_error)?;
|
||||
Ok(format!(
|
||||
"{}|{}|{}",
|
||||
request.package_instance, request.capability, request.trace_id
|
||||
)
|
||||
.into_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
fn map_runtime_error(error: RuntimeError) -> PluginError {
|
||||
PluginError::capability(error.code(), error.message)
|
||||
}
|
||||
|
||||
fn audit_entries(broker: &CapabilityBroker) -> Vec<marqdo_host::CapabilityAudit> {
|
||||
let audit = broker.audit();
|
||||
let guard = audit.lock().expect("audit lock");
|
||||
guard.clone()
|
||||
}
|
||||
|
||||
/// CAP-01: a declared vendor capability without a trusted policy entry is
|
||||
/// denied, even when a handler happens to be installed.
|
||||
#[test]
|
||||
fn cap_01_vendor_capability_without_policy_is_denied() {
|
||||
let mut broker = CapabilityBroker::new(Capabilities::project());
|
||||
broker
|
||||
.register_handler(FixedHandler {
|
||||
name: "vendor.device.read",
|
||||
reply: b"device".to_vec(),
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
let error = broker
|
||||
.execute("demo.solver", "vendor.device.read", b"{}")
|
||||
.unwrap_err();
|
||||
assert_eq!(error.code, "permission.denied", "{error:?}");
|
||||
assert!(broker
|
||||
.authorize("demo.solver", "vendor.device.read", b"{}")
|
||||
.is_err());
|
||||
|
||||
let audit = audit_entries(&broker);
|
||||
assert_eq!(audit.len(), 2);
|
||||
assert!(audit.iter().all(|entry| entry.decision == "denied"));
|
||||
assert!(audit
|
||||
.iter()
|
||||
.all(|entry| entry.capability == "vendor.device.read"));
|
||||
}
|
||||
|
||||
/// CAP-02: authorized but no handler installed is `capability.unavailable`;
|
||||
/// `authorize` alone does not execute and cannot fake an empty success.
|
||||
#[test]
|
||||
fn cap_02_authorized_capability_without_handler_is_unavailable() {
|
||||
let mut broker = CapabilityBroker::new(Capabilities::project().allow_write("out/**"));
|
||||
broker.allow_vendor("vendor.device.read");
|
||||
|
||||
let unavailable = broker
|
||||
.execute("demo.solver", "vendor.device.read", b"{}")
|
||||
.unwrap_err();
|
||||
assert_eq!(
|
||||
unavailable.code, "capability.unavailable",
|
||||
"{unavailable:?}"
|
||||
);
|
||||
|
||||
// Policy is a grant, not an executor: authorize() succeeds, execute() still
|
||||
// refuses because no trusted handler was registered.
|
||||
assert!(broker
|
||||
.authorize(
|
||||
"demo.solver",
|
||||
"fs.write",
|
||||
br#"{"path":"out/result.txt","contents":"x"}"#,
|
||||
)
|
||||
.is_ok());
|
||||
let write = broker
|
||||
.execute(
|
||||
"demo.solver",
|
||||
"fs.write",
|
||||
br#"{"path":"out/result.txt","contents":"x"}"#,
|
||||
)
|
||||
.unwrap_err();
|
||||
assert_eq!(write.code, "capability.unavailable", "{write:?}");
|
||||
|
||||
let audit = audit_entries(&broker);
|
||||
assert!(audit
|
||||
.iter()
|
||||
.any(|entry| entry.decision == "unavailable" && entry.capability == "fs.write"));
|
||||
assert!(audit
|
||||
.iter()
|
||||
.any(|entry| entry.decision == "granted" && entry.capability == "fs.write"));
|
||||
}
|
||||
|
||||
/// CAP-03: a handler bound to one trusted package rejects cross-package
|
||||
/// impersonation, while the bound package still executes and the request
|
||||
/// identity/trace id reach the handler.
|
||||
#[test]
|
||||
fn cap_03_package_bound_handler_rejects_impersonation() {
|
||||
let mut broker = CapabilityBroker::new(Capabilities::project()).with_trace_id("trace-9");
|
||||
broker.allow_vendor("vendor.device.read");
|
||||
broker
|
||||
.register_package_handler(
|
||||
"demo.solver",
|
||||
EchoHandler {
|
||||
name: "vendor.device.read",
|
||||
fuel: 0,
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let spoofed = broker
|
||||
.execute("attacker.solver", "vendor.device.read", b"{}")
|
||||
.unwrap_err();
|
||||
assert_eq!(spoofed.code, "permission.denied", "{spoofed:?}");
|
||||
|
||||
let reply = broker
|
||||
.execute("demo.solver", "vendor.device.read", b"{}")
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
String::from_utf8(reply).unwrap(),
|
||||
"demo.solver|vendor.device.read|trace-9"
|
||||
);
|
||||
|
||||
let audit = audit_entries(&broker);
|
||||
assert!(audit
|
||||
.iter()
|
||||
.any(|entry| entry.decision == "denied" && entry.package == "attacker.solver"));
|
||||
assert!(audit
|
||||
.iter()
|
||||
.any(|entry| entry.decision == "granted" && entry.package == "demo.solver"));
|
||||
}
|
||||
|
||||
/// CAP-04: duplicate, reserved, empty and malformed handler names are rejected
|
||||
/// and never installed.
|
||||
#[test]
|
||||
fn cap_04_registration_rejects_duplicate_reserved_and_empty_names() {
|
||||
let mut broker = CapabilityBroker::new(Capabilities::project());
|
||||
let rejected = |broker: &mut CapabilityBroker, name: &'static str, expected: &str| {
|
||||
let error = broker
|
||||
.register_handler(FixedHandler {
|
||||
name,
|
||||
reply: Vec::new(),
|
||||
})
|
||||
.unwrap_err();
|
||||
assert_eq!(error.code, expected, "name `{name}`: {error:?}");
|
||||
assert!(!broker.has_handler(name));
|
||||
};
|
||||
|
||||
rejected(&mut broker, "", "capability.handler_invalid");
|
||||
rejected(&mut broker, "host.terminal", "capability.handler_reserved");
|
||||
rejected(
|
||||
&mut broker,
|
||||
"marqdo.internal.read",
|
||||
"capability.handler_reserved",
|
||||
);
|
||||
rejected(
|
||||
&mut broker,
|
||||
"not a capability",
|
||||
"capability.handler_invalid",
|
||||
);
|
||||
rejected(&mut broker, "vendor..read", "capability.handler_invalid");
|
||||
|
||||
broker
|
||||
.register_handler(FixedHandler {
|
||||
name: "vendor.device.read",
|
||||
reply: b"device".to_vec(),
|
||||
})
|
||||
.unwrap();
|
||||
let duplicate = broker
|
||||
.register_handler(FixedHandler {
|
||||
name: "vendor.device.read",
|
||||
reply: b"other".to_vec(),
|
||||
})
|
||||
.unwrap_err();
|
||||
assert_eq!(
|
||||
duplicate.code, "capability.handler_conflict",
|
||||
"{duplicate:?}"
|
||||
);
|
||||
|
||||
// The trusted built-in fs.read handler cannot be replaced either.
|
||||
let dir = TempDir::new();
|
||||
let mut with_fs =
|
||||
CapabilityBroker::new(Capabilities::project()).with_fs(ProjectFs::new(dir.path()));
|
||||
assert!(with_fs.has_handler("fs.read"));
|
||||
let conflict = with_fs
|
||||
.register_handler(FixedHandler {
|
||||
name: "fs.read",
|
||||
reply: Vec::new(),
|
||||
})
|
||||
.unwrap_err();
|
||||
assert_eq!(conflict.code, "capability.handler_conflict", "{conflict:?}");
|
||||
}
|
||||
|
||||
/// CAP-05: the real `fs.read` handler enforces ProjectFs root/scope limits; a
|
||||
/// permissive glob grant cannot be used to read outside the project root or to
|
||||
/// traverse with `..`.
|
||||
#[test]
|
||||
fn cap_05_fs_read_handler_enforces_scope() {
|
||||
let dir = TempDir::new();
|
||||
std::fs::create_dir_all(dir.path().join("data")).unwrap();
|
||||
std::fs::write(dir.path().join("data/allowed.txt"), "allowed").unwrap();
|
||||
std::fs::create_dir_all(dir.path().join("private")).unwrap();
|
||||
std::fs::write(dir.path().join("private/secret.txt"), "secret").unwrap();
|
||||
|
||||
let mut broker = CapabilityBroker::new(Capabilities::project().allow_read("data/**"))
|
||||
.with_fs(ProjectFs::new(dir.path()));
|
||||
let allowed = broker
|
||||
.execute("demo.solver", "fs.read", br#"{"path":"data/allowed.txt"}"#)
|
||||
.unwrap();
|
||||
assert_eq!(String::from_utf8(allowed).unwrap(), "allowed");
|
||||
|
||||
for out_of_scope in [
|
||||
br#"{"path":"private/secret.txt"}"#.as_slice(),
|
||||
br#"{"path":"../escape.txt"}"#.as_slice(),
|
||||
br#"{"path":"/etc/passwd"}"#.as_slice(),
|
||||
] {
|
||||
let error = broker
|
||||
.execute("demo.solver", "fs.read", out_of_scope)
|
||||
.unwrap_err();
|
||||
assert_eq!(
|
||||
error.code, "permission.denied",
|
||||
"{out_of_scope:?}: {error:?}"
|
||||
);
|
||||
}
|
||||
|
||||
// Even a `**` grant only authorizes project-relative paths: traversal and
|
||||
// absolute paths are rejected by ProjectFs::resolve at the handler.
|
||||
let mut permissive = CapabilityBroker::new(Capabilities::project().allow_read("**"))
|
||||
.with_fs(ProjectFs::new(dir.path()));
|
||||
for rejected in [
|
||||
br#"{"path":"data/../private/secret.txt"}"#.as_slice(),
|
||||
br#"{"path":"/etc/passwd"}"#.as_slice(),
|
||||
] {
|
||||
let error = permissive
|
||||
.execute("demo.solver", "fs.read", rejected)
|
||||
.unwrap_err();
|
||||
assert_eq!(error.code, "permission.denied", "{rejected:?}: {error:?}");
|
||||
}
|
||||
}
|
||||
|
||||
/// CAP-06: a handler reply over the broker budget is a resource limit, and the
|
||||
/// handler receives (and charges) the broker budget.
|
||||
#[test]
|
||||
fn cap_06_handler_reply_over_budget_is_resource_limit() {
|
||||
let mut broker = CapabilityBroker::new(Capabilities::project())
|
||||
.with_reply_limit(64)
|
||||
.with_budget(Budget {
|
||||
fuel: 10,
|
||||
..Budget::default()
|
||||
});
|
||||
broker.allow_vendor("vendor.device.big");
|
||||
broker.allow_vendor("vendor.device.fuel");
|
||||
broker
|
||||
.register_handler(FixedHandler {
|
||||
name: "vendor.device.big",
|
||||
reply: vec![b'x'; 65],
|
||||
})
|
||||
.unwrap();
|
||||
broker
|
||||
.register_handler(EchoHandler {
|
||||
name: "vendor.device.fuel",
|
||||
fuel: 4,
|
||||
})
|
||||
.unwrap();
|
||||
|
||||
let oversized = broker
|
||||
.execute("demo.solver", "vendor.device.big", b"{}")
|
||||
.unwrap_err();
|
||||
assert_eq!(oversized.code, "resource.exhausted", "{oversized:?}");
|
||||
|
||||
let reply = broker
|
||||
.execute("demo.solver", "vendor.device.fuel", b"{}")
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
String::from_utf8(reply).unwrap(),
|
||||
"demo.solver|vendor.device.fuel|"
|
||||
);
|
||||
assert_eq!(broker.budget().remaining_fuel(), 6);
|
||||
|
||||
let audit = audit_entries(&broker);
|
||||
assert!(audit.iter().all(|entry| entry.duration_micros <= 5_000_000));
|
||||
assert!(audit
|
||||
.iter()
|
||||
.any(|entry| entry.capability == "vendor.device.big" && entry.decision == "denied"));
|
||||
}
|
||||
|
||||
/// CAP-07: forged plugin handles are rejected by the host table, and a handle
|
||||
/// hidden inside an untrusted extension value is detected.
|
||||
#[test]
|
||||
fn cap_07_forged_handles_are_rejected() {
|
||||
let mut table = PluginHandleTable::default();
|
||||
let handle = table.mint("demo.capability", "capability.read", "demo.capability", 1);
|
||||
table
|
||||
.validate(&handle, "demo.capability", 1)
|
||||
.expect("host-minted handle is valid");
|
||||
|
||||
let mut forged = handle.clone();
|
||||
forged.id = handle.id.wrapping_add(1000);
|
||||
let error = table
|
||||
.validate(&forged, "demo.capability", 1)
|
||||
.expect_err("forged id must be rejected");
|
||||
assert!(
|
||||
error.message.contains("plugin_handle.untrusted"),
|
||||
"{error:?}"
|
||||
);
|
||||
|
||||
let wrong_owner = table
|
||||
.validate(&handle, "other.package", 1)
|
||||
.expect_err("cross-owner handle must be rejected");
|
||||
assert!(
|
||||
wrong_owner.message.contains("plugin_handle.untrusted"),
|
||||
"{wrong_owner:?}"
|
||||
);
|
||||
|
||||
let untrusted_value = Value::Map(
|
||||
[("handle".to_string(), Value::PluginHandle(handle.clone()))]
|
||||
.into_iter()
|
||||
.collect(),
|
||||
);
|
||||
assert!(contains_plugin_handle(&untrusted_value));
|
||||
}
|
||||
|
||||
/// CAP-08: a handle from an old run or a cleared (unloaded) run is rejected on
|
||||
/// replay.
|
||||
#[test]
|
||||
fn cap_08_replayed_handle_after_run_end_is_rejected() {
|
||||
let mut table = PluginHandleTable::default();
|
||||
let handle = table.mint("demo.capability", "capability.read", "demo.capability", 7);
|
||||
table
|
||||
.validate(&handle, "demo.capability", 7)
|
||||
.expect("current run accepts its own handle");
|
||||
|
||||
let new_run = table
|
||||
.validate(&handle, "demo.capability", 8)
|
||||
.expect_err("a new run must not accept the old handle");
|
||||
assert!(
|
||||
new_run.message.contains("plugin_handle.untrusted"),
|
||||
"{new_run:?}"
|
||||
);
|
||||
|
||||
table.clear_run(7);
|
||||
assert!(table.is_empty());
|
||||
let replayed = table
|
||||
.validate(&handle, "demo.capability", 7)
|
||||
.expect_err("a cleared run must reject the replayed handle");
|
||||
assert!(
|
||||
replayed.message.contains("plugin_handle.untrusted"),
|
||||
"{replayed:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// CAP-09: the capability audit records decisions and durations but never file
|
||||
/// contents or request secrets.
|
||||
#[test]
|
||||
fn cap_09_audit_contains_no_file_contents_or_secrets() {
|
||||
const SECRET: &str = "TOP-SECRET-PAYLOAD";
|
||||
const API_KEY: &str = "sk-live-not-audited";
|
||||
let dir = TempDir::new();
|
||||
std::fs::create_dir_all(dir.path().join("data")).unwrap();
|
||||
std::fs::write(dir.path().join("data/secret.txt"), SECRET).unwrap();
|
||||
|
||||
let mut broker = CapabilityBroker::new(Capabilities::project().allow_read("data/**"))
|
||||
.with_fs(ProjectFs::new(dir.path()));
|
||||
let request = format!(r#"{{"path":"data/secret.txt","api_key":"{API_KEY}"}}"#);
|
||||
let contents = broker
|
||||
.execute("demo.solver", "fs.read", request.as_bytes())
|
||||
.unwrap();
|
||||
assert_eq!(String::from_utf8(contents).unwrap(), SECRET);
|
||||
|
||||
let denied = broker
|
||||
.execute("demo.solver", "fs.read", br#"{"path":"private/key.txt"}"#)
|
||||
.unwrap_err();
|
||||
assert_eq!(denied.code, "permission.denied");
|
||||
|
||||
let audit_debug = format!("{:?}", audit_entries(&broker));
|
||||
assert!(!audit_debug.contains(SECRET), "{audit_debug}");
|
||||
assert!(!audit_debug.contains(API_KEY), "{audit_debug}");
|
||||
|
||||
let audit = audit_entries(&broker);
|
||||
assert!(audit
|
||||
.iter()
|
||||
.any(|entry| entry.decision == "granted" && entry.capability == "fs.read"));
|
||||
assert!(audit
|
||||
.iter()
|
||||
.any(|entry| entry.decision == "denied" && entry.capability == "fs.read"));
|
||||
assert!(audit.iter().all(|entry| entry.duration_micros <= 5_000_000));
|
||||
}
|
||||
|
||||
/// CAP-10: low-level resource open/read/write/close stays on a sane performance
|
||||
/// path. This is a catastrophic-regression guard with a deliberately generous
|
||||
/// bound, not a benchmark (CI timeouts remain authoritative).
|
||||
#[test]
|
||||
fn cap_10_resource_stream_operations_do_not_regress_catastrophically() {
|
||||
const CHUNK: usize = 256 * 1024;
|
||||
const ITERATIONS: usize = 32;
|
||||
let payload = vec![b'x'; CHUNK];
|
||||
let mut table = ResourceTable::default();
|
||||
|
||||
let started = Instant::now();
|
||||
for _ in 0..ITERATIONS {
|
||||
let reader = table.open_file("data/blob.bin", payload.clone());
|
||||
let read = table.read(reader, payload.len(), None).unwrap();
|
||||
assert_eq!(read.bytes.len(), CHUNK);
|
||||
assert!(read.eof);
|
||||
table.close(reader, None).unwrap();
|
||||
|
||||
let writer = table.open_file_writer("out/blob.bin");
|
||||
assert_eq!(table.write(writer, &payload, None).unwrap(), CHUNK);
|
||||
table.close(writer, None).unwrap();
|
||||
}
|
||||
let elapsed = started.elapsed();
|
||||
// 32 x (256 KiB read + 256 KiB write) is ~16 MiB of in-memory copying.
|
||||
assert!(
|
||||
elapsed < Duration::from_secs(5),
|
||||
"resource open/read/write/close took {elapsed:?} for {ITERATIONS} iterations"
|
||||
);
|
||||
}
|
||||
|
||||
/// CAP-11: the default `project` mode is not an OS-strict mode and grants
|
||||
/// nothing beyond `pure` by default.
|
||||
#[test]
|
||||
fn cap_11_default_project_mode_is_not_os_strict() {
|
||||
let dir = TempDir::new();
|
||||
let host = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
|
||||
assert_eq!(host.mode, SandboxMode::Project);
|
||||
assert_ne!(host.mode, SandboxMode::Strict);
|
||||
assert_eq!(host.mode.as_str(), "project");
|
||||
|
||||
// Project defaults equal pure defaults: no implicit fs/process/plugin grant.
|
||||
assert!(host.capabilities.fs_read.is_empty());
|
||||
assert!(host.capabilities.fs_write.is_empty());
|
||||
assert!(!host.capabilities.process);
|
||||
assert!(!host.capabilities.foreign);
|
||||
assert!(host.capabilities.plugins.is_empty());
|
||||
|
||||
// Strict isolation availability is a property of `strict`; project mode is
|
||||
// cooperative in-process checks, whether or not the platform could isolate.
|
||||
assert_eq!(
|
||||
SandboxMode::parse(host.mode.as_str()),
|
||||
Some(SandboxMode::Project)
|
||||
);
|
||||
}
|
||||
|
||||
/// CAP-12: core capabilities without an executing handler are always
|
||||
/// `capability.unavailable`, never a silent empty success.
|
||||
#[test]
|
||||
fn cap_12_core_capabilities_without_handler_are_unavailable() {
|
||||
let dir = TempDir::new();
|
||||
std::fs::create_dir_all(dir.path().join("data")).unwrap();
|
||||
std::fs::write(dir.path().join("data/ok.txt"), "ok").unwrap();
|
||||
|
||||
let mut capabilities = Capabilities::project()
|
||||
.allow_read("data/**")
|
||||
.allow_write("data/**")
|
||||
.allow_net("127.0.0.1")
|
||||
.allow_process("/bin/true");
|
||||
capabilities.process = true;
|
||||
let mut broker = CapabilityBroker::new(capabilities).with_fs(ProjectFs::new(dir.path()));
|
||||
|
||||
// The one migrated capability has a real handler.
|
||||
let read = broker
|
||||
.execute("demo.solver", "fs.read", br#"{"path":"data/ok.txt"}"#)
|
||||
.unwrap();
|
||||
assert_eq!(String::from_utf8(read).unwrap(), "ok");
|
||||
|
||||
// Every other core capability is authorized by its grant but has no
|
||||
// executor in this build.
|
||||
for name in CORE_CAPABILITIES {
|
||||
if name == "fs.read" {
|
||||
continue;
|
||||
}
|
||||
let payload: &[u8] = match name {
|
||||
"fs.write" => br#"{"path":"data/x","contents":"x"}"#,
|
||||
"net.request" => br#"{"host":"127.0.0.1"}"#,
|
||||
_ => b"{}",
|
||||
};
|
||||
let error = broker.execute("demo.solver", name, payload).unwrap_err();
|
||||
assert_eq!(
|
||||
error.code, "capability.unavailable",
|
||||
"core capability `{name}` must not silently succeed: {error:?}"
|
||||
);
|
||||
assert!(!broker.has_handler(name));
|
||||
}
|
||||
|
||||
let unknown = broker
|
||||
.execute("demo.solver", "teleport", b"{}")
|
||||
.unwrap_err();
|
||||
assert_eq!(unknown.code, "capability.unknown", "{unknown:?}");
|
||||
|
||||
let audit = audit_entries(&broker);
|
||||
assert!(
|
||||
audit
|
||||
.iter()
|
||||
.filter(|entry| entry.decision == "unavailable")
|
||||
.count()
|
||||
>= CORE_CAPABILITIES.len() - 1,
|
||||
"{audit:?}"
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,201 @@
|
||||
//! Resolved static call bindings (RFC-EXT-0001 §4.2, ADR-0054).
|
||||
//!
|
||||
//! The frontend resolves every statically visible call site exactly once and
|
||||
//! records the decision here. The runtime only dispatches by this table: it
|
||||
//! never re-resolves a qualified name and never guesses a plugin from a string
|
||||
//! prefix. The table is internal IR (not part of Analysis/Trace v2 wire
|
||||
//! contracts) and is deliberately skipped when a [`crate::Program`] is
|
||||
//! serialized.
|
||||
//!
|
||||
//! Identity rules:
|
||||
//!
|
||||
//! * a [`CallSiteKey`] is `(module, function, byte span)` and is therefore
|
||||
//! independent of disk paths, registration order and hash-map iteration;
|
||||
//! * a bound target names the *resolved implementation*, including the plugin
|
||||
//! package, complete export name and verified artifact digest for ABI calls;
|
||||
//! * [`BoundTarget::DynamicValue`] is the only non-static path and may only
|
||||
//! call language values (for example a `FunctionRef`); it is never a back
|
||||
//! door to an unknown ABI export.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::fmt;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::{FunctionId, ModuleId};
|
||||
|
||||
/// Stable identity of one static call site.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
pub struct CallSiteKey {
|
||||
/// Module containing the call.
|
||||
pub module: ModuleId,
|
||||
/// Function containing the call.
|
||||
pub function: FunctionId,
|
||||
/// First byte offset of the call expression.
|
||||
pub span_start: usize,
|
||||
/// End byte offset (exclusive) of the call expression.
|
||||
pub span_end: usize,
|
||||
}
|
||||
|
||||
impl CallSiteKey {
|
||||
/// Construct a key from a function identity and an expression span.
|
||||
pub fn new(function: FunctionId, span: marqdo_span::Span) -> Self {
|
||||
Self {
|
||||
module: function.module.clone(),
|
||||
function,
|
||||
span_start: span.start,
|
||||
span_end: span.end,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Display for CallSiteKey {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
write!(
|
||||
f,
|
||||
"{}@{}..{}",
|
||||
self.function, self.span_start, self.span_end
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolved implementation target for a call site.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||
pub enum BoundTarget {
|
||||
/// A fixed language builtin, by canonical builtin name.
|
||||
Builtin {
|
||||
/// Canonical name (for example `math.eval`), not a user alias.
|
||||
canonical_name: String,
|
||||
},
|
||||
/// A function defined by a Markdown heading.
|
||||
Marqdo {
|
||||
/// Exact function identity.
|
||||
function: FunctionId,
|
||||
},
|
||||
/// A statically declared ABI plugin export.
|
||||
AbiPlugin {
|
||||
/// Plugin package that owns the export.
|
||||
package: String,
|
||||
/// Complete dotted export name.
|
||||
export: String,
|
||||
/// Verified artifact digest from the selected manifest artifact.
|
||||
digest: String,
|
||||
/// Declared parameter names in manifest order; used to normalise
|
||||
/// positional arguments deterministically for the ABI v1 map wire.
|
||||
param_names: Vec<String>,
|
||||
},
|
||||
/// The callee is a language value resolved at runtime (local variable or
|
||||
/// an expression result). Only language callables such as `FunctionRef`
|
||||
/// are valid; this path cannot invoke an ABI plugin.
|
||||
DynamicValue,
|
||||
}
|
||||
|
||||
impl BoundTarget {
|
||||
/// Whether this binding names a statically known implementation.
|
||||
pub fn is_static(&self) -> bool {
|
||||
!matches!(self, BoundTarget::DynamicValue)
|
||||
}
|
||||
}
|
||||
|
||||
/// One deterministic binding table for a prepared program.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct CallBindingTable {
|
||||
sites: BTreeMap<CallSiteKey, BoundTarget>,
|
||||
}
|
||||
|
||||
impl CallBindingTable {
|
||||
/// Insert a binding. Returns `false` when the site was already bound to a
|
||||
/// different target; re-binding the same target is idempotent.
|
||||
pub fn insert(&mut self, key: CallSiteKey, target: BoundTarget) -> bool {
|
||||
match self.sites.get(&key) {
|
||||
Some(existing) => existing == &target,
|
||||
None => {
|
||||
self.sites.insert(key, target);
|
||||
true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Look up the bound target for a call site.
|
||||
pub fn get(&self, key: &CallSiteKey) -> Option<&BoundTarget> {
|
||||
self.sites.get(key)
|
||||
}
|
||||
|
||||
/// Number of bound call sites.
|
||||
pub fn len(&self) -> usize {
|
||||
self.sites.len()
|
||||
}
|
||||
|
||||
/// Whether no call site is bound.
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.sites.is_empty()
|
||||
}
|
||||
|
||||
/// Iterate bindings deterministically (call site order).
|
||||
pub fn iter(&self) -> impl Iterator<Item = (&CallSiteKey, &BoundTarget)> {
|
||||
self.sites.iter()
|
||||
}
|
||||
|
||||
/// Number of statically bound (non-dynamic) call sites.
|
||||
pub fn static_count(&self) -> usize {
|
||||
self.sites
|
||||
.values()
|
||||
.filter(|target| target.is_static())
|
||||
.count()
|
||||
}
|
||||
|
||||
/// Number of dynamically resolved call sites.
|
||||
pub fn dynamic_count(&self) -> usize {
|
||||
self.sites.len() - self.static_count()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use marqdo_span::Span;
|
||||
|
||||
fn function(name: &str) -> FunctionId {
|
||||
FunctionId {
|
||||
module: ModuleId::root_default("index.mq.md"),
|
||||
name: name.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
fn key(name: &str, span: Span) -> CallSiteKey {
|
||||
CallSiteKey::new(function(name), span)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn keys_are_deterministic_and_span_based() {
|
||||
let left = key("f", Span::new(10, 20));
|
||||
let right = key("f", Span::new(10, 20));
|
||||
let other = key("f", Span::new(21, 30));
|
||||
assert_eq!(left, right);
|
||||
assert_ne!(left, other);
|
||||
assert_eq!(left.to_string(), "root:default::index.mq.md::f@10..20");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn insert_is_idempotent_but_rejects_conflicting_targets() {
|
||||
let mut table = CallBindingTable::default();
|
||||
let site = key("f", Span::new(1, 2));
|
||||
assert!(table.insert(
|
||||
site.clone(),
|
||||
BoundTarget::Builtin {
|
||||
canonical_name: "长度".into()
|
||||
}
|
||||
));
|
||||
assert!(table.insert(
|
||||
site.clone(),
|
||||
BoundTarget::Builtin {
|
||||
canonical_name: "长度".into()
|
||||
}
|
||||
));
|
||||
assert!(!table.insert(site, BoundTarget::DynamicValue));
|
||||
assert_eq!(table.len(), 1);
|
||||
assert_eq!(table.static_count(), 1);
|
||||
assert_eq!(table.dynamic_count(), 0);
|
||||
}
|
||||
}
|
||||
@@ -10,7 +10,7 @@ use std::fmt;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::{builtins, Function, Module, ModuleId, Param, Program, SignatureKind};
|
||||
use crate::{builtins, Function, FunctionId, Module, ModuleId, Param, Program, SignatureKind};
|
||||
use marqdo_span::Span;
|
||||
|
||||
/// Globally unique callable identity.
|
||||
@@ -138,6 +138,42 @@ pub struct CallableRegistry {
|
||||
builtins: BTreeMap<String, CallableDescriptor>,
|
||||
functions: BTreeMap<ModuleId, BTreeMap<String, CallableDescriptor>>,
|
||||
external: BTreeMap<String, CallableDescriptor>,
|
||||
/// Official stdlib public aliases (for example `打印`/`print`) mapped to
|
||||
/// the bundled Markdown function that implements them. Only the verified
|
||||
/// stdlib loader may populate this table; third-party packages cannot.
|
||||
official_aliases: BTreeMap<String, FunctionId>,
|
||||
}
|
||||
|
||||
/// One official bundled-stdlib public alias.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct OfficialAlias {
|
||||
/// Public names that resolve to the same target (for example both
|
||||
/// `打印` and `print`).
|
||||
pub public_names: Vec<String>,
|
||||
/// Bundled Marqdo function implementing the alias.
|
||||
pub function: FunctionId,
|
||||
/// Effective signature; the loader must have checked it against the
|
||||
/// verified stdlib metadata before registration.
|
||||
pub signature: CallableSignature,
|
||||
/// Owner package instance (the verified stdlib instance).
|
||||
pub owner_package: String,
|
||||
/// Conservative effect request summary for Trace/Analysis.
|
||||
pub effects_requested: Vec<String>,
|
||||
}
|
||||
|
||||
/// A deterministic static-table conflict.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct CallableConflict {
|
||||
/// Colliding public name.
|
||||
pub public_name: String,
|
||||
/// Human-readable reason; never used as a machine code.
|
||||
pub reason: String,
|
||||
}
|
||||
|
||||
impl fmt::Display for CallableConflict {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
write!(f, "`{}`: {}", self.public_name, self.reason)
|
||||
}
|
||||
}
|
||||
|
||||
impl CallableRegistry {
|
||||
@@ -150,21 +186,46 @@ impl CallableRegistry {
|
||||
}
|
||||
|
||||
/// Build a registry for all modules and declared external descriptors.
|
||||
pub fn from_program(program: &Program) -> Self {
|
||||
///
|
||||
/// Duplicate external names are rejected rather than silently keeping the
|
||||
/// first entry: a static table must never depend on registration order.
|
||||
pub fn try_from_program(program: &Program) -> Result<Self, CallableConflict> {
|
||||
let mut registry = Self::default();
|
||||
registry.add_builtins();
|
||||
for module in program.modules.values() {
|
||||
registry.add_module(module);
|
||||
}
|
||||
for descriptor in &program.external_callables {
|
||||
// Conflicting declarations are reported by the loader; keep the
|
||||
// first deterministic entry here rather than overwriting silently.
|
||||
registry
|
||||
.external
|
||||
.entry(descriptor.public_name.clone())
|
||||
.or_insert_with(|| descriptor.clone());
|
||||
.insert_external(descriptor.clone())
|
||||
.map_err(|reason| CallableConflict {
|
||||
public_name: descriptor.public_name.clone(),
|
||||
reason,
|
||||
})?;
|
||||
}
|
||||
registry
|
||||
Ok(registry)
|
||||
}
|
||||
|
||||
/// Build a registry for all modules and declared external descriptors.
|
||||
///
|
||||
/// Prefer [`CallableRegistry::try_from_program`] on the CLI composition
|
||||
/// path; this compatibility wrapper keeps the first declaration for
|
||||
/// already-validated callers (tests and embedders).
|
||||
pub fn from_program(program: &Program) -> Self {
|
||||
Self::try_from_program(program).unwrap_or_else(|_| {
|
||||
let mut registry = Self::default();
|
||||
registry.add_builtins();
|
||||
for module in program.modules.values() {
|
||||
registry.add_module(module);
|
||||
}
|
||||
for descriptor in &program.external_callables {
|
||||
registry
|
||||
.external
|
||||
.entry(descriptor.public_name.clone())
|
||||
.or_insert_with(|| descriptor.clone());
|
||||
}
|
||||
registry
|
||||
})
|
||||
}
|
||||
|
||||
/// Look up a builtin by canonical name or alias.
|
||||
@@ -188,6 +249,85 @@ impl CallableRegistry {
|
||||
self.external.values()
|
||||
}
|
||||
|
||||
/// Register one official bundled-stdlib public alias.
|
||||
///
|
||||
/// The loader calls this only after verifying the bundled stdlib digest and
|
||||
/// the metadata digest, so a third-party package can never claim `打印`,
|
||||
/// `print` or any other official public name through this path.
|
||||
pub fn insert_official_alias(&mut self, alias: OfficialAlias) -> Result<(), String> {
|
||||
if alias.public_names.is_empty() {
|
||||
return Err("official alias must declare at least one public name".to_string());
|
||||
}
|
||||
if alias.function.module.instance.is_empty() || alias.function.name.is_empty() {
|
||||
return Err("official alias target function identity must not be empty".to_string());
|
||||
}
|
||||
for name in &alias.public_names {
|
||||
if name.is_empty() {
|
||||
return Err("official alias public name must not be empty".to_string());
|
||||
}
|
||||
if self.external.contains_key(name) {
|
||||
return Err(format!(
|
||||
"official alias `{name}` conflicts with a declared plugin export"
|
||||
));
|
||||
}
|
||||
let descriptor = CallableDescriptor {
|
||||
id: CallableId(format!("marqdo:{}", alias.function)),
|
||||
public_name: name.clone(),
|
||||
owner_package: alias.owner_package.clone(),
|
||||
signature: alias.signature.clone(),
|
||||
effects_requested: alias.effects_requested.clone(),
|
||||
origin: CallableOrigin::Marqdo,
|
||||
implementation_digest: None,
|
||||
source_span: None,
|
||||
};
|
||||
// Official aliases intentionally replace the reserved placeholder
|
||||
// builtin (for example `打印`); no third-party path can do this.
|
||||
self.builtins.insert(name.clone(), descriptor);
|
||||
self.official_aliases
|
||||
.insert(name.clone(), alias.function.clone());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Target Marqdo function of an official alias, if `name` is one.
|
||||
pub fn alias_target(&self, name: &str) -> Option<&FunctionId> {
|
||||
self.official_aliases.get(name)
|
||||
}
|
||||
|
||||
/// Iterate official aliases deterministically (public name, target).
|
||||
pub fn official_aliases(&self) -> impl Iterator<Item = (&String, &FunctionId)> {
|
||||
self.official_aliases.iter()
|
||||
}
|
||||
|
||||
/// Whether a dotted builtin namespace starts with `prefix.`.
|
||||
pub fn has_builtin_dotted_prefix(&self, prefix: &str) -> bool {
|
||||
self.builtins.keys().any(|name| {
|
||||
name.strip_prefix(prefix)
|
||||
.is_some_and(|rest| rest.starts_with('.'))
|
||||
})
|
||||
}
|
||||
|
||||
/// Whether a declared plugin export starts with `prefix.`.
|
||||
pub fn has_external_dotted_prefix(&self, prefix: &str) -> bool {
|
||||
self.external.keys().any(|name| {
|
||||
name.strip_prefix(prefix)
|
||||
.is_some_and(|rest| rest.starts_with('.'))
|
||||
})
|
||||
}
|
||||
|
||||
/// Resolve a module function to its exact identity.
|
||||
pub fn function_id(&self, module: &ModuleId, name: &str) -> Option<FunctionId> {
|
||||
let descriptor = self.function(module, name)?;
|
||||
if matches!(descriptor.origin, CallableOrigin::Marqdo) {
|
||||
Some(FunctionId {
|
||||
module: module.clone(),
|
||||
name: descriptor.public_name.clone(),
|
||||
})
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Insert an external descriptor, rejecting duplicate public names.
|
||||
pub fn insert_external(&mut self, descriptor: CallableDescriptor) -> Result<(), String> {
|
||||
if self.builtins.contains_key(&descriptor.public_name) {
|
||||
|
||||
@@ -17,6 +17,7 @@ use marqdo_span::Span;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
pub mod builtins;
|
||||
pub mod call_binding;
|
||||
pub mod callable;
|
||||
|
||||
/// Canonical logical identity of a module: package instance plus a path
|
||||
@@ -476,10 +477,13 @@ pub struct Program {
|
||||
/// metadata only: `check` never loads a library.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub external_callables: Vec<callable::CallableDescriptor>,
|
||||
/// Compat-B/C `打印`/`print` destination (ADR-0030/0031). When set, the
|
||||
/// runtime calls this stdlib function; there is no legacy HostOp path.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub print_override: Option<FunctionId>,
|
||||
/// Resolved static call bindings (RFC-EXT-0001 §4.2, ADR-0054).
|
||||
///
|
||||
/// Internal IR: deliberately skipped on the wire so Analysis/Trace v2
|
||||
/// consumers are unaffected. A prepared program always carries a complete
|
||||
/// table; the runtime never re-resolves names.
|
||||
#[serde(skip)]
|
||||
pub call_bindings: call_binding::CallBindingTable,
|
||||
}
|
||||
|
||||
impl Program {
|
||||
@@ -517,6 +521,19 @@ impl Program {
|
||||
pub fn callable_registry(&self) -> callable::CallableRegistry {
|
||||
callable::CallableRegistry::from_program(self)
|
||||
}
|
||||
|
||||
/// Resolved static call binding for one call site, if any.
|
||||
pub fn call_binding(
|
||||
&self,
|
||||
key: &call_binding::CallSiteKey,
|
||||
) -> Option<&call_binding::BoundTarget> {
|
||||
self.call_bindings.get(key)
|
||||
}
|
||||
|
||||
/// Install the prepared binding table.
|
||||
pub fn set_call_bindings(&mut self, bindings: call_binding::CallBindingTable) {
|
||||
self.call_bindings = bindings;
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -54,7 +54,6 @@ pub struct RunOutcome {
|
||||
/// Execution frame for one function call.
|
||||
struct Frame {
|
||||
function: FunctionId,
|
||||
namespace: Option<String>,
|
||||
locals: BTreeMap<String, Value>,
|
||||
}
|
||||
|
||||
@@ -179,7 +178,6 @@ impl<'a> Interpreter<'a> {
|
||||
};
|
||||
let mut frame = Frame {
|
||||
function: function.id.clone(),
|
||||
namespace: function.namespace.clone(),
|
||||
locals,
|
||||
};
|
||||
self.depth += 1;
|
||||
@@ -277,7 +275,6 @@ impl<'a> Interpreter<'a> {
|
||||
// an empty frame so they cannot reach caller state.
|
||||
let mut empty = Frame {
|
||||
function: function.id.clone(),
|
||||
namespace: function.namespace.clone(),
|
||||
locals: BTreeMap::new(),
|
||||
};
|
||||
let value = self.eval_expr(default, &mut empty)?;
|
||||
@@ -616,113 +613,96 @@ impl<'a> Interpreter<'a> {
|
||||
span: Span,
|
||||
frame: &mut Frame,
|
||||
) -> Result<Value, RuntimeError> {
|
||||
let caller_module = frame.function.module.clone();
|
||||
match callee {
|
||||
Expr::Ident(ident) => {
|
||||
if let Some(value) = frame.locals.get(&ident.name).cloned() {
|
||||
let args = self.eval_args(args, frame)?;
|
||||
return self.call_value(program, value, args, Some(span), Some(&caller_module));
|
||||
}
|
||||
if matches!(ident.name.as_str(), "打印" | "print") {
|
||||
if let Some(target) = program.print_override.clone() {
|
||||
let args = self.eval_args(args, frame)?;
|
||||
let Some(function) = program.function(&target) else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::UnknownSymbol,
|
||||
Some(span),
|
||||
format!("print override `{target}` does not exist"),
|
||||
));
|
||||
};
|
||||
return self.call_function(
|
||||
program,
|
||||
function,
|
||||
args,
|
||||
Some(span),
|
||||
Some(&caller_module),
|
||||
);
|
||||
}
|
||||
}
|
||||
if let Some(builtin) = builtins::find(&ident.name) {
|
||||
let args = self.eval_args(args, frame)?;
|
||||
return self.call_builtin(builtin, args, Some(span), frame);
|
||||
}
|
||||
let args = self.eval_args(args, frame)?;
|
||||
let (module, namespace) = (frame.function.module.clone(), frame.namespace.clone());
|
||||
let Some(function) =
|
||||
resolve_function(program, &module, namespace.as_deref(), &ident.name)
|
||||
else {
|
||||
// ADR-0054 / RFC-EXT-0001 §4: the frontend resolved this call site
|
||||
// exactly once. Dispatch by the recorded identity; never re-resolve a
|
||||
// name, never infer a plugin from a dotted prefix.
|
||||
let key = marqdo_ir::call_binding::CallSiteKey::new(frame.function.clone(), span);
|
||||
let Some(target) = program.call_bindings.get(&key).cloned() else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::Internal,
|
||||
Some(span),
|
||||
format!(
|
||||
"call site `{key}` has no static binding; run a prepared program (ADR-0054)"
|
||||
),
|
||||
));
|
||||
};
|
||||
match target {
|
||||
marqdo_ir::call_binding::BoundTarget::Builtin { canonical_name } => {
|
||||
let Some(builtin) = builtins::find(&canonical_name) else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::UnknownSymbol,
|
||||
ErrorCategory::Internal,
|
||||
Some(span),
|
||||
format!("unknown function `{}`", ident.name),
|
||||
format!("bound builtin `{canonical_name}` is not registered"),
|
||||
));
|
||||
};
|
||||
self.call_function(program, function, args, Some(span), Some(&caller_module))
|
||||
let args = self.eval_args(args, frame)?;
|
||||
self.call_builtin(builtin, args, Some(span), frame)
|
||||
}
|
||||
Expr::Member { base, name, .. } => {
|
||||
if let Expr::Ident(alias) = base.as_ref() {
|
||||
if let Some(target_module) =
|
||||
program.resolve_import(&frame.function.module, &alias.name)
|
||||
{
|
||||
let args = self.eval_args(args, frame)?;
|
||||
let Some(function) = resolve_function(program, target_module, None, name)
|
||||
else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::UnknownSymbol,
|
||||
Some(span),
|
||||
format!("unknown function `{name}` in module `{target_module}`"),
|
||||
));
|
||||
};
|
||||
return self.call_function(
|
||||
program,
|
||||
function,
|
||||
args,
|
||||
Some(span),
|
||||
Some(&caller_module),
|
||||
);
|
||||
}
|
||||
let qualified = format!("{}.{name}", alias.name);
|
||||
if let Some(builtin) = builtins::find(&qualified) {
|
||||
let args = self.eval_args(args, frame)?;
|
||||
return self.call_builtin(builtin, args, Some(span), frame);
|
||||
}
|
||||
if !frame.locals.contains_key(&alias.name) {
|
||||
// A declared external descriptor is the only static
|
||||
// source of plugin exports. Calls without a descriptor
|
||||
// are still accepted for compatibility only when no
|
||||
// plugin descriptors exist at all; this fallback is
|
||||
// removed once the CLI always supplies descriptors.
|
||||
let qualified = format!("{}.{}", alias.name, name);
|
||||
let declared = program.external_callable(&qualified).is_some();
|
||||
if !declared && !program.external_callables.is_empty() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::UnknownSymbol,
|
||||
Some(span),
|
||||
format!("unknown plugin export `{qualified}`"),
|
||||
));
|
||||
}
|
||||
// Plugin calls take named arguments only in this build.
|
||||
let evaluated = self.eval_args(args, frame)?;
|
||||
let mut named = std::collections::BTreeMap::new();
|
||||
for argument in evaluated {
|
||||
let Some(argument_name) = argument.name else {
|
||||
marqdo_ir::call_binding::BoundTarget::Marqdo { function } => {
|
||||
let Some(target_function) = program.function(&function) else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::Internal,
|
||||
Some(span),
|
||||
format!("bound function `{function}` does not exist"),
|
||||
));
|
||||
};
|
||||
let args = self.eval_args(args, frame)?;
|
||||
self.call_function(
|
||||
program,
|
||||
target_function,
|
||||
args,
|
||||
Some(span),
|
||||
Some(&frame.function.module.clone()),
|
||||
)
|
||||
}
|
||||
marqdo_ir::call_binding::BoundTarget::AbiPlugin {
|
||||
package: _,
|
||||
export,
|
||||
digest: _,
|
||||
param_names,
|
||||
} => {
|
||||
// ABI v1 transports named arguments as a map. Normalise
|
||||
// positional arguments deterministically against the static
|
||||
// manifest order; static validation already rejected unknown,
|
||||
// duplicate or missing arguments.
|
||||
let evaluated = self.eval_args(args, frame)?;
|
||||
let mut named = std::collections::BTreeMap::new();
|
||||
let mut position = 0usize;
|
||||
for argument in evaluated {
|
||||
let name = match argument.name {
|
||||
Some(name) => name,
|
||||
None => {
|
||||
let Some(name) = param_names.get(position).cloned() else {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::Type,
|
||||
Some(argument.span),
|
||||
"plugin calls require named arguments",
|
||||
ErrorCategory::Internal,
|
||||
Some(span),
|
||||
format!(
|
||||
"positional argument {} was not normalised for `{export}`",
|
||||
position + 1
|
||||
),
|
||||
));
|
||||
};
|
||||
named.insert(argument_name, argument.value);
|
||||
position += 1;
|
||||
name
|
||||
}
|
||||
let module = frame.function.module.clone();
|
||||
return self.host_invoke(
|
||||
HostOp::PluginInvoke,
|
||||
vec![Value::Text(qualified), Value::Map(named)],
|
||||
};
|
||||
if named.insert(name.clone(), argument.value).is_some() {
|
||||
return Err(RuntimeError::new(
|
||||
ErrorCategory::Arity,
|
||||
Some(span),
|
||||
&module,
|
||||
);
|
||||
format!("duplicate argument `{name}` for `{export}`"),
|
||||
));
|
||||
}
|
||||
}
|
||||
let module = frame.function.module.clone();
|
||||
self.host_invoke(
|
||||
HostOp::PluginInvoke,
|
||||
vec![Value::Text(export), Value::Map(named)],
|
||||
Some(span),
|
||||
&module,
|
||||
)
|
||||
}
|
||||
marqdo_ir::call_binding::BoundTarget::DynamicValue => {
|
||||
let callee_value = self.eval_expr(callee, frame)?;
|
||||
let args = self.eval_args(args, frame)?;
|
||||
self.call_value(
|
||||
@@ -730,18 +710,7 @@ impl<'a> Interpreter<'a> {
|
||||
callee_value,
|
||||
args,
|
||||
Some(span),
|
||||
Some(&caller_module),
|
||||
)
|
||||
}
|
||||
other => {
|
||||
let callee_value = self.eval_expr(other, frame)?;
|
||||
let args = self.eval_args(args, frame)?;
|
||||
self.call_value(
|
||||
program,
|
||||
callee_value,
|
||||
args,
|
||||
Some(span),
|
||||
Some(&caller_module),
|
||||
Some(&frame.function.module.clone()),
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -935,21 +904,6 @@ impl<'a> Interpreter<'a> {
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_function<'p>(
|
||||
program: &'p Program,
|
||||
module: &ModuleId,
|
||||
namespace: Option<&str>,
|
||||
name: &str,
|
||||
) -> Option<&'p Function> {
|
||||
let module = program.module(module)?;
|
||||
if let Some(namespace) = namespace {
|
||||
if let Some(found) = module.namespaced_function(namespace, name) {
|
||||
return Some(found);
|
||||
}
|
||||
}
|
||||
module.function(name)
|
||||
}
|
||||
|
||||
/// Bind already-evaluated arguments to a builtin parameter list.
|
||||
fn bind_builtin_args(
|
||||
builtin: &'static builtins::Builtin,
|
||||
|
||||
@@ -19,6 +19,14 @@ fn build(source: &str) -> Program {
|
||||
..Program::default()
|
||||
};
|
||||
program.insert(result.module);
|
||||
let registry = program.callable_registry();
|
||||
let bound = marqdo_frontend::calls::bind_program_calls(&program, ®istry);
|
||||
assert!(
|
||||
bound.diagnostics.is_empty(),
|
||||
"unexpected binding diagnostics: {:#?}",
|
||||
bound.diagnostics
|
||||
);
|
||||
program.set_call_bindings(bound.bindings);
|
||||
program
|
||||
}
|
||||
|
||||
|
||||
@@ -81,6 +81,34 @@ pub mod codes {
|
||||
pub const PROJECT_LOCK: &str = "project.lock";
|
||||
/// Dependency resolution failed.
|
||||
pub const PROJECT_DEPENDENCY: &str = "project.dependency";
|
||||
/// A statically visible callable export is declared more than once
|
||||
/// (RFC-EXT-0001 §16.1).
|
||||
pub const EXT_DUPLICATE_CALLABLE: &str = "ext.duplicate_callable";
|
||||
/// An import alias collides with a builtin or plugin dotted namespace.
|
||||
pub const EXT_IMPORT_NAMESPACE_CONFLICT: &str = "ext.import_namespace_conflict";
|
||||
/// A loaded plugin's exports/signature/owner/artifact digest do not exactly
|
||||
/// match the statically prepared descriptors.
|
||||
pub const EXT_PLUGIN_BINDING_MISMATCH: &str = "ext.plugin_binding_mismatch";
|
||||
/// Two resource providers try to register the same non-reserved kind.
|
||||
pub const EXT_DUPLICATE_PROVIDER_KIND: &str = "ext.duplicate_provider_kind";
|
||||
/// A legacy/future provider manifest is passed to the v2-only CLI entry.
|
||||
pub const EXT_UNSUPPORTED_PROVIDER_SCHEMA: &str = "ext.unsupported_provider_schema";
|
||||
/// No provider artifact matches the current host target triple.
|
||||
pub const EXT_NO_MATCHING_ARTIFACT: &str = "ext.no_matching_artifact";
|
||||
/// A provider artifact differs from its declared sha256, escapes its
|
||||
/// bundle, is not a regular file or is an unallowed symlink.
|
||||
pub const EXT_ARTIFACT_INTEGRITY: &str = "ext.artifact_integrity";
|
||||
/// The matching provider worker/helper program is missing or cannot start.
|
||||
pub const EXT_WORKER_UNAVAILABLE: &str = "ext.worker_unavailable";
|
||||
/// The current platform explicitly does not support the requested
|
||||
/// extension transport.
|
||||
pub const EXT_PLATFORM_UNSUPPORTED: &str = "ext.platform_unsupported";
|
||||
/// A locked extension identity does not match the actual manifest or
|
||||
/// artifact on disk.
|
||||
pub const EXT_EXTENSION_LOCK_MISMATCH: &str = "ext.extension_lock_mismatch";
|
||||
/// An extension is explicitly selected but is not part of the project lock
|
||||
/// (development-only use; release mode rejects it).
|
||||
pub const EXT_UNLOCKED_EXTENSION: &str = "ext.unlocked_extension";
|
||||
/// Internal invariant violation; never expected in normal operation.
|
||||
pub const INTERNAL: &str = "internal.error";
|
||||
}
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
# ADR-0054: One static callable binding, one runtime dispatch
|
||||
|
||||
- Status: Accepted (2026-11-21)
|
||||
- Supersedes the resolution model of ADR-0009/ADR-0010 where they conflict
|
||||
- Implements: RFC-EXT-0001 §4 (PR-01/PR-02)
|
||||
|
||||
## Context
|
||||
|
||||
The tree had four independent ways to decide what a call means:
|
||||
|
||||
1. `marqdo-frontend/src/calls.rs` checked a Member call's builtin name before
|
||||
its import alias;
|
||||
2. `marqdo-runtime/src/interpreter.rs` checked the import alias before the
|
||||
builtin for the same expression;
|
||||
3. the runtime treated an undeclared `alias.name` as a plugin invocation when
|
||||
`program.external_callables` was empty ("try it at run time");
|
||||
4. `marqdo-cli` synthesised an empty-parameter descriptor (`params: Vec::new()`)
|
||||
for exports that were loaded but not statically described.
|
||||
|
||||
Consequences observed while building RFC-EXT-0001's fixtures:
|
||||
|
||||
* `check` could accept a call that `run` dispatched to a different
|
||||
implementation (or vice versa);
|
||||
* an unknown symbol failed only at run time, with a Host side effect already
|
||||
attempted;
|
||||
* an import alias that happened to equal a builtin dotted namespace (`math`)
|
||||
silently changed the meaning of `math.eval(...)` depending on the pipeline;
|
||||
* plugin arity/name errors could pass static checking and only fail (or
|
||||
succeed with different arguments) at the ABI boundary.
|
||||
|
||||
The language contract (RFC-LANG-0002) already says calls are expressions and
|
||||
that `check` and `run` must agree; the implementation had no such guarantee.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **One registry.** `marqdo_ir::callable::CallableRegistry` is the only static
|
||||
table. It contains builtins (including official bundled-stdlib public
|
||||
aliases), module functions keyed by canonical name plus unique local name,
|
||||
and exact plugin exports (`namespace.function`). Duplicate declarations are
|
||||
rejected when the registry is built; order never decides the answer.
|
||||
2. **One binding pass.** `marqdo_frontend::calls::bind_program_calls(program,
|
||||
registry)` walks every statically visible call expression exactly once and
|
||||
produces `CallBindingTable` plus diagnostics. It replaces the split
|
||||
`validate_module`/`validate_program` priority logic on the CLI path (the
|
||||
standalone frontend keeps a wrapper for single-document tests). The result
|
||||
is installed on `Program::call_bindings`.
|
||||
3. **Deterministic identity.** A call site is
|
||||
`(module id, function id, byte span)` (`CallSiteKey`), never a disk path,
|
||||
registration order or hash-map iteration. A bound target is one of:
|
||||
* `Builtin { canonical_name }` — fixed core function;
|
||||
* `Marqdo { function }` — exact `FunctionId` (module functions and official
|
||||
stdlib aliases such as `打印`/`print` → `io.mq.md::写值`);
|
||||
* `AbiPlugin { package, export, digest, param_names }` — complete export
|
||||
name plus the statically verified artifact digest and manifest argument
|
||||
order;
|
||||
* `DynamicValue` — a language value (for example a `FunctionRef`); this is
|
||||
the **only** non-static path and cannot reach an ABI export by name.
|
||||
4. **Runtime dispatches by identity.** `eval_call` looks up the site's binding
|
||||
and calls the recorded target. It never calls `builtins::find(qualified)`,
|
||||
never inspects `external_callables.is_empty()` and never guesses a plugin.
|
||||
A missing binding is an internal error (`internal.error`), not a reason to
|
||||
re-resolve.
|
||||
5. **Prepare finalises once.** `marqdo-cli` has one composition step
|
||||
(`project_prepare::prepare`): load sources → read extension descriptors
|
||||
(no `dlopen`/spawn) → build the registry → bind calls once. `check`,
|
||||
`run`/`call`, `inspect` and Analysis consume that one prepared program.
|
||||
Static plugin descriptors are inserted before binding, not appended after a
|
||||
first validation.
|
||||
6. **Exact run-time reconciliation.** `run` verifies each selected plugin
|
||||
artifact, loads it, then compares the Host's actual exports (name, raw
|
||||
signature, artifact digest, owner package) with the static descriptors
|
||||
one-to-one. Any extra, missing or altered export fails with
|
||||
`ext.plugin_binding_mismatch` before the entry function runs. No descriptor
|
||||
is synthesised at run time.
|
||||
7. **Print is not special.** `Program.print_override` and the runtime
|
||||
`打印`/`print` branch are removed. The builtin placeholder remains only for
|
||||
embedders without the official bundle and fails with
|
||||
`kernel.unavailable`; the official bundle declares aliases in verified
|
||||
metadata (ADR-0058).
|
||||
|
||||
## Priority matrix (normative)
|
||||
|
||||
See `spec/callable-resolution.md` for the complete matrix and diagnostics. In
|
||||
short:
|
||||
|
||||
* `f(...)`: local variable → official alias/reserved builtin → current
|
||||
namespace → canonical name → unique local name → otherwise
|
||||
`symbol.unknown` (ambiguity is an error, never a first-match);
|
||||
* `alias.f(...)`: explicit import alias → builtin dotted namespace → exact
|
||||
plugin export → otherwise `symbol.unknown`;
|
||||
* `some_value.f(...)`: dynamic language dispatch only.
|
||||
|
||||
Import aliases that collide with a builtin or plugin dotted namespace are
|
||||
rejected at prepare with `ext.import_namespace_conflict`; duplicate plugin
|
||||
exports are rejected with `ext.duplicate_callable`.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `check` and `run` now share binding identity; CALL-01…CALL-16 in
|
||||
`crates/cli/tests/callable_resolution.rs` pin the matrix.
|
||||
* 0.x breaking change: a project importing a module under an alias that equals
|
||||
a builtin namespace (for example `math`) must rename the alias. There is no
|
||||
`--legacy` switch and no dual semantics.
|
||||
* Analysis v2's `calls[].state/target` are now projected from the same binding
|
||||
(official aliases appear as `function` targets). Analysis v2 schema and
|
||||
fields are otherwise unchanged; private `CallBindingTable` is not serialized.
|
||||
* Runtime no longer performs per-call name resolution, so the dynamic path is
|
||||
limited to language values. A text or map value called as a function is
|
||||
`call.not_callable` and never becomes a Host plugin request.
|
||||
* Plugin manifests must be complete before binding: a manifest with a missing
|
||||
or malformed signature is a prepare error; ABI v1 default values are
|
||||
rejected until the wire semantics are versioned.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
* **Pick a fixed precedence and keep two implementations.** Rejected: the RFC
|
||||
requires one static answer, and duplicated precedence is exactly the defect.
|
||||
* **Build the registry lazily in the runtime.** Rejected: `check` would still
|
||||
not prove the binding, and runtime loading would change static resolution.
|
||||
* **Keep the empty-`external_callables` fallback for "developer convenience".**
|
||||
Rejected: it turns a compile error into a Host call attempt and breaks
|
||||
`check`'s no-execution guarantee.
|
||||
* **Expose `CallableId`/`CallBindingTable` in Analysis v2.** Rejected:
|
||||
private IR must not become a public wire contract without a version bump.
|
||||
EOF
|
||||
@@ -0,0 +1,196 @@
|
||||
# ADR-0055: Resource Provider v2 through the normal CLI
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Relates to: RFC-EXT-0001 §5/§16/§18, ADR-0033, ADR-0034, ADR-0035, ADR-0036,
|
||||
ADR-0037, ADR-0038, ADR-0021 (strict L1), ADR-0052 (single Markdown semantics)
|
||||
Specs: `spec/resource-provider-v2.md`
|
||||
Implementation: `crates/cli/src/provider_manifest.rs`,
|
||||
`crates/cli/src/commands/sandbox.rs`, `crates/cli/src/commands/run.rs`,
|
||||
`crates/cli/src/main.rs`, `crates/cli/tests/provider_cli.rs`
|
||||
|
||||
## Context
|
||||
|
||||
The host already implements versioned resource-provider registration
|
||||
(`register_resource_provider`, `register_worker_resource_provider`,
|
||||
`register_c_resource_provider`, `register_provider_package`) and
|
||||
`resource.open` dispatches unknown kinds through that registry with a provider
|
||||
name grant (ADR-0033/0035/0036/0037/0038). What was missing was a **public
|
||||
composition-root entry point**: `main.rs`/`SandboxSettings`/`run.rs` had no
|
||||
`--provider`/`--allow-provider`, so an external repository could not use a new
|
||||
resource kind on a frozen `marqdo` binary without patching and rebuilding the
|
||||
CLI (RFC-EXT-0001 EXT-07).
|
||||
|
||||
The existing `marqdo.provider` manifest (`schema_version: 1`, ADR-0037) is a
|
||||
host-embedding installation format: `name/kinds/abi/library`, no `version`, no
|
||||
targeted artifact digest and no transport. It cannot support portable,
|
||||
integrity-checked CLI loading. RFC-EXT-0001 §5.3 asks for a v2 document rather
|
||||
than silently overloading v1. It also requires that selecting an extension is
|
||||
not authorizing it, that `pure`/`strict` keep failing closed, and that no new
|
||||
`HostOp`/builtin/business branch is introduced.
|
||||
|
||||
This ADR records the PR-03 decision on the current branch.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. CLI surface and ordering
|
||||
|
||||
`--provider PATH` (repeatable, a `provider.json` file) and
|
||||
`--allow-provider NAME` (repeatable, exact name) are added to `SandboxArgs`, so
|
||||
they are shared by `run` and `call` only. `check`/`inspect` do not declare them
|
||||
and never read a provider manifest. `SandboxSettings` carries
|
||||
`providers: Vec<PathBuf>` and `allow_provider: Vec<String>`; the `project`
|
||||
capability set calls the existing `Capabilities::allow_resource_provider(name)`
|
||||
for each grant.
|
||||
|
||||
Run ordering follows RFC-EXT-0001 §18.2: `prepare` (no provider work) →
|
||||
diagnostics → mode/strict preflight → capabilities → host assembly and budgets
|
||||
→ kernels → **all provider manifests/targets/artifacts/grants/duplicate kinds
|
||||
verified and plugin artifacts verified** → plugin workers loaded and exports
|
||||
reconciled → provider helpers spawned → entry-function check → execute. No
|
||||
worker (plugin or provider) starts before every T6/T6b verification passes, and
|
||||
the main function never runs after a provider failure.
|
||||
|
||||
`pure` and `strict` reject both flags in preflight (`cli.invalid_grants` /
|
||||
`strict.unsupported_grant`, exit 2) without silently switching to `project`.
|
||||
`--allow-provider` rejects wildcards (`*`, `?`, `[`, `]`) and empty names; a
|
||||
grant that matches no selected manifest is `ext.provider_grant_unmatched`
|
||||
(exit 2). A `--provider` selected without its exact grant is
|
||||
`permission.denied` (exit 1) before the helper is spawned.
|
||||
|
||||
### 2. Strict v2 manifest parser
|
||||
|
||||
New `crates/cli/src/provider_manifest.rs` owns the document, separate from the
|
||||
plugin `plugin.yaml` parser:
|
||||
|
||||
* accepts exactly `schema = "marqdo.provider"`, `schema_version = 2`,
|
||||
`abi = "c-abi-1"`, `transport = "worker"`; other versions/ABIs/transports are
|
||||
`ext.unsupported_provider_schema` (no silent migration, no v1 fallback from
|
||||
the CLI);
|
||||
* strict dotted lowercase name/kind grammar, strict SemVer, non-empty unique
|
||||
non-reserved kinds, bounded counts, known/vendor `permissions_requested`
|
||||
declarations only, optional non-executable `metadata`;
|
||||
* unknown top-level/artifact fields are rejected (`ext.manifest_invalid`);
|
||||
* artifacts need unique parseable targets and 64-lowercase-hex sha256;
|
||||
the current `HostTarget` artifact is selected (`ext.no_matching_artifact`
|
||||
when absent), resolved inside the manifest bundle (absolute/`..`/symlink
|
||||
escape/non-regular/size limit/sha mismatch → `ext.artifact_integrity`);
|
||||
* duplicate names with different manifest digests (`ext.duplicate_provider`)
|
||||
and duplicate kinds across providers (`ext.duplicate_provider_kind`) are
|
||||
deterministic because paths are canonicalised, sorted and de-duplicated
|
||||
before conflict detection.
|
||||
|
||||
`VerifiedProvider` is the static `{name, kinds, manifest_path, base_dir,
|
||||
manifest_digest, transport, selected_artifact_path/relative/sha256/target,
|
||||
version}` record. It is still declaration/verification only: no `dlopen`, no
|
||||
spawn, no grant.
|
||||
|
||||
### 3. Activation reuses the existing host API
|
||||
|
||||
`load_resource_provider_into_host(host, verified, allow_provider)` repeats the
|
||||
exact grant check, re-verifies the manifest digest and the artifact
|
||||
(regular file, containment, size, sha256) immediately before use, resolves the
|
||||
helper, and calls **the existing**
|
||||
`StdHost::register_worker_resource_provider(name, kinds, helper, [artifact])`.
|
||||
It never falls back to `register_c_resource_provider`,
|
||||
`register_provider_package`, in-process native loading or another provider.
|
||||
`resource.open` still re-checks the provider name grant in the Host. Provider
|
||||
registration keeps using `register_provider_checked` for name/kind/duplicate
|
||||
validation, so no second provider registry is introduced.
|
||||
|
||||
### 4. Helper resolution and platform claim
|
||||
|
||||
The helper is resolved only as a sibling of the running `marqdo` executable:
|
||||
`<exe dir>/marqdo-provider-worker[.exe]`. `PATH`, the current directory,
|
||||
environment overrides and downloads are not consulted. Missing/non-regular
|
||||
helper, spawn failure or failed `ping` → `ext.worker_unavailable` (exit 1).
|
||||
|
||||
The `worker` transport is claimed only on **linux/x86_64**; every other target
|
||||
returns `ext.platform_unsupported` (exit 1) instead of pretending success.
|
||||
Release artifacts must ship the helper next to the CLI for the same target
|
||||
triple. macOS/Windows stay buildable and explicitly unsupported until they have
|
||||
a native fixture and release evidence (RFC-EXT-0001 §5.6).
|
||||
|
||||
### 5. No new host op or language surface
|
||||
|
||||
There is no new `HostOp`, builtin, `resource_request` string branch, `Value`
|
||||
variant or Markdown syntax. A provider kind is still an argument to
|
||||
`resource.open("<kind>", descriptor, rights)`; the only change is the CLI
|
||||
composition root that selects, verifies and authorizes the existing host
|
||||
transport. Provider tokens stay inside the helper; the language sees only the
|
||||
opaque Host `Resource` handle. Worker errors are structured `RuntimeError`s
|
||||
surfaced as CLI errors — the CLI never panics on a helper crash/timeout.
|
||||
|
||||
### 6. Tests
|
||||
|
||||
`crates/cli/tests/provider_cli.rs` drives the real CLI binary with a real C
|
||||
`c-abi-1` sensor library and the real `marqdo-provider-worker` helper built by
|
||||
cargo and installed next to the CLI (the same layout a release tarball uses;
|
||||
production resolution is not weakened for tests). Coverage: PROV-01/02/03/04/
|
||||
05/06/07/08/09/10/13, plus `check` refusing the flag. Marker files written by
|
||||
the C library's ABI-version hook prove whether a worker actually started, so
|
||||
"no worker on unauthorized/invalid input" is evidence, not an assertion about
|
||||
code paths.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A third party can add a new resource kind to a frozen `marqdo` binary through
|
||||
`provider.json` v2 + `--provider` + `--allow-provider`, without a core
|
||||
rebuild, new builtin or new CLI command.
|
||||
* v1 `marqdo.provider` remains a host-embedding format; CLI users get a clear
|
||||
`ext.unsupported_provider_schema` instead of a silently unverified load.
|
||||
* Integrity hashing raises the cost of accidental corruption/tampering but is
|
||||
not a trust root; a compromised or malicious helper still runs with the
|
||||
user's privileges. Documentation states this explicitly.
|
||||
* `pure`/`strict` semantics and the existing resource/provider host contracts
|
||||
are unchanged (strict still L1-only and fail-closed).
|
||||
* The provider helper becomes a release artifact that must be packaged next to
|
||||
the CLI; a missing helper is a first-class, deterministic error.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
1. **Reuse `plugin.yaml`/plugin ABI directly for providers.** Rejected: a
|
||||
resource provider's contract is open/read/write/close with provider-private
|
||||
tokens and stream kinds, not callable exports; sharing the manifest/ABI
|
||||
parser would conflate two independent protocols (RFC-EXT-0001 §1.3).
|
||||
2. **Extend v1 in place with optional `version`/`artifacts`.** Rejected:
|
||||
v1 is host-embedding installation metadata and is already accepted by
|
||||
`register_provider_package`; silently changing its meaning or accepting a
|
||||
half-populated v1 would break the frozen ADR-0037 contract. v2 is explicit.
|
||||
3. **Allow in-process C ABI loading when the worker helper is missing.**
|
||||
Rejected: it would silently lower the process boundary and RFC-EXT-0001
|
||||
§5.2/§5.4 forbids falling back to Native. `ext.worker_unavailable` is the
|
||||
correct fail-closed result.
|
||||
4. **PATH lookup for `marqdo-provider-worker`.** Rejected: `PATH` is
|
||||
attacker-controlled state and would make the executed artifact
|
||||
non-deterministic; the sibling rule binds the helper to the release.
|
||||
5. **Register providers lazily at the first `resource.open`.** Rejected for
|
||||
this slice: authorization and integrity must be decided before the main
|
||||
function executes, and the existing host API is synchronous spawn-on-register.
|
||||
6. **Accept a `*` provider grant for developer convenience.** Rejected: grants
|
||||
are exact names (RFC-EXT-0001 §5.2); wildcards would let a typo authorize an
|
||||
arbitrary selected provider.
|
||||
|
||||
## Deviations recorded
|
||||
|
||||
* `ext.manifest_invalid`, `ext.duplicate_provider` and
|
||||
`ext.provider_grant_unmatched` are **additive** `ext.*` codes beyond the list
|
||||
frozen in RFC-EXT-0001 §18.1. The frozen list has no generic malformed-v2
|
||||
code, no duplicate-provider-name code and no unmatched-grant code; reusing
|
||||
`ext.unsupported_provider_schema`/`ext.duplicate_provider_kind` for these
|
||||
would report the wrong reason. They follow the same `ext.` naming and exit
|
||||
code contract and are frozen by this ADR.
|
||||
* `ext.artifact_integrity` failures exit 1 (load/verify), while static
|
||||
manifest-selection failures exit 2, matching RFC-EXT-0001's exit-code
|
||||
guidance; plugin artifact checking already uses the same split.
|
||||
* Only linux/x86_64 is claimed for the provider `worker` transport in this
|
||||
release (RFC-EXT-0001 §5.6 permits this explicit platform limit; Windows and
|
||||
macOS return `ext.platform_unsupported`).
|
||||
|
||||
## Verification
|
||||
|
||||
`crates/cli/tests/provider_cli.rs` (12 real tests, linux/x86_64) plus
|
||||
`crates/cli/src/provider_manifest.rs` unit tests pass; `spec/security.md`,
|
||||
`spec/resource-provider-v2.md` and the README provider section document the
|
||||
same contract. Evidence is recorded in the PR-03 task report (commands, exit
|
||||
codes, per-PROV matrix).
|
||||
@@ -0,0 +1,229 @@
|
||||
# ADR-0057: Trusted capability-handler registration
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Implements: RFC-EXT-0001 §7 (PR-05)
|
||||
Extends: ADR-0011 (capability broker)
|
||||
Scope: `crates/marqdo-host/src/capability.rs` (host-internal registry)
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0011 introduced `CapabilityBroker` with two separate ideas but only one
|
||||
implementation path:
|
||||
|
||||
* `decide()` answered "is this capability granted?" from `Capabilities` and a
|
||||
trusted vendor-policy allow-list; and
|
||||
* `execute()` special-cased the single real core implementation (`fs.read`)
|
||||
inline, returning `capability.unavailable` for everything else.
|
||||
|
||||
That shape is why adding a capability looked like a one-line allow-list edit,
|
||||
and it made "declared in policy" and "can actually execute" the same object.
|
||||
RFC-EXT-0001 §7.1/§7.2 requires the chain:
|
||||
|
||||
```text
|
||||
manifest requests capability
|
||||
-> operator/CLI grants it
|
||||
-> broker asks: is a trusted handler installed?
|
||||
-> handler re-checks owner/grant/scope/budget
|
||||
-> handler executes
|
||||
-> audit { identity, effect, decision, duration, trace_id }
|
||||
```
|
||||
|
||||
A name in an allow-list must never be enough to execute, and a capability with
|
||||
no executor must never look like an empty success.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Two independent registries
|
||||
|
||||
`CapabilityBroker` keeps:
|
||||
|
||||
* `vendor_policy: BTreeSet<String>` — names the trusted host *policy* accepts
|
||||
(from `allow_vendor`); and
|
||||
* `handlers: BTreeMap<String, HandlerEntry>` — names that have an explicitly
|
||||
registered trusted executor.
|
||||
|
||||
`HandlerEntry` additionally carries an optional exact trusted package binding
|
||||
(`None` = authorized by the capability grant alone).
|
||||
|
||||
The two sets cannot substitute for each other:
|
||||
|
||||
* vendor capability without a policy entry -> `permission.denied`
|
||||
(declaration is not a grant; CAP-01);
|
||||
* vendor policy entry without a handler -> `capability.unavailable` (CAP-02);
|
||||
* core capability granted by `Capabilities` but without a handler ->
|
||||
`capability.unavailable` (CAP-12).
|
||||
|
||||
### 2. Handler interface
|
||||
|
||||
```rust
|
||||
pub struct CapabilityRequest<'a> {
|
||||
pub package_instance: &'a str, // trusted identity from the host, not wire data
|
||||
pub capability: &'a str,
|
||||
pub payload: &'a [u8],
|
||||
pub trace_id: &'a str,
|
||||
}
|
||||
|
||||
pub trait CapabilityHandler: Send {
|
||||
fn name(&self) -> &str;
|
||||
fn execute(
|
||||
&mut self,
|
||||
request: &CapabilityRequest<'_>,
|
||||
capabilities: &Capabilities,
|
||||
budget: &mut Budget,
|
||||
) -> Result<Vec<u8>, PluginError>;
|
||||
}
|
||||
```
|
||||
|
||||
The registry key is `handler.name()`, so the handler's declared capability can
|
||||
never disagree with the name it is registered under (the RFC's "declared
|
||||
protocol mismatch" rejection cannot occur by construction).
|
||||
|
||||
Registration is host-only; there is no wire path from a plugin manifest to this
|
||||
API. `register_handler` installs a handler usable by any package holding a
|
||||
grant; `register_package_handler(package, handler)` binds it to one exact
|
||||
trusted package identity and denies every other claimed package (CAP-03, the
|
||||
"trust the package the host loaded, not a self-reported name" rule of §7.4).
|
||||
|
||||
### 3. Registration validation
|
||||
|
||||
`register_handler` / `register_package_handler` reject:
|
||||
|
||||
| condition | stable code |
|
||||
|---|---|
|
||||
| empty name, surrounding whitespace, or not a core/vendor name | `capability.handler_invalid` |
|
||||
| name under a reserved host prefix (`marqdo.`, `host.`) | `capability.handler_reserved` |
|
||||
| name already registered (including the built-in `fs.read`) | `capability.handler_conflict` |
|
||||
|
||||
A rejected handler is never installed. The component that constructs the
|
||||
broker is the trust boundary: only trusted host/embedder code can register, so
|
||||
"untrusted source" is rejected by the absence of an API rather than a flag.
|
||||
|
||||
### 4. Fixed execution chain and audit
|
||||
|
||||
`execute()` is one function with one order:
|
||||
|
||||
1. `decide()` — `Capabilities`, vendor policy, request payload shape;
|
||||
2. handler installed? — otherwise audit `unavailable`, return
|
||||
`capability.unavailable`;
|
||||
3. package-bound handler owns the claimed package? — otherwise audit `denied`,
|
||||
return `permission.denied`;
|
||||
4. handler executes under the broker `Budget` and the broker reply limit;
|
||||
5. audit one entry with `package`, `capability`, `decision`
|
||||
(`granted`/`denied`/`unavailable`), reason and `duration_micros`.
|
||||
|
||||
`authorize()` remains the grant decision only (`granted`/`denied`); it never
|
||||
executes and therefore cannot demonstrate an executor. This keeps the
|
||||
authorization and execution questions separate and auditable.
|
||||
|
||||
The reply is bounded by `reply_limit` (default
|
||||
`MAX_CAPABILITY_REPLY_BYTES`) and oversize replies are `resource.exhausted`
|
||||
(CAP-06). Handlers also receive `&mut Budget`, so an embedder can wire the
|
||||
per-run budget and handlers can charge fuel/output (tested). Response payloads
|
||||
are never written to the audit log, so file contents and request secrets stay
|
||||
out of it (CAP-09).
|
||||
|
||||
### 5. `fs.read` becomes the first standard handler
|
||||
|
||||
The former inline `fs.read` match arm moved into `FsReadHandler`, installed by
|
||||
`CapabilityBroker::with_fs(fs)`. It:
|
||||
|
||||
* resolves the path with `ProjectFs::resolve` (absolute paths, `..`, empty
|
||||
paths rejected) **before** I/O;
|
||||
* re-checks `Capabilities::check_read` against the normalized
|
||||
project-relative path at the execution point, so the grant is enforced on the
|
||||
same path the filesystem uses;
|
||||
* reads through `ProjectFs` (regular file, UTF-8, project root, 8 MiB limit).
|
||||
|
||||
`with_fs` still installs exactly one handler; `capability_broker.rs` and
|
||||
`host.rs` call sites are unchanged. Every other core capability and every
|
||||
vendor capability without a registered handler keeps returning
|
||||
`capability.unavailable`; nothing returns an empty `Ok` as a stand-in for
|
||||
"implemented".
|
||||
|
||||
## RFC deviations (explicit)
|
||||
|
||||
1. **`EffectiveGrant` -> `Capabilities`; `ResourceBudget` -> `Budget`.** The
|
||||
RFC (§7.2) says these are design-interface names and the implementation must
|
||||
reuse the existing grant and limit structures instead of building a second
|
||||
permission system; that is exactly what happened.
|
||||
2. **`CapabilityError` -> `PluginError`.** The capability bridge across the
|
||||
plugin/worker ABI already uses `marqdo_plugin_abi::PluginError`; reusing it
|
||||
avoids a second error taxonomy. Codes remain the stable contract.
|
||||
3. **`trace_id` is plumbed through `CapabilityRequest` and settable with
|
||||
`with_trace_id`, but the host wiring is a follow-up.** `host.rs` does not set
|
||||
it yet, so audit entries from the current CLI plugin path have an empty
|
||||
trace id. Wiring it requires the run/session id from `ExecutionContext`,
|
||||
which is outside this PR's write scope.
|
||||
4. **`resource.exhausted` preserves the broker code but not the runtime
|
||||
category.** `PluginError` kind `Capability` maps to
|
||||
`RuntimeError` category `PermissionDenied` in `host.rs::map_plugin_error`.
|
||||
The stable code is `resource.exhausted` (what CAP-06 asserts at the broker
|
||||
boundary); teaching the host mapping about a resource-limit plugin code is a
|
||||
follow-up because `host.rs` is outside this PR's scope.
|
||||
5. **`CapabilityBroker` is no longer `Clone`** because it owns `Box<dyn
|
||||
CapabilityHandler>`. No in-tree caller cloned it; a future embedder that
|
||||
needs a clone should share the broker behind the same lock instead, which is
|
||||
also what the one-call-at-a-time ABI requires.
|
||||
|
||||
## Alternatives
|
||||
|
||||
1. Keep the closed match arm and add capabilities to it. Rejected: every
|
||||
vendor operation would need a core release and the broker would still have
|
||||
no separation between "granted" and "executable".
|
||||
2. Treat a vendor policy entry as proof of an executor. Rejected: that is the
|
||||
exact "declaration becomes privilege" failure the RFC forbids; CAP-02 locks
|
||||
it out.
|
||||
3. Register handlers by `(name, package)` strings only and trust the caller's
|
||||
package argument. Rejected: a wire-supplied package name must not authorize
|
||||
anything; package-bound registration pins the identity the host selected at
|
||||
load time.
|
||||
4. Give every core capability an empty handler so `execute()` always returns
|
||||
`Ok`. Rejected: it converts "not implemented" into a silent success, the
|
||||
failure mode RFC §7.1 calls out.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Existing `Capabilities`, `with_fs`, `with_audit`, `allow_vendor`,
|
||||
`authorize`, `execute` and `capability_audit()` call sites are unchanged;
|
||||
`host.rs` needed no edit.
|
||||
* One observable change: a vendor capability without a trusted policy entry is
|
||||
now `permission.denied` instead of `capability.unavailable`. This matches
|
||||
ADR-0011 ("a missing grant is `permission.denied`") and RFC CAP-01.
|
||||
* `CapabilityAudit` gained `duration_micros`; all fields remain public and the
|
||||
existing tests that read `package`/`capability`/`decision` compile unchanged.
|
||||
* `CapabilityBroker` lost `Clone` (see deviation 5).
|
||||
* No ABI, wire, manifest or CLI change.
|
||||
|
||||
## Security
|
||||
|
||||
* Registration is a trusted-host operation with no manifest/wire entry point;
|
||||
a plugin cannot install, replace or enumerate handlers.
|
||||
* Package-bound handlers reject cross-package impersonation even when policy
|
||||
and grant match.
|
||||
* `fs.read` keeps `ProjectFs` root confinement, glob scope and size limits; the
|
||||
grant is re-checked on the normalized path at the execution point.
|
||||
* Audit records decisions and durations but never payload contents.
|
||||
* This remains a cooperative in-process API contract, not an OS sandbox: a
|
||||
native plugin can still issue direct syscalls. The registry narrows what the
|
||||
host will *do on request*; it does not confine plugin memory or syscalls.
|
||||
|
||||
## Testing
|
||||
|
||||
`crates/marqdo-host/tests/capability_handlers.rs` covers CAP-01..CAP-12:
|
||||
policy-vs-handler separation, package binding, registration rejection, real
|
||||
`fs.read` scope enforcement, reply budget and handler budget charging, forged
|
||||
and replayed handles, audit hygiene, resource-stream regression smoke, project
|
||||
mode not being OS-strict, and "no handler is never a silent success".
|
||||
Unit tests in `capability.rs` keep the `CORE_CAPABILITIES` list in sync with
|
||||
`decide()` and assert that the built-in `fs.read` handler is installed by
|
||||
`with_fs`.
|
||||
|
||||
## Migration & rollout
|
||||
|
||||
`with_fs` installs the migrated `fs.read` handler automatically, so existing
|
||||
plugin loads keep working. Future core capabilities migrate by adding a handler
|
||||
and registering it in the trusted composition root; until then they remain
|
||||
`capability.unavailable`. Vendor operations that need a non-streaming API
|
||||
should first use the existing `resource.open/read/write/close` stream path
|
||||
(RFC-EXT-0001 §7.3) instead of registering a new host handler.
|
||||
@@ -0,0 +1,105 @@
|
||||
# ADR-0058: Official stdlib public aliases replace print magic
|
||||
|
||||
- Status: Accepted (2026-11-21)
|
||||
- Implements: RFC-EXT-0001 §8 (PR-06)
|
||||
- Related: ADR-0028/0030/0031/0039, ADR-0054
|
||||
|
||||
## Context
|
||||
|
||||
`打印`/`print` were implemented twice:
|
||||
|
||||
1. `crates/cli/src/loader.rs::program_uses_print` walked the entire AST, and if
|
||||
any function called `打印`/`print` it injected the bundled
|
||||
`stdlib/io.mq.md` module and set `Program.print_override` to its `写值`
|
||||
function;
|
||||
2. `crates/marqdo-runtime/src/interpreter.rs` contained a `打印`/`print`
|
||||
special case that redirected to the override, while the builtin table also
|
||||
contained a placeholder `打印` entry.
|
||||
|
||||
Copying this pattern to the next official high-level function (formatting,
|
||||
console input, …) would add one AST scanner and one runtime branch per name.
|
||||
The RFC forbids that ("为标准库每个高级函数复制 print_override 模式"), while
|
||||
also requiring byte-identical output and no new Host op.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Verified metadata.** The bundled `stdlib/stdlib.json` declares
|
||||
|
||||
```json
|
||||
"public_aliases": [
|
||||
{
|
||||
"public_names": ["print", "打印"],
|
||||
"target_module": "io.mq.md",
|
||||
"target_function": "写值",
|
||||
"expected_signature": "params(值)",
|
||||
"effects": ["host"]
|
||||
}
|
||||
],
|
||||
"aliases_digest": "sha256:<64 lowercase hex>"
|
||||
```
|
||||
|
||||
`aliases_digest` is the SHA-256 of the canonical binary encoding: for each
|
||||
alias sorted by `(target_module, target_function)`, the module bytes, NUL,
|
||||
the function bytes, NUL, then every public name in sorted order followed by
|
||||
NUL. The same encoding is implemented by `tools/pack-stdlib.py` and
|
||||
`crates/cli/src/stdlib.rs::aliases_digest`; `stdlib.json` cannot be edited
|
||||
without updating the digest, and the loader verifies both digests before any
|
||||
module loads. Metadata is integrity-checked, not signed: a SHA-256 proves
|
||||
content equality, not publisher identity.
|
||||
2. **Lazy, generic loading.** The loader reads the metadata, decides whether the
|
||||
program references any public alias name (a generic identifier scan, not a
|
||||
per-function hard code), and only then loads the declared target module.
|
||||
The target signature is checked against `expected_signature`; a mismatch is
|
||||
a load error. A program without `打印`/`print` keeps its previous module
|
||||
graph and startup cost.
|
||||
3. **Ordinary Marqdo binding.** For each used alias the loader produces an
|
||||
`OfficialAlias { public_names, function, signature, owner_package,
|
||||
effects_requested }`; `project_prepare` registers it in the
|
||||
`CallableRegistry`. The binding pass targets
|
||||
`Marqdo { function: <stdlib instance>::io.mq.md::写值 }`. The runtime calls
|
||||
an ordinary Markdown function; it has no print branch.
|
||||
4. **Reserved names stay reserved.** Official aliases may replace the builtin
|
||||
placeholder for the same public name. Third-party registry insertion rejects
|
||||
builtin *and* official-alias names (`ext.duplicate_callable`), so no plugin
|
||||
or imported module can claim `打印`/`print`, `std.*` or `marqdo.*`.
|
||||
`plugin.yaml` still rejects reserved export prefixes at manifest validation.
|
||||
5. **No Host print operation.** Output continues to flow through
|
||||
`io.写值` → `format.显示` → UTF-8 bytes → `Memory` →
|
||||
`resource.write(stdout)`. There is no `HostOp::Print` and no second
|
||||
formatter. Unbundled embedders that call the placeholder builtin receive
|
||||
`kernel.unavailable`.
|
||||
6. **Removed code.** `program_uses_print`, `stmts_use_print`,
|
||||
`expr_uses_print`, `Program.print_override`, the runtime redirect branch and
|
||||
the whole-AST print scanner are deleted. A structural test
|
||||
(`architecture_boundaries.rs::callable_binding_is_the_only_static_resolution_path`)
|
||||
fails if they reappear.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `打印` and `print` are now ordinary calls in Analysis/Trace: the call target
|
||||
is the stdlib function, `state` is `function`, and no synthetic `builtin`
|
||||
target is reported for the alias.
|
||||
* Output must stay byte-identical. `crates/cli/tests/stdlib_alias_routing.rs`
|
||||
plus the existing print/format/Trace goldens pin this; a diff is a blocking
|
||||
regression, not an acceptable "new design".
|
||||
* A tampered alias table (metadata digest or expected signature) is rejected
|
||||
before module loading; `stdlib.json` and the `.mq.md` tree remain separately
|
||||
digest-pinned, and `tools/pack-stdlib.py` refuses to build an inconsistent
|
||||
bundle.
|
||||
* The generic metadata path is the only place a future official high-level name
|
||||
needs to be declared locally. Any *new language syntax* still requires a
|
||||
language RFC; aliases cannot change `@exec`/`@knowledge` semantics.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
* **Keep `print_override` and add a second alias for another function.**
|
||||
Rejected: it multiplies hard-coded scans/branches and violates RFC-EXT-0001
|
||||
§8 and §14.10.
|
||||
* **Move the print formatter into the Host as a `Print` op.** Rejected: it
|
||||
would reintroduce an un-audited output path and duplicate the stdlib
|
||||
formatter; output is a resource write.
|
||||
* **Let third-party packages register `print`/`打印` aliases.** Rejected:
|
||||
official public names are a language surface, not an extension registry.
|
||||
* **Put the alias table in Rust `match` arms instead of metadata.** Rejected:
|
||||
the table must be versioned with the verified bundle so a stale/mixed bundle
|
||||
cannot run a different function.
|
||||
@@ -15,6 +15,15 @@ replace `spec/` or a recorded test log.
|
||||
> bundled `stdlib/` implement it. The workspace gate is green at `9c1723d`;
|
||||
> rollout evidence and historical banners are indexed in
|
||||
> [`../migration/README.md`](../migration/README.md).
|
||||
>
|
||||
> **Extensibility closure (RFC-EXT-0001).** The current tree implements the
|
||||
> single static callable binding (ADR-0054), official stdlib public aliases
|
||||
> replacing print magic (ADR-0058), third-party resource providers through the
|
||||
> normal CLI (ADR-0055), and the capability-handler registry (ADR-0057). The
|
||||
> extension selection/lock v4 composition and the frozen external-binary
|
||||
> evidence are tracked as remaining PR-04/PR-08 work, not claimed here. The
|
||||
> normative resolution contract is
|
||||
> [`spec/callable-resolution.md`](../../spec/callable-resolution.md).
|
||||
|
||||
## 1. Four state dimensions
|
||||
|
||||
@@ -25,7 +34,7 @@ platform".
|
||||
| Dimension | Current state | Source |
|
||||
|---|---|---|
|
||||
| Implemented | Rust workspace `0.1.0`: RFC-LANG-0002 Markdown/frontend/IR, v2 statement parser, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, migrated bundled stdlib, CLI composition root | `Cargo.toml`, `crates/`, `stdlib/` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **507 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline; fmt and clippy clean | `docs/STATUS.md` §0, `docs/evidence/2026-11-21-semantics-v2-green.md` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **584 passed, 0 failed** across 66 targets on Linux x86_64 in the RFC-EXT-0001 tree; fmt and clippy clean. Historical baseline: 507 passed at `9c1723d`. | `docs/STATUS.md`, `docs/evidence/2026-11-22-ext-implementation.md` |
|
||||
| Bundled stdlib released | In-repo bundle `marqdo-stdlib@0.1.0`, SHA-256 digest verified before loading; it is **not** an external registry package | `stdlib/stdlib.json`, `stdlib/README.md` |
|
||||
| Multi-platform verified | Linux local gate is the only executed lane here; Windows/macOS native lanes are **UNVERIFIED**, and the GitHub security lane remains a **VM-only** gate | `docs/STATUS.md` §1, §4 |
|
||||
|
||||
@@ -53,6 +62,12 @@ OS / external services / devices
|
||||
|
||||
CLI is only the composition root: parse arguments, load the project,
|
||||
assemble Host + Runtime, route output, exit with a status.
|
||||
`project_prepare::prepare` is the single finalisation step: it loads sources,
|
||||
reads static extension descriptors (never `dlopen`/spawn), builds the unique
|
||||
`CallableRegistry`, binds every call site once and hands the same
|
||||
`PreparedProgram` to `check`/`run`/`call`/`inspect`/Analysis. The runtime
|
||||
dispatches only by the resulting `CallBindingTable`; it never re-resolves a
|
||||
name or guesses a plugin.
|
||||
Callable Plugin / Foreign kernel / Tool Protocol are separate extension
|
||||
categories; they do not enter the standard library or the language IR.
|
||||
```
|
||||
@@ -112,7 +127,10 @@ review RFC, not a new crate per provider.
|
||||
| network endpoints | `tcp`/`udp`/`tcp-listener`/`tls` open + `resource.accept`; exact host grant, DNS after grant, TLS certificate validation | implemented (project mode) | `resource_streams` |
|
||||
| `std:net/http` | byte-exact HTTP/1.1 parse/build; chunk sizes are byte counts; text wrapper compatible | implemented (pure `.mq.md`) | `stdlib_http` |
|
||||
| HTTP/1.1 | `stdlib/net/http.mq.md`: status/header/chunked-byte parse, request build | implemented over an already-authorized TCP stream; not a DNS/TLS client | `stdlib_http`, `http_over_tcp`, `stdlib_product` |
|
||||
| print | bundled `io.mq.md`/`console.mq.md` through `resource.write`; legacy `HostOp::Print` removed | implemented (Compat-C) | `print_routing`, `stdlib_product` |
|
||||
| print / official stdlib aliases | `打印`/`print` are verified `stdlib.json` public aliases bound to `io.mq.md::写值`; ordinary Marqdo calls, no `print_override`, no runtime branch, no `HostOp::Print` | implemented (ADR-0058) | `stdlib_alias_routing`, `print_routing`, `stdlib_product` |
|
||||
| callable resolution | one registry + one `bind_program_calls` pass; `check`/`run`/`inspect`/Analysis share `CallSiteKey` bindings; plugin exports reconcile exactly with static manifests; no runtime name fallback | implemented (ADR-0054) | `callable_resolution`, `plugin_worker`, `analysis_trace` |
|
||||
| resource provider v2 CLI | `--provider`/`--allow-provider` select and authorize a `provider.json` schema-2 `c-abi-1` worker artifact; manifest/artifact/grants/duplicate kinds verified before spawn; sibling helper only | implemented linux/x86_64; other platforms fail closed (ADR-0055) | `provider_cli` |
|
||||
| capability handlers | trusted handler registry executes granted capabilities under budget/audit; missing handler is `capability.unavailable`; `fs.read` is the first real handler | implemented (ADR-0057) | `capability_handlers` |
|
||||
| strict mode | only the pure/Math execution path and the built-in stdout console channel; project/foreign/plugin/tool/trace/file/network grants are rejected | implemented L1; L2 not implemented | `tests/security`, `resource_streams`, `stdlib_product` |
|
||||
|
||||
## 4. Extension path classification
|
||||
@@ -120,9 +138,10 @@ review RFC, not a new crate per provider.
|
||||
| Extension path | Responsibility | State |
|
||||
|---|---|---|
|
||||
| Tool Protocol | external `marqdo` CLI tools, inspect/verify/IDE commands; versioned manifest and install/remove | implemented v1; not a standard-I/O replacement inside a user program |
|
||||
| Callable Plugin ABI | native callable functions and controlled experimental acceleration | implemented v1 (native + worker); not an ordinary JSON/CSV formatter |
|
||||
| Resource Provider | new resource kinds/devices/network/file endpoints | implemented v1 in-process, worker and trusted C ABI; not one provider per high-level algorithm |
|
||||
| Foreign kernel | explicit cross-language compatibility for Python/Node one-shot execution | implemented; not a Marqdo implementation of common algorithms |
|
||||
| Callable Plugin ABI | native callable functions and controlled experimental acceleration | implemented v1 (native + worker); static manifest is the contract and run reconciles exports exactly; not an ordinary JSON/CSV formatter |
|
||||
| Resource Provider | new resource kinds/devices/network/file endpoints | v2 CLI entry implemented for trusted worker transport on linux/x86_64 (ADR-0055); v1 in-process/trusted C ABI remain embedder APIs; not one provider per high-level algorithm |
|
||||
| Foreign kernel | explicit cross-language compatibility for Python/Node one-shot execution | implemented; registry accepts an explicitly declared unknown language; not a Marqdo implementation of common algorithms |
|
||||
| Official stdlib public alias | language-surface aliases such as `打印`/`print` declared in verified bundle metadata | implemented (ADR-0058); third parties cannot claim these names |
|
||||
| Shared registry/policy/budget/audit/trace | package/adapter identity, descriptor/version checks, per-request grants, timeouts, error classes and trace metadata | housed inside `marqdo-host`; do not invent a shared binary ABI for function calls and resource streams |
|
||||
|
||||
## 5. Gap and closure backlog
|
||||
@@ -146,6 +165,9 @@ that a phase is complete.
|
||||
| P3 | No concurrency tasks, atomics or cancellable long-running scheduling | open; separate ADR |
|
||||
| P3 | No native optimizer/JIT or columnar numeric backend | open; benchmark-first |
|
||||
| P3 | Windows Native/Worker ABI is unsupported; macOS/Windows CI is unverified | open |
|
||||
| P0 | RFC-EXT-0001 PR-04: `marqdo.yaml.extensions` + lock v4 extension selection, CLI composition-root wiring, `inspect --extensions` | **open / in progress** |
|
||||
| P0 | RFC-EXT-0001 PR-08: frozen release binary, external conformance rerun, SHA-bound evidence manifest, final docs/status convergence | **open / in progress** |
|
||||
| P1 | Provider worker transport only linux/x86_64; macOS/Windows provider lanes fail closed | open; separate platform work |
|
||||
|
||||
## 6. Documentation governance
|
||||
|
||||
@@ -184,3 +206,9 @@ that a phase is complete.
|
||||
- [ADR-0052: byte-exact HTTP framing](../adr/0051-byte-exact-http-framing.md) — accepted byte-list HTTP/1.1 parser/builder.
|
||||
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
|
||||
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
|
||||
- [RFC-EXT-0001](RFC-EXT-0001.md) — extensibility closure task specification (implementation status tracked here and in STATUS).
|
||||
- [ADR-0054: single callable binding](../adr/0054-callable-binding-single-source.md) — one registry/binding pass, runtime dispatch by identity.
|
||||
- [ADR-0055: resource provider CLI v2](../adr/0055-resource-provider-cli-v2.md) — provider v2 manifest/worker CLI closure and trust boundary.
|
||||
- [ADR-0057: capability handler registration](../adr/0057-capability-handler-registration.md) — declared capability vs installed handler vs audit.
|
||||
- [ADR-0058: stdlib public aliases](../adr/0058-stdlib-public-alias.md) — verified metadata replaces print magic.
|
||||
- [`spec/callable-resolution.md`](../../spec/callable-resolution.md) and [`spec/resource-provider-v2.md`](../../spec/resource-provider-v2.md) — normative extension boundaries.
|
||||
+42
@@ -72,3 +72,45 @@ cargo build --release --locked -p marqdo-cli
|
||||
The ignored `.secrets/dev.env` may contain a Woodpecker API token for local
|
||||
queries/triggers. The token is not needed by the pipeline itself and must never
|
||||
be committed.
|
||||
|
||||
## External extension evidence (RFC-EXT-0001 PR-07/PR-08)
|
||||
|
||||
A release candidate must also prove that a **frozen** CLI binary can use
|
||||
extensions from physically separate repositories without modifying or
|
||||
rebuilding this workspace. This is a new necessary condition; it does **not**
|
||||
replace the security/platform/performance gates above.
|
||||
|
||||
Required inputs:
|
||||
|
||||
* a release build of this commit: `cargo build --release --locked -p marqdo-cli`;
|
||||
* the provider helper shipped next to it for the same target:
|
||||
`marqdo-provider-worker` (the provider transport resolves only that sibling;
|
||||
no `PATH` fallback);
|
||||
* a clean checkout of the external fixture repository
|
||||
`https://git.cflmy.cn/cflmy/marqdo-extension-conformance`.
|
||||
|
||||
Generation and validation:
|
||||
|
||||
```bash
|
||||
CLI="$(realpath target/release/marqdo)"
|
||||
python3 tools/ci/verify_extension_contracts.py \
|
||||
--cli "$CLI" \
|
||||
--fixtures /path/to/marqdo-extension-conformance
|
||||
python3 tools/ci/verify_extension_contracts.py \
|
||||
--validate docs/evidence/extension/<40-hex-source-SHA>/manifest.json
|
||||
```
|
||||
|
||||
The generator hashes the CLI before and after the external suite, records the
|
||||
source SHA and fixture repo SHA, and refuses to write evidence for a changed
|
||||
binary, a dirty fixture checkout, a `FAIL`, or an unexplained
|
||||
`UNSUPPORTED`/`SKIPPED` result. The external suite itself
|
||||
(`tests/run_all.py` in that repository) uses only `$MARQDO_BIN` (absolute),
|
||||
never Cargo, and covers: pinned-Git `.mq.md` package, worker ABI plugin,
|
||||
unknown foreign kernel, provider v2 worker, and an independent Analysis/Trace
|
||||
consumer.
|
||||
|
||||
The schema is [`evidence/extension/schema.json`](evidence/extension/schema.json);
|
||||
the generator is `tools/ci/verify_extension_contracts.py`. A manifest whose
|
||||
`tested_sha` is not the release SHA is invalid. Platform-specific results must
|
||||
be reported as `UNSUPPORTED` (for example provider worker on non-linux/x86_64);
|
||||
a Linux PASS never substitutes for a macOS/Windows lane.
|
||||
@@ -0,0 +1,195 @@
|
||||
# RFC-EXT-0001 implementation evidence (P0 core + PR-03/05/06/07 first pass)
|
||||
|
||||
Status: **partial, honest record.** PR-00/01/02/03/05/06 are implemented and
|
||||
tested on Linux x86_64 in this tree; PR-04 (extension selection/lock v4) and the
|
||||
final frozen-release evidence are still in progress. This document is not a
|
||||
release claim. The authoritative executed status is regenerated in
|
||||
`docs/STATUS.md` and `docs/evidence/extension/<SHA>/manifest.json`.
|
||||
|
||||
## 1. Identity and environment
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Trees | `/root/work/marqdo-cli` (branch `feat/extensible-ecosystem`) and separate `/root/work/marqdo-extension-conformance` |
|
||||
| Baseline | `595279fe38048b52b96d9ff1e7d57a867ec5c4c6`; green baseline in `docs/evidence/ext-baseline-595279f.md` |
|
||||
| Rust | `rustc 1.100.0`, `cargo 1.100.0` |
|
||||
| OS / arch | Linux x86_64 |
|
||||
| Commands | logged under `.cache/ext/` (gitignored) |
|
||||
|
||||
No commit SHA is claimed for the implementation yet: work is uncommitted on the
|
||||
feature branch, so this file cannot be a same-SHA release artifact.
|
||||
|
||||
## 2. Executed workspace gate (current tree)
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS (exit 0)
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS (exit 0)
|
||||
cargo test --workspace --locked PASS (exit 0)
|
||||
66 test targets: 584 passed, 0 failed, 0 ignored
|
||||
```
|
||||
|
||||
Targeted new suites:
|
||||
|
||||
```text
|
||||
cargo test -p marqdo-cli --test callable_resolution 14 passed
|
||||
cargo test -p marqdo-cli --test stdlib_alias_routing 5 passed
|
||||
cargo test -p marqdo-cli --test provider_cli 12 passed
|
||||
cargo test -p marqdo-cli --test analysis_trace 7 passed
|
||||
cargo test -p marqdo-cli --test plugin_worker 4 passed
|
||||
cargo test -p marqdo-cli --test print_routing 2 passed
|
||||
cargo test -p marqdo-cli --test architecture_boundaries 9 passed
|
||||
cargo test -p marqdo-host (includes capability_handlers 12)
|
||||
```
|
||||
|
||||
## 3. What was implemented (source paths)
|
||||
|
||||
### PR-01/02 — one static binding, bound runtime dispatch
|
||||
|
||||
* `crates/marqdo-ir/src/call_binding.rs` — `CallSiteKey`, `BoundTarget`
|
||||
(`Builtin`/`Marqdo`/`AbiPlugin`/`DynamicValue`), deterministic
|
||||
`CallBindingTable`, unit tests.
|
||||
* `crates/marqdo-ir/src/callable.rs` — `try_from_program` (rejects duplicate
|
||||
exports instead of first-wins), `OfficialAlias` registration, dotted-prefix
|
||||
conflict helpers, `function_id`.
|
||||
* `crates/marqdo-frontend/src/calls.rs` — the single `bind_program_calls` pass,
|
||||
RFC priority matrix, import namespace conflicts, exact plugin signature
|
||||
validation; `validate_module` is a standalone-frontend wrapper.
|
||||
* `crates/marqdo-frontend/src/analyze.rs` — `analyze_source_for_program` defers
|
||||
call checking to the composition root; standalone `analyze_source` keeps its
|
||||
single-document checks for the conformance corpora.
|
||||
* `crates/marqdo-runtime/src/interpreter.rs` — dispatch by `CallSiteKey`; the
|
||||
unknown-plugin fallback, `print_override` branch and dead `resolve_function`
|
||||
are removed; bound ABI calls normalise positional arguments to manifest order.
|
||||
* `crates/cli/src/extension_descriptors.rs` — static plugin manifest reading
|
||||
(no `dlopen`/spawn), signature parsing (defaults rejected), bundle-relative
|
||||
path checks, artifact sha helpers.
|
||||
* `crates/cli/src/project_prepare.rs` — one `prepare()`: load → descriptors →
|
||||
registry → bind once → `PreparedProgram`; diagnostics are appended once.
|
||||
* `crates/cli/src/commands/{check,run,inspect}.rs` + `loader.rs` — all consume
|
||||
`prepare_project`; `run` verifies artifacts before load and reconciles the
|
||||
Host exports one-to-one (`ext.plugin_binding_mismatch` on any difference).
|
||||
* `crates/cli/src/analysis.rs` — Analysis `calls[].state/target` are projected
|
||||
from the same binding table (no second resolver).
|
||||
* `crates/cli/tests/callable_resolution.rs` — CALL-01/02/03/04/05/06/07/08/09/
|
||||
11/12/13/14/15/16 as black-box CLI tests (CALL-10 has unit coverage in
|
||||
`commands/run.rs` for missing/extra runtime exports).
|
||||
|
||||
Normative docs: `spec/callable-resolution.md`, `docs/adr/0054-*`,
|
||||
`spec/plugin-abi.md` §12, `spec/analysis-v2.md` calls note.
|
||||
|
||||
### PR-03 — resource provider v2 through the normal CLI
|
||||
|
||||
* `crates/cli/src/main.rs` — repeatable `--provider PATH` and
|
||||
`--allow-provider NAME` on `run`/`call` only.
|
||||
* `crates/cli/src/commands/sandbox.rs` — grants map to
|
||||
`allow_resource_provider`; `pure`/`strict` preflight refuse the flags;
|
||||
wildcards rejected.
|
||||
* `crates/cli/src/provider_manifest.rs` — strict schema v2 parser/verifier,
|
||||
target selection, bundle containment, sha256, reserved-kind rejection,
|
||||
duplicate kind/provider checks, sibling `marqdo-provider-worker` resolution
|
||||
(no `PATH` fallback; `ext.worker_unavailable`/`ext.platform_unsupported`).
|
||||
* `crates/cli/src/commands/run.rs` — verify all manifests/artifacts/grants and
|
||||
duplicate kinds before spawning; spawn only granted+verified providers; no
|
||||
native fallback; no new `HostOp`/`resource_request` branch.
|
||||
* Tests: `crates/cli/tests/provider_cli.rs` (real C sensor + real worker
|
||||
helper), 12 tests; `spec/resource-provider-v2.md`, ADR-0055, README section.
|
||||
|
||||
Provider evidence recorded by the PR-03 implementer:
|
||||
|
||||
| ID | Result | Evidence summary |
|
||||
|---|---|---|
|
||||
| PROV-01 | PASS | exit 0, `count=5`, `eof=true`, one worker/open/close marker |
|
||||
| PROV-02 | PASS | exit 1 `permission.denied`; helper never started |
|
||||
| PROV-03 | PASS | exit 2 `ext.provider_grant_unmatched`; no worker |
|
||||
| PROV-04 | PASS | exit 2 `ext.duplicate_provider_kind`; no worker |
|
||||
| PROV-05 | PASS | one provider, two kinds, counts `[5,2]`, both closed |
|
||||
| PROV-06 | PASS | `stdio/file/tcp/tls/udp/tcp-listener` rejected `ext.manifest_invalid` |
|
||||
| PROV-07 | PASS | sha mismatch / symlink escape / `..` → `ext.artifact_integrity` before spawn |
|
||||
| PROV-08 | PASS | `ext.no_matching_artifact` |
|
||||
| PROV-09 | PASS | v1 → `ext.unsupported_provider_schema`, no migration |
|
||||
| PROV-10 | PASS | pure `cli.invalid_grants`, strict `strict.unsupported_grant`, `*` rejected |
|
||||
| PROV-13 | PASS | sibling helper missing → `ext.worker_unavailable`, PATH helper unused |
|
||||
|
||||
Platform: provider worker transport is claimed **linux/x86_64 only**; macOS and
|
||||
Windows are explicitly `UNSUPPORTED`/fail-closed (`ext.platform_unsupported`),
|
||||
not claimed supported. PROV-11/12/14/15 belong to host-protocol and PR-07
|
||||
external closure and are not claimed by the PR-03 CLI matrix.
|
||||
|
||||
### PR-05 — capability handler registry
|
||||
|
||||
`crates/marqdo-host/src/capability.rs` now has a trusted handler directory,
|
||||
`fs.read` as a real handler, decide → handler-installed → execute under budget →
|
||||
audit with duration; missing handler stays `capability.unavailable`, duplicate/
|
||||
reserved/empty registrations are rejected. `crates/marqdo-host/tests/
|
||||
capability_handlers.rs` has 12 CAP tests (CAP-01…CAP-12); ADR-0057 and
|
||||
`spec/security.md` updated. `cargo test -p marqdo-host` PASS.
|
||||
|
||||
### PR-06 — official stdlib public aliases
|
||||
|
||||
`stdlib/stdlib.json` declares `public_aliases` (`print`/`打印` →
|
||||
`io.mq.md::写值`) and `aliases_digest`; `crates/cli/src/stdlib.rs` verifies the
|
||||
metadata digest and expected target signature; `loader.rs` loads the target
|
||||
module only when the alias name is referenced and produces `OfficialAlias`
|
||||
records; `project_prepare` registers them and the binding pass targets the
|
||||
ordinary `Marqdo` function. `program_uses_print`, `Program.print_override`, the
|
||||
runtime print branch and the AST scanner are deleted; the placeholder builtin
|
||||
still fails with `kernel.unavailable` for unbundled embedders. Tests:
|
||||
`crates/cli/tests/stdlib_alias_routing.rs` (5) + existing print/format/Trace
|
||||
goldens; the stdlib fixture aliases that collided with builtin namespaces
|
||||
(`table`/`text`/`fs`) were renamed and the bundle digest regenerated.
|
||||
`tools/pack-stdlib.py` verifies `aliases_digest` before packaging.
|
||||
ADR-0058 documents the decision.
|
||||
|
||||
### PR-07 (first pass) — independent external repository
|
||||
|
||||
`/root/work/marqdo-extension-conformance` is a separate Git repository (own
|
||||
history, remote `https://git.cflmy.cn/cflmy/marqdo-extension-conformance`,
|
||||
pushed; commit `b729afc` at the time of this record). It contains:
|
||||
|
||||
* `pure-lib/` + `consumer/` with a pinned full-SHA Git dependency;
|
||||
* `plugin/` — real C ABI v1 worker plugin, three exports under two prefixes;
|
||||
* `kernel/` — `mocklang` provider with a language the CLI does not know;
|
||||
* `provider/` — `provider.json` v2 + real C ABI sensor library;
|
||||
* `tool/` — stdlib-only Python consumer of Analysis v2 / Trace v2;
|
||||
* `tests/run_all.py`, `tests/verify_frozen_cli.py`, `CONTRACT.md`,
|
||||
`.github/workflows/conformance.yml`.
|
||||
|
||||
Provisional run against the current development binary (debug build, not a
|
||||
release candidate):
|
||||
|
||||
```text
|
||||
[PASS] plugin_worker, callable_registry, foreign_kernel,
|
||||
resource_provider_cli, package_pinned_git,
|
||||
tool_analysis_consumer, negative_security
|
||||
(all categories PASS; frozen CLI sha256 recorded in evidence/results.json)
|
||||
```
|
||||
|
||||
This is **not** final PR-07/PR-08 evidence: the binary was not a `--release`
|
||||
candidate, and `tools/ci/verify_extension_contracts.py` has not yet emitted a
|
||||
`docs/evidence/extension/<SHA>/manifest.json`. The external suite will be rerun
|
||||
against the frozen release binary after PR-04 lands.
|
||||
|
||||
## 4. Known gaps / deviations (not hidden)
|
||||
|
||||
1. **PR-04 not landed yet.** `marqdo.yaml.extensions` and lock v4 are still in
|
||||
progress; the CLI composition root for extension selection is not wired.
|
||||
2. **Provider platform claims are Linux x86_64 only**; macOS/Windows are
|
||||
explicitly unsupported for the worker transport. The RFC's mac/Windows
|
||||
provider lanes are not claimed.
|
||||
3. **Additive provider error codes** `ext.manifest_invalid`,
|
||||
`ext.duplicate_provider`, `ext.provider_grant_unmatched` were added because
|
||||
the RFC's frozen code list has no precise code for malformed v2, duplicate
|
||||
provider names, or an unmatched `--allow-provider`; documented in
|
||||
`spec/resource-provider-v2.md` and ADR-0055.
|
||||
4. **`--allow-provider` without a selected manifest** aborts with
|
||||
`permission.denied` before spawning rather than silently ignoring the grant
|
||||
(needed to satisfy PROV-02 with the existing spawn-on-register Host API).
|
||||
5. **Analysis v2 `calls[].state/target` semantics changed for official aliases**
|
||||
(print now projects as a `function` target). This is a projection of
|
||||
existing fields, documented in `spec/analysis-v2.md`/ADR-0054/ADR-0058;
|
||||
there is no v3 bump and the private binding table is not exposed.
|
||||
6. **CALL-10's host-side mismatches** are unit-tested for missing/extra exports;
|
||||
a live TOCTOU signature/digest mismatch is covered by artifact-tamper tests
|
||||
plus exact reconciliation, not by swapping a manifest mid-run.
|
||||
7. Full `--release` build, frozen external rerun, SHA-bound evidence manifest,
|
||||
docs/STATUS/CURRENT/COMPATIBILITY/CHANGELOG finalisation remain for PR-08.
|
||||
@@ -0,0 +1,82 @@
|
||||
# RFC-EXT-0001 baseline audit — `595279f`
|
||||
|
||||
Status: **baseline evidence recorded before implementation.** This is not a
|
||||
release claim and none of the new RFC-EXT-0001 behaviour existed at this SHA.
|
||||
|
||||
## 1. Identity and environment
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Repository | `cflmy/marqdo-cli` (private remote `https://git.cflmy.cn/cflmy/marqdo-cli`) |
|
||||
| Baseline commit | `595279fe38048b52b96d9ff1e7d57a867ec5c4c6` (`master`, 2026-11-21) |
|
||||
| Working branch | `feat/extensible-ecosystem` |
|
||||
| Rust | `rustc 1.100.0 (0ef1bc386 2026-11-09)` |
|
||||
| Cargo | `cargo 1.100.0 (3acc79c0c 2026-11-05)` |
|
||||
| OS / arch | Linux `x86_64` (Ubuntu container) |
|
||||
| Local homes | `RUSTUP_HOME=.rustup-home`, `CARGO_HOME=.cargo-home` |
|
||||
| Command log | `.cache/ext/baseline.log` (local, gitignored) |
|
||||
|
||||
## 2. Executed gate (same SHA, before any RFC-EXT-0001 edit)
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check exit 0 PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings exit 0 PASS
|
||||
cargo test --workspace --locked exit 0 PASS
|
||||
62 test targets: 521 passed, 0 failed, 2 ignored
|
||||
```
|
||||
|
||||
The RFC §2.2 static inventory commands were also run and their findings are
|
||||
recorded in §3. A green baseline is only the starting point, not evidence for
|
||||
any new feature.
|
||||
|
||||
## 3. Source facts at the baseline (RFC §2.1 refresh)
|
||||
|
||||
Each row was checked against the actual tree, not against the RFC prose. None of
|
||||
the rows was `already_closed` at `595279f`.
|
||||
|
||||
| ID | Observed at `595279f` | Impact | RFC treatment |
|
||||
|---|---|---|---|
|
||||
| EXT-01 | `marqdo-frontend/src/units.rs` `SemanticMode` is `Exec`/`Knowledge` only; unknown `@xxx` is `sem.invalid_mode_marker` | third parties cannot add an execution mode (intended closure) | keep; metadata is inert Markdown |
|
||||
| EXT-02 | `frontend/src/calls.rs` checks Member builtin before import; `runtime/src/interpreter.rs` checks import before builtin | check and run can bind different targets | PR-01/02 single binding |
|
||||
| EXT-03 | `ir/src/callable.rs` has `CallableRegistry` but calls still use `builtins::find` and runtime fallbacks | registry was not the single truth | PR-01/02 |
|
||||
| EXT-04 | interpreter tolerated undeclared `alias.name` as a plugin call when `external_callables` was empty | static-unknown symbol failed only at run time | PR-02 remove fallback |
|
||||
| EXT-05 | `cli/src/commands/run.rs` synthesised `params: Vec::new()` descriptors for loaded exports | manifest/runtime signature could differ silently | PR-02 exact reconciliation |
|
||||
| EXT-06 | `loader.rs` validated the program before the CLI appended static plugin descriptors, then validated again | duplicated diagnostics and ordering hazards | PR-01 single `prepare` finalize |
|
||||
| EXT-07 | `host/src/host/providers.rs` has Rust/C/Worker provider registration; CLI had no `--provider` | providers were only reachable by trusted embedders | PR-03 |
|
||||
| EXT-08 | `host/src/capability.rs::execute` implemented only `fs.read`; other names returned unavailable | declaring a capability did not mean executing it | PR-05 handler registry |
|
||||
| EXT-09 | `host/src/host.rs::resource_request` has per-string built-in operations | adding a non-streaming operation required core edits | PR-03/05; keep core operations |
|
||||
| EXT-10 | `cli/src/loader.rs::program_uses_print` injected `stdlib/io.mq.md` and set `Program.print_override`; runtime special-cased `打印/print` | migrating another stdlib function by copying this pattern would hard-code every name | PR-06 public aliases |
|
||||
| EXT-11 | `foreign/src/lib.rs::KernelRegistry` supports unknown languages; old `normalize_language` only knows Python/Node | registry is the execution path, legacy helper is display-only | keep registry as the only query |
|
||||
| EXT-12 | Plugin ABI v1 Native/Worker implemented; Wasm unsupported; Tool Protocol tested | do not rebuild plugin/tool platform | keep + add identity reconciliation |
|
||||
| EXT-13 | `package/src/lib.rs` supports path + pinned `git` + package instances; `stdlib/` is a verified bundle | real `.mq.md` source decoupling exists | keep + external black-box test |
|
||||
| EXT-14 | strict L1 permits only limited pure computation and stdout and rejects Worker/Foreign/Provider | project mode is not an OS sandbox | keep fail-closed; strict L2 is a separate RFC |
|
||||
|
||||
## 4. Issues reproduced while recording this baseline
|
||||
|
||||
They were reproduced while writing the RFC acceptance fixtures; all are fixed by
|
||||
the PR series recorded later in `docs/STATUS.md` and
|
||||
`docs/evidence/extension/<SHA>/manifest.json`:
|
||||
|
||||
1. **Duplicate resolution (EXT-02/03/04).** A fixture with an import alias that
|
||||
matched a builtin dotted namespace (`import: {math: ./math.mq.md}` plus
|
||||
`math.eval(...)`) had no deterministic binding rule; the frontend checked the
|
||||
builtin while the runtime checked the import first.
|
||||
2. **Runtime plugin guessing (EXT-04).** `solver.add(...)` without `--plugin`
|
||||
reached the runtime and attempted `PluginInvoke` instead of failing during
|
||||
`check` with `symbol.unknown`.
|
||||
3. **Synthesised plugin signatures (EXT-05).** `run` could execute a plugin
|
||||
export whose static descriptor had an empty parameter list, so an arity or
|
||||
position/name mismatch was only discovered (or not) at the ABI boundary.
|
||||
4. **Print magic (EXT-10).** `打印`/`print` were routed by a whole-AST
|
||||
`program_uses_print` scan plus a runtime name special case; there was no
|
||||
generic de-magicking path for a second official stdlib function.
|
||||
5. **Provider CLI gap (EXT-07).** There was no `--provider`/`--allow-provider`;
|
||||
an external `provider.json` could not be used through the shipped binary.
|
||||
|
||||
## 5. Not claimed by this baseline
|
||||
|
||||
* No `--provider`/`--allow-provider` existed; strict L2 remains unimplemented.
|
||||
* No frozen-binary external repository evidence existed.
|
||||
* No `docs/evidence/extension/<SHA>/manifest.json` existed.
|
||||
* G3 remote security and G4 performance status is unchanged by this document;
|
||||
see `docs/STATUS.md` and `docs/releasing.md`.
|
||||
@@ -0,0 +1,32 @@
|
||||
# External-extension evidence
|
||||
|
||||
RFC-EXT-0001 §10.2/§11.3 requires SHA-bound proof that an **unchanged** CLI
|
||||
binary can use extensions from physically separate Git repositories. This
|
||||
directory holds that evidence.
|
||||
|
||||
* `schema.json` — JSON Schema for the evidence manifest.
|
||||
* `<40-hex-source-SHA>/manifest.json` — generated by
|
||||
`tools/ci/verify_extension_contracts.py`; never hand-written.
|
||||
|
||||
Generation (from a clean fixture checkout and a release build of this repo):
|
||||
|
||||
```bash
|
||||
cargo build --release --locked -p marqdo-cli
|
||||
CLI="$(realpath target/release/marqdo)"
|
||||
python3 tools/ci/verify_extension_contracts.py \
|
||||
--cli "$CLI" \
|
||||
--fixtures /path/to/marqdo-extension-conformance
|
||||
python3 tools/ci/verify_extension_contracts.py \
|
||||
--validate docs/evidence/extension/<SHA>/manifest.json
|
||||
```
|
||||
|
||||
The generator refuses to write evidence when:
|
||||
|
||||
* the CLI binary hash changes between the start and end of the suite;
|
||||
* the fixture repository has uncommitted changes;
|
||||
* a required result is missing or `FAIL`;
|
||||
* `UNSUPPORTED`/`SKIPPED` is recorded without `--allow-unverified` and a report
|
||||
entry explaining the platform limitation.
|
||||
|
||||
A local or container run is **not** a substitute for the release gates in
|
||||
`docs/releasing.md` (G3/G4), and no tag is created by this evidence.
|
||||
@@ -0,0 +1,63 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://marqdo.dev/schemas/extension-evidence-v1.json",
|
||||
"title": "Marqdo external-extension evidence manifest",
|
||||
"description": "SHA-bound evidence that independent repositories can use the frozen CLI without modifying or recompiling marqdo-cli (RFC-EXT-0001 §11.3). Generated by tools/ci/verify_extension_contracts.py; never hand-edited.",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"schema",
|
||||
"schema_version",
|
||||
"tested_sha",
|
||||
"cli_binary_sha256",
|
||||
"fixture_repo_sha",
|
||||
"os",
|
||||
"arch",
|
||||
"results",
|
||||
"commands",
|
||||
"failures",
|
||||
"unverified"
|
||||
],
|
||||
"properties": {
|
||||
"schema": { "const": "marqdo.extension-evidence" },
|
||||
"schema_version": { "const": 1 },
|
||||
"tested_sha": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{40}$",
|
||||
"description": "Exact marqdo-cli source commit the frozen binary was built from."
|
||||
},
|
||||
"cli_binary_sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
},
|
||||
"fixture_repo_sha": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{40}$"
|
||||
},
|
||||
"os": { "type": "string", "enum": ["linux", "macos", "windows"] },
|
||||
"arch": { "type": "string" },
|
||||
"results": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "string",
|
||||
"enum": ["PASS", "FAIL", "UNSUPPORTED", "SKIPPED"]
|
||||
},
|
||||
"required": [
|
||||
"callable_registry",
|
||||
"plugin_worker",
|
||||
"foreign_kernel",
|
||||
"resource_provider_cli",
|
||||
"package_pinned_git",
|
||||
"tool_analysis_consumer",
|
||||
"negative_security"
|
||||
]
|
||||
},
|
||||
"commands": {
|
||||
"type": "array",
|
||||
"items": { "type": "string" },
|
||||
"minItems": 1
|
||||
},
|
||||
"failures": { "type": "array", "items": { "type": "string" } },
|
||||
"unverified": { "type": "array", "items": { "type": "string" } }
|
||||
}
|
||||
}
|
||||
@@ -129,3 +129,33 @@ self-hosted G-C URL. See
|
||||
delete the failing test.
|
||||
- If the security gate regresses: set strict back to `Unavailable`, publish a
|
||||
security note, and do not claim isolation until the lane is green again.
|
||||
|
||||
## 7. Extension-ecosystem gate (RFC-EXT-0001)
|
||||
|
||||
Before a tag, in addition to §3 and §4:
|
||||
|
||||
1. `cargo build --release --locked -p marqdo-cli` and build
|
||||
`-p marqdo-provider-worker`; ship both binaries in the release archive for
|
||||
the same target. A provider request without the sibling helper must fail
|
||||
with `ext.worker_unavailable` — never fall back to `PATH`.
|
||||
2. Run the external frozen-binary suite from
|
||||
`https://git.cflmy.cn/cflmy/marqdo-extension-conformance` against the exact
|
||||
release binaries and generate
|
||||
`docs/evidence/extension/<40-hex-source-SHA>/manifest.json` with
|
||||
`tools/ci/verify_extension_contracts.py`. The manifest must validate against
|
||||
`docs/evidence/extension/schema.json`, its `tested_sha` must equal the
|
||||
tagged commit, and every mandatory result must be `PASS` on the claimed
|
||||
platform. `UNSUPPORTED` is allowed only for a platform explicitly excluded
|
||||
from the support matrix in `COMPATIBILITY.md`.
|
||||
3. `check`/`run`/`inspect`/Analysis share one prepared call binding
|
||||
(`spec/callable-resolution.md`); `check` never loads an extension binary;
|
||||
plugin exports reconcile exactly with static manifests; official stdlib
|
||||
aliases are verified metadata with byte-identical print output.
|
||||
4. Existing gates remain mandatory and unchanged: G3 strict isolation evidence,
|
||||
G4 measured performance on the fixed node, native platform lanes, schema
|
||||
consumers and `tools/ci/release_evidence.py`. The extension evidence is an
|
||||
additional gate; it never substitutes for them, and local/container smoke is
|
||||
not a release claim.
|
||||
5. If any extension gate is missing or mismatched, the release decision is
|
||||
**DO NOT RELEASE**. Tagging and asset publication still require explicit
|
||||
owner approval.
|
||||
@@ -95,6 +95,13 @@ a separate statement. `imports`, `references` and `calls` remain static facts; a
|
||||
Markdown reference is not a call. `effects` remains `["unknown"]` (a
|
||||
conservative summary, not a purity proof).
|
||||
|
||||
`calls[]` is projected from the single prepared call binding
|
||||
(`spec/callable-resolution.md` §6): `state` remains
|
||||
`function`/`builtin`/`plugin`/`unresolved`, and official stdlib aliases such as
|
||||
`打印` are reported as `function` targets because that is the implementation
|
||||
that runs. The private `CallBindingTable` is not serialized; Analysis v2's
|
||||
schema and version are unchanged.
|
||||
|
||||
An `assign` node keeps the v1 string `target` for a local rebinding. Addressable
|
||||
memory adds one additive target form (ADR-0022); consumers that only understand
|
||||
local assignments can ignore an object target:
|
||||
|
||||
@@ -0,0 +1,281 @@
|
||||
# Marqdo callable resolution and binding
|
||||
|
||||
Status: **normative for the current implementation** (ADR-0054/ADR-0058,
|
||||
RFC-EXT-0001 §4). This document describes the single static resolution used by
|
||||
`check`, `run`, `call`, `inspect` and Analysis. There is no second priority
|
||||
order and no runtime name guessing.
|
||||
|
||||
## 1. Identity
|
||||
|
||||
A call is an expression (`Expr::Call`). Each statically visible call site has a
|
||||
deterministic key:
|
||||
|
||||
```text
|
||||
CallSiteKey = (module id, function id, span.start, span.end)
|
||||
```
|
||||
|
||||
The key is independent of the checkout path, package cache location,
|
||||
registration order and map iteration. For a signature-default expression the
|
||||
owning function is the function that declares the default, because the runtime
|
||||
evaluates defaults in that function's empty frame.
|
||||
|
||||
A binding is one of:
|
||||
|
||||
| Bound target | Meaning |
|
||||
|---|---|
|
||||
| `Builtin { canonical_name }` | fixed language builtin, addressed by its canonical name (never by a user alias) |
|
||||
| `Marqdo { function }` | exact `FunctionId` of a Markdown-defined function, including an official bundled-stdlib public alias |
|
||||
| `AbiPlugin { package, export, digest, param_names }` | exact plugin export; `digest` is the selected, verified artifact SHA-256 and `param_names` is the manifest order used to normalise positional arguments |
|
||||
| `DynamicValue` | the callee is a language value (for example a `FunctionRef`); resolved at run time from the value, never by name |
|
||||
|
||||
`CallBindingTable` is private IR. It is not serialized into Analysis or Trace.
|
||||
The public `calls[].state/target` projection is derived from it (see §6).
|
||||
|
||||
## 2. Resolution matrix
|
||||
|
||||
### 2.1 Unqualified `f(...)`
|
||||
|
||||
In this exact order; the first applicable step decides:
|
||||
|
||||
1. **Local variable.** If `f` is a parameter or has been assigned anywhere in
|
||||
the function body, the site binds `DynamicValue`. At run time the value must
|
||||
be callable (`FunctionRef` in this release); a `Text`/`Map`/etc. value is
|
||||
`call.not_callable`. This is a language variable rule, not an escape hatch to
|
||||
a plugin.
|
||||
2. **Official alias or reserved builtin.** The registry's builtin table contains
|
||||
fixed builtins and, when the verified bundle is present and the program uses
|
||||
them, official stdlib aliases. An official alias wins over the placeholder
|
||||
builtin of the same name. Third parties cannot insert names here.
|
||||
3. **Current namespace function.** If the enclosing level-1 namespace `N`
|
||||
declares `N.f`, that function is chosen.
|
||||
4. **Canonical function name.** A function whose canonical name is exactly `f`.
|
||||
5. **Unique local name.** A function whose `local_name` is `f` in exactly one
|
||||
namespace. If two namespaces define the same local name and none of steps
|
||||
1–4 applies, the call is an error (`symbol.unknown`, message reports the
|
||||
ambiguity); it is never "first in document order".
|
||||
6. Otherwise `symbol.unknown`.
|
||||
|
||||
### 2.2 Qualified `alias.f(...)`
|
||||
|
||||
1. **Local variable named `alias`.** If `alias` is a local value, the site binds
|
||||
`DynamicValue`; `alias.f` is ordinary language member access. This cannot
|
||||
become a plugin call.
|
||||
2. **Explicit import.** If the module declares `import: { alias: <target> }`,
|
||||
the call binds only a function of that imported module (canonical name first,
|
||||
then unique local name). A missing function is `symbol.unknown`; it never
|
||||
falls back to a builtin or plugin with the same dotted spelling.
|
||||
3. **Builtin dotted namespace.** Otherwise, an exact builtin
|
||||
`alias.f` (for example `math.eval`) binds `Builtin`.
|
||||
4. **Exact plugin export.** Otherwise, if a selected static plugin declares the
|
||||
complete name `alias.f`, the site binds that export. The manifest signature
|
||||
is validated here; no library is loaded.
|
||||
5. Otherwise `symbol.unknown`.
|
||||
|
||||
`alias` in step 1/2 must be exactly an explicit import or a local value; there
|
||||
is no implicit namespace search across packages.
|
||||
|
||||
### 2.3 Other callees
|
||||
|
||||
`some_value.f(...)`, `f_list[i](...)`, `(expr)(...)` and any other non-Ident,
|
||||
non-member-of-Ident callee bind `DynamicValue`. Only language callables can be
|
||||
invoked at run time.
|
||||
|
||||
### 2.4 Import namespace conflicts
|
||||
|
||||
Before any call is bound, each module's import aliases are checked against:
|
||||
|
||||
* builtin dotted namespaces (`alias.` must not prefix any builtin name), and
|
||||
* visible plugin dotted namespaces (`alias.` must not prefix any selected
|
||||
plugin export).
|
||||
|
||||
A collision is `ext.import_namespace_conflict` at the front-matter import span,
|
||||
with a rename hint. Example: importing `./math.mq.md` as `math` conflicts with
|
||||
the builtin namespace `math.`; rename the alias to `mymath`. This is a
|
||||
deliberate 0.x breaking change with no compatibility switch (ADR-0054).
|
||||
|
||||
### 2.5 Duplicate callables
|
||||
|
||||
The static table rejects:
|
||||
|
||||
* the same complete plugin export declared by two selected packages
|
||||
(`ext.duplicate_callable` at prepare);
|
||||
* a plugin export that equals a builtin name or an official alias
|
||||
(`ext.duplicate_callable`);
|
||||
* two different manifests for the same plugin package identity in one selection
|
||||
(`ext.duplicate_callable`); the exact same package identity is idempotent;
|
||||
* a malformed/ambiguous static table entry (`ext.duplicate_callable`).
|
||||
|
||||
Order of `--plugin` flags never decides which declaration wins.
|
||||
|
||||
## 3. Argument checking
|
||||
|
||||
### 3.1 Markdown functions and builtins
|
||||
|
||||
`CallableSignature::validate_args` enforces:
|
||||
|
||||
* named arguments must exist exactly once; unknown names are `call.arity`;
|
||||
* positional arguments require an explicit language signature and map to the
|
||||
declared parameter order; overflow is `call.arity`;
|
||||
* required parameters without defaults must be supplied; missing is
|
||||
`call.arity`;
|
||||
* inferred signatures accept named arguments only (`call.arity` otherwise).
|
||||
|
||||
Messages are deterministic and identical between `check` and `run`.
|
||||
|
||||
### 3.2 ABI plugin exports
|
||||
|
||||
* The static manifest signature is authoritative. Unknown, duplicate, missing
|
||||
or excess arguments are rejected at prepare/bind time (`call.arity`); the
|
||||
plugin is never spawned for a statically invalid call.
|
||||
* ABI v1 transports arguments as a named map. Positional arguments are
|
||||
statically validated and deterministically normalised to the manifest's
|
||||
declared parameter order at run time. Example: `solver.add(1, b=2)` becomes
|
||||
`{a:1,b:2}`.
|
||||
* Manifest signature default values are rejected in ABI v1 until the wire
|
||||
semantics are versioned; no default is injected at run time.
|
||||
* A plugin that is declared but not selected (`--plugin` missing) is
|
||||
`symbol.unknown`; a selected plugin without an exact export descriptor is
|
||||
`symbol.unknown`. There is no runtime fallback.
|
||||
|
||||
## 4. Run-time reconciliation
|
||||
|
||||
`run`/`call` performs, in order:
|
||||
|
||||
1. static prepare (sources, manifests, signatures, binding) — no binary loaded;
|
||||
2. verify each selected artifact path, target and SHA-256;
|
||||
3. verify the `--allow-plugin` grant;
|
||||
4. load exactly the selected artifacts;
|
||||
5. compare the Host's actual exports with the static descriptors one-to-one
|
||||
(`name`, raw signature, artifact digest, owner package).
|
||||
|
||||
A mismatch in step 5 is `ext.plugin_binding_mismatch`, reported before the
|
||||
entry function runs. `run` must never synthesise a missing descriptor or an
|
||||
empty signature.
|
||||
|
||||
## 5. `check` versus `run`
|
||||
|
||||
`check` and `run` consume the same `PreparedProgram` binding table. Therefore:
|
||||
|
||||
* the same call site has the same `BoundTarget` in both;
|
||||
* a statically invalid call fails `check` and `run` with the same diagnostic
|
||||
code and message; `run` refuses before any Host side effect;
|
||||
* `check --plugin` reads manifests and signatures only: it never `dlopen`s a
|
||||
library, spawns a worker or executes a side-effect script; a missing plugin
|
||||
binary is not an error for `check`;
|
||||
* `--emit` Analysis is derived from the same binding (no second resolver).
|
||||
|
||||
## 6. Analysis projection
|
||||
|
||||
`calls[]` keeps the v2 fields `{function, span, callee, state, target}`.
|
||||
`state` is one of `function`, `builtin`, `plugin`, `unresolved`; it is now
|
||||
derived from the binding table:
|
||||
|
||||
| Bound target | `state` | `target` |
|
||||
|---|---|---|
|
||||
| `Builtin` | `builtin` | canonical builtin name |
|
||||
| `Marqdo` | `function` | exact `FunctionId` string |
|
||||
| `AbiPlugin` | `plugin` | complete export name |
|
||||
| `DynamicValue` / unbound | `unresolved` | `null` |
|
||||
|
||||
Official stdlib aliases (for example `打印`) therefore project as `function`
|
||||
targets, because that is the implementation that runs. This is a projection of
|
||||
the existing fields, not a new public schema. The private `CallBindingTable` is
|
||||
never exposed; a future explicit `resolved_call_target` field would require an
|
||||
Analysis version bump.
|
||||
|
||||
## 7. Diagnostics
|
||||
|
||||
| Code | Stage | Trigger |
|
||||
|---|---|---|
|
||||
| `symbol.unknown` | prepare/bind | no target for the call; ambiguity; unknown imported function; undeclared plugin export |
|
||||
| `call.arity` | prepare/bind | bad argument name/number/duplication, invalid positional use |
|
||||
| `ext.import_namespace_conflict` | prepare | import alias collides with a builtin/plugin dotted namespace |
|
||||
| `ext.duplicate_callable` | prepare | duplicate complete export, builtin shadowing, conflicting package manifests |
|
||||
| `ext.plugin_binding_mismatch` | run init | loaded exports/signatures/owner/digest differ from static descriptors |
|
||||
| `ext.no_matching_artifact` | manifest read | plugin/provider has no artifact for the current target |
|
||||
| `ext.artifact_integrity` | manifest read / run init | sha256 mismatch, path traversal, non-regular file, symlink escape |
|
||||
| `call.not_callable` | run | dynamic value is not callable |
|
||||
| `internal.error` | run | a call site has no binding (a prepared program must always be complete) |
|
||||
|
||||
Existing codes are reused rather than duplicated; `ext.capability_handler_unavailable`
|
||||
is internal-only — the public wire code remains `capability.unavailable`.
|
||||
|
||||
## 8. Examples
|
||||
|
||||
### 8.1 Imported function, no conflict
|
||||
|
||||
```markdown
|
||||
---
|
||||
import:
|
||||
mymath: "./math.mq.md"
|
||||
---
|
||||
|
||||
## @exec main
|
||||
|
||||
`return mymath.eval(a=1)`
|
||||
```
|
||||
|
||||
Binds `Marqdo { function: <package instance>::math.mq.md::eval }` in both
|
||||
`check` and `run`.
|
||||
|
||||
### 8.2 Builtin without an import
|
||||
|
||||
```markdown
|
||||
## @exec main
|
||||
|
||||
`return math.eval(ref("#公式"), {"r": 2}, target="A")`
|
||||
```
|
||||
|
||||
Binds `Builtin { canonical_name: "math.eval" }`.
|
||||
|
||||
### 8.3 Import namespace conflict
|
||||
|
||||
```markdown
|
||||
---
|
||||
import:
|
||||
math: "./math.mq.md"
|
||||
---
|
||||
|
||||
## @exec main
|
||||
|
||||
`return math.eval(a=1)`
|
||||
```
|
||||
|
||||
`check` fails with `ext.import_namespace_conflict`; `run` refuses before
|
||||
executing anything. Rename the alias.
|
||||
|
||||
### 8.4 Plugin export
|
||||
|
||||
```markdown
|
||||
## @exec main
|
||||
|
||||
`return solver.add(1, b=2)`
|
||||
```
|
||||
|
||||
* without `--plugin solver/plugin.yaml` or with a manifest that does not
|
||||
declare `solver.add`: `symbol.unknown` (no Host fallback);
|
||||
* with the manifest: binds `AbiPlugin { package: "demo.solver", export:
|
||||
"solver.add", digest: "<sha256>", param_names: ["a","b"] }`;
|
||||
* with a changed artifact byte: `ext.artifact_integrity` before the entry
|
||||
function;
|
||||
* with a missing `b`: `call.arity` at prepare.
|
||||
|
||||
### 8.5 Dynamic `FunctionRef`
|
||||
|
||||
```markdown
|
||||
## @exec 平方
|
||||
|
||||
`params(n: Int)`
|
||||
|
||||
`return n * n`
|
||||
|
||||
## @exec main
|
||||
|
||||
`f = ref("#平方")`
|
||||
|
||||
`return f(n=4)`
|
||||
```
|
||||
|
||||
The call site binds `DynamicValue`; the runtime calls the `FunctionRef` value.
|
||||
A `Text` value in the same position fails `call.not_callable` and never reaches
|
||||
the Host.
|
||||
@@ -459,3 +459,33 @@ when no C compiler is available.
|
||||
| Positional plugin arguments | Not supported; named only. |
|
||||
| Thread-safe / concurrent invocation | Not supported (single-threaded v1 contract). |
|
||||
| Old `marqdo_abi.h` v2 compatibility | Not implemented and not claimed; a separate adapter would be required. |
|
||||
|
||||
## 12. Static descriptor / run-time reconciliation (RFC-EXT-0001 §4.5)
|
||||
|
||||
The CLI never derives a callable signature from a loaded library. The
|
||||
**manifest is the static contract** and the runtime is checked against it:
|
||||
|
||||
1. `check --plugin <manifest>` reads and validates the manifest, including every
|
||||
export name and signature; it does not load the artifact and does not need
|
||||
the artifact file to exist.
|
||||
2. `run`/`call` first verifies the selected artifact path, host target and
|
||||
SHA-256, then checks the plugin grant, then loads exactly the selected
|
||||
artifacts.
|
||||
3. After loading, the Host's actual exports are compared one-to-one with the
|
||||
static descriptors: complete name, raw signature string, owner package and
|
||||
verified artifact digest. Extra, missing, renamed or differently-signed
|
||||
exports fail with `ext.plugin_binding_mismatch` before the entry function
|
||||
runs.
|
||||
4. The binding pass validates call arguments against the manifest signature.
|
||||
ABI v1 transports a named map, so positional arguments are statically
|
||||
validated and deterministically normalised to manifest order
|
||||
(`solver.add(1, b=2)` → `{a:1,b:2}`). Unknown, duplicate, missing or excess
|
||||
arguments are `call.arity` before any process is started.
|
||||
5. Manifest signatures with default values are rejected in ABI v1 until the
|
||||
default-value wire semantics are versioned. The host never injects a default.
|
||||
6. Duplicate complete export names across selected packages, or exports that
|
||||
shadow a builtin/official name, are `ext.duplicate_callable` at prepare.
|
||||
Selection order never decides which declaration wins.
|
||||
|
||||
None of this changes ABI v1's C layout or wire format; it closes the static
|
||||
side of the contract described by `spec/callable-resolution.md`.
|
||||
@@ -0,0 +1,287 @@
|
||||
# Marqdo Resource Provider v2 (`provider.json`)
|
||||
|
||||
Status: **frozen provider manifest schema version 2** for the `marqdo` CLI
|
||||
composition root. Implementation: `crates/cli/src/provider_manifest.rs`;
|
||||
prerequisite host transports: ADR-0033/0035/0036/0037/0038; CLI decision:
|
||||
ADR-0055. The worker helper is `marqdo-provider-worker`
|
||||
(`crates/marqdo-provider-worker`, ADR-0038).
|
||||
|
||||
> **A trusted helper is not a sandbox.** The provider `worker` transport runs a
|
||||
> separate process with the invoking user's OS privileges. It isolates a crash
|
||||
> or protocol failure from the interpreter; it does **not** restrict file
|
||||
> system, network, process or memory access. `sha256` values are content
|
||||
> *integrity* checks, not publisher authentication. `strict` L1 refuses every
|
||||
> provider (fail-closed) until a real isolation adapter exists.
|
||||
|
||||
## 1. CLI surface
|
||||
|
||||
```text
|
||||
marqdo run FILE [--provider PATH/provider.json ...] [--allow-provider NAME ...]
|
||||
marqdo call FILE FUNCTION [--arg NAME=VALUE ...] \
|
||||
[--provider PATH/provider.json ...] [--allow-provider NAME ...]
|
||||
marqdo check FILE # no --provider/--allow-provider options exist
|
||||
```
|
||||
|
||||
* `--provider PATH` is repeatable. `PATH` names a `provider.json` file
|
||||
(directories are rejected). Only explicitly passed manifests are considered;
|
||||
there is no directory scan, package-manager discovery, registry download or
|
||||
network fetch.
|
||||
* `--allow-provider NAME` is repeatable and matches the manifest `name`
|
||||
**exactly**. There is no `*`/glob wildcard, prefix match or implicit
|
||||
authorization. Selecting a provider with `--provider` never grants it.
|
||||
* `check`/`inspect` do not accept these options and never read a provider
|
||||
manifest, start a helper or call provider code. Provider kinds are not new
|
||||
callables; they are values passed to `resource.open(kind, ...)`.
|
||||
* `--mode pure` and `--mode strict` reject `--provider`/`--allow-provider` in
|
||||
preflight (exit 2, no helper spawn, no silent downgrade to `project`).
|
||||
`pure` reports `cli.invalid_grants`; `strict` reports
|
||||
`strict.unsupported_grant`.
|
||||
|
||||
### 1.1 Ordering
|
||||
|
||||
`run`/`call` follow RFC-EXT-0001 §18.2:
|
||||
|
||||
1. T1–T5: load the Markdown graph and prepare call bindings (unchanged;
|
||||
providers are not part of `prepare`).
|
||||
2. T6: parse all `--provider` manifests, validate schema/name/version/ABI/
|
||||
transport/kinds/artifact declarations, select and verify the current-target
|
||||
artifact (regular file, inside the bundle, size bound, sha256), reject
|
||||
duplicate provider names and duplicate kinds across providers, and check
|
||||
the exact `--allow-provider` grants.
|
||||
3. T6b: verify all selected plugin manifests/artifacts (existing behavior).
|
||||
4. T7: start plugin workers and reconcile exports, then spawn one
|
||||
`marqdo-provider-worker` per granted+verified provider.
|
||||
5. T9: execute the entry function; every `resource.open` re-checks the provider
|
||||
name grant in the Host.
|
||||
|
||||
No worker process (plugin or provider) is started before every T6/T6b check
|
||||
passes. A manifest, target, integrity, grant or duplicate-kind failure leaves
|
||||
the main function unexecuted.
|
||||
|
||||
An `--allow-provider NAME` that matches no selected manifest is a deterministic
|
||||
option error (`ext.provider_grant_unmatched`, exit 2) rather than silent
|
||||
discovery fan-out.
|
||||
|
||||
## 2. Manifest file
|
||||
|
||||
* File name convention: `provider.json`. It must be a UTF-8 JSON object of at
|
||||
most 1 MiB and is read from an explicit path, never from `PATH`.
|
||||
* The CLI accepts **schema version 2 only**. A `schema_version: 1` manifest is
|
||||
rejected with `ext.unsupported_provider_schema`; there is no silent
|
||||
migration and no v1/v2 auto-detection. (v1 remains usable only through the
|
||||
host embedding API `StdHost::register_provider_package`, ADR-0037.)
|
||||
* Unknown top-level fields are rejected inside schema v2. Extensible,
|
||||
non-executable data must live in the `metadata` object, which has no
|
||||
execution meaning. A future schema version must be a new document, not an
|
||||
in-place extension of v2.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "marqdo.provider",
|
||||
"schema_version": 2,
|
||||
"name": "demo.sensor",
|
||||
"version": "0.1.0",
|
||||
"abi": "c-abi-1",
|
||||
"transport": "worker",
|
||||
"kinds": ["demo.sensor"],
|
||||
"artifacts": [
|
||||
{
|
||||
"target": "x86_64-unknown-linux-gnu",
|
||||
"path": "lib/libdemo_sensor.so",
|
||||
"sha256": "<64-lowercase-hex-sha256>"
|
||||
}
|
||||
],
|
||||
"permissions_requested": []
|
||||
}
|
||||
```
|
||||
|
||||
### 2.1 Fields
|
||||
|
||||
| Field | Required | Rules |
|
||||
|---|---|---|
|
||||
| `schema` | yes | Exactly `"marqdo.provider"`. |
|
||||
| `schema_version` | yes | Exactly `2` (integer). |
|
||||
| `name` | yes | Stable grant name. Non-empty dotted name: each `.`-separated segment starts with a lowercase ASCII letter and continues with lowercase ASCII letters, digits, `-` or `_`; at most 128 bytes. A name equal to a reserved core resource kind is rejected. |
|
||||
| `version` | yes | Strict SemVer (`semver::Version`, three numeric components). |
|
||||
| `abi` | yes | Exactly `"c-abi-1"` (ABI v1, ADR-0036/0038). Anything else is `ext.unsupported_provider_schema`. |
|
||||
| `transport` | yes | Exactly `"worker"` (framed-JSON helper, ADR-0035/0038). Anything else is `ext.unsupported_provider_schema`. |
|
||||
| `kinds` | yes | Non-empty, at most 64 unique dotted names with the same grammar as `name`. Core kinds are reserved (see §2.2). |
|
||||
| `artifacts` | yes | Non-empty, at most 32 entries with unique target triples. Each entry has `target`, `path`, `sha256`; unknown entry fields are rejected. |
|
||||
| `permissions_requested` | no | Declarations only, at most 64 unique strings: a known host capability (`marqdo_plugin_abi::KNOWN_CAPABILITIES`) or a dotted vendor capability (at least three segments, no core namespace prefix). Declarations never grant anything. |
|
||||
| `metadata` | no | Arbitrary JSON. Ignored for execution; never used to select artifacts or grant permissions. |
|
||||
|
||||
`artifacts[].target` must parse as a target triple. `artifacts[].path` is
|
||||
bundle-relative: absolute paths, `..` components and other escapes are
|
||||
rejected. `artifacts[].sha256` must be 64 lowercase hexadecimal characters.
|
||||
`artifacts[].path` is resolved against the manifest's own directory (the
|
||||
bundle root); a symlink is followed only when the canonical target stays inside
|
||||
that root, and the final target must be a regular file no larger than 256 MiB.
|
||||
|
||||
### 2.2 Reserved core kinds
|
||||
|
||||
`stdio`, `file`, `tcp`, `tls`, `udp` and `tcp-listener` are implemented by
|
||||
`marqdo-host` before the provider registry is consulted. A provider may not
|
||||
declare them (nor use one as its `name`); registration can never shadow them.
|
||||
|
||||
Reserved core kinds are rejected with `ext.manifest_invalid`.
|
||||
|
||||
### 2.3 Artifact selection
|
||||
|
||||
The only artifact used is the one whose `target` equals
|
||||
`marqdo_plugin_abi::HostTarget::current().triple()`. A missing current target
|
||||
is `ext.no_matching_artifact` (exit 2); another platform's artifact or a
|
||||
`default` fallback is never substituted.
|
||||
|
||||
## 3. Identity and integrity
|
||||
|
||||
* The manifest content digest is `sha256:<hex>` over the manifest bytes; the
|
||||
selected artifact digest is the declared lowercase `sha256`. Both are
|
||||
re-verified immediately before the helper is spawned, and again when the
|
||||
registration helper is called, so a file replaced between preparation and
|
||||
activation fails closed.
|
||||
* These hashes prove the manifest/artifact pair is byte-identical to what was
|
||||
selected. They do **not** prove who published it, that it is trustworthy or
|
||||
that it may be granted. Publisher trust needs a signature/trust-root layer
|
||||
that is out of scope for schema v2.
|
||||
* No version of this schema stores a secret, environment value or shell
|
||||
fragment. Manifests are never concatenated into a shell command.
|
||||
|
||||
### 3.1 Duplicate identity
|
||||
|
||||
* The same provider `name` selected with two different manifest digests is
|
||||
`ext.duplicate_provider`; the exact same manifest selected twice is
|
||||
idempotent.
|
||||
* One non-reserved `kind` declared by two providers is
|
||||
`ext.duplicate_provider_kind`. Registration never overwrites a previous
|
||||
provider ("last wins" is forbidden).
|
||||
* Error selection is deterministic: manifests are canonicalised, de-duplicated
|
||||
and sorted before conflicts are reported, independent of CLI argument order.
|
||||
* One provider may serve several non-reserved kinds; all of them are
|
||||
registered together and must be usable.
|
||||
|
||||
## 4. Worker helper resolution
|
||||
|
||||
The CLI resolves the helper **only** as a sibling of the running `marqdo`
|
||||
executable:
|
||||
|
||||
```text
|
||||
<directory of the marqdo executable>/marqdo-provider-worker[.exe]
|
||||
```
|
||||
|
||||
* No `PATH` lookup, no current-directory search, no environment override and no
|
||||
download fallback. A helper found on `PATH` is ignored.
|
||||
* A missing sibling (or a non-regular file) is `ext.worker_unavailable`
|
||||
(exit 1). A helper that fails to start or fails the protocol ping is also
|
||||
`ext.worker_unavailable`; there is no fallback to in-process/native loading.
|
||||
* Release archives must ship the `marqdo` binary and the
|
||||
`marqdo-provider-worker` helper built for the same target triple.
|
||||
* The `worker` transport is supported only on **linux/x86_64** in this release.
|
||||
Other platforms compile but fail closed with `ext.platform_unsupported`
|
||||
(exit 1). Windows and macOS are explicitly unsupported until they have a
|
||||
native fixture and release evidence; Linux success is not a platform claim.
|
||||
|
||||
## 5. Worker protocol v1 and token lifecycle
|
||||
|
||||
The helper is `marqdo-provider-worker <selected-artifact-path>`. It loads
|
||||
exactly one `c-abi-1` shared library, verifies
|
||||
`marqdo_resource_provider_abi_version() == 1`, and speaks the ADR-0035 framed
|
||||
protocol over stdin/stdout: 4-byte big-endian length + JSON, 1 MiB maximum per
|
||||
frame.
|
||||
|
||||
```text
|
||||
{"op":"ping"} -> {"ok":true} (handshake)
|
||||
{"op":"open","kind","descriptor","rights"} -> {"ok":true,"token":u64}
|
||||
{"op":"read","token","max_bytes"} -> {"ok":true,"bytes":[u8...],"eof":bool}
|
||||
{"op":"write","token","bytes":[u8...]} -> {"ok":true,"count":u64}
|
||||
{"op":"close","token"} -> {"ok":true}
|
||||
{"op":"shutdown"} -> process exit
|
||||
{"ok":false,"code","message"} -> structured failure
|
||||
```
|
||||
|
||||
Token lifecycle:
|
||||
|
||||
1. `ping` must succeed before the host registers the provider. A failed
|
||||
handshake aborts the run (`ext.worker_unavailable`).
|
||||
2. `open` creates a provider-private `u64` token in the helper. The language
|
||||
never sees it: the Host issues the opaque language `Resource` handle that
|
||||
binds run id + provider name + token. Provider tokens cannot be forged into
|
||||
language values or wire messages.
|
||||
3. `read`/`write`/`close` are dispatched by the Host with the token it stored.
|
||||
A read returns at most `max_bytes`; bytes cross the boundary by copy.
|
||||
4. `close` closes exactly one token. Repeated or unknown tokens must fail with
|
||||
a structured error; the Host never panics.
|
||||
5. When the run ends (normal, error or timeout) every open token is closed and
|
||||
the helper receives `shutdown`; the Host kills and reaps the child on drop.
|
||||
A helper crash/protocol violation becomes a structured `host.error` for the
|
||||
offending operation (CLI exit 1) and is never retried against another
|
||||
provider or an in-process transport.
|
||||
6. Provider error codes map to existing categories: `permission.denied`,
|
||||
`resource.closed`, `resource.exhausted`, `resource.unsupported`, otherwise
|
||||
`host.error`.
|
||||
|
||||
## 6. Authorization
|
||||
|
||||
* Registration, manifest parsing and artifact verification are not
|
||||
authorization. Every `resource.open("<kind>", ...)` still requires:
|
||||
project mode, the provider name in `Capabilities.resource_providers`
|
||||
(populated only by exact `--allow-provider NAME`), and the Host's provider
|
||||
name check. `permissions_requested` declarations do not affect this.
|
||||
* `pure` and `strict` never reach provider kinds. `project` is the only mode
|
||||
that can use a third-party provider.
|
||||
* Unauthorized selection fails closed before the worker is spawned:
|
||||
`--provider` without a matching `--allow-provider` is `permission.denied`
|
||||
(exit 1) and the helper is never started.
|
||||
|
||||
## 7. Error codes and exit codes
|
||||
|
||||
Manifest/CLI option failures exit 2; load/verification/spawn failures exit 1;
|
||||
`permission.denied` exits 1. With `--json` the CLI emits exactly one JSON
|
||||
document with a `code` (nested under `error.code` for these structured
|
||||
errors).
|
||||
|
||||
| Code | Stage | Trigger |
|
||||
|---|---|---|
|
||||
| `ext.manifest_unreadable` | manifest read (2) | missing/unreadable path, directory instead of `provider.json` |
|
||||
| `ext.unsupported_provider_schema` | manifest read (2) | `schema` mismatch, `schema_version != 2`, unsupported `abi`/`transport` |
|
||||
| `ext.manifest_invalid` | manifest read (2) | invalid UTF-8/JSON, unknown field, bad name/kind/version/permission, reserved kind, duplicate kind/target inside one manifest |
|
||||
| `ext.no_matching_artifact` | manifest read (2) | no artifact for the current host target |
|
||||
| `ext.duplicate_provider` | manifest read (2) | same provider name with two different manifest digests |
|
||||
| `ext.duplicate_provider_kind` | manifest read (2) | two providers declare the same non-reserved kind |
|
||||
| `ext.provider_grant_unmatched` | grant check (2) | `--allow-provider` names no selected manifest |
|
||||
| `ext.artifact_integrity` | artifact verify / pre-spawn re-verify (1) | sha256 mismatch, absolute/`..`/symlink escape, non-regular file, size limit, manifest changed after verification |
|
||||
| `permission.denied` | grant check (1) | `--provider` selected without its exact `--allow-provider` grant |
|
||||
| `ext.worker_unavailable` | spawn (1) | sibling helper missing/not a regular file, spawn failure, ping failure |
|
||||
| `ext.platform_unsupported` | spawn (1) | provider `worker` transport on a non-linux/x86_64 build |
|
||||
| `cli.invalid_grants` | CLI preflight (2) | wildcard/empty `--allow-provider`, provider flags in `pure` mode |
|
||||
| `strict.unsupported_grant` | strict preflight (2) | provider flags in `strict` mode |
|
||||
|
||||
`ext.manifest_invalid`, `ext.duplicate_provider` and
|
||||
`ext.provider_grant_unmatched` are additive codes: RFC-EXT-0001 §18.1 froze the
|
||||
provider-specific codes it could name, but a generic malformed-v2-manifest
|
||||
code, a duplicate-provider-name code and an unmatched-grant code are required
|
||||
to keep failure reasons distinct. They follow the same `ext.` naming and exit
|
||||
code contract and are stable.
|
||||
|
||||
## 8. Test matrix (PR-03)
|
||||
|
||||
`crates/cli/tests/provider_cli.rs` builds a real C `c-abi-1` sensor provider,
|
||||
runs the real `marqdo-provider-worker` next to the real CLI binary, and covers:
|
||||
|
||||
| ID | Evidence |
|
||||
|---|---|
|
||||
| PROV-01 | granted v2 worker + `resource.open/read/close("demo.sensor")` returns count 5, EOF, one close, `resource.open` in the host trace |
|
||||
| PROV-02 | `--provider` without grant → `permission.denied`; marker proves the helper was never started |
|
||||
| PROV-03 | `--allow-provider` without `--provider` → `ext.provider_grant_unmatched`, no worker |
|
||||
| PROV-04 | same kind from two providers → `ext.duplicate_provider_kind` before any worker |
|
||||
| PROV-05 | one provider with two non-reserved kinds → both open/read/close, one helper |
|
||||
| PROV-06 | `stdio`/`file`/`tcp`/`tls`/`udp`/`tcp-listener` kinds rejected, no worker |
|
||||
| PROV-07 | digest mismatch, symlink escape, `..` traversal → `ext.artifact_integrity`, no worker |
|
||||
| PROV-08 | no current-target artifact → `ext.no_matching_artifact` |
|
||||
| PROV-09 | `schema_version: 1` → `ext.unsupported_provider_schema` |
|
||||
| PROV-10 | `pure`/`strict` + provider flags → preflight refusal, no worker; wildcard grant rejected |
|
||||
| PROV-13 | relocated CLI without sibling helper (helper present on `PATH`) → `ext.worker_unavailable`, PATH is never used |
|
||||
|
||||
PROV-11 (worker crash/timeout/size-limit), PROV-12 (closed/stale/forged token),
|
||||
PROV-14 (concurrent lifecycle) and PROV-15 (frozen-binary external repository)
|
||||
are covered by the host worker/protocol tests or belong to the PR-07 external
|
||||
conformance repository; they are not part of this CLI slice's manifest matrix.
|
||||
+71
-12
@@ -9,6 +9,7 @@ Normative implementation:
|
||||
|---|---|
|
||||
| Sandbox modes, strict refusal | `crates/marqdo-sandbox/src/lib.rs` |
|
||||
| Capability sets and checks | `crates/marqdo-host/src/capability.rs` |
|
||||
| Trusted capability handlers | `crates/marqdo-host/src/capability.rs` (`CapabilityBroker`, `CapabilityHandler`, `FsReadHandler`; ADR-0057) |
|
||||
| Host dispatch and per-op checks | `crates/marqdo-host/src/host.rs` |
|
||||
| Project-rooted filesystem | `crates/marqdo-host/src/fs.rs` |
|
||||
| Budgets | `crates/marqdo-host/src/budget.rs` |
|
||||
@@ -175,18 +176,72 @@ before reading. Full symlink/race hardening is an open P4 item.
|
||||
* plugin invoke → the **complete export name** must be registered, then
|
||||
`capabilities.check_plugin(package)` on **every call**; loading also checks
|
||||
the same grant once;
|
||||
* plugin host-capability requests → `CapabilityBroker` checks the package and
|
||||
effective grants and **executes** granted `fs.read` through `ProjectFs`,
|
||||
returning the real file bytes. Other core capabilities (`fs.write`,
|
||||
`net.request`, `process`, console/math/log/...), and vendor capabilities
|
||||
without a trusted executing handler, return `capability.unavailable`; they
|
||||
never pretend an empty reply is success. Unknown/vendor names without a
|
||||
trusted policy entry become `capability.unavailable`/`permission.denied`;
|
||||
* plugin host-capability requests → `CapabilityBroker` runs a fixed chain:
|
||||
`decide` (package, effective grants, trusted vendor policy, payload shape) →
|
||||
a trusted `CapabilityHandler` must be installed for the exact capability name
|
||||
→ a package-bound handler must own the trusted package identity → the handler
|
||||
executes under the broker budget and reply limit → one audit entry. Granted
|
||||
`fs.read` executes through `FsReadHandler`, which resolves the path with
|
||||
`ProjectFs` and re-checks the read grant on the normalized project-relative
|
||||
path before returning the real file bytes. Every capability without an
|
||||
installed handler — other core capabilities (`fs.write`, `net.request`,
|
||||
`process`, console/math/log/clock/random/artifact.record) and vendor
|
||||
capabilities without a trusted executor — returns `capability.unavailable`;
|
||||
it never pretends an empty reply is success. A vendor name with no trusted
|
||||
policy entry is `permission.denied` (ADR-0011, ADR-0057);
|
||||
* print → output budget only.
|
||||
|
||||
Static analysis may warn, but it never replaces these runtime checks. There is
|
||||
no host bypass for tests: tests use the same `StdHost`/`Capabilities` types.
|
||||
|
||||
### 2.3 Executable capabilities need an installed trusted handler
|
||||
|
||||
`CapabilityBroker` (`crates/marqdo-host/src/capability.rs`, ADR-0057) separates
|
||||
*being authorized* from *being executable*:
|
||||
|
||||
| Question | Registry | Failure |
|
||||
|---|---|---|
|
||||
| Is this capability granted for this trusted package? | `Capabilities` / `vendor_policy` | `permission.denied` |
|
||||
| Is a trusted executor installed for this exact name? | `handlers: BTreeMap<String, HandlerEntry>` | `capability.unavailable` |
|
||||
| Does a package-bound handler own the claimed package? | `HandlerEntry.package` | `permission.denied` |
|
||||
|
||||
Execution order is fixed: `decide` → handler installed → package binding →
|
||||
handler execution under the broker `Budget` and reply limit → audit. A name in
|
||||
`vendor_policy` is policy only; it does not install anything, and it cannot
|
||||
turn into an executor. A core capability granted by `Capabilities` but missing
|
||||
a handler is `capability.unavailable`, never an empty `Ok`.
|
||||
|
||||
* **Registration is host-only.** `register_handler(handler)` /
|
||||
`register_package_handler(package, handler)` are called by the trusted host /
|
||||
embedder composition root; no manifest, plugin or wire message can reach the
|
||||
registry. The registry key is `handler.name()`, so a handler cannot be
|
||||
registered under a name it does not declare.
|
||||
* **Rejected names** (stable codes): empty/whitespace/malformed names are
|
||||
`capability.handler_invalid`; names under the reserved host prefixes
|
||||
`marqdo.` / `host.` are `capability.handler_reserved`; an already registered
|
||||
name (including the built-in `fs.read`) is `capability.handler_conflict`. A
|
||||
rejected handler is not installed.
|
||||
* **`fs.read`** is the first migrated standard handler (`FsReadHandler`),
|
||||
installed by `CapabilityBroker::with_fs(fs)`. It keeps the `ProjectFs` root,
|
||||
traversal/absolute-path rejection, regular-file/UTF-8/8 MiB limits and the
|
||||
glob grant, and it re-checks the grant against the normalized path at the
|
||||
execution point. The broker reply limit (`MAX_CAPABILITY_REPLY_BYTES` by
|
||||
default) bounds the returned bytes; an oversize reply is
|
||||
`resource.exhausted`.
|
||||
* **Audit**: every decision records package, capability,
|
||||
`granted`/`denied`/`unavailable`, a reason and `duration_micros`. Payloads
|
||||
and file contents are never recorded, so ordinary secrets do not appear in
|
||||
the audit log.
|
||||
* **Budgets**: handlers receive `&mut Budget` and can charge fuel/output; the
|
||||
broker also enforces the reply byte limit. Native handlers still run
|
||||
in-process and are trusted code, not a sandbox.
|
||||
|
||||
Known gaps in this build: the broker `trace_id` is settable
|
||||
(`with_trace_id`) but `host.rs` does not supply it yet; and a broker-level
|
||||
`resource.exhausted` reply keeps its stable code but the host plugin-error
|
||||
mapping currently collapses `PluginError` kind `capability` to the
|
||||
`permission.denied` runtime category (changing that mapping is a follow-up).
|
||||
|
||||
## 3. Resource budgets
|
||||
|
||||
`Budget` (`crates/marqdo-host/src/budget.rs`) is per run:
|
||||
@@ -338,11 +393,15 @@ Neither command executes project code. Specifically:
|
||||
4. Write-path symlink TOCTOU (see §2.1).
|
||||
5. Plugin **worker crash isolation** and the worker capability bridge are
|
||||
implemented (not usable in strict L1): both native and worker plugins can
|
||||
reach the broker's
|
||||
`fs.read` execution when granted. Other core capabilities and vendor
|
||||
handlers remain unavailable. Broker execution is a cooperative API
|
||||
contract, not an OS sandbox; a worker and a native plugin can still issue
|
||||
direct syscalls.
|
||||
reach the broker's `fs.read` handler when granted (ADR-0057). Only
|
||||
`fs.read` has a registered handler; every other core capability and every
|
||||
vendor capability without an explicitly registered trusted handler remains
|
||||
`capability.unavailable`, and there is no API to register one from a plugin,
|
||||
manifest or CLI flag. Broker execution is a cooperative API contract, not an
|
||||
OS sandbox; a worker and a native plugin can still issue direct syscalls.
|
||||
Follow-ups: wire the run `trace_id` into the broker audit and teach the host
|
||||
plugin-error mapping to report `resource.exhausted` as a resource category
|
||||
rather than `permission.denied`.
|
||||
6. `plugin_handle` wire values are structurally reconstructable, but host
|
||||
authority comes only from the `PluginHandleTable` owner/run/generation
|
||||
check; untrusted plugin/foreign output containing a handle is rejected
|
||||
|
||||
+16
-1
@@ -1,5 +1,20 @@
|
||||
{
|
||||
"name": "marqdo-stdlib",
|
||||
"version": "0.1.0",
|
||||
"digest": "sha256:b94f1a48ca6619689584aca3af7befac64ac9fdb7ea7c0a9ccb491df8df226b8"
|
||||
"digest": "sha256:5407d3665ba2fe105820d0f1af5efaafa10c277a69eb819c3225719813c8c713",
|
||||
"public_aliases": [
|
||||
{
|
||||
"public_names": [
|
||||
"print",
|
||||
"打印"
|
||||
],
|
||||
"target_module": "io.mq.md",
|
||||
"target_function": "写值",
|
||||
"expected_signature": "params(值)",
|
||||
"effects": [
|
||||
"host"
|
||||
]
|
||||
}
|
||||
],
|
||||
"aliases_digest": "sha256:5f513c0fdbb3c1e9829be3c9fa23a2d1e89cb8510073fb5eb1e8e2f5265a83bb"
|
||||
}
|
||||
+15
-15
@@ -1,38 +1,38 @@
|
||||
---
|
||||
import:
|
||||
fs: "std:fs/file"
|
||||
fs_lib: "std:fs/file"
|
||||
---
|
||||
|
||||
# @exec fs
|
||||
|
||||
## main
|
||||
|
||||
**创建 = fs.创建目录("out")**
|
||||
**创建 = fs_lib.创建目录("out")**
|
||||
|
||||
**写入 = fs.写文本("out/fs-a.txt", "你好\n")**
|
||||
**写入 = fs_lib.写文本("out/fs-a.txt", "你好\n")**
|
||||
|
||||
**存在 = fs.存在("out/fs-a.txt")**
|
||||
**存在 = fs_lib.存在("out/fs-a.txt")**
|
||||
|
||||
**元数据 = fs.元数据("out/fs-a.txt")**
|
||||
**元数据 = fs_lib.元数据("out/fs-a.txt")**
|
||||
|
||||
**列表 = fs.列出目录("out")**
|
||||
**列表 = fs_lib.列出目录("out")**
|
||||
|
||||
**是文件 = fs.是文件("out/fs-a.txt")**
|
||||
**是文件 = fs_lib.是文件("out/fs-a.txt")**
|
||||
|
||||
**是目录 = fs.是目录("out")**
|
||||
**是目录 = fs_lib.是目录("out")**
|
||||
|
||||
**重命名 = fs.重命名("out/fs-a.txt", "out/fs-b.txt")**
|
||||
**重命名 = fs_lib.重命名("out/fs-a.txt", "out/fs-b.txt")**
|
||||
|
||||
**读取 = fs.读文本("out/fs-b.txt")**
|
||||
**读取 = fs_lib.读文本("out/fs-b.txt")**
|
||||
|
||||
**追加 = fs.追加文本("out/fs-b.txt", "世界\n")**
|
||||
**追加 = fs_lib.追加文本("out/fs-b.txt", "世界\n")**
|
||||
|
||||
**片段 = fs.读取片段("out/fs-b.txt", 0, 100)**
|
||||
**片段 = fs_lib.读取片段("out/fs-b.txt", 0, 100)**
|
||||
|
||||
**遍历 = fs.遍历("out")**
|
||||
**遍历 = fs_lib.遍历("out")**
|
||||
|
||||
**删除 = fs.删除("out/fs-b.txt")**
|
||||
**删除 = fs_lib.删除("out/fs-b.txt")**
|
||||
|
||||
**删除目录 = fs.删除("out")**
|
||||
**删除目录 = fs_lib.删除("out")**
|
||||
|
||||
`return [创建.ok, 写入.value.count, 存在.value, 元数据.value.kind, 元数据.value.size, 列表.value, 是文件.value, 是目录.value, 重命名.ok, 读取.value, 追加.value, 片段.value, 遍历.value, 删除.ok, 删除目录.ok]`
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
import:
|
||||
table: "std:data/table"
|
||||
table_lib: "std:data/table"
|
||||
---
|
||||
|
||||
# @exec table
|
||||
@@ -9,22 +9,22 @@ import:
|
||||
|
||||
**数据 = 表(["名称", "分数"], [["甲", 87], ["乙", 92], ["丙", 71]])**
|
||||
|
||||
**选择 = table.选择列(数据, 1)**
|
||||
**选择 = table_lib.选择列(数据, 1)**
|
||||
|
||||
**过滤 = table.过滤数值大于(数据, 1, 80)**
|
||||
**过滤 = table_lib.过滤数值大于(数据, 1, 80)**
|
||||
|
||||
**排序 = table.排序(数据, 1)**
|
||||
**排序 = table_lib.排序(数据, 1)**
|
||||
|
||||
**追加 = table.追加行(数据, ["丁", 65])**
|
||||
**追加 = table_lib.追加行(数据, ["丁", 65])**
|
||||
|
||||
**设置 = table.设置单元格(数据, 0, 1, 99)**
|
||||
**设置 = table_lib.设置单元格(数据, 0, 1, 99)**
|
||||
|
||||
**左 = 表(["名称", "分数"], [["甲", 87], ["乙", 92]])**
|
||||
|
||||
**右 = 表(["名称", "等级"], [["乙", "A"], ["甲", "B"]])**
|
||||
|
||||
**连接 = table.连接(左, 右, 0, 0)**
|
||||
**连接 = table_lib.连接(左, 右, 0, 0)**
|
||||
|
||||
**分组 = table.分组(表(["类别", "值"], [["a", 1], ["b", 2], ["a", 3]]), 0, 1)**
|
||||
**分组 = table_lib.分组(表(["类别", "值"], [["a", 1], ["b", 2], ["a", 3]]), 0, 1)**
|
||||
|
||||
`return [选择, 过滤, 排序, 追加, 设置, 连接, 分组, 数据, table.转GFM(数据)]`
|
||||
`return [选择, 过滤, 排序, 追加, 设置, 连接, 分组, 数据, table_lib.转GFM(数据)]`
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
import:
|
||||
text: "std:text"
|
||||
text_lib: "std:text"
|
||||
---
|
||||
|
||||
# @exec text
|
||||
@@ -9,6 +9,6 @@ import:
|
||||
|
||||
**内存 = memory.new(1, 1)**
|
||||
|
||||
**长度 = text.utf8编码("é", 内存, 0)**
|
||||
**长度 = text_lib.utf8编码("é", 内存, 0)**
|
||||
|
||||
`return [长度, 内存[0], 内存[1]]`
|
||||
Executable
+245
@@ -0,0 +1,245 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Frozen-CLI external-extension evidence generator and validator.
|
||||
|
||||
RFC-EXT-0001 §10.2/§11.3: a release candidate must prove that independent
|
||||
repositories work against an *unchanged* CLI binary. This script:
|
||||
|
||||
1. hashes the CLI binary before and after the external suite and refuses to
|
||||
emit PASS if the hash moved;
|
||||
2. records the exact marqdo-cli source SHA and the external fixture repo SHA;
|
||||
3. runs the fixture repository's own black-box suite with ``MARQDO_BIN`` set to
|
||||
the frozen binary (the suite itself must not run cargo or read the CLI
|
||||
source tree);
|
||||
4. validates/normalises the per-case results into the repository evidence
|
||||
schema (``docs/evidence/extension/schema.json``);
|
||||
5. writes ``docs/evidence/extension/<tested_sha>/manifest.json``.
|
||||
|
||||
Usage:
|
||||
|
||||
python3 tools/ci/verify_extension_contracts.py \
|
||||
--cli target/release/marqdo \
|
||||
--fixtures ../marqdo-extension-conformance \
|
||||
--out docs/evidence/extension/<sha>/manifest.json
|
||||
|
||||
python3 tools/ci/verify_extension_contracts.py --validate manifest.json
|
||||
|
||||
Only Python's standard library is used. No network access is performed by this
|
||||
script; the external suite documents its own offline cache behaviour.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
SCHEMA = REPO / "docs" / "evidence" / "extension" / "schema.json"
|
||||
|
||||
REQUIRED_RESULTS = [
|
||||
"callable_registry",
|
||||
"plugin_worker",
|
||||
"foreign_kernel",
|
||||
"resource_provider_cli",
|
||||
"package_pinned_git",
|
||||
"tool_analysis_consumer",
|
||||
"negative_security",
|
||||
]
|
||||
|
||||
ALLOWED = {"PASS", "FAIL", "UNSUPPORTED", "SKIPPED"}
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def git_sha(directory: Path) -> str:
|
||||
output = subprocess.run(
|
||||
["git", "-C", str(directory), "rev-parse", "HEAD"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
return output.stdout.strip()
|
||||
|
||||
|
||||
def os_name() -> str:
|
||||
name = platform.system().lower()
|
||||
if name.startswith("linux"):
|
||||
return "linux"
|
||||
if name == "darwin":
|
||||
return "macos"
|
||||
if name.startswith("windows"):
|
||||
return "windows"
|
||||
raise SystemExit(f"unsupported OS for extension evidence: {name}")
|
||||
|
||||
|
||||
def validate_manifest(manifest: dict) -> list[str]:
|
||||
problems: list[str] = []
|
||||
if manifest.get("schema") != "marqdo.extension-evidence":
|
||||
problems.append("schema must be marqdo.extension-evidence")
|
||||
if manifest.get("schema_version") != 1:
|
||||
problems.append("schema_version must be 1")
|
||||
for field in ["tested_sha", "fixture_repo_sha"]:
|
||||
value = manifest.get(field, "")
|
||||
if not (isinstance(value, str) and len(value) == 40 and all(c in "0123456789abcdef" for c in value)):
|
||||
problems.append(f"{field} must be a 40-character lowercase git SHA")
|
||||
cli = manifest.get("cli_binary_sha256", "")
|
||||
if not (isinstance(cli, str) and len(cli) == 64 and all(c in "0123456789abcdef" for c in cli)):
|
||||
problems.append("cli_binary_sha256 must be a 64-character lowercase sha256")
|
||||
if manifest.get("os") not in {"linux", "macos", "windows"}:
|
||||
problems.append("os must be linux, macos or windows")
|
||||
results = manifest.get("results")
|
||||
if not isinstance(results, dict):
|
||||
problems.append("results must be an object")
|
||||
else:
|
||||
for name in REQUIRED_RESULTS:
|
||||
if results.get(name) not in ALLOWED:
|
||||
problems.append(f"results.{name} must be one of {sorted(ALLOWED)}")
|
||||
for key, value in results.items():
|
||||
if key not in REQUIRED_RESULTS:
|
||||
problems.append(f"unknown result key {key}")
|
||||
if value not in ALLOWED:
|
||||
problems.append(f"result {key}={value} is not allowed")
|
||||
commands = manifest.get("commands")
|
||||
if not isinstance(commands, list) or not commands or not all(isinstance(c, str) for c in commands):
|
||||
problems.append("commands must be a non-empty array of strings")
|
||||
for field in ["failures", "unverified"]:
|
||||
if not isinstance(manifest.get(field), list):
|
||||
problems.append(f"{field} must be an array")
|
||||
if not problems:
|
||||
for name, value in (manifest.get("results") or {}).items():
|
||||
if value in {"FAIL"}:
|
||||
problems.append(f"result {name} is FAIL")
|
||||
return problems
|
||||
|
||||
|
||||
def run_suite(fixtures: Path, cli: Path) -> tuple[bool, dict, list[str]]:
|
||||
"""Run the external suite and return (ok, results, commands)."""
|
||||
results: dict[str, str] = {}
|
||||
commands: list[str] = []
|
||||
env = dict(os.environ)
|
||||
env["MARQDO_BIN"] = str(cli)
|
||||
env["MARQDO_FROZEN_CLI"] = str(cli)
|
||||
runner = fixtures / "tests" / "run_all.py"
|
||||
if not runner.is_file():
|
||||
return False, results, commands
|
||||
commands.append(f"python3 {runner} (cwd={fixtures})")
|
||||
completed = subprocess.run(
|
||||
[sys.executable, str(runner)],
|
||||
cwd=fixtures,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
print(completed.stdout, end="")
|
||||
print(completed.stderr, end="", file=sys.stderr)
|
||||
results_path = fixtures / "evidence" / "results.json"
|
||||
if results_path.is_file():
|
||||
with results_path.open("r", encoding="utf-8") as handle:
|
||||
payload = json.load(handle)
|
||||
for key, value in (payload.get("results") or {}).items():
|
||||
results[key] = str(value).upper()
|
||||
return completed.returncode == 0, results, commands
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--cli", type=Path, help="Absolute/relative frozen marqdo binary")
|
||||
parser.add_argument("--fixtures", type=Path, help="External fixture repository root")
|
||||
parser.add_argument("--source-repo", type=Path, default=REPO, help="marqdo-cli checkout")
|
||||
parser.add_argument("--out", type=Path, help="Evidence manifest output path")
|
||||
parser.add_argument("--validate", type=Path, help="Only validate an existing manifest")
|
||||
parser.add_argument("--allow-unverified", action="store_true", help="Record UNSUPPORTED/SKIPPED without failing")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.validate:
|
||||
manifest = json.loads(args.validate.read_text(encoding="utf-8"))
|
||||
problems = validate_manifest(manifest)
|
||||
if problems:
|
||||
for problem in problems:
|
||||
print(f"INVALID: {problem}", file=sys.stderr)
|
||||
return 1
|
||||
print(f"valid extension evidence: {args.validate}")
|
||||
return 0
|
||||
|
||||
if not args.cli or not args.fixtures:
|
||||
parser.error("--cli and --fixtures are required unless --validate is used")
|
||||
cli = args.cli.resolve()
|
||||
fixtures = args.fixtures.resolve()
|
||||
if not cli.is_file():
|
||||
print(f"error: CLI binary not found: {cli}", file=sys.stderr)
|
||||
return 2
|
||||
if not (fixtures / ".git").exists():
|
||||
print(f"error: fixture path is not a git repository: {fixtures}", file=sys.stderr)
|
||||
return 2
|
||||
if subprocess.run(
|
||||
["git", "-C", str(fixtures), "status", "--porcelain"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
).stdout.strip():
|
||||
print("error: fixture repository has uncommitted changes", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
cli_before = sha256_file(cli)
|
||||
tested_sha = git_sha(args.source_repo.resolve())
|
||||
fixture_sha = git_sha(fixtures)
|
||||
ok, results, commands = run_suite(fixtures, cli)
|
||||
cli_after = sha256_file(cli)
|
||||
if cli_before != cli_after:
|
||||
print("error: frozen CLI hash changed during the external suite", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
failures = [name for name, value in results.items() if value == "FAIL"]
|
||||
unverified = [
|
||||
name for name, value in results.items() if value in {"UNSUPPORTED", "SKIPPED"}
|
||||
]
|
||||
missing = [name for name in REQUIRED_RESULTS if name not in results]
|
||||
if missing:
|
||||
failures.extend(f"missing:{name}" for name in missing)
|
||||
manifest = {
|
||||
"schema": "marqdo.extension-evidence",
|
||||
"schema_version": 1,
|
||||
"tested_sha": tested_sha,
|
||||
"cli_binary_sha256": cli_before,
|
||||
"fixture_repo_sha": fixture_sha,
|
||||
"os": os_name(),
|
||||
"arch": platform.machine(),
|
||||
"results": {name: results.get(name, "SKIPPED") for name in REQUIRED_RESULTS},
|
||||
"commands": commands,
|
||||
"failures": failures,
|
||||
"unverified": unverified,
|
||||
}
|
||||
problems = validate_manifest(manifest)
|
||||
if unverified and not args.allow_unverified:
|
||||
problems.append(
|
||||
"UNSUPPORTED/SKIPPED results require --allow-unverified and an explicit report entry"
|
||||
)
|
||||
if not ok and not failures:
|
||||
failures.append("external suite exited non-zero without structured results")
|
||||
problems.append("external suite failed")
|
||||
if problems:
|
||||
for problem in problems:
|
||||
print(f"FAIL: {problem}", file=sys.stderr)
|
||||
print(json.dumps(manifest, indent=2))
|
||||
return 1
|
||||
out = args.out or (REPO / "docs" / "evidence" / "extension" / tested_sha / "manifest.json")
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8")
|
||||
print(f"wrote extension evidence: {out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -18,6 +18,22 @@ STDLIB = ROOT / "stdlib"
|
||||
DIST = ROOT / "dist"
|
||||
|
||||
|
||||
def aliases_digest(manifest: dict) -> str:
|
||||
digest = hashlib.sha256()
|
||||
for alias in sorted(
|
||||
manifest.get("public_aliases", []),
|
||||
key=lambda a: (a["target_module"], a["target_function"]),
|
||||
):
|
||||
digest.update(alias["target_module"].encode())
|
||||
digest.update(b"\0")
|
||||
digest.update(alias["target_function"].encode())
|
||||
digest.update(b"\0")
|
||||
for name in sorted(alias["public_names"]):
|
||||
digest.update(name.encode())
|
||||
digest.update(b"\0")
|
||||
return f"sha256:{digest.hexdigest()}"
|
||||
|
||||
|
||||
def tree_digest() -> str:
|
||||
digest = hashlib.sha256()
|
||||
for path in sorted(
|
||||
@@ -37,6 +53,9 @@ def main() -> int:
|
||||
digest = tree_digest()
|
||||
if manifest["digest"] != f"sha256:{digest}":
|
||||
raise SystemExit("stdlib.json digest does not match the tree")
|
||||
alias_digest = aliases_digest(manifest)
|
||||
if manifest.get("aliases_digest") != alias_digest:
|
||||
raise SystemExit("stdlib.json aliases_digest does not match public_aliases")
|
||||
|
||||
files = sorted(
|
||||
[p for p in STDLIB.rglob("*") if p.is_file() and p.name != "stdlib.json"]
|
||||
@@ -65,6 +84,7 @@ def main() -> int:
|
||||
"name": manifest["name"],
|
||||
"version": version,
|
||||
"tree_digest": f"sha256:{digest}",
|
||||
"aliases_digest": alias_digest,
|
||||
"archive": archive.name,
|
||||
"archive_sha256": f"sha256:{archive_sha}",
|
||||
"files": [p.relative_to(STDLIB).as_posix() for p in files],
|
||||
|
||||
Reference in new issue
Block a user