From 89555382e0a8cb19f629757ed6d611fc9305ebba Mon Sep 17 00:00:00 2001 From: cflmy Date: Sun, 22 Nov 2026 00:43:18 +0800 Subject: [PATCH] feat(ext): extension selection + lock v4, inspect --extensions (RFC-EXT-0001 PR-04) - marqdo.yaml.extensions accepts only ./path or @alias/path, resolves to a regular file inside its owning package, never grants - lock_version 4 records plugin/kernel/provider manifest+artifact identity; v3 requires explicit 'marqdo lock' migration; sync/check/run compare identity - check stays static; run re-verifies artifact bytes; pure/strict refuse - inspect --extensions reports declared/locked identity, plugin descriptors and stdlib aliases with authorization 'not_granted' - black-box extension_selection suite (13) + package-layer tests; specs/ADR-0056 - workspace gate: fmt/clippy clean, 599 passed across 67 targets --- CHANGELOG.md | 37 ++ COMPATIBILITY.md | 37 +- crates/cli/src/commands/inspect.rs | 123 +++- crates/cli/src/commands/package.rs | 73 ++- crates/cli/src/commands/run.rs | 42 +- crates/cli/src/extension_selection.rs | 318 +++++++++ crates/cli/src/loader.rs | 15 +- crates/cli/src/main.rs | 10 +- crates/cli/src/project_prepare.rs | 52 +- crates/cli/tests/extension_selection.rs | 495 ++++++++++++++ crates/cli/tests/package_env.rs | 2 +- crates/marqdo-package/src/lib.rs | 649 ++++++++++++++++++- docs/STATUS.md | 39 +- docs/adr/0056-extension-selection-locking.md | 127 ++++ docs/adr/README.md | 19 +- examples/multi-module/marqdo.lock | 2 +- examples/packages-multi-version/marqdo.lock | 2 +- spec/extension-selection.md | 191 ++++++ spec/package-format.md | 70 +- 19 files changed, 2240 insertions(+), 63 deletions(-) create mode 100644 crates/cli/src/extension_selection.rs create mode 100644 crates/cli/tests/extension_selection.rs create mode 100644 docs/adr/0056-extension-selection-locking.md create mode 100644 spec/extension-selection.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 87ea795..21879e9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,43 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601. ## Unreleased (stabilization candidate) +### Breaking changes — extensibility closure (RFC-EXT-0001) + +Implements [`docs/architecture/RFC-EXT-0001.md`](docs/architecture/RFC-EXT-0001.md) +(ADR-0054/0055/0056/0057/0058). 0.x breaking changes: + +- **One callable resolution.** `check`/`run`/`call`/`inspect`/Analysis share a + single registry and binding pass (`CallSiteKey`). The runtime no longer + re-resolves names or guesses a plugin when no descriptor is loaded. An import + alias that collides with a builtin/plugin dotted namespace (for example + importing a module as `math`) is now `ext.import_namespace_conflict`; rename + the alias. Unknown plugin exports are `symbol.unknown` at compile time. +- **Exact plugin binding.** A plugin manifest is the static signature contract; + `run` verifies artifact bytes and reconciles exports/signatures/owners/digests + one-to-one (`ext.plugin_binding_mismatch`). Empty/synthesised signatures are + gone; ABI v1 signature defaults are rejected; positional arguments are + normalised to manifest order. +- **Lock v4.** `marqdo.lock` upgrades to `lock_version: 4` with an `extensions:` + identity table. Lock v3 is rejected with an explicit `marqdo lock` migration + error; it is never reinterpreted silently. Projects can declare explicit + activation sources under `marqdo.yaml.extensions` (`./path` or `@alias/path` + only); a declaration is not a grant. +- **Analysis calls projection.** `calls[].state/target` now come from the same + binding; official stdlib aliases such as `打印` project as `function` targets. + Analysis v2 schema/version are unchanged and the private binding table is not + serialized. +- **Print magic removed.** `打印`/`print` are verified `stdlib.json` + `public_aliases` bound to `io.mq.md::写值`; `Program.print_override`, the + AST-wide print scan and the runtime print branch are deleted. Output bytes are + unchanged; programs without print do not load the stdlib module graph. +- **Third-party resource providers via CLI.** `--provider PATH` / + `--allow-provider NAME` on `run`/`call` select and authorize a + `provider.json` schema-2 worker artifact; verified before spawn; pure/strict + refuse; linux/x86_64 only (other platforms fail closed). +- **Capability handler registry.** Granted capabilities execute only through a + trusted installed handler under budget/audit; a declared-but-unhandled + capability stays `capability.unavailable`. + ### Breaking changes — language source syntax (RFC-LANG-0002) **The `.mq.md` source language changed incompatibly. Old source files are not diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 7a7fd5d..c6fe281 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -1,9 +1,10 @@ # Compatibility and support matrix -Snapshot: workspace version **0.1.0**, language SHA **`9c1723d`**, security SHA -**`8af7262`**, no release tag cut. This file records what was actually verified and separates it -from what is only implemented. Executed status is in -[`docs/STATUS.md`](docs/STATUS.md). +Snapshot: workspace version **0.1.0**, RFC-EXT-0001 tree on +`feat/extensible-ecosystem`, no release tag cut. This file records what was +actually verified and separates it from what is only implemented. Executed +status is in [`docs/STATUS.md`](docs/STATUS.md); SHA-bound external evidence is +under [`docs/evidence/extension/`](docs/evidence/extension/README.md). ## Language source syntax (breaking change) @@ -47,11 +48,15 @@ Consequences for compatibility: |---|---|---| | Markdown profile | `marqdo.md.v1` | Frozen parser/options. The RFC-LANG-0002 mapping rewrite keeps this profile; the source-language break is tracked above, not by a new profile id. | | Language / CLI | workspace `0.1.0` | Pre-1.0. The `.mq.md` source language changed incompatibly (RFC-LANG-0002); there is no old-syntax execution path. CLI flags may change with a changelog note; `strict` refusal is contractual. | -| `marqdo.lock` | v3 | v1/v2 are rejected with a relock instruction; `run`/`call`/`check` never rewrite or fetch the lock. | +| `marqdo.lock` | v4 | v3 is rejected with an explicit `marqdo lock` migration error; v1/v2 are rejected with a relock instruction; `run`/`call`/`check` never rewrite or fetch the lock. `extensions:` records manifest/artifact identity (integrity, not publisher auth). | | Analysis | v2 | Consumers reject unknown majors; incompatible field changes require v3. | | Trace | v2 | Binds to the exact Analysis bytes; unknown majors rejected. | | CLI Tool | v1 | Manifest schema, SemVer negotiation and whole-bundle digest are stable for v1. | -| Plugin ABI | v1 | Struct layout/version negotiation frozen; `--trust-native-plugin` is required for in-process loading. | +| Plugin ABI | v1 | Struct layout/version negotiation frozen; `--trust-native-plugin` is required for in-process loading. The static manifest signature is authoritative; `run` reconciles loaded exports/signatures/owners/artifact digests exactly (`ext.plugin_binding_mismatch`); ABI v1 defaults are rejected. | +| Resource Provider manifest | v2 | `schema_version: 2`, `abi: c-abi-1`, `transport: worker`; v1 is rejected (`ext.unsupported_provider_schema`), no silent migration. CLI `--provider`/`--allow-provider`; trusted worker, not an OS sandbox. | +| Callable resolution | ADR-0054 | One registry + one binding pass; `check` and `run` bind the same `CallSiteKey`; no runtime name fallback. Import aliases colliding with builtin/plugin dotted namespaces are rejected. | +| Analysis `calls[].state/target` | v2 (projection) | Now projected from the unique binding; official stdlib aliases appear as `function`. Schema/version unchanged; private binding table not serialized. | +| Extension selection | lock v4 | `marqdo.yaml.extensions` only `./path` or `@alias/path`; declaration is not a grant; declared identity must match the lock (`ext.extension_lock_mismatch`). | | Foreign wire | v1 | One-shot request envelope; provider descriptors are explicit. | ## Platform / feature evidence @@ -67,7 +72,13 @@ portable/tested locally but the native platform lane has not run here; | Addressable `Memory` P1 (ADR-0022) | PASS | UNVERIFIED | UNVERIFIED | | Resource streams CM-A06 (ADR-0025/0026: stdout + project file read/write) | PASS | UNVERIFIED | **FAIL-CLOSED** (`strict` refuses resource opens) | | `run` / `call` / `check` / `inspect` / `fmt` | PASS | UNVERIFIED | UNVERIFIED | -| lock v3 path/Git graph + sync + tamper | PASS | UNVERIFIED | UNVERIFIED | +| lock v4 path/Git graph + sync + tamper | PASS | UNVERIFIED | UNVERIFIED | +| Extension selection/lock v4 (`extensions:`, identities, `inspect --extensions`) | PASS | UNVERIFIED | UNVERIFIED | +| Callable single binding + `ext.import_namespace_conflict`/`symbol.unknown`/plugin reconciliation | PASS | UNVERIFIED | UNVERIFIED | +| Official stdlib public aliases (print de-magic; byte-identical output) | PASS | UNVERIFIED | UNVERIFIED | +| Resource Provider v2 CLI (`--provider`/`--allow-provider`, worker helper) | PASS (linux/x86_64; real C sensor + helper) | **FAIL-CLOSED** (`ext.platform_unsupported`) | **UNSUPPORTED** | +| Capability handler registry (`fs.read` handler, audit, unavailable) | PASS | UNVERIFIED | UNVERIFIED | +| External conformance repository (`marqdo-extension-conformance`, frozen CLI) | PASS (provisional development binary; release rerun pending) | UNVERIFIED | UNVERIFIED | | Analysis v2 / Trace v2 + Python consumers | PASS | UNVERIFIED | UNVERIFIED | | Tool Protocol v1 + 3 independent consumers | PASS | UNVERIFIED | UNVERIFIED | | Native ABI (`plugin inspect`, trusted call) | PASS (C fixture) | UNVERIFIED (host-native fixture) | **UNSUPPORTED** | @@ -107,6 +118,18 @@ reproduce the Windows checkout byte behavior. permits user, mount, PID and network namespaces. See [`docs/security-gate.md`](docs/security-gate.md) and ADR-0021. +## RFC-EXT-0001 platform statements + +* Provider worker transport: linux/x86_64 only; macOS/Windows return + `ext.platform_unsupported` (compile-only, fail-closed). The RFC's macOS + provider lane is **not claimed**. +* Windows Native/Worker ABI remains **UNSUPPORTED**; a Linux PASS does not + substitute for it. +* `strict` L1 continues to refuse plugins/providers/foreign kernels; strict L2 + is not implemented. A worker is a trusted process, not an OS sandbox. +* The external conformance suite is a mandatory **additional** release gate + (`docs/releasing.md` §7); it does not replace G3/G4/native lanes. + ## Explicitly unsupported in this candidate - Pre-RFC `.mq.md` source syntax (Strong-execute / Emph-return / ordered-list diff --git a/crates/cli/src/commands/inspect.rs b/crates/cli/src/commands/inspect.rs index d1665ce..ebaa51d 100644 --- a/crates/cli/src/commands/inspect.rs +++ b/crates/cli/src/commands/inspect.rs @@ -17,24 +17,31 @@ pub struct InspectFlags { pub refs: bool, /// Show platform capability report. pub platform: bool, + /// Show declared/loaded extension identities (never grants). + pub extensions: bool, } -pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool) -> i32 { - let prepared = match prepare_project(file, json, &[]) { +pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool, plugins: &[PathBuf]) -> i32 { + let prepared = match prepare_project(file, json, plugins) { Ok(prepared) => prepared, Err(code) => return code, }; - let project = prepared.loaded; + let project = &prepared.loaded; let entry = project.entry_file(); let module = &entry.analysis.module; - let selected_any = - flags.ast || flags.units || flags.signature || flags.ir || flags.refs || flags.platform; + let selected_any = flags.ast + || flags.units + || flags.signature + || flags.ir + || flags.refs + || flags.platform + || flags.extensions; if json { let mut object = serde_json::Map::new(); object.insert("command".into(), json!("inspect")); object.insert("module".into(), json!(entry.module_id.to_string())); if !selected_any { - object.insert("summary".into(), summary_json(&project)); + object.insert("summary".into(), summary_json(project)); } if flags.ast { object.insert( @@ -58,7 +65,7 @@ pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool) -> i32 { ); } if flags.refs { - object.insert("refs".into(), json!(refs_json(&project))); + object.insert("refs".into(), json!(refs_json(project))); } if flags.platform { object.insert( @@ -66,6 +73,9 @@ pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool) -> i32 { serde_json::to_value(PlatformCapabilities::detect()).unwrap_or_default(), ); } + if flags.extensions { + object.insert("extensions".into(), extensions_json(&prepared, project)); + } output::print_json(&serde_json::Value::Object(object)); return 0; } @@ -120,7 +130,7 @@ pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool) -> i32 { if flags.refs { println!( "{}", - serde_json::to_string_pretty(&refs_json(&project)).unwrap_or_default() + serde_json::to_string_pretty(&refs_json(project)).unwrap_or_default() ); } if flags.platform { @@ -129,5 +139,102 @@ pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool) -> i32 { serde_json::to_string_pretty(&PlatformCapabilities::detect()).unwrap_or_default() ); } + if flags.extensions { + let value = extensions_json(&prepared, project); + println!( + "{}", + serde_json::to_string_pretty(&value).unwrap_or_default() + ); + } 0 } + +/// Static extension identity projection for `inspect --extensions`. +/// +/// Reports declared sources, lock identity, static plugin descriptors and +/// official stdlib aliases. It deliberately never shows environment variables, +/// secrets, binary contents or an authorization result beyond +/// `not_granted` (inspection does not grant anything). +fn extensions_json( + prepared: &crate::project_prepare::PreparedProgram, + project: &crate::loader::LoadedProject, +) -> serde_json::Value { + let root = project.root.as_path(); + let relative = |path: &Path| { + path.strip_prefix(root) + .map(|value| value.to_string_lossy().to_string()) + .unwrap_or_else(|_| path.to_string_lossy().to_string()) + }; + let declared: Vec = project + .extension_sources + .iter() + .map(|extension| { + let identity = prepared.extension_identities.iter().find(|identity| { + identity.kind == extension.kind + && identity.source == extension.from + && identity.logical_owner == extension.logical_owner + }); + let locked = project.lock.as_ref().is_some_and(|lock| { + identity.is_some_and(|identity| { + lock.extension(identity.kind, &identity.name, &identity.logical_owner) + .is_some() + }) + }); + json!({ + "kind": extension.kind.as_str(), + "source": extension.from, + "path": relative(&extension.path), + "logical_owner": extension.logical_owner, + "name": identity.map(|identity| identity.name.clone()), + "version": identity.map(|identity| identity.version.clone()), + "manifest_digest": identity.map(|identity| identity.manifest_digest.clone()), + "artifact_digest": identity.map(|identity| identity.artifact_digest.clone()), + "selected_target": identity.map(|identity| identity.selected_target.clone()), + "status": if locked { "locked" } else { "unlocked" }, + "authorization": "not_granted", + }) + }) + .collect(); + let plugins: Vec = prepared + .plugins + .plugins + .iter() + .map(|plugin| { + json!({ + "package": plugin.package, + "version": plugin.version, + "kind": plugin.kind.as_str(), + "manifest_path": relative(&plugin.manifest_path), + "manifest_digest": plugin.manifest_sha256, + "selected_target": plugin.artifact_target, + "artifact_digest": plugin.artifact_sha256, + "artifact_path": relative(&plugin.artifact_path()), + "exports": plugin.exports.iter().map(|export| json!({ + "name": export.name, + "signature": export.signature, + })).collect::>(), + "validation": "static_manifest_ok", + "authorization": "not_granted", + }) + }) + .collect(); + let aliases: Vec = project + .official_aliases + .iter() + .map(|alias| { + json!({ + "public_names": alias.public_names, + "target": alias.function.to_string(), + "owner_package": alias.owner_package, + "source": "verified stdlib bundle", + "authorization": "official_bundle", + }) + }) + .collect(); + json!({ + "declared": declared, + "plugins": plugins, + "stdlib_aliases": aliases, + "note": "inspect never loads or executes an extension and never grants a permission", + }) +} diff --git a/crates/cli/src/commands/package.rs b/crates/cli/src/commands/package.rs index 7e961f3..3687a6a 100644 --- a/crates/cli/src/commands/package.rs +++ b/crates/cli/src/commands/package.rs @@ -1,6 +1,7 @@ //! Package lock/sync/add/remove commands. use super::*; +use crate::extension_selection; /// `marqdo lock`. pub fn lock(json: bool) -> i32 { @@ -18,19 +19,40 @@ pub fn lock(json: bool) -> i32 { return 2; } }; - let graph = - match marqdo_package::PackageResolver::new(root.clone(), Some(manifest.clone()), None) - .with_fetch(true) - .with_unlocked(true) - .resolve_lock_graph() - { - Ok(graph) => graph, - Err(error) => { - eprintln!("error: {error}"); - return 1; - } - }; - let lockfile = graph.lock; + let resolver = marqdo_package::PackageResolver::new(root.clone(), Some(manifest.clone()), None) + .with_fetch(true) + .with_unlocked(true); + let graph = match resolver.resolve_lock_graph() { + Ok(graph) => graph, + Err(error) => { + eprintln!("error: {error}"); + return 1; + } + }; + let mut lockfile = graph.lock; + // Lock v4: record the exact extension identity declared by the project. + // Reading manifests/declared digests is static; no binary is loaded and no + // grant is implied. + let extension_sources = match resolver.resolve_extension_sources() { + Ok(sources) => sources, + Err(error) => { + eprintln!("error: extension selection: {error}"); + return 1; + } + }; + let identities = match extension_selection::extension_identities(&extension_sources) { + Ok(identities) => identities, + Err(error) => { + eprintln!("error[{}]: {}", error.code, error.message); + return error.exit_code; + } + }; + for identity in identities { + if let Err(error) = lockfile.merge_extension(identity) { + eprintln!("error[ext.extension_lock_mismatch]: {error}"); + return 1; + } + } let path = root.join(marqdo_package::LOCK_FILE); if let Err(error) = write_atomic(&path, lockfile.to_yaml().as_bytes()) { eprintln!("error: cannot write {}: {error}", path.display()); @@ -41,6 +63,7 @@ pub fn lock(json: bool) -> i32 { "command": "lock", "success": true, "packages": lockfile.packages, + "extensions": lockfile.extensions, })); } else { println!( @@ -90,6 +113,30 @@ pub fn sync(locked: bool, json: bool) -> i32 { let resolver = marqdo_package::PackageResolver::new(root, Some(manifest.clone()), Some(lockfile.clone())) .with_fetch(!locked); + // Verify the declared extension selection against the lock before using + // any materialized content. A missing/changed identity fails closed. + let extension_sources = match resolver.resolve_extension_sources() { + Ok(sources) => sources, + Err(error) => { + eprintln!("error: extension selection: {error}"); + return 1; + } + }; + let identities = match extension_selection::extension_identities(&extension_sources) { + Ok(identities) => identities, + Err(error) => { + eprintln!("error[{}]: {}", error.code, error.message); + return error.exit_code; + } + }; + if let Err(error) = lockfile.validate_extension_declarations(&manifest) { + eprintln!("error[ext.extension_lock_mismatch]: {error}"); + return 1; + } + if let Err(error) = extension_selection::validate_against_lock(&lockfile, &identities) { + eprintln!("error[{}]: {}", error.code, error.message); + return error.exit_code; + } if let Err(error) = resolver.materialize_env(&lockfile) { eprintln!("error: cannot materialize project environment: {error}"); return 1; diff --git a/crates/cli/src/commands/run.rs b/crates/cli/src/commands/run.rs index 4a2871f..7141309 100644 --- a/crates/cli/src/commands/run.rs +++ b/crates/cli/src/commands/run.rs @@ -20,6 +20,7 @@ pub fn execute( Ok(prepared) => prepared, Err(code) => return code, }; + let extension_sources = prepared.loaded.extension_sources.clone(); let project = prepared.loaded; let static_plugins = prepared.plugins; if project.has_errors() { @@ -51,6 +52,43 @@ pub fn execute( return 2; } }; + // A project-declared extension is activation, not authorization, but it is + // still third-party executable code. `pure`/`strict` never activate it. + if !extension_sources.is_empty() && mode != SandboxMode::Project { + let code = if mode == SandboxMode::Strict { + "strict.unsupported_grant" + } else { + "cli.invalid_grants" + }; + let message = format!( + "this project declares {} extension activation source(s) in {}; \ + --mode {} cannot activate third-party extensions. Use --mode project \ + with explicit grants, or remove the extensions block", + extension_sources.len(), + marqdo_package::MANIFEST_FILE, + settings.mode + ); + emit_error(json, command_name, code, &message); + return 2; + } + let mut kernel_paths = settings.kernels.clone(); + kernel_paths.extend( + extension_sources + .iter() + .filter(|extension| extension.kind == marqdo_package::ExtensionKind::Kernel) + .map(|extension| extension.path.clone()), + ); + kernel_paths.sort(); + kernel_paths.dedup(); + let mut provider_paths = settings.providers.clone(); + provider_paths.extend( + extension_sources + .iter() + .filter(|extension| extension.kind == marqdo_package::ExtensionKind::Provider) + .map(|extension| extension.path.clone()), + ); + provider_paths.sort(); + provider_paths.dedup(); if mode == SandboxMode::Strict { if let Err(error) = settings.strict_preflight() { emit_error(json, command_name, "strict.unsupported_grant", &error); @@ -121,7 +159,7 @@ pub fn execute( if let Some(timeout) = settings.timeout { host.budget.deadline = Some(Instant::now() + Duration::from_secs(timeout)); } - for kernel_path in &settings.kernels { + for kernel_path in &kernel_paths { if let Err(error) = load_kernel_descriptor(&mut host, kernel_path) { if json { output::print_json(&json!({"success": false, "error": error})); @@ -139,7 +177,7 @@ pub fn execute( // current target artifact, check the exact grant and reject duplicate // names/kinds *before* any provider or plugin worker process is spawned // (RFC-EXT-0001 §5.4.6/§18.2 T6–T7). - let providers = match provider_manifest::read_provider_manifests(&settings.providers) { + let providers = match provider_manifest::read_provider_manifests(&provider_paths) { Ok(providers) => providers, Err(error) => { emit_prepare_error(json, command_name, &error); diff --git a/crates/cli/src/extension_selection.rs b/crates/cli/src/extension_selection.rs new file mode 100644 index 0000000..cbf4e50 --- /dev/null +++ b/crates/cli/src/extension_selection.rs @@ -0,0 +1,318 @@ +//! Extension selection identities (RFC-EXT-0001 §6, ADR-0056). +//! +//! `marqdo.yaml.extensions` declares activation sources; the lock file records +//! the exact selected identity (manifest digest, artifact digest, target). +//! This module computes those identities from static manifests only: +//! +//! * it never `dlopen`s a library, spawns a worker or runs a script; +//! * it reads the declared artifact SHA-256 (plus the manifest bytes) so +//! `check`/`sync` can compare against the lock without requiring a built +//! binary; `run` separately verifies the actual artifact bytes before use. +//! +//! A declaration is visibility/activation, never a grant. Grants stay in +//! `--allow-plugin`/`--allow-provider`/`--allow-foreign` and the capability +//! broker. + +use std::collections::BTreeSet; + +use marqdo_package::{ExtensionKind, LockedExtension, ResolvedExtension}; +use sha2::{Digest, Sha256}; + +use crate::extension_descriptors::PrepareError; + +/// Compute the declared identity of every resolved extension source. +/// +/// Duplicate `(kind, name, logical_owner)` identities with different digests +/// are rejected deterministically; the same identity declared twice is +/// idempotent. +pub fn extension_identities( + resolved: &[ResolvedExtension], +) -> Result, PrepareError> { + let mut identities = Vec::new(); + let mut seen: BTreeSet<(ExtensionKind, String, String)> = BTreeSet::new(); + for extension in resolved { + let identity = match extension.kind { + ExtensionKind::Plugin => plugin_identity(extension)?, + ExtensionKind::Provider => provider_identity(extension)?, + ExtensionKind::Kernel => kernel_identity(extension)?, + }; + let key = ( + identity.kind, + identity.name.clone(), + identity.logical_owner.clone(), + ); + if !seen.insert(key.clone()) { + let existing = identities + .iter() + .find(|item: &&LockedExtension| { + item.kind == key.0 && item.name == key.1 && item.logical_owner == key.2 + }) + .expect("identity was inserted with its key"); + if existing == &identity { + continue; + } + return Err(PrepareError::option( + "ext.duplicate_callable", + format!( + "extension {} `{}` for owner `{}` is declared with two different manifests", + key.0, key.1, key.2 + ), + )); + } + identities.push(identity); + } + identities.sort_by(|left, right| { + (&left.kind, &left.name, &left.logical_owner).cmp(&( + &right.kind, + &right.name, + &right.logical_owner, + )) + }); + Ok(identities) +} + +fn plugin_identity(extension: &ResolvedExtension) -> Result { + let manifest_bytes = std::fs::read(&extension.path).map_err(|error| { + PrepareError::option( + "ext.manifest_unreadable", + format!( + "cannot read plugin manifest `{}`: {error}", + extension.path.display() + ), + ) + })?; + let manifest = marqdo_plugin_abi::PluginManifest::load(&extension.path) + .map_err(|error| PrepareError::option(error.code.clone(), error.message.clone()))?; + manifest + .validate() + .map_err(|error| PrepareError::option(error.code.clone(), error.message.clone()))?; + let host = marqdo_plugin_abi::HostTarget::current(); + let artifact = manifest + .resolve_artifact(&host) + .map_err(|error| PrepareError::option("ext.no_matching_artifact", error.message))?; + Ok(LockedExtension { + kind: ExtensionKind::Plugin, + logical_owner: extension.logical_owner.clone(), + name: manifest.plugin.name.clone(), + version: manifest.plugin.version.clone(), + manifest_digest: hex(&Sha256::digest(&manifest_bytes)), + artifact_digest: artifact.sha256.clone(), + selected_target: host.triple().to_string(), + source: extension.from.clone(), + }) +} + +fn provider_identity(extension: &ResolvedExtension) -> Result { + let manifest_bytes = std::fs::read(&extension.path).map_err(|error| { + PrepareError::option( + "ext.manifest_unreadable", + format!( + "cannot read provider manifest `{}`: {error}", + extension.path.display() + ), + ) + })?; + let manifest: serde_json::Value = serde_json::from_slice(&manifest_bytes).map_err(|error| { + PrepareError::option( + "ext.manifest_unreadable", + format!( + "invalid provider manifest `{}`: {error}", + extension.path.display() + ), + ) + })?; + if manifest.get("schema").and_then(serde_json::Value::as_str) != Some("marqdo.provider") { + return Err(PrepareError::option( + "ext.manifest_invalid", + format!( + "`{}` must declare schema `marqdo.provider`", + extension.path.display() + ), + )); + } + if manifest + .get("schema_version") + .and_then(serde_json::Value::as_u64) + != Some(2) + { + return Err(PrepareError::option( + "ext.unsupported_provider_schema", + format!( + "`{}` is not a provider schema version 2 manifest; run `marqdo lock` only with v2", + extension.path.display() + ), + )); + } + let name = manifest + .get("name") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + PrepareError::option( + "ext.manifest_invalid", + format!("`{}` has no name", extension.path.display()), + ) + })?; + let version = manifest + .get("version") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + PrepareError::option( + "ext.manifest_invalid", + format!("`{}` has no version", extension.path.display()), + ) + })?; + let target = marqdo_plugin_abi::HostTarget::current() + .triple() + .to_string(); + let artifacts = manifest + .get("artifacts") + .and_then(serde_json::Value::as_array) + .ok_or_else(|| { + PrepareError::option( + "ext.manifest_invalid", + format!("`{}` has no artifacts array", extension.path.display()), + ) + })?; + let artifact = artifacts + .iter() + .find(|artifact| { + artifact.get("target").and_then(serde_json::Value::as_str) == Some(&target) + }) + .ok_or_else(|| { + PrepareError::option( + "ext.no_matching_artifact", + format!("provider `{name}` has no artifact for target `{target}`"), + ) + })?; + let sha256 = artifact + .get("sha256") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + PrepareError::option( + "ext.manifest_invalid", + format!("provider `{name}` artifact has no sha256"), + ) + })?; + if sha256.len() != 64 || !sha256.bytes().all(|byte| byte.is_ascii_hexdigit()) { + return Err(PrepareError::option( + "ext.manifest_invalid", + format!("provider `{name}` artifact sha256 is not 64 hex characters"), + )); + } + Ok(LockedExtension { + kind: ExtensionKind::Provider, + logical_owner: extension.logical_owner.clone(), + name: name.to_string(), + version: version.to_string(), + manifest_digest: hex(&Sha256::digest(&manifest_bytes)), + artifact_digest: sha256.to_ascii_lowercase(), + selected_target: target, + source: extension.from.clone(), + }) +} + +fn kernel_identity(extension: &ResolvedExtension) -> Result { + let manifest_bytes = std::fs::read(&extension.path).map_err(|error| { + PrepareError::option( + "ext.manifest_unreadable", + format!( + "cannot read kernel descriptor `{}`: {error}", + extension.path.display() + ), + ) + })?; + let manifest: serde_json::Value = serde_json::from_slice(&manifest_bytes).map_err(|error| { + PrepareError::option( + "ext.manifest_unreadable", + format!( + "invalid kernel descriptor `{}`: {error}", + extension.path.display() + ), + ) + })?; + let name = manifest + .get("provider_id") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + PrepareError::option( + "ext.manifest_invalid", + format!("`{}` has no provider_id", extension.path.display()), + ) + })?; + let wire_version = manifest + .get("wire_version") + .and_then(serde_json::Value::as_u64) + .ok_or_else(|| { + PrepareError::option( + "ext.manifest_invalid", + format!("`{}` has no wire_version", extension.path.display()), + ) + })?; + if wire_version != 1 { + return Err(PrepareError::option( + "ext.manifest_invalid", + format!("kernel descriptor `{name}` uses unsupported wire_version {wire_version}"), + )); + } + let digest = manifest + .get("digest") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + PrepareError::option( + "ext.manifest_invalid", + format!("kernel descriptor `{name}` has no digest"), + ) + })?; + if digest.len() != 64 || !digest.bytes().all(|byte| byte.is_ascii_hexdigit()) { + return Err(PrepareError::option( + "ext.manifest_invalid", + format!("kernel descriptor `{name}` digest is not 64 hex characters"), + )); + } + let entrypoint = manifest + .get("entrypoint") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + PrepareError::option( + "ext.manifest_invalid", + format!("kernel descriptor `{name}` has no entrypoint"), + ) + })?; + // The descriptor's entrypoint is package/descriptor-relative or absolute + // (existing foreign.md contract). Reject explicit path escaping here. + if entrypoint.is_empty() { + return Err(PrepareError::option( + "ext.manifest_invalid", + format!("kernel descriptor `{name}` has an empty entrypoint"), + )); + } + Ok(LockedExtension { + kind: ExtensionKind::Kernel, + logical_owner: extension.logical_owner.clone(), + name: name.to_string(), + version: format!("wire:{wire_version}"), + manifest_digest: hex(&Sha256::digest(&manifest_bytes)), + artifact_digest: digest.to_ascii_lowercase(), + selected_target: marqdo_plugin_abi::HostTarget::current() + .triple() + .to_string(), + source: extension.from.clone(), + }) +} + +/// Validate that the lock still describes the current declared selection. +pub fn validate_against_lock( + lock: &marqdo_package::LockFile, + identities: &[LockedExtension], +) -> Result<(), PrepareError> { + lock.validate_extension_selection(identities) + .map_err(|message| PrepareError::load("ext.extension_lock_mismatch", message)) +} + +fn hex(bytes: &[u8]) -> String { + let mut out = String::with_capacity(bytes.len() * 2); + for byte in bytes { + out.push_str(&format!("{byte:02x}")); + } + out +} diff --git a/crates/cli/src/loader.rs b/crates/cli/src/loader.rs index 2072935..c2a99f0 100644 --- a/crates/cli/src/loader.rs +++ b/crates/cli/src/loader.rs @@ -163,6 +163,12 @@ pub struct LoadedProject { /// These are ordinary `Marqdo` bindings; the runtime has no print special /// case and no `Program.print_override`. pub official_aliases: Vec, + /// Project manifest, when the entry belongs to a project. + pub manifest: Option, + /// Verified lock file, when present (lock v4). + pub lock: Option, + /// Resolved `marqdo.yaml.extensions` activation sources (static paths only). + pub extension_sources: Vec, } impl LoadedProject { @@ -271,7 +277,7 @@ pub fn load(entry: &Path) -> Result { } None }; - let resolver = PackageResolver::new(root.clone(), manifest.clone(), lock); + let resolver = PackageResolver::new(root.clone(), manifest.clone(), lock.clone()); let entry_module = resolver.resolve_entry(&entry)?; let stdlib = StdlibBundle::discover().map_err(|error| format!("stdlib: {error}"))?; let mut loader = Loader { @@ -290,6 +296,10 @@ pub fn load(entry: &Path) -> Result { // the target module/function; only aliases the program actually references // load their module, and the runtime binds them as ordinary Marqdo calls. let official_aliases = loader.load_official_aliases()?; + let extension_sources = loader + .resolver + .resolve_extension_sources() + .map_err(|error| format!("extension selection: {error}"))?; let stdlib_info = loader.stdlib.as_ref().map(|bundle| StdlibInfo { version: bundle.version().to_string(), @@ -314,6 +324,9 @@ pub fn load(entry: &Path) -> Result { diagnostics: loader.diagnostics, stdlib: stdlib_info, official_aliases, + manifest, + lock, + extension_sources, }) } diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index 108953e..d63e92c 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -8,6 +8,7 @@ mod analysis; mod args; mod commands; mod extension_descriptors; +mod extension_selection; mod loader; mod output; mod project_prepare; @@ -261,6 +262,12 @@ struct InspectArgs { /// Print sandbox/platform capability report. #[arg(long)] platform: bool, + /// Print declared/loaded extension identities (never grants). + #[arg(long)] + extensions: bool, + /// Explicitly selected plugin manifest for extension inspection (repeatable). + #[arg(long = "plugin", value_name = "PATH")] + plugin: Vec, /// Machine readable JSON output. #[arg(long)] json: bool, @@ -421,8 +428,9 @@ fn dispatch() -> i32 { ir: args.ir, refs: args.refs, platform: args.platform, + extensions: args.extensions, }; - commands::inspect(args.file.as_deref(), &flags, args.json) + commands::inspect(args.file.as_deref(), &flags, args.json, &args.plugin) } Command::Run(args) => commands::execute( args.file.as_deref(), diff --git a/crates/cli/src/project_prepare.rs b/crates/cli/src/project_prepare.rs index a7c5edd..856c1d6 100644 --- a/crates/cli/src/project_prepare.rs +++ b/crates/cli/src/project_prepare.rs @@ -14,8 +14,10 @@ use std::path::PathBuf; use marqdo_ir::callable::CallableRegistry; +use marqdo_package::{ExtensionKind, LockedExtension}; use crate::extension_descriptors::{read_plugin_descriptors, PrepareError, StaticPluginSet}; +use crate::extension_selection; use crate::loader::{self, FileDiagnostic, LoadedProject}; /// Options that influence static preparation. @@ -37,6 +39,11 @@ pub struct PreparedProgram { /// evidence) can report the same static table without rebuilding it. #[allow(dead_code)] pub registry: CallableRegistry, + /// Static identities of the declared extensions, already compared with the + /// lock when one is present (never grants). Consumed by + /// `inspect --extensions` and evidence. + #[allow(dead_code)] + pub extension_identities: Vec, } /// Prepare one document without executing any project or extension code. @@ -49,8 +56,48 @@ pub fn prepare( let mut loaded = loader::load(file.unwrap_or_else(|| std::path::Path::new("index.mq.md"))) .map_err(|message| default_load_error(&message))?; - // T2: extension descriptors are read from manifests only. - let plugins = read_plugin_descriptors(&options.plugins)?; + // T2: extension descriptors are read from manifests only. Explicit CLI + // `--plugin` paths are merged with the project's declared and locked + // activation sources; CLI paths remain a development form and grant + // nothing by themselves. + let mut plugin_paths = options.plugins.clone(); + for extension in &loaded.extension_sources { + if extension.kind == ExtensionKind::Plugin { + plugin_paths.push(extension.path.clone()); + } + } + plugin_paths.sort(); + plugin_paths.dedup(); + let plugins = read_plugin_descriptors(&plugin_paths)?; + + // T3a: declared extensions must have a locked identity (lock v4). The + // declaration is activation, not authorization; a mismatched/missing lock + // is fail-closed rather than silently "latest". + let extension_identities = + extension_selection::extension_identities(&loaded.extension_sources)?; + if !loaded.extension_sources.is_empty() { + match &loaded.lock { + Some(lock) => { + lock.validate_extension_declarations( + loaded + .manifest + .as_ref() + .expect("extension sources require a project manifest"), + ) + .map_err(|message| PrepareError::load("ext.extension_lock_mismatch", message))?; + extension_selection::validate_against_lock(lock, &extension_identities)?; + } + None => { + return Err(PrepareError::load( + "ext.extension_lock_mismatch", + format!( + "the project declares {} extension(s) but marqdo.lock is missing; run `marqdo lock` first", + loaded.extension_sources.len() + ), + )); + } + } + } // T3: unique static table. Duplicate exports/builtin shadowing are rejected // before any call site can be bound. @@ -126,6 +173,7 @@ pub fn prepare( loaded, plugins, registry, + extension_identities, }) } diff --git a/crates/cli/tests/extension_selection.rs b/crates/cli/tests/extension_selection.rs new file mode 100644 index 0000000..9eaf69a --- /dev/null +++ b/crates/cli/tests/extension_selection.rs @@ -0,0 +1,495 @@ +//! Extension selection and lock v4 conformance (RFC-EXT-0001 §6.6). +//! +//! These are black-box CLI tests: `marqdo.yaml.extensions` declares activation +//! sources, `marqdo lock` records their exact identity, and `check`/`run` +//! compare the current declaration with the lock before using any extension. +#![cfg(unix)] + +use std::path::{Path, PathBuf}; +use std::process::{Command, Output}; +use std::sync::atomic::{AtomicU32, Ordering}; + +use marqdo_plugin_abi::{sha256_file, HostTarget}; + +static COUNTER: AtomicU32 = AtomicU32::new(0); + +struct TempDir(PathBuf); + +impl TempDir { + fn new(tag: &str) -> Self { + let id = COUNTER.fetch_add(1, Ordering::Relaxed); + let path = std::env::temp_dir().join(format!( + "marqdo-ext-selection-{}-{id}-{tag}", + std::process::id() + )); + std::fs::create_dir_all(&path).unwrap(); + Self(path) + } + + fn path(&self) -> &Path { + &self.0 + } +} + +impl Drop for TempDir { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.0); + } +} + +fn repo() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf() +} + +fn cli(args: &[&str], dir: &Path) -> Output { + Command::new(env!("CARGO_BIN_EXE_marqdo")) + .args(args) + .current_dir(dir) + .output() + .unwrap() +} + +fn stderr(output: &Output) -> String { + String::from_utf8_lossy(&output.stderr).to_string() +} + +fn stdout(output: &Output) -> String { + String::from_utf8_lossy(&output.stdout).to_string() +} + +fn have_cc() -> bool { + Command::new("cc") + .arg("--version") + .output() + .map(|output| output.status.success()) + .unwrap_or(false) +} + +fn library_name() -> &'static str { + if cfg!(target_os = "macos") { + "plugin.dylib" + } else { + "plugin.so" + } +} + +fn compile_plugin(dir: &Path) -> PathBuf { + let output = dir.join(library_name()); + let flags: &[&str] = if cfg!(target_os = "macos") { + &["-dynamiclib", "-fPIC"] + } else { + &["-shared", "-fPIC"] + }; + let status = Command::new("cc") + .args(["-std=c11", "-O2"]) + .args(flags) + .arg("-I") + .arg(repo().join("include")) + .arg(repo().join("examples/plugin-demo/solver.c")) + .arg("-o") + .arg(&output) + .status() + .expect("cc"); + assert!(status.success(), "cc failed"); + output +} + +fn write_plugin_manifest(adapter: &Path, artifact: &Path) { + let sha = sha256_file(artifact).unwrap(); + std::fs::write( + adapter.join("plugin.yaml"), + format!( + r#"plugin: + name: demo.solver + version: 0.1.0 + abi: "^1.0" + kind: worker + exports: + - name: solver.add + signature: "($a: Int, $b: Int) -> Int" + requires: + - math.compute + artifacts: + - target: {} + file: {} + sha256: "{sha}" +"#, + HostTarget::current().triple(), + artifact.file_name().unwrap().to_string_lossy() + ), + ) + .unwrap(); +} + +fn write_project(project: &Path, extension_yaml: &str) { + std::fs::create_dir_all(project.join("adapter")).unwrap(); + std::fs::write( + project.join("marqdo.yaml"), + format!("project:\n name: ext-project\n version: 0.1.0\n{extension_yaml}"), + ) + .unwrap(); + std::fs::write( + project.join("index.mq.md"), + "## @exec main\n\n`return solver.add(a=20, b=22)`\n", + ) + .unwrap(); +} + +/// Build a project with one worker plugin declared through `extensions:`. +fn plugin_project(tag: &str) -> Option { + if !have_cc() { + eprintln!("skipping: cc is unavailable"); + return None; + } + let project = TempDir::new(tag); + let artifact = compile_plugin(&project.path().join("adapter").create_dir_all_ok()); + write_plugin_manifest(&project.path().join("adapter"), &artifact); + write_project( + project.path(), + "extensions:\n plugins:\n - from: ./adapter/plugin.yaml\n", + ); + Some(project) +} + +fn lock(project: &Path) -> Output { + cli(&["lock"], project) +} + +#[test] +fn declared_plugin_extension_locks_and_runs() { + let Some(project) = plugin_project("positive") else { + return; + }; + let locked = lock(project.path()); + assert_eq!(locked.status.code(), Some(0), "{}", stderr(&locked)); + let lock_text = std::fs::read_to_string(project.path().join("marqdo.lock")).unwrap(); + assert!(lock_text.contains("lock_version: 4")); + assert!(lock_text.contains("kind: plugin")); + assert!(lock_text.contains("source: ./adapter/plugin.yaml")); + let check = cli(&["check", "index.mq.md"], project.path()); + assert_eq!(check.status.code(), Some(0), "{}", stderr(&check)); + let run = cli( + &["run", "index.mq.md", "--allow-plugin", "demo.solver"], + project.path(), + ); + assert_eq!(run.status.code(), Some(0), "{}", stderr(&run)); +} + +#[test] +fn declared_extension_without_a_lock_is_fail_closed() { + let Some(project) = plugin_project("no-lock") else { + return; + }; + let check = cli(&["check", "index.mq.md"], project.path()); + assert_eq!(check.status.code(), Some(1), "{}", stderr(&check)); + assert!( + stderr(&check).contains("ext.extension_lock_mismatch"), + "{}", + stderr(&check) + ); + assert!(stderr(&check).contains("marqdo lock"), "{}", stderr(&check)); +} + +#[test] +fn changed_manifest_after_lock_is_rejected() { + let Some(project) = plugin_project("manifest-change") else { + return; + }; + assert_eq!(lock(project.path()).status.code(), Some(0)); + let adapter = project.path().join("adapter"); + let manifest = std::fs::read_to_string(adapter.join("plugin.yaml")).unwrap(); + std::fs::write( + adapter.join("plugin.yaml"), + manifest.replace("version: 0.1.0", "version: 0.2.0"), + ) + .unwrap(); + let check = cli(&["check", "index.mq.md"], project.path()); + assert_eq!(check.status.code(), Some(1), "{}", stderr(&check)); + assert!( + stderr(&check).contains("ext.extension_lock_mismatch"), + "{}", + stderr(&check) + ); +} + +#[test] +fn changed_artifact_after_lock_fails_before_main_runs() { + let Some(project) = plugin_project("artifact-change") else { + return; + }; + assert_eq!(lock(project.path()).status.code(), Some(0)); + let artifact = project.path().join("adapter").join(library_name()); + let mut bytes = std::fs::read(&artifact).unwrap(); + bytes.push(0); + std::fs::write(&artifact, bytes).unwrap(); + let run = cli( + &["run", "index.mq.md", "--allow-plugin", "demo.solver"], + project.path(), + ); + assert_eq!(run.status.code(), Some(1), "{}", stderr(&run)); + assert!( + stderr(&run).contains("ext.artifact_integrity"), + "{}", + stderr(&run) + ); + assert!(stdout(&run).is_empty()); +} + +#[test] +fn extension_source_traversal_is_rejected_at_load() { + let project = TempDir::new("traversal"); + write_project( + project.path(), + "extensions:\n plugins:\n - from: ../escape/plugin.yaml\n", + ); + let check = cli(&["check", "index.mq.md"], project.path()); + assert_eq!(check.status.code(), Some(2), "{}", stderr(&check)); + assert!( + stderr(&check).contains("extension") || stderr(&check).contains("must stay inside"), + "{}", + stderr(&check) + ); +} + +#[test] +fn undeclared_dependency_alias_is_rejected() { + let project = TempDir::new("alias"); + write_project( + project.path(), + "extensions:\n plugins:\n - from: \"@missing/plugin.yaml\"\n", + ); + let check = cli(&["check", "index.mq.md"], project.path()); + assert_eq!(check.status.code(), Some(2), "{}", stderr(&check)); + assert!( + stderr(&check).contains("not a declared dependency"), + "{}", + stderr(&check) + ); +} + +#[test] +fn lock_v3_is_rejected_with_an_explicit_migration() { + let Some(project) = plugin_project("v3") else { + return; + }; + let lock_path = project.path().join("marqdo.lock"); + std::fs::write( + &lock_path, + "lock_version: 3\ncontent_profile_version: 1\nroot:\n package: root:ext-project@0.1.0\n manifest_digest: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\npackages: []\nroot_dependencies: {}\n", + ) + .unwrap(); + let check = cli(&["check", "index.mq.md"], project.path()); + assert!(!check.status.success(), "{}", stderr(&check)); + assert!( + stderr(&check).contains("lock v3 is superseded by lock v4"), + "{}", + stderr(&check) + ); + assert!(!stdout(&check).contains("ok")); +} + +#[test] +fn pure_and_strict_refuse_project_declared_extensions() { + let Some(project) = plugin_project("modes") else { + return; + }; + assert_eq!(lock(project.path()).status.code(), Some(0)); + let pure = cli(&["run", "index.mq.md", "--mode", "pure"], project.path()); + assert_eq!(pure.status.code(), Some(2), "{}", stderr(&pure)); + let strict = cli(&["run", "index.mq.md", "--mode", "strict"], project.path()); + assert_eq!(strict.status.code(), Some(2), "{}", stderr(&strict)); +} + +#[test] +fn provider_extension_is_locked_but_still_requires_an_exact_grant() { + let project = TempDir::new("provider"); + std::fs::create_dir_all(project.path().join("vendor/lib")).unwrap(); + let artifact = project.path().join("vendor/lib/libsensor.so"); + std::fs::write(&artifact, b"fixture provider artifact").unwrap(); + let sha = sha256_file(&artifact).unwrap(); + std::fs::write( + project.path().join("vendor/provider.json"), + format!( + r#"{{ + "schema": "marqdo.provider", + "schema_version": 2, + "name": "demo.sensor", + "version": "0.1.0", + "abi": "c-abi-1", + "transport": "worker", + "kinds": ["demo.sensor"], + "artifacts": [{{"target": "{}", "path": "lib/libsensor.so", "sha256": "{sha}"}}], + "permissions_requested": [] +}}"#, + HostTarget::current().triple() + ), + ) + .unwrap(); + write_project( + project.path(), + "extensions:\n providers:\n - from: ./vendor/provider.json\n", + ); + std::fs::write( + project.path().join("index.mq.md"), + "## @exec main\n\n`流 = resource.open(\"demo.sensor\", \"unit-1\", [\"read\"])`\n\n`return resource.close(流)`\n", + ) + .unwrap(); + let locked = lock(project.path()); + assert_eq!(locked.status.code(), Some(0), "{}", stderr(&locked)); + let lock_text = std::fs::read_to_string(project.path().join("marqdo.lock")).unwrap(); + assert!(lock_text.contains("kind: provider")); + // Locked identity is not a grant: the provider is selected from the + // manifest but `--allow-provider` is still required before any worker. + let no_grant = cli(&["run", "index.mq.md"], project.path()); + assert_eq!(no_grant.status.code(), Some(1), "{}", stderr(&no_grant)); + assert!( + stderr(&no_grant).contains("permission.denied"), + "{}", + stderr(&no_grant) + ); +} + +trait CreateDirOk { + fn create_dir_all_ok(&self) -> PathBuf; +} + +impl CreateDirOk for PathBuf { + fn create_dir_all_ok(&self) -> PathBuf { + std::fs::create_dir_all(self).unwrap(); + self.clone() + } +} + +#[test] +fn inspect_extensions_reports_lock_identity_without_executing() { + let Some(project) = plugin_project("inspect") else { + return; + }; + assert_eq!(lock(project.path()).status.code(), Some(0)); + let artifact = project.path().join("adapter").join(library_name()); + std::fs::remove_file(&artifact).unwrap(); + // Inspection is static: a missing artifact does not prevent identity + // reporting, and no worker is started. + let output = cli( + &["inspect", "index.mq.md", "--extensions", "--json"], + project.path(), + ); + assert_eq!(output.status.code(), Some(0), "{}", stderr(&output)); + let report: serde_json::Value = serde_json::from_str(&stdout(&output)).unwrap(); + let declared = report["extensions"]["declared"].as_array().unwrap(); + assert_eq!(declared.len(), 1); + assert_eq!(declared[0]["kind"], "plugin"); + assert_eq!(declared[0]["status"], "locked"); + assert_eq!(declared[0]["authorization"], "not_granted"); + assert_eq!(declared[0]["name"], "demo.solver"); + let plugins = report["extensions"]["plugins"].as_array().unwrap(); + assert_eq!(plugins[0]["package"], "demo.solver"); + assert_eq!(plugins[0]["exports"][0]["name"], "solver.add"); + assert_eq!(plugins[0]["authorization"], "not_granted"); +} + +#[test] +fn cli_and_manifest_selection_of_the_same_plugin_is_idempotent() { + let Some(project) = plugin_project("idempotent") else { + return; + }; + assert_eq!(lock(project.path()).status.code(), Some(0)); + let adapter = project.path().join("adapter"); + let check = cli( + &[ + "check", + "index.mq.md", + "--plugin", + adapter.to_str().unwrap(), + ], + project.path(), + ); + assert_eq!(check.status.code(), Some(0), "{}", stderr(&check)); +} + +#[test] +fn two_different_manifests_for_one_package_conflict() { + let Some(project) = plugin_project("conflict") else { + return; + }; + let second = project.path().join("adapter-v2"); + std::fs::create_dir_all(&second).unwrap(); + let artifact = project.path().join("adapter").join(library_name()); + std::fs::copy(&artifact, second.join(library_name())).unwrap(); + write_plugin_manifest(&second, &second.join(library_name())); + let manifest = std::fs::read_to_string(second.join("plugin.yaml")).unwrap(); + std::fs::write( + second.join("plugin.yaml"), + manifest.replace("version: 0.1.0", "version: 0.2.0"), + ) + .unwrap(); + let check = cli( + &[ + "check", + "index.mq.md", + "--plugin", + project.path().join("adapter").to_str().unwrap(), + "--plugin", + second.to_str().unwrap(), + ], + project.path(), + ); + assert_eq!(check.status.code(), Some(2), "{}", stderr(&check)); + assert!( + stderr(&check).contains("ext.duplicate_callable"), + "{}", + stderr(&check) + ); +} + +#[test] +fn declared_kernel_extension_is_activated_from_the_manifest() { + let project = TempDir::new("kernel"); + std::fs::create_dir_all(project.path().join("kernel")).unwrap(); + let provider = project.path().join("kernel/mocklang"); + std::fs::write( + &provider, + "#!/bin/sh\ncat >/dev/null\nprintf '%s' '{\"type\":\"int\",\"decimal\":\"42\"}'\n", + ) + .unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&provider, std::fs::Permissions::from_mode(0o755)).unwrap(); + } + let digest = sha256_file(&provider).unwrap(); + std::fs::write( + project.path().join("kernel/kernel.json"), + format!( + r#"{{ + "provider_id": "mocklang-provider", + "languages": ["mocklang-only"], + "wire_version": 1, + "entrypoint": "{}", + "permissions_requested": [], + "digest": "{digest}" +}}"#, + provider.display() + ), + ) + .unwrap(); + write_project( + project.path(), + "extensions:\n kernels:\n - from: ./kernel/kernel.json\n", + ); + std::fs::write( + project.path().join("index.mq.md"), + "## @exec main\n\n`结果 = foreign.call(ref(\"#代码\"), {})`\n\n`return 结果`\n\n### @knowledge 代码\n\n```mocklang-only\ninert\n```\n", + ) + .unwrap(); + assert_eq!(lock(project.path()).status.code(), Some(0)); + let lock_text = std::fs::read_to_string(project.path().join("marqdo.lock")).unwrap(); + assert!(lock_text.contains("kind: kernel")); + let run = cli(&["run", "index.mq.md", "--allow-foreign"], project.path()); + assert_eq!(run.status.code(), Some(0), "{}", stderr(&run)); +} diff --git a/crates/cli/tests/package_env.rs b/crates/cli/tests/package_env.rs index 26d00f7..b98247d 100644 --- a/crates/cli/tests/package_env.rs +++ b/crates/cli/tests/package_env.rs @@ -170,7 +170,7 @@ fn pinned_git_dependency_locks_syncs_and_runs() { String::from_utf8_lossy(&lock.stderr) ); let lock_text = std::fs::read_to_string(root.join("marqdo.lock")).unwrap(); - assert!(lock_text.contains("lock_version: 3")); + assert!(lock_text.contains("lock_version: 4")); assert!(lock_text.contains("root_dependencies:")); assert!(lock_text.contains("instance_id: dep:remote@0.3.0#")); assert!(lock_text.contains("kind: git")); diff --git a/crates/marqdo-package/src/lib.rs b/crates/marqdo-package/src/lib.rs index 0ce27f8..8f77c72 100644 --- a/crates/marqdo-package/src/lib.rs +++ b/crates/marqdo-package/src/lib.rs @@ -116,6 +116,233 @@ pub struct NetRequests { pub hosts: Vec, } +/// Extension category explicitly selected by a project manifest +/// (RFC-EXT-0001 §6). +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum ExtensionKind { + /// Callable ABI plugin (`plugin.yaml`). + Plugin, + /// Foreign kernel descriptor (`kernel.json`). + Kernel, + /// Resource provider manifest (`provider.json`). + Provider, +} + +impl ExtensionKind { + /// Stable lowercase name used in lock files and diagnostics. + pub const fn as_str(self) -> &'static str { + match self { + ExtensionKind::Plugin => "plugin", + ExtensionKind::Kernel => "kernel", + ExtensionKind::Provider => "provider", + } + } + + /// Parse the stable lowercase name. + pub fn parse(value: &str) -> Option { + match value { + "plugin" => Some(ExtensionKind::Plugin), + "kernel" => Some(ExtensionKind::Kernel), + "provider" => Some(ExtensionKind::Provider), + _ => None, + } + } +} + +impl std::fmt::Display for ExtensionKind { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(self.as_str()) + } +} + +/// One explicit extension activation source. +/// +/// Allowed forms are deliberately narrow (RFC-EXT-0001 §6.3): +/// +/// * `./relative/path` — a regular file inside the current project; +/// * `@alias/relative/path` — a regular file inside a locked dependency. +/// +/// Absolute paths, `..` traversal, HTTP(S) URLs, globs and implicit searches +/// are rejected. A declaration is activation/visibility, never a grant. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct ExtensionSource { + /// Declared source string. + pub from: String, +} + +impl ExtensionSource { + /// Parse and validate the source form. + pub fn resolve( + &self, + dependencies: &BTreeMap, + ) -> Result { + let value = self.from.trim(); + if value.is_empty() { + return Err("extension `from` must not be empty".to_string()); + } + if let Some(rest) = value.strip_prefix("./") { + let relative = safe_relative(rest, "extension source")?; + return Ok(ResolvedExtensionSource::ProjectRelative(relative)); + } + if let Some(rest) = value.strip_prefix('@') { + let (alias, relative) = rest.split_once('/').ok_or_else(|| { + format!("extension source `{value}` must be `@alias/relative/path`") + })?; + if alias.is_empty() { + return Err(format!("extension source `{value}` has an empty alias")); + } + if !dependencies.contains_key(alias) { + return Err(format!( + "extension source `{value}` uses `@{alias}`, which is not a declared dependency" + )); + } + let relative = safe_relative(relative, "extension source")?; + return Ok(ResolvedExtensionSource::Dependency { + alias: alias.to_string(), + relative, + }); + } + Err(format!( + "extension source `{value}` must be `./relative/path` or `@alias/relative/path`; absolute paths, URLs and implicit discovery are not allowed" + )) + } +} + +/// A resolved extension activation source (regular file inside a package). +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ResolvedExtension { + /// Extension category declared by the manifest block. + pub kind: ExtensionKind, + /// Declared `from` string (identity key in the lock). + pub from: String, + /// Canonical manifest/descriptor file path. + pub path: PathBuf, + /// Logical owner package instance (`root:...` or a dependency instance id). + pub logical_owner: String, + /// Canonical package root that contains the file. + pub package_root: PathBuf, +} + +/// A parsed extension source relative to a project or dependency. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ResolvedExtensionSource { + /// `./relative/path` below the project root. + ProjectRelative(PathBuf), + /// `@alias/relative/path` below a declared dependency. + Dependency { + /// Dependency alias. + alias: String, + /// Package-relative path. + relative: PathBuf, + }, +} + +fn resolve_extension_file( + package_root: &Path, + relative: &Path, + kind: ExtensionKind, + from: &str, +) -> Result { + let candidate = package_root.join(relative); + let canonical = candidate.canonicalize().map_err(|error| { + format!( + "cannot open {kind} extension source `{from}` at `{}`: {error}", + candidate.display() + ) + })?; + if !canonical.starts_with(package_root) { + return Err(format!( + "{kind} extension source `{from}` escapes its owning package" + )); + } + let metadata = std::fs::metadata(&canonical) + .map_err(|error| format!("cannot stat {kind} extension source `{from}`: {error}"))?; + if !metadata.is_file() { + return Err(format!( + "{kind} extension source `{from}` must be a regular file" + )); + } + Ok(canonical) +} + +fn safe_relative(value: &str, what: &str) -> Result { + if value.is_empty() { + return Err(format!("{what} path must not be empty")); + } + let mut normalised = PathBuf::new(); + for component in Path::new(value).components() { + match component { + std::path::Component::Normal(part) => normalised.push(part), + std::path::Component::CurDir => {} + std::path::Component::ParentDir + | std::path::Component::RootDir + | std::path::Component::Prefix(_) => { + return Err(format!( + "{what} path `{value}` must stay inside its package (no `..`, absolute or rooted paths)" + )) + } + } + } + if normalised.as_os_str().is_empty() { + return Err(format!("{what} path must not be empty")); + } + Ok(normalised) +} + +/// The `extensions:` block of `marqdo.yaml` (explicit activation sources). +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct Extensions { + /// Callable plugin manifests. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub plugins: Vec, + /// Foreign kernel descriptors. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub kernels: Vec, + /// Resource provider manifests. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub providers: Vec, +} + +impl Extensions { + /// Whether no extension is declared. + pub fn is_empty(&self) -> bool { + self.plugins.is_empty() && self.kernels.is_empty() && self.providers.is_empty() + } + + /// Iterate `(kind, source)` in deterministic kind order. + pub fn iter(&self) -> impl Iterator { + self.plugins + .iter() + .map(|source| (ExtensionKind::Plugin, source)) + .chain( + self.kernels + .iter() + .map(|source| (ExtensionKind::Kernel, source)), + ) + .chain( + self.providers + .iter() + .map(|source| (ExtensionKind::Provider, source)), + ) + } + + fn validate(&self, dependencies: &BTreeMap) -> Result<(), String> { + let mut seen: std::collections::BTreeSet<(ExtensionKind, String)> = + std::collections::BTreeSet::new(); + for (kind, source) in self.iter() { + source.resolve(dependencies)?; + if !seen.insert((kind, source.from.clone())) { + return Err(format!( + "{kind} extension source `{}` is declared more than once", + source.from + )); + } + } + Ok(()) + } +} + /// The `marqdo.yaml` project manifest. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct Manifest { @@ -130,6 +357,9 @@ pub struct Manifest { /// Capability requests. #[serde(default, skip_serializing_if = "CapabilityRequests::is_empty")] pub capabilities: CapabilityRequests, + /// Explicit extension activation sources (never authorization). + #[serde(default, skip_serializing_if = "Option::is_none")] + pub extensions: Option, } /// A requested plugin (installation is not authorization). @@ -173,6 +403,9 @@ impl Manifest { )); } } + if let Some(extensions) = &self.extensions { + extensions.validate(&self.dependencies)?; + } Ok(()) } @@ -192,6 +425,7 @@ impl Manifest { dependencies: BTreeMap::new(), plugins: BTreeMap::new(), capabilities: CapabilityRequests::default(), + extensions: None, } } } @@ -392,6 +626,59 @@ impl PackageResolver { &self.root } + /// Resolve and validate every `marqdo.yaml.extensions` source to a real + /// regular file inside its owning package (RFC-EXT-0001 §6.3). + /// + /// This is pure static path resolution: no extension binary is loaded, no + /// script is executed and no grant is implied. Resolution order is sorted by + /// `(kind, from)` so the resulting fact table never depends on YAML order. + pub fn resolve_extension_sources(&self) -> Result, String> { + let Some(manifest) = &self.manifest else { + return Ok(Vec::new()); + }; + let Some(extensions) = &manifest.extensions else { + return Ok(Vec::new()); + }; + let root = self + .root + .canonicalize() + .map_err(|error| format!("cannot resolve project root: {error}"))?; + let owner = root_instance(Some(manifest)); + let mut resolved = Vec::new(); + for (kind, source) in extensions.iter() { + match source.resolve(&manifest.dependencies)? { + ResolvedExtensionSource::ProjectRelative(relative) => { + let path = resolve_extension_file(&root, &relative, kind, &source.from)?; + resolved.push(ResolvedExtension { + kind, + from: source.from.clone(), + path, + logical_owner: owner.clone(), + package_root: root.clone(), + }); + } + ResolvedExtensionSource::Dependency { alias, relative } => { + let module = self.resolve_alias(&alias)?; + let package_root = + module.package.module_root.canonicalize().map_err(|error| { + format!("cannot resolve package root for dependency `{alias}`: {error}") + })?; + let path = + resolve_extension_file(&package_root, &relative, kind, &source.from)?; + resolved.push(ResolvedExtension { + kind, + from: source.from.clone(), + path, + logical_owner: module.package.instance_id.clone(), + package_root, + }); + } + } + } + resolved.sort_by(|left, right| (&left.kind, &left.from).cmp(&(&right.kind, &right.from))); + Ok(resolved) + } + /// The project manifest, if the entry is part of a project. pub fn manifest(&self) -> Option<&Manifest> { self.manifest.as_ref() @@ -849,6 +1136,7 @@ impl PackageResolver { }, packages, root_dependencies, + extensions: Vec::new(), }; lock.validate()?; Ok(ResolvedLockGraph { @@ -1115,7 +1403,7 @@ fn package_name_version(manifest: &Option, alias: &str) -> (String, St } /// Lock format version written by this build. -pub const LOCK_VERSION: u32 = 3; +pub const LOCK_VERSION: u32 = 4; /// One package instance in `marqdo.lock` v3. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -1138,7 +1426,7 @@ pub struct LockedPackage { pub dependencies: BTreeMap, } -/// A structured package source in lock v3. +/// A structured package source in lock v4. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(tag = "kind", rename_all = "snake_case")] pub enum LockedSource { @@ -1194,7 +1482,7 @@ impl LockedSource { } } -/// Root entry of a lock v3 graph. +/// Root entry of a lock v4 graph. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct LockedRoot { /// Logical root package instance (`root:@`). @@ -1212,6 +1500,32 @@ impl Default for LockedRoot { } } +/// One locked extension identity (lock v4). +/// +/// Records the exact selected extension source, manifest digest, artifact +/// digest and target so a later `run` can prove it uses the same bytes the +/// project locked. A lock is identity, not authorization: grants remain +/// separate CLI/operator decisions. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct LockedExtension { + /// Extension category. + pub kind: ExtensionKind, + /// Owner package instance (`root:...` or a dependency instance id). + pub logical_owner: String, + /// Stable declared name (plugin package, kernel provider id, provider name). + pub name: String, + /// Declared extension version. + pub version: String, + /// SHA-256 of the exact manifest file bytes (lowercase hex). + pub manifest_digest: String, + /// SHA-256 of the selected target artifact (lowercase hex). + pub artifact_digest: String, + /// Selected host target triple. + pub selected_target: String, + /// Source string as declared in `marqdo.yaml.extensions`. + pub source: String, +} + /// The `marqdo.lock` package-instance graph. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct LockFile { @@ -1227,6 +1541,9 @@ pub struct LockFile { /// Root alias -> target package instance. #[serde(default)] pub root_dependencies: BTreeMap, + /// Locked extension identities, sorted by `(kind, name, logical_owner)`. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub extensions: Vec, } impl Default for LockFile { @@ -1237,6 +1554,7 @@ impl Default for LockFile { root: LockedRoot::default(), packages: Vec::new(), root_dependencies: BTreeMap::new(), + extensions: Vec::new(), } } } @@ -1264,7 +1582,7 @@ pub fn manifest_digest(manifest: &Manifest) -> String { } impl LockFile { - /// Parse and validate a lockfile. Only lock v3 is accepted. + /// Parse and validate a lockfile. Only lock v4 is accepted. pub fn from_yaml(yaml: &str) -> Result { #[derive(Deserialize)] struct VersionProbe { @@ -1273,8 +1591,13 @@ impl LockFile { let probe: VersionProbe = serde_yaml::from_str(yaml).map_err(|error| format!("invalid {LOCK_FILE}: {error}"))?; if probe.lock_version != LOCK_VERSION { + if probe.lock_version == LOCK_VERSION - 1 { + return Err(format!( + "lock v3 is superseded by lock v4 (extension identities); run `marqdo lock` to migrate explicitly. {LOCK_FILE} was not rewritten" + )); + } return Err(format!( - "unsupported lock_version {}; lock v3 is required. Run `marqdo lock` to regenerate the graph", + "unsupported lock_version {}; lock v{LOCK_VERSION} is required. Run `marqdo lock` to regenerate the graph", probe.lock_version )); } @@ -1282,6 +1605,7 @@ impl LockFile { serde_yaml::from_str(yaml).map_err(|error| format!("invalid {LOCK_FILE}: {error}"))?; lock.packages .sort_by(|a, b| a.instance_id.cmp(&b.instance_id)); + lock.sort_extensions(); lock.validate()?; Ok(lock) } @@ -1299,6 +1623,7 @@ impl LockFile { sorted .packages .sort_by(|a, b| a.instance_id.cmp(&b.instance_id)); + sorted.sort_extensions(); serde_yaml::to_string(&sorted).unwrap_or_default() } @@ -1309,6 +1634,115 @@ impl LockFile { hex::encode(hasher.finalize()) } + fn sort_extensions(&mut self) { + self.extensions.sort_by(|left, right| { + (&left.kind, &left.name, &left.logical_owner).cmp(&( + &right.kind, + &right.name, + &right.logical_owner, + )) + }); + } + + /// Look up one locked extension identity. + pub fn extension( + &self, + kind: ExtensionKind, + name: &str, + logical_owner: &str, + ) -> Option<&LockedExtension> { + self.extensions.iter().find(|extension| { + extension.kind == kind + && extension.name == name + && extension.logical_owner == logical_owner + }) + } + + /// Compare an actual extension selection with the lock (RFC-EXT-0001 §6.4). + /// + /// The same complete identity and digests are idempotent; any missing, + /// extra or changed identity is an error. The caller turns this into + /// `ext.extension_lock_mismatch`; this layer returns a descriptive string. + pub fn validate_extension_selection(&self, actual: &[LockedExtension]) -> Result<(), String> { + let mut seen: BTreeSet<(ExtensionKind, String, String)> = BTreeSet::new(); + for extension in actual { + let key = ( + extension.kind, + extension.name.clone(), + extension.logical_owner.clone(), + ); + if !seen.insert(key.clone()) { + return Err(format!( + "extension {} `{}` for owner `{}` is selected more than once", + key.0, key.1, key.2 + )); + } + match self.extension(key.0, &key.1, &key.2) { + Some(locked) if locked == extension => {} + Some(locked) => { + return Err(format!( + "extension {} `{}` identity mismatch: lock has manifest {} artifact {} target {}, selection has manifest {} artifact {} target {}", + key.0, + key.1, + locked.manifest_digest, + locked.artifact_digest, + locked.selected_target, + extension.manifest_digest, + extension.artifact_digest, + extension.selected_target + )) + } + None => { + return Err(format!( + "extension {} `{}` for owner `{}` is not present in {LOCK_FILE}; run `marqdo lock` and pass the selected manifest explicitly", + key.0, key.1, key.2 + )) + } + } + } + for locked in &self.extensions { + if !seen.contains(&( + locked.kind, + locked.name.clone(), + locked.logical_owner.clone(), + )) { + return Err(format!( + "locked extension {} `{}` for owner `{}` is not selected in this run", + locked.kind, locked.name, locked.logical_owner + )); + } + } + Ok(()) + } + + /// Insert one exact extension identity; identical insertion is idempotent. + pub fn merge_extension(&mut self, extension: LockedExtension) -> Result<(), String> { + match self.extension( + extension.kind, + &extension.name, + &extension.logical_owner, + ) { + Some(existing) if existing == &extension => Ok(()), + Some(existing) => Err(format!( + "extension {} `{}` for owner `{}` already has manifest {} artifact {} target {}; refusing to replace it with manifest {} artifact {} target {}", + extension.kind, + extension.name, + extension.logical_owner, + existing.manifest_digest, + existing.artifact_digest, + existing.selected_target, + extension.manifest_digest, + extension.artifact_digest, + extension.selected_target + )), + None => { + self.extensions.push(extension); + self.sort_extensions(); + Ok(()) + } + } + } + /// Find a locked package by instance id. pub fn package(&self, instance_id: &str) -> Option<&LockedPackage> { self.packages @@ -1445,6 +1879,112 @@ impl LockFile { )); } } + + let mut extension_keys = BTreeSet::new(); + let mut extension_sources = BTreeSet::new(); + for extension in &self.extensions { + if !extension_keys.insert(( + extension.kind, + extension.name.clone(), + extension.logical_owner.clone(), + )) { + return Err(format!( + "duplicate extension {} `{}` for owner `{}` in {LOCK_FILE}", + extension.kind, extension.name, extension.logical_owner + )); + } + if extension.name.trim().is_empty() { + return Err("lock extension name must not be empty".to_string()); + } + if extension.version.trim().is_empty() { + return Err(format!( + "extension {} `{}` has an empty version", + extension.kind, extension.name + )); + } + if extension.logical_owner.trim().is_empty() { + return Err(format!( + "extension {} `{}` has an empty logical_owner", + extension.kind, extension.name + )); + } + if !is_hex_digest(&extension.manifest_digest) { + return Err(format!( + "extension {} `{}` manifest_digest is not a 64-character lowercase hex digest", + extension.kind, extension.name + )); + } + if !is_hex_digest(&extension.artifact_digest) { + return Err(format!( + "extension {} `{}` artifact_digest is not a 64-character lowercase hex digest", + extension.kind, extension.name + )); + } + if extension.selected_target.trim().is_empty() { + return Err(format!( + "extension {} `{}` has an empty selected_target", + extension.kind, extension.name + )); + } + if extension.source.trim().is_empty() { + return Err(format!( + "extension {} `{}` has an empty source", + extension.kind, extension.name + )); + } + if !extension_sources.insert((extension.kind, extension.source.clone())) { + return Err(format!( + "extension {} source `{}` is locked more than once", + extension.kind, extension.source + )); + } + } + Ok(()) + } + + /// Validate that locked extensions match the manifest declarations. + /// + /// Declaration is activation, not authorization; this only checks identity + /// bookkeeping. A manifest that declares an extension without a lock entry + /// (or vice versa) requires an explicit `marqdo lock`. + pub fn validate_extension_declarations(&self, manifest: &Manifest) -> Result<(), String> { + let declared: Vec<(ExtensionKind, String)> = manifest + .extensions + .as_ref() + .map(|extensions| { + extensions + .iter() + .map(|(kind, source)| (kind, source.from.trim().to_string())) + .collect() + }) + .unwrap_or_default(); + if declared.is_empty() && !self.extensions.is_empty() { + return Err(format!( + "{LOCK_FILE} records extension(s) but the manifest declares none; run `marqdo lock`" + )); + } + for (kind, source) in &declared { + let found = self + .extensions + .iter() + .any(|extension| &extension.kind == kind && &extension.source == source); + if !found { + return Err(format!( + "extension {kind} source `{source}` is declared but not locked; run `marqdo lock`" + )); + } + } + for extension in &self.extensions { + if !declared + .iter() + .any(|(kind, source)| kind == &extension.kind && source == &extension.source) + { + return Err(format!( + "locked extension {} source `{}` is not declared by the manifest; run `marqdo lock`", + extension.kind, extension.source + )); + } + } Ok(()) } @@ -1555,6 +2095,7 @@ capabilities: dependencies: BTreeMap::new(), }, ], + extensions: Vec::new(), root_dependencies: BTreeMap::from([ ( "a".to_string(), @@ -1576,13 +2117,107 @@ capabilities: } #[test] - fn v1_and_v2_locks_are_rejected_with_a_relock_hint() { + fn old_locks_are_rejected_with_an_explicit_migration() { for version in [1, 2] { let yaml = format!("lock_version: {version}\npackages: []\n"); let error = LockFile::from_yaml(&yaml).unwrap_err(); - assert!(error.contains("lock v3 is required"), "{error}"); + assert!(error.contains("lock v4 is required"), "{error}"); assert!(error.contains("marqdo lock"), "{error}"); } + // v3 is the immediately previous public format: the error must say the + // migration is explicit, not silently reinterpret the document. + let error = LockFile::from_yaml("lock_version: 3\npackages: []\n").unwrap_err(); + assert!( + error.contains("lock v3 is superseded by lock v4"), + "{error}" + ); + assert!(error.contains("run `marqdo lock`"), "{error}"); + } + + #[test] + fn extensions_manifest_sources_are_narrow_and_validated() { + let yaml = "project:\n name: p\n version: 0.1.0\ndependencies:\n q:\n path: ./q\nextensions:\n plugins:\n - from: ./adapter/plugin.yaml\n kernels: []\n providers:\n - from: \"@q/adapter/provider.json\"\n"; + let manifest = Manifest::from_yaml(yaml).unwrap(); + let extensions = manifest.extensions.as_ref().unwrap(); + assert_eq!(extensions.plugins.len(), 1); + assert_eq!(extensions.providers.len(), 1); + let resolved = extensions.plugins[0] + .resolve(&manifest.dependencies) + .unwrap(); + assert_eq!( + resolved, + ResolvedExtensionSource::ProjectRelative(PathBuf::from("adapter/plugin.yaml")) + ); + for bad in [ + "extensions:\n plugins:\n - from: /abs/plugin.yaml\n", + "extensions:\n plugins:\n - from: https://example/plugin.yaml\n", + "extensions:\n plugins:\n - from: ../escape/plugin.yaml\n", + "extensions:\n plugins:\n - from: \"@missing/plugin.yaml\"\n", + "extensions:\n plugins:\n - from: \"@q/../../escape\"\n", + "extensions:\n plugins:\n - from: \"\"\n", + ] { + let yaml = format!( + "project:\n name: p\n version: 0.1.0\ndependencies:\n q:\n path: ./q\n{bad}" + ); + assert!( + Manifest::from_yaml(&yaml).is_err(), + "extension source must be rejected: {bad}" + ); + } + } + + fn sample_extension(name: &str, digest: char) -> LockedExtension { + LockedExtension { + kind: ExtensionKind::Plugin, + logical_owner: "root:p@0.1.0".into(), + name: name.into(), + version: "0.1.0".into(), + manifest_digest: digest.to_string().repeat(64), + artifact_digest: "b".repeat(64), + selected_target: "x86_64-unknown-linux-gnu".into(), + source: format!("./adapter/{name}/plugin.yaml"), + } + } + + #[test] + fn lock_v4_extension_identity_is_deterministic_and_exact() { + let mut lock = LockFile::default(); + lock.root.manifest_digest = "a".repeat(64); + lock.merge_extension(sample_extension("demo.ext", 'a')) + .unwrap(); + lock.merge_extension(sample_extension("demo.ext", 'a')) + .unwrap(); + assert_eq!(lock.extensions.len(), 1); + assert!(lock + .merge_extension(sample_extension("demo.ext", 'c')) + .is_err()); + let second = sample_extension("other.ext", 'd'); + lock.merge_extension(second.clone()).unwrap(); + assert_eq!(lock.extensions[0].name, "demo.ext"); + let yaml = lock.to_yaml(); + let parsed = LockFile::from_yaml(&yaml).unwrap(); + assert_eq!(parsed, lock); + assert!(parsed + .validate_extension_selection(&[ + sample_extension("demo.ext", 'a'), + sample_extension("other.ext", 'd'), + ]) + .is_ok()); + assert!(parsed + .validate_extension_selection(&[sample_extension("demo.ext", 'a')]) + .is_err()); + assert!(parsed + .validate_extension_selection(&[ + sample_extension("demo.ext", 'a'), + sample_extension("other.ext", 'd'), + sample_extension("extra.ext", 'e'), + ]) + .is_err()); + let mut changed = sample_extension("demo.ext", 'a'); + changed.artifact_digest = "f".repeat(64); + assert!(parsed + .validate_extension_selection(&[changed, sample_extension("other.ext", 'd')]) + .is_err()); } #[test] diff --git a/docs/STATUS.md b/docs/STATUS.md index 83a8cdc..80ee12e 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -19,6 +19,34 @@ not be read as current status. Status is generated from executed commands and logs; a gate row without a command, log or source path is a defect, not a state. +## 0.0 RFC-EXT-0001 extensibility closure (current tree) + +Executed on Linux x86_64, Rust 1.100.0, branch `feat/extensible-ecosystem`: + +```text +cargo fmt --all -- --check PASS +cargo clippy --workspace --all-targets -- -D warnings PASS +cargo test --workspace --locked PASS + 67 test targets: 599 passed, 0 failed +cargo build --release --locked -p marqdo-cli PASS +``` + +Implemented and tested in this tree (PR-00…PR-07; no release claim): + +| Area | State | Tests | +|---|---|---| +| One `CallableRegistry` + one binding pass; runtime dispatch by `CallSiteKey`; no runtime name fallback; exact plugin reconciliation | **IMPLEMENTED / TESTED** | `callable_resolution` (14), `plugin_worker`, `analysis_trace`, `architecture_boundaries` | +| `provider.json` v2 through `--provider`/`--allow-provider`; verified before spawn; sibling helper only | **IMPLEMENTED / TESTED (linux/x86_64)** | `provider_cli` (12); PROV-01…10/13; macOS/Windows FAIL-CLOSED | +| `marqdo.yaml.extensions` + lock v4 identity + `inspect --extensions`; CLI/manifest merge and conflicts | **IMPLEMENTED / TESTED** | `extension_selection` (13), `package_env`, package crate tests | +| Capability handler registry; `fs.read` real handler; audit/unavailable | **IMPLEMENTED / TESTED** | `marqdo-host` `capability_handlers` (12) | +| Official stdlib public aliases; print_override/AST scan/runtime branch removed | **IMPLEMENTED / TESTED** | `stdlib_alias_routing` (5), print/format/Trace goldens | +| Independent external repo + frozen-CLI suite (pure pinned Git, plugin, kernel, provider, tool) | **PASS against a development binary; release-binary rerun + SHA-bound manifest pending** | `/root/work/marqdo-extension-conformance` `tests/run_all.py`: all 7 categories PASS | + +Evidence: [`evidence/2026-11-22-ext-implementation.md`](evidence/2026-11-22-ext-implementation.md) +and baseline [`evidence/ext-baseline-595279f.md`](evidence/ext-baseline-595279f.md). +The RFC-EXT-0001 release gate additionally requires a `--release` frozen-binary +rerun and `docs/evidence/extension//manifest.json` on the tagged commit. + ## 0. Language migration completed (RFC-LANG-0002) The repository now has **one** language contract, implemented end to end: @@ -215,7 +243,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL | 5 | Markdown/power/control-flow corpus | **Green on semantics v2** | 160 AST + 73 semantic + 49 compile-negative + 2 warning + 21 runtime-negative fixtures; `conformance_ast`/`semantics`/`negative` pass | | 6 | Payload/control source spans | **Green on semantics v2** | UTF-8 spans, control-expression offsets and heading boundaries covered by fixtures and CLI tests; old condition headers were removed | | 7 | Independent Tool consumers | **Done locally** | `sample-view`/`sample-knowledge`/`sample-proof` E2E test | -| 8 | lock v3 reproducibility | **Implemented; tamper test added** | `package_env` PASS | +| 8 | lock v4 reproducibility + extension identities | **Implemented; tamper/migration tests added** | `package_env`, `extension_selection` PASS | | 9 | Analysis/Trace/Tool/ABI/Foreign | **Implemented; supported-matrix bounded** | protocol tests, plugin/worker/foreign tests | | 10 | Linux strict L1 | **Local VM PASS at 5fcc111; remote G3 runner still missing** | 7/7 + positive CLI + strict JSON/inert-`@knowledge`; [`strict-gc-5fcc111`](evidence/2026-11-21-strict-gc-5fcc111.md) | | 11 | Linux strict L2 | **NOT IMPLEMENTED / UNSUPPORTED** | strict rejects Worker/Foreign/native ABI | @@ -233,10 +261,11 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL - Native `**` power is pinned end-to-end: right associativity, unary-minus precedence, parentheses, float negative exponents, branch/loop conditions, Strong+CodeSpan payloads, and inert prose/code fences. -- lock v3 full transitive package-instance graph, verified Git cache, strict - `sync --locked`, atomic `.marqdo/env` materialization; `run`/`check` never - fetch or rewrite the lock. Locked path content tamper is rejected without - rewriting the lock or environment. +- lock v4 full transitive package-instance graph plus extension identity + records, verified Git cache, strict `sync --locked`, atomic `.marqdo/env` + materialization; `run`/`check` never fetch or rewrite the lock. Locked path + content tamper is rejected without rewriting the lock or environment; lock v3 + requires an explicit `marqdo lock` migration. - Analysis v2 (logical identity, resources, byte digest) and Trace v2 (module ownership, bounds, truncation, stale detection) with independent Python consumers and JSON Schema validation. diff --git a/docs/adr/0056-extension-selection-locking.md b/docs/adr/0056-extension-selection-locking.md new file mode 100644 index 0000000..49c6e4e --- /dev/null +++ b/docs/adr/0056-extension-selection-locking.md @@ -0,0 +1,127 @@ +# ADR-0056: Explicit extension selection and lock v4 + +- Status: Accepted (2026-11-21) +- Implements: RFC-EXT-0001 §6 (PR-04) +- Related: ADR-0008/0016/0039, ADR-0054/0055/0057/0058 + +## Context + +`marqdo-package` already locks a reproducible package-instance graph (path and +pinned Git sources) and materializes it under `.marqdo/env`. Extensions had no +equivalent: plugin/provider/kernel paths were only accepted as repeatable CLI +flags (`--plugin`, `--kernel`, `--provider`). An independent repository could +therefore publish a `.mq.md` package and a callable/extension artifact, but a +consumer could not record *which exact manifest/artifact* the project selected, +and a project manifest could not declare activation sources at all. + +Copying CLI paths on every run makes deployments non-reproducible and invites a +second, hidden package manager. The existing lock is the right place, but its +v3 schema has no extension identity. + +## Decision + +### 1. `marqdo.yaml.extensions` declares activation sources, never grants + +```yaml +extensions: + plugins: + - from: ./adapter/plugin.yaml + kernels: [] + providers: + - from: "@quantum/adapter/provider.json" +``` + +Only two `from` forms exist: + +* `./relative/path` inside the project; +* `@alias/relative/path` inside a **declared** dependency in the same fixed + package graph. + +Absolute paths, URLs, globs, `..` traversal and implicit discovery are rejected +at manifest validation; the resolved file must be a regular file inside its +owning package. A declaration means "this project activates this extension" — +it never means permission: `--allow-plugin`/`--allow-provider`/`--allow-foreign` +and the capability broker remain separate. `pure`/`strict` refuse to activate +project-declared extensions. + +CLI flags remain supported as an explicit temporary/dev selection. Selection is +deduplicated by canonical path; the same manifest selected from the project +and the CLI is idempotent, while two different manifests for one package +identity are a deterministic `ext.duplicate_callable` error. + +### 2. Lock v4 records extension identity + +`marqdo.lock` becomes `lock_version: 4`. The package-instance graph is +unchanged; a new `extensions:` array records, per selected source: + +```yaml +- kind: plugin # plugin | kernel | provider + logical_owner: root:example@0.1.0 + name: demo.solver + version: 0.1.0 + manifest_digest: + artifact_digest: + selected_target: x86_64-unknown-linux-gnu + source: ./adapter/plugin.yaml +``` + +The array is sorted by `(kind, name, logical_owner)`. Duplicate identities or +duplicate sources are rejected. `manifest_digest`/`artifact_digest` are +**integrity** values, not publisher authentication; a release signature/trust +root is separate work. + +### 3. Explicit migration, no silent reinterpretation + +`LockFile::from_yaml` of a v3 document returns a deterministic error: lock v3 is +superseded by v4 (extension identities), run `marqdo lock` to migrate. v1/v2 +keep their relock error but now name v4. `run`/`check`/`sync` never rewrite the +lock and never fetch a floating revision. + +### 4. Who verifies what + +| Stage | Behaviour | +|---|---| +| `marqdo lock` | Resolves declared `from` sources statically, reads manifests and declared artifact digests, writes v4. No extension binary is loaded or executed. | +| `check` | Loads project + static descriptors and compares declared identities with the lock (`ext.extension_lock_mismatch` on missing/extra/changed). Never starts a worker; a missing artifact is not fatal for a pure static check. | +| `sync` | Re-validates declarations against the lock before materializing package content. | +| `run`/`call` | Re-verifies the actual artifact bytes/hashes for selected plugins/providers (and the kernel executable) before spawning/loading anything; exact grants are still required. | +| `inspect --extensions` | Projects declared sources, lock identity, static plugin descriptors and official stdlib aliases; authorization is always `not_granted` and inspection never executes. | + +### 5. Selection priority and conflicts + +1. Project declaration (locked) and explicit CLI selection are merged by + canonical path; identical identity is idempotent. +2. Same package/kernel/provider logical identity with different manifest digest + or artifact digest is an error; no `last wins`, no auto-suffix. +3. Extension logical owners are recorded as package instance ids (root or + dependency), so moving the cache/checkout does not change identity. +4. A dependency extension cannot escape its package root (`@alias/../` is + rejected; canonical path must stay under the package root). +5. Multi-version dependencies remain distinguishable by package instance + + alias; the lock records the exact owner. + +## Consequences + +* Third-party `.mq.md`/plugin/kernel/provider repositories can publish into a + project with a reproducible, auditable selection; +* `check` stays side-effect free while still validating extension identity; +* pre-existing projects migrate with one `marqdo lock`; their v3 lock is not + silently accepted; +* the existing package graph, Git pinning, strict `sync` and env materialization + are unchanged apart from the version bump and new section; +* no package registry, network discovery or automatic extension installation is + introduced. + +## Rejected alternatives + +* **Keep extensions CLI-only.** Rejected: projects cannot reproduce/audit the + selection, and `check` cannot validate it. +* **Put extension paths in the existing `plugins:` requirement map.** Rejected: + it is a version *requirement*, not a runnable artifact identity, and mixing + them would make manifests ambiguous. +* **Add fields to lock v3 without bumping the version.** Rejected by + RFC-EXT-0001 §6.3 and the repository's public-lock rule. +* **Let extensions be discovered from directories, PATH or an HTTP index.** + Rejected: implicit discovery is non-deterministic and un-auditable. +* **Auto-grant from a manifest declaration.** Rejected: selection is not + authorization (ADR-0054/0055). \ No newline at end of file diff --git a/docs/adr/README.md b/docs/adr/README.md index b3da25f..f59529f 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -29,7 +29,18 @@ architecture; read them as decision history. [`docs/architecture/CURRENT.md`](../architecture/CURRENT.md); for executed status see [`docs/STATUS.md`](../STATUS.md). -Files: `0001`–`0050`, plus `0051-byte-exact-http-framing.md` (HTTP framing), -`0052-unified-markdown-semantics.md` (current language mapping) and -`0053-strict-child-inspection-and-json-failure-contract.md` (strict CI/JSON -failure contract). +Files: `0001`–`0050`, plus: + +* `0051-byte-exact-http-framing.md` — HTTP framing; +* `0052-unified-markdown-semantics.md` — current language mapping; +* `0053-strict-child-inspection-and-json-failure-contract.md` — strict CI/JSON + failure contract; +* `0054-callable-binding-single-source.md` — one registry/binding pass and + runtime dispatch by identity; +* `0055-resource-provider-cli-v2.md` — `provider.json` v2 through the normal + CLI, worker trust boundary; +* `0056-extension-selection-locking.md` — `marqdo.yaml.extensions` and lock v4; +* `0057-capability-handler-registration.md` — declared capability vs installed + handler vs audit; +* `0058-stdlib-public-alias.md` — verified stdlib `public_aliases` replace + print magic. diff --git a/examples/multi-module/marqdo.lock b/examples/multi-module/marqdo.lock index 0aaef4b..891c68c 100644 --- a/examples/multi-module/marqdo.lock +++ b/examples/multi-module/marqdo.lock @@ -1,4 +1,4 @@ -lock_version: 3 +lock_version: 4 content_profile_version: 1 root: package: root:multi-module@0.1.0 diff --git a/examples/packages-multi-version/marqdo.lock b/examples/packages-multi-version/marqdo.lock index c981a6a..8a50a2a 100644 --- a/examples/packages-multi-version/marqdo.lock +++ b/examples/packages-multi-version/marqdo.lock @@ -1,4 +1,4 @@ -lock_version: 3 +lock_version: 4 content_profile_version: 1 root: package: root:multi-version@0.1.0 diff --git a/spec/extension-selection.md b/spec/extension-selection.md new file mode 100644 index 0000000..f41671a --- /dev/null +++ b/spec/extension-selection.md @@ -0,0 +1,191 @@ +# Marqdo extension selection and lock v4 + +Status: **implemented** (RFC-EXT-0001 §6, ADR-0056). This is the normative +contract for `marqdo.yaml.extensions`, `marqdo.lock` extension identities, and +how `lock`/`sync`/`check`/`run`/`inspect` treat them. + +## 1. Scope + +An **extension** is one of: + +| Kind | Manifest/descriptor | Execution form | +|---|---|---| +| `plugin` | `plugin.yaml` (Plugin ABI v1) | native/worker callable plugin | +| `kernel` | `kernel.json` (Foreign wire v1) | one-shot child process | +| `provider` | `provider.json` (Resource Provider v2) | worker helper + C ABI library | + +Extension *selection* is distinct from extension *authorization*: selecting a +manifest makes its names visible and allows `run` to consider it. It never +creates a grant, loads a library, starts a worker or executes a script. Actual +permissions are the CLI grants plus the capability broker at the effect site. + +## 2. Project manifest + +```yaml +extensions: + plugins: + - from: ./adapter/plugin.yaml + kernels: + - from: "@quantum/adapter/kernel.json" + providers: + - from: "@quantum/adapter/provider.json" +``` + +Rules: + +1. `from` is exactly one of: + * `./relative/path` — below the project root; + * `@alias/relative/path` — below a dependency declared in + `dependencies` in the same project. +2. Absolute/rooted paths, `http(s)://` URLs, globs, empty strings, `.`-only + paths and any `ParentDir` (`..`) component are rejected. The alias must + exist in `dependencies`. +3. The resolved path must be a regular file whose canonical form stays inside + its owning package root (project root or dependency package root). Symlink + escapes are rejected. +4. Duplicate `(kind, from)` entries are rejected. +5. A dependency's extension file is identified by the dependency package + instance id (`dep:name@version#content-digest`), never by a physical cache + path. +6. Unknown YAML fields are ignored by the parser, but they never imply + execution or authorization. + +The existing `plugins:` block is a **version requirement** and is not a +runnable artifact; it remains declarative only. + +## 3. Lock v4 + +```yaml +lock_version: 4 +content_profile_version: 1 +root: + package: root:example@0.1.0 + manifest_digest: <64 hex> +packages: [] +root_dependencies: {} +extensions: + - kind: plugin + logical_owner: root:example@0.1.0 + name: demo.solver + version: 0.1.0 + manifest_digest: <64 hex> + artifact_digest: <64 hex> + selected_target: x86_64-unknown-linux-gnu + source: ./adapter/plugin.yaml +``` + +| Field | Rule | +|---|---| +| `kind` | `plugin` \| `kernel` \| `provider`; unknown values are rejected | +| `logical_owner` | non-empty package instance id, root or dependency | +| `name` | stable declared name; for a kernel, `provider_id` | +| `version` | declared SemVer/string; a kernel has no version field, so `wire:` | +| `manifest_digest` | SHA-256 of the exact manifest/descriptor bytes (lowercase hex) | +| `artifact_digest` | declared selected-target artifact SHA-256; for a kernel the pinned provider executable digest | +| `selected_target` | host target triple selected at lock time | +| `source` | `from` string exactly as declared | + +* Extensions are sorted by `(kind, name, logical_owner)`; the whole array is + omitted when empty. +* Duplicate `(kind, name, logical_owner)` or duplicate `(kind, source)` is an + error. +* `manifest_digest`/`artifact_digest` are integrity values. They are not + signatures and do not establish publisher identity. +* Lock v3 is rejected with an explicit migration message; run `marqdo lock`. + No v3 field is reinterpreted as v4 and the lock is not rewritten implicitly. + +## 4. Stage behaviour + +### 4.1 `marqdo lock` + +1. Resolves the package graph as before. +2. Resolves every `extensions` source to a canonical regular file. +3. Reads each manifest/descriptor statically and computes the identity above. +4. Rejects missing files, malformed manifests, unsupported provider/kernel + schema versions, missing current-target artifacts and duplicate identities. +5. Writes lock v4 atomically. It never loads a shared library, starts a helper + or executes a kernel. + +### 4.2 `marqdo sync [--locked]` + +1. Loads and validates the lock (v3 → explicit migration error). +2. Resolves the current declared sources and computes their identities. +3. Compares the declared selection with the lock (same rules as §4.3). +4. `--locked` forbids fetching; any lock/graph mismatch is fail-closed. + +`sync` materializes package content but does not authorize or execute an +extension binary. + +### 4.3 `marqdo check` + +`check`: + +* loads sources, static manifests and the lock; +* compares declared extension identity with the lock and fails with + `ext.extension_lock_mismatch` on a missing/extra/changed identity; +* may read manifests and declared digests, but **never** loads a plugin + library, starts a provider worker, spawns a foreign kernel or runs a + side-effect script; +* does not require a plugin/provider/kernel artifact file to exist for the + static check path (execution-artifact verification belongs to `run`). + +### 4.4 `marqdo run` / `marqdo call` + +Before the entry function runs: + +1. `prepare` performs the same static binding and lock comparison as `check`. +2. The merged selection (project declaration + explicit CLI flags) determines + which plugin/provider/kernel descriptors are considered. +3. Real artifact bytes/hashes, bundle containment, target and exact grants are + verified before any worker/library/kernel is started. +4. A failed verification produces a structured `ext.*` error and the main + function does not run. There is no fallback to "latest", PATH discovery or + a trusted in-process native provider. + +`--mode pure` and `--mode strict` reject project-declared extension activation +(`cli.invalid_grants` / `strict.unsupported_grant`) before any extension is +used. + +## 5. Selection priority and conflicts + +1. Project declaration and CLI selection are merged and deduplicated by + canonical path. +2. Identical logical identity + digests is idempotent. +3. Same package/kernel/provider logical identity with different manifest or + artifact digest is `ext.duplicate_callable` (prepare) — never last-wins and + never auto-suffixed. +4. Two providers declaring the same resource kind is + `ext.duplicate_provider_kind` (before spawn). +5. Ordering of CLI flags and YAML entries never decides which implementation is + used. + +## 6. `inspect --extensions` + +`inspect FILE --extensions [--json] [--plugin PATH]` prints: + +* declared sources with `kind`, `source`, package-relative `path`, logical + owner and resolution status (`locked`/`unlocked`); +* the locked identity fields when available; +* static plugin descriptors (package/version/kind/target/artifact digest/ + manifest digest/exports), validation `static_manifest_ok`; +* official stdlib public aliases (`stdlib_aliases`) with the verified target. + +Authorization is reported as `not_granted`; inspection never grants, loads or +executes anything. Environment variables, secrets and binary contents are never +printed. + +## 7. Diagnostics + +| Code | Stage | Trigger | +|---|---|---| +| `ext.extension_lock_mismatch` | prepare/sync | declared selection missing from lock, extra locked extension, changed manifest/artifact identity, or extensions declared without a lock | +| `ext.duplicate_callable` | prepare | two different manifests for one package identity, or duplicate callable names | +| `ext.duplicate_provider_kind` | prepare/run init | two providers declare the same resource kind | +| `ext.no_matching_artifact` | identity/manifest read | no artifact for the current target | +| `ext.artifact_integrity` | run init | artifact bytes/hash/containment mismatch | +| `project.lock` | load | lock v3/older version, malformed lock | +| `project.manifest` | load | invalid `extensions` source form, missing file, alias not declared | + +Exit codes follow the CLI convention: manifest/option errors are `2`, load/ +verify/runtime errors are `1`; `check` exit code is `1` on any compile/load +error, `0` on success. diff --git a/spec/package-format.md b/spec/package-format.md index 6e6464f..23d1773 100644 --- a/spec/package-format.md +++ b/spec/package-format.md @@ -1,10 +1,12 @@ -# Marqdo Project Package Format v3 +# Marqdo Project Package Format v4 -Status: **lock_version 3 package-instance graph as implemented by S1 -(ADR-0016).** Path and pinned Git sources, transitive closure locking, -fail-closed locked resolution, strict `sync`, and content-profile-1 file-set -hashing/materialization are implemented. Registries and SemVer solving remain -out of scope; see §8. +Status: **lock_version 4 package-instance graph plus extension identities +(RFC-EXT-0001 §6, ADR-0056).** Path and pinned Git sources, transitive closure +locking, fail-closed locked resolution, strict `sync`, content-profile-1 +file-set hashing/materialization, and explicit `extensions:` activation +sources/identities are implemented. Registries and SemVer solving remain out of +scope; see §8. Lock v3 is rejected with an explicit `marqdo lock` migration +error; it is never reinterpreted silently. Implementation: `crates/marqdo-package/src/lib.rs` (manifest/lock types and validation), `crates/cli/src/commands.rs` (`init`, `add`, `remove`, `lock`, @@ -46,6 +48,12 @@ capabilities: # requests only; not grants net: hosts: [] process: false +extensions: # explicit activation sources; never grants + plugins: + - from: ./adapter/plugin.yaml + kernels: [] + providers: + - from: "@quantum/adapter/provider.json" ``` ### 2.1 Fields @@ -62,6 +70,7 @@ capabilities: # requests only; not grants | `capabilities.fs.read` / `.write` | list of strings | no | requested project-relative globs | | `capabilities.net.hosts` | list of strings | no | requested hosts | | `capabilities.process` | bool | no | requested process execution | +| `extensions.plugins[]` / `.kernels[]` / `.providers[]` | `{from}` | no | explicit extension activation sources; `from` is `./relative/path` in the project or `@alias/relative/path` in a declared dependency. The alias must be in `dependencies`. Absolute paths, URLs, globs and `..` traversal are rejected. A declaration is not a grant. | Unknown fields are ignored (the types do not `deny_unknown_fields`). Duplicate YAML keys follow `serde_yaml` behavior (last one wins); there is no duplicate-key @@ -78,7 +87,12 @@ validates: * neither → error "must declare a path or a pinned git source"; * `git` without `rev` → error requiring an exact revision; * an absolute `path` → error "path must be project-relative"; -* for every plugin requirement, `version` is non-empty. +* for every plugin requirement, `version` is non-empty; +* every `extensions.*[].from` is one of the two allowed forms and resolves to a + regular file inside its owning package; duplicates within a kind are + rejected; `@alias` must name a declared dependency. Resolution happens when + the project is loaded (`PackageResolver::resolve_extension_sources`), not in + plain `Manifest::validate`. Note: a *git* dependency passes manifest validation when pinned, but the dependency is not resolvable or lockable in this build (§5). @@ -92,11 +106,13 @@ grants (`spec/security.md` §2). `net` has no implementing host op, and ## 3. `marqdo.lock` -`lock_version: 3` records the complete package-instance closure reachable from -the root manifest. Packages are sorted by `instance_id`: +`lock_version: 4` records the complete package-instance closure reachable from +the root manifest plus the exact identity of every declared extension. +Packages are sorted by `instance_id`; extensions by `(kind, name, +logical_owner)`: ```yaml -lock_version: 3 +lock_version: 4 content_profile_version: 1 root: package: root:quantum@0.1.0 @@ -124,8 +140,42 @@ packages: dependencies: {} root_dependencies: math: dep:math@1.0.0#9e2... +extensions: +- kind: plugin + logical_owner: root:quantum@0.1.0 + name: demo.solver + version: 0.1.0 + manifest_digest: 4f5c... # sha256 of the exact plugin.yaml bytes + artifact_digest: 9e2... # selected target artifact sha256 + selected_target: x86_64-unknown-linux-gnu + source: ./adapter/plugin.yaml ``` +An extension lock record binds: + +* `kind` — `plugin` | `kernel` | `provider`; +* `logical_owner` — root instance or dependency package instance id; +* `name` / `version` — stable declared identity (kernel descriptors have no + version, so the lock records `wire:`); +* `manifest_digest` — SHA-256 of the exact manifest/descriptor bytes; +* `artifact_digest` — declared selected-target artifact SHA-256 (for a kernel, + the pinned provider executable digest); +* `selected_target` — the host triple selected at lock time; +* `source` — the `from` string exactly as declared. + +`marqdo lock` records these identities statically (no extension is executed). +`check`/`sync` compare the current declaration to the lock and fail with +`ext.extension_lock_mismatch` on a missing, extra or changed identity; `run` +additionally verifies the real artifact bytes before starting any worker. +`extensions` is omitted when no extension is declared. + +### 3.1 Lock v3 migration + +`from_yaml` of a v3 document returns a deterministic error: lock v3 is +superseded by v4 (extension identities), and the user must run `marqdo lock` +to migrate explicitly. No v3 field is silently reinterpreted. Lock v1/v2 keep +their existing relock error with the v4 requirement. + | Field | Semantics | |---|---| | `lock_version` | writers emit `3`; readers accept only `3` and tell v1/v2 users to run `marqdo lock` |