feat(ext): single callable binding, provider v2 CLI, capability handlers, stdlib aliases
Implements RFC-EXT-0001 PR-00..PR-07 first pass: - CallSiteKey/CallBindingTable + one frontend binding pass; runtime dispatch by identity - prepare composition root shared by check/run/inspect/Analysis; exact plugin reconciliation - provider.json v2 --provider/--allow-provider worker CLI with fail-closed verification - capability handler registry with real fs.read and audit - verified stdlib public aliases replace print_override/AST scan - docs: specs, ADR-0054/0055/0057/0058, README, CI/release gates, evidence PR-04 extension selection/lock v4 and final frozen-binary evidence remain.
This commit is contained in:
1 parent
9aa536c2c9
commit
1f6fd644a0
54 files changed
+8710
-828
No files matched your search
@@ -95,6 +95,13 @@ a separate statement. `imports`, `references` and `calls` remain static facts; a
|
||||
Markdown reference is not a call. `effects` remains `["unknown"]` (a
|
||||
conservative summary, not a purity proof).
|
||||
|
||||
`calls[]` is projected from the single prepared call binding
|
||||
(`spec/callable-resolution.md` §6): `state` remains
|
||||
`function`/`builtin`/`plugin`/`unresolved`, and official stdlib aliases such as
|
||||
`打印` are reported as `function` targets because that is the implementation
|
||||
that runs. The private `CallBindingTable` is not serialized; Analysis v2's
|
||||
schema and version are unchanged.
|
||||
|
||||
An `assign` node keeps the v1 string `target` for a local rebinding. Addressable
|
||||
memory adds one additive target form (ADR-0022); consumers that only understand
|
||||
local assignments can ignore an object target:
|
||||
|
||||
@@ -0,0 +1,281 @@
|
||||
# Marqdo callable resolution and binding
|
||||
|
||||
Status: **normative for the current implementation** (ADR-0054/ADR-0058,
|
||||
RFC-EXT-0001 §4). This document describes the single static resolution used by
|
||||
`check`, `run`, `call`, `inspect` and Analysis. There is no second priority
|
||||
order and no runtime name guessing.
|
||||
|
||||
## 1. Identity
|
||||
|
||||
A call is an expression (`Expr::Call`). Each statically visible call site has a
|
||||
deterministic key:
|
||||
|
||||
```text
|
||||
CallSiteKey = (module id, function id, span.start, span.end)
|
||||
```
|
||||
|
||||
The key is independent of the checkout path, package cache location,
|
||||
registration order and map iteration. For a signature-default expression the
|
||||
owning function is the function that declares the default, because the runtime
|
||||
evaluates defaults in that function's empty frame.
|
||||
|
||||
A binding is one of:
|
||||
|
||||
| Bound target | Meaning |
|
||||
|---|---|
|
||||
| `Builtin { canonical_name }` | fixed language builtin, addressed by its canonical name (never by a user alias) |
|
||||
| `Marqdo { function }` | exact `FunctionId` of a Markdown-defined function, including an official bundled-stdlib public alias |
|
||||
| `AbiPlugin { package, export, digest, param_names }` | exact plugin export; `digest` is the selected, verified artifact SHA-256 and `param_names` is the manifest order used to normalise positional arguments |
|
||||
| `DynamicValue` | the callee is a language value (for example a `FunctionRef`); resolved at run time from the value, never by name |
|
||||
|
||||
`CallBindingTable` is private IR. It is not serialized into Analysis or Trace.
|
||||
The public `calls[].state/target` projection is derived from it (see §6).
|
||||
|
||||
## 2. Resolution matrix
|
||||
|
||||
### 2.1 Unqualified `f(...)`
|
||||
|
||||
In this exact order; the first applicable step decides:
|
||||
|
||||
1. **Local variable.** If `f` is a parameter or has been assigned anywhere in
|
||||
the function body, the site binds `DynamicValue`. At run time the value must
|
||||
be callable (`FunctionRef` in this release); a `Text`/`Map`/etc. value is
|
||||
`call.not_callable`. This is a language variable rule, not an escape hatch to
|
||||
a plugin.
|
||||
2. **Official alias or reserved builtin.** The registry's builtin table contains
|
||||
fixed builtins and, when the verified bundle is present and the program uses
|
||||
them, official stdlib aliases. An official alias wins over the placeholder
|
||||
builtin of the same name. Third parties cannot insert names here.
|
||||
3. **Current namespace function.** If the enclosing level-1 namespace `N`
|
||||
declares `N.f`, that function is chosen.
|
||||
4. **Canonical function name.** A function whose canonical name is exactly `f`.
|
||||
5. **Unique local name.** A function whose `local_name` is `f` in exactly one
|
||||
namespace. If two namespaces define the same local name and none of steps
|
||||
1–4 applies, the call is an error (`symbol.unknown`, message reports the
|
||||
ambiguity); it is never "first in document order".
|
||||
6. Otherwise `symbol.unknown`.
|
||||
|
||||
### 2.2 Qualified `alias.f(...)`
|
||||
|
||||
1. **Local variable named `alias`.** If `alias` is a local value, the site binds
|
||||
`DynamicValue`; `alias.f` is ordinary language member access. This cannot
|
||||
become a plugin call.
|
||||
2. **Explicit import.** If the module declares `import: { alias: <target> }`,
|
||||
the call binds only a function of that imported module (canonical name first,
|
||||
then unique local name). A missing function is `symbol.unknown`; it never
|
||||
falls back to a builtin or plugin with the same dotted spelling.
|
||||
3. **Builtin dotted namespace.** Otherwise, an exact builtin
|
||||
`alias.f` (for example `math.eval`) binds `Builtin`.
|
||||
4. **Exact plugin export.** Otherwise, if a selected static plugin declares the
|
||||
complete name `alias.f`, the site binds that export. The manifest signature
|
||||
is validated here; no library is loaded.
|
||||
5. Otherwise `symbol.unknown`.
|
||||
|
||||
`alias` in step 1/2 must be exactly an explicit import or a local value; there
|
||||
is no implicit namespace search across packages.
|
||||
|
||||
### 2.3 Other callees
|
||||
|
||||
`some_value.f(...)`, `f_list[i](...)`, `(expr)(...)` and any other non-Ident,
|
||||
non-member-of-Ident callee bind `DynamicValue`. Only language callables can be
|
||||
invoked at run time.
|
||||
|
||||
### 2.4 Import namespace conflicts
|
||||
|
||||
Before any call is bound, each module's import aliases are checked against:
|
||||
|
||||
* builtin dotted namespaces (`alias.` must not prefix any builtin name), and
|
||||
* visible plugin dotted namespaces (`alias.` must not prefix any selected
|
||||
plugin export).
|
||||
|
||||
A collision is `ext.import_namespace_conflict` at the front-matter import span,
|
||||
with a rename hint. Example: importing `./math.mq.md` as `math` conflicts with
|
||||
the builtin namespace `math.`; rename the alias to `mymath`. This is a
|
||||
deliberate 0.x breaking change with no compatibility switch (ADR-0054).
|
||||
|
||||
### 2.5 Duplicate callables
|
||||
|
||||
The static table rejects:
|
||||
|
||||
* the same complete plugin export declared by two selected packages
|
||||
(`ext.duplicate_callable` at prepare);
|
||||
* a plugin export that equals a builtin name or an official alias
|
||||
(`ext.duplicate_callable`);
|
||||
* two different manifests for the same plugin package identity in one selection
|
||||
(`ext.duplicate_callable`); the exact same package identity is idempotent;
|
||||
* a malformed/ambiguous static table entry (`ext.duplicate_callable`).
|
||||
|
||||
Order of `--plugin` flags never decides which declaration wins.
|
||||
|
||||
## 3. Argument checking
|
||||
|
||||
### 3.1 Markdown functions and builtins
|
||||
|
||||
`CallableSignature::validate_args` enforces:
|
||||
|
||||
* named arguments must exist exactly once; unknown names are `call.arity`;
|
||||
* positional arguments require an explicit language signature and map to the
|
||||
declared parameter order; overflow is `call.arity`;
|
||||
* required parameters without defaults must be supplied; missing is
|
||||
`call.arity`;
|
||||
* inferred signatures accept named arguments only (`call.arity` otherwise).
|
||||
|
||||
Messages are deterministic and identical between `check` and `run`.
|
||||
|
||||
### 3.2 ABI plugin exports
|
||||
|
||||
* The static manifest signature is authoritative. Unknown, duplicate, missing
|
||||
or excess arguments are rejected at prepare/bind time (`call.arity`); the
|
||||
plugin is never spawned for a statically invalid call.
|
||||
* ABI v1 transports arguments as a named map. Positional arguments are
|
||||
statically validated and deterministically normalised to the manifest's
|
||||
declared parameter order at run time. Example: `solver.add(1, b=2)` becomes
|
||||
`{a:1,b:2}`.
|
||||
* Manifest signature default values are rejected in ABI v1 until the wire
|
||||
semantics are versioned; no default is injected at run time.
|
||||
* A plugin that is declared but not selected (`--plugin` missing) is
|
||||
`symbol.unknown`; a selected plugin without an exact export descriptor is
|
||||
`symbol.unknown`. There is no runtime fallback.
|
||||
|
||||
## 4. Run-time reconciliation
|
||||
|
||||
`run`/`call` performs, in order:
|
||||
|
||||
1. static prepare (sources, manifests, signatures, binding) — no binary loaded;
|
||||
2. verify each selected artifact path, target and SHA-256;
|
||||
3. verify the `--allow-plugin` grant;
|
||||
4. load exactly the selected artifacts;
|
||||
5. compare the Host's actual exports with the static descriptors one-to-one
|
||||
(`name`, raw signature, artifact digest, owner package).
|
||||
|
||||
A mismatch in step 5 is `ext.plugin_binding_mismatch`, reported before the
|
||||
entry function runs. `run` must never synthesise a missing descriptor or an
|
||||
empty signature.
|
||||
|
||||
## 5. `check` versus `run`
|
||||
|
||||
`check` and `run` consume the same `PreparedProgram` binding table. Therefore:
|
||||
|
||||
* the same call site has the same `BoundTarget` in both;
|
||||
* a statically invalid call fails `check` and `run` with the same diagnostic
|
||||
code and message; `run` refuses before any Host side effect;
|
||||
* `check --plugin` reads manifests and signatures only: it never `dlopen`s a
|
||||
library, spawns a worker or executes a side-effect script; a missing plugin
|
||||
binary is not an error for `check`;
|
||||
* `--emit` Analysis is derived from the same binding (no second resolver).
|
||||
|
||||
## 6. Analysis projection
|
||||
|
||||
`calls[]` keeps the v2 fields `{function, span, callee, state, target}`.
|
||||
`state` is one of `function`, `builtin`, `plugin`, `unresolved`; it is now
|
||||
derived from the binding table:
|
||||
|
||||
| Bound target | `state` | `target` |
|
||||
|---|---|---|
|
||||
| `Builtin` | `builtin` | canonical builtin name |
|
||||
| `Marqdo` | `function` | exact `FunctionId` string |
|
||||
| `AbiPlugin` | `plugin` | complete export name |
|
||||
| `DynamicValue` / unbound | `unresolved` | `null` |
|
||||
|
||||
Official stdlib aliases (for example `打印`) therefore project as `function`
|
||||
targets, because that is the implementation that runs. This is a projection of
|
||||
the existing fields, not a new public schema. The private `CallBindingTable` is
|
||||
never exposed; a future explicit `resolved_call_target` field would require an
|
||||
Analysis version bump.
|
||||
|
||||
## 7. Diagnostics
|
||||
|
||||
| Code | Stage | Trigger |
|
||||
|---|---|---|
|
||||
| `symbol.unknown` | prepare/bind | no target for the call; ambiguity; unknown imported function; undeclared plugin export |
|
||||
| `call.arity` | prepare/bind | bad argument name/number/duplication, invalid positional use |
|
||||
| `ext.import_namespace_conflict` | prepare | import alias collides with a builtin/plugin dotted namespace |
|
||||
| `ext.duplicate_callable` | prepare | duplicate complete export, builtin shadowing, conflicting package manifests |
|
||||
| `ext.plugin_binding_mismatch` | run init | loaded exports/signatures/owner/digest differ from static descriptors |
|
||||
| `ext.no_matching_artifact` | manifest read | plugin/provider has no artifact for the current target |
|
||||
| `ext.artifact_integrity` | manifest read / run init | sha256 mismatch, path traversal, non-regular file, symlink escape |
|
||||
| `call.not_callable` | run | dynamic value is not callable |
|
||||
| `internal.error` | run | a call site has no binding (a prepared program must always be complete) |
|
||||
|
||||
Existing codes are reused rather than duplicated; `ext.capability_handler_unavailable`
|
||||
is internal-only — the public wire code remains `capability.unavailable`.
|
||||
|
||||
## 8. Examples
|
||||
|
||||
### 8.1 Imported function, no conflict
|
||||
|
||||
```markdown
|
||||
---
|
||||
import:
|
||||
mymath: "./math.mq.md"
|
||||
---
|
||||
|
||||
## @exec main
|
||||
|
||||
`return mymath.eval(a=1)`
|
||||
```
|
||||
|
||||
Binds `Marqdo { function: <package instance>::math.mq.md::eval }` in both
|
||||
`check` and `run`.
|
||||
|
||||
### 8.2 Builtin without an import
|
||||
|
||||
```markdown
|
||||
## @exec main
|
||||
|
||||
`return math.eval(ref("#公式"), {"r": 2}, target="A")`
|
||||
```
|
||||
|
||||
Binds `Builtin { canonical_name: "math.eval" }`.
|
||||
|
||||
### 8.3 Import namespace conflict
|
||||
|
||||
```markdown
|
||||
---
|
||||
import:
|
||||
math: "./math.mq.md"
|
||||
---
|
||||
|
||||
## @exec main
|
||||
|
||||
`return math.eval(a=1)`
|
||||
```
|
||||
|
||||
`check` fails with `ext.import_namespace_conflict`; `run` refuses before
|
||||
executing anything. Rename the alias.
|
||||
|
||||
### 8.4 Plugin export
|
||||
|
||||
```markdown
|
||||
## @exec main
|
||||
|
||||
`return solver.add(1, b=2)`
|
||||
```
|
||||
|
||||
* without `--plugin solver/plugin.yaml` or with a manifest that does not
|
||||
declare `solver.add`: `symbol.unknown` (no Host fallback);
|
||||
* with the manifest: binds `AbiPlugin { package: "demo.solver", export:
|
||||
"solver.add", digest: "<sha256>", param_names: ["a","b"] }`;
|
||||
* with a changed artifact byte: `ext.artifact_integrity` before the entry
|
||||
function;
|
||||
* with a missing `b`: `call.arity` at prepare.
|
||||
|
||||
### 8.5 Dynamic `FunctionRef`
|
||||
|
||||
```markdown
|
||||
## @exec 平方
|
||||
|
||||
`params(n: Int)`
|
||||
|
||||
`return n * n`
|
||||
|
||||
## @exec main
|
||||
|
||||
`f = ref("#平方")`
|
||||
|
||||
`return f(n=4)`
|
||||
```
|
||||
|
||||
The call site binds `DynamicValue`; the runtime calls the `FunctionRef` value.
|
||||
A `Text` value in the same position fails `call.not_callable` and never reaches
|
||||
the Host.
|
||||
@@ -459,3 +459,33 @@ when no C compiler is available.
|
||||
| Positional plugin arguments | Not supported; named only. |
|
||||
| Thread-safe / concurrent invocation | Not supported (single-threaded v1 contract). |
|
||||
| Old `marqdo_abi.h` v2 compatibility | Not implemented and not claimed; a separate adapter would be required. |
|
||||
|
||||
## 12. Static descriptor / run-time reconciliation (RFC-EXT-0001 §4.5)
|
||||
|
||||
The CLI never derives a callable signature from a loaded library. The
|
||||
**manifest is the static contract** and the runtime is checked against it:
|
||||
|
||||
1. `check --plugin <manifest>` reads and validates the manifest, including every
|
||||
export name and signature; it does not load the artifact and does not need
|
||||
the artifact file to exist.
|
||||
2. `run`/`call` first verifies the selected artifact path, host target and
|
||||
SHA-256, then checks the plugin grant, then loads exactly the selected
|
||||
artifacts.
|
||||
3. After loading, the Host's actual exports are compared one-to-one with the
|
||||
static descriptors: complete name, raw signature string, owner package and
|
||||
verified artifact digest. Extra, missing, renamed or differently-signed
|
||||
exports fail with `ext.plugin_binding_mismatch` before the entry function
|
||||
runs.
|
||||
4. The binding pass validates call arguments against the manifest signature.
|
||||
ABI v1 transports a named map, so positional arguments are statically
|
||||
validated and deterministically normalised to manifest order
|
||||
(`solver.add(1, b=2)` → `{a:1,b:2}`). Unknown, duplicate, missing or excess
|
||||
arguments are `call.arity` before any process is started.
|
||||
5. Manifest signatures with default values are rejected in ABI v1 until the
|
||||
default-value wire semantics are versioned. The host never injects a default.
|
||||
6. Duplicate complete export names across selected packages, or exports that
|
||||
shadow a builtin/official name, are `ext.duplicate_callable` at prepare.
|
||||
Selection order never decides which declaration wins.
|
||||
|
||||
None of this changes ABI v1's C layout or wire format; it closes the static
|
||||
side of the contract described by `spec/callable-resolution.md`.
|
||||
@@ -0,0 +1,287 @@
|
||||
# Marqdo Resource Provider v2 (`provider.json`)
|
||||
|
||||
Status: **frozen provider manifest schema version 2** for the `marqdo` CLI
|
||||
composition root. Implementation: `crates/cli/src/provider_manifest.rs`;
|
||||
prerequisite host transports: ADR-0033/0035/0036/0037/0038; CLI decision:
|
||||
ADR-0055. The worker helper is `marqdo-provider-worker`
|
||||
(`crates/marqdo-provider-worker`, ADR-0038).
|
||||
|
||||
> **A trusted helper is not a sandbox.** The provider `worker` transport runs a
|
||||
> separate process with the invoking user's OS privileges. It isolates a crash
|
||||
> or protocol failure from the interpreter; it does **not** restrict file
|
||||
> system, network, process or memory access. `sha256` values are content
|
||||
> *integrity* checks, not publisher authentication. `strict` L1 refuses every
|
||||
> provider (fail-closed) until a real isolation adapter exists.
|
||||
|
||||
## 1. CLI surface
|
||||
|
||||
```text
|
||||
marqdo run FILE [--provider PATH/provider.json ...] [--allow-provider NAME ...]
|
||||
marqdo call FILE FUNCTION [--arg NAME=VALUE ...] \
|
||||
[--provider PATH/provider.json ...] [--allow-provider NAME ...]
|
||||
marqdo check FILE # no --provider/--allow-provider options exist
|
||||
```
|
||||
|
||||
* `--provider PATH` is repeatable. `PATH` names a `provider.json` file
|
||||
(directories are rejected). Only explicitly passed manifests are considered;
|
||||
there is no directory scan, package-manager discovery, registry download or
|
||||
network fetch.
|
||||
* `--allow-provider NAME` is repeatable and matches the manifest `name`
|
||||
**exactly**. There is no `*`/glob wildcard, prefix match or implicit
|
||||
authorization. Selecting a provider with `--provider` never grants it.
|
||||
* `check`/`inspect` do not accept these options and never read a provider
|
||||
manifest, start a helper or call provider code. Provider kinds are not new
|
||||
callables; they are values passed to `resource.open(kind, ...)`.
|
||||
* `--mode pure` and `--mode strict` reject `--provider`/`--allow-provider` in
|
||||
preflight (exit 2, no helper spawn, no silent downgrade to `project`).
|
||||
`pure` reports `cli.invalid_grants`; `strict` reports
|
||||
`strict.unsupported_grant`.
|
||||
|
||||
### 1.1 Ordering
|
||||
|
||||
`run`/`call` follow RFC-EXT-0001 §18.2:
|
||||
|
||||
1. T1–T5: load the Markdown graph and prepare call bindings (unchanged;
|
||||
providers are not part of `prepare`).
|
||||
2. T6: parse all `--provider` manifests, validate schema/name/version/ABI/
|
||||
transport/kinds/artifact declarations, select and verify the current-target
|
||||
artifact (regular file, inside the bundle, size bound, sha256), reject
|
||||
duplicate provider names and duplicate kinds across providers, and check
|
||||
the exact `--allow-provider` grants.
|
||||
3. T6b: verify all selected plugin manifests/artifacts (existing behavior).
|
||||
4. T7: start plugin workers and reconcile exports, then spawn one
|
||||
`marqdo-provider-worker` per granted+verified provider.
|
||||
5. T9: execute the entry function; every `resource.open` re-checks the provider
|
||||
name grant in the Host.
|
||||
|
||||
No worker process (plugin or provider) is started before every T6/T6b check
|
||||
passes. A manifest, target, integrity, grant or duplicate-kind failure leaves
|
||||
the main function unexecuted.
|
||||
|
||||
An `--allow-provider NAME` that matches no selected manifest is a deterministic
|
||||
option error (`ext.provider_grant_unmatched`, exit 2) rather than silent
|
||||
discovery fan-out.
|
||||
|
||||
## 2. Manifest file
|
||||
|
||||
* File name convention: `provider.json`. It must be a UTF-8 JSON object of at
|
||||
most 1 MiB and is read from an explicit path, never from `PATH`.
|
||||
* The CLI accepts **schema version 2 only**. A `schema_version: 1` manifest is
|
||||
rejected with `ext.unsupported_provider_schema`; there is no silent
|
||||
migration and no v1/v2 auto-detection. (v1 remains usable only through the
|
||||
host embedding API `StdHost::register_provider_package`, ADR-0037.)
|
||||
* Unknown top-level fields are rejected inside schema v2. Extensible,
|
||||
non-executable data must live in the `metadata` object, which has no
|
||||
execution meaning. A future schema version must be a new document, not an
|
||||
in-place extension of v2.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "marqdo.provider",
|
||||
"schema_version": 2,
|
||||
"name": "demo.sensor",
|
||||
"version": "0.1.0",
|
||||
"abi": "c-abi-1",
|
||||
"transport": "worker",
|
||||
"kinds": ["demo.sensor"],
|
||||
"artifacts": [
|
||||
{
|
||||
"target": "x86_64-unknown-linux-gnu",
|
||||
"path": "lib/libdemo_sensor.so",
|
||||
"sha256": "<64-lowercase-hex-sha256>"
|
||||
}
|
||||
],
|
||||
"permissions_requested": []
|
||||
}
|
||||
```
|
||||
|
||||
### 2.1 Fields
|
||||
|
||||
| Field | Required | Rules |
|
||||
|---|---|---|
|
||||
| `schema` | yes | Exactly `"marqdo.provider"`. |
|
||||
| `schema_version` | yes | Exactly `2` (integer). |
|
||||
| `name` | yes | Stable grant name. Non-empty dotted name: each `.`-separated segment starts with a lowercase ASCII letter and continues with lowercase ASCII letters, digits, `-` or `_`; at most 128 bytes. A name equal to a reserved core resource kind is rejected. |
|
||||
| `version` | yes | Strict SemVer (`semver::Version`, three numeric components). |
|
||||
| `abi` | yes | Exactly `"c-abi-1"` (ABI v1, ADR-0036/0038). Anything else is `ext.unsupported_provider_schema`. |
|
||||
| `transport` | yes | Exactly `"worker"` (framed-JSON helper, ADR-0035/0038). Anything else is `ext.unsupported_provider_schema`. |
|
||||
| `kinds` | yes | Non-empty, at most 64 unique dotted names with the same grammar as `name`. Core kinds are reserved (see §2.2). |
|
||||
| `artifacts` | yes | Non-empty, at most 32 entries with unique target triples. Each entry has `target`, `path`, `sha256`; unknown entry fields are rejected. |
|
||||
| `permissions_requested` | no | Declarations only, at most 64 unique strings: a known host capability (`marqdo_plugin_abi::KNOWN_CAPABILITIES`) or a dotted vendor capability (at least three segments, no core namespace prefix). Declarations never grant anything. |
|
||||
| `metadata` | no | Arbitrary JSON. Ignored for execution; never used to select artifacts or grant permissions. |
|
||||
|
||||
`artifacts[].target` must parse as a target triple. `artifacts[].path` is
|
||||
bundle-relative: absolute paths, `..` components and other escapes are
|
||||
rejected. `artifacts[].sha256` must be 64 lowercase hexadecimal characters.
|
||||
`artifacts[].path` is resolved against the manifest's own directory (the
|
||||
bundle root); a symlink is followed only when the canonical target stays inside
|
||||
that root, and the final target must be a regular file no larger than 256 MiB.
|
||||
|
||||
### 2.2 Reserved core kinds
|
||||
|
||||
`stdio`, `file`, `tcp`, `tls`, `udp` and `tcp-listener` are implemented by
|
||||
`marqdo-host` before the provider registry is consulted. A provider may not
|
||||
declare them (nor use one as its `name`); registration can never shadow them.
|
||||
|
||||
Reserved core kinds are rejected with `ext.manifest_invalid`.
|
||||
|
||||
### 2.3 Artifact selection
|
||||
|
||||
The only artifact used is the one whose `target` equals
|
||||
`marqdo_plugin_abi::HostTarget::current().triple()`. A missing current target
|
||||
is `ext.no_matching_artifact` (exit 2); another platform's artifact or a
|
||||
`default` fallback is never substituted.
|
||||
|
||||
## 3. Identity and integrity
|
||||
|
||||
* The manifest content digest is `sha256:<hex>` over the manifest bytes; the
|
||||
selected artifact digest is the declared lowercase `sha256`. Both are
|
||||
re-verified immediately before the helper is spawned, and again when the
|
||||
registration helper is called, so a file replaced between preparation and
|
||||
activation fails closed.
|
||||
* These hashes prove the manifest/artifact pair is byte-identical to what was
|
||||
selected. They do **not** prove who published it, that it is trustworthy or
|
||||
that it may be granted. Publisher trust needs a signature/trust-root layer
|
||||
that is out of scope for schema v2.
|
||||
* No version of this schema stores a secret, environment value or shell
|
||||
fragment. Manifests are never concatenated into a shell command.
|
||||
|
||||
### 3.1 Duplicate identity
|
||||
|
||||
* The same provider `name` selected with two different manifest digests is
|
||||
`ext.duplicate_provider`; the exact same manifest selected twice is
|
||||
idempotent.
|
||||
* One non-reserved `kind` declared by two providers is
|
||||
`ext.duplicate_provider_kind`. Registration never overwrites a previous
|
||||
provider ("last wins" is forbidden).
|
||||
* Error selection is deterministic: manifests are canonicalised, de-duplicated
|
||||
and sorted before conflicts are reported, independent of CLI argument order.
|
||||
* One provider may serve several non-reserved kinds; all of them are
|
||||
registered together and must be usable.
|
||||
|
||||
## 4. Worker helper resolution
|
||||
|
||||
The CLI resolves the helper **only** as a sibling of the running `marqdo`
|
||||
executable:
|
||||
|
||||
```text
|
||||
<directory of the marqdo executable>/marqdo-provider-worker[.exe]
|
||||
```
|
||||
|
||||
* No `PATH` lookup, no current-directory search, no environment override and no
|
||||
download fallback. A helper found on `PATH` is ignored.
|
||||
* A missing sibling (or a non-regular file) is `ext.worker_unavailable`
|
||||
(exit 1). A helper that fails to start or fails the protocol ping is also
|
||||
`ext.worker_unavailable`; there is no fallback to in-process/native loading.
|
||||
* Release archives must ship the `marqdo` binary and the
|
||||
`marqdo-provider-worker` helper built for the same target triple.
|
||||
* The `worker` transport is supported only on **linux/x86_64** in this release.
|
||||
Other platforms compile but fail closed with `ext.platform_unsupported`
|
||||
(exit 1). Windows and macOS are explicitly unsupported until they have a
|
||||
native fixture and release evidence; Linux success is not a platform claim.
|
||||
|
||||
## 5. Worker protocol v1 and token lifecycle
|
||||
|
||||
The helper is `marqdo-provider-worker <selected-artifact-path>`. It loads
|
||||
exactly one `c-abi-1` shared library, verifies
|
||||
`marqdo_resource_provider_abi_version() == 1`, and speaks the ADR-0035 framed
|
||||
protocol over stdin/stdout: 4-byte big-endian length + JSON, 1 MiB maximum per
|
||||
frame.
|
||||
|
||||
```text
|
||||
{"op":"ping"} -> {"ok":true} (handshake)
|
||||
{"op":"open","kind","descriptor","rights"} -> {"ok":true,"token":u64}
|
||||
{"op":"read","token","max_bytes"} -> {"ok":true,"bytes":[u8...],"eof":bool}
|
||||
{"op":"write","token","bytes":[u8...]} -> {"ok":true,"count":u64}
|
||||
{"op":"close","token"} -> {"ok":true}
|
||||
{"op":"shutdown"} -> process exit
|
||||
{"ok":false,"code","message"} -> structured failure
|
||||
```
|
||||
|
||||
Token lifecycle:
|
||||
|
||||
1. `ping` must succeed before the host registers the provider. A failed
|
||||
handshake aborts the run (`ext.worker_unavailable`).
|
||||
2. `open` creates a provider-private `u64` token in the helper. The language
|
||||
never sees it: the Host issues the opaque language `Resource` handle that
|
||||
binds run id + provider name + token. Provider tokens cannot be forged into
|
||||
language values or wire messages.
|
||||
3. `read`/`write`/`close` are dispatched by the Host with the token it stored.
|
||||
A read returns at most `max_bytes`; bytes cross the boundary by copy.
|
||||
4. `close` closes exactly one token. Repeated or unknown tokens must fail with
|
||||
a structured error; the Host never panics.
|
||||
5. When the run ends (normal, error or timeout) every open token is closed and
|
||||
the helper receives `shutdown`; the Host kills and reaps the child on drop.
|
||||
A helper crash/protocol violation becomes a structured `host.error` for the
|
||||
offending operation (CLI exit 1) and is never retried against another
|
||||
provider or an in-process transport.
|
||||
6. Provider error codes map to existing categories: `permission.denied`,
|
||||
`resource.closed`, `resource.exhausted`, `resource.unsupported`, otherwise
|
||||
`host.error`.
|
||||
|
||||
## 6. Authorization
|
||||
|
||||
* Registration, manifest parsing and artifact verification are not
|
||||
authorization. Every `resource.open("<kind>", ...)` still requires:
|
||||
project mode, the provider name in `Capabilities.resource_providers`
|
||||
(populated only by exact `--allow-provider NAME`), and the Host's provider
|
||||
name check. `permissions_requested` declarations do not affect this.
|
||||
* `pure` and `strict` never reach provider kinds. `project` is the only mode
|
||||
that can use a third-party provider.
|
||||
* Unauthorized selection fails closed before the worker is spawned:
|
||||
`--provider` without a matching `--allow-provider` is `permission.denied`
|
||||
(exit 1) and the helper is never started.
|
||||
|
||||
## 7. Error codes and exit codes
|
||||
|
||||
Manifest/CLI option failures exit 2; load/verification/spawn failures exit 1;
|
||||
`permission.denied` exits 1. With `--json` the CLI emits exactly one JSON
|
||||
document with a `code` (nested under `error.code` for these structured
|
||||
errors).
|
||||
|
||||
| Code | Stage | Trigger |
|
||||
|---|---|---|
|
||||
| `ext.manifest_unreadable` | manifest read (2) | missing/unreadable path, directory instead of `provider.json` |
|
||||
| `ext.unsupported_provider_schema` | manifest read (2) | `schema` mismatch, `schema_version != 2`, unsupported `abi`/`transport` |
|
||||
| `ext.manifest_invalid` | manifest read (2) | invalid UTF-8/JSON, unknown field, bad name/kind/version/permission, reserved kind, duplicate kind/target inside one manifest |
|
||||
| `ext.no_matching_artifact` | manifest read (2) | no artifact for the current host target |
|
||||
| `ext.duplicate_provider` | manifest read (2) | same provider name with two different manifest digests |
|
||||
| `ext.duplicate_provider_kind` | manifest read (2) | two providers declare the same non-reserved kind |
|
||||
| `ext.provider_grant_unmatched` | grant check (2) | `--allow-provider` names no selected manifest |
|
||||
| `ext.artifact_integrity` | artifact verify / pre-spawn re-verify (1) | sha256 mismatch, absolute/`..`/symlink escape, non-regular file, size limit, manifest changed after verification |
|
||||
| `permission.denied` | grant check (1) | `--provider` selected without its exact `--allow-provider` grant |
|
||||
| `ext.worker_unavailable` | spawn (1) | sibling helper missing/not a regular file, spawn failure, ping failure |
|
||||
| `ext.platform_unsupported` | spawn (1) | provider `worker` transport on a non-linux/x86_64 build |
|
||||
| `cli.invalid_grants` | CLI preflight (2) | wildcard/empty `--allow-provider`, provider flags in `pure` mode |
|
||||
| `strict.unsupported_grant` | strict preflight (2) | provider flags in `strict` mode |
|
||||
|
||||
`ext.manifest_invalid`, `ext.duplicate_provider` and
|
||||
`ext.provider_grant_unmatched` are additive codes: RFC-EXT-0001 §18.1 froze the
|
||||
provider-specific codes it could name, but a generic malformed-v2-manifest
|
||||
code, a duplicate-provider-name code and an unmatched-grant code are required
|
||||
to keep failure reasons distinct. They follow the same `ext.` naming and exit
|
||||
code contract and are stable.
|
||||
|
||||
## 8. Test matrix (PR-03)
|
||||
|
||||
`crates/cli/tests/provider_cli.rs` builds a real C `c-abi-1` sensor provider,
|
||||
runs the real `marqdo-provider-worker` next to the real CLI binary, and covers:
|
||||
|
||||
| ID | Evidence |
|
||||
|---|---|
|
||||
| PROV-01 | granted v2 worker + `resource.open/read/close("demo.sensor")` returns count 5, EOF, one close, `resource.open` in the host trace |
|
||||
| PROV-02 | `--provider` without grant → `permission.denied`; marker proves the helper was never started |
|
||||
| PROV-03 | `--allow-provider` without `--provider` → `ext.provider_grant_unmatched`, no worker |
|
||||
| PROV-04 | same kind from two providers → `ext.duplicate_provider_kind` before any worker |
|
||||
| PROV-05 | one provider with two non-reserved kinds → both open/read/close, one helper |
|
||||
| PROV-06 | `stdio`/`file`/`tcp`/`tls`/`udp`/`tcp-listener` kinds rejected, no worker |
|
||||
| PROV-07 | digest mismatch, symlink escape, `..` traversal → `ext.artifact_integrity`, no worker |
|
||||
| PROV-08 | no current-target artifact → `ext.no_matching_artifact` |
|
||||
| PROV-09 | `schema_version: 1` → `ext.unsupported_provider_schema` |
|
||||
| PROV-10 | `pure`/`strict` + provider flags → preflight refusal, no worker; wildcard grant rejected |
|
||||
| PROV-13 | relocated CLI without sibling helper (helper present on `PATH`) → `ext.worker_unavailable`, PATH is never used |
|
||||
|
||||
PROV-11 (worker crash/timeout/size-limit), PROV-12 (closed/stale/forged token),
|
||||
PROV-14 (concurrent lifecycle) and PROV-15 (frozen-binary external repository)
|
||||
are covered by the host worker/protocol tests or belong to the PR-07 external
|
||||
conformance repository; they are not part of this CLI slice's manifest matrix.
|
||||
+71
-12
@@ -9,6 +9,7 @@ Normative implementation:
|
||||
|---|---|
|
||||
| Sandbox modes, strict refusal | `crates/marqdo-sandbox/src/lib.rs` |
|
||||
| Capability sets and checks | `crates/marqdo-host/src/capability.rs` |
|
||||
| Trusted capability handlers | `crates/marqdo-host/src/capability.rs` (`CapabilityBroker`, `CapabilityHandler`, `FsReadHandler`; ADR-0057) |
|
||||
| Host dispatch and per-op checks | `crates/marqdo-host/src/host.rs` |
|
||||
| Project-rooted filesystem | `crates/marqdo-host/src/fs.rs` |
|
||||
| Budgets | `crates/marqdo-host/src/budget.rs` |
|
||||
@@ -175,18 +176,72 @@ before reading. Full symlink/race hardening is an open P4 item.
|
||||
* plugin invoke → the **complete export name** must be registered, then
|
||||
`capabilities.check_plugin(package)` on **every call**; loading also checks
|
||||
the same grant once;
|
||||
* plugin host-capability requests → `CapabilityBroker` checks the package and
|
||||
effective grants and **executes** granted `fs.read` through `ProjectFs`,
|
||||
returning the real file bytes. Other core capabilities (`fs.write`,
|
||||
`net.request`, `process`, console/math/log/...), and vendor capabilities
|
||||
without a trusted executing handler, return `capability.unavailable`; they
|
||||
never pretend an empty reply is success. Unknown/vendor names without a
|
||||
trusted policy entry become `capability.unavailable`/`permission.denied`;
|
||||
* plugin host-capability requests → `CapabilityBroker` runs a fixed chain:
|
||||
`decide` (package, effective grants, trusted vendor policy, payload shape) →
|
||||
a trusted `CapabilityHandler` must be installed for the exact capability name
|
||||
→ a package-bound handler must own the trusted package identity → the handler
|
||||
executes under the broker budget and reply limit → one audit entry. Granted
|
||||
`fs.read` executes through `FsReadHandler`, which resolves the path with
|
||||
`ProjectFs` and re-checks the read grant on the normalized project-relative
|
||||
path before returning the real file bytes. Every capability without an
|
||||
installed handler — other core capabilities (`fs.write`, `net.request`,
|
||||
`process`, console/math/log/clock/random/artifact.record) and vendor
|
||||
capabilities without a trusted executor — returns `capability.unavailable`;
|
||||
it never pretends an empty reply is success. A vendor name with no trusted
|
||||
policy entry is `permission.denied` (ADR-0011, ADR-0057);
|
||||
* print → output budget only.
|
||||
|
||||
Static analysis may warn, but it never replaces these runtime checks. There is
|
||||
no host bypass for tests: tests use the same `StdHost`/`Capabilities` types.
|
||||
|
||||
### 2.3 Executable capabilities need an installed trusted handler
|
||||
|
||||
`CapabilityBroker` (`crates/marqdo-host/src/capability.rs`, ADR-0057) separates
|
||||
*being authorized* from *being executable*:
|
||||
|
||||
| Question | Registry | Failure |
|
||||
|---|---|---|
|
||||
| Is this capability granted for this trusted package? | `Capabilities` / `vendor_policy` | `permission.denied` |
|
||||
| Is a trusted executor installed for this exact name? | `handlers: BTreeMap<String, HandlerEntry>` | `capability.unavailable` |
|
||||
| Does a package-bound handler own the claimed package? | `HandlerEntry.package` | `permission.denied` |
|
||||
|
||||
Execution order is fixed: `decide` → handler installed → package binding →
|
||||
handler execution under the broker `Budget` and reply limit → audit. A name in
|
||||
`vendor_policy` is policy only; it does not install anything, and it cannot
|
||||
turn into an executor. A core capability granted by `Capabilities` but missing
|
||||
a handler is `capability.unavailable`, never an empty `Ok`.
|
||||
|
||||
* **Registration is host-only.** `register_handler(handler)` /
|
||||
`register_package_handler(package, handler)` are called by the trusted host /
|
||||
embedder composition root; no manifest, plugin or wire message can reach the
|
||||
registry. The registry key is `handler.name()`, so a handler cannot be
|
||||
registered under a name it does not declare.
|
||||
* **Rejected names** (stable codes): empty/whitespace/malformed names are
|
||||
`capability.handler_invalid`; names under the reserved host prefixes
|
||||
`marqdo.` / `host.` are `capability.handler_reserved`; an already registered
|
||||
name (including the built-in `fs.read`) is `capability.handler_conflict`. A
|
||||
rejected handler is not installed.
|
||||
* **`fs.read`** is the first migrated standard handler (`FsReadHandler`),
|
||||
installed by `CapabilityBroker::with_fs(fs)`. It keeps the `ProjectFs` root,
|
||||
traversal/absolute-path rejection, regular-file/UTF-8/8 MiB limits and the
|
||||
glob grant, and it re-checks the grant against the normalized path at the
|
||||
execution point. The broker reply limit (`MAX_CAPABILITY_REPLY_BYTES` by
|
||||
default) bounds the returned bytes; an oversize reply is
|
||||
`resource.exhausted`.
|
||||
* **Audit**: every decision records package, capability,
|
||||
`granted`/`denied`/`unavailable`, a reason and `duration_micros`. Payloads
|
||||
and file contents are never recorded, so ordinary secrets do not appear in
|
||||
the audit log.
|
||||
* **Budgets**: handlers receive `&mut Budget` and can charge fuel/output; the
|
||||
broker also enforces the reply byte limit. Native handlers still run
|
||||
in-process and are trusted code, not a sandbox.
|
||||
|
||||
Known gaps in this build: the broker `trace_id` is settable
|
||||
(`with_trace_id`) but `host.rs` does not supply it yet; and a broker-level
|
||||
`resource.exhausted` reply keeps its stable code but the host plugin-error
|
||||
mapping currently collapses `PluginError` kind `capability` to the
|
||||
`permission.denied` runtime category (changing that mapping is a follow-up).
|
||||
|
||||
## 3. Resource budgets
|
||||
|
||||
`Budget` (`crates/marqdo-host/src/budget.rs`) is per run:
|
||||
@@ -338,11 +393,15 @@ Neither command executes project code. Specifically:
|
||||
4. Write-path symlink TOCTOU (see §2.1).
|
||||
5. Plugin **worker crash isolation** and the worker capability bridge are
|
||||
implemented (not usable in strict L1): both native and worker plugins can
|
||||
reach the broker's
|
||||
`fs.read` execution when granted. Other core capabilities and vendor
|
||||
handlers remain unavailable. Broker execution is a cooperative API
|
||||
contract, not an OS sandbox; a worker and a native plugin can still issue
|
||||
direct syscalls.
|
||||
reach the broker's `fs.read` handler when granted (ADR-0057). Only
|
||||
`fs.read` has a registered handler; every other core capability and every
|
||||
vendor capability without an explicitly registered trusted handler remains
|
||||
`capability.unavailable`, and there is no API to register one from a plugin,
|
||||
manifest or CLI flag. Broker execution is a cooperative API contract, not an
|
||||
OS sandbox; a worker and a native plugin can still issue direct syscalls.
|
||||
Follow-ups: wire the run `trace_id` into the broker audit and teach the host
|
||||
plugin-error mapping to report `resource.exhausted` as a resource category
|
||||
rather than `permission.denied`.
|
||||
6. `plugin_handle` wire values are structurally reconstructable, but host
|
||||
authority comes only from the `PluginHandleTable` owner/run/generation
|
||||
check; untrusted plugin/foreign output containing a handle is rejected
|
||||
|
||||
Reference in new issue
Block a user