Files
marqdo-cli/spec/resource-provider-v2.md
T
cflmy 1f6fd644a0 feat(ext): single callable binding, provider v2 CLI, capability handlers, stdlib aliases
Implements RFC-EXT-0001 PR-00..PR-07 first pass:
- CallSiteKey/CallBindingTable + one frontend binding pass; runtime dispatch by identity
- prepare composition root shared by check/run/inspect/Analysis; exact plugin reconciliation
- provider.json v2 --provider/--allow-provider worker CLI with fail-closed verification
- capability handler registry with real fs.read and audit
- verified stdlib public aliases replace print_override/AST scan
- docs: specs, ADR-0054/0055/0057/0058, README, CI/release gates, evidence

PR-04 extension selection/lock v4 and final frozen-binary evidence remain.
2026-11-22 00:17:36 +08:00

15 KiB
Raw Blame History

Marqdo Resource Provider v2 (provider.json)

Status: frozen provider manifest schema version 2 for the marqdo CLI composition root. Implementation: crates/cli/src/provider_manifest.rs; prerequisite host transports: ADR-0033/0035/0036/0037/0038; CLI decision: ADR-0055. The worker helper is marqdo-provider-worker (crates/marqdo-provider-worker, ADR-0038).

A trusted helper is not a sandbox. The provider worker transport runs a separate process with the invoking user's OS privileges. It isolates a crash or protocol failure from the interpreter; it does not restrict file system, network, process or memory access. sha256 values are content integrity checks, not publisher authentication. strict L1 refuses every provider (fail-closed) until a real isolation adapter exists.

1. CLI surface

marqdo run  FILE [--provider PATH/provider.json ...] [--allow-provider NAME ...]
marqdo call FILE FUNCTION [--arg NAME=VALUE ...] \
             [--provider PATH/provider.json ...] [--allow-provider NAME ...]
marqdo check FILE            # no --provider/--allow-provider options exist
  • --provider PATH is repeatable. PATH names a provider.json file (directories are rejected). Only explicitly passed manifests are considered; there is no directory scan, package-manager discovery, registry download or network fetch.
  • --allow-provider NAME is repeatable and matches the manifest name exactly. There is no */glob wildcard, prefix match or implicit authorization. Selecting a provider with --provider never grants it.
  • check/inspect do not accept these options and never read a provider manifest, start a helper or call provider code. Provider kinds are not new callables; they are values passed to resource.open(kind, ...).
  • --mode pure and --mode strict reject --provider/--allow-provider in preflight (exit 2, no helper spawn, no silent downgrade to project). pure reports cli.invalid_grants; strict reports strict.unsupported_grant.

1.1 Ordering

run/call follow RFC-EXT-0001 §18.2:

  1. T1–T5: load the Markdown graph and prepare call bindings (unchanged; providers are not part of prepare).
  2. T6: parse all --provider manifests, validate schema/name/version/ABI/ transport/kinds/artifact declarations, select and verify the current-target artifact (regular file, inside the bundle, size bound, sha256), reject duplicate provider names and duplicate kinds across providers, and check the exact --allow-provider grants.
  3. T6b: verify all selected plugin manifests/artifacts (existing behavior).
  4. T7: start plugin workers and reconcile exports, then spawn one marqdo-provider-worker per granted+verified provider.
  5. T9: execute the entry function; every resource.open re-checks the provider name grant in the Host.

No worker process (plugin or provider) is started before every T6/T6b check passes. A manifest, target, integrity, grant or duplicate-kind failure leaves the main function unexecuted.

An --allow-provider NAME that matches no selected manifest is a deterministic option error (ext.provider_grant_unmatched, exit 2) rather than silent discovery fan-out.

2. Manifest file

  • File name convention: provider.json. It must be a UTF-8 JSON object of at most 1 MiB and is read from an explicit path, never from PATH.
  • The CLI accepts schema version 2 only. A schema_version: 1 manifest is rejected with ext.unsupported_provider_schema; there is no silent migration and no v1/v2 auto-detection. (v1 remains usable only through the host embedding API StdHost::register_provider_package, ADR-0037.)
  • Unknown top-level fields are rejected inside schema v2. Extensible, non-executable data must live in the metadata object, which has no execution meaning. A future schema version must be a new document, not an in-place extension of v2.
{
  "schema": "marqdo.provider",
  "schema_version": 2,
  "name": "demo.sensor",
  "version": "0.1.0",
  "abi": "c-abi-1",
  "transport": "worker",
  "kinds": ["demo.sensor"],
  "artifacts": [
    {
      "target": "x86_64-unknown-linux-gnu",
      "path": "lib/libdemo_sensor.so",
      "sha256": "<64-lowercase-hex-sha256>"
    }
  ],
  "permissions_requested": []
}

2.1 Fields

Field Required Rules
schema yes Exactly "marqdo.provider".
schema_version yes Exactly 2 (integer).
name yes Stable grant name. Non-empty dotted name: each .-separated segment starts with a lowercase ASCII letter and continues with lowercase ASCII letters, digits, - or _; at most 128 bytes. A name equal to a reserved core resource kind is rejected.
version yes Strict SemVer (semver::Version, three numeric components).
abi yes Exactly "c-abi-1" (ABI v1, ADR-0036/0038). Anything else is ext.unsupported_provider_schema.
transport yes Exactly "worker" (framed-JSON helper, ADR-0035/0038). Anything else is ext.unsupported_provider_schema.
kinds yes Non-empty, at most 64 unique dotted names with the same grammar as name. Core kinds are reserved (see §2.2).
artifacts yes Non-empty, at most 32 entries with unique target triples. Each entry has target, path, sha256; unknown entry fields are rejected.
permissions_requested no Declarations only, at most 64 unique strings: a known host capability (marqdo_plugin_abi::KNOWN_CAPABILITIES) or a dotted vendor capability (at least three segments, no core namespace prefix). Declarations never grant anything.
metadata no Arbitrary JSON. Ignored for execution; never used to select artifacts or grant permissions.

artifacts[].target must parse as a target triple. artifacts[].path is bundle-relative: absolute paths, .. components and other escapes are rejected. artifacts[].sha256 must be 64 lowercase hexadecimal characters. artifacts[].path is resolved against the manifest's own directory (the bundle root); a symlink is followed only when the canonical target stays inside that root, and the final target must be a regular file no larger than 256 MiB.

2.2 Reserved core kinds

stdio, file, tcp, tls, udp and tcp-listener are implemented by marqdo-host before the provider registry is consulted. A provider may not declare them (nor use one as its name); registration can never shadow them.

Reserved core kinds are rejected with ext.manifest_invalid.

2.3 Artifact selection

The only artifact used is the one whose target equals marqdo_plugin_abi::HostTarget::current().triple(). A missing current target is ext.no_matching_artifact (exit 2); another platform's artifact or a default fallback is never substituted.

3. Identity and integrity

  • The manifest content digest is sha256:<hex> over the manifest bytes; the selected artifact digest is the declared lowercase sha256. Both are re-verified immediately before the helper is spawned, and again when the registration helper is called, so a file replaced between preparation and activation fails closed.
  • These hashes prove the manifest/artifact pair is byte-identical to what was selected. They do not prove who published it, that it is trustworthy or that it may be granted. Publisher trust needs a signature/trust-root layer that is out of scope for schema v2.
  • No version of this schema stores a secret, environment value or shell fragment. Manifests are never concatenated into a shell command.

3.1 Duplicate identity

  • The same provider name selected with two different manifest digests is ext.duplicate_provider; the exact same manifest selected twice is idempotent.
  • One non-reserved kind declared by two providers is ext.duplicate_provider_kind. Registration never overwrites a previous provider ("last wins" is forbidden).
  • Error selection is deterministic: manifests are canonicalised, de-duplicated and sorted before conflicts are reported, independent of CLI argument order.
  • One provider may serve several non-reserved kinds; all of them are registered together and must be usable.

4. Worker helper resolution

The CLI resolves the helper only as a sibling of the running marqdo executable:

<directory of the marqdo executable>/marqdo-provider-worker[.exe]
  • No PATH lookup, no current-directory search, no environment override and no download fallback. A helper found on PATH is ignored.
  • A missing sibling (or a non-regular file) is ext.worker_unavailable (exit 1). A helper that fails to start or fails the protocol ping is also ext.worker_unavailable; there is no fallback to in-process/native loading.
  • Release archives must ship the marqdo binary and the marqdo-provider-worker helper built for the same target triple.
  • The worker transport is supported only on linux/x86_64 in this release. Other platforms compile but fail closed with ext.platform_unsupported (exit 1). Windows and macOS are explicitly unsupported until they have a native fixture and release evidence; Linux success is not a platform claim.

5. Worker protocol v1 and token lifecycle

The helper is marqdo-provider-worker <selected-artifact-path>. It loads exactly one c-abi-1 shared library, verifies marqdo_resource_provider_abi_version() == 1, and speaks the ADR-0035 framed protocol over stdin/stdout: 4-byte big-endian length + JSON, 1 MiB maximum per frame.

{"op":"ping"}                              -> {"ok":true}          (handshake)
{"op":"open","kind","descriptor","rights"} -> {"ok":true,"token":u64}
{"op":"read","token","max_bytes"}          -> {"ok":true,"bytes":[u8...],"eof":bool}
{"op":"write","token","bytes":[u8...]}     -> {"ok":true,"count":u64}
{"op":"close","token"}                     -> {"ok":true}
{"op":"shutdown"}                          -> process exit
{"ok":false,"code","message"}              -> structured failure

Token lifecycle:

  1. ping must succeed before the host registers the provider. A failed handshake aborts the run (ext.worker_unavailable).
  2. open creates a provider-private u64 token in the helper. The language never sees it: the Host issues the opaque language Resource handle that binds run id + provider name + token. Provider tokens cannot be forged into language values or wire messages.
  3. read/write/close are dispatched by the Host with the token it stored. A read returns at most max_bytes; bytes cross the boundary by copy.
  4. close closes exactly one token. Repeated or unknown tokens must fail with a structured error; the Host never panics.
  5. When the run ends (normal, error or timeout) every open token is closed and the helper receives shutdown; the Host kills and reaps the child on drop. A helper crash/protocol violation becomes a structured host.error for the offending operation (CLI exit 1) and is never retried against another provider or an in-process transport.
  6. Provider error codes map to existing categories: permission.denied, resource.closed, resource.exhausted, resource.unsupported, otherwise host.error.

6. Authorization

  • Registration, manifest parsing and artifact verification are not authorization. Every resource.open("<kind>", ...) still requires: project mode, the provider name in Capabilities.resource_providers (populated only by exact --allow-provider NAME), and the Host's provider name check. permissions_requested declarations do not affect this.
  • pure and strict never reach provider kinds. project is the only mode that can use a third-party provider.
  • Unauthorized selection fails closed before the worker is spawned: --provider without a matching --allow-provider is permission.denied (exit 1) and the helper is never started.

7. Error codes and exit codes

Manifest/CLI option failures exit 2; load/verification/spawn failures exit 1; permission.denied exits 1. With --json the CLI emits exactly one JSON document with a code (nested under error.code for these structured errors).

Code Stage Trigger
ext.manifest_unreadable manifest read (2) missing/unreadable path, directory instead of provider.json
ext.unsupported_provider_schema manifest read (2) schema mismatch, schema_version != 2, unsupported abi/transport
ext.manifest_invalid manifest read (2) invalid UTF-8/JSON, unknown field, bad name/kind/version/permission, reserved kind, duplicate kind/target inside one manifest
ext.no_matching_artifact manifest read (2) no artifact for the current host target
ext.duplicate_provider manifest read (2) same provider name with two different manifest digests
ext.duplicate_provider_kind manifest read (2) two providers declare the same non-reserved kind
ext.provider_grant_unmatched grant check (2) --allow-provider names no selected manifest
ext.artifact_integrity artifact verify / pre-spawn re-verify (1) sha256 mismatch, absolute/../symlink escape, non-regular file, size limit, manifest changed after verification
permission.denied grant check (1) --provider selected without its exact --allow-provider grant
ext.worker_unavailable spawn (1) sibling helper missing/not a regular file, spawn failure, ping failure
ext.platform_unsupported spawn (1) provider worker transport on a non-linux/x86_64 build
cli.invalid_grants CLI preflight (2) wildcard/empty --allow-provider, provider flags in pure mode
strict.unsupported_grant strict preflight (2) provider flags in strict mode

ext.manifest_invalid, ext.duplicate_provider and ext.provider_grant_unmatched are additive codes: RFC-EXT-0001 §18.1 froze the provider-specific codes it could name, but a generic malformed-v2-manifest code, a duplicate-provider-name code and an unmatched-grant code are required to keep failure reasons distinct. They follow the same ext. naming and exit code contract and are stable.

8. Test matrix (PR-03)

crates/cli/tests/provider_cli.rs builds a real C c-abi-1 sensor provider, runs the real marqdo-provider-worker next to the real CLI binary, and covers:

ID Evidence
PROV-01 granted v2 worker + resource.open/read/close("demo.sensor") returns count 5, EOF, one close, resource.open in the host trace
PROV-02 --provider without grant → permission.denied; marker proves the helper was never started
PROV-03 --allow-provider without --provider → ext.provider_grant_unmatched, no worker
PROV-04 same kind from two providers → ext.duplicate_provider_kind before any worker
PROV-05 one provider with two non-reserved kinds → both open/read/close, one helper
PROV-06 stdio/file/tcp/tls/udp/tcp-listener kinds rejected, no worker
PROV-07 digest mismatch, symlink escape, .. traversal → ext.artifact_integrity, no worker
PROV-08 no current-target artifact → ext.no_matching_artifact
PROV-09 schema_version: 1 → ext.unsupported_provider_schema
PROV-10 pure/strict + provider flags → preflight refusal, no worker; wildcard grant rejected
PROV-13 relocated CLI without sibling helper (helper present on PATH) → ext.worker_unavailable, PATH is never used

PROV-11 (worker crash/timeout/size-limit), PROV-12 (closed/stale/forged token), PROV-14 (concurrent lifecycle) and PROV-15 (frozen-binary external repository) are covered by the host worker/protocol tests or belong to the PR-07 external conformance repository; they are not part of this CLI slice's manifest matrix.