Complete the documentation set for the M0.5 toolchain, grounded in the code: - spec/references.md: import/reference/call edges, identities, loader rules, anchors, `ref` resolution/ambiguity, `check --refs` validation. - spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal string rule, special floats, limits and exact float round-trip. - spec/security.md: modes (strict refuses without OS isolation), capabilities, path rules, budgets, tracing, foreign process caveats, plugin trust. - spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync, unsupported registry/git/environment work. - spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the implemented CLI plugin-invocation flow with its limitations. - spec/math.md and spec/foreign.md: the implemented math subset and the foreign request/response/timeout/capability contract. - docs/adr/0001..0007: accepted decisions for the span foundation, inert executables outside function bodies, ref ambiguity (S-07), strict refusal, AST normalization, CLI contract and BigInt/typed wire (S-08). - docs/contributor-guide.md: repo map, workflow, fixture/golden updates, quality gate, ADR/commit policy. - docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18 Alpha checklist (conclusion: not yet Alpha-complete), including known divergences rather than overclaims. - README.md rewritten from "design-only starter" to the working implementation with quickstart, implemented phases, deliberate limits and doc index. - AGENTS.md: updated pointers/workflow only; architectural rules unchanged. - Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside functions, unsupported math.simplify/differentiate).
18 KiB
Marqdo Plugin ABI v1
Status: frozen ABI v1.0 (MQ_ABI_MAJOR=1, MQ_ABI_MINOR=0). The canonical
C header is include/marqdo_plugin_abi_v1.h; the Rust mirrors are in
crates/marqdo-plugin-abi/src/abi.rs; the plugin-side SDK is
crates/marqdo-plugin-sdk.
A C ABI is not a sandbox. A native plugin is loaded into the host process. It can read host memory, call arbitrary syscalls, bypass the capability model and crash the host. Loading requires an explicit trust decision. The isolated worker transport described by the blueprint is not implemented in this build.
1. Independent versions
Three versions exist and are never conflated:
| Version | Current | Where |
|---|---|---|
| Marqdo language/semantics | M0.5, spec/* |
docs/DEVELOPMENT_BLUEPRINT.md, specs |
| Plugin ABI major.minor | 1.0 |
MQ_ABI_MAJOR/MQ_ABI_MINOR, marqdo_plugin_abi::ABI_MAJOR/ABI_MINOR |
| Plugin package SemVer | e.g. 0.1.0 |
plugin.yaml plugin.version |
The JSON wire protocol has its own wire_version = 1
(spec/value-wire.md); it is not the ABI major version.
2. Stable-layout types
The header defines only fixed-width integers, size_t-based length+pointer
buffers, opaque host-owned handles and extern "C" function pointers. Rust
String/Vec/trait objects/Result/panics, C++ exceptions, Go objects and
raw host pointers never cross the boundary.
On a 64-bit target (include/marqdo_plugin_abi_v1.h static assertions and
crates/marqdo-plugin-abi/src/abi.rs compile-time assertions):
| Type | Size | Align |
|---|---|---|
MqBytes |
16 | 8 |
MqOwnedBytes |
16 | 8 |
MqPluginApi |
32 | 8 |
MqHostApi |
56 | 8 |
MqPluginInvoke (fn ptr) |
8 | — |
On a 32-bit target the Rust side asserts MqBytes/MqOwnedBytes = 8,
MqPluginApi = 20 and MqHostApi = 28. The layout uses size_t, so an
artifact is only loadable by a host built for the same target triple, pointer
width and byte order. (NULL, 0) is the only empty buffer; a non-empty
buffer with a NULL pointer is a protocol error.
2.1 Ownership and lifetimes
| Buffer | Producer | Lifetime / free rule |
|---|---|---|
args_wire, export_name, signature_wire, capability request_wire, plugin_metadata_wire (as seen by host) |
caller passing it | borrowed for the duration of the call; consumer must copy and must not retain or free |
result_wire, error_wire, capability reply_wire |
the producing side | must be allocated with host->alloc(host->host_data, size) and is released with host->free(host->host_data, data) by the consumer after copying; never malloc/new/GC memory |
plugin_metadata_wire (as returned by plugin) |
plugin static storage | valid only during mq_plugin_entry_v1; host copies it immediately |
MqCallContext is opaque, host-owned and valid only for the invocation it was
minted for. Plugins must not dereference, cache or copy it.
2.2 Size limits (enforced by host / SDK)
| Constant | Bytes |
|---|---|
MQ_MAX_EXPORT_NAME_BYTES |
256 |
MQ_MAX_SIGNATURE_WIRE_BYTES |
4096 |
MQ_MAX_METADATA_WIRE_BYTES |
65536 |
MQ_MAX_REQUEST_WIRE_BYTES |
16 MiB (16,777,216) |
MQ_MAX_RESULT_WIRE_BYTES |
64 MiB (67,108,864) |
MQ_MAX_ERROR_WIRE_BYTES |
1 MiB (1,048,576) |
MQ_MAX_CAPABILITY_REPLY_BYTES |
16 MiB |
Wire JSON decoding additionally applies the default WireLimits from
spec/value-wire.md (depth 64, 1,000,000 items, 16 MiB strings). Oversized
payloads are rejected, never truncated and reparsed.
3. Exported entry point and tables
Required symbol: mq_plugin_entry_v1 (MQ_PLUGIN_ENTRY_SYMBOL):
int32_t mq_plugin_entry_v1(const MqHostApi *host, MqPluginApi *plugin);
Called exactly once, by the loading thread:
- the host supplies a valid
MqHostApitable and a zeroedMqPluginApiwhosestruct_size = sizeof(MqPluginApi); - the plugin must validate the host table, fill
plugin.abi_major,plugin.abi_minor,plugin.plugin_metadata_wireand register every export viahost->register_function; - it returns
MQ_OKonly after all exports are registered; on non-zero the host ignores the plugin table and fails the load.
MQ_OK and error codes (stable integers):
| Code | Name | Meaning |
|---|---|---|
| 0 | MQ_OK |
success |
| 1 | MQ_ERR_INVALID_ARGUMENT |
NULL pointer, bad pointer/length pair |
| 2 | MQ_ERR_ABI_MISMATCH |
ABI major mismatch or incompatible struct_size |
| 3 | MQ_ERR_REGISTRATION |
an export could not be registered |
| 4 | MQ_ERR_UNSUPPORTED |
operation/type/capability unsupported |
| 5 | MQ_ERR_OUT_OF_MEMORY |
host allocation failed |
| 6 | MQ_ERR_BAD_WIRE |
payload is not valid wire_version 1 JSON |
| 7 | MQ_ERR_INTERNAL |
unexpected failure, including caught panic |
| 8 | MQ_ERR_CAPABILITY |
capability request denied/failed |
3.1 Version negotiation
Host side (negotiate in crates/marqdo-plugin-abi/src/abi.rs):
plugin.abi_major != 1→ load fails withabi_major_mismatch;plugin.abi_minorgreater than the host minor is accepted but clamped to the host minor (currently 0); only features defined at that minor may be used;- the plugin must reject a host table with
host == NULL,host->struct_size < sizeof(MqHostApi),host->abi_major != 1or NULL alloc/free/register_function/request_capability pointers.
struct_size is the extension mechanism: fields may only be appended in a
later ABI minor, never reordered or resized; a consumer only reads a field when
struct_size covers it.
3.2 Threading and lifetime
v1 is single-threaded and synchronous: entry is called once; invocations and
host callbacks for one plugin instance run on that thread with at most one call
in flight. A plugin must not start threads that call back into MqHostApi.
v1 has no shutdown callback and no hot unload/reload; the LoadedPlugin
wrapper unloads only when dropped, and the docs require one instance per
artifact path (ABI v1 plugins may keep the host table in a process-global).
No panic/exception/longjmp may cross the boundary; the SDK catches Rust panics
and returns MQ_ERR_INTERNAL with a structured error envelope.
4. plugin.yaml
plugin:
name: demo.solver
version: 0.1.0
abi: "^1.0"
kind: native
exports:
- name: solver.add
signature: "($a: Int, $b: Int) -> Int"
requires:
- math.compute
artifacts:
- target: x86_64-unknown-linux-gnu
file: solver.so
sha256: "<64 lowercase hex>"
Validation (crates/marqdo-plugin-abi/src/manifest.rs::PluginManifest::validate):
name: dotted segments, each starting with a lowercase ASCII letter and containing only lowercase letters, digits,-/_; ≤ 256 bytes; package names starting withmarqdo.orbuiltin.are reserved;version: semver-ishmajor.minor.patchwith optional-prereleaseand+build; numeric components have no leading zeros; not a SemVer range resolver;abi: caret-style requirement^1,^1.0,1or1.0. Caret semantics:^M.mmeans>=M.m, <M+1.0; the0.xspecial case pins the minor. It must be compatible with the host's1.0;kind:native,workerorwasm; onlynativeis loadable in-process in this build;exports: at least one, at most 256, unique, namespaced (must contain.), no reserved prefix (marqdo.,builtin.,core.,std.,host.), non-empty signature ≤ 4096 bytes, signature starts with(and contains->, no control characters;requires: at most 64, unique names from the whitelistlog,math.compute,fs.read,fs.write,net.request,clock,random,artifact.record; this is a declaration, not a grant;artifacts: 1–32, one per target triple; target parses as a 3–4 component triple (lowercase, alphanumeric/underscore components);fileis a non-empty relative path ≤ 256 bytes with no absolute/..escape;sha256is exactly 64 hex characters.
Manifest parsing/validation never executes plugin code and performs no filesystem access (artifact verification is a separate step).
5. Artifact selection and verification
HostTarget::current()derives a conventional triple fromstd::env::consts(x86_64-unknown-linux-gnu,aarch64-apple-darwin,x86_64-pc-windows-msvc, ...).resolve_artifactrequires exact triple equality; pointer width and byte order follow from the triple. No fallback to another architecture.verify_artifactresolves the path below the manifest directory and compares the file's SHA-256 with the manifest (case-insensitive hex). A missing file ismissing_artifact; a digest mismatch ishash_mismatch.- Verification proves the artifact is the file described by the manifest; it proves nothing about trust or authorization.
6. Loader lifecycle (host side)
crates/marqdo-plugin-abi/src/loader.rs::LoadedPlugin::open_with:
manifest.validate();- require
kind == native(otherwiseplugin_kind_not_loadable_in_process); - resolve artifact for the host target and verify its sha256;
libloading::Library::new(a trust decision, not a sandbox);- build the host API table and
HostState, initializeMqPluginApi; - call
mq_plugin_entry_v1once; non-zero → structured load error; - require
plugin_api.struct_size >= sizeof(MqPluginApi); - negotiate ABI major/minor;
- copy
plugin_metadata_wireand require metadataname/versionto match the manifest exactly; - take registered exports and cross-check against the manifest: duplicate registrations, missing declared exports, signature mismatches and undeclared exports all fail the load;
- keep the library mapped for the lifetime of
LoadedPlugin.
Invocation (LoadedPlugin::invoke): validate the request envelope, check the
request size, find the registered export, mint a MqCallContext, call the
plugin's MqPluginInvoke, copy and free both output buffers, then enforce the
envelope contract:
- success (
MQ_OK):error_wiremust be empty;result_wiremust be a non-emptywire_version1 success envelope; the typed value is optional (absentvaluemeansUnit); - failure:
error_wiremust hold awire_version1 error envelope; a missing or invalid error envelope still fails the call (structuredplugin_call_failed).
Capability requests from the plugin go through
MqHostApi::request_capability into the host's CapabilityHandler, on the
same thread, one at a time. The handler receives the capability name and
request bytes and returns reply bytes or an error. When no handler is
installed (the CLI's case) every request is denied (MQ_ERR_CAPABILITY).
7. Wire envelopes
All payloads are UTF-8 JSON with wire_version = 1. Values inside args and
value use the type-tagged encoding of spec/value-wire.md.
Request:
{
"wire_version": 1,
"kind": "request",
"function": "solver.add",
"args": {
"a": {"type": "int", "decimal": "20"},
"b": {"type": "int", "decimal": "22"}
},
"context": {"trace_id": "optional", "capability_handle": "opaque-and-scoped"}
}
Response (success):
{"wire_version":1,"kind":"response","value":{"type":"int","decimal":"42"}}
A success response with neither value nor error means Unit. A response
with both is rejected (ambiguous_response). Error:
{"wire_version":1,"kind":"response","error":{
"kind":"invoke","code":"missing_argument",
"message":"required argument `a` is missing","recoverable":true
}}
Envelope limits: at most MAX_ARGS = 1024 request arguments; request ≤ 16 MiB,
response ≤ 64 MiB, error ≤ 1 MiB. context carries no secrets and no
reconstructable OS handles. signature_wire is
{"signature":"<signature string>"}; plugin_metadata_wire is
{"name":"<package name>","version":"<semver>"} (extra fields ignored).
8. Error model
Host-side errors are PluginError { kind, code, message, detail?, recoverable }
with stable kind values: manifest, abi, artifact, load, wire,
capability, invoke, unsupported. Examples: abi_major_mismatch,
no_matching_artifact, hash_mismatch, missing_entry_symbol,
metadata_name_mismatch, export_signature_mismatch,
unsupported_wire_version. message/detail must never contain secrets.
They are serializable and are used as the wire error body.
When a plugin is used from Marqdo code, host errors become structured
RuntimeErrors (crates/marqdo-host/src/host.rs::map_plugin_error):
| Plugin error kind | Runtime code |
|---|---|
abi |
plugin.abi_mismatch |
capability |
permission.denied |
wire |
runtime.type |
unsupported |
kernel.unavailable |
manifest/artifact/load/invoke |
plugin.failure |
9. Using a plugin from .mq.md
This build supports explicitly trusted in-process native plugins only.
# 1. Build the C example and refresh its sha256 in plugin.yaml
./examples/plugin-demo/build.sh
# 2. Inspect without loading
marqdo plugin inspect examples/plugin-demo
# 3. Load with an explicit trust decision and a capability grant, then run
marqdo run examples/plugin-demo/plugin-call.mq.md \
--plugin examples/plugin-demo \
--trust-native-plugin \
--allow-plugin demo.solver
Flags (run and call): --plugin <manifest-or-dir> (repeatable),
--trust-native-plugin, --allow-plugin <package-name>. --allow-plugin
matches the manifest plugin.name (for example demo.solver), not the call
namespace (solver).
Document syntax (examples/plugin-demo/plugin-call.mq.md):
## main
**结果 = solver.add(a=20, b=22)**
> 打印(结果)
Semantics:
- the call namespace is derived from the first manifest export prefix
(
solver.add→ namespacesolver); a manifest whose exports use several prefixes registers only the first namespace, so exports under other prefixes are not reachable in this build; - calls accept named arguments only; positional arguments are a runtime type error ("plugin calls require named arguments");
- each call becomes
HostOp::PluginInvokewith a typed wire request; the plugin's typed response becomes a Marqdo value (Unitwhen absent); - the capability grant is re-checked on every call, not only at load;
- plugin namespace calls are not statically validated:
checkpasses a document that calls a namespace no plugin provides; at runtime the call fails withsymbol.unknown("no plugin namespace ... is loaded") unless--pluginloaded it; - plugin host-capability requests are denied because the CLI installs no
capability handler (
LoadOptions::default()); --mode strictrefuses to run before any plugin is loaded;--mode purecannot grant--allow-plugin, so loading fails withpermission.denied.
Trust boundary: --trust-native-plugin is an explicit unsafe decision; the
process shares its address space with the plugin. In-process load is the only
transport implemented — there is no worker process, Wasm component, hot
unload, or plugin capability bridge. A native plugin crash is a host crash.
10. SDK usage (crates/marqdo-plugin-sdk)
The SDK hides the raw C layout and exports the entry symbol with
mq_plugin_entry!. A complete Rust plugin crate:
use marqdo_plugin_sdk::{mq_plugin_entry, CallContext, PluginError, Value};
fn add(_ctx: &mut CallContext, args: &[u8]) -> Result<Vec<u8>, PluginError> {
let request = marqdo_plugin_sdk::wire::decode_request(args)?;
let values = request.typed_args()?;
let a = marqdo_plugin_sdk::wire::int_arg(&values, "a")?;
let b = marqdo_plugin_sdk::wire::int_arg(&values, "b")?;
marqdo_plugin_sdk::wire::success(Some(Value::Int((a + b).into())))
}
mq_plugin_entry! {
name: "demo.solver",
version: "0.1.0",
exports: {
"solver.add" => "($a: Int, $b: Int) -> Int" => add,
}
}
The macro validates the host table, reports the plugin ABI version and metadata,
registers every export and returns MQ_OK only if all registrations succeed.
Handlers receive a &mut CallContext; use
CallContext::capability_request(name, request_bytes) for an already-authorized
host capability (the CLI grants none today), and
CallContext::log(level, message) as a convenience. Panics are caught at the
ABI boundary and converted to a structured error. The SDK depends only on
marqdo-plugin-abi, never on host-only crates or the interpreter.
11. C demo (examples/plugin-demo)
examples/plugin-demo/solver.c is a dependency-free C implementation of one
export (solver.add, Int + Int) using a minimal JSON scanner; it validates
the host table, reports metadata and registers the export. build.sh compiles
it (cc -shared -fPIC -I include) and rewrites the sha256 in plugin.yaml
(the checked-in manifest uses a zero placeholder until you build). The
examples/plugin-demo/README.md walks through the security model, build,
inspect and call flow.
The integration test crates/marqdo-plugin-abi/tests/loader_unix.rs compiles
this exact solver.c with cc, loads it with libloading, round-trips a wire
request, and rebuilds it with -DMQ_SOLVER_REPORT_ABI_MAJOR=2 to prove that a
wrong ABI major fails the load. It also covers a missing entry symbol, missing
artifact, hash mismatch, malformed manifest and target mismatch. The tests skip
when no C compiler is available.
12. Not implemented / explicit limitations
| Item | Status |
|---|---|
| Isolated plugin worker transport (length-prefixed IPC) | Not implemented; PlatformCapabilities::worker_processes = false. |
| Wasm Component + WIT | Not implemented. |
| Hot unload/reload | Not supported by ABI v1 (no shutdown callback). |
| Plugin capability bridge from the CLI | Not implemented; CLI supplies no handler, so all plugin host-capability requests are denied. |
| Plugin installation by package manager | Not implemented; marqdo.yaml plugins: is declarative only. |
| Static validation of plugin namespaces/exports/signatures against loaded plugins | Not implemented; errors are runtime-only. |
| Multiple namespaces per plugin | Not implemented; single namespace from the first export prefix. |
| Positional plugin arguments | Not supported; named only. |
| Thread-safe / concurrent invocation | Not supported (single-threaded v1 contract). |
Old marqdo_abi.h v2 compatibility |
Not implemented and not claimed; a separate adapter would be required. |