Files
marqdo-cli/spec/plugin-abi.md
T
cflmy 0116160399 docs: formal specs, ADRs, STATUS matrix, contributor guide and honest README
Complete the documentation set for the M0.5 toolchain, grounded in the code:

- spec/references.md: import/reference/call edges, identities, loader rules,
  anchors, `ref` resolution/ambiguity, `check --refs` validation.
- spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal
  string rule, special floats, limits and exact float round-trip.
- spec/security.md: modes (strict refuses without OS isolation), capabilities,
  path rules, budgets, tracing, foreign process caveats, plugin trust.
- spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync,
  unsupported registry/git/environment work.
- spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact
  rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the
  implemented CLI plugin-invocation flow with its limitations.
- spec/math.md and spec/foreign.md: the implemented math subset and the
  foreign request/response/timeout/capability contract.
- docs/adr/0001..0007: accepted decisions for the span foundation, inert
  executables outside function bodies, ref ambiguity (S-07), strict refusal,
  AST normalization, CLI contract and BigInt/typed wire (S-08).
- docs/contributor-guide.md: repo map, workflow, fixture/golden updates,
  quality gate, ADR/commit policy.
- docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18
  Alpha checklist (conclusion: not yet Alpha-complete), including known
  divergences rather than overclaims.
- README.md rewritten from "design-only starter" to the working implementation
  with quickstart, implemented phases, deliberate limits and doc index.
- AGENTS.md: updated pointers/workflow only; architectural rules unchanged.
- Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside
  functions, unsupported math.simplify/differentiate).
2026-11-20 10:56:26 +08:00

420 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Marqdo Plugin ABI v1
Status: **frozen ABI v1.0** (`MQ_ABI_MAJOR=1`, `MQ_ABI_MINOR=0`). The canonical
C header is `include/marqdo_plugin_abi_v1.h`; the Rust mirrors are in
`crates/marqdo-plugin-abi/src/abi.rs`; the plugin-side SDK is
`crates/marqdo-plugin-sdk`.
> **A C ABI is not a sandbox.** A native plugin is loaded into the host
> process. It can read host memory, call arbitrary syscalls, bypass the
> capability model and crash the host. Loading requires an explicit trust
> decision. The isolated worker transport described by the blueprint is **not
> implemented** in this build.
## 1. Independent versions
Three versions exist and are never conflated:
| Version | Current | Where |
|---|---|---|
| Marqdo language/semantics | M0.5, `spec/*` | `docs/DEVELOPMENT_BLUEPRINT.md`, specs |
| Plugin ABI major.minor | `1.0` | `MQ_ABI_MAJOR`/`MQ_ABI_MINOR`, `marqdo_plugin_abi::ABI_MAJOR/ABI_MINOR` |
| Plugin package SemVer | e.g. `0.1.0` | `plugin.yaml` `plugin.version` |
The JSON wire protocol has its own `wire_version = 1`
(`spec/value-wire.md`); it is not the ABI major version.
## 2. Stable-layout types
The header defines only fixed-width integers, `size_t`-based length+pointer
buffers, opaque host-owned handles and `extern "C"` function pointers. Rust
`String`/`Vec`/trait objects/`Result`/panics, C++ exceptions, Go objects and
raw host pointers never cross the boundary.
On a 64-bit target (`include/marqdo_plugin_abi_v1.h` static assertions and
`crates/marqdo-plugin-abi/src/abi.rs` compile-time assertions):
| Type | Size | Align |
|---|---|---|
| `MqBytes` | 16 | 8 |
| `MqOwnedBytes` | 16 | 8 |
| `MqPluginApi` | 32 | 8 |
| `MqHostApi` | 56 | 8 |
| `MqPluginInvoke` (fn ptr) | 8 | — |
On a 32-bit target the Rust side asserts `MqBytes`/`MqOwnedBytes` = 8,
`MqPluginApi` = 20 and `MqHostApi` = 28. The layout uses `size_t`, so an
artifact is only loadable by a host built for the **same target triple, pointer
width and byte order**. `(NULL, 0)` is the only empty buffer; a non-empty
buffer with a NULL pointer is a protocol error.
### 2.1 Ownership and lifetimes
| Buffer | Producer | Lifetime / free rule |
|---|---|---|
| `args_wire`, `export_name`, `signature_wire`, capability `request_wire`, `plugin_metadata_wire` (as seen by host) | caller passing it | borrowed for the duration of the call; consumer must copy and must not retain or free |
| `result_wire`, `error_wire`, capability `reply_wire` | the producing side | must be allocated with `host->alloc(host->host_data, size)` and is released with `host->free(host->host_data, data)` by the consumer after copying; never `malloc`/`new`/GC memory |
| `plugin_metadata_wire` (as returned by plugin) | plugin static storage | valid only during `mq_plugin_entry_v1`; host copies it immediately |
`MqCallContext` is opaque, host-owned and valid only for the invocation it was
minted for. Plugins must not dereference, cache or copy it.
### 2.2 Size limits (enforced by host / SDK)
| Constant | Bytes |
|---|---|
| `MQ_MAX_EXPORT_NAME_BYTES` | 256 |
| `MQ_MAX_SIGNATURE_WIRE_BYTES` | 4096 |
| `MQ_MAX_METADATA_WIRE_BYTES` | 65536 |
| `MQ_MAX_REQUEST_WIRE_BYTES` | 16 MiB (16,777,216) |
| `MQ_MAX_RESULT_WIRE_BYTES` | 64 MiB (67,108,864) |
| `MQ_MAX_ERROR_WIRE_BYTES` | 1 MiB (1,048,576) |
| `MQ_MAX_CAPABILITY_REPLY_BYTES` | 16 MiB |
Wire JSON decoding additionally applies the default `WireLimits` from
`spec/value-wire.md` (depth 64, 1,000,000 items, 16 MiB strings). Oversized
payloads are rejected, never truncated and reparsed.
## 3. Exported entry point and tables
Required symbol: `mq_plugin_entry_v1` (`MQ_PLUGIN_ENTRY_SYMBOL`):
```c
int32_t mq_plugin_entry_v1(const MqHostApi *host, MqPluginApi *plugin);
```
Called exactly once, by the loading thread:
* the host supplies a valid `MqHostApi` table and a zeroed `MqPluginApi` whose
`struct_size = sizeof(MqPluginApi)`;
* the plugin must validate the host table, fill `plugin.abi_major`,
`plugin.abi_minor`, `plugin.plugin_metadata_wire` and register every export
via `host->register_function`;
* it returns `MQ_OK` only after all exports are registered; on non-zero the
host ignores the plugin table and fails the load.
`MQ_OK` and error codes (stable integers):
| Code | Name | Meaning |
|---|---|---|
| 0 | `MQ_OK` | success |
| 1 | `MQ_ERR_INVALID_ARGUMENT` | NULL pointer, bad pointer/length pair |
| 2 | `MQ_ERR_ABI_MISMATCH` | ABI major mismatch or incompatible `struct_size` |
| 3 | `MQ_ERR_REGISTRATION` | an export could not be registered |
| 4 | `MQ_ERR_UNSUPPORTED` | operation/type/capability unsupported |
| 5 | `MQ_ERR_OUT_OF_MEMORY` | host allocation failed |
| 6 | `MQ_ERR_BAD_WIRE` | payload is not valid `wire_version` 1 JSON |
| 7 | `MQ_ERR_INTERNAL` | unexpected failure, including caught panic |
| 8 | `MQ_ERR_CAPABILITY` | capability request denied/failed |
### 3.1 Version negotiation
Host side (`negotiate` in `crates/marqdo-plugin-abi/src/abi.rs`):
* `plugin.abi_major != 1` → load fails with `abi_major_mismatch`;
* `plugin.abi_minor` greater than the host minor is accepted but **clamped** to
the host minor (currently 0); only features defined at that minor may be
used;
* the plugin must reject a host table with `host == NULL`,
`host->struct_size < sizeof(MqHostApi)`, `host->abi_major != 1` or NULL
alloc/free/register_function/request_capability pointers.
`struct_size` is the extension mechanism: fields may only be appended in a
later ABI minor, never reordered or resized; a consumer only reads a field when
`struct_size` covers it.
### 3.2 Threading and lifetime
v1 is single-threaded and synchronous: entry is called once; invocations and
host callbacks for one plugin instance run on that thread with at most one call
in flight. A plugin must not start threads that call back into `MqHostApi`.
v1 has no shutdown callback and no hot unload/reload; the `LoadedPlugin`
wrapper unloads only when dropped, and the docs require one instance per
artifact path (ABI v1 plugins may keep the host table in a process-global).
No panic/exception/longjmp may cross the boundary; the SDK catches Rust panics
and returns `MQ_ERR_INTERNAL` with a structured error envelope.
## 4. `plugin.yaml`
```yaml
plugin:
name: demo.solver
version: 0.1.0
abi: "^1.0"
kind: native
exports:
- name: solver.add
signature: "($a: Int, $b: Int) -> Int"
requires:
- math.compute
artifacts:
- target: x86_64-unknown-linux-gnu
file: solver.so
sha256: "<64 lowercase hex>"
```
Validation (`crates/marqdo-plugin-abi/src/manifest.rs::PluginManifest::validate`):
* `name`: dotted segments, each starting with a lowercase ASCII letter and
containing only lowercase letters, digits, `-`/`_`; ≤ 256 bytes; package
names starting with `marqdo.` or `builtin.` are reserved;
* `version`: semver-ish `major.minor.patch` with optional `-prerelease` and
`+build`; numeric components have no leading zeros; not a SemVer range
resolver;
* `abi`: caret-style requirement `^1`, `^1.0`, `1` or `1.0`. Caret semantics:
`^M.m` means `>=M.m, <M+1.0`; the `0.x` special case pins the minor. It must
be compatible with the host's `1.0`;
* `kind`: `native`, `worker` or `wasm`; only `native` is loadable in-process in
this build;
* `exports`: at least one, at most 256, unique, namespaced (must contain `.`),
no reserved prefix (`marqdo.`, `builtin.`, `core.`, `std.`, `host.`),
non-empty signature ≤ 4096 bytes, signature starts with `(` and contains
`->`, no control characters;
* `requires`: at most 64, unique names from the whitelist `log`,
`math.compute`, `fs.read`, `fs.write`, `net.request`, `clock`, `random`,
`artifact.record`; this is a declaration, not a grant;
* `artifacts`: 1–32, one per target triple; target parses as a 3–4 component
triple (lowercase, alphanumeric/underscore components); `file` is a
non-empty relative path ≤ 256 bytes with no absolute/`..` escape; `sha256` is
exactly 64 hex characters.
Manifest parsing/validation never executes plugin code and performs no
filesystem access (artifact verification is a separate step).
## 5. Artifact selection and verification
* `HostTarget::current()` derives a conventional triple from
`std::env::consts` (`x86_64-unknown-linux-gnu`, `aarch64-apple-darwin`,
`x86_64-pc-windows-msvc`, ...).
* `resolve_artifact` requires **exact triple equality**; pointer width and byte
order follow from the triple. No fallback to another architecture.
* `verify_artifact` resolves the path below the manifest directory and compares
the file's SHA-256 with the manifest (case-insensitive hex). A missing file
is `missing_artifact`; a digest mismatch is `hash_mismatch`.
* Verification proves the artifact is the file described by the manifest; it
proves nothing about trust or authorization.
## 6. Loader lifecycle (host side)
`crates/marqdo-plugin-abi/src/loader.rs::LoadedPlugin::open_with`:
1. `manifest.validate()`;
2. require `kind == native` (otherwise `plugin_kind_not_loadable_in_process`);
3. resolve artifact for the host target and verify its sha256;
4. `libloading::Library::new` (a trust decision, not a sandbox);
5. build the host API table and `HostState`, initialize `MqPluginApi`;
6. call `mq_plugin_entry_v1` once; non-zero → structured load error;
7. require `plugin_api.struct_size >= sizeof(MqPluginApi)`;
8. negotiate ABI major/minor;
9. copy `plugin_metadata_wire` and require metadata `name`/`version` to match
the manifest exactly;
10. take registered exports and cross-check against the manifest: duplicate
registrations, missing declared exports, signature mismatches and
undeclared exports all fail the load;
11. keep the library mapped for the lifetime of `LoadedPlugin`.
Invocation (`LoadedPlugin::invoke`): validate the request envelope, check the
request size, find the registered export, mint a `MqCallContext`, call the
plugin's `MqPluginInvoke`, copy and free both output buffers, then enforce the
envelope contract:
* success (`MQ_OK`): `error_wire` must be empty; `result_wire` must be a
non-empty `wire_version` 1 success envelope; the typed value is optional
(absent `value` means `Unit`);
* failure: `error_wire` must hold a `wire_version` 1 error envelope; a missing
or invalid error envelope still fails the call (structured
`plugin_call_failed`).
Capability requests from the plugin go through
`MqHostApi::request_capability` into the host's `CapabilityHandler`, on the
same thread, one at a time. The handler receives the capability name and
request bytes and returns reply bytes or an error. When no handler is
installed (the CLI's case) **every request is denied** (`MQ_ERR_CAPABILITY`).
## 7. Wire envelopes
All payloads are UTF-8 JSON with `wire_version = 1`. Values inside `args` and
`value` use the type-tagged encoding of `spec/value-wire.md`.
Request:
```json
{
"wire_version": 1,
"kind": "request",
"function": "solver.add",
"args": {
"a": {"type": "int", "decimal": "20"},
"b": {"type": "int", "decimal": "22"}
},
"context": {"trace_id": "optional", "capability_handle": "opaque-and-scoped"}
}
```
Response (success):
```json
{"wire_version":1,"kind":"response","value":{"type":"int","decimal":"42"}}
```
A success response with neither `value` nor `error` means `Unit`. A response
with both is rejected (`ambiguous_response`). Error:
```json
{"wire_version":1,"kind":"response","error":{
"kind":"invoke","code":"missing_argument",
"message":"required argument `a` is missing","recoverable":true
}}
```
Envelope limits: at most `MAX_ARGS = 1024` request arguments; request ≤ 16 MiB,
response ≤ 64 MiB, error ≤ 1 MiB. `context` carries no secrets and no
reconstructable OS handles. `signature_wire` is
`{"signature":"<signature string>"}`; `plugin_metadata_wire` is
`{"name":"<package name>","version":"<semver>"}` (extra fields ignored).
## 8. Error model
Host-side errors are `PluginError { kind, code, message, detail?, recoverable }`
with stable `kind` values: `manifest`, `abi`, `artifact`, `load`, `wire`,
`capability`, `invoke`, `unsupported`. Examples: `abi_major_mismatch`,
`no_matching_artifact`, `hash_mismatch`, `missing_entry_symbol`,
`metadata_name_mismatch`, `export_signature_mismatch`,
`unsupported_wire_version`. `message`/`detail` must never contain secrets.
They are serializable and are used as the wire error body.
When a plugin is used from Marqdo code, host errors become structured
`RuntimeError`s (`crates/marqdo-host/src/host.rs::map_plugin_error`):
| Plugin error kind | Runtime code |
|---|---|
| `abi` | `plugin.abi_mismatch` |
| `capability` | `permission.denied` |
| `wire` | `runtime.type` |
| `unsupported` | `kernel.unavailable` |
| `manifest`/`artifact`/`load`/`invoke` | `plugin.failure` |
## 9. Using a plugin from `.mq.md`
This build supports explicitly trusted **in-process native** plugins only.
```bash
# 1. Build the C example and refresh its sha256 in plugin.yaml
./examples/plugin-demo/build.sh
# 2. Inspect without loading
marqdo plugin inspect examples/plugin-demo
# 3. Load with an explicit trust decision and a capability grant, then run
marqdo run examples/plugin-demo/plugin-call.mq.md \
--plugin examples/plugin-demo \
--trust-native-plugin \
--allow-plugin demo.solver
```
Flags (`run` and `call`): `--plugin <manifest-or-dir>` (repeatable),
`--trust-native-plugin`, `--allow-plugin <package-name>`. `--allow-plugin`
matches the manifest `plugin.name` (for example `demo.solver`), not the call
namespace (`solver`).
Document syntax (`examples/plugin-demo/plugin-call.mq.md`):
```markdown
## main
**结果 = solver.add(a=20, b=22)**
> 打印(结果)
```
Semantics:
* the call namespace is derived from the **first** manifest export prefix
(`solver.add` → namespace `solver`); a manifest whose exports use several
prefixes registers only the first namespace, so exports under other prefixes
are not reachable in this build;
* calls accept **named arguments only**; positional arguments are a runtime
type error ("plugin calls require named arguments");
* each call becomes `HostOp::PluginInvoke` with a typed wire request; the
plugin's typed response becomes a Marqdo value (`Unit` when absent);
* the capability grant is re-checked on every call, not only at load;
* plugin namespace calls are **not statically validated**: `check` passes a
document that calls a namespace no plugin provides; at runtime the call fails
with `symbol.unknown` ("no plugin namespace ... is loaded") unless
`--plugin` loaded it;
* plugin host-capability requests are denied because the CLI installs no
capability handler (`LoadOptions::default()`);
* `--mode strict` refuses to run before any plugin is loaded; `--mode pure`
cannot grant `--allow-plugin`, so loading fails with `permission.denied`.
Trust boundary: `--trust-native-plugin` is an explicit unsafe decision; the
process shares its address space with the plugin. In-process load is the only
transport implemented — there is no worker process, Wasm component, hot
unload, or plugin capability bridge. A native plugin crash is a host crash.
## 10. SDK usage (`crates/marqdo-plugin-sdk`)
The SDK hides the raw C layout and exports the entry symbol with
`mq_plugin_entry!`. A complete Rust plugin crate:
```rust
use marqdo_plugin_sdk::{mq_plugin_entry, CallContext, PluginError, Value};
fn add(_ctx: &mut CallContext, args: &[u8]) -> Result<Vec<u8>, PluginError> {
let request = marqdo_plugin_sdk::wire::decode_request(args)?;
let values = request.typed_args()?;
let a = marqdo_plugin_sdk::wire::int_arg(&values, "a")?;
let b = marqdo_plugin_sdk::wire::int_arg(&values, "b")?;
marqdo_plugin_sdk::wire::success(Some(Value::Int((a + b).into())))
}
mq_plugin_entry! {
name: "demo.solver",
version: "0.1.0",
exports: {
"solver.add" => "($a: Int, $b: Int) -> Int" => add,
}
}
```
The macro validates the host table, reports the plugin ABI version and metadata,
registers every export and returns `MQ_OK` only if all registrations succeed.
Handlers receive a `&mut CallContext`; use
`CallContext::capability_request(name, request_bytes)` for an already-authorized
host capability (the CLI grants none today), and
`CallContext::log(level, message)` as a convenience. Panics are caught at the
ABI boundary and converted to a structured error. The SDK depends only on
`marqdo-plugin-abi`, never on host-only crates or the interpreter.
## 11. C demo (`examples/plugin-demo`)
`examples/plugin-demo/solver.c` is a dependency-free C implementation of one
export (`solver.add`, `Int + Int`) using a minimal JSON scanner; it validates
the host table, reports metadata and registers the export. `build.sh` compiles
it (`cc -shared -fPIC -I include`) and rewrites the `sha256` in `plugin.yaml`
(the checked-in manifest uses a zero placeholder until you build). The
`examples/plugin-demo/README.md` walks through the security model, build,
inspect and call flow.
The integration test `crates/marqdo-plugin-abi/tests/loader_unix.rs` compiles
this exact `solver.c` with `cc`, loads it with `libloading`, round-trips a wire
request, and rebuilds it with `-DMQ_SOLVER_REPORT_ABI_MAJOR=2` to prove that a
wrong ABI major fails the load. It also covers a missing entry symbol, missing
artifact, hash mismatch, malformed manifest and target mismatch. The tests skip
when no C compiler is available.
## 12. Not implemented / explicit limitations
| Item | Status |
|---|---|
| Isolated plugin worker transport (length-prefixed IPC) | Not implemented; `PlatformCapabilities::worker_processes = false`. |
| Wasm Component + WIT | Not implemented. |
| Hot unload/reload | Not supported by ABI v1 (no shutdown callback). |
| Plugin capability bridge from the CLI | Not implemented; CLI supplies no handler, so all plugin host-capability requests are denied. |
| Plugin installation by package manager | Not implemented; `marqdo.yaml` `plugins:` is declarative only. |
| Static validation of plugin namespaces/exports/signatures against loaded plugins | Not implemented; errors are runtime-only. |
| Multiple namespaces per plugin | Not implemented; single namespace from the first export prefix. |
| Positional plugin arguments | Not supported; named only. |
| Thread-safe / concurrent invocation | Not supported (single-threaded v1 contract). |
| Old `marqdo_abi.h` v2 compatibility | Not implemented and not claimed; a separate adapter would be required. |