Files
cflmy 92bfbf2568 feat(rfc-cm): baseline P0-P3 compute machine, stdlib and resource slices
Implements the first vertical slices of RFC-CM-0001/0002 on top of a113d89:

- P1: opaque Memory regions (64 KiB pages, budgets, grow-only), byte
  read/write, Memory-index place assignment, wire/host-boundary rejection.
- CM-A04/A05: value-semantics List/Map/Text/Table primitives and the
  recoverable Result map convention.
- CM-A06: opaque Resource handles, resource.open/read/write/close, stdout
  write, granted project-file read and buffered write, strict fail-closed.
- CM-A07 Compat-A: explicit .mq.md format/utf8/io chain matching builtin
  print for the covered value types; builtin remains default.
- P2 fixtures: allocators, list/tree/graph/hash, integer codecs, UTF-8
  encode/decode, CSV/JSON subsets, table filter/sort/join, bytecode VM.
- ADRs 0022-0027, spec updates, evidence reports and 68 semantic goldens.

Gate: cargo fmt --check, clippy -D warnings, 407 tests passed.
2026-11-20 21:16:54 +08:00

10 KiB

Marqdo Value Wire Format v1

Status: frozen for M0.5; extended by ADR-0022 (non-transferable Memory). Changes require an ADR, a wire_version bump and cross-ABI compatibility tests. Adding a rejected tag is not a version bump; accepting a new value shape is.

Canonical implementation: crates/marqdo-value/src/wire.rs (to_wire, from_wire, from_wire_with_limits, WireLimits, WireError). The plugin envelope that carries these values is defined in crates/marqdo-plugin-abi/src/wire.rs and documented in plugin-abi.md; the foreign-kernel stdin envelope is documented in foreign.md and implemented in crates/marqdo-foreign/src/lib.rs.

1. Why a typed wire format

JSON has one numeric type. Marqdo has arbitrary-precision Int, IEEE-754 Float (including NaN and both infinities) and 15 distinct runtime value types (crates/marqdo-value/src/value.rs). Encoding an Int as a JSON number would silently lose digits above 2^53 and would make Int(1) and Float(1.0) indistinguishable after a round trip. The wire format is therefore fully type-tagged and is not plain JSON.

Addressable Memory and host Resource handles are deliberately not transferable. to_wire emits {"type":"memory"} only so that an accidental transfer is visible; decoding that tag is always WireError::Invalid, and the host boundary rejects any argument containing a live handle before an ABI call. Copying bytes requires an explicit standard-library snapshot to a managed value, not serializing a handle (ADR-0022).

The rules that follow from this:

  1. Every value is a JSON object with a mandatory "type" string.
  2. Integers never travel as JSON numbers. Int is {"type":"int","decimal":"<base-10 string>"}, which preserves every digit, sign and leading-zero-free canonical form.
  3. Float uses a JSON number only for finite values; NaN, Infinity and -Infinity use an explicit "special" tag.
  4. Unknown tags, missing fields, wrong JSON types and invalid payloads are structured decode errors, never silent coercions.

JSON numbers still appear in two non-value positions: plugin_handle.id (a host generation counter) and span.start / span.end (byte offsets). Neither is a user arithmetic value.

2. Version and entry points

  • WIRE_VERSION: u32 = 1 (marqdo_value::WIRE_VERSION).
  • to_wire(&Value) -> serde_json::Value encodes.
  • from_wire(&Json) -> Result<Value, WireError> decodes with WireLimits::default().
  • from_wire_with_limits(&Json, WireLimits) -> Result<Value, WireError> decodes with explicit limits.
  • Decoding is recursive and never panics on untrusted input; every failure is a WireError:
    • UnknownType(String) — missing/non-string/unknown "type";
    • MissingField(&'static str) — required field absent;
    • Invalid(String) — wrong JSON type, invalid decimal, bad table shape, unknown float special, non-object value;
    • LimitExceeded(&'static str) — nesting depth, list/map length, string or map-key length.

WireError implements Display but has no stable numeric code of its own. Boundary crates wrap it in their own stable codes: the plugin layer uses invalid_argument_wire / invalid_result_wire (crates/marqdo-plugin-abi/src/wire.rs), and the foreign layer maps it to Protocol (crates/marqdo-foreign/src/lib.rs).

3. Value encodings

The table is normative and mirrors encode/decode in crates/marqdo-value/src/wire.rs.

Marqdo type Wire JSON Notes
Unit {"type":"unit"} No result; distinct from None.
None {"type":"none"} Explicit empty value.
Bool {"type":"bool","value":true} value must be a JSON boolean.
Int {"type":"int","decimal":"-123..."} Decimal string; decoded with num_bigint::BigInt::from_str. No +, no separators, no hex. -0 decodes to 0 because BigInt has no signed zero.
Float (finite) {"type":"float","value":1.5} JSON number decoded with as_f64(). -0.0 round-trips as a JSON negative-zero number and decodes to Float(-0.0).
Float (NaN) {"type":"float","special":"nan"} Never a JSON number.
Float (+∞) {"type":"float","special":"infinity"}
Float (−∞) {"type":"float","special":"-infinity"}
Text {"type":"text","value":"..."} UTF-8 JSON string.
List {"type":"list","items":[<value>, ...]} Order preserved.
Map {"type":"map","entries":{"key":<value>, ...}} Decoding rebuilds a BTreeMap, so key order is canonical (sorted) after a round trip. Duplicate keys cannot occur in a JSON object.
Table {"type":"table","header":["a","b"],"rows":[[<value>,<value>], ...]} Header cells must be JSON strings; every row must have exactly header.len() cells or decoding fails.
Formula {"type":"formula","latex":"...","display":false} plus optional "unit" (string) and "span":{"start":N,"end":M} display defaults to false on decode when absent. The formula is inert data; this encoding never evaluates it.
Code {"type":"code","language":"python","source":"...","content_hash":"<sha256 hex>"} plus optional "unit" and "span" language and content_hash default to "" on decode when absent; source is required.
FunctionRef {"type":"function_ref","module":"index.mq.md","name":"main"} Both fields required strings.
PluginHandle {"type":"plugin_handle","plugin":"demo.solver","export":"solver.add","id":1} id must be a non-negative JSON integer (as_u64).
Memory {"type":"memory"} Never decodable. No handle, address or bytes are encoded; a round trip is intentionally impossible (wire::tests::memory_handles_are_not_transferable).
Resource {"type":"resource"} Never decodable. Host-private stream handle; from_wire rejects the tag (resource_handles_are_not_transferable, ADR-0025).

Optional fields (unit, span, display, language, content_hash) are emitted only when present on the value. Encoders always emit display for Formula and language/content_hash/source for Code.

3.1 Round-trip guarantees

For the values covered by wire::tests::scalar_roundtrip, from_wire(to_wire(v)) == v. Exact guarantees:

  • all Int digits survive (big_integers_keep_all_digits);
  • NaN/±Infinity survive with their sign/tag (special_floats_are_tagged);
  • Unit and None never collapse into each other;
  • Map iteration order after decoding is sorted, which is the canonical order used by Value::display/repr.

Finite Float values are encoded as JSON numbers. The workspace enables serde_json's float_roundtrip feature, so a finite binary64 value survives an encode/decode cycle exactly (verified by the Python math.pi * 9 semantic fixture: 28.274333882308138 stays 28.274333882308138). NaN/infinities are tagged and exact. Use Text when a decimal string must be exchanged verbatim rather than as a binary64 float.

4. Limits

WireLimits::default() (crates/marqdo-value/src/wire.rs):

Limit Default Applies to
max_depth 64 Nesting of values: the top-level object is depth 0, every child decoded through list.items, map.entries, table.rows or table.header cells is depth+1. A value at depth > 64 is rejected with LimitExceeded("nesting depth").
max_items 1,000,000 list.items length and map.entries key count.
max_string_bytes 16 MiB text.value byte length and map key byte length.

4.1 Known gaps in the current decoder

The following are not bounded by WireLimits in crates/marqdo-value/src/wire.rs today and must be treated as open items (they are also bounded indirectly at the ABI layer by envelope size limits):

  • table.header length, table.rows length and the number of columns are not checked against max_items (only row/header width equality is checked);
  • table header cell strings are not checked against max_string_bytes;
  • formula.latex, code.source, function_ref.module/name, plugin_handle.plugin/export are not checked against max_string_bytes;
  • a plugin_handle decoded from wire data is a structurally reconstructed Value::PluginHandle; it is not an authorization token. The host owns a PluginHandleTable with owner/run/generation scope (ADR-0014), and plugin or foreign output containing any handle is rejected with plugin_handle.untrusted. Host code must therefore treat a plugin handle as opaque intent and re-authorize any operation, not as proof of prior authorization. The header comment "cannot be forged from wire data" is aspirational for this point and should be fixed or reworded in a future ADR.
  • JSON itself may contain duplicate object keys; serde_json keeps the last one, so map.entries with duplicate keys is not rejected as an error.

5. Where the format is used

Boundary Envelope Value positions
Plugin ABI v1 request/response {"wire_version":1,"kind":"request"/"response",...} (crates/marqdo-plugin-abi/src/wire.rs) request.args is an object of name → typed value; response.value is one typed value.
Plugin host capability request/reply opaque bytes, shape per capability defined by the capability, not by this file.
Foreign kernel stdin {"wire_version":1,"kind":"request","input":<typed value or null>} (crates/marqdo-foreign/src/lib.rs) input is a to_wire(value) result, or JSON null when absent.
Foreign typed return (foreign.call) stdout must be exactly one typed value decoded with from_wire and default limits.

The --json output of run/call embeds "value": to_wire(outcome.value) (crates/cli/src/commands.rs), so CLI JSON consumers see the same typed representation.

6. Compatibility rules

  • wire_version is part of the frozen contract. A decoder rejects a different version rather than guessing.
  • Adding a new "type" tag requires a spec change and an ADR; removing or renaming a tag is a breaking change.
  • The plugin envelope independently carries wire_version; do not assume the C ABI major version and wire_version are the same number (see spec/plugin-abi.md §2).
  • All payloads are UTF-8 JSON. Endianness is irrelevant for the JSON layer, but the C ABI structs that carry these bytes are target-dependent.