Files
marqdo-cli/spec/value-wire.md
T
cflmy 0116160399 docs: formal specs, ADRs, STATUS matrix, contributor guide and honest README
Complete the documentation set for the M0.5 toolchain, grounded in the code:

- spec/references.md: import/reference/call edges, identities, loader rules,
  anchors, `ref` resolution/ambiguity, `check --refs` validation.
- spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal
  string rule, special floats, limits and exact float round-trip.
- spec/security.md: modes (strict refuses without OS isolation), capabilities,
  path rules, budgets, tracing, foreign process caveats, plugin trust.
- spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync,
  unsupported registry/git/environment work.
- spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact
  rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the
  implemented CLI plugin-invocation flow with its limitations.
- spec/math.md and spec/foreign.md: the implemented math subset and the
  foreign request/response/timeout/capability contract.
- docs/adr/0001..0007: accepted decisions for the span foundation, inert
  executables outside function bodies, ref ambiguity (S-07), strict refusal,
  AST normalization, CLI contract and BigInt/typed wire (S-08).
- docs/contributor-guide.md: repo map, workflow, fixture/golden updates,
  quality gate, ADR/commit policy.
- docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18
  Alpha checklist (conclusion: not yet Alpha-complete), including known
  divergences rather than overclaims.
- README.md rewritten from "design-only starter" to the working implementation
  with quickstart, implemented phases, deliberate limits and doc index.
- AGENTS.md: updated pointers/workflow only; architectural rules unchanged.
- Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside
  functions, unsupported math.simplify/differentiate).
2026-11-20 10:56:26 +08:00

162 lines
9.2 KiB
Markdown

# Marqdo Value Wire Format v1
Status: **frozen for M0.5**. Changes require an ADR, a `wire_version` bump and
cross-ABI compatibility tests.
Canonical implementation: `crates/marqdo-value/src/wire.rs`
(`to_wire`, `from_wire`, `from_wire_with_limits`, `WireLimits`, `WireError`).
The plugin envelope that *carries* these values is defined in
`crates/marqdo-plugin-abi/src/wire.rs` and documented in
[`plugin-abi.md`](plugin-abi.md); the foreign-kernel stdin envelope is
documented in [`foreign.md`](foreign.md) and implemented in
`crates/marqdo-foreign/src/lib.rs`.
## 1. Why a typed wire format
JSON has one numeric type. Marqdo has arbitrary-precision `Int`, IEEE-754
`Float` (including `NaN` and both infinities) and 13 distinct runtime value
types (`crates/marqdo-value/src/value.rs`). Encoding an `Int` as a JSON number
would silently lose digits above 2^53 and would make `Int(1)` and
`Float(1.0)` indistinguishable after a round trip. The wire format is
therefore **fully type-tagged** and is not plain JSON.
The rules that follow from this:
1. Every value is a JSON **object** with a mandatory `"type"` string.
2. **Integers never travel as JSON numbers.** `Int` is
`{"type":"int","decimal":"<base-10 string>"}`, which preserves every digit,
sign and leading-zero-free canonical form.
3. `Float` uses a JSON number only for finite values; `NaN`, `Infinity` and
`-Infinity` use an explicit `"special"` tag.
4. Unknown tags, missing fields, wrong JSON types and invalid payloads are
structured decode errors, never silent coercions.
JSON numbers still appear in two non-value positions: `plugin_handle.id` (a
host generation counter) and `span.start` / `span.end` (byte offsets). Neither
is a user arithmetic value.
## 2. Version and entry points
* `WIRE_VERSION: u32 = 1` (`marqdo_value::WIRE_VERSION`).
* `to_wire(&Value) -> serde_json::Value` encodes.
* `from_wire(&Json) -> Result<Value, WireError>` decodes with
`WireLimits::default()`.
* `from_wire_with_limits(&Json, WireLimits) -> Result<Value, WireError>`
decodes with explicit limits.
* Decoding is recursive and never panics on untrusted input; every failure is
a `WireError`:
* `UnknownType(String)` — missing/non-string/unknown `"type"`;
* `MissingField(&'static str)` — required field absent;
* `Invalid(String)` — wrong JSON type, invalid decimal, bad table shape,
unknown float `special`, non-object value;
* `LimitExceeded(&'static str)` — nesting depth, list/map length, string or
map-key length.
`WireError` implements `Display` but has no stable numeric code of its own.
Boundary crates wrap it in their own stable codes: the plugin layer uses
`invalid_argument_wire` / `invalid_result_wire`
(`crates/marqdo-plugin-abi/src/wire.rs`), and the foreign layer maps it to
`Protocol` (`crates/marqdo-foreign/src/lib.rs`).
## 3. Value encodings
The table is normative and mirrors `encode`/`decode` in
`crates/marqdo-value/src/wire.rs`.
| Marqdo type | Wire JSON | Notes |
|---|---|---|
| `Unit` | `{"type":"unit"}` | No result; distinct from `None`. |
| `None` | `{"type":"none"}` | Explicit empty value. |
| `Bool` | `{"type":"bool","value":true}` | `value` must be a JSON boolean. |
| `Int` | `{"type":"int","decimal":"-123..."}` | Decimal string; decoded with `num_bigint::BigInt::from_str`. No `+`, no separators, no hex. `-0` decodes to `0` because `BigInt` has no signed zero. |
| `Float` (finite) | `{"type":"float","value":1.5}` | JSON number decoded with `as_f64()`. `-0.0` round-trips as a JSON negative-zero number and decodes to `Float(-0.0)`. |
| `Float` (`NaN`) | `{"type":"float","special":"nan"}` | Never a JSON number. |
| `Float` (`+∞`) | `{"type":"float","special":"infinity"}` | |
| `Float` (`−∞`) | `{"type":"float","special":"-infinity"}` | |
| `Text` | `{"type":"text","value":"..."}` | UTF-8 JSON string. |
| `List` | `{"type":"list","items":[<value>, ...]}` | Order preserved. |
| `Map` | `{"type":"map","entries":{"key":<value>, ...}}` | Decoding rebuilds a `BTreeMap`, so key order is canonical (sorted) after a round trip. Duplicate keys cannot occur in a JSON object. |
| `Table` | `{"type":"table","header":["a","b"],"rows":[[<value>,<value>], ...]}` | Header cells must be JSON strings; every row must have exactly `header.len()` cells or decoding fails. |
| `Formula` | `{"type":"formula","latex":"...","display":false}` plus optional `"unit"` (string) and `"span":{"start":N,"end":M}` | `display` defaults to `false` on decode when absent. The formula is inert data; this encoding never evaluates it. |
| `Code` | `{"type":"code","language":"python","source":"...","content_hash":"<sha256 hex>"}` plus optional `"unit"` and `"span"` | `language` and `content_hash` default to `""` on decode when absent; `source` is required. |
| `FunctionRef` | `{"type":"function_ref","module":"index.mq.md","name":"main"}` | Both fields required strings. |
| `PluginHandle` | `{"type":"plugin_handle","plugin":"demo.solver","export":"solver.add","id":1}` | `id` must be a non-negative JSON integer (`as_u64`). |
Optional fields (`unit`, `span`, `display`, `language`, `content_hash`) are
emitted only when present on the value. Encoders always emit `display` for
`Formula` and `language`/`content_hash`/`source` for `Code`.
### 3.1 Round-trip guarantees
For the values covered by `wire::tests::scalar_roundtrip`,
`from_wire(to_wire(v)) == v`. Exact guarantees:
* all `Int` digits survive (`big_integers_keep_all_digits`);
* `NaN`/`±Infinity` survive with their sign/tag (`special_floats_are_tagged`);
* `Unit` and `None` never collapse into each other;
* `Map` iteration order after decoding is sorted, which is the canonical order
used by `Value::display`/`repr`.
Finite `Float` values are encoded as JSON numbers. The workspace enables
`serde_json`'s `float_roundtrip` feature, so a finite binary64 value survives an
encode/decode cycle exactly (verified by the Python `math.pi * 9` semantic
fixture: `28.274333882308138` stays `28.274333882308138`). `NaN`/infinities are
tagged and exact. Use `Text` when a decimal string must be exchanged verbatim
rather than as a binary64 float.
## 4. Limits
`WireLimits::default()` (`crates/marqdo-value/src/wire.rs`):
| Limit | Default | Applies to |
|---|---|---|
| `max_depth` | 64 | Nesting of *values*: the top-level object is depth 0, every child decoded through `list.items`, `map.entries`, `table.rows` or `table.header` cells is depth+1. A value at depth > 64 is rejected with `LimitExceeded("nesting depth")`. |
| `max_items` | 1,000,000 | `list.items` length and `map.entries` key count. |
| `max_string_bytes` | 16 MiB | `text.value` byte length and map key byte length. |
### 4.1 Known gaps in the current decoder
The following are **not** bounded by `WireLimits` in
`crates/marqdo-value/src/wire.rs` today and must be treated as open items
(they are also bounded indirectly at the ABI layer by envelope size limits):
* `table.header` length, `table.rows` length and the number of columns are not
checked against `max_items` (only row/header width equality is checked);
* table header cell strings are not checked against `max_string_bytes`;
* `formula.latex`, `code.source`, `function_ref.module`/`name`,
`plugin_handle.plugin`/`export` are not checked against
`max_string_bytes`;
* a `plugin_handle` decoded from wire data is a structurally reconstructed
`Value::PluginHandle`; this build does not carry an unforgeable capability
token. Host code must therefore treat a plugin handle as opaque *intent* and
re-authorize any operation, not as proof of prior authorization. The header
comment "cannot be forged from wire data" is aspirational for this point and
should be fixed or reworded in a future ADR.
* JSON itself may contain duplicate object keys; `serde_json` keeps the last
one, so `map.entries` with duplicate keys is not rejected as an error.
## 5. Where the format is used
| Boundary | Envelope | Value positions |
|---|---|---|
| Plugin ABI v1 request/response | `{"wire_version":1,"kind":"request"/"response",...}` (`crates/marqdo-plugin-abi/src/wire.rs`) | `request.args` is an object of name → typed value; `response.value` is one typed value. |
| Plugin host capability request/reply | opaque bytes, shape per capability | defined by the capability, not by this file. |
| Foreign kernel stdin | `{"wire_version":1,"kind":"request","input":<typed value or null>}` (`crates/marqdo-foreign/src/lib.rs`) | `input` is a `to_wire(value)` result, or JSON `null` when absent. |
| Foreign typed return (`foreign.call`) | stdout must be exactly one typed value | decoded with `from_wire` and default limits. |
The `--json` output of `run`/`call` embeds `"value": to_wire(outcome.value)`
(`crates/cli/src/commands.rs`), so CLI JSON consumers see the same typed
representation.
## 6. Compatibility rules
* `wire_version` is part of the frozen contract. A decoder rejects a different
version rather than guessing.
* Adding a new `"type"` tag requires a spec change and an ADR; removing or
renaming a tag is a breaking change.
* The plugin envelope independently carries `wire_version`; do not assume the
C ABI major version and `wire_version` are the same number
(see `spec/plugin-abi.md` §2).
* All payloads are UTF-8 JSON. Endianness is irrelevant for the JSON layer,
but the C ABI structs that carry these bytes are target-dependent.