feat(trace): Trace v2 identity, module-owned events and capacity bounds (B-16)

- bind traces to the logical project identity and the exact emitted Analysis
  digest; every span-bearing event carries module_id
- add host TraceState budgets (100k events / 32 MiB defaults), CLI
  --trace-max-events/--trace-max-bytes, terminal trace.truncated events, and a
  final serialized byte cap; budgets never change run.status
- disable semantic recording when no trace is requested
- cap worker frames at 8 MiB, align request/result/capability wire caps, make
  clean EOF distinct from protocol errors, and mark/kill a worker after a
  transport failure instead of reusing it
- add trace-v2 spec/schema, independent trace consumer, CI check and tests
- record the S2/C state in STATUS
This commit is contained in:
cflmy committed 2026-11-20 13:01:22 +08:00
1 parent 55dcb57553
commit d890686a2b
21 files changed
+732 -84

No files matched your search

+9 -6
View File
@@ -68,10 +68,10 @@ minted for. Plugins must not dereference, cache or copy it.
| `MQ_MAX_EXPORT_NAME_BYTES` | 256 |
| `MQ_MAX_SIGNATURE_WIRE_BYTES` | 4096 |
| `MQ_MAX_METADATA_WIRE_BYTES` | 65536 |
| `MQ_MAX_REQUEST_WIRE_BYTES` | 16 MiB (16,777,216) |
| `MQ_MAX_RESULT_WIRE_BYTES` | 64 MiB (67,108,864) |
| `MQ_MAX_REQUEST_WIRE_BYTES` | 8 MiB - 4096 (8,384,512) |
| `MQ_MAX_RESULT_WIRE_BYTES` | 8 MiB - 4096 (8,384,512) |
| `MQ_MAX_ERROR_WIRE_BYTES` | 1 MiB (1,048,576) |
| `MQ_MAX_CAPABILITY_REPLY_BYTES` | 16 MiB |
| `MQ_MAX_CAPABILITY_REPLY_BYTES` | 8 MiB - 4096 |
Wire JSON decoding additionally applies the default `WireLimits` from
`spec/value-wire.md` (depth 64, 1,000,000 items, 16 MiB strings). Oversized
@@ -270,8 +270,8 @@ with both is rejected (`ambiguous_response`). Error:
}}
```
Envelope limits: at most `MAX_ARGS = 1024` request arguments; request ≤ 16 MiB,
response ≤ 64 MiB, error ≤ 1 MiB. `context` carries no secrets and no
Envelope limits: at most `MAX_ARGS = 1024` request arguments; request and
response ≤ 8 MiB - 4096, error ≤ 1 MiB. `context` carries no secrets and no
reconstructable OS handles. `signature_wire` is
`{"signature":"<signature string>"}`; `plugin_metadata_wire` is
`{"name":"<package name>","version":"<semver>"}` (extra fields ignored).
@@ -364,7 +364,10 @@ full plugin capability broker for worker requests remain future work.
### 9.1 Worker frames
Frames are a 4-byte big-endian length followed by one JSON object (max 64 MiB).
Frames are a 4-byte big-endian length followed by one JSON object (max 8 MiB).
A zero/oversized length, partial header, truncated payload or invalid JSON is a
protocol error; clean EOF at a frame boundary is not. The host marks the worker
dead after a transport failure and does not reuse it.
Worker → host: `{"kind":"ready","exports":[...]}`. Host → worker:
`{"kind":"invoke","request":<wire request>}` and `{"kind":"shutdown"}`.
Worker → host responses are `{"kind":"response","response":<wire response>}` or
+86
View File
@@ -0,0 +1,86 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://marqdo.dev/schemas/trace-v2.schema.json",
"title": "Marqdo Trace v2",
"type": "object",
"required": ["schema", "schema_version", "run", "source", "limits", "truncated", "events"],
"properties": {
"schema": {"const": "marqdo.trace"},
"schema_version": {"const": 2},
"run": {
"type": "object",
"required": ["id", "entry", "status"],
"properties": {
"id": {"type": "string", "minLength": 1},
"entry": {"type": "string", "minLength": 1},
"status": {"type": "string", "minLength": 1}
},
"additionalProperties": true
},
"source": {
"type": "object",
"required": ["root_id", "analysis_digest"],
"properties": {
"root_id": {"type": "string", "pattern": "^(root:|sha256:[0-9a-f]{64}$)"},
"analysis_digest": {"type": "string", "pattern": "^sha256:[0-9a-f]{64}$"}
},
"additionalProperties": true
},
"limits": {
"type": "object",
"required": ["max_events", "max_bytes"],
"properties": {
"max_events": {"type": "integer", "minimum": 0},
"max_bytes": {"type": "integer", "minimum": 0}
},
"additionalProperties": false
},
"truncated": {
"oneOf": [
{"type": "null"},
{
"type": "object",
"required": ["reason", "limit", "written"],
"properties": {
"reason": {"enum": ["event_budget", "byte_budget"]},
"limit": {"type": "integer", "minimum": 0},
"written": {"type": "integer", "minimum": 0}
},
"additionalProperties": true
}
]
},
"events": {
"type": "array",
"items": {
"type": "object",
"required": ["seq", "kind", "module_id", "span", "detail"],
"properties": {
"seq": {"type": "integer", "minimum": 0},
"kind": {"type": "string", "minLength": 1},
"function_id": {"type": ["string", "null"]},
"module_id": {"type": ["string", "null"]},
"span": {
"oneOf": [
{"type": "null"},
{
"type": "object",
"required": ["start", "end"],
"properties": {
"start": {"type": "integer", "minimum": 0},
"end": {"type": "integer", "minimum": 0}
},
"additionalProperties": true
}
]
},
"detail": {"type": ["string", "null"]},
"operation": {"type": "string"},
"duration_micros": {"type": "integer", "minimum": 0}
},
"additionalProperties": true
}
}
},
"additionalProperties": true
}
+2 -1
View File
@@ -1,6 +1,7 @@
# Marqdo Trace v1
Status: **implemented by `marqdo run/call --trace-file <path>` (M0.6 R4).**
Status: **superseded by [Trace v2](trace-v2.md); never released, do not write
v1.** Kept as history.
## 1. Invariants
+112
View File
@@ -0,0 +1,112 @@
# Marqdo Trace v2
Status: **implemented by `marqdo run/call --trace-file <path>` (S2).**
Supersedes Trace v1; v1 was never released and is kept as history.
## 1. Invariants
* Tracing is observation, not execution. Only `run`/`call` produce a trace;
`check`, `inspect` and `tool list` never execute a document.
* Events come from the interpreter (`call.*`, `branch.eval`,
`loop.iteration`, `return`) and the host (`effect`). Both share one sequence.
* The writer is atomic; an error run still records its terminal status.
* No argument values, secrets, handles or file contents are recorded.
* Trace capacity is a bounded observation budget. Reaching it truncates the
trace; it never changes `run.status` or the program’s execution result.
Execution budgets (`--fuel`, `--timeout`, ...) remain separate and still
produce `resource.exhausted`.
## 2. Shape
```json
{
"schema": "marqdo.trace",
"schema_version": 2,
"run": {"id": "pid-nanos", "entry": "root:q@0.1.0::index.mq.md::main", "status": "ok"},
"source": {"root_id": "root:q@0.1.0", "analysis_digest": "sha256:<64 hex>"},
"limits": {"max_events": 100000, "max_bytes": 33554432},
"truncated": null,
"events": [
{"seq": 0, "kind": "call.enter", "function_id": "root:q@0.1.0::index.mq.md::main",
"module_id": "root:q@0.1.0::index.mq.md", "span": null, "detail": null},
{"seq": 1, "kind": "effect", "module_id": "dep:p@0.1.0#<digest>::index.mq.md",
"span": {"start": 40, "end": 52}, "detail": "ok", "operation": "print",
"duration_micros": 3}
]
}
```
`seq` is globally increasing. `run.status` is `ok` or a stable runtime error
code. Every span-bearing event has a `module_id`; the span is a UTF-8 byte
offset range in that module. `source` only carries project identity and the
Analysis digest; it does not identify the span owner.
`source.root_id` is the Analysis v2 `project.root_id`
(`root:<name>@<version>` or `sha256:<workspace snapshot>`).
`source.analysis_digest` is `sha256:` plus the SHA-256 of the exact Analysis
v2 bytes produced for the same project with the same projection configuration.
A consumer verifies it by hashing the Analysis file.
`truncated` is `null` or:
```json
{"reason": "event_budget", "limit": 20, "written": 20}
```
`reason` is `event_budget` or `byte_budget`. When truncation occurs, the final
event is `{"kind":"trace.truncated","detail":<reason>,"module_id":null,
"span":null,"seq":<one past the last written event>}`.
## 3. Event kinds and span owners
| kind | detail | `module_id` owner |
|---|---|---|
| `call.enter` | null | caller module (entry call: callee) |
| `call.exit` | `ok` / `return` / error code | caller module (entry call: callee) |
| `return` | value type name | current function’s module |
| `branch.eval` | `true` / `false` | current function’s module |
| `loop.iteration` | `continue` / `exit` | current function’s module |
| `effect` | result summary | module from `ExecutionContext` |
| `trace.truncated` | `event_budget` / `byte_budget` | null |
`function_id` is always the canonical callee function id. Future kinds are
additive; consumers must ignore unknown kinds. `effect.operation` names the
`HostOp` (`print`, `fs.read`, ...).
## 4. Capacity
Defaults: 100 000 events and 32 MiB approximate serialized size. `0` means
unlimited for either limit. `run`/`call` expose `--trace-max-events` and
`--trace-max-bytes`; these are options on existing commands, not new commands.
The host stops accepting events once a limit is reached; the writer also
enforces the byte limit on the final serialized document, dropping trailing
events and recording `byte_budget`. Semantic recording is disabled entirely
when no trace was requested.
The `trace` array in `run --json` uses the same budget.
## 5. Staleness
A trace is stale when its `source.analysis_digest` differs from the digest of
the Analysis emitted for the current document. Consumers must not present a
stale trace as current. Moving a project does not change the manifest identity
or digest; changing executable source does.
## 6. Worker transport limits
Plugin worker frames are a 4-byte big-endian length plus JSON, capped at
8 MiB (`8 * 1024 * 1024`). The wire request/result/capability caps are aligned
to `8 MiB - 4096`; error frames remain 1 MiB. A partial header, zero/oversized
length, truncated payload or invalid JSON is a protocol error. A clean EOF is
distinct from a protocol failure. The host marks a worker dead after a transport
failure and never reuses its stdin.
## 7. Compatibility and tests
* Consumers must reject unknown `schema_version` and validate `module_id`
against Analysis modules.
* JSON Schema: `spec/schemas/trace-v2.schema.json`.
* `tests/consumers/trace_consumer.py` verifies identity, digest and event
module references.
* CLI tests cover call/effect events, no-secret redaction, error terminal
status, and event-budget truncation without changing the run status.