feat(trace): Trace v2 identity, module-owned events and capacity bounds (B-16)
- bind traces to the logical project identity and the exact emitted Analysis digest; every span-bearing event carries module_id - add host TraceState budgets (100k events / 32 MiB defaults), CLI --trace-max-events/--trace-max-bytes, terminal trace.truncated events, and a final serialized byte cap; budgets never change run.status - disable semantic recording when no trace is requested - cap worker frames at 8 MiB, align request/result/capability wire caps, make clean EOF distinct from protocol errors, and mark/kill a worker after a transport failure instead of reusing it - add trace-v2 spec/schema, independent trace consumer, CI check and tests - record the S2/C state in STATUS
This commit is contained in:
1 parent
55dcb57553
commit
d890686a2b
21 files changed
+732
-84
No files matched your search
+9
-6
@@ -68,10 +68,10 @@ minted for. Plugins must not dereference, cache or copy it.
|
||||
| `MQ_MAX_EXPORT_NAME_BYTES` | 256 |
|
||||
| `MQ_MAX_SIGNATURE_WIRE_BYTES` | 4096 |
|
||||
| `MQ_MAX_METADATA_WIRE_BYTES` | 65536 |
|
||||
| `MQ_MAX_REQUEST_WIRE_BYTES` | 16 MiB (16,777,216) |
|
||||
| `MQ_MAX_RESULT_WIRE_BYTES` | 64 MiB (67,108,864) |
|
||||
| `MQ_MAX_REQUEST_WIRE_BYTES` | 8 MiB - 4096 (8,384,512) |
|
||||
| `MQ_MAX_RESULT_WIRE_BYTES` | 8 MiB - 4096 (8,384,512) |
|
||||
| `MQ_MAX_ERROR_WIRE_BYTES` | 1 MiB (1,048,576) |
|
||||
| `MQ_MAX_CAPABILITY_REPLY_BYTES` | 16 MiB |
|
||||
| `MQ_MAX_CAPABILITY_REPLY_BYTES` | 8 MiB - 4096 |
|
||||
|
||||
Wire JSON decoding additionally applies the default `WireLimits` from
|
||||
`spec/value-wire.md` (depth 64, 1,000,000 items, 16 MiB strings). Oversized
|
||||
@@ -270,8 +270,8 @@ with both is rejected (`ambiguous_response`). Error:
|
||||
}}
|
||||
```
|
||||
|
||||
Envelope limits: at most `MAX_ARGS = 1024` request arguments; request ≤ 16 MiB,
|
||||
response ≤ 64 MiB, error ≤ 1 MiB. `context` carries no secrets and no
|
||||
Envelope limits: at most `MAX_ARGS = 1024` request arguments; request and
|
||||
response ≤ 8 MiB - 4096, error ≤ 1 MiB. `context` carries no secrets and no
|
||||
reconstructable OS handles. `signature_wire` is
|
||||
`{"signature":"<signature string>"}`; `plugin_metadata_wire` is
|
||||
`{"name":"<package name>","version":"<semver>"}` (extra fields ignored).
|
||||
@@ -364,7 +364,10 @@ full plugin capability broker for worker requests remain future work.
|
||||
|
||||
### 9.1 Worker frames
|
||||
|
||||
Frames are a 4-byte big-endian length followed by one JSON object (max 64 MiB).
|
||||
Frames are a 4-byte big-endian length followed by one JSON object (max 8 MiB).
|
||||
A zero/oversized length, partial header, truncated payload or invalid JSON is a
|
||||
protocol error; clean EOF at a frame boundary is not. The host marks the worker
|
||||
dead after a transport failure and does not reuse it.
|
||||
Worker → host: `{"kind":"ready","exports":[...]}`. Host → worker:
|
||||
`{"kind":"invoke","request":<wire request>}` and `{"kind":"shutdown"}`.
|
||||
Worker → host responses are `{"kind":"response","response":<wire response>}` or
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://marqdo.dev/schemas/trace-v2.schema.json",
|
||||
"title": "Marqdo Trace v2",
|
||||
"type": "object",
|
||||
"required": ["schema", "schema_version", "run", "source", "limits", "truncated", "events"],
|
||||
"properties": {
|
||||
"schema": {"const": "marqdo.trace"},
|
||||
"schema_version": {"const": 2},
|
||||
"run": {
|
||||
"type": "object",
|
||||
"required": ["id", "entry", "status"],
|
||||
"properties": {
|
||||
"id": {"type": "string", "minLength": 1},
|
||||
"entry": {"type": "string", "minLength": 1},
|
||||
"status": {"type": "string", "minLength": 1}
|
||||
},
|
||||
"additionalProperties": true
|
||||
},
|
||||
"source": {
|
||||
"type": "object",
|
||||
"required": ["root_id", "analysis_digest"],
|
||||
"properties": {
|
||||
"root_id": {"type": "string", "pattern": "^(root:|sha256:[0-9a-f]{64}$)"},
|
||||
"analysis_digest": {"type": "string", "pattern": "^sha256:[0-9a-f]{64}$"}
|
||||
},
|
||||
"additionalProperties": true
|
||||
},
|
||||
"limits": {
|
||||
"type": "object",
|
||||
"required": ["max_events", "max_bytes"],
|
||||
"properties": {
|
||||
"max_events": {"type": "integer", "minimum": 0},
|
||||
"max_bytes": {"type": "integer", "minimum": 0}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"truncated": {
|
||||
"oneOf": [
|
||||
{"type": "null"},
|
||||
{
|
||||
"type": "object",
|
||||
"required": ["reason", "limit", "written"],
|
||||
"properties": {
|
||||
"reason": {"enum": ["event_budget", "byte_budget"]},
|
||||
"limit": {"type": "integer", "minimum": 0},
|
||||
"written": {"type": "integer", "minimum": 0}
|
||||
},
|
||||
"additionalProperties": true
|
||||
}
|
||||
]
|
||||
},
|
||||
"events": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["seq", "kind", "module_id", "span", "detail"],
|
||||
"properties": {
|
||||
"seq": {"type": "integer", "minimum": 0},
|
||||
"kind": {"type": "string", "minLength": 1},
|
||||
"function_id": {"type": ["string", "null"]},
|
||||
"module_id": {"type": ["string", "null"]},
|
||||
"span": {
|
||||
"oneOf": [
|
||||
{"type": "null"},
|
||||
{
|
||||
"type": "object",
|
||||
"required": ["start", "end"],
|
||||
"properties": {
|
||||
"start": {"type": "integer", "minimum": 0},
|
||||
"end": {"type": "integer", "minimum": 0}
|
||||
},
|
||||
"additionalProperties": true
|
||||
}
|
||||
]
|
||||
},
|
||||
"detail": {"type": ["string", "null"]},
|
||||
"operation": {"type": "string"},
|
||||
"duration_micros": {"type": "integer", "minimum": 0}
|
||||
},
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"additionalProperties": true
|
||||
}
|
||||
+2
-1
@@ -1,6 +1,7 @@
|
||||
# Marqdo Trace v1
|
||||
|
||||
Status: **implemented by `marqdo run/call --trace-file <path>` (M0.6 R4).**
|
||||
Status: **superseded by [Trace v2](trace-v2.md); never released, do not write
|
||||
v1.** Kept as history.
|
||||
|
||||
## 1. Invariants
|
||||
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# Marqdo Trace v2
|
||||
|
||||
Status: **implemented by `marqdo run/call --trace-file <path>` (S2).**
|
||||
Supersedes Trace v1; v1 was never released and is kept as history.
|
||||
|
||||
## 1. Invariants
|
||||
|
||||
* Tracing is observation, not execution. Only `run`/`call` produce a trace;
|
||||
`check`, `inspect` and `tool list` never execute a document.
|
||||
* Events come from the interpreter (`call.*`, `branch.eval`,
|
||||
`loop.iteration`, `return`) and the host (`effect`). Both share one sequence.
|
||||
* The writer is atomic; an error run still records its terminal status.
|
||||
* No argument values, secrets, handles or file contents are recorded.
|
||||
* Trace capacity is a bounded observation budget. Reaching it truncates the
|
||||
trace; it never changes `run.status` or the program’s execution result.
|
||||
Execution budgets (`--fuel`, `--timeout`, ...) remain separate and still
|
||||
produce `resource.exhausted`.
|
||||
|
||||
## 2. Shape
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "marqdo.trace",
|
||||
"schema_version": 2,
|
||||
"run": {"id": "pid-nanos", "entry": "root:q@0.1.0::index.mq.md::main", "status": "ok"},
|
||||
"source": {"root_id": "root:q@0.1.0", "analysis_digest": "sha256:<64 hex>"},
|
||||
"limits": {"max_events": 100000, "max_bytes": 33554432},
|
||||
"truncated": null,
|
||||
"events": [
|
||||
{"seq": 0, "kind": "call.enter", "function_id": "root:q@0.1.0::index.mq.md::main",
|
||||
"module_id": "root:q@0.1.0::index.mq.md", "span": null, "detail": null},
|
||||
{"seq": 1, "kind": "effect", "module_id": "dep:p@0.1.0#<digest>::index.mq.md",
|
||||
"span": {"start": 40, "end": 52}, "detail": "ok", "operation": "print",
|
||||
"duration_micros": 3}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`seq` is globally increasing. `run.status` is `ok` or a stable runtime error
|
||||
code. Every span-bearing event has a `module_id`; the span is a UTF-8 byte
|
||||
offset range in that module. `source` only carries project identity and the
|
||||
Analysis digest; it does not identify the span owner.
|
||||
|
||||
`source.root_id` is the Analysis v2 `project.root_id`
|
||||
(`root:<name>@<version>` or `sha256:<workspace snapshot>`).
|
||||
`source.analysis_digest` is `sha256:` plus the SHA-256 of the exact Analysis
|
||||
v2 bytes produced for the same project with the same projection configuration.
|
||||
A consumer verifies it by hashing the Analysis file.
|
||||
|
||||
`truncated` is `null` or:
|
||||
|
||||
```json
|
||||
{"reason": "event_budget", "limit": 20, "written": 20}
|
||||
```
|
||||
|
||||
`reason` is `event_budget` or `byte_budget`. When truncation occurs, the final
|
||||
event is `{"kind":"trace.truncated","detail":<reason>,"module_id":null,
|
||||
"span":null,"seq":<one past the last written event>}`.
|
||||
|
||||
## 3. Event kinds and span owners
|
||||
|
||||
| kind | detail | `module_id` owner |
|
||||
|---|---|---|
|
||||
| `call.enter` | null | caller module (entry call: callee) |
|
||||
| `call.exit` | `ok` / `return` / error code | caller module (entry call: callee) |
|
||||
| `return` | value type name | current function’s module |
|
||||
| `branch.eval` | `true` / `false` | current function’s module |
|
||||
| `loop.iteration` | `continue` / `exit` | current function’s module |
|
||||
| `effect` | result summary | module from `ExecutionContext` |
|
||||
| `trace.truncated` | `event_budget` / `byte_budget` | null |
|
||||
|
||||
`function_id` is always the canonical callee function id. Future kinds are
|
||||
additive; consumers must ignore unknown kinds. `effect.operation` names the
|
||||
`HostOp` (`print`, `fs.read`, ...).
|
||||
|
||||
## 4. Capacity
|
||||
|
||||
Defaults: 100 000 events and 32 MiB approximate serialized size. `0` means
|
||||
unlimited for either limit. `run`/`call` expose `--trace-max-events` and
|
||||
`--trace-max-bytes`; these are options on existing commands, not new commands.
|
||||
The host stops accepting events once a limit is reached; the writer also
|
||||
enforces the byte limit on the final serialized document, dropping trailing
|
||||
events and recording `byte_budget`. Semantic recording is disabled entirely
|
||||
when no trace was requested.
|
||||
|
||||
The `trace` array in `run --json` uses the same budget.
|
||||
|
||||
## 5. Staleness
|
||||
|
||||
A trace is stale when its `source.analysis_digest` differs from the digest of
|
||||
the Analysis emitted for the current document. Consumers must not present a
|
||||
stale trace as current. Moving a project does not change the manifest identity
|
||||
or digest; changing executable source does.
|
||||
|
||||
## 6. Worker transport limits
|
||||
|
||||
Plugin worker frames are a 4-byte big-endian length plus JSON, capped at
|
||||
8 MiB (`8 * 1024 * 1024`). The wire request/result/capability caps are aligned
|
||||
to `8 MiB - 4096`; error frames remain 1 MiB. A partial header, zero/oversized
|
||||
length, truncated payload or invalid JSON is a protocol error. A clean EOF is
|
||||
distinct from a protocol failure. The host marks a worker dead after a transport
|
||||
failure and never reuses its stdin.
|
||||
|
||||
## 7. Compatibility and tests
|
||||
|
||||
* Consumers must reject unknown `schema_version` and validate `module_id`
|
||||
against Analysis modules.
|
||||
* JSON Schema: `spec/schemas/trace-v2.schema.json`.
|
||||
* `tests/consumers/trace_consumer.py` verifies identity, digest and event
|
||||
module references.
|
||||
* CLI tests cover call/effect events, no-secret redaction, error terminal
|
||||
status, and event-budget truncation without changing the run status.
|
||||
Reference in new issue
Block a user