8.0 KiB
ADR-0059: Runtime observability and Execution Events v1
- Status: Accepted
- Date: 2026-11-22
- Decision owners: CLI, runtime and host maintainers
- Inputs: RFC-OBS-0001
(issue
#1ofcflmy/marqdo-cli), Trace v2, Analysis v2, Tool Protocol v1
Context
Marqdo has two public, versioned machine-readable artifacts: Analysis v2 (static, never executes) and Trace v2 (one complete document published atomically after a run). Neither can stream program stdout/stderr while a task is still running, and Trace v2 deliberately has no argument or local-variable values.
External tools (marqdo-view, agents, IDEs, Knowledge/Prove) therefore had two
bad options: parse human terminal output, or embed/link private runtime types
and build a second execution path. Both break the repository boundary this
project exists to maintain: the CLI is the only runtime.
Decision
-
A third, independent public protocol exists: Execution Events v1.
schema = "marqdo.execution-events",schema_version = 1.- It is a one-way NDJSON (JSON Lines) stream over the CLI's protocol stdout.
- It is not Trace v2, not Analysis v2 and not Tool Protocol v1. Each has its own version axis. Adding optional events/fields is allowed in v1; changing required field meaning, ordering, redaction promises or the source-digest definition requires a new major.
- The CLI implements no HTTP, SSE, WebSocket, browser session or View/Agent/IDE-specific branch. Consumers convert the stream themselves.
-
The stream is opt-in and never changes semantics.
marqdo run/call --event-stream jsonlturns the CLI stdout into a pure JSONL channel. Without the option, stdout, stderr, exit codes, Trace and Analysis behavior are byte-for-byte unchanged. Observation is not a capability grant and cannot widenpure/project/strictauthorization. -
--event-stream jsonland--jsonare mutually exclusive.--trace-filemay be combined; Trace v2 keeps its atomic, post-run, low-sensitivity contract. Program stdout/stderr are encapsulated asstdout.chunk/stderr.chunkevents and are never written raw to the protocol stdout. -
Events are emitted at real commit points, not from final buffers. Stdout/stderr chunks originate in
StdHost::resource_writeimmediately after the host accepts and commits bytes, inside the same budget path. The finaltake_output()buffer is never split into fake "live" chunks. Selected local-variable events originate after the assignment has been committed toFrame.locals; a rejected memory write never produces a success event. -
The host owns the observation facade.
marqdo-host::observedefines typed internal events (ObservationEvent), theObservationSinktrait,ObservationConfig, frame identity (FrameId) and the observation budget. The runtime emits borrowed events through defaultHostmethods; it never hands the sinkFrame.locals, the wholeMemoryStore, or raw OS handles. The CLI owns only JSONL encoding and the public envelope. No new crate is introduced. -
Call instances get an opaque identity. Each entered frame gets a run-unique
frame_id(f1,f2, ...) and aparent_frame_id(nullat the top level). Frame IDs exist only in Execution Events v1; Trace v2 is not given a new field. Argument binding happens before a frame is announced, so a failed binding never produces a dangling frame. -
Values are selected, bounded and off by default.
- Default event mode publishes types/status only (
value_state: not_requested). --observe-variable NAME/--observe-parameter NAMEare exact-name, repeatable selections. There is no wildcard, regex, path expression or--dump-all-locals.- Supported value types are the scalars
Int,Float,Bool;Textadditionally requires--observe-text-values. Lists, maps, tables, formulas, code, memory, resources and plugin handles reportunsupportedand are never expanded or serialized byDebug. - Every value is capped by
--observe-max-value-bytes(default 256) and reportsnot_requested,redacted,truncated,unsupportedorobserveddistinctly. A text truncation never splits a UTF-8 scalar. - Observation lowers confidentiality: it is an explicit developer action. It is never enabled from a project manifest, never recorded in Trace v2 or Analysis v2, and never automatically enabled by View.
- Default event mode publishes types/status only (
-
Budgets, ordering and failure are defined.
- One dispatcher assigns
seqfrom 0; stdout/stderr, structural and value events share the observable order of admission. - Structural/value events have a bounded budget (default 100 000 events /
16 MiB). Exhaustion emits one
observation.truncatedand stops only non-critical events; stdout/stderr and the terminal event still flow. stream.opened(seq 0),run.started, stdout/stderr and the terminalrun.completed/run.failed/run.rejectedare not dropped by the structural budget.- Writes are bounded and synchronous: a slow consumer applies IO
backpressure instead of growing an unbounded queue. A write failure
aborts the run with
observe.transport_failed; the CLI must not keep executing a program with side effects after its event pipe broke, and it must not write a half JSON line. - A killed/crashed process may have no terminal event. Consumers must
report
incomplete, never synthesize success.
- One dispatcher assigns
-
run.rejectedcovers pre-execution failure in event mode. After the writer is open, prepare/authorization/compile failures produce a structured terminalrun.rejectedwith the existing stable error code and phase, a non-zero exit, and norun.started. CLI option parsing errors still use the ordinary clap stderr path before the protocol exists. -
The source binding is the Analysis digest.
run.started.payload.sourcecarriesroot_idandanalysis_digest=sha256:plus the SHA-256 of the exact Analysis v2 bytes for the same project/projection configuration. A changed source makes old events stale; consumers must not render them onto new source. -
The Trace v2 run identity is shared, without changing Trace v2. In a combined run the CLI composition root creates one opaque
run_idand passes it to both writers, sotrace.run.id == execution stream run_id. Trace v2 keepsschema_version: 2and gains no frame/value fields.
Consequences
- External tools can consume live output with a released binary and a documented schema; they still need no private Rust crate.
- The runtime gains observation hooks at existing commit points. The default
(no option) path installs no sink and does not clone
Values or serialize observation data. - The value policy is intentionally smaller than a debugger. Breakpoints, stepping, time travel, raw memory snapshots, memory-byte events and a bidirectional Debug Protocol remain a separate future RFC.
- Memory metadata events (
memory.region.*,memory.budget) are P2 in RFC-OBS-0001 and are not part of this decision's implemented scope; no raw memory content or handle is ever exposed.
Rejected alternatives
- Stream Trace v2 while running. Breaks existing complete-file consumers and cannot naturally carry live stdout/stderr or reject-before-run events.
- Make events a flag of
--json. Mixes a complete result envelope with a line stream on one channel and violates the clean-stdout guarantee. - Emit stdout by splitting the final buffer. It is not real-time and defeats the purpose; the RFC explicitly forbids it.
- Put JSONL/SSE/HTTP in the CLI. Adds transport and product concerns to the composition root; View's SSE belongs to the View repository.
- Publish all locals as a default debugging dump. Unbounded confidentiality loss and no stable value encoding; rejected in favour of exact, bounded selections.