obs(03-05): fixtures, black-box schema test, CI/consumer gate and protocol docs
This commit is contained in:
1 parent
e6e497e68b
commit
2d51f08340
21 files changed
+467
-13
No files matched your search
@@ -94,6 +94,49 @@ exports reconcile exactly with the static manifest. The independent
|
||||
frozen-binary suite is the beta gate above; the older `6a7bb98` manifest is
|
||||
historical and superseded by the release-SHA manifest.
|
||||
|
||||
## 0.5 Post-beta addition — Execution Events v1 (unreleased)
|
||||
|
||||
This section records work **after** the `v3.0.0-beta.1` tag. It is not part of
|
||||
the tagged beta archive and no new release tag has been created. The design
|
||||
input is RFC-OBS-0001 (issue `#1`), the frozen decision is ADR-0059 and the
|
||||
normative contract is [`spec/execution-events-v1.md`](../spec/execution-events-v1.md).
|
||||
|
||||
- **Implemented (P0/P1):** `run/call --event-stream jsonl` writes a pure NDJSON
|
||||
`marqdo.execution-events` v1 stream; lifecycle/terminal events; byte-exact
|
||||
base64 stdout/stderr chunks emitted at the Host commit point; run-unique
|
||||
`frame_id`/`parent_frame_id`; `return`/`branch.eval`/`loop.iteration`/`effect`
|
||||
attribution; exact-name, bounded value observation with a separate
|
||||
`--observe-text-values` consent; structural/value budget with one
|
||||
`observation.truncated`; broken-pipe failure via `observe.transport_failed`;
|
||||
Trace v2 coexistence with a shared run id and unchanged Trace schema.
|
||||
- **Unchanged:** Analysis v2 and Trace v2 schemas/versions, `--json` envelope,
|
||||
default stdout/stderr/exit codes, language semantics and capability
|
||||
authorization.
|
||||
- **Deferred:** OBS-06 memory metadata (P2), debugger/stepping/time travel,
|
||||
bidirectional debug protocol, SSE/HTTP in the CLI (View implements that).
|
||||
- **Explicitly unsupported combination:** `--mode strict` rejects
|
||||
`--event-stream` with a structured `run.rejected`; there is no downgrade.
|
||||
|
||||
Executed local gate on this tree (Linux x86_64, Rust 1.100.0):
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS: 628 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-cli --test execution_events PASS: 12 passed
|
||||
cargo test -p marqdo-host --test observation PASS: 8 passed
|
||||
cargo test -p marqdo-runtime --test observation PASS: 4 passed
|
||||
python3 tests/consumers/execution_events_consumer.py PASS on a live real-binary stream (schema, seq, terminal, byte_count)
|
||||
```
|
||||
|
||||
The Python consumer is an independent black-box parser using only the checked-in
|
||||
JSON Schema; it was run against the local binary's stream and the checked-in
|
||||
valid/invalid fixtures. This is local evidence, not a frozen release-archive or
|
||||
same-SHA CI gate. `cargo test --workspace --locked` includes the new schema
|
||||
fixture test when `python3` + `jsonschema` are available (the Woodpecker test
|
||||
lane installs `python3-jsonschema` and also runs the consumer against a real
|
||||
`--event-stream jsonl` invocation).
|
||||
|
||||
## 1. Historical: pre-beta remote baseline and evidence
|
||||
|
||||
> **Historical record.** Everything in §1 onward was executed before the
|
||||
|
||||
@@ -26,6 +26,14 @@ replace `spec/` or a recorded test log.
|
||||
> Normative boundaries: [`spec/callable-resolution.md`](../../spec/callable-resolution.md),
|
||||
> [`spec/extension-selection.md`](../../spec/extension-selection.md),
|
||||
> [`spec/resource-provider-v2.md`](../../spec/resource-provider-v2.md).
|
||||
>
|
||||
> **Post-baseline observability (ADR-0059, RFC-OBS-0001).** After the tagged
|
||||
> baseline the tree adds Execution Events v1: a typed host observation facade,
|
||||
> real-commit stdout/stderr chunks, run-unique call frames and explicitly
|
||||
> selected bounded values over `run/call --event-stream jsonl`. Analysis v2 and
|
||||
> Trace v2 are unchanged. This is locally tested on the post-baseline tree
|
||||
> ([`../STATUS.md`](../STATUS.md) §0.5) and is not part of the tagged beta
|
||||
> archive or its same-SHA evidence.
|
||||
|
||||
## 1. Four state dimensions
|
||||
|
||||
@@ -35,8 +43,8 @@ platform".
|
||||
|
||||
| Dimension | Current state | Source |
|
||||
|---|---|---|
|
||||
| Implemented | Rust workspace `3.0.0-beta.1`: RFC-LANG-0002 Markdown/frontend/IR, v2 statement parser, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, migrated bundled stdlib, CLI composition root | `Cargo.toml`, `crates/`, `stdlib/` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **599 passed, 0 failed** across 67 targets on Linux x86_64 in the RFC-EXT-0001 tree; fmt and clippy clean. Historical baseline: 507 passed at `9c1723d`. | `docs/STATUS.md`, `docs/evidence/2026-11-22-ext-implementation.md` |
|
||||
| Implemented | Rust workspace `3.0.0-beta.1`: RFC-LANG-0002 Markdown/frontend/IR, v2 statement parser, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, migrated bundled stdlib, CLI composition root; post-baseline Execution Events v1 observation (ADR-0059) | `Cargo.toml`, `crates/`, `stdlib/` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **628 passed, 0 failed, 2 ignored** on the post-baseline observability tree (Linux x86_64); `cargo fmt --check` and `clippy -D warnings` clean. Historical baseline: 599 passed before Execution Events v1, 507 at `9c1723d`. | `docs/STATUS.md` §0.5, `docs/evidence/2026-11-22-ext-implementation.md` |
|
||||
| Bundled stdlib released | In-repo bundle `marqdo-stdlib@0.1.0`, SHA-256 digest verified before loading; it is **not** an external registry package | `stdlib/stdlib.json`, `stdlib/README.md` |
|
||||
| Multi-platform verified | Linux local gate is the only executed lane here; Windows/macOS native lanes are **UNVERIFIED**, and the GitHub security lane remains a **VM-only** gate | `docs/STATUS.md` §1, §4 |
|
||||
|
||||
@@ -110,6 +118,7 @@ review RFC, not a new crate per provider.
|
||||
| Public contract | API (`.mq.md`) | State | Tests |
|
||||
|---|---|---|---|
|
||||
| stdio stdin | `resource.open("stdio","stdin",["read"])`, `resource.read`, `resource.close`; one run-scoped cursor | implemented, read-only | `resource_streams`, `p2_closure`, `tests/conformance` |
|
||||
| Execution Events v1 | `run/call --event-stream jsonl`; `--observe-variable`, `--observe-parameter`, `--observe-text-values` | implemented P0/P1 (post-beta); P2 memory metadata deferred; strict refuses the option | `execution_events` CLI, `observation` host/runtime, `execution_events_consumer.py` |
|
||||
| stdio stdout | `resource.open("stdio","stdout",["write"])`, `resource.write`, `resource.close`; separate output budget | implemented | `stdlib_bundle`, `print_routing`, `resource_streams` |
|
||||
| stdio stderr | `resource.open("stdio","stderr",["write"])`; separate from stdout and CLI diagnostics | implemented | `resource_streams`, `stdlib_product` |
|
||||
| project file read | `resource.open("file", path, ["read"])`; path/grant checked; bounded | implemented, project mode | `resource_streams`, `stdlib_product` |
|
||||
|
||||
+13
-2
@@ -16,14 +16,25 @@ The default Docker pipeline runs on push and pull request:
|
||||
```text
|
||||
test:
|
||||
rustup component add clippy rustfmt
|
||||
apt install curl/ca-certificates/python3 + Node.js 22
|
||||
apt install curl/ca-certificates/python3/python3-jsonschema + Node.js 22
|
||||
cargo fmt --all -- --check
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
cargo build --locked -p marqdo-provider-worker
|
||||
cargo test --workspace --locked
|
||||
cargo run --locked -p marqdo-cli -- run \
|
||||
tests/consumers/fixtures/execution-events/basic.mq.md --event-stream jsonl \
|
||||
> /tmp/execution-events.jsonl
|
||||
python3 tests/consumers/execution_events_consumer.py /tmp/execution-events.jsonl
|
||||
cargo test -p marqdo-cli --test docs_consistency
|
||||
cargo test -p marqdo-cli --test architecture_boundaries
|
||||
cargo test -p marqdo-cli --test stdlib_product
|
||||
python3 -m unittest discover -s tools/ci -p "test_*.py"
|
||||
rustup target add x86_64-pc-windows-gnu
|
||||
cargo check --workspace --all-targets --target x86_64-pc-windows-gnu
|
||||
|
||||
build (depends on test):
|
||||
cargo build --release --locked -p marqdo-cli
|
||||
cargo build --release --locked -p marqdo-cli -p marqdo-provider-worker
|
||||
package + cold-install smoke (which now validates live Execution Events v1)
|
||||
```
|
||||
|
||||
Node.js 22 is required because the optional foreign-kernel tests use ES-module
|
||||
|
||||
@@ -7,9 +7,9 @@ core checkout, or host memory layout. Adding a new domain function must not
|
||||
require a core commit.
|
||||
|
||||
Product version for these docs: **`3.0.0-beta.1`** (Linux x86_64 beta).
|
||||
Protocol versions are independent: Analysis v2, Trace v2, Tool v1, Plugin ABI
|
||||
v1, Foreign wire v1, Provider manifest v2, lock v4, markdown profile
|
||||
`marqdo.md.v1`.
|
||||
Protocol versions are independent: Analysis v2, Trace v2, Execution Events v1,
|
||||
Tool v1, Plugin ABI v1, Foreign wire v1, Provider manifest v2, lock v4,
|
||||
markdown profile `marqdo.md.v1`.
|
||||
|
||||
## Pick your entry point
|
||||
|
||||
@@ -33,7 +33,7 @@ v1, Foreign wire v1, Provider manifest v2, lock v4, markdown profile
|
||||
4. [extensions.md](extensions.md) — plugins, foreign kernels, resource
|
||||
providers, extension selection and lock v4.
|
||||
5. [analysis-trace.md](analysis-trace.md) — the public static/dynamic data
|
||||
contracts and digest binding.
|
||||
contracts, Execution Events v1 and digest binding.
|
||||
6. [compatibility.md](compatibility.md) — version axes, platform matrix and the
|
||||
prerelease policy.
|
||||
7. [release-limits.md](release-limits.md) — the honest beta limitations (this
|
||||
|
||||
@@ -60,6 +60,51 @@ marqdo run index.mq.md --trace-file ./trace.json
|
||||
`spec/schemas/trace-v2.schema.json`; consumer:
|
||||
`tests/consumers/trace_consumer.py`.
|
||||
|
||||
## Execution Events v1 (live)
|
||||
|
||||
```bash
|
||||
marqdo run index.mq.md --event-stream jsonl
|
||||
marqdo call index.mq.md main --arg n=5 --event-stream jsonl
|
||||
marqdo run index.mq.md --event-stream jsonl --trace-file ./trace.json
|
||||
```
|
||||
|
||||
- `schema: "marqdo.execution-events"`, `schema_version: 1`. This is a separate
|
||||
protocol from Analysis/Trace and has its own version axis.
|
||||
- While active, the CLI stdout is a **pure NDJSON channel**: one JSON object per
|
||||
line, `seq` starts at 0 and is strictly increasing. Program stdout/stderr are
|
||||
only `stdout.chunk`/`stderr.chunk` events with `encoding: "base64"`,
|
||||
`data_base64` and a matching `byte_count`. Concatenated decoded bytes equal a
|
||||
normal run's output; invalid UTF-8/NUL/binary bytes are preserved.
|
||||
- `stream.opened` is always seq 0. A run then has at most one `run.started` and
|
||||
exactly one terminal event (`run.completed`, `run.failed`, `run.rejected`) on
|
||||
a complete stream. An EOF without a terminal event is `incomplete`
|
||||
(SIGKILL/crash/broken pipe); never synthesize success.
|
||||
- `run.started.source` contains `root_id` and the `sha256:` digest of the exact
|
||||
Analysis v2 bytes. `--trace-file` may be combined; both writers share one
|
||||
opaque run id, and Trace v2 remains atomic and value-free.
|
||||
- Structural events: `call.enter`/`call.exit` (`frame_id`, `parent_frame_id`),
|
||||
`return`, `branch.eval`, `loop.iteration`, `effect`. Frames identify call
|
||||
instances, not functions; a failed argument binding creates no frame.
|
||||
- Values are off by default (`value_state: not_requested`).
|
||||
`--observe-variable NAME` and `--observe-parameter NAME` are exact-name
|
||||
selections. `Int`/`Float`/`Bool` are bounded by
|
||||
`--observe-max-value-bytes` (default 256); selected `Text` is `redacted` until
|
||||
`--observe-text-values` is also passed. Containers, `Memory`, resources and
|
||||
plugin handles are `unsupported`; handles are never serialized.
|
||||
- Structural/value events have a separate budget
|
||||
(`--observe-max-events`, `--observe-max-bytes`; defaults 100 000 / 16 MiB).
|
||||
Exhaustion emits one `observation.truncated` and still delivers stdout/stderr
|
||||
and the terminal event. A broken output pipe fails the run with
|
||||
`observe.transport_failed`; it never keeps executing side effects silently.
|
||||
- `--event-stream` is mutually exclusive with `--json`, rejected by
|
||||
`check`/`inspect`/`fmt`/`tool`, and currently rejected by `--mode strict`
|
||||
(fail-closed, no downgrade). It is not a capability grant.
|
||||
- Contract: [`../../spec/execution-events-v1.md`](../../spec/execution-events-v1.md);
|
||||
schema: `spec/schemas/execution-events-v1.schema.json`; consumer:
|
||||
`tests/consumers/execution_events_consumer.py`; fixtures and Rust gate:
|
||||
`tests/consumers/fixtures/execution-events/` and
|
||||
`crates/cli/tests/execution_events.rs`.
|
||||
|
||||
## Consumer rules
|
||||
|
||||
1. Reject unknown schema majors; there is no silent "use latest" fallback.
|
||||
|
||||
@@ -11,6 +11,7 @@ This page summarizes the public support policy. The authoritative matrix is
|
||||
| Product / CLI | `3.0.0-beta.1` | SemVer prerelease; breaking changes inside the 3.0 beta line are allowed and appear in `CHANGELOG.md`. |
|
||||
| Analysis | v2 | New major for incompatible changes; consumers reject unknown majors. |
|
||||
| Trace | v2 | New major for incompatible changes; binds to exact Analysis bytes. |
|
||||
| Execution Events | v1 | One-line NDJSON stream; new major for incompatible required-field/ordering/redaction changes. Consumers reject unknown majors. |
|
||||
| CLI Tool | v1 | Frozen manifest/negotiation/bundle contract for v1. |
|
||||
| Plugin ABI | v1 (`1.0`) | Frozen C layout and version negotiation. |
|
||||
| Foreign wire | v1 | One-shot request envelope. |
|
||||
@@ -35,6 +36,9 @@ ranges**. Beta cores require a prerelease range such as
|
||||
|
||||
Strict is Linux-only and probe-gated. Plugin/provider worker transports are not
|
||||
OS sandboxes. `pure`/`project` are execution modes, not security boundaries.
|
||||
Execution Events v1 is a live observation protocol: it is not a capability
|
||||
grant, values are off by default, and `--mode strict` currently refuses
|
||||
`--event-stream` explicitly (no downgrade).
|
||||
|
||||
## Language compatibility
|
||||
|
||||
|
||||
@@ -61,8 +61,10 @@ Even when available:
|
||||
* `3.0.0-beta.1` is a **prerelease**, not stable and not LTS. Breaking changes
|
||||
inside the 3.0 beta series are allowed and announced in the changelog.
|
||||
* The product version is independent of protocol versions: Analysis v2, Trace
|
||||
v2, Tool v1, Plugin ABI v1, Foreign wire v1, Provider manifest v2 and lock v4
|
||||
keep their own major numbers. Do not infer them from `3.0.0-beta.1`.
|
||||
v2, Execution Events v1, Tool v1, Plugin ABI v1, Foreign wire v1, Provider
|
||||
manifest v2 and lock v4 keep their own major numbers. Do not infer them from
|
||||
`3.0.0-beta.1`. Execution Events v1 is a post-beta addition and is not in the
|
||||
tagged beta archive.
|
||||
* The official bundled stdlib keeps its independent package version `0.1.0`.
|
||||
A plugin `.mq.md` package version such as `0.1.0` is also independent.
|
||||
* `project.marqdo` in `marqdo.yaml` is enforced in this release. SemVer
|
||||
@@ -80,6 +82,20 @@ Archives are accompanied by SHA256 checksums and `RELEASE_INFO.json`
|
||||
integrity only; they are **not** a publisher signature. A digest obtained from
|
||||
an untrusted Git URL does not make the code trustworthy.
|
||||
|
||||
## Execution Events v1 (post-beta addition)
|
||||
|
||||
* Implemented: live stdout/stderr chunks, lifecycle/terminal events, call
|
||||
instance ids, branch/loop/effect events, explicit bounded value observation.
|
||||
* Not implemented: memory region/budget metadata (OBS-06/P2), raw memory
|
||||
content, breakpoints/stepping/time travel, and a bidirectional debug
|
||||
protocol. Do not infer a debugger from this protocol.
|
||||
* `--mode strict` currently rejects `--event-stream` explicitly; there is no
|
||||
silent fallback to `project`.
|
||||
* The live lane is tested on Linux x86_64; macOS/Windows are unverified for
|
||||
this option like the rest of the beta CLI.
|
||||
* Observation lowers confidentiality when values are explicitly enabled. Text
|
||||
needs a second opt-in; View/agents must not enable these flags by default.
|
||||
|
||||
## Deliberately deferred
|
||||
|
||||
strict L2, Wasm plugins, macOS/Windows provider worker, automatic uv/npm
|
||||
|
||||
@@ -91,6 +91,20 @@ Dynamic input:
|
||||
"$MARQDO_BIN" run index.mq.md --trace-file ./trace.json
|
||||
```
|
||||
|
||||
Live dynamic input (Execution Events v1):
|
||||
|
||||
```bash
|
||||
"$MARQDO_BIN" run index.mq.md --event-stream jsonl
|
||||
"$MARQDO_BIN" call index.mq.md main --arg n=5 --event-stream jsonl
|
||||
```
|
||||
|
||||
Consume the stdout line stream as NDJSON; do not pass `--json` in the same
|
||||
invocation. Negotiate by reading `stream.opened` (`marqdo.execution-events`,
|
||||
`schema_version: 1`) and reject unknown majors. A tool server may convert the
|
||||
stream to its own SSE/websocket transport; the CLI never implements HTTP/SSE.
|
||||
This option is unavailable under `--mode strict` in the current tree and the
|
||||
tool must surface that refusal instead of pretending real-time support.
|
||||
|
||||
See [analysis-trace.md](analysis-trace.md) for fields and digest rules. Do not
|
||||
depend on `inspect --ir`: that JSON is not a frozen cross-repository protocol.
|
||||
The `view` tool renders the **original Markdown**, not the Analysis JSON; the
|
||||
|
||||
Reference in new issue
Block a user