obs(03-05): fixtures, black-box schema test, CI/consumer gate and protocol docs
This commit is contained in:
1 parent
e6e497e68b
commit
2d51f08340
21 files changed
+467
-13
No files matched your search
@@ -10,7 +10,7 @@ steps:
|
||||
commands:
|
||||
- rustup component add clippy rustfmt
|
||||
- apt-get update
|
||||
- apt-get install -y --no-install-recommends curl ca-certificates python3 build-essential gcc-mingw-w64-x86-64
|
||||
- apt-get install -y --no-install-recommends curl ca-certificates python3 python3-jsonschema build-essential gcc-mingw-w64-x86-64
|
||||
- curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
|
||||
- apt-get install -y --no-install-recommends nodejs
|
||||
- cargo fmt --all -- --check
|
||||
@@ -19,6 +19,10 @@ steps:
|
||||
# binary. Build it once so the test harness does not need a nested build.
|
||||
- cargo build --locked -p marqdo-provider-worker
|
||||
- cargo test --workspace --locked
|
||||
# Execution Events v1 black-box contract: a released-binary-shaped run
|
||||
# validated by the independent Python consumer and the public schema.
|
||||
- cargo run --quiet --locked -p marqdo-cli -- run tests/consumers/fixtures/execution-events/basic.mq.md --event-stream jsonl > /tmp/execution-events.jsonl
|
||||
- python3 tests/consumers/execution_events_consumer.py /tmp/execution-events.jsonl
|
||||
- cargo test -p marqdo-cli --test docs_consistency
|
||||
- cargo test -p marqdo-cli --test architecture_boundaries
|
||||
- python3 -m unittest discover -s tools/ci -p "test_*.py"
|
||||
|
||||
@@ -10,6 +10,36 @@ Foreign wire v1, Provider manifest v2, lock v4) and independent package
|
||||
versions (including the bundled stdlib `0.1.0`) are versioned separately and
|
||||
do not follow the product version. Dates use ISO 8601.
|
||||
|
||||
## Unreleased (post `3.0.0-beta.1`)
|
||||
|
||||
### Runtime observability — Execution Events v1 (RFC-OBS-0001, ADR-0059)
|
||||
|
||||
- `marqdo run` and `marqdo call` accept `--event-stream jsonl`: the CLI stdout
|
||||
becomes a pure NDJSON `marqdo.execution-events` v1 channel. Program
|
||||
stdout/stderr are carried as byte-exact base64 chunk events emitted at the
|
||||
Host commit point, not by splitting the final buffer.
|
||||
- Lifecycle and terminal events (`stream.opened`, `run.started`,
|
||||
`run.completed`, `run.failed`, `run.rejected`) have defined ordering and exit
|
||||
codes. `--event-stream` is mutually exclusive with `--json`, is rejected by
|
||||
static commands and by `--mode strict`, and may be combined with
|
||||
`--trace-file`; the two writers share one run id and Analysis digest.
|
||||
- Call instances get run-unique `frame_id`/`parent_frame_id`; `call.enter`,
|
||||
`call.exit`, `return`, `branch.eval`, `loop.iteration`, `effect` and
|
||||
`variable.assigned` events are bounded by a structural budget with a single
|
||||
`observation.truncated` marker.
|
||||
- Values are off by default. `--observe-variable` / `--observe-parameter` are
|
||||
exact-name selections; `Int`/`Float`/`Bool` are bounded and `Text` requires
|
||||
the separate `--observe-text-values` consent. Unselected, redacted,
|
||||
truncated and unsupported states are distinct.
|
||||
- New public contract
|
||||
[`spec/execution-events-v1.md`](spec/execution-events-v1.md) and
|
||||
[`spec/schemas/execution-events-v1.schema.json`](spec/schemas/execution-events-v1.schema.json),
|
||||
with fixtures and an independent Python consumer. Analysis v2 and Trace v2
|
||||
schemas/versions are unchanged.
|
||||
- Limitations: memory metadata events (OBS-06/P2), a bidirectional debug
|
||||
protocol and SSE/HTTP remain out of scope; strict mode refuses event mode
|
||||
explicitly.
|
||||
|
||||
## 3.0.0-beta.1 (2026-11-22)
|
||||
|
||||
First public prerelease of the independent Marqdo toolchain. This is a
|
||||
|
||||
@@ -53,6 +53,7 @@ Consequences for compatibility:
|
||||
| `marqdo.lock` | v4 | v3 is rejected with an explicit `marqdo lock` migration error; v1/v2 are rejected with a relock instruction; `run`/`call`/`check` never rewrite or fetch the lock. `extensions:` records manifest/artifact identity (integrity, not publisher auth). |
|
||||
| Analysis | v2 | Consumers reject unknown majors; incompatible field changes require v3. |
|
||||
| Trace | v2 | Binds to the exact Analysis bytes; unknown majors rejected. |
|
||||
| Execution Events | v1 | Live NDJSON for `run/call --event-stream jsonl`; independent version axis. Values are opt-in and bounded; unknown majors rejected. Not yet part of the tagged beta archive (post-beta addition). |
|
||||
| CLI Tool | v1 | Manifest schema, SemVer negotiation and whole-bundle digest are stable for v1. |
|
||||
| Plugin ABI | v1 | Struct layout/version negotiation frozen; `--trust-native-plugin` is required for in-process loading. The static manifest signature is authoritative; `run` reconciles loaded exports/signatures/owners/artifact digests exactly (`ext.plugin_binding_mismatch`); ABI v1 defaults are rejected. |
|
||||
| Resource Provider manifest | v2 | `schema_version: 2`, `abi: c-abi-1`, `transport: worker`; v1 is rejected (`ext.unsupported_provider_schema`), no silent migration. CLI `--provider`/`--allow-provider`; trusted worker, not an OS sandbox. |
|
||||
@@ -82,6 +83,7 @@ portable/tested locally but the native platform lane has not run here;
|
||||
| Capability handler registry (`fs.read` handler, audit, unavailable) | PASS | UNVERIFIED | UNVERIFIED |
|
||||
| External conformance repository (`marqdo-extension-conformance`, frozen CLI) | PASS (`--release` binary sha256 `28e3776d…`, fixture repo `e4b7352c…`; see `docs/evidence/extension/6a7bb983.../manifest.json`) | UNVERIFIED | UNVERIFIED |
|
||||
| Analysis v2 / Trace v2 + Python consumers | PASS | UNVERIFIED | UNVERIFIED |
|
||||
| Execution Events v1 (`--event-stream jsonl`, P0/P1) | PASS locally (12 CLI contract tests + 8 host + 4 runtime; independent Python schema consumer); post-beta, not in the tagged archive | UNVERIFIED | UNVERIFIED |
|
||||
| Tool Protocol v1 + 3 independent consumers | PASS | UNVERIFIED | UNVERIFIED |
|
||||
| Native ABI (`plugin inspect`, trusted call) | PASS (C fixture) | UNVERIFIED (host-native fixture) | **UNSUPPORTED** |
|
||||
| Worker ABI + `fs.read` bridge | PASS (Unix) | UNVERIFIED | **UNSUPPORTED** (Unix-gated) |
|
||||
|
||||
@@ -67,6 +67,7 @@ marqdo init demo
|
||||
cd demo
|
||||
marqdo check index.mq.md
|
||||
marqdo run index.mq.md # executes ## @exec main
|
||||
marqdo run index.mq.md --event-stream jsonl # live Execution Events v1 NDJSON
|
||||
marqdo inspect index.mq.md --ast
|
||||
marqdo inspect index.mq.md --signature
|
||||
marqdo call index.mq.md main
|
||||
@@ -230,7 +231,9 @@ rules are [`RFC-LANG-0002`](docs/architecture/RFC-LANG-0002-unified-semantics.md
|
||||
(`--plugin --trust-native-plugin --allow-plugin`). A C ABI is not a sandbox.
|
||||
* **Public protocols** — Analysis v2 and Trace v2 with JSON Schemas and
|
||||
independent Python consumers; Tool Protocol v1 with arbitrary command names,
|
||||
SemVer negotiation and whole-bundle digest verification.
|
||||
SemVer negotiation and whole-bundle digest verification; **Execution Events
|
||||
v1** (`run/call --event-stream jsonl`) for live stdout/stderr, call frames and
|
||||
explicitly selected, bounded values (post-beta addition, ADR-0059).
|
||||
|
||||
Run everything:
|
||||
|
||||
|
||||
@@ -149,6 +149,123 @@ fn assert_envelope(events: &[Value]) {
|
||||
assert!(events_of(events, "run.started").len() <= 1);
|
||||
}
|
||||
|
||||
fn repo_root() -> PathBuf {
|
||||
Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.unwrap()
|
||||
.parent()
|
||||
.unwrap()
|
||||
.to_path_buf()
|
||||
}
|
||||
|
||||
fn python_consumer_available() -> bool {
|
||||
Command::new("python3")
|
||||
.args(["-c", "import jsonschema"])
|
||||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.status()
|
||||
.map(|status| status.success())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn checked_in_fixtures_validate_against_the_public_schema() {
|
||||
if !python_consumer_available() {
|
||||
eprintln!("python3 + jsonschema unavailable; schema consumer skipped");
|
||||
return;
|
||||
}
|
||||
let fixtures = repo_root().join("tests/consumers/fixtures/execution-events");
|
||||
let consumer = repo_root().join("tests/consumers/execution_events_consumer.py");
|
||||
for name in ["minimal.jsonl", "full.jsonl"] {
|
||||
let path = fixtures.join("valid").join(name);
|
||||
let output = Command::new("python3")
|
||||
.arg(&consumer)
|
||||
.arg(&path)
|
||||
.output()
|
||||
.expect("run execution events consumer");
|
||||
assert!(
|
||||
output.status.success(),
|
||||
"valid fixture {name} rejected: {}",
|
||||
String::from_utf8_lossy(&output.stdout)
|
||||
);
|
||||
}
|
||||
for entry in fs::read_dir(fixtures.join("invalid")).unwrap() {
|
||||
let path = entry.unwrap().path();
|
||||
let output = Command::new("python3")
|
||||
.arg(&consumer)
|
||||
.arg(&path)
|
||||
.output()
|
||||
.expect("run execution events consumer");
|
||||
assert!(
|
||||
!output.status.success(),
|
||||
"invalid fixture {} was accepted",
|
||||
path.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn structure_fixture_attributes_repeated_recursive_frames() {
|
||||
let dir = TempDir::new("structure");
|
||||
let source = fs::read_to_string(
|
||||
repo_root().join("tests/consumers/fixtures/execution-events/structure.mq.md"),
|
||||
)
|
||||
.unwrap();
|
||||
dir.write("index.mq.md", &source);
|
||||
let output = cli(
|
||||
dir.path(),
|
||||
&[
|
||||
"run",
|
||||
"index.mq.md",
|
||||
"--event-stream",
|
||||
"jsonl",
|
||||
"--observe-max-events",
|
||||
"1000",
|
||||
],
|
||||
);
|
||||
assert_eq!(
|
||||
output.status.code(),
|
||||
Some(0),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let events = parse_events(&output.stdout);
|
||||
assert_envelope(&events);
|
||||
assert_eq!(events.last().unwrap()["payload"]["value"]["decimal"], "3");
|
||||
|
||||
let enters = events_of(&events, "call.enter");
|
||||
assert_eq!(enters.len(), 4, "main + f(2) + g(2) + f(0)");
|
||||
let frames: Vec<&str> = enters
|
||||
.iter()
|
||||
.map(|event| event["payload"]["frame_id"].as_str().unwrap())
|
||||
.collect();
|
||||
let unique: std::collections::BTreeSet<&str> = frames.iter().copied().collect();
|
||||
assert_eq!(unique.len(), 4, "frame identities are run-unique");
|
||||
let main = enters[0]["payload"]["frame_id"].as_str().unwrap();
|
||||
assert!(enters[0]["payload"]["parent_frame_id"].is_null());
|
||||
let f2 = enters[1]["payload"]["frame_id"].as_str().unwrap();
|
||||
let g2 = enters[2]["payload"]["frame_id"].as_str().unwrap();
|
||||
let f0 = enters[3]["payload"]["frame_id"].as_str().unwrap();
|
||||
assert_eq!(enters[1]["payload"]["parent_frame_id"], main);
|
||||
assert_eq!(enters[2]["payload"]["parent_frame_id"], f2);
|
||||
assert_eq!(enters[3]["payload"]["parent_frame_id"], main);
|
||||
let exit_frames: Vec<&str> = events_of(&events, "call.exit")
|
||||
.iter()
|
||||
.map(|event| event["payload"]["frame_id"].as_str().unwrap())
|
||||
.collect();
|
||||
for frame in [main, f2, g2, f0] {
|
||||
assert!(exit_frames.contains(&frame), "frame {frame} must close");
|
||||
}
|
||||
assert_eq!(events_of(&events, "branch.eval").len(), 2);
|
||||
assert_eq!(
|
||||
events_of(&events, "loop.iteration")
|
||||
.iter()
|
||||
.map(|event| event["payload"]["decision"].as_str().unwrap())
|
||||
.collect::<Vec<_>>(),
|
||||
vec!["continue", "continue", "exit"]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn basic_stream_is_pure_jsonl_and_reconstructs_program_output() {
|
||||
let dir = TempDir::new("basic");
|
||||
|
||||
@@ -94,6 +94,49 @@ exports reconcile exactly with the static manifest. The independent
|
||||
frozen-binary suite is the beta gate above; the older `6a7bb98` manifest is
|
||||
historical and superseded by the release-SHA manifest.
|
||||
|
||||
## 0.5 Post-beta addition — Execution Events v1 (unreleased)
|
||||
|
||||
This section records work **after** the `v3.0.0-beta.1` tag. It is not part of
|
||||
the tagged beta archive and no new release tag has been created. The design
|
||||
input is RFC-OBS-0001 (issue `#1`), the frozen decision is ADR-0059 and the
|
||||
normative contract is [`spec/execution-events-v1.md`](../spec/execution-events-v1.md).
|
||||
|
||||
- **Implemented (P0/P1):** `run/call --event-stream jsonl` writes a pure NDJSON
|
||||
`marqdo.execution-events` v1 stream; lifecycle/terminal events; byte-exact
|
||||
base64 stdout/stderr chunks emitted at the Host commit point; run-unique
|
||||
`frame_id`/`parent_frame_id`; `return`/`branch.eval`/`loop.iteration`/`effect`
|
||||
attribution; exact-name, bounded value observation with a separate
|
||||
`--observe-text-values` consent; structural/value budget with one
|
||||
`observation.truncated`; broken-pipe failure via `observe.transport_failed`;
|
||||
Trace v2 coexistence with a shared run id and unchanged Trace schema.
|
||||
- **Unchanged:** Analysis v2 and Trace v2 schemas/versions, `--json` envelope,
|
||||
default stdout/stderr/exit codes, language semantics and capability
|
||||
authorization.
|
||||
- **Deferred:** OBS-06 memory metadata (P2), debugger/stepping/time travel,
|
||||
bidirectional debug protocol, SSE/HTTP in the CLI (View implements that).
|
||||
- **Explicitly unsupported combination:** `--mode strict` rejects
|
||||
`--event-stream` with a structured `run.rejected`; there is no downgrade.
|
||||
|
||||
Executed local gate on this tree (Linux x86_64, Rust 1.100.0):
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS: 628 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-cli --test execution_events PASS: 12 passed
|
||||
cargo test -p marqdo-host --test observation PASS: 8 passed
|
||||
cargo test -p marqdo-runtime --test observation PASS: 4 passed
|
||||
python3 tests/consumers/execution_events_consumer.py PASS on a live real-binary stream (schema, seq, terminal, byte_count)
|
||||
```
|
||||
|
||||
The Python consumer is an independent black-box parser using only the checked-in
|
||||
JSON Schema; it was run against the local binary's stream and the checked-in
|
||||
valid/invalid fixtures. This is local evidence, not a frozen release-archive or
|
||||
same-SHA CI gate. `cargo test --workspace --locked` includes the new schema
|
||||
fixture test when `python3` + `jsonschema` are available (the Woodpecker test
|
||||
lane installs `python3-jsonschema` and also runs the consumer against a real
|
||||
`--event-stream jsonl` invocation).
|
||||
|
||||
## 1. Historical: pre-beta remote baseline and evidence
|
||||
|
||||
> **Historical record.** Everything in §1 onward was executed before the
|
||||
|
||||
@@ -26,6 +26,14 @@ replace `spec/` or a recorded test log.
|
||||
> Normative boundaries: [`spec/callable-resolution.md`](../../spec/callable-resolution.md),
|
||||
> [`spec/extension-selection.md`](../../spec/extension-selection.md),
|
||||
> [`spec/resource-provider-v2.md`](../../spec/resource-provider-v2.md).
|
||||
>
|
||||
> **Post-baseline observability (ADR-0059, RFC-OBS-0001).** After the tagged
|
||||
> baseline the tree adds Execution Events v1: a typed host observation facade,
|
||||
> real-commit stdout/stderr chunks, run-unique call frames and explicitly
|
||||
> selected bounded values over `run/call --event-stream jsonl`. Analysis v2 and
|
||||
> Trace v2 are unchanged. This is locally tested on the post-baseline tree
|
||||
> ([`../STATUS.md`](../STATUS.md) §0.5) and is not part of the tagged beta
|
||||
> archive or its same-SHA evidence.
|
||||
|
||||
## 1. Four state dimensions
|
||||
|
||||
@@ -35,8 +43,8 @@ platform".
|
||||
|
||||
| Dimension | Current state | Source |
|
||||
|---|---|---|
|
||||
| Implemented | Rust workspace `3.0.0-beta.1`: RFC-LANG-0002 Markdown/frontend/IR, v2 statement parser, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, migrated bundled stdlib, CLI composition root | `Cargo.toml`, `crates/`, `stdlib/` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **599 passed, 0 failed** across 67 targets on Linux x86_64 in the RFC-EXT-0001 tree; fmt and clippy clean. Historical baseline: 507 passed at `9c1723d`. | `docs/STATUS.md`, `docs/evidence/2026-11-22-ext-implementation.md` |
|
||||
| Implemented | Rust workspace `3.0.0-beta.1`: RFC-LANG-0002 Markdown/frontend/IR, v2 statement parser, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, migrated bundled stdlib, CLI composition root; post-baseline Execution Events v1 observation (ADR-0059) | `Cargo.toml`, `crates/`, `stdlib/` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **628 passed, 0 failed, 2 ignored** on the post-baseline observability tree (Linux x86_64); `cargo fmt --check` and `clippy -D warnings` clean. Historical baseline: 599 passed before Execution Events v1, 507 at `9c1723d`. | `docs/STATUS.md` §0.5, `docs/evidence/2026-11-22-ext-implementation.md` |
|
||||
| Bundled stdlib released | In-repo bundle `marqdo-stdlib@0.1.0`, SHA-256 digest verified before loading; it is **not** an external registry package | `stdlib/stdlib.json`, `stdlib/README.md` |
|
||||
| Multi-platform verified | Linux local gate is the only executed lane here; Windows/macOS native lanes are **UNVERIFIED**, and the GitHub security lane remains a **VM-only** gate | `docs/STATUS.md` §1, §4 |
|
||||
|
||||
@@ -110,6 +118,7 @@ review RFC, not a new crate per provider.
|
||||
| Public contract | API (`.mq.md`) | State | Tests |
|
||||
|---|---|---|---|
|
||||
| stdio stdin | `resource.open("stdio","stdin",["read"])`, `resource.read`, `resource.close`; one run-scoped cursor | implemented, read-only | `resource_streams`, `p2_closure`, `tests/conformance` |
|
||||
| Execution Events v1 | `run/call --event-stream jsonl`; `--observe-variable`, `--observe-parameter`, `--observe-text-values` | implemented P0/P1 (post-beta); P2 memory metadata deferred; strict refuses the option | `execution_events` CLI, `observation` host/runtime, `execution_events_consumer.py` |
|
||||
| stdio stdout | `resource.open("stdio","stdout",["write"])`, `resource.write`, `resource.close`; separate output budget | implemented | `stdlib_bundle`, `print_routing`, `resource_streams` |
|
||||
| stdio stderr | `resource.open("stdio","stderr",["write"])`; separate from stdout and CLI diagnostics | implemented | `resource_streams`, `stdlib_product` |
|
||||
| project file read | `resource.open("file", path, ["read"])`; path/grant checked; bounded | implemented, project mode | `resource_streams`, `stdlib_product` |
|
||||
|
||||
+13
-2
@@ -16,14 +16,25 @@ The default Docker pipeline runs on push and pull request:
|
||||
```text
|
||||
test:
|
||||
rustup component add clippy rustfmt
|
||||
apt install curl/ca-certificates/python3 + Node.js 22
|
||||
apt install curl/ca-certificates/python3/python3-jsonschema + Node.js 22
|
||||
cargo fmt --all -- --check
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
cargo build --locked -p marqdo-provider-worker
|
||||
cargo test --workspace --locked
|
||||
cargo run --locked -p marqdo-cli -- run \
|
||||
tests/consumers/fixtures/execution-events/basic.mq.md --event-stream jsonl \
|
||||
> /tmp/execution-events.jsonl
|
||||
python3 tests/consumers/execution_events_consumer.py /tmp/execution-events.jsonl
|
||||
cargo test -p marqdo-cli --test docs_consistency
|
||||
cargo test -p marqdo-cli --test architecture_boundaries
|
||||
cargo test -p marqdo-cli --test stdlib_product
|
||||
python3 -m unittest discover -s tools/ci -p "test_*.py"
|
||||
rustup target add x86_64-pc-windows-gnu
|
||||
cargo check --workspace --all-targets --target x86_64-pc-windows-gnu
|
||||
|
||||
build (depends on test):
|
||||
cargo build --release --locked -p marqdo-cli
|
||||
cargo build --release --locked -p marqdo-cli -p marqdo-provider-worker
|
||||
package + cold-install smoke (which now validates live Execution Events v1)
|
||||
```
|
||||
|
||||
Node.js 22 is required because the optional foreign-kernel tests use ES-module
|
||||
|
||||
@@ -7,9 +7,9 @@ core checkout, or host memory layout. Adding a new domain function must not
|
||||
require a core commit.
|
||||
|
||||
Product version for these docs: **`3.0.0-beta.1`** (Linux x86_64 beta).
|
||||
Protocol versions are independent: Analysis v2, Trace v2, Tool v1, Plugin ABI
|
||||
v1, Foreign wire v1, Provider manifest v2, lock v4, markdown profile
|
||||
`marqdo.md.v1`.
|
||||
Protocol versions are independent: Analysis v2, Trace v2, Execution Events v1,
|
||||
Tool v1, Plugin ABI v1, Foreign wire v1, Provider manifest v2, lock v4,
|
||||
markdown profile `marqdo.md.v1`.
|
||||
|
||||
## Pick your entry point
|
||||
|
||||
@@ -33,7 +33,7 @@ v1, Foreign wire v1, Provider manifest v2, lock v4, markdown profile
|
||||
4. [extensions.md](extensions.md) — plugins, foreign kernels, resource
|
||||
providers, extension selection and lock v4.
|
||||
5. [analysis-trace.md](analysis-trace.md) — the public static/dynamic data
|
||||
contracts and digest binding.
|
||||
contracts, Execution Events v1 and digest binding.
|
||||
6. [compatibility.md](compatibility.md) — version axes, platform matrix and the
|
||||
prerelease policy.
|
||||
7. [release-limits.md](release-limits.md) — the honest beta limitations (this
|
||||
|
||||
@@ -60,6 +60,51 @@ marqdo run index.mq.md --trace-file ./trace.json
|
||||
`spec/schemas/trace-v2.schema.json`; consumer:
|
||||
`tests/consumers/trace_consumer.py`.
|
||||
|
||||
## Execution Events v1 (live)
|
||||
|
||||
```bash
|
||||
marqdo run index.mq.md --event-stream jsonl
|
||||
marqdo call index.mq.md main --arg n=5 --event-stream jsonl
|
||||
marqdo run index.mq.md --event-stream jsonl --trace-file ./trace.json
|
||||
```
|
||||
|
||||
- `schema: "marqdo.execution-events"`, `schema_version: 1`. This is a separate
|
||||
protocol from Analysis/Trace and has its own version axis.
|
||||
- While active, the CLI stdout is a **pure NDJSON channel**: one JSON object per
|
||||
line, `seq` starts at 0 and is strictly increasing. Program stdout/stderr are
|
||||
only `stdout.chunk`/`stderr.chunk` events with `encoding: "base64"`,
|
||||
`data_base64` and a matching `byte_count`. Concatenated decoded bytes equal a
|
||||
normal run's output; invalid UTF-8/NUL/binary bytes are preserved.
|
||||
- `stream.opened` is always seq 0. A run then has at most one `run.started` and
|
||||
exactly one terminal event (`run.completed`, `run.failed`, `run.rejected`) on
|
||||
a complete stream. An EOF without a terminal event is `incomplete`
|
||||
(SIGKILL/crash/broken pipe); never synthesize success.
|
||||
- `run.started.source` contains `root_id` and the `sha256:` digest of the exact
|
||||
Analysis v2 bytes. `--trace-file` may be combined; both writers share one
|
||||
opaque run id, and Trace v2 remains atomic and value-free.
|
||||
- Structural events: `call.enter`/`call.exit` (`frame_id`, `parent_frame_id`),
|
||||
`return`, `branch.eval`, `loop.iteration`, `effect`. Frames identify call
|
||||
instances, not functions; a failed argument binding creates no frame.
|
||||
- Values are off by default (`value_state: not_requested`).
|
||||
`--observe-variable NAME` and `--observe-parameter NAME` are exact-name
|
||||
selections. `Int`/`Float`/`Bool` are bounded by
|
||||
`--observe-max-value-bytes` (default 256); selected `Text` is `redacted` until
|
||||
`--observe-text-values` is also passed. Containers, `Memory`, resources and
|
||||
plugin handles are `unsupported`; handles are never serialized.
|
||||
- Structural/value events have a separate budget
|
||||
(`--observe-max-events`, `--observe-max-bytes`; defaults 100 000 / 16 MiB).
|
||||
Exhaustion emits one `observation.truncated` and still delivers stdout/stderr
|
||||
and the terminal event. A broken output pipe fails the run with
|
||||
`observe.transport_failed`; it never keeps executing side effects silently.
|
||||
- `--event-stream` is mutually exclusive with `--json`, rejected by
|
||||
`check`/`inspect`/`fmt`/`tool`, and currently rejected by `--mode strict`
|
||||
(fail-closed, no downgrade). It is not a capability grant.
|
||||
- Contract: [`../../spec/execution-events-v1.md`](../../spec/execution-events-v1.md);
|
||||
schema: `spec/schemas/execution-events-v1.schema.json`; consumer:
|
||||
`tests/consumers/execution_events_consumer.py`; fixtures and Rust gate:
|
||||
`tests/consumers/fixtures/execution-events/` and
|
||||
`crates/cli/tests/execution_events.rs`.
|
||||
|
||||
## Consumer rules
|
||||
|
||||
1. Reject unknown schema majors; there is no silent "use latest" fallback.
|
||||
|
||||
@@ -11,6 +11,7 @@ This page summarizes the public support policy. The authoritative matrix is
|
||||
| Product / CLI | `3.0.0-beta.1` | SemVer prerelease; breaking changes inside the 3.0 beta line are allowed and appear in `CHANGELOG.md`. |
|
||||
| Analysis | v2 | New major for incompatible changes; consumers reject unknown majors. |
|
||||
| Trace | v2 | New major for incompatible changes; binds to exact Analysis bytes. |
|
||||
| Execution Events | v1 | One-line NDJSON stream; new major for incompatible required-field/ordering/redaction changes. Consumers reject unknown majors. |
|
||||
| CLI Tool | v1 | Frozen manifest/negotiation/bundle contract for v1. |
|
||||
| Plugin ABI | v1 (`1.0`) | Frozen C layout and version negotiation. |
|
||||
| Foreign wire | v1 | One-shot request envelope. |
|
||||
@@ -35,6 +36,9 @@ ranges**. Beta cores require a prerelease range such as
|
||||
|
||||
Strict is Linux-only and probe-gated. Plugin/provider worker transports are not
|
||||
OS sandboxes. `pure`/`project` are execution modes, not security boundaries.
|
||||
Execution Events v1 is a live observation protocol: it is not a capability
|
||||
grant, values are off by default, and `--mode strict` currently refuses
|
||||
`--event-stream` explicitly (no downgrade).
|
||||
|
||||
## Language compatibility
|
||||
|
||||
|
||||
@@ -61,8 +61,10 @@ Even when available:
|
||||
* `3.0.0-beta.1` is a **prerelease**, not stable and not LTS. Breaking changes
|
||||
inside the 3.0 beta series are allowed and announced in the changelog.
|
||||
* The product version is independent of protocol versions: Analysis v2, Trace
|
||||
v2, Tool v1, Plugin ABI v1, Foreign wire v1, Provider manifest v2 and lock v4
|
||||
keep their own major numbers. Do not infer them from `3.0.0-beta.1`.
|
||||
v2, Execution Events v1, Tool v1, Plugin ABI v1, Foreign wire v1, Provider
|
||||
manifest v2 and lock v4 keep their own major numbers. Do not infer them from
|
||||
`3.0.0-beta.1`. Execution Events v1 is a post-beta addition and is not in the
|
||||
tagged beta archive.
|
||||
* The official bundled stdlib keeps its independent package version `0.1.0`.
|
||||
A plugin `.mq.md` package version such as `0.1.0` is also independent.
|
||||
* `project.marqdo` in `marqdo.yaml` is enforced in this release. SemVer
|
||||
@@ -80,6 +82,20 @@ Archives are accompanied by SHA256 checksums and `RELEASE_INFO.json`
|
||||
integrity only; they are **not** a publisher signature. A digest obtained from
|
||||
an untrusted Git URL does not make the code trustworthy.
|
||||
|
||||
## Execution Events v1 (post-beta addition)
|
||||
|
||||
* Implemented: live stdout/stderr chunks, lifecycle/terminal events, call
|
||||
instance ids, branch/loop/effect events, explicit bounded value observation.
|
||||
* Not implemented: memory region/budget metadata (OBS-06/P2), raw memory
|
||||
content, breakpoints/stepping/time travel, and a bidirectional debug
|
||||
protocol. Do not infer a debugger from this protocol.
|
||||
* `--mode strict` currently rejects `--event-stream` explicitly; there is no
|
||||
silent fallback to `project`.
|
||||
* The live lane is tested on Linux x86_64; macOS/Windows are unverified for
|
||||
this option like the rest of the beta CLI.
|
||||
* Observation lowers confidentiality when values are explicitly enabled. Text
|
||||
needs a second opt-in; View/agents must not enable these flags by default.
|
||||
|
||||
## Deliberately deferred
|
||||
|
||||
strict L2, Wasm plugins, macOS/Windows provider worker, automatic uv/npm
|
||||
|
||||
@@ -91,6 +91,20 @@ Dynamic input:
|
||||
"$MARQDO_BIN" run index.mq.md --trace-file ./trace.json
|
||||
```
|
||||
|
||||
Live dynamic input (Execution Events v1):
|
||||
|
||||
```bash
|
||||
"$MARQDO_BIN" run index.mq.md --event-stream jsonl
|
||||
"$MARQDO_BIN" call index.mq.md main --arg n=5 --event-stream jsonl
|
||||
```
|
||||
|
||||
Consume the stdout line stream as NDJSON; do not pass `--json` in the same
|
||||
invocation. Negotiate by reading `stream.opened` (`marqdo.execution-events`,
|
||||
`schema_version: 1`) and reject unknown majors. A tool server may convert the
|
||||
stream to its own SSE/websocket transport; the CLI never implements HTTP/SSE.
|
||||
This option is unavailable under `--mode strict` in the current tree and the
|
||||
tool must surface that refusal instead of pretending real-time support.
|
||||
|
||||
See [analysis-trace.md](analysis-trace.md) for fields and digest rules. Do not
|
||||
depend on `inspect --ir`: that JSON is not a frozen cross-repository protocol.
|
||||
The `view` tool renders the **original Markdown**, not the Analysis JSON; the
|
||||
|
||||
@@ -18,6 +18,9 @@ design input is [RFC-OBS-0001](../docs/architecture/RFC-OBS-0001-runtime-observa
|
||||
* `--event-stream jsonl` is available only on `run` and `call`. It is
|
||||
mutually exclusive with `--json`; `check`, `inspect`, `fmt`, `tool` and the
|
||||
other static commands reject the option. `--trace-file` may be combined.
|
||||
* `--mode strict` currently rejects `--event-stream` with a structured
|
||||
`run.rejected` (`strict.unsupported_grant`); there is no silent downgrade to
|
||||
`project`.
|
||||
* The consumer checks the negotiated protocol from `stream.opened`
|
||||
(`schema` + `schema_version`). Consumers **must reject unknown major
|
||||
versions**, and must not silently treat another major as v1.
|
||||
@@ -204,7 +207,9 @@ truncated text the object also carries `"truncated": true` and
|
||||
Lists, maps, tables, formulas, code, memory, resources and plugin handles are
|
||||
`unsupported`; they are never recursed into and handles are never serialized.
|
||||
A text value is never published without the explicit
|
||||
`--observe-text-values` consent, even when the name was selected.
|
||||
`--observe-text-values` consent, even when the name was selected. A
|
||||
`--observe-max-value-bytes` value of `0` means unlimited; otherwise text is
|
||||
cut only at a UTF-8 scalar boundary and marked `truncated`.
|
||||
|
||||
`variable.assigned` is emitted only after the assignment was actually
|
||||
committed. It never re-evaluates the right-hand side and never back-fills the
|
||||
|
||||
@@ -51,6 +51,7 @@ Rust harnesses (all in `crates/cli/tests/`):
|
||||
| `negative.rs` | Current-syntax diagnostics are **hard errors** (never prose fallbacks): exact sorted `sem.*`/`symbol.*`/`call.*`/`frontmatter.*` code lists, the `sem.inferred_signature` warning model with inert-knowledge controls, and runtime error codes — each asserted in its own suite. |
|
||||
| `properties.rs` | Blueprint §14.2 prose invariance: inserting/removing pure narration paragraphs never changes the executable function IR (modulo source spans) and diagnostics stay error-free. |
|
||||
| `integration_cli.rs` | Real CLI contract: `init`, `check`, `run`, `call`, `inspect`, `fmt`, multi-module imports/cycles, `lock`/`sync --locked`, plugin inspection and explicitly trusted plugin invocation. |
|
||||
| `execution_events.rs` | Execution Events v1 black-box contract: pure JSONL, seq/terminal ordering, byte-exact base64 chunks, live delivery before exit, value redaction/selection, trace coexistence, truncation, output budget and broken-pipe failure. |
|
||||
|
||||
## Running
|
||||
|
||||
@@ -75,6 +76,21 @@ cargo test -p marqdo-cli --test properties
|
||||
cargo test -p marqdo-cli --test integration_cli
|
||||
```
|
||||
|
||||
Execution Events v1 (RFC-OBS-0001, ADR-0059) is pinned by:
|
||||
|
||||
* `crates/cli/tests/execution_events.rs` — real-binary NDJSON contract, the
|
||||
checked-in JSON Schema fixtures, the structure fixture, live output, values,
|
||||
trace coexistence and a broken pipe;
|
||||
* `crates/marqdo-host/tests/observation.rs` — real commit-point chunks, frame
|
||||
pairing, budget truncation and the broken-sink failure boundary;
|
||||
* `crates/marqdo-runtime/tests/observation.rs` — recursive/repeated call frames,
|
||||
branch/loop/assignment attribution and exact value selection;
|
||||
* `tests/consumers/execution_events_consumer.py` and
|
||||
`tests/consumers/fixtures/execution-events/` — the independent Python consumer
|
||||
and the RFC §13 fixtures. `python3` + `jsonschema` enable the schema test and
|
||||
the Woodpecker lane runs the consumer against a real `--event-stream jsonl`
|
||||
invocation.
|
||||
|
||||
The power/control-flow slice is pinned by:
|
||||
|
||||
* AST fixtures 151–160: code-span vs tight/spaced `**`, multi-backtick spans,
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# Execution Events v1 fixtures
|
||||
|
||||
These are the minimal executable fixtures from RFC-OBS-0001 §13 plus the
|
||||
schema-validation fixtures used by
|
||||
[`execution_events_consumer.py`](../../execution_events_consumer.py).
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `basic.mq.md` | Fixture A: live stdout ordering and return value. |
|
||||
| `realtime.mq.md` | Fixture B: `READY` must be observable before the run exits. |
|
||||
| `structure.mq.md` | Fixture C: `main -> f -> g`, repeated calls, branch, loop. |
|
||||
| `values.mq.md` | Fixture D: default privacy and exact value selection. |
|
||||
| `valid/*.jsonl` | Schema-valid streams (minimal and full event table). |
|
||||
| `invalid/*.json` | One-object negative fixtures: unknown major, missing/`negative` seq, bad chunk/span/terminal. |
|
||||
|
||||
The `.mq.md` files are inputs to the released CLI:
|
||||
|
||||
```bash
|
||||
marqdo run tests/consumers/fixtures/execution-events/basic.mq.md --event-stream jsonl
|
||||
python3 tests/consumers/execution_events_consumer.py /tmp/events.jsonl
|
||||
```
|
||||
@@ -0,0 +1,11 @@
|
||||
# @knowledge Execution Events fixture A
|
||||
|
||||
## @exec main
|
||||
|
||||
`打印("START")`
|
||||
|
||||
`n = 10`
|
||||
|
||||
`打印("DONE")`
|
||||
|
||||
`return n`
|
||||
@@ -0,0 +1,17 @@
|
||||
# @knowledge Execution Events fixture B
|
||||
|
||||
## @exec main
|
||||
|
||||
`打印("READY")`
|
||||
|
||||
`i = 0`
|
||||
|
||||
`while i < 2000000`
|
||||
|
||||
`i = i + 1`
|
||||
|
||||
`end`
|
||||
|
||||
`打印("FINISH")`
|
||||
|
||||
`return i`
|
||||
@@ -0,0 +1,35 @@
|
||||
# @knowledge Execution Events fixture C
|
||||
|
||||
## @exec g
|
||||
|
||||
`params(n: Int)`
|
||||
|
||||
`return n + 1`
|
||||
|
||||
## @exec f
|
||||
|
||||
`params(n: Int)`
|
||||
|
||||
`if n > 0`
|
||||
|
||||
`return g(n=n)`
|
||||
|
||||
`end`
|
||||
|
||||
`return n`
|
||||
|
||||
## @exec main
|
||||
|
||||
`i = 0`
|
||||
|
||||
`while i < 2`
|
||||
|
||||
`i = i + 1`
|
||||
|
||||
`end`
|
||||
|
||||
`a = f(n=2)`
|
||||
|
||||
`b = f(n=0)`
|
||||
|
||||
`return a + b`
|
||||
@@ -0,0 +1,9 @@
|
||||
# @knowledge Execution Events fixture D
|
||||
|
||||
## @exec main
|
||||
|
||||
`secret_token = "DO_NOT_LEAK_SENTINEL_1"`
|
||||
|
||||
`visible_count = 42`
|
||||
|
||||
`return visible_count`
|
||||
@@ -185,6 +185,48 @@ def main(argv: list[str]) -> int:
|
||||
result = subprocess.run(command, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
fail(f"{consumer} failed:\n{result.stdout}\n{result.stderr}")
|
||||
# Execution Events v1: the released binary's live NDJSON stream,
|
||||
# validated by the independent consumer shipped in the archive.
|
||||
run_help = runner.run(["run", "--help"]).stdout
|
||||
if "--event-stream" not in run_help or "--observe-variable" not in run_help:
|
||||
fail("run --help does not advertise Execution Events v1 options")
|
||||
live = runner.run(
|
||||
[
|
||||
"run",
|
||||
"tests/consumers/fixtures/execution-events/basic.mq.md",
|
||||
"--event-stream",
|
||||
"jsonl",
|
||||
]
|
||||
)
|
||||
events_path = install / "execution-events.jsonl"
|
||||
events_path.write_text(live.stdout, encoding="utf-8")
|
||||
events = [
|
||||
json.loads(line)
|
||||
for line in live.stdout.splitlines()
|
||||
if line.strip()
|
||||
]
|
||||
if not events or events[0].get("kind") != "stream.opened":
|
||||
fail("Execution Events stream does not start with stream.opened")
|
||||
if events[-1].get("kind") != "run.completed":
|
||||
fail("Execution Events stream has no successful terminal event")
|
||||
validate_events = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
str(install / "tests" / "consumers" / "execution_events_consumer.py"),
|
||||
str(events_path),
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
if validate_events.returncode != 0:
|
||||
if "jsonschema" in validate_events.stderr and "ModuleNotFoundError" in validate_events.stderr:
|
||||
print("smoke: note: jsonschema is not installed; Execution Events schema validation skipped")
|
||||
else:
|
||||
fail(
|
||||
"execution_events_consumer.py failed:\n"
|
||||
f"{validate_events.stdout}\n{validate_events.stderr}"
|
||||
)
|
||||
|
||||
validate = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
|
||||
Reference in new issue
Block a user