diff --git a/CHANGELOG.md b/CHANGELOG.md index 994d760..2c9f677 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -127,6 +127,8 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601. `std:` resolution, tamper rejection and frozen-binary hash equality. - P2 closure gate: add a pure no-Foreign/no-Plugin lane for C01–C06/C11/C12 (20 fixtures + stdlib table), source shortcut scan and golden comparison. +- P6 baseline profile: add `tools/perf/bench.py` and a checked-in 7-workload + JSON report; no bytecode/JIT/native backend is introduced. - P4 cross-storage table equivalence: fixture 72 compares row-store Table sort/filter output with a `.mq.md` columnar store cell-by-cell. - P5 HTTP framing/client: `stdlib/net/http.mq.md` parses status/headers/chunked diff --git a/docs/STATUS.md b/docs/STATUS.md index ad2cb97..f5f39c2 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -147,6 +147,9 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL providers register additional resource kinds without parser/runtime/HostOp changes; project-mode + name grant required; pure/strict refuse provider kinds; C/Worker transport remains open. +- P6 baseline profile: `tools/perf/bench.py` measures 7 `.mq.md` workloads with + wall time and peak RSS; no optimizer/backend was added and P6 is not claimed + complete. - Cross-storage table equivalence (P4, ADR-0032): fixture 72 compares T0 row Table results with a `.mq.md` columnar List/Map store for sort/filter. - Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md` @@ -207,7 +210,7 @@ behavior. This table records only what is in this tree now. | P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Still missing: directories, network, generic provider registry, Table storage, strict file/network resources. | | P4 Table logical/physical migration | **PARTIAL** | ADR-0032/CM-A08: `Value::Table` logical semantics unchanged (T0 `Vec>`); `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group/`转GFM` as value-returning `.mq.md` functions with input immutability. GFM rendering and cross-storage equivalence (fixture 72: row Table vs `.mq.md` columnar store, same headers/rows/cells) are covered. No T1/T2/T3 physical storage, native column store, zero-copy views or Memory-backed table yet. | | P5 third-party Provider / TCP / HTTP | **PARTIAL** | `stdlib/net/http.mq.md` + semantic fixture 71 implement HTTP/1.1 status/header/chunked parsing and request construction in `.mq.md`; `http_over_tcp.rs` runs a local HTTP exchange over `resource.open("tcp",...)`; no host HTTP/TLS and no external network test. ADR-0034 blocking TCP resource endpoint (`resource.open("tcp", ":", ["read","write"])`, exact IP grant, project-only, 5s timeouts; loopback host+CLI tests). ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, host DNS/TLS/HTTP implementation, UDP, clock/random/process/device providers, or provider package installation. | -| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work | +| P6 performance / native backends | **BASELINE ONLY** | reproducible profile harness `tools/perf/bench.py` and [`p6-profile`](evidence/2026-11-20-p6-profile.md) for 7 workloads; no bytecode/JIT/columnar numeric backend or equivalence/optimization gate yet | The P1 memory domain is pure computation: it needs no grant and is available in `pure`, `project` and `strict` (bounded by the memory budgets). It adds no diff --git a/docs/evidence/2026-11-20-p6-profile.json b/docs/evidence/2026-11-20-p6-profile.json new file mode 100644 index 0000000..2c8fa6a --- /dev/null +++ b/docs/evidence/2026-11-20-p6-profile.json @@ -0,0 +1,57 @@ +{ + "schema": "marqdo.perf-baseline", + "schema_version": 1, + "binary": ".cache/target/release/marqdo", + "iterations": 3, + "workloads": [ + { + "name": "allocator-free-list", + "source": "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md", + "seconds_median": 0.0173, + "seconds_min": 0.017026568995788693, + "max_rss_kb_median": 8428 + }, + { + "name": "json-parser", + "source": "tests/conformance/semantics/62-json-array-parser.mq.md", + "seconds_median": 0.0188, + "seconds_min": 0.01880078506655991, + "max_rss_kb_median": 8960 + }, + { + "name": "table-sort", + "source": "tests/conformance/semantics/65-table-sort-stdlib.mq.md", + "seconds_median": 0.0152, + "seconds_min": 0.014747997978702188, + "max_rss_kb_median": 7476 + }, + { + "name": "bytecode-vm", + "source": "tests/conformance/semantics/63-memory-bytecode-vm.mq.md", + "seconds_median": 0.0155, + "seconds_min": 0.015298803802579641, + "max_rss_kb_median": 7692 + }, + { + "name": "matrix-multiply", + "source": "tests/conformance/semantics/70-matrix-multiply.mq.md", + "seconds_median": 0.0151, + "seconds_min": 0.014752828981727362, + "max_rss_kb_median": 7368 + }, + { + "name": "http-parser", + "source": "tests/conformance/semantics/71-http-parser.mq.md", + "seconds_median": 0.018, + "seconds_min": 0.017968795029446483, + "max_rss_kb_median": 8196 + }, + { + "name": "stdlib-print-demo", + "source": "stdlib/demo.mq.md", + "seconds_median": 0.0205, + "seconds_min": 0.02037098305299878, + "max_rss_kb_median": 9636 + } + ] +} diff --git a/docs/evidence/2026-11-20-p6-profile.md b/docs/evidence/2026-11-20-p6-profile.md new file mode 100644 index 0000000..1352839 --- /dev/null +++ b/docs/evidence/2026-11-20-p6-profile.md @@ -0,0 +1,53 @@ +# RFC-CM P6 baseline profile + +Date: 2026-11-20 (Asia/Shanghai) +Repository: `marqdo-cli` +Baseline: `b020904` plus this step's changes. +Scope: **baseline measurement only**; no bytecode/JIT/native backend was added. + +## Harness + +`tools/perf/bench.py` builds `marqdo-cli --release`, runs each workload three +times through `/usr/bin/time -f "%e %M"`, measures wall time with +`time.perf_counter`, verifies `--json` success and writes a schema-tagged JSON +report. The checked-in report is +[`2026-11-20-p6-profile.json`](2026-11-20-p6-profile.json). + +Environment: Ubuntu 26.04 container, Linux x86_64, Rust 1.100.0, release +profile, 7 selected `.mq.md` workloads. These are small conformance workloads, +so most times are dominated by process/startup and Markdown parsing; they are +not throughput benchmarks of large scientific problems. + +## Results (median of 3) + +| Workload | Wall s | Peak RSS KiB | +|---|---:|---:| +| allocator-free-list | 0.0173 | 8428 | +| json-parser | 0.0188 | 8960 | +| table-sort | 0.0152 | 7476 | +| bytecode-vm | 0.0155 | 7692 | +| matrix-multiply | 0.0151 | 7368 | +| http-parser | 0.0180 | 8196 | +| stdlib-print-demo | 0.0205 | 9636 | + +## Interpretation + +* Startup + parse + load dominate these small workloads; the interpreter's + per-byte/per-call cost is visible mainly in the VM/matrix/table loops. +* `stdlib-print-demo` is highest RSS/wall because it loads the bundled stdlib + modules and formats all value types through `.mq.md`. +* The current release binary is a tree-walking Rust interpreter; no bytecode + compiler, JIT, columnar numeric backend or copy-elision pass exists. +* No claim of P6 pass: the RFC requires an optimized implementation that is + equivalent to the reference interpreter and reproducible performance/budget + evidence. This report only establishes the baseline and a reproducible + harness for future work. + +## Reproduce + +```bash +export PATH=/root/.cargo/bin:$PATH +export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo +export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target +python3 tools/perf/bench.py > /tmp/marqdo-perf.json +``` diff --git a/tests/README.md b/tests/README.md index aa37d0f..c273f13 100644 --- a/tests/README.md +++ b/tests/README.md @@ -185,6 +185,9 @@ grant, pure/strict, bad protocol version, reserved kinds and duplicate kinds are rejected. RFC-CM P2 closure (C01–C06, C11, C12) is pinned by +`tools/perf/bench.py` is the P6 baseline harness (not a test; it is not part of +the cargo gate). + `crates/cli/tests/p2_closure.rs`: 20 semantic fixtures run through `run --mode pure --json`, sources are scanned for foreign/plugin/import shortcuts, and value/stdout are compared with the goldens; the stdlib table diff --git a/tools/perf/bench.py b/tools/perf/bench.py new file mode 100755 index 0000000..b22d163 --- /dev/null +++ b/tools/perf/bench.py @@ -0,0 +1,100 @@ +#!/usr/bin/env python3 +"""Reproducible P6 baseline profile for selected .mq.md workloads. + +Builds the release CLI, runs each workload three times through /usr/bin/time, +and writes a JSON report with wall seconds and peak RSS. This is a baseline +profile, not a claim that P6 optimization is complete. +""" + +import json +import os +import statistics +import subprocess +import sys +import tempfile +import time +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[2] +target_dir = Path(os.environ.get("CARGO_TARGET_DIR", ROOT / "target")) +if not target_dir.is_absolute(): + target_dir = ROOT / target_dir +BIN = target_dir / "release" / "marqdo" +WORKLOADS = [ + ("allocator-free-list", "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md"), + ("json-parser", "tests/conformance/semantics/62-json-array-parser.mq.md"), + ("table-sort", "tests/conformance/semantics/65-table-sort-stdlib.mq.md"), + ("bytecode-vm", "tests/conformance/semantics/63-memory-bytecode-vm.mq.md"), + ("matrix-multiply", "tests/conformance/semantics/70-matrix-multiply.mq.md"), + ("http-parser", "tests/conformance/semantics/71-http-parser.mq.md"), + ("stdlib-print-demo", "stdlib/demo.mq.md"), +] + + +def run_once(relative: str) -> dict: + with tempfile.NamedTemporaryFile("r", delete=False) as handle: + time_path = handle.name + command = [ + "/usr/bin/time", + "-f", + "%e %M", + "-o", + time_path, + str(BIN), + "run", + relative, + "--json", + ] + env = dict(os.environ) + env["MARQDO_STDLIB_DIR"] = str(ROOT / "stdlib") + started = time.perf_counter() + output = subprocess.run( + command, cwd=ROOT, env=env, capture_output=True, text=True, timeout=120 + ) + elapsed = time.perf_counter() - started + timing = Path(time_path).read_text().split() + Path(time_path).unlink(missing_ok=True) + if output.returncode != 0: + raise RuntimeError(f"{relative} failed: {output.stderr}") + report = json.loads(output.stdout) + if not report.get("success"): + raise RuntimeError(f"{relative} reported failure") + return { + "seconds": elapsed, + "max_rss_kb": int(timing[1]), + } + + +def main() -> int: + subprocess.run( + ["cargo", "build", "--release", "-p", "marqdo-cli"], + cwd=ROOT, + check=True, + ) + report = { + "schema": "marqdo.perf-baseline", + "schema_version": 1, + "binary": str(BIN.relative_to(ROOT)), + "iterations": 3, + "workloads": [], + } + for name, relative in WORKLOADS: + runs = [run_once(relative) for _ in range(3)] + report["workloads"].append( + { + "name": name, + "source": relative, + "seconds_median": round(statistics.median(r["seconds"] for r in runs), 4), + "seconds_min": min(r["seconds"] for r in runs), + "max_rss_kb_median": int( + statistics.median(r["max_rss_kb"] for r in runs) + ), + } + ) + json.dump(report, sys.stdout, ensure_ascii=False, indent=2) + sys.stdout.write("\n") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main())