perf(p6): add a reproducible baseline profile harness
- tools/perf/bench.py builds the release CLI and measures 7 .mq.md workloads three times with wall time and peak RSS, validating --json success. - Check in the schema-tagged JSON report and interpretation; no bytecode/JIT/ native backend was added, so P6 remains baseline-only, not complete. - Update STATUS/changelog/tests README. Gate: fmt, clippy -D warnings, 430 tests passed.
This commit is contained in:
1 parent
b020904199
commit
4c8e7fd8ab
6 files changed
+219
-1
No files matched your search
+4
-1
@@ -147,6 +147,9 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
providers register additional resource kinds without parser/runtime/HostOp
|
||||
changes; project-mode + name grant required; pure/strict refuse provider
|
||||
kinds; C/Worker transport remains open.
|
||||
- P6 baseline profile: `tools/perf/bench.py` measures 7 `.mq.md` workloads with
|
||||
wall time and peak RSS; no optimizer/backend was added and P6 is not claimed
|
||||
complete.
|
||||
- Cross-storage table equivalence (P4, ADR-0032): fixture 72 compares T0 row
|
||||
Table results with a `.mq.md` columnar List/Map store for sort/filter.
|
||||
- Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md`
|
||||
@@ -207,7 +210,7 @@ behavior. This table records only what is in this tree now.
|
||||
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Still missing: directories, network, generic provider registry, Table storage, strict file/network resources. |
|
||||
| P4 Table logical/physical migration | **PARTIAL** | ADR-0032/CM-A08: `Value::Table` logical semantics unchanged (T0 `Vec<Vec<Value>>`); `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group/`转GFM` as value-returning `.mq.md` functions with input immutability. GFM rendering and cross-storage equivalence (fixture 72: row Table vs `.mq.md` columnar store, same headers/rows/cells) are covered. No T1/T2/T3 physical storage, native column store, zero-copy views or Memory-backed table yet. |
|
||||
| P5 third-party Provider / TCP / HTTP | **PARTIAL** | `stdlib/net/http.mq.md` + semantic fixture 71 implement HTTP/1.1 status/header/chunked parsing and request construction in `.mq.md`; `http_over_tcp.rs` runs a local HTTP exchange over `resource.open("tcp",...)`; no host HTTP/TLS and no external network test. ADR-0034 blocking TCP resource endpoint (`resource.open("tcp", "<ip>:<port>", ["read","write"])`, exact IP grant, project-only, 5s timeouts; loopback host+CLI tests). ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, host DNS/TLS/HTTP implementation, UDP, clock/random/process/device providers, or provider package installation. |
|
||||
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
|
||||
| P6 performance / native backends | **BASELINE ONLY** | reproducible profile harness `tools/perf/bench.py` and [`p6-profile`](evidence/2026-11-20-p6-profile.md) for 7 workloads; no bytecode/JIT/columnar numeric backend or equivalence/optimization gate yet |
|
||||
|
||||
The P1 memory domain is pure computation: it needs no grant and is available in
|
||||
`pure`, `project` and `strict` (bounded by the memory budgets). It adds no
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
{
|
||||
"schema": "marqdo.perf-baseline",
|
||||
"schema_version": 1,
|
||||
"binary": ".cache/target/release/marqdo",
|
||||
"iterations": 3,
|
||||
"workloads": [
|
||||
{
|
||||
"name": "allocator-free-list",
|
||||
"source": "tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md",
|
||||
"seconds_median": 0.0173,
|
||||
"seconds_min": 0.017026568995788693,
|
||||
"max_rss_kb_median": 8428
|
||||
},
|
||||
{
|
||||
"name": "json-parser",
|
||||
"source": "tests/conformance/semantics/62-json-array-parser.mq.md",
|
||||
"seconds_median": 0.0188,
|
||||
"seconds_min": 0.01880078506655991,
|
||||
"max_rss_kb_median": 8960
|
||||
},
|
||||
{
|
||||
"name": "table-sort",
|
||||
"source": "tests/conformance/semantics/65-table-sort-stdlib.mq.md",
|
||||
"seconds_median": 0.0152,
|
||||
"seconds_min": 0.014747997978702188,
|
||||
"max_rss_kb_median": 7476
|
||||
},
|
||||
{
|
||||
"name": "bytecode-vm",
|
||||
"source": "tests/conformance/semantics/63-memory-bytecode-vm.mq.md",
|
||||
"seconds_median": 0.0155,
|
||||
"seconds_min": 0.015298803802579641,
|
||||
"max_rss_kb_median": 7692
|
||||
},
|
||||
{
|
||||
"name": "matrix-multiply",
|
||||
"source": "tests/conformance/semantics/70-matrix-multiply.mq.md",
|
||||
"seconds_median": 0.0151,
|
||||
"seconds_min": 0.014752828981727362,
|
||||
"max_rss_kb_median": 7368
|
||||
},
|
||||
{
|
||||
"name": "http-parser",
|
||||
"source": "tests/conformance/semantics/71-http-parser.mq.md",
|
||||
"seconds_median": 0.018,
|
||||
"seconds_min": 0.017968795029446483,
|
||||
"max_rss_kb_median": 8196
|
||||
},
|
||||
{
|
||||
"name": "stdlib-print-demo",
|
||||
"source": "stdlib/demo.mq.md",
|
||||
"seconds_median": 0.0205,
|
||||
"seconds_min": 0.02037098305299878,
|
||||
"max_rss_kb_median": 9636
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
# RFC-CM P6 baseline profile
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `b020904` plus this step's changes.
|
||||
Scope: **baseline measurement only**; no bytecode/JIT/native backend was added.
|
||||
|
||||
## Harness
|
||||
|
||||
`tools/perf/bench.py` builds `marqdo-cli --release`, runs each workload three
|
||||
times through `/usr/bin/time -f "%e %M"`, measures wall time with
|
||||
`time.perf_counter`, verifies `--json` success and writes a schema-tagged JSON
|
||||
report. The checked-in report is
|
||||
[`2026-11-20-p6-profile.json`](2026-11-20-p6-profile.json).
|
||||
|
||||
Environment: Ubuntu 26.04 container, Linux x86_64, Rust 1.100.0, release
|
||||
profile, 7 selected `.mq.md` workloads. These are small conformance workloads,
|
||||
so most times are dominated by process/startup and Markdown parsing; they are
|
||||
not throughput benchmarks of large scientific problems.
|
||||
|
||||
## Results (median of 3)
|
||||
|
||||
| Workload | Wall s | Peak RSS KiB |
|
||||
|---|---:|---:|
|
||||
| allocator-free-list | 0.0173 | 8428 |
|
||||
| json-parser | 0.0188 | 8960 |
|
||||
| table-sort | 0.0152 | 7476 |
|
||||
| bytecode-vm | 0.0155 | 7692 |
|
||||
| matrix-multiply | 0.0151 | 7368 |
|
||||
| http-parser | 0.0180 | 8196 |
|
||||
| stdlib-print-demo | 0.0205 | 9636 |
|
||||
|
||||
## Interpretation
|
||||
|
||||
* Startup + parse + load dominate these small workloads; the interpreter's
|
||||
per-byte/per-call cost is visible mainly in the VM/matrix/table loops.
|
||||
* `stdlib-print-demo` is highest RSS/wall because it loads the bundled stdlib
|
||||
modules and formats all value types through `.mq.md`.
|
||||
* The current release binary is a tree-walking Rust interpreter; no bytecode
|
||||
compiler, JIT, columnar numeric backend or copy-elision pass exists.
|
||||
* No claim of P6 pass: the RFC requires an optimized implementation that is
|
||||
equivalent to the reference interpreter and reproducible performance/budget
|
||||
evidence. This report only establishes the baseline and a reproducible
|
||||
harness for future work.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
python3 tools/perf/bench.py > /tmp/marqdo-perf.json
|
||||
```
|
||||
Reference in new issue
Block a user