Compare commits

..
10 Commits
Author SHA1 Message Date
cflmyandCursor a87c2154ff release: v2.4.0 oracle journal language values
Expose foreign.oracle_journal/replay_check as values, bump CLI/ext/wasm to
2.4.0, and fix notebook OnceLock wait for MSRV 1.81 so EK can compile.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-06 15:09:34 +08:00
cflmyandCursor 8f1010f967 Ship oracle journal values, bytecode/frontend hardening, and replay-policy Lean.
Host exposes oracle journals as language values with refuse/copy gates; bytecode and foreign session paths get modularized; applications and verification replay-policy sources land with Lake builds ignored.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-06 15:04:27 +08:00
cflmyandCursor e3d9021bb6 docs: align foreign Strong correspondence with mechanized Thm. corr
Record corr_fresh_functional and host discharge (foreign gold + R1–R3)
for the closed/fresh/functional fragment instead of conformance-only wording.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-01 20:03:44 +08:00
cflmyandCursor 142ab523d6 release: v2.3.0 Foreign Strong session/handle/replay/arity
Opaque session handles, affine close, replay_check vs oracle journal,
table arity refuse, and Ω∈V_safe gate for the Strong foreign narrative.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-01 14:46:08 +08:00
cflmyandCursor 82c8d78fd3 release: v2.2.2 V_safe JSON integer refuse on foreign bridge
Align host json encode/decode with Lean ForeignJSON: refuse OOD integers
instead of silent Node distortion; keep decimal-string BigInt author path.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-01 09:43:51 +08:00
cflmyandCursor 58791fc7d9 release: v2.2.1 fix view outline rail under document
Occlusion left .doc-block open and mid-width CSS stacked the Functions/Variables rail below Structure; close the block and stack by panel container width instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-30 11:41:49 +08:00
cflmyandCursor 1730f26fc7 release: v2.2.0 call one function in any .mq.md
The same page is the document, the knowledge and the program. marqdo call runs that function with arguments, so reuse, tools and tests do not need a second skill system.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-30 09:07:23 +08:00
cflmyandCursor 4018e9928a fix(ai): stream usage events and accept free-form plan replies
Request stream_options.include_usage, emit usage SSE events for collect_usage,
propagate plan tokens through run, and treat unknown prose decisions as DONE.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-28 11:56:51 +08:00
cflmyandCursor 628e3b6e5d chore(ppa): bump 2.1.5 to 1ppa2 after Launchpad no-log failure
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 17:38:04 +08:00
cflmyandCursor 5c6c612edd fix(test): coerce MARQDO_SITE_PORT for MR-01 isolated execute
Env port strings must be json.parsed to int; pin main site to 18220 so
inherited MARQDO_SITE_PORT cannot steal the fixture bind.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 16:05:24 +08:00
159 changed files with 21220 additions and 1692 deletions

No files matched your search

+31 -13
View File
@@ -21,7 +21,7 @@ Marqdo = **Markdown markers as syntax**. A `.mq.md` file is both documentation a
These rules are the **current default**. Do **not** teach Classic `[key](map)` index or “bold is return of print”. Classic migration only: [migration-classic.md](migration-classic.md).
1. **File suffix** `.mq.md`. Print with `> print text=…` / `> 打印 内容=…` — never use `**…**` to print.
2. **`**…**` = code / return expression.** `*…*` = statement (bind / expr). Closing markers touch the last token (no trailing space).
2. **`**…**` = statement** (assign / call / expression). `*…*` = return. Closing markers touch the last token (no trailing space).
3. **`[描述](payload)` = occlusion:** `[]` is prose (never executed); `()` is the only executed part. Empty `()` → `None` stub.
4. **Read collections with `access(coll, key)`** (aliases `取` / `取值`). **Never** use `[key](map)` as index under default Occlusion.
5. **Foreach:** `- [描述](遍历 coll 为 item)` (`遍历`/`for`, `为`/`as`/`in`). Prefer explicit traversal when generating new code.
@@ -34,8 +34,8 @@ These rules are the **current default**. Do **not** teach Classic `[key](map)` i
| Marker | Meaning (v2.1) |
|--------|----------------|
| `#` / `##`+ | Object/type · function/method |
| `*…*` | Statement |
| `**…**` | Return / code expression |
| `**…**` | Statement (assign / call / expression) |
| `*…*` | Return |
| `[描述](payload)` | Occlusion step (payload executes) |
| `access(c, k)` | Collection read |
| `> fn …` | Call |
@@ -57,6 +57,24 @@ Executable-knowledge gate: [`tests/executable_knowledge/`](../../tests/executabl
8. Browser: Marqdo on WASM + `lib/browser` tables; do not hand-write business JS.
## Call one function
```markdown
# 计算器
## 加法
计算两个数字`a`和`b`的和。
*[计算结果](a + b)*
```
```
marqdo call calculator.mq.md 计算器.加法 a=2 b=3
```
prints `5`. The same command works on any `.mq.md`; the file does not need a special skill name.
## Minimal program
```markdown
@@ -96,13 +114,13 @@ Chinese builtins (same functions, no import):
## Statements, returns, branches
```markdown
*x = 1*
*y = x + 2*
**x = 1**
**y = x + 2**
## add_one
+ `n`
**n + 1**
*n + 1*
```
```markdown
@@ -130,9 +148,9 @@ import:
# main
*xs = > text.split value="a,b,c" sep=","*
**xs = > text.split value="a,b,c" sep=","**
> print text=`xs`
*t = > clock.now_unix*
**t = > clock.now_unix**
```
- Import **English** or **Chinese** library file; call that file’s API names via `lib.member` (do not mix languages).
@@ -191,8 +209,8 @@ Install: `marqdo ext add quantum` (ZH id: `量子`). Build: `cargo build --relea
**Q8 themes (circuit / probs / bloch):** `theme="dark"|"light"|"bw"` (ZH `主题=`; **default `dark`** — slate lab look, gate-family colors, label chips so wires never cross `q0` text). View plot chrome follows `data-theme` on the SVG. Advanced kinds (hinton/…) still Q8c pending full re-skin.
```markdown
*_ = > `qc`.draw kind="circuit" theme="dark"*
*_ = > `qc`.draw kind="probs" theme="light"*
**_ = > `qc`.draw kind="circuit" theme="dark"**
**_ = > `qc`.draw kind="probs" theme="light"**
```
Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7.md](../../doc/design/ext-quantum-q7.md) · Q8 viz: [ext-quantum-viz-style.md](../../doc/design/ext-quantum-viz-style.md) · examples: [quantum-bell](../../examples/quantum-bell/) · [quantum-entanglement](../../examples/quantum-entanglement/).
@@ -203,7 +221,7 @@ Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7
- `invoke` returns a record (`ok`, `code`, `stdout`, `stderr`, `ms`, `timed_out`, `truncated`, `result`); pass `input` for JSON stdin and `output="json"` for a parsed result.
- `call` is JSON-in / JSON-out; `call_many` fans out one program over a list with ordered results.
- `session_open` starts a persistent Python/Node kernel; `session_eval` evaluates source; `session_set` / `session_get` bridge values explicitly as JSON. Close it with `session_close`.
- `session_open` returns an opaque `handle` (not text; not JSON-serializable). `session_eval` / `set` / `get` borrow the handle; `session_close` consumes it (double-close / use-after-close fault). Prefer `foreign.replay_check` / language-level `foreign.oracle_journal` when comparing recordings (`replay_snapshot` only re-runs). JSON integers outside \(\pm(2^{53}-1)\) and ragged table columns are refused (`doc/design/v-safe-json.md`, `foreign-strong-correspondence.md`, `oracle-journal-values.md`).
- A view document opts into kernel state with `会话 = > foreign.session kernels=... cells=...`; the table names are free. `view` consumes only the `会话` handle and never scans source lines or reserved table names. Without `会话`, cells are stateless one-shot runs; no kernel is created. Clearing outputs calls `/api/session/clear` and keeps persistent kernel state. There is no separate notebook CLI mode or command, and no new Markdown marker exists.
- The same source remains the authority; state crossing kernels is explicit, never implicitly mirrored.
@@ -223,8 +241,8 @@ Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7
| `# main` then prose then code with no blank line | Blank line before code |
| `if x > 0:` | `1. `x` > 0` |
| `* > print text=hi *` | `> print text=hi` |
| `` *`a` = 1 * `` (trailing space + backticks) | `*a = 1*` — bare bind, no trailing space before `*` / `**` |
| `*xs = > text.split value=a,b sep=,*` (bare text args) | `*xs = > text.split value="a,b" sep=","*` — quote text literals; bare words are variables |
| `` *`a` = 1 * `` (trailing space + backticks) | `**a = 1**` — bare bind, no trailing space before `*` / `**` |
| `*xs = > text.split value=a,b sep=,*` (italic is return, and bare text args) | `**xs = > text.split value="a,b" sep=","**` — statement; quote text literals |
| Forgetting empty return after nested `##` helper when outer continues | Add `*None*` / `**` / `****` (not `---`) |
| Import `lib/text` then call bare `split` | `> text.split …` (qualified) |
| Import `lib/text` then call `拆分` | Match file language (`text.split`, not 文本) |
+12
View File
@@ -126,6 +126,18 @@ jobs:
## Marqdo ${{ github.ref_name }}
### 亮点 / Highlights
- **v2.4.0**:Oracle journal 作语言值——`foreign.oracle_journal`/`replay_check`;bytecode/frontend 硬化;MSRV 1.81
- **v2.4.0**: Oracle journal as language values — foreign.oracle_journal/replay_check; bytecode/frontend hardening; MSRV 1.81
- **v2.4.0 ext**:扩展包同步 2.4.0 / Ext pack 2.4.0
- **v2.3.0**:Foreign Strong——不透明 `handle`、仿射 close、`replay_check`/oracle journal、表 arity refuse、Ω∈V_safe
- **v2.3.0**: Foreign Strong — opaque handle, affine close, replay_check/oracle journal, table arity refuse, Ω∈V_safe
- **v2.3.0 ext**:扩展包同步 2.3.0 / Ext pack 2.3.0
- **v2.2.1**:view / 静态导出——修复 occlusion 与中等宽度下大纲侧栏掉到正文下方
- **v2.2.1**: view / static export — fix outline rail nested under document (occlusion + mid-width layout)
- **v2.2.1 ext**:扩展包同步 2.2.1 / Ext pack 2.2.1
- **v2.2.0**:函数级调用 `marqdo call`——同一份 .mq.md 就是文档、知识和工具;Tree/Bytecode 执行被点名的函数;外部调用默认只跑纯函数
- **v2.2.0**: `marqdo call` runs one function in any .mq.md — the file is the document, the knowledge and the tool; Tree and Bytecode execute that function; external calls stay pure
- **v2.2.0 ext**:扩展包同步 2.2.0 / Ext pack 2.2.0
- **v2.1.5**:Agent 可靠工作台——ResourceView / ToolSpec / CheckpointV2 / EffectLedger · EKC ir.json 暖机 · Web 短读槽隔离
- **v2.1.5**: Agent reliability workbench — ResourceView / ToolSpec / CheckpointV2 / EffectLedger, EKC ir.json warm-load, web short/read lane isolation
- **v2.1.5 ext**:扩展包同步 2.1.5 / Ext pack 2.1.5
+3
View File
@@ -79,6 +79,9 @@ plugins/*/tmp/
# Plot SVGs emitted beside tests / notebooks when demos run
/tests/**/*-plot-*.svg
# Lean Lake build outputs (sources under verification/ stay tracked)
**/.lake/
# PLDI / thesis harness scratch must never land in this tree
.lx*-tmp/
+122
View File
@@ -8,6 +8,128 @@
### Changed
## v2.4.0 — 2026-11-06
### Highlights
**Oracle journal 作语言值**:`foreign.oracle_journal` / `foreign.replay_check` 把观测与纯比较暴露为普通 Marqdo 值(非仅 notebook API);handle/code 不得入 journal;`copy` 拒绝 code。Bytecode / frontend / foreign session 模块化与 applications 金样例;`verification/replay-policy` Lean 策略身份。CLI · wasm · ext **2.4.0**。
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——变更在 `lib/foreign` 宿主表面与实现结构,不增 Markdown 标记。
### Added
- **`foreign.oracle_journal` / `foreign.replay_check`**:文档级观测列表与纯 journal 比较(`doc/design/oracle-journal-values.md`)。
- **回归**:`foreign-oracle-{journal,names,failure}`、`foreign-{,session-}copy-code-refuse`、`tests/oracle_journal.rs`、applications 四场景、`verification/replay-policy`。
- **文档**:`oracle-journal-values.md`;frontend-call / foreign-library 方向笔记。
### Fixed
- **view notebook API**:`OnceLock` 初始化等待兼容 MSRV **1.81**(不再调用 ≥1.84 的 `wait`)。
- JSON `float_roundtrip` 特征,减少外联浮点往返失真。
### Changed
- Bytecode compile/VM、foreign_doc/session、sema resolution、notebook interrupt 路径拆模块(行为对齐既有 Strong 契约)。
## v2.3.0 — 2026-11-01
### Highlights
**Foreign Strong(session / value / replay)**:会话句柄改为不透明 `handle`(不可 JSON 过桥、禁止 text 伪造);`close` 仿射消费(双 close / use-after-close 故障);`replay_check` 对照 oracle journal(旧 `replay_snapshot` 仅重跑保留);GFM 表行 arity 与列长映射拒绝静默补空;Ω 返回域外整数故障。对齐 PLDI Strong 叙事与 Lean SessionReg / \(V_{\mathsf{safe}}\)。CLI · wasm · ext **2.3.0**。
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——破坏面在宿主 foreign/session/JSON 与表解析纪律,不增 Markdown 标记。
### Added
- **`Value::Handle`**:`session_open` 返回句柄;session 操作拒绝 text 伪造。
- **`DocumentSession::replay_check`** + API `/api/session/snapshot/replay_check`;snapshot 录音字段 + **`oracle_journal`**(与 execution journal 分名)。
- **API** `/api/session/oracle_journal`。
- **文档**:`doc/design/foreign-strong-correspondence.md`;`v-safe-json.md` 补 Handle / 列 arity / Ω 门。
- **回归**:`foreign-session-{use-before-open,use-after-close,double-close,cold-replay-empty,isolation,handle-no-serialize,result-vsafe-refuse}`;`tests/errors/table-arity.mq.md`。
### Fixed
- 不等宽 GFM 表不再静默补空单元格;解析报 `table row arity mismatch`。
- Py/Node session bootstrap:Ω 结果 OOD 整数不再 `default=str` 偷渡,宿主升级为故障。
### Changed
- **`session_close`**:未知 / 已关闭句柄 → 故障(不再返回 `False`)。
- **`session_list`**:`id` 字段为 `handle`。
- Snapshot export 含 `ok`/`code`/`stdout`/`result`/`source_hash` 与 `oracle_journal`。
## v2.2.2 — 2026-11-01
### Highlights
**\(V_{\mathsf{safe}}\) JSON 整数契约落地**:外联 / `json.parse` / `json.stringify` / `session_set` 等共用映射对齐 Lean ForeignJSON——`|n| > 2^{53}-1` 的 JSON 整数显式拒绝(错误前缀 `json: integer outside V_safe`),禁止 Node 路径静默扭曲;宽整数仍用十进制字符串 + 对端 `BigInt`。CLI · wasm · ext **2.2.2**。
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——宿主 JSON 契约与测试,不改 Markdown 标记。
### Added
- **`doc/design/v-safe-json.md`**:宿主 ↔ Lean \(V_{\mathsf{safe}}\) 契约锁定(`MAX_SAFE`、encode/decode refuse、嵌套、float 为宿主扩展)。
- **回归**:`tests/lib/json-vsafe-*`、`foreign-vsafe-*`、`foreign-session-vsafe-refuse`(含算术后越界 refuse)。
### Fixed
- **`src/host/json.rs`**:对域外整数与 `u64 > i64::MAX` 显式失败;不再经 `f64`/`Text` 静默降级。
- **`doc/design/foreign-kernel.md`**:Int→number 仅当 `|n| ≤ 2^{53}-1`;宽整数走 string 桥。
### Changed
- 外联 JSON 桥与 `lib/json` 行为:OOD 整数从「可能静默失真」改为「诊断失败」。
## v2.2.1 — 2026-10-30
### Highlights
**view / 静态导出大纲侧栏修复**:带遮蔽(occlusion)语句的文档页里,Functions / Variables 右侧栏会掉到正文下面;中等窗口宽度下也会被错误叠成单列。CLI · wasm · ext **2.2.1**。
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——布局与 HTML 闭合修复,不改 Markdown 标记。
### Fixed
- **`marqdo view` / `view output`**:occlusion 行提前返回时未关闭 `.doc-block`,把 `#outline-rail` 嵌进 `document-surface`,侧栏排到文档下方;现已闭合并保留标注带。
- **view 壳 CSS**:大纲折叠改为按主栏 `@container mq-panel` 宽度(32rem),不再用视口 `960px` 媒体查询把右侧栏压到正文下;极窄时大纲上移到文档上方。
## v2.2.0 — 2026-10-30
### Highlights
**函数级调用 `marqdo call`**:同一份 `.mq.md` 已经是文档、知识和程序。这一版补上从外部调用其中某个函数的入口。人继续读这一页;开发者、测试和 Agent 用同一条命令传入参数并执行。知识复用、把函数当工具、逐个函数做测试、改完再跑,都是这一入口的用法,不是另一套 Skill 系统。CLI · wasm · ext **2.2.0**。
```bash
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3
# 5
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3 --json
```
**作用**:最小可执行单位从「整份文件」扩展到「文件里的一个函数」。不必为了试一个函数再写 `# main`、打印和包装程序。
**价值**:说明和实现仍在同一页。Agent 读函数旁边的叙述,定位函数,传入参数,得到可检查的结果。正式发布和权限仍走已有的项目验证;`call` 默认不开放写文件、进程、网络和插件。
说明页:[public/features/22-函数调用.mq.md](public/features/22-函数调用.mq.md) · [public/features/22-call.mq.md](public/features/22-call.mq.md)
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——新增的是 CLI 入口,不新增 Markdown 标记。`**…**` 仍是语句,`*…*` 仍是返回。
### Added
- **`marqdo call <文件.mq.md> <函数> [名字=值 ...]`**:加载该文件及其 import,按显示路径、SymbolId 或唯一短名定位函数,用现有签名和契约绑定参数,再用现有运行时执行。
- **`--json`**:成功 `{"ok":true,"result":...}`,失败 `{"ok":false,"error":...}`,错误带诊断 `code`。`--backend tree|bytecode` 与 `marqdo run` 相同;指定函数时两个后端都不会改去跑 `# main`。
- **默认纯函数**:效应含写入、网络、进程或插件时,执行前返回 `artifact.effect_denied`,副作用不会发生。
- **参照文档** `examples/calculator.mq.md`,以及用户说明页 `public/features/22-*`。
### Fixed
- 官方语法说明与 Agent 语法卡的标记极性与解析器对齐:粗体是语句,斜体是返回。同步测试会实际运行说明里的调用示例。
- Bytecode 的 `marqdo run --call-lib-path` 执行被点名的导入函数,不再忽略该参数去跑入口。
- 隔离运行时冒烟测试补上 `InvokeContext.fs_root`,`--tests` 可以编过。
### Changed
- CLI / wasm / `ext/VERSION` **同步 2.2.0**。扩展包版本号对齐;这一版的新能力在 CLI。
## v2.1.5 — 2026-10-27
### Highlights
Generated
+2 -2
View File
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "marqdo"
version = "2.1.5"
version = "2.4.0"
dependencies = [
"anyhow",
"assert_cmd",
@@ -611,7 +611,7 @@ dependencies = [
[[package]]
name = "marqdo-wasm"
version = "2.1.5"
version = "2.4.0"
dependencies = [
"marqdo",
"serde_json",
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo"
version = "2.1.5"
version = "2.4.0"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo — Markdown markers as an interpreted programming language"
@@ -30,7 +30,7 @@ chrono = { version = "0.4", default-features = false, features = ["clock", "std"
clap = { version = "=4.4.18", features = ["derive"], optional = true }
pulldown-cmark = "0.12"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
serde_json = { version = "1", features = ["float_roundtrip"] }
thiserror = "1"
tiny_http = { version = "0.12", optional = true }
sha2 = { version = "=0.10.9", default-features = false }
+15 -5
View File
@@ -4,6 +4,8 @@
Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,既是给人读的文稿,也是给机器跑的程序。写文档的过程就是在写可执行逻辑——大型项目里,说明、示例与实现不再分叉成三份真相。它不是「又一个 Markdown 编程语言」,而是**文档原生的编程运行时(Document-native runtime)**——一门关于**可执行知识(Executable Knowledge)**的语言。
v2.2.0 起,`marqdo call` 可以直接调用这份文件里的一个函数。读的是同一页说明,跑的是同一个实现。知识因此可以当工具用,测试也不必再包一层 `# main`。
欢迎访问 [marqdo 官方网站](https://www.marqdo.com/) 阅读更多特性与可执行文档。
**链接语法**:[doc/design/link-syntax.md](doc/design/link-syntax.md) · 语法宪法:[doc/design/markdown-mapping.md](doc/design/markdown-mapping.md) · 对象:[objects.md](doc/design/objects.md) · 调用实参:[call-arguments.md](doc/design/call-arguments.md) · 用户站:[user-site.md](doc/design/user-site.md) · 浏览:[view.md](doc/design/view.md) · 调试:[view-debug.md](doc/design/view-debug.md) · OKF 清单:[catalog-cli.md](doc/design/catalog-cli.md) · Engineering Knowledge:[engineering-knowledge.md](doc/design/engineering-knowledge.md) · VS Code 扩展:[vscode-extension.md](doc/design/vscode-extension.md)(分支 **`vscode-extension`**;提交约定 [vscode-extension-commit.md](doc/design/vscode-extension-commit.md)) · **AI Skill**:[skills/marqdo/](skills/marqdo/)(说明 [ai-skill.md](doc/design/ai-skill.md)) · 官方扩展:[ext-llm.md](doc/design/ext-llm.md)(`ext/llm`)· [ext-agent.md](doc/design/ext-agent.md)(智能体开发框架)· [ext-cli.md](doc/design/ext-cli.md)(`marqdo ext list/add/remove`) · 原生插件 ABI:[ext-abi.md](doc/design/ext-abi.md)(`lib/plugin`) · 金样例:[tests/](tests/) · 用户文档:[public/](public/) · 变更:[CHANGELOG.md](CHANGELOG.md)
@@ -29,6 +31,7 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
| 真正的核心优势 | 说明 |
|------|------|
| **知识-代码一致性**<br>Knowledge-Code Coherence | 说明、数据、示例与实现**同一份源文件**:文档里的表格就是程序的数据,文档里的调用就是真实执行。「文档过期」不再是悄悄漂移,而是可被机器发现的**知识不一致**——知识陈述本身可以被测试。 |
| **函数即工具**<br>`marqdo call` | 最小可执行单位是文件中的函数,不只是整份文件。人阅读函数旁边的叙述;开发者、测试和 Agent 用 `marqdo call 文件 函数 名字=值` 执行它。知识复用、工具调用、函数级测试和局部改进是同一入口的四种用法。 |
| **默认安全,AI 友好** | 无标记文字默认是叙述(注释),只有显式标记才进入执行。AI 自由发挥**不会污染**文档与代码的边界——这比「自然语言编程」安全得多,是 AI 原生语言的重要前提。 |
| **multi-view artifact**<br>`.mq.md` 双后缀即开放接口 | `.mq` 告诉解释器「可执行」,`.md` 告诉整个 Markdown 生态「仍是文档」。GitHub 直接读、VS Code 直接改、搜索引擎直接索引、LLM 直接理解、Marqdo 直接运行。**Marqdo 不占有知识,只是给 Markdown 增加执行语义。** |
| **表格即数据结构** | 程序的数据结构直接就是文档的数据结构:单列竖表 → `List`;横表(1 数据行)→ `Map`;横表(多数据行)→ `Map` of `List`(列向聚合);首列写 `@` / `行` / `row` → 行向记录 `List` of `Map`。取元使用 `access(集合, "键")`,链接 `[描述](表达式)` 只承载叙述与执行体。这是 literate programming 的下一步。 |
@@ -281,8 +284,13 @@ marqdo catalog public -o .marqdo
---
## 现状(v2.1.5)
## 现状(v2.4.0)
- **Oracle journal 语言值(v2.4.0)**:`foreign.oracle_journal` / `foreign.replay_check` 作普通值;handle/code 拒入 journal;bytecode/frontend 硬化;扩展包同步 **2.4.0**
- **Foreign Strong(v2.3.0)**:会话 `handle`(不可过桥)· 仿射 close · `replay_check`/oracle journal · 表 arity refuse · Ω∈\(V_{\mathsf{safe}}\)。扩展包同步 **2.3.0**
- **\(V_{\mathsf{safe}}\) JSON 整数(v2.2.2)**:外联与 `lib/json` 拒绝 `|n| > 2^{53}-1` 的 JSON 整数(对齐 Lean ForeignJSON);宽整数用十进制字符串 + Node `BigInt`。
- **view 大纲侧栏(v2.2.1)**:修复 occlusion 文档与中等宽度下 Functions / Variables 掉到正文下方的问题;静态 `view output` 同源。
- **函数级调用(v2.2.0)**:`marqdo call <文件.mq.md> <函数> 名字=值`。同一份文档给人读,也按函数执行;Tree 与 Bytecode 都跑被点名的函数,不回退到 `# main`。默认只执行纯函数。`--json` 给 Agent 读取结果或诊断。说明与价值见 [22-函数调用](public/features/22-函数调用.mq.md)。示例:`marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3` → `5`。扩展包同步 **2.2.1**
- **Agent 可靠工作台(v2.1.5)**:ResourceView(JSON/CSV/PDF/image)· ToolSpec 授权先于调用 · CheckpointV2 / EffectLedger / ContextPackV2 · EKC `ir.json` 暖机与 `--refresh` · Web 短读/控制与 execute 槽隔离;扩展包同步 **2.1.5**
- **默认隔离 HTTP execute(v2.1.4 · ADR 0009)**:`execute` / `sse-live` / `stream:true` → Supervisor worker 池(`--call-lib-path`);父进程不再占 `INVOKE_LOCK`;`/_runtime/*`;`runtime.isolated_execution=1`;MR-01 金样例
- **知识工作台运行时(v2.1.3)**:ModelSelection / RunContext / usage null≠0 · `view document` 只读嵌入 · LLM selection/messages/cancel · agent.step selection e2e · live smoke(设计 Partial:原生适配器 unavailable)
@@ -325,7 +333,7 @@ marqdo catalog public -o .marqdo
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
### 如何使用最新 Marqdo(v2.1.5)
### 如何使用最新 Marqdo(v2.4.0)
**安装解释器(任选其一)**
@@ -340,14 +348,16 @@ marqdo catalog public -o .marqdo
```bash
# 1) 源码安装(跟 tag 或 main)
git clone https://github.com/cflmy/marqdo.git && cd marqdo
git checkout v2.1.5 # 或留在 main
git checkout v2.4.0 # 或留在 main
cargo build --release
export PATH="$PWD/target/release:$PATH"
marqdo version
marqdo version --check
# 2) 跑解释器 / 文档站 / catalog
# 2) 跑整个文件,或只调用其中一个函数
marqdo run tests/structure/hello.mq.md
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3 --json
marqdo run tests/structure/hello.mq.md --backend bytecode
marqdo view public --no-open
marqdo debug public --no-open
@@ -379,7 +389,7 @@ marqdo wasm build
# → dist/wasm/ … 见 examples/browser-hello/
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
# https://github.com/cflmy/marqdo/releases/tag/v2.1.5
# https://github.com/cflmy/marqdo/releases/tag/v2.4.0
```
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo-wasm"
version = "2.1.5"
version = "2.4.0"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
+43
View File
@@ -1,3 +1,46 @@
marqdo (2.4.0-1ppa1) noble; urgency=medium
* Marqdo 2.4.0: oracle journal as language values (foreign.oracle_journal /
replay_check), bytecode/frontend hardening, MSRV 1.81 OnceLock wait fix.
-- Marqdo Maintainers <cflmy@users.noreply.github.com> Fri, 06 Nov 2026 15:15:00 +0800
marqdo (2.3.0-1ppa1) noble; urgency=medium
* Marqdo 2.3.0: Strong foreign session — opaque Handle, affine close,
replay_check vs oracle journal, table arity refuse, Omega V_safe gate.
-- Marqdo Maintainers <cflmy@users.noreply.github.com> Sat, 01 Nov 2026 14:45:00 +0800
marqdo (2.2.2-1ppa1) noble; urgency=medium
* Marqdo 2.2.2: V_safe JSON integer contract on foreign/json bridge —
refuse integers outside +/- (2^53-1); wide ints via decimal string.
-- Marqdo Maintainers <noreply@marqdo.com> Sat, 01 Nov 2026 09:45:00 +0800
marqdo (2.2.1-1ppa1) noble; urgency=medium
* Marqdo 2.2.1: fix view/static-export outline rail under document
(occlusion doc-block close + panel container query layout).
-- Marqdo Maintainers <noreply@marqdo.com> Fri, 30 Oct 2026 11:45:00 +0800
marqdo (2.2.0-1ppa1) noble; urgency=medium
* Marqdo 2.2.0: marqdo call runs one named function in any .mq.md.
The same file stays the document, the knowledge and the program.
Tree and bytecode both execute that function. External calls stay
pure: write, process, network and plugin stay off by default.
-- Marqdo Maintainers <noreply@marqdo.com> Fri, 30 Oct 2026 09:20:00 +0800
marqdo (2.1.5-1ppa2) noble; urgency=medium
* Rebuild 2.1.5 against Launchpad (previous 1ppa1 failed with no build log).
-- Marqdo Maintainers <noreply@marqdo.com> Tue, 27 Oct 2026 09:35:49 +0000
marqdo (2.1.5-1ppa1) noble; urgency=medium
* Marqdo 2.1.5: agent reliability workbench — ResourceView / ToolSpec /
+9 -5
View File
@@ -214,19 +214,23 @@ template = ["php", "-r", "{inline}"]
### 4.1 值映射
Canonical integer contract: [`v-safe-json.md`](v-safe-json.md) (Lean \(V_{\mathsf{safe}}\): `|n| ≤ 2^{53}-1`).
| Marqdo | JSON |
|---|---|
| None | null |
| Bool | bool |
| Int | number |
| Num | number(NaN/Inf → null) |
| Text | string |
| List | array |
| Map | object |
| Int | number **仅当** `|n| ≤ 2^{53}-1`;越界 → 显式失败(禁止静默扭曲) |
| Num | number(NaN/Inf → null);float 为宿主扩展,∉ mechanized \(V_{\mathsf{safe}}\) |
| Text | string(宽整数用十进制字符串过桥,对端 `BigInt` / Python `int`) |
| List | array(子树递归;任一越界整数使整体失败) |
| Map | object(同上) |
| Formula | 矩阵 → 二维数组;其它 → 数字或字符串 |
| Code | 源码字符串 |
| Secret | JSON 中 `<secret>`;`env` 显式交付时取真实值 |
错误前缀:`json: integer outside V_safe (±(2^53-1)): <n>`。
### 4.2 输入
- `input=` → JSON;写 stdin;同时提供 `MARQDO_INPUT`(≤8 KiB)与 `MARQDO_INPUT_FILE`(任意大小)。
@@ -0,0 +1,24 @@
# 外联库论文的研究方向
作者于 2026-11-05 将论文主线明确为:**叙述本身是程序,外部代码是值**。
Marqdo 的外联库用于在这一宿主模型中组合既有语言的计算能力,统一说明、控制流、
代码复用、状态与显式数据交换。论文应先解释 Notebook/文学编程中的实际设计问题,
再给出解决方案、可核验的收益、形式化依据和应用实验。
后续实现工作围绕绑定的代码值、参数化调用、持久会话、复制值和文档计划展开。
生命周期、值域和重放机制为这个统一设计提供保证;不能将重放检查器重新提升成
取代外联库与叙述式编程的论文主题。不得把未测的人因优势或未证明的实现正确性
写成既成事实。
完整方向文件在论文工作区:
- 方向(当前):`MarqdoThesis/PLDI/design/V7-NARRATIVE-FIRST-DIRECTION-2026-11-05.md`
- v6 方向(历史):`MarqdoThesis/PLDI/design/V6-NARRATIVE-FIRST-DIRECTION-2026-11-05.md`
- 证据映射:`MarqdoThesis/PLDI/design/V6-EVIDENCE-MAP-2026-11-05.md`
- 章节与页预算:`MarqdoThesis/PLDI/design/V6-OUTLINE-2026-11-05.md`
当前稿位于 `MarqdoThesis/PLDI/paper-v7`;v6 保留,历史 v5 及其证据保留。方向文件取代了
此前把论文缩成“外联会话值 + 因果重放”的 N3 路线:重放是支撑机制,不是主线。
2026-11-05 交付:`MarqdoThesis/PLDI/paper-v7/main.pdf`,正文(含 AI Use)20 页。
v6 保留为上一版尝试;更早的 v6 草稿保留在 `paper-v6-draft-2026-11-05/`。
@@ -0,0 +1,44 @@
# Foreign Strong: Lean ↔ host correspondence
2026-11-02 补充:`ForeignSource.lean` 对显式局部环境、返回、函数调用/递归、
分支和有界 while 的源解释器与独立 effect 编译器证明一般精化,并推出安全
观察和旧句柄不可复活。`CheckForeignSource.lean` 消费真实加载后 AST 的显式
适配格式,用独立 counter oracle 对照 tree/bytecode 的返回、journal、fault
与故障后 n。它已用于发现并修复 tree 的父词法作用域查找缺陷。
这是源控制流层的证明与实现 translation validation;Rust parser/adapter、
全部 AST 构造、机器数值运算和物理调度的完整 refinement 仍须完成。
模型中的 fuel 是分析界限,不能称为真实程序必然终止或有界取消。
Pinned with PLDI Strong / paper-v5 (sessions as values; three gates).
Marqdo **2.3.0** host implements the same objects the Lean development decides.
| Lean / paper rule | Host surface | Evidence |
|---|---|---|
| `SessionReg` + `sessionOpen` / `Close` / `sessionEval` | `HostContext.foreign_sessions` + `Value::Handle` | `DocCalc.lean` S1–S6, `requires_live`; `foreign_session.rs` |
| `Ω(s, τ, c)` | Persistent kernel + process order | Oracle journal `(cell, source_hash, stdout, result)` |
| `SafeVal` / Encode / Decode | `src/host/json.rs` + bootstrap sanitize | `ForeignJSON.lean`; gold `foreign-*-vsafe-*` |
| Handle ∉ \(V_{\mathsf{safe}}\) | `value_to_json(Handle) → Err` | `foreign-session-handle-no-serialize.mq.md` |
| Closed / Closed-Close | use-after-close / double-close `Err` | gold misuse; `sessionClose_then_eval_none_w` |
| Oracle-Domain | Ω OOD integer → host `Err` | `foreign-session-result-vsafe-refuse.mq.md` |
| Table arity stuck | Parse refuse unequal GFM; JSON unequal columns | `Surface.lean`; `tests/errors/table-arity.mq.md` |
| Snapshot hash match/mismatch | `view/output.rs` `snapshot_page` | unit snapshot hash tests |
| **Thm. corr** (`corr_fresh_functional`) | Cold `evaldoc`; step values = Ω | `corr_step_eq_omega`; `corr_doc_fresh_foreign`; foreign-session gold |
| `OracleJournal` / `replayCheck` | `oracle_journal` + `DocumentSession::replay_check` | `replayCheck_*`; fate R1–R3 |
| `corr_fail_warm_store` / `corr_fail_reordered_plan` | cold vs warm / schedule order | DocCalc witnesses; Table corr-fail |
| Dual-backend differential | `tests/gold.rs` tree + bytecode | 308 × 2; foreign session/V_safe subset |
## Scope (not distance)
- SafeVal mechanization covers the **integer** fragment; finite floats are admitted by the host under the same refuse-not-coerce bridge policy for non-integers already in SafeVal.
- Session discipline is a **dynamic registry** (affine close + borrow-use). Static linear types over OS-killable processes are out of scope by design, not an unfinished host feature.
- Correspondence is mechanized for the **closed / fresh-open / functional-Ω** fragment (`corr_fresh_functional`): per-step interactive value = Ω answer, interactive runner = normative runner, `replayCheck` on the journal. Broken hypotheses are `corr_fail_*`. Host discharge is foreign-session gold plus R1–R3, not R1–R3 alone.
## Reviewer-driven implementation update (2026-11-01)
The independent live cursor and multisession action-plan simulation are in
`DocCalc.runForeignLive_simulation` and `MarqdoProof.ForeignTrace`. The language
now exposes oracle observations and pure journal comparison through
`foreign.oracle_journal` / `foreign.replay_check`. Strict interactive snapshot
replay uses the full oracle history with source identity and per-step input.
See `oracle-journal-values.md` for the API and the research experiment manifest
under `PLDI/experiments/reviewer-v5/` for exact source hashes and actual runs.
@@ -0,0 +1,336 @@
# Executable call and integer semantics
This describes the implementation contract exercised by
`src/frontend_elaboration_tests.rs`; it is not a claim of a verified parser.
All explicit actual arguments evaluate once in authored order, including extra
named arguments whose values the callee does not read. Binding then permutes
already computed values into parameter slots. Named arguments take precedence
over positional filling; duplicate names and excess positional arguments fail.
A duplicate-actual binding fault occurs only when its call is reached, after
all explicit actuals have evaluated; it cannot reject an unused function at
compile time or erase an earlier argument effect.
Local calls and imported calls use the same order. Builtins and host primitives
must not reorder expression evaluation when selecting their parameter values.
All supplied parameters are available before defaults. Missing defaults execute
in parameter declaration order, once per call, in the target definition's
module and lexical scope. A default can see supplied parameters, earlier
computed defaults, and an explicitly bound method receiver. It cannot capture
the caller's locals. A still omitted later parameter is unbound. Explicit None
is a supplied value and never triggers a default. Required-argument validation
precedes default execution. Default expressions participate in receiver
classification, dependency analysis and transitive effect inference.
A dynamic method's signature belongs to the resolved receiver and selected
method; a different object's same-named method cannot supply its parameter
list. Imported definitions retain their module scope. A compiled document's
entry is selected only from its own module; an imported `main` is not an entry.
The bytecode retains argument names and explicit presence bits. Callee
prologues evaluate defaults in their own frame. Local slots distinguish an
unbound slot from a slot holding None. Local, constant, call-layout and jump
indices use container-sized indices rather than unchecked 8/16-bit casts.
Scratch slots are reused across sequential calls while pending outer actuals
remain live. Scalar constant interning uses a hash table; floating constants
are keyed by their exact bits.
Host integers are signed 64-bit values. Add, subtract, multiply, divide and
negation report a language error on overflow in every build profile; division
by zero is a separate error. Division truncates toward zero. The negative
endpoint has a valid signed literal. Float-to-int conversion rejects nonfinite
or out-of-range inputs instead of silently saturating. This local integer
domain is distinct from the narrower continuous foreign-copy domain.
The Lean call-combinator proof preserves complete interaction programs for
arbitrary argument evaluators, including failure and reply-dependent requests.
The integer model specifies exact arithmetic followed by representability
checking. Native documents, independently derived Python expectations, a Lean
arithmetic consumer and isolated source mutations validate the implementation
links. These do not prove the entire Rust parser, binder, object representation
or adapter. The research coverage ledger must keep those obligations explicit.
## Module identity, entry inputs and source-bound invocation
Instances retain the existing map representation and safe-copy domain. The
reserved `_type_module` field is a list of import binding names, paired with
`_type`. Class selection uses that module instance within the active import
tree; inherited methods resolve their bases in the defining module. Equal
class names in different imports remain distinct. These fields are ordinary,
forgeable data, not capabilities or globally portable nominal identities.
Round trips through Python/Node preserve these descriptors in the same module
context. Opaque session handles remain outside the copy domain.
Entry metadata is passed to `Vm::run`, never compiled into `main`'s body.
Explicit named/recursive invocations of `main` therefore keep their own inputs.
Defaults run with the invocation's actual registry; a session-valued default
must not be evaluated in a separate interpreter and then injected into a VM.
Reads appearing only in explicit/prose defaults participate in input inference
and effect analysis. Default failures use the parameter declaration's span.
Source-bound APIs preserve the selected declaration's source identity and
span, even when two declarations have the same dotted path. Textual path calls
retain the existing first-declaration rule. Duplicate parameter names are
rejected because they cannot define distinct binding slots.
Both backends retain parameter, return and local field contracts. Bytecode
checks supplied/defaulted inputs before the body, explicit returns before
returning, table bindings before writing a local and direct footnote accesses
before/after lookup. Unknown fields report the contract error before a generic
missing-key error. Entry/raw invocations also check their completed inputs.
Contract diagnostics retain the declaration quote and the failure location.
## Refinement obligations and current evidence
| Link | Mechanized result or implementation evidence | Remaining obligation |
|---|---|---|
| Raw Markdown to executable AST | Native explicit/prose defaults, source snapshots, parameter spans, inference and duplicate-parameter diagnostics | Prove the actual Rust classifier/parser and extraction rules; the older abstract grammar is not that proof |
| Loaded module tree to flattened declarations | Independent inventories of source digests, module instances, lexical parents, class/method ownership and resolved bases | Connect the actual resolver to the Lean namespace lookup model, including failed/ambiguous resolution |
| Effectful actual evaluation to parameter values | `FrontendCalls` proves interaction-program equality for arbitrary expression evaluators; native authored-order/unused-actual/builtin/foreign probes | Prove Rust scratch allocation, every expression case and its stack invariant against that combinator |
| Default evaluation and invocation inputs | Native callee-scope, presence, receiver, entry/raw/metadata, inference, error-span and stateful-session cases | A complete default-binding semantics and universal compiler refinement, including errors and host context |
| Module-qualified object dispatch | `ModularObjects` proves namespace flattening lookup, resolved-parent dispatch and safe descriptor round trips | Connect every real object constructor, resolver and copied descriptor to the model; justify Rust module-root lifetimes |
| Machine integers | `MachineInts` exact signed-64 model, checked boundary implementation, independent Python and Lean consumers | Connect Rust checked arithmetic and every conversion to the mathematical evaluator; floats remain an extension |
| Progressive contracts | Four real source boundary failures preserve codes, anchors and failure spans; valid contract execution | Universal contract-preservation theorem and imported declaration/call-site attribution |
| Source-bound external calls | Two same-path declarations execute their separately selected source lines on both backends | Full external argument validation, effect-policy and environment-injection refinement |
The ledger deliberately does not turn a new model lemma, a finite passing
corpus, or independent reconstruction into a complete frontend theorem. Those
remaining proof links are required to discharge the full frontend objective.
## Default prologue proof and imported contract attribution (November 4)
`DefaultPrologue.default_prologue_refines` relates source default binding, which
rechecks the current environment for omission, to a guarded block machine with
an immutable initial presence mask. It proves whole interaction-program equality
for arbitrary effectful evaluators, assuming distinct parameter slots and an
initially faithful presence mask. Earlier default stores preserve that invariant
for later parameters. The duplicate-slot counterexample issues two requests in
the machine versus one in the source, demonstrating why slot uniqueness matters.
Explicit None skips and immediate failure stopping are separately checked.
This is a model of guard/evaluate/store blocks, not a proof of the Rust emitter,
flat instruction stepping, actual expression compilation or parser.
The generated implementation family crosses all 27 omitted/int/None states for
three parameters with all six declaration orders. An independent environment
model checks 162 documents and 324 executions, including 34 faults when a default
reads a later omitted parameter. Source text is reconstructed independently;
expected traces are not derived from either backend. This finite family does
not prove arbitrary defaults. Imported argument-contract diagnostics now retain
the caller's failure span and source revision while pointing the declaration
anchor and suggestion at the actual library. Supplied and defaulted invalid
arguments both refuse before entering the callee body.
## Concrete name and flat-code interfaces
The compiler now borrows each original declaration and owning module while
flattening. Bare-call selection uses the same source `lookup_local` operation
as Tree, then maps the selected declaration identity to its flat index. This
removes a second, subtly different lookup algorithm: repeated top-level names
previously selected the first declaration in Tree but failed compilation in VM.
It does not prove the source lookup operation itself.
`NameSlots` proves reads, writes and the complete named-argument binding fold
commute with an injective local-name vector. Unknown names and repeated raw
API names are covered; repeated raw names retain last-write semantics. A
presence mask computed from initialized slots is faithful by construction.
The Rust raw entry and argument binder now derive their mask from the actual
initial logical slots. Method receiver aliases are overlaid only for default
evaluation, and restored into the body frame after logical argument checks.
`ReceiverDefaults` supplies the additional environment reasoning for explicitly
declared self/自 parameters; the slot theorem alone does not cover that case.
Every compiled Program is structurally verified before return, and the VM
verifies even publicly constructed/mutated Programs before any instruction.
All current opcode variants are handled exhaustively. Checks include operand
bounds, parameter/local prefix and injectivity, declaration and inheritance
forests, call-layout and function references, spans, reachable stack underflow,
branch/loop join heights and absence of leaked stack values on return. Missing
variable values remain language faults; this checker does not prohibit them.
Rust exports the computed per-PC heights alongside the real emitted opcode
text. `CheckFlatCertificate.lean` independently decodes that text and applies
an executable checker whose soundness is proved in `FlatCertificate`. Accepted
certificates preserve heights along every finite path and guarantee a valid
fetch with sufficient operands on each reachable PC, including loop prefixes.
The independent reader rejects unmodeled opcodes. Rust debug serialization and
opcode-to-concrete-VM-step correspondence remain interfaces to justify; a
structural certificate does not establish source/bytecode functional equality.
Neither this ordinary bytecode verification technique nor reuse of a resolver
is an originality claim. Complete functional lowering, parser semantics and the
Rust implementation correspondence remain open proof obligations.
## Functional expression checks and receiver views (November 4)
The emitter now records source-directed expression rules before emission, with
child order and exact instruction intervals. The mandatory native verifier
checks literal identity (including floating-point bits), local names, unary and
binary operators, short-circuit targets, interpolation, collections, indexing
and occlusion. Calls delegate staging and selection checks to `bytecode/call.rs`;
resolution and dispatch semantics remain explicit boundaries.
All seven statement forms have source-directed templates; a separate forest
check covers the entire default prologue, statement body and implicit return.
These checks detect stack-preserving semantic corruption, including Add to Sub,
and refuse it before host effects. A forged source witness remains outside this
check's guarantee; the recorder and actual AST correspondence need justification.
`ExpressionLowering` proves postfix expression compilation preserves complete
interaction programs for arbitrary effectful leaves and operator interpretations,
including faults and an arbitrary caller stack and continuation. `ShortCircuit`
proves the exact absolute-PC AND/OR macros under arbitrary effectful child
evaluation. `CheckExpressionLowering.lean` independently reads actual emitted
operators, constants and local names, and applies the proved core checker.
It counts call, collection, index and interpolation boundaries explicitly.
Child inlining, Rust VM stepping, leaf interpretation and source-witness fidelity
remain obligations; none of these results is whole-frontend functional refinement.
`ReceiverDefaults` proves that the source's separately maintained default
environment equals the machine's receiver view after every successful prefix.
The mask describes logical argument slots, while omitted self/自 defaults may
replace the evaluation view. Supplied None remains present. The real receiver
is restored before entering the body; argument contracts check both the logical
arguments at the call site and the restored frame at the declaration, preserving
source identity and diagnostic frames. The theorem composes this view invariant
with `DefaultPrologue` for arbitrary effectful defaults and distinct slots.
Concrete repairs also share scalar/unary semantics across evaluators, stringify
every interpolation part, allocate fresh loop temporaries, restrict dynamic
calls to their defining module scope, and enforce the existing one-million
truthy-iteration limit in VM as well as Tree. Regression evidence records each
repair separately from proof claims. These compiler techniques are not new
originality claims.
## Actual call staging and bounded control-flow macros
Every emitted call now has a separate source obligation captured before actuals
are replaced with scratch reads. The checker covers authored actual order,
evaluate/store/pop triples, source-local and loop-slot separation, nested-call
scratch separation, argument reloads, layouts, builtin selection/defaults, host
primitive identity, plugin name/pair ordering, method name/receiver and resolved
user target/tag. Every call expression must have exactly one obligation. The
reserved source-local prefix is established before any synthetic slots are
allocated. Disjoint nested calls may reuse scratch slots after their lifetimes.
`CallStaging` proves the flat evaluate/save/pop program restores an arbitrary
caller stack and has the same effects and failures as authored-order collection.
It proves outside slots unchanged and unique staged slots read back the exact
actual vector. The dispatch continuation is arbitrary and effectful, so binding,
defaults and receiver effects can follow collection without commuting effects.
`CheckCallStaging.lean` decodes real stores and reloads, applies the proved stage
checker and independently checks the complete selection/dispatch template.
Source-expression callback correspondence, name resolution and the semantics of
each dispatch continuation remain proof interfaces. Host parameter metadata is
an exported registry interface, not an independently verified registry theorem.
`WhileLowering` proves the condition/JF/check/body/backedge macro preserves the
complete interaction program for a bounded loop, including a final false
condition, condition/body faults, the truthy-iteration limit and early returns.
It relates an increasing native counter to a decreasing model budget and proves
admitted increments below one million fit u32. Model machine-step fuel is
distinct from this language-visible iteration budget.
`CheckStatementLowering.lean` checks actual default prologues, all seven statement
templates, forests and implicit returns, including the exact while macro layout.
The while theorem uses condition/body callbacks; concrete child inlining and the
full Rust state machine are not discharged by checking those layouts. Branches
and foreach currently have independent template evidence, not new functional
refinement theorems. No ordinary compiler-checking technique is a novelty claim.
## Loaded executable AST and coherent witness corruption
`SourceFunction` now captures parameters, defaults, executable bodies and complete
contract data directly from the loaded declaration before emission. The native
source checker walks that snapshot separately from the rule recorder, checks
ordered expression/statement correspondence and complete witness ownership,
and accounts for the generated scratch reloads explicitly. It also compares
full contracts with structural Rust equality, including types, nullable fields,
descriptions, source lines and declaration quotes. Contract presence alone would
not detect a Number-to-Any weakening.
The evidence exports the executable AST separately from the original module and
from the compiled snapshot. `CheckSourceWitness.lean` matches declaration
identities and applies `SourceWitness`'s proved ordered-tree reconstruction
checker. Interpolation parts, index labels, code payloads and formula structure
are retained; all formula floats and matrix entries retain their exact bits.
The old Debug formula text was insufficient for NaN payload identity. Inert
occlusion prose and pre-inference parser metadata are not execution-tree nodes.
Ten native coordinated-corruption cases change code together with its witness,
including operators, constants, variables, assignments, callees, NaN payloads,
index keys, interpolation, contract metadata and an orphan expression record.
Eight independent source-witness negatives include changing the compiled snapshot
as well. A coordinated wrong operator is a positive acceptance control for all
four earlier independent consumers and a rejection control for the AST checker.
This distinguishes trusted source correspondence from agreement among emitted
metadata fields.
The theorem proves exact correspondence of the supplied executable trees, hence
preservation under any interpretation of those trees. The native and JSON AST
adapters, normalization helpers and snapshot capture remain trusted interfaces.
No theorem here verifies the actual Markdown classifier/parser/inference or
connects every concrete Rust operation to the semantic models. A caller who
changes both the reference AST and its program has supplied a different
translation-validation input; this is not source authentication.
## Runtime argument-preparation faults
The expanded AST audit exposed a concrete acceptance mismatch: Tree can execute
main while an unused function contains a conflicting bracket modifier, whereas
VM compilation previously rejected the entire module. Modifier preparation now
lowers its failure to `RejectArgumentExpansion` before actual evaluation. A
provided default parameter, including explicit None, skips that fault; reaching
it preserves prior output and stops before actual effects or a suffix. The
error's message, span, source revision and diagnostic frames match Tree.
`CallExpr::for_evaluation` normalizes bare aliases before modifier expansion,
matching Tree's existing order. The expression checker validates an effect-free
fault leaf and the independent decoder handles its actual terminal opcode.
The AST adapter explicitly represents that executable fault without children;
unexecuted actual expressions are not falsely counted as evaluated code.
Two independent fault-opcode/message negatives and a positive AST-identity-only
control keep representation checking distinct from instruction interpretation.
The independent Python oracle now compares JSON values with exact recursive
types and requires integer arithmetic inputs to be actual integers. Python's
True==1 and 1==1.0 equalities must not erase language-visible value distinctions.
Four type-corruption controls are added. These improvements do not discharge
other call-error timing, parser, resolver, dispatch or Rust-step proof obligations.
## Native binding and declaration graph validation
Builtins, host primitives and plugins retain authored labels in native bytecode.
The shared value-independent binding plan selects named parameters before
positionals, preserves omitted optional bindings and unknown named extras, and
reports duplicate, missing and surplus positional arguments after actual effects.
Plugin lookup precedes signature binding. Eighteen reached/unreached error cases
check explicit expected output, messages, spans, source revisions and frames.
A registered C ABI echo probe checks mixed and extra named arguments. Execution
uses the native VM and plugin bridge throughout.
The source and flat declaration views share ordered resolution. A loaded snapshot
captures ordinal declaration paths, levels, child/root counts, imports and ordered
use aliases before emission. Verification checks identity, parent/child/root order,
module scopes and object/method names, then derives call targets independently of
emitted target records. Two-component paths check runtime receiver presence before
static fallback; an unbound reserved slot differs from a supplied None. Imports
retain priority. Resolution failures are native terminal instructions after actuals.
`NameResolution` proves first-match, path and lexical commutation under an
ordered embedding. Its rank checker proves sufficient depth and connects a
well-founded total source policy to flat lookup. Rust uses the same parent-chain
policy; parent-view extraction remains a trusted adapter. The finite checkers consume actual AST/flat evidence in
`CheckNameResolution.lean`. Nine native corruptions and seven independent graph
negatives test this boundary. A coordinated wrong target remains a positive control
for the older stack, staging and AST-shape consumers. Ordinary compiler correctness
ingredients are not novelty claims. Snapshot/JSON adapters, primitive registries,
parent-view extraction and concrete Rust
instruction refinement remain explicit obligations; universal frontend verification
and overall paper readiness are not established by these results.
`ArgumentBinding` separately models per-parameter omission permission and proves
value substitution preserves selections and faults. A successful index plan has
a finite-index witness. `CheckArgumentBinding.lean` checks actual static call
plans and an independently enumerated mask/label corpus. User VM calls now use
the shared planner with the callee default mask, correcting their surplus-count
diagnostic and preserving missing-required-before-default priority. Default
evaluation, contracts, dynamic signature selection and concrete Rust/JSON steps
remain separate interfaces. The mathematical planner is not a proof of the
entire native binding implementation.
+30
View File
@@ -1,5 +1,35 @@
# 文档会话(View Cells)
## 2026-11-02:并发控制与完整历史 v5
以下为当前实现;后文保留首版设计与当时的状态表。
- HTTP `kernel/interrupt` 查找已存在的会话,经独立控制器发送请求,不等待单元执行锁或其他文档的初始化。未打开、尚在初始化的会话返回错误,不启动新内核。已加载文档的源码文件被删除后仍可取消正在执行的单元。
- 内核的中断能力绑定其持有的子进程,并在回收前撤销;旧能力不能指向后来启动的内核。引导阶段的取消被暂存,收到 ready 后投递。逻辑取消输入会合并重复信号;投递错误明确返回。Windows 尚无对应的保状态中断实现。
- 取消使 Run All 在当前单元结束后停止,返回实际 `ran`、`remaining` 和 `cancelled`;不会继续运行后续单元。内核没有退出时,取消前已发生的状态修改保留。close 撤销 API 会话,重新打开不会激活旧引用。
- 完整历史 v5 将并发取消作为所在操作的 `controls` 输入记录:操作起点之后的微秒偏移及各内核的真实接收结果。严格回放按输入重新投递并重新观察回执;操作先结束时撤销剩余定时请求,不能流入下一操作。实际回执或观察不匹配就报告差异。v4 无控制输入的历史仍可检查;v4 中出现 controls 字段会被拒绝。
- 严格检查先完成纯验证,再启动回放内核。实际执行中,每次协议请求必须先与预期请求一致;实际回复和每个单元的观察、批次结果也立即比较。首个差异后停止依赖后缀,包括同一个 Run All 内的后续单元。未执行的观察标记 `compared=false`,不冒充已比较的第二个差异。
快照导出支持可选 `compact:true`。默认仍为完整 JSON;紧凑格式
`marqdo.history.dag` v1 对相同子值做自底向上共享,整数使用十进制文本、
浮点数保存 IEEE 位模式,保留映射顺序和全部历史字段。导入和严格回放会自动
识别格式。SHA-256 校验只检测内容一致性,不提供来源认证。
解码要求规范节点、向后引用、全部可达和唯一键,并在展开前检查最多
1,000,000 个节点、8,000,000 个展开值、512 MiB 保守展开预算、128 层深度;
展开后再次检查安全值域。此格式减少导出字节数,尚未减少录制时内存,也不保证更快。
实际取消实验覆盖 Unix 上同步 Python/Node 持久会话;不能推导任意外部代码均有界停止。
[Python 3.14 信号文档](https://docs.python.org/3.14/library/signal.html#execution-of-python-signal-handlers)
说明纯 C 计算可延迟信号处理;
[Node 22.22.1 vm 文档](https://nodejs.org/download/release/v22.22.1/docs/api/vm.html#timeout-interactions-with-asynchronous-tasks-and-promises)
说明异步任务可能逃出同步求值的中断范围。完整源码 elaboration、控制适配器、
真实时钟与信号调度到形式模型的 refinement 仍需完成,不能以这些测试代替。
相关实现:`foreign_session/control.rs`、`foreign_doc/control.rs`、
`foreign_doc/packing.rs`、`foreign_doc/replay.rs` 与 `view/notebook_api.rs`。
真实 TCP/HTTP 取消、逐单元副作用屏障和独立快照消费记录位于论文
`PLDI/experiments/reviewer-v5/`。
| | |
|---|---|
| 状态 | **D1–D3 首版已实现;D4/D5 待做** |
+131
View File
@@ -0,0 +1,131 @@
# Document oracle observations and strict replay
`foreign.oracle_journal` returns a copy of this evaluation's observations as
ordinary `index/channel/value` records. Successful admitted `session_eval`,
`session_get`, and document `json.parse` operations append `eval`, `get`, and
`decode`, respectively. Set/open/close do not append. An observation is
validated recursively before append; handles and code objects cannot enter.
Each new HostContext starts with an empty observation list.
`foreign.replay_check expected=J observed=J2` is a pure comparison, with no
foreign execution. It validates contiguous zero-based indices, channel tags
and crossing-domain membership. Records compare structurally by field name,
independent of field order. The result contains `ok`, the least mismatching
`index`, and both records (None for a missing entry). Two empty journals
match. Failure to provide a valid journal is an error, not a match.
The interactive `DocumentSession::replay_check(snapshot)` is a different layer:
it executes a recorded trace from fresh state. Version-2 snapshots contain the
full chronological oracle trace, recorded stdin per step, its declared count,
the plan and source identity. Preflight validates every step before any foreign
execution. Every step is compared; none is skipped. Repeated cell executions
are distinct observations. Execution timing logs and cell display summaries
are not comparison inputs. Matching input values in a supplied snapshot is
consistency checking, not cryptographic authentication of the recording.
Snapshots carry the SHA-256 of the loaded document text. Import checks it
before restoring foreign bindings, and preflights all crossing payloads before
mutation. Reload retains prior oracle observations with their original source
hashes, so a later checker can reject an incompatible trace.
Lean: DocCalc's independent `runForeignLive` cursor simulates `runForeignAcc`
for arbitrary prefixes, observation accumulators and code sequences, including
oracle failures. `ForeignTrace` composes the registry's open/eval/close
judgments using independent normative and interactive execution control;
`trace_simulation`, `trace_correspondence`, and `trace_replay` quantify arbitrary
multisession action sequences, also when a run faults. These supplement the
existing theorems without removing them.
Tests: `tests/oracle_journal.rs` runs the public document API on both evaluators;
`host::oracle::tests` covers journal equality and length localization;
`host::foreign_doc::tests` covers the live trace preflight and actual replay;
`host::foreign_session::tests` includes generated two-session model checks and
Python/Node boundary and process-failure experiments. The research runner is
`MarqdoThesis/PLDI/experiments/reviewer-v5/run.py`.
## Stateful requests and semantic projection
`HostContext.foreign_protocol_steps` records internal request attempts and
replies, including set values, copied eval input and raw stdin. It is separate
from the admitted language journal: rejected inbound responses belong to the
protocol evidence, while only admitted values belong to the oracle journal.
An input refused before send produces no protocol request. A response refused
after execution does not undo kernel effects; the post-state is retained.
Language observations are recorded at the document API boundary and by live
cell evaluation. Low-level get/names used during snapshot export do not append
language observations. A successful eval projects to its result. A failed eval
projects to a record of stable status, output and diagnostics. Process timing,
commands and handles are not semantic values. The exact source path allocated
for one-shot execution is alpha-renamed only in diagnostic fields; raw stderr
and stdout remain available unchanged, and ordinary paths/error differences
are still compared. Live oracle records carry this same projected `value`.
Python/Node replies mark serialization-domain failure with `domain_error`.
Error text is not used to infer this flag. Python refuses non-string object
keys; Node sanitizes objects with a null prototype to preserve special keys.
The common strict JSON decoder checks duplicate keys before materializing a
map, rather than losing them through overwrite.
`ForeignState.lean` refines the earlier eval-only fragment with opaque kernel
state, complete request histories, set/get/eval input/decode, monotonic handle
allocation and fault outcomes carrying the admitted prefix. An adaptive
Program has continuations on actual observations; its correspondence theorem
covers data-dependent next actions, not just a fixed schedule. Mapping actual
document elaboration and host bindings to this program is a further proof
obligation recorded in the research status file.
## Complete history, snapshot version 4 (2026-11-02)
Strict interactive replay now records operations, not just successful cells.
Each public execution/mutation boundary records typed arguments, success or
failure, admitted observations, cell-oracle records, and stable resulting
state. Nested implementation calls are grouped under the user's operation.
The history includes set/get, run/step/run-all, restart, reload, plan edits,
clear, snapshot inspection and imported values. Failed attempts remain in the
history; rejected handles and secrets have only non-capability/redacted tags.
Clearing display outputs cannot erase the history's oracle projection.
The snapshot stores the materialized initial plan and historical reload plans.
Entry text is parsed at its original path, retaining relative imports without
rewriting the user's document. Initial bindings are reapplied when a kernel is
reopened; unchanged reload kernels retain state, whereas changed language or
command retires the incompatible process. Loaded source/metadata and cached
cell outputs are updated consistently.
Preflight checks the entire operation stream, all source revisions, plan edits,
binding/cursor/generation transitions, and oracle projections before starting
any replay kernel. Executing the trace compares each result, admitted value,
cell result and stable state. The first mismatch stops its dependent suffix;
`executed_actions` and `remaining_actions` make this explicit. `matched` counts
admitted observations and `matched_actions` counts operations. No missing
record is silently accepted or skipped.
The HTTP replay-check endpoint performs an isolated comparison and retains an
existing warm session. It validates malformed input before starting even a
seed kernel; duplicate JSON fields are refused before overwrite. Verification
can still run recorded foreign work and its external effects: isolation is of
the document session, not a process sandbox. `replay_snapshot` remains the
separate legacy cell re-run API and makes no strict-history comparison claim.
Older version-2/3 snapshots may be used by legacy restoration/re-run surfaces;
they cannot prove a complete action history and strict checking requires v4.
`ForeignHistory.lean` models explicit command boundaries. A fault terminates
its command, a later command receives the actual retained registry, and the
history retains all command faults and admitted observations. It proves
command-history correspondence, journal membership and no alias resurrection
through arbitrary recovery histories. This extends the adaptive Program
results; the general source-elaboration refinement is a separate outstanding
proof obligation.
Experiment source inputs are frozen before running, checked again afterward,
and archived with their hashes. The current passing bundle is identified by
`MarqdoThesis/PLDI/experiments/reviewer-v5/LATEST-PASSING.json`.
Each operation also retains its internal protocol attempts/replies, including
refused inspection results. Those records are compared even when an inspector
could not return a value. They are control evidence and do not admit rejected
payloads into the language journal. An explicit document session_names request
adds its admitted array on the decode channel; internal names/get probes do not.
Reload records either a historical source/plan input or its input-read error;
replay consumes that captured version input and does not rewrite past files.
+90
View File
@@ -0,0 +1,90 @@
# \(V_{\mathsf{safe}}\) JSON integer contract (host ↔ Lean)
Canonical Lean model: `MarqdoThesis/PLDI/proofs/MarqdoProof/MarqdoProof/ForeignJSON.lean`.
Host mapper: `src/host/json.rs` (`json_to_value` / `value_to_json`).
This document freezes **runtime conformance** of the Marqdo JSON bridge to the
mechanized \(V_{\mathsf{safe}}\) integer domain. It is **not** a verified
compilation of the Rust interpreter.
## Domain
```
MAX_SAFE = 9007199254740991 # 2^53 - 1
```
\(V_{\mathsf{safe}}\) values (mechanized):
| JSON | Marqdo |
|---|---|
| `null` | `None` |
| `bool` | `Bool` |
| integer with `\|n\| ≤ MAX_SAFE` | `Int` |
| `string` | `Text` |
| `array` | `List` |
| `object` | `Map` |
**Host extension (not in Lean `SafeVal`):** JSON non-integer numbers → `Value::Num(f64)`. Non-finite numbers (NaN/Inf) are refused on encode and in Python/Node oracle responses, including nested payloads. Floats are documented host behavior; they are **not** claimed as mechanized \(V_{\mathsf{safe}}\) fidelity.
## Encode (`value_to_json`)
- `Value::Int(n)` with `|n| ≤ MAX_SAFE` → JSON number.
- `Value::Int(n)` with `|n| > MAX_SAFE` → **refuse** (error).
- `Value::Handle(_)` → **refuse** (`json: session handle cannot be serialized`).
- Column-table `Map` whose fields are all `List` with **unequal lengths** →
**refuse** (`json: table column arity mismatch`).
- Nested `List` / `Map`: any child refuse fails the whole tree.
- GFM table **parse** also refuses unequal row width
(`table row arity mismatch`) before a value is built.
- Wide integers that must cross a language boundary: author converts to a
**decimal string** (∈ \(V_{\mathsf{safe}}\)); the foreign program may use
`BigInt` / Python `int` on that string.
## Foreign oracle (`session_eval` / `session_get`)
Python/Node session bootstraps sanitize results with the same integer gate
before JSON emission. An Ω response containing an OOD integer is a **fault**
(host `Err`), not a soft `ok:false` cell record. Stdout text that authors
`json.parse` themselves is still gated by host `json_to_value`.
## Decode (`json_to_value`)
- JSON integer with `|n| ≤ MAX_SAFE` → `Int`.
- JSON integer with `|n| > MAX_SAFE` → **refuse**.
- JSON unsigned integer `> i64::MAX` → **refuse** (do **not** coerce via `f64` or `Text`).
- Nested array/object: any child refuse fails the whole tree.
- Decimal-string encodings of wide integers → `Text` (in domain).
## Error prefix
All refuse messages use a fixed prefix for probes and diagnostics:
```
json: integer outside V_safe (±(2^53-1)): <n>
```
## In-domain roundtrip
`±MAX_SAFE`, bool, null, string, array, and object roundtrips through
`encode → decode` must succeed. Integers `MAX_SAFE+1`, `-MAX_SAFE-1`, `2^53`,
`2^60`, `2^60+1` (and nested forms) must refuse.
## Non-claims
- Not verified Rust ↔ Lean compilation.
- Not arbitrary host-object fidelity (DataFrame, functions, heap sharing).
- Not automatic BigInt promotion inside Node/Python bootstraps.
- Document-level `Value::Int` may still hold i64 values that are OOD for JSON;
the gate fires when those values enter JSON encode/decode.
## Strict request boundary update
The numeric gate applies to the represented value, including integer-valued
floats and exponent notation. `9007199254740992.0` and
`9.007199254740992e15` are refused; fractions such as `1.5` remain in the host
extension. `serde_json` uses `float_roundtrip` to avoid endpoint decoding loss.
The strict text decoder rejects duplicate object keys and unequal column maps
before information is lost. One-shot, call_many and persistent input paths
use recursive domain admission; code, handle and secret values are not copied
implicitly. Auto-detection may leave non-JSON output as text, but valid JSON
outside the domain is a refusal, including in batch results.
+374
View File
@@ -0,0 +1,374 @@
请你重新阅读marqdo的官方库,掌握marqdo的最新进度。在我对marqdo项目的仔细审视之下我发现有一点没有闭环,我终于找到了,考虑现在智能体开发最常用的SKILL,这是一个markdown文件写明了智能体该干什么,我想要更推荐一步,因为marqdo使用markdown语法,支持叙述和代码相互混合,这就使得marqdo满足这样一个条件,当分章节的SKILL撰写下来,只要经过调整就能成为一份marqdo代码其中包含若干函数,我打算开发新的CLI命令marqdo > 函数名 函数参数或者marqdo call 函数名 函数参数 此时SKILL的不同章节就变成了不同的函数,智能体就真正的成为了将知识复现的作用,这让代码即文档、文档即知识,知识即可执行推进了一步变成了知识即工具,真正发挥了marqdo的优势,同样这样的执行还以为着智能体能够自行改进SKILL并接受确定的问题后修改而不会出错,这让自我进化的智能体成为了具备理论支撑和清晰边界的可能
# Marqdo 从“可执行知识”走向“知识即工具”:最新仓库审查与架构建议
我认为你这次找到了 Marqdo 一个非常重要、而且比单纯增加 `marqdo call` 命令更深层的突破口。
你实际上提出了一个新的统一关系:
> SKILL 是人和 AI 都能阅读的操作知识;Marqdo 可以让其中具有确定性执行语义的章节,直接成为可调用的工具。
这样,Agent 的职责就有机会从“阅读说明、理解步骤、临时生成实现”,转向“定位已有能力、填入参数、执行和验证结果”。
这与 Marqdo 原来的三句话是连续的,不需要推翻原来的定位:
代码即文档 → 文档即知识库 → 知识即可执行 → 知识即可作为工具使用。
不过我建议把“知识即工具”视为可执行知识的对外使用形态,而不是再创造一套与语言、知识库平行的 Skill 运行系统。
## 一、重新检查仓库后的最新情况
本次核对的是 [cflmy/marqdo](https://github.com/cflmy/marqdo?utm_source=chatgpt.com) 的 `main` 分支,HEAD 为 `4018e992`,`Cargo.toml` 版本为 2.1.5。最近的 v2.1.5 版本记录日期是 2026 年 10 月 27 日。
与我们之前讨论 v2.0.0 时相比,项目已经完成了不少与你的构想直接相关的基础建设。
| 相关能力 | 最新仓库证据 | 判断 |
| --------------------- | ---------------------------------------------------------------- | -------- |
| 跨文档确定性函数绑定 | `BoundCall`、`SymbolId`,Tree/Bytecode 双后端测试 | 已有核心基础 |
| 已编译知识的执行复用 | MLSP `artifact.bind`、`artifact.execute`、`artifact.reuse_execute` | 已有子集 |
| Agent 复用已有函数 | `agent_eng_bind_execute`、发布门禁 | 已有子集 |
| 指定函数执行 | `marqdo run --call-lib-path`、JSON 参数、隔离 worker | 已有底层入口 |
| 智能体工具规范 | `ToolSpecV1`、效应分类、权限检查与恢复设计 | 已有框架基础 |
| 章节式 SKILL 自动成为 CLI 工具 | 目前没有独立的 `marqdo call` 或 Skill 编译命令 | 待开发 |
| SKILL 自修改、自动测试和受控发布 | 已有局部修改、验证与沙盒组件 | 尚未形成完整闭环 |
因此,现在不是重新开发一个 Skill 执行器,而是将已存在的 Marqdo 函数、知识制品、工具协议和 Agent 机制统一起来。
还有一个与你的新构想直接相关的问题值得先修复:
最新官方 `skills/marqdo/SKILL.md` 的核心语法说明,与仓库中的现行解析器、v0.3 语法规范和可执行测试存在冲突。
目前解析器与测试使用的是:
- `**…**`:代码、赋值、调用等语句;
- `*…*`:返回值。
但官方 Skill 顶部却把两者反过来描述。这不是小的文档笔误:如果 Agent 按照错误规则自我修改 Skill,就可能产生与实际运行时不一致的程序。
这个真实存在的不一致,恰好证明了你所说的闭环为什么重要——SKILL 不能仅靠自然语言说明自己的正确性,它需要由语言编译器和实际测试提供独立验证。
## 二、你真正提出的是一种新的 Skill 体系
我建议把它暂时命名为 Executable Skill(可执行技能)。
它不应被理解为“给 Markdown Skill 增加一点代码”,而应当满足一个更严格的定义:
> Executable Skill 是一份以 Markdown 为人类可读接口、以 Marqdo 函数为确定性执行单元、以编译后的语义制品为工具身份、能够由 Agent 发现、调用、验证和改进的技能文档。
传统 Skill 与你的目标之间,存在一个非常重要的区别。
| 维度 | 普通 Markdown Skill | Marqdo Executable Skill |
| ---- | ----------------------- | ----------------------- |
| 技能说明 | 自然语言 | 自然语言 |
| 操作步骤 | Agent 阅读后自行实施 | 可以绑定已有 Marqdo 函数 |
| 函数接口 | 通常需要额外编写工具代码 | 从 `.mq.md` 的函数和契约生成 |
| 执行 | 依赖 Agent 选择工具、生成代码或操作软件 | 对已实现的步骤直接执行 |
| 输出验证 | 往往需要单独建立 | 可连接编译器、契约和测试 |
| 改进 | 编辑说明或脚本 | 编辑同源的知识与实现,验证后发布 |
| 可追踪性 | 主要依赖工具日志 | 可关联 SymbolId、源码、版本和执行结果 |
但不是每个 Skill 章节都应该自动变成可执行函数。
比如“适用场景”“注意事项”“禁止操作”是知识或约束;“检查订单”“生成报价”“汇总报告”才可能成为可调用能力。编译器应区分文档章节与可执行章节,不能把所有二级标题不加判断地暴露成工具。
### 这实际上使 Agent 的结构发生变化
SKILL.md
目标 · 流程 · 函数 · 契约 · 示例 · 约束
Marqdo Skill Compiler
章节解析 · 符号绑定 · 类型检查 · 效应分析
Knowledge View
Agent 阅读技能与限制
Tool Registry
Agent 调用已编译函数
Agent:选择、组合、执行、观察与改进
由语言运行时执行确定性部分,由模型处理非确定性判断
特别值得注意的是:这不仅是执行接口的统一,还是知识交付接口的统一。
同一份 Skill 能够投影成可读说明、CLI 函数、Agent Tool Schema、MCP 工具描述,以及知识库中的能力节点。实现这些投影的系统仍是同一个 Marqdo 编译器。
## 三、CLI 应采用 `marqdo call`,不要采用 `marqdo >`
你提出了两个命令形式:
```
marqdo > 函数名 函数参数
marqdo call 函数名 函数参数
```
我明确建议采用第二种。
原因是 `>` 在 Bash、PowerShell、CMD 等 Shell 中已经具有输出重定向语义。`marqdo > xxx` 会被 Shell 先行解释,无法可靠地作为传递给 Marqdo 的函数调用语法。
推荐设计为:
```
# 按函数名调用
marqdo call "订单计算.商品小计" \
--skill ./skills/order-calculator \
--arg 单价=30 \
--arg 数量=2
# 结构化调用,供 Agent 使用
marqdo call "订单计算.商品小计" \
--skill ./skills/order-calculator \
--args-json inputs.json \
--output json \
--policy pure
```
上述是建议新增的 CLI 接口,不是 v2.1.5 已支持的命令。
我还建议在 CLI 层保留一个便利别名:`marqdo skill call`。它与 `marqdo call` 进入同一个 dispatcher,不应分别实现两套调用逻辑。
### 最小必要的 CLI 能力
| 命令 | 作用 |
| ---------------------------- | ----------------------- |
| `marqdo skill inspect PATH` | 解析 Skill 文档、章节、可调用函数和依赖 |
| `marqdo skill list PATH` | 列出对 Agent 暴露的工具 |
| `marqdo skill check PATH` | 检查语法、绑定、签名、效应和工具导出规则 |
| `marqdo call TARGET` | 调用指定 Marqdo 函数并返回结果 |
| `marqdo skill test PATH` | 执行 Skill 的测试与契约检查 |
| `marqdo skill export PATH` | 生成通用工具描述或 MCP 暴露接口 |
| `marqdo skill trial PATH` | 对修改后的候选版本进行隔离验证 |
| `marqdo skill promote PATH` | 在门禁通过后发布新版本 |
| `marqdo skill rollback PATH` | 恢复到指定已接受版本 |
不过不建议一次性实现这九项命令。最小可用产品只需要 `inspect`、`call`、`check`、`test` 四项,版本发布与改进机制可以随后补齐。
最重要的技术决策是:
`marqdo call` 必须调用现有编译器绑定的函数,不得重新用字符串搜索源码、猜测章节位置再执行。
当前 [src/mlsp.rs](https://github.com/cflmy/marqdo/blob/main/src/mlsp.rs?utm_source=chatgpt.com) 已有 `artifact.bind` 与 `artifact.execute`,能够检查制品指纹、依赖闭包、来源摘要和效应策略。新的 CLI 应复用相同的应用服务和语义校验,而不是通过进程内调用 MLSP JSON 接口制造另一层重复协议。
还有一个现有实现细节:`marqdo run --call-lib-path` 已支持隔离 worker 通过 JSON 参数调用函数,但当前 `src/lib.rs` 的这个入口只在树遍历分支中处理,Bytecode 分支仍按普通程序入口执行。新的公开 `call` 命令需要补齐这一差异,或者明确在第一版只支持 Tree backend,不能静默忽略所选后端。
## 四、完整示例:让一份 Skill 同时成为文档、知识和工具
我建议第一版尽量保持作者体验简单:人只维护一份 Markdown 源文件,编译器自动生成工具描述、函数索引和知识制品。
例如,我们建立一个订单计算 Skill。
下面是目标设计示例。其中 `runtime` 和 `tools` 是我建议为 Skill 编译增加的元数据约定,目前不能将这份 `SKILL.md` 直接交给 v2.1.5 的 CLI,期待自动注册工具。
```
---
name: order-calculator
description: 订单价格计算,包括商品小计和含税金额
runtime: marqdo
tools:
- 订单计算.商品小计
- 订单计算.含税金额
---
# 订单计算
这份技能用于计算订单金额。
所有金额使用同一货币单位。税率使用小数表示,
例如 0.10 表示 10%。
## 商品小计
给定商品单价`单价`和购买数量`数量`,
计算未税商品小计。
*[商品小计](单价 * 数量)*
## 含税金额
给定商品单价`单价`、购买数量`数量`
和适用税率`税率`,计算含税金额。
**小计 = [计算商品小计](订单计算.商品小计(单价=单价, 数量=数量))**
*[最终含税金额](小计 * (1 + 税率))*
```
这里的计算是演示规则。正式业务技能还需要校验数量范围、金额精度、货币单位、税率适用条件与舍入规则;这些不能由函数名字自动推断为正确。
### 这份 Skill 会产生三种投影
给人阅读: 它是一份正常的 Markdown 文档。读者能理解每个步骤的目的,以及所需输入和返回值。
给 Marqdo 编译器: 两个 `##` 单元构成可解析的函数;第二个函数可以确定性地调用第一个函数;符号索引能够记录它们的依赖关系。
给 Agent 使用: 编译器输出两个工具的结构化描述。Agent 不需要重新生成乘法计算程序,只需选择工具并提供参数。
例如,编译后的工具描述可以是:
```
{
"id": "order-calculator::订单计算.商品小计",
"description": "计算未税商品小计",
"input_schema": {
"type": "object",
"properties": {
"单价": {"type": "number"},
"数量": {"type": "number"}
},
"required": ["单价", "数量"]
},
"output_schema": {
"type": "number"
},
"effect": "pure"
}
```
但这个 Schema 必须由明确的类型契约和编译器检查生成。若源文件没有足够的类型信息,应标记为未知或要求补全,不能因为变量参加了乘法就无条件断言它一定是 `number`。
需要强调:不是 `##` 标题自动等于一个公开工具。 `##` 让章节具备 Marqdo 函数的结构,`tools` 则声明哪些函数允许对外暴露。这样才能避免“安全说明”一类章节被意外注册成工具,也避免内部辅助函数被 Agent 随意调用。
### 为什么采用显式的 `runtime: marqdo`?
因为普通 Skill 的 Markdown 本来就可能包含 `**加粗内容**`、编号列表、代码块和二级标题。这些在现有 Marqdo 中可能具有执行语义。
如果直接把所有 `SKILL.md` 当成 `.mq.md` 执行,很可能发生严重的语义误读。
所以我建议:
- 未明确声明 Marqdo 运行时的 `SKILL.md`,仍然是普通 Markdown 文档。
- 明确声明 `runtime: marqdo` 的 Skill 才进入 Marqdo 编译流程。
- 内嵌示例代码块不应因展示而执行。
- 编辑器应对具有执行意义的 Markdown 标记提供可视化提示。
- 导出的工具必须经过独立检查和授权。
未来还可以支持从已有普通 Skill 中自动提取候选函数,但那应当是生成可审阅的转换补丁,而不是默认把自然语言步骤当成已经实现的代码。
## 五、这次最有价值的部分:自我改进的 Skill
你提出的第二层意义,我认为甚至比 `marqdo call` 更值得研究。
传统 Agent 修改一个 Skill 时,常见问题是:它修改了一段说明或脚本,却无法可靠判断其他步骤是否受到影响。
Marqdo 已经具备一些适合解决这个问题的基础,例如 `SymbolId`、依赖闭包、`artifact.patch`、`artifact.verify` 和知识影响分析。
但是,不能由此推出“智能体修改 Skill 就不会出错”。
更精确的目标是:智能体可以提出改进,系统能够判定一组明确、可测试的性质是否被保持,并且只有通过门禁的版本才有资格替换原有 Skill。
我建议采用以下演化协议:
已发布 Skill v1
现有稳定版本与完整依赖
1\. 发现确定的问题
失败测试、诊断、复现步骤或用户提出的明确修改要求
2\. AI 生成局部修改候选
只修改获准的章节或函数,记录版本与修改意图
3\. 编译与静态验证
语法、符号、参数、返回类型、效应、依赖关系
4\. 隔离测试与回归验证
目标问题修复、历史行为保持、异常输入、性能与权限边界
5\. 发布门禁
仅当策略允许且必要验证通过时晋升新版本;否则保留旧版本
Skill v2 · 已接受版本
可追溯、可回滚、后续调用绑定到明确版本
### 举一个真正的自我改进案例
原 Skill 中存在:
```
## 商品小计
商品单价`单价`,购买数量`数量`。
*[小计](单价 * 数量)*
```
现在发现一个确定的问题:输入 `数量=-1` 时,函数返回负数,但业务要求数量必须大于零。
Agent 不应该只是把说明修改成“数量必须大于零”。这不会改变执行行为。
它应当在候选版本中增加可执行的校验逻辑,并补充至少三个测试:
| 测试 | 输入 | 期望 |
| ---- | ----------- | ------------- |
| 正常订单 | 单价 30,数量 2 | 返回 60 |
| 非法数量 | 单价 30,数量 -1 | 返回结构化参数错误 |
| 边界数量 | 单价 30,数量 0 | 根据业务契约决定接受或拒绝 |
如果 Marqdo 尚不具备合适的语言级错误返回或异常处理方式,这个案例还会暴露一个语言能力缺口:需要一种由 Marqdo 程序自身构造、传递和处理错误的机制,而不是只能依赖 Rust 宿主报错。
这与此前我们讨论的通用语言能力重构是直接关联的。
### 自我进化的边界必须比普通代码修改更严格
我建议把以下约束固定下来:
1. 修改不能自动扩大权限。 原 Skill 只有纯计算权限,新版本不能因为 Agent 自己添加网络或文件操作就获得这些权限。
2. 候选版不能直接覆盖运行中的正式版本。 需要隔离工作区、原子发布和固定引用版本。
3. 必须证明目标问题被修复。 不允许通过删除测试、放宽断言或隐藏失败来宣称成功。
4. 测试通过不是绝对正确的证明。 未覆盖的业务情况、外部事实和依赖服务仍可能出错。
5. 高风险修改需要更严格的发布决策。 尤其是支付、外部写入、删除、凭证和安全策略变更,不能由被修改的 Skill 自己批准。
最新仓库的 `artifact.patch` 已经提供基于摘要的修改和有限回滚,但它还不是完整的候选版本事务与测试后晋升系统。现有 `HostCaps` 的默认值也允许文件写入、执行、网络和插件,因此新 `call` 的安全默认值应当收紧,不能简单继承普通本地运行的全部宿主权限。
这就是“有界自我进化”的含义:不保证 Agent 永不犯错,而是让它能够在可验证、可回滚、可限制权限的范围内持续改进。
## 六、我建议如何安排开发
| 阶段 | 目标 | 关键验收 |
| ---------------- | ---------------------------------------- | ------------------------ |
| S0 · Skill 语义一致性 | 修复官方 Skill 与解析器的标记冲突,建立 Skill 语法同步测试 | Skill 记载的每条核心语法都有可运行金样例 |
| S1 · 函数调用 CLI | `marqdo call` 调用已绑定的普通函数、类函数和跨文档函数 | JSON 参数正确绑定;错误路径和双后端行为明确 |
| S2 · Skill 编译 | 对显式启用的 `SKILL.md` 解析章节和可执行函数 | 自动产生 SymbolId、签名、依赖、导出清单 |
| S3 · 知识即工具 | 将公开函数编译为 ToolSpec,并接入现有 Agent dispatcher | Agent 不重新生成实现即可调用并取得结果 |
| S4 · Skill 测试 | 增加 `skill check/test`、负例、效应门禁和回归测试 | 未通过验证的 Skill 不得自动发布 |
| S5 · Skill 演化 | 实现候选修改、隔离验证、影响分析、版本晋升与回滚 | 修改失败不影响已发布版本,权限不能自行扩大 |
这里还需要一项对你的研究目标尤其重要的对照实验。
让相同模型分别使用普通 Markdown Skill、Markdown Skill 加外部脚本工具、以及 Marqdo Executable Skill 完成相同的一组任务。分别测量正确率、模型需要重新生成的代码量、工具调用参数错误、修改后的回归失败率、token 消耗和完成时间。
只有与“Markdown + 现有工具调用”进行公平对照,才能证明 Marqdo 的优势来自文档、程序与知识的统一,而不只是因为它额外提供了一个执行器。
## 七、我对这项构想的最终评价
我认为这次构想具有三个非常值得肯定的性质。
第一,它有明确的语言基础。`#`、`##`、自然语言叙述、函数参数、`[]()`、契约表和跨文档符号绑定并不是为了 Skill 重新创造的。Skill 可以成为现有语言的一种应用形态。
第二,它有真实的工程基础。当前 Marqdo 已经具备部分确定性绑定、制品验证、Agent 复用与隔离运行能力。你需要做的是将这些能力收敛到统一的公开接口,而不是重新实现一套 Agent 框架。
第三,它能够引出一个更有研究价值的问题:自然语言描述的操作知识,如何逐步转化为可调用、可检查、可维护,并可由 Agent 在受限条件下自行改进的程序制品?
但是,“SKILL 章节即函数”本身还不足以证明新的编程范式。Markdown 章节映射函数,与文学编程、文档驱动工作流及现有工具包装有明显的共同点。
更有价值的是把三个原本需要单独维护的对象——Skill 说明、函数实现、Agent Tool Schema——统一为一个可编译的源事实,并展示这种统一确实降低了错误、维护成本和重复生成成本。
我建议为 Marqdo 保留原来的核心理念,并将这一阶段的技术方向明确为:
> 知识即工具(Knowledge as Tool):将可执行知识直接编译为 Agent 能够发现、调用和验证的工具,并允许智能体在明确的语义、测试和权限边界内持续改进这些知识。
这可能是 Marqdo 从“AI 可以使用的编程语言”,进一步走向“AI 能够使用并维护的可执行知识体系”的关键一步。
备注,这里实际上有一点叙述错误,marqdo支持mq.md文件,所以最好针对SKILL也是SKILL.mq.md这样会比较统一。
+249
View File
@@ -0,0 +1,249 @@
SKILL的名称应该是.mq.md有一点必须提及,SKILL其实命名不一定必须得是SKILL,平常的一个marqdo文件也可以是一份skill这里的设计太复杂了没有抽离出来核心本质,你看对于一个平常的marqdo文件,完全也可以和SKILL一样,这都是没有区别的,所谓的知识即工具作为高度抽象本质就是带参数调用函数,所以你的inspect等等命令其实完全可以对传统的marqdo也生效而不是非要绑定SKILL对吧,而且这样处理还有好处是方面智能体做单元测试,单个函数是否可以带参数直接运行,换句话说,你仔细思考就会发现这些好处是marqdo的设计哲学在增加一点点call自然派生的优势,所以不需要繁复的设计
你说得对。这次我认为你抓住了真正的核心,而且我的上一版设计确实把一个本来很自然的语言能力复杂化了。
Marqdo 根本不需要引入一种叫作 Executable Skill 的新概念,也不需要专门建立 Skill 编译器。
因为在 Marqdo 的设计哲学下:
> 一个 `.mq.md` 文件本身就是文档、知识与程序。只要允许从外部直接调用其中的函数,它自然也就能够成为 Skill。
这不是为 Skill 开发一套新系统,而是为 Marqdo 补齐一个非常基础的语言使用入口。
## 一、真正缺少的只有一件事:函数级执行入口
Marqdo 当前已经支持函数定义、参数、返回值、模块引用,以及确定性符号绑定。
但是日常使用时,我们往往仍然站在“运行整个文件”的角度:
```
marqdo run example.mq.md
```
你现在提出的是:
```
marqdo call example.mq.md 函数名 参数
```
这实际上意味着:
Marqdo 的最小可执行单位,从文件进一步扩展为文件中的任意可调用函数。
而这一点会自然派生大量优势。
普通 `.mq.md` 文件
自然语言叙述 + 函数 + 参数 + 返回值
marqdo call
指定函数 + 绑定参数 + 直接执行
知识复用
调用已存在的实现
Agent Skill
直接暴露函数为工具
单元测试
逐个函数验证结果
自我改进
修改函数并重新测试
这里不存在四套实现,只有一个底层能力的四种使用方式。
## 二、一个普通的 Marqdo 文件就足够了
比如,文件 `calculator.mq.md`:
```
# 计算器
这是一个普通的 Marqdo 文档。
其中包含一些可以复用的数学计算知识。
## 加法
计算两个数字`a`和`b`的和。
*[计算结果](a + b)*
## 乘法
计算两个数字`a`和`b`的积。
*[计算结果](a * b)*
```
它既是可阅读的文档,也包含两个函数。
现在只需要支持:
```
marqdo call calculator.mq.md 计算器.加法 a=2 b=3
```
返回:
```
5
```
再比如:
```
marqdo call calculator.mq.md 计算器.乘法 a=4 b=6
```
返回:
```
24
```
以上 `marqdo call` 是拟新增的命令形式;函数示例遵循当前 `*…*` 返回、`[]()` 遮蔽的语义。
这时候,一个 Agent 只要能够读取这个 `.mq.md` 文档,就可以理解函数的用途;只要能够调用 Marqdo CLI,就能执行其中的能力。
它根本不需要关心这个文件是不是 Skill,也不需要一份特殊的 `SKILL.md`。
如果 Agent 把它当成 Skill 阅读和使用,它就是 Skill;如果开发者把它当成程序库调用,它就是程序库;如果研究者把它作为知识库中的一部分,它就是可执行知识。
文件的身份没有变化,变化的是使用方式。
## 三、这项能力会自然解决三个问题
### 1. 让知识成为工具
传统 Skill:
Agent 阅读操作说明,决定如何完成任务。
Marqdo:
Agent 阅读函数说明,找到适合的函数,传入参数,直接执行。
这就将原来的:
知识 → Agent 理解 → 重新实现 → 执行
缩短为:
知识 → 函数定位 → 参数调用 → 执行
但模型仍然需要负责判断应该调用哪个函数、传入哪些参数。确定性的是被调用函数的语言语义,而不是整个 Agent 决策过程。
### 2. 让单元测试成为语言的天然能力
我非常赞同你提到的单元测试价值。
有了 `marqdo call`,可以直接测试:
```
marqdo call calculator.mq.md 计算器.加法 a=2 b=3
# 期望 5
marqdo call calculator.mq.md 计算器.加法 a=-2 b=3
# 期望 1
marqdo call calculator.mq.md 计算器.乘法 a=0 b=100
# 期望 0
```
不必为了测试一个函数,再创建一个 `# main`,再编写调用、打印和结果检查的包装程序。
未来即使增加 `marqdo test`,底层仍然可以复用同一个函数调用接口。
### 3. 让 Agent 可以针对知识进行局部改进
这是你提出的最有意义的延伸。
假设一个 Agent 发现 `计算器.乘法` 对某类输入存在错误。
它可以:
1. 阅读相应函数及其上下文。
2. 修改这个 `.mq.md` 文件中的函数实现。
3. 直接用 `marqdo call` 传入错误输入验证。
4. 再运行该函数的历史测试。
5. 如果检查通过,就提交候选修改;否则保留原有实现。
这使得 Agent 的改进对象不再仅仅是自然语言提示词,而是有明确输入、输出和可执行语义的知识单元。
当然,测试不能证明函数在所有输入上都正确;正式发布和权限控制仍需要遵循已有的项目验证机制。但不需要为此创造一套 Skill 专属的演化协议。
## 四、真正的 CLI 设计可以非常简单
我建议第一阶段只做一个命令:
```
marqdo call <文件> <函数> [具名参数...]
```
比如:
```
marqdo call book.mq.md 订单.计算总价 单价=30 数量=2
```
它只需要完成五件事:
| 步骤 | 职责 |
| -- | -------------------- |
| 1 | 加载指定 `.mq.md` 文件及其导入 |
| 2 | 解析并唯一定位目标函数 |
| 3 | 根据现有函数签名绑定、校验参数 |
| 4 | 使用现有运行时执行该函数 |
| 5 | 返回函数结果和结构化错误 |
额外提供一个 `--json` 输出选项就足够方便 Agent 调用。
如果未来需要 `marqdo inspect`,它也应该针对任何 `.mq.md` 文件,列出其中函数、参数和文档描述,而非限定 Skill。
此外,当前仓库已经有 `--call-lib-path` 和 `artifact.execute` 的相关基础,因此实现时应尽量复用现有函数定位与执行机制,而不是重复开发解释器。
唯一值得坚持的边界是:外部调用仍必须经过现有权限检查,不能因为新增了 `call` 就默认允许 Agent 对任意函数执行文件删除、网络写入等操作。
## 五、我认为你的设计哲学现在更加清晰了
你原来的理念:
> 代码即文档,文档即知识库,知识即可执行。
实际上已经足够完整。
“知识即工具”不是额外的设计理念,而是“知识即可执行”在函数级调用能力成熟后自然出现的结果。
进一步看,这件事还有一个很漂亮的对称性:
| 参与者 | 对同一个 `.mq.md` 的使用方式 |
| ---------- | ------------------- |
| 人类 | 阅读文档、理解知识 |
| 编译器 | 解析并验证函数 |
| 开发者 | 调用函数、组合程序 |
| Agent | 发现知识、调用工具 |
| 测试系统 | 输入参数、检查结果 |
| 维护者或 Agent | 修改实现、验证改进 |
这些角色并没有要求作者维护六份不同的东西。
同一份文件,同一套语义,同一个函数调用接口。
这也恰恰是我认为 Marqdo 相比传统“Markdown 文档 + 外部工具”组合最值得证明的优势:知识描述与执行实现共址,而且能够直接由语言工具链消费,避免维护重复的工具包装和操作说明。
所以,我现在的建议与上一轮不同:
不要开发 Executable Skill 系统。不要设计 Skill 专属语法。不要引入新的 Skill 注册、编译和执行机制。
先完成 `marqdo call`,使普通 `.mq.md` 中的可调用函数都能被精确定位、传参执行,并用现有语义系统验证。
这是一个相对很小的接口变化,却可能自然打通 Marqdo 的编程、知识复用、Agent 工具调用和函数级测试四个场景。
我认为这才是你所说的:让 Marqdo 自己的设计哲学发挥作用,而不是不断增加外部框架来证明它的价值。
+17
View File
@@ -0,0 +1,17 @@
# 计算器
这是一个普通的 Marqdo 文档。
其中包含一些可以复用的数学计算知识。
## 加法
计算两个数字`a`和`b`的和。
*[计算结果](a + b)*
## 乘法
计算两个数字`a`和`b`的积。
*[计算结果](a * b)*
+1 -1
View File
@@ -1 +1 @@
2.1.5
2.4.0
+82 -18
View File
@@ -67,7 +67,11 @@ One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{te
1. `stream`
**evs = > model.stream prompt=`prompt` echo=`echo` stream_kind=`stream_kind`**
**text = > llm.collect events=`evs`**
**tokens = 0**
**tokens = > llm.collect_usage events=`evs`**
1. `tokens` == None
**tokens = 0**
2. *
**_ = 1**
**err = None**
**calls = 1**
1. not `text`
@@ -89,12 +93,16 @@ One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{te
**tokens = > json.get value=`r` key="tokens"**
**err = > json.get value=`r` key="error"**
**fin = > json.get value=`r` key="finish"**
1. `tokens` == None
**usage = > json.get value=`r` key="usage"**
1. `usage`
**tokens = > json.get value=`usage` key="total_tokens"**
2. *
**text = > str value=`r`**
**tokens = 0**
**err = None**
**fin = None**
1. not `tokens`
1. `tokens` == None
**tokens = 0**
2. *
**_ = 1**
@@ -1339,7 +1347,7 @@ Write a scratch tool workbook under `.marqdo/agent-runs/tools/<name>.mq.md`.
Protected v2.1 Occlusion card always prepended to skill_brief (M05). Must not be truncated by budget.
**esc = > json.parse text={"card":"## CORE SYNTAX (v2.1 Occlusion)\n1. Print with > print / > 打印 — never use bold markers to print.\n2. Bold markers = return/code; italic markers = statement.\n3. [desc](payload) = occlusion; brackets never execute.\n4. Read with access(coll, key) — never Classic [key](map) index.\n5. Foreach: - [desc](for coll as item). import + one language per file.\n6. # type · ## function · + params · numbered branches · - loops.\n"}**
**esc = > json.parse text={"card":"## CORE SYNTAX (v2.1 Occlusion)\n1. Print with > print / > 打印 — never use bold markers to print.\n2. Bold markers = statement; italic markers = return.\n3. [desc](payload) = occlusion; brackets never execute.\n4. Read with access(coll, key) — never Classic [key](map) index.\n5. Foreach: - [desc](for coll as item). import + one language per file.\n6. # type · ## function · + params · numbered branches · - loops.\n"}**
*> json.get value=`esc` key="card"*
---
@@ -2813,19 +2821,31 @@ Primary author entry (v2 + P4): Adaptive `route` → execute (policy/kb skill) o
2. *
**out = > `self`.plan goal=`task` kb_dir=`kb_dir` writeback=`writeback` reuse=`reuse` near_match=`near_match` near_threshold=`near_threshold` promote=`promote` improve_every=`improve_every` explore_n=`explore_n` max_rounds=`max_rounds` stream=`stream` echo=`echo` trace=`trace` force=`force`**
**exec_mode = "explore"**
**llm_calls = 1**
**llm_calls = > json.get value=`out` key="llm_calls"**
**tokens = > json.get value=`out` key="tokens"**
1. `llm_calls` == None
**llm_calls = 1**
1. `tokens` == None
**tokens = 0**
2. *
**out = > `self`.plan goal=`task` kb_dir=`kb_dir` writeback=`writeback` reuse=`reuse` near_match=`near_match` near_threshold=`near_threshold` promote=`promote` improve_every=`improve_every` explore_n=`explore_n` max_rounds=`max_rounds` stream=`stream` echo=`echo` trace=`trace` force=`force`**
**cache = > json.get value=`out` key="cache"**
1. `cache` == "hit"
**exec_mode = "compiled"**
**llm_calls = 0**
**tokens = 0**
2. `cache` == "soft-hit"
**exec_mode = "near"**
**llm_calls = 0**
**tokens = 0**
3. *
**exec_mode = "explore"**
**llm_calls = 1**
**llm_calls = > json.get value=`out` key="llm_calls"**
**tokens = > json.get value=`out` key="tokens"**
1. `llm_calls` == None
**llm_calls = 1**
1. `tokens` == None
**tokens = 0**
**st = > json.get value=`out` key="status"**
**res = > json.get value=`out` key="result"**
@@ -3492,6 +3512,8 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
**status = "error"**
**err = "max_rounds exhausted"**
**skip_loop = None**
**plan_llm_calls = 0**
**plan_tokens = 0**
> plan_echo_decompose workbook=`path` stream=`stream` echo=`echo`
**last_obs = > inspect_workbook path=`path`**
@@ -3505,6 +3527,12 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
**last_reply = > json.get value=`turn` key="reply"**
**last_obs = > json.get value=`turn` key="observation"**
**events = > json.get value=`turn` key="events"**
**_c = > json.get value=`turn` key="llm_calls"**
**_t = > json.get value=`turn` key="tokens"**
1. `_c` != None
**plan_llm_calls = plan_llm_calls + _c**
1. `_t` != None
**plan_tokens = plan_tokens + _t**
2. *
**dec = "TRANSPORT_ERROR"**
**last_reply = "plan_llm_act_loop_not_map"**
@@ -3553,11 +3581,27 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
2. *
**_ = 1**
5. *
**done = 1**
**status = "error"**
**err = "unrecognized plan decision"**
**summary = last_reply**
**skip_loop = 1**
Free-form parent reply with no DECISION/CALL/READ (dec=unknown): accept as DONE
when there is prose — chat soft-recover depends on result/summary surviving.
1. `dec` == "unknown"
1. `last_reply`
**done = 1**
**status = "ok"**
**err = None**
**summary = `last_reply`**
**skip_loop = 1**
2. *
**done = 1**
**status = "error"**
**err = "unrecognized plan decision"**
**summary = last_reply**
**skip_loop = 1**
2. *
**done = 1**
**status = "error"**
**err = "unrecognized plan decision"**
**summary = last_reply**
**skip_loop = 1**
1. `skip_loop`
**left = 0**
@@ -3606,6 +3650,12 @@ Deterministic success stop (loop engineering): when the child already returned a
**last_reply = > json.get value=`turn` key="reply"**
**last_obs = > json.get value=`turn` key="observation"**
**events = > json.get value=`turn` key="events"**
**_c = > json.get value=`turn` key="llm_calls"**
**_t = > json.get value=`turn` key="tokens"**
1. `_c` != None
**plan_llm_calls = plan_llm_calls + _c**
1. `_t` != None
**plan_tokens = plan_tokens + _t**
1. `dec` == DONE
> agent_workbook_solidify path=`path` observation=`last_obs`
**done = 1**
@@ -3626,11 +3676,25 @@ Deterministic success stop (loop engineering): when the child already returned a
3. `dec` == RUN
**left = left - 1**
4. *
**done = 1**
**status = "error"**
**err = "unrecognized plan decision"**
**summary = last_reply**
**left = 0**
1. `dec` == "unknown"
1. `last_reply`
**done = 1**
**status = "ok"**
**err = None**
**summary = `last_reply`**
**left = 0**
2. *
**done = 1**
**status = "error"**
**err = "unrecognized plan decision"**
**summary = last_reply**
**left = 0**
2. *
**done = 1**
**status = "error"**
**err = "unrecognized plan decision"**
**summary = last_reply**
**left = 0**
1. `status` == "ok"
1. `promote`
@@ -3643,9 +3707,9 @@ Deterministic success stop (loop engineering): when the child already returned a
`out` =
| status | goal | workbook | rounds | cache |
|--------|------|----------|--------|-------|
| `status` | `goal` | `path` | `round` | `cache` |
| status | goal | workbook | rounds | cache | llm_calls | tokens |
|--------|------|----------|--------|-------|-----------|--------|
| `status` | `goal` | `path` | `round` | `cache` | `plan_llm_calls` | `plan_tokens` |
1. `summary`
**out = table.put in=`out` at="summary" value=`summary`**
+29
View File
@@ -92,6 +92,35 @@ Alias: `stream_result`.
**answer = ""**
*answer*
## collect_usage
+ `events`
Best-effort `total_tokens` from stream frames (requires `stream_options.include_usage`).
**tokens = None**
1. `events`
- [ev](遍历 events 为 ev)
**t = > json.get value=`ev` key="type"**
**raw = None**
1. `t` == "data"
**raw = > json.get value=`ev` key="data"**
2. *
**u0 = > json.get value=`ev` key="usage"**
1. `u0`
**tt0 = > json.get value=`u0` key="total_tokens"**
1. `tt0` != None
**tokens = `tt0`**
1. `raw`
1. not `raw` == "[DONE]"
**parsed = > json.parse text=`raw`**
1. `parsed`
**u = > json.get value=`parsed` key="usage"**
1. `u`
**tt = > json.get value=`u` key="total_tokens"**
1. `tt` != None
**tokens = `tt`**
*tokens*
## stream_result
+ `events`
+7
View File
@@ -115,6 +115,13 @@ Pass either `messages=` (role/content list) or legacy `prompt=` (single user mes
1. `stream`
**req = > table.put in=`req` at="stream" value=True**
`stream_opts` =
| include_usage |
|---------------|
| True |
**req = > table.put in=`req` at="stream_options" value=`stream_opts`**
**body = > json.stringify value=`req`**
**resp = > net.http_post_sse url=`url` body=`body` headers=`headers` echo=`echo`**
**st = > json.get value=`resp` key="status"**
+23 -3
View File
@@ -177,8 +177,9 @@ view has no persistent kernel state.
+ `cwd`=None
+ `timeout`=None
Start a persistent Python/Node kernel; returns a session id. `timeout` is the
default per-eval timeout.
Start a persistent Python/Node kernel; returns an opaque `handle` value.
Text fabrications are refused by session ops; handles cannot cross the JSON
bridge. `timeout` is the default per-eval timeout.
*> host_foreign_session_open lang=`lang` cmd=`cmd` env=`env` cwd=`cwd` timeout=`timeout`*
@@ -223,7 +224,9 @@ List kernel variable names.
## session_close
+ `id`
Kill and remove a session; returns whether it existed.
Consume (affine close) a live session handle: kill and remove it. Returns
`True` on success. Closing an unknown or already-closed id is a fault
(`foreign session … not found`), matching use-after-close on eval/set/get.
*> host_foreign_session_close id=`id`*
@@ -246,3 +249,20 @@ List live sessions with id/lang/cmd/evals/alive/started_ms.
List configured foreign languages.
*> host_foreign_langs*
## oracle_journal
Return this document evaluation's admitted eval/get/decode observations as
ordinary records with index, channel, value. Opening, setting and closing do
not append observations.
*> host_oracle_journal *
## replay_check
+ `expected`
+ `observed`
Compare two oracle journals without executing foreign code. Returns ok and
the least mismatching index, with the expected and observed records.
*> host_oracle_replay_check expected=`expected` observed=`observed`*
+9
View File
@@ -211,6 +211,10 @@ fn parse_sse_events(
{
accumulated.push_str(c);
}
// Preserve provider usage as a first-class event for llm.collect_usage.
if let Some(u) = v.get("usage").filter(|u| u.is_object()) {
events.push(json!({"type": "usage", "usage": u}));
}
}
events.push(json!({"type": "data", "data": data}));
}
@@ -365,6 +369,11 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
}
if stream {
req_body.insert("stream".into(), Value::Bool(true));
// OpenAI / DeepSeek / compatible: final SSE chunk carries usage only when requested.
req_body.insert(
"stream_options".into(),
json!({ "include_usage": true }),
);
}
let body = Value::Object(req_body).to_string();
let auth = format!("{bearer}{api_key}");
+3 -1
View File
@@ -17,7 +17,9 @@ The language is still early: structural markers, English/Chinese builtin aliases
中文文档请看同目录的 [`00-欢迎.mq.md`](00-欢迎.mq.md)(叙述与内置均用中文)。
Want a quick hello? Calls only run when they sit in the entry `# main` — `##` children are methods until something calls them. Try **greet who="marqdo"**. Names can include spaces: **greet who="marqdo user"**.
Want a quick hello? `marqdo run` executes the entry `# main`. Try **greet who="marqdo"**. Names can include spaces: **greet who="marqdo user"**.
This version also runs one function on its own. `marqdo call` takes the file, the function and named arguments. The page you read is the program that runs. What that is for, and why it matters, is in [features/22-call.mq.md](features/22-call.mq.md).
Beyond the basics, Marqdo ships table, time, text, files, system, math, foreign, and more. Math and foreign deserve a special mention.
+3 -1
View File
@@ -17,7 +17,9 @@ AI 偏爱 Markdown 结构的输出。我当时的念头很简单——何必再
English docs: [`00-welcome.mq.md`](00-welcome.mq.md)(English prose and English builtins).
想先打个招呼?调用写在入口 `# main` 里才会真的跑——子标题里的方法要等你叫到它。比如:**问候 谁="marqdo"**。名字里可以带空格:**问候 谁="marqdo user"**。
想先打个招呼?`marqdo run` 会运行入口 `# main`。比如:**问候 谁="marqdo"**。名字里可以带空格:**问候 谁="marqdo user"**。
从这一版起,不必为了试用一个函数再包一层入口。`marqdo call` 直接运行文件里的某个函数,参数写在命令上。同一页仍然是给人读的文档。作用和价值写在 [features/22-函数调用.mq.md](features/22-函数调用.mq.md)。
除了这些基本面,标准库里还有表、时间、文本、文件、系统、数学、外联等。我特别想提数学和外联。
+1
View File
@@ -14,6 +14,7 @@
| [`features/19-foreign-view-cells.mq.md`](features/19-foreign-view-cells.mq.md) / [`19-外联视图单元.mq.md`](features/19-外联视图单元.mq.md) | 外联 v2 完整 Python 示例:probe / JSON call / 并行 / 持久内核 / 状态桥 / view 单元 |
| [`features/20-okf-ekc.mq.md`](features/20-okf-ekc.mq.md) / [`20-OKF与EKC.mq.md`](features/20-OKF与EKC.mq.md) | 代码即文档、文档即知识:OKF 形态与 EKC 工程知识编译器 |
| [`features/21-persistence.mq.md`](features/21-persistence.mq.md) / [`21-持久化.mq.md`](features/21-持久化.mq.md) | 外联结果持久化:静态快照与 writeback |
| [`features/22-call.mq.md`](features/22-call.mq.md) / [`22-函数调用.mq.md`](features/22-函数调用.mq.md) | `marqdo call`:同一份文档按函数执行,知识可以直接当工具 |
官方扩展(非 stdlib):`ext/llm`、`ext/agent`、`ext/web`、`ext/quantum`、`ext/linalg` — 见 `features/05-*.mq.md`、`06-*.mq.md`、`07-linalg.mq.md` / `07-线性代数.mq.md` 与 `doc/design/ext-*.md`。
+59
View File
@@ -0,0 +1,59 @@
---
markup: occlusion
title: Call one function: the same document is the tool
description: v2.2.0 marqdo call runs a function inside any .mq.md. The explanation, the implementation and the call are one page.
---
# main
v2.2.0 adds one command: `marqdo call`. It adds no Markdown marker and no skill system. An ordinary .mq.md file is already a document, a piece of knowledge and a program. What was missing is a way to call one of its functions from outside.
The usual command runs the whole file. Functions inside it run only when the entry calls them. Trying one calculation meant writing an entry, a print and a check. The smallest unit you can execute is now a function.
```text
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3
```
Standard output is 5. `--json` is for programs and agents: success carries the result, failure carries a diagnostic code, and the process exits non-zero. `--backend tree` and `--backend bytecode` both run that function. Another entry in the file is not used as a fallback.
The second half of this page is itself a calculator. This command calls this page:
```text
marqdo call public/features/22-call.mq.md calculator.add a=2 b=3
```
---
The change is how the file is used.
A person reads the sentences next to the function and sees what add computes. The compiler reads the same markers and checks parameters and the return. A developer composes the function into another program. A test passes inputs to this function and looks at the output, without a wrapper whose only job is to print. An agent reads what the function is for and runs the implementation that is already there, instead of reading the page and rewriting it.
Those four uses are one call. Knowledge becomes a tool because knowledge can already run, now at function scope. The file does not need a special name, and it does not need to declare itself a skill.
---
The command keeps a boundary.
An external call runs a pure function. Writing files, starting a process, using the network or loading a plugin is refused before the function runs, so the side effect does not happen. Wider permission stays on the checks the project already has. An instance method needs its receiver; this command does not invent one.
The function must resolve to exactly one declaration. No match, or one short name that hits two functions, is a structured error. The command does not pick the first hit. Arguments bind through the signature and contract that the function already has. Missing, unknown and ill-typed arguments are diagnostics.
After editing a function, call it again with the inputs that failed. A passing check is a reason to keep the edit. A test does not prove every input. It gives the edit a check that can be repeated.
# calculator
This is an ordinary Marqdo document.
It holds a little reusable arithmetic.
## add
Add the numbers `a` and `b`.
*[result](a + b)*
## multiply
Multiply the numbers `a` and `b`.
*[result](a * b)*
+59
View File
@@ -0,0 +1,59 @@
---
markup: occlusion
title: 函数级调用:同一份文档成为工具
description: v2.2.0 的 marqdo call 让任意 .mq.md 里的函数可以带参数执行。说明、实现和调用是同一页。
---
# main
v2.2.0 增加一条命令:`marqdo call`。它不新增 Markdown 标记,也不另建一套技能系统。一个普通的 .mq.md 本来就是文档、知识和程序。缺的是从外部直接调用其中某个函数。
以前日常用法是运行整份文件。入口对象跑完,里面的函数要等文件自己去调用。要试验一个计算,往往得再写入口、打印和检查。现在最小可执行单位是函数。
```text
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3
```
标准输出是 5。`--json` 给程序和 Agent 读:成功时是带结果的对象,失败时是带诊断码的对象,退出码非 0。`--backend tree` 与 `--backend bytecode` 执行同一个函数。文件里即使还有别的入口,也不会改去跑那个入口。
本页后半本身就是一份计算器。下面这条命令调用的就是这一页:
```text
marqdo call public/features/22-函数调用.mq.md 计算器.加法 a=2 b=3
```
---
它解决的是使用方式,不是再写一份说明书。
人打开这一页,读函数旁边的句子,知道加法在算什么。编译器解析同一批标记,检查参数和返回。开发者把函数嵌进别的程序。测试对这个函数传入输入,看输出是不是期望值,不必为它包一层只用来打印的程序。Agent 读到的是函数的用途,执行的是已经写好的实现,而不是读完说明再临时重写一遍。
这四件事共用一次调用。知识成为工具,是「知识即可执行」在函数这一层上的结果。文件不需要改名,也不需要声明自己是技能。运行时和工具清单都不会把一页普通文档挡在门外。
---
边界也写在命令里。
外部调用默认只执行纯函数。写文件、起进程、访问网络、加载插件,会在运行前被拒绝,副作用不会发生。要放宽权限,仍走项目里已有的验证,而不是在这条命令上新开一个开关。实例方法需要接收者,不能靠这条命令凭空调用。
函数名必须能唯一对上。没有这个函数,或同一个短名指向两个函数,都会返回结构化错误,不会擅自挑第一个。参数按函数已有的签名和契约绑定:缺了、多了、类型不对,都是诊断,不是静默猜一个值。
改一个函数之后,用原来的输入再调用一次。结果对了,这个实现就留下来;不对,就还是原来的实现。测试不能证明所有输入都正确。它让修改有一个可以重复的检查点。
# 计算器
这是一个普通的 Marqdo 文档。
其中包含一些可以复用的数学计算知识。
## 加法
计算两个数字`a`和`b`的和。
*[计算结果](a + b)*
## 乘法
计算两个数字`a`和`b`的积。
*[计算结果](a * b)*
+30 -12
View File
@@ -21,7 +21,7 @@ Marqdo = **Markdown markers as syntax**. A `.mq.md` file is both documentation a
These rules are the **current default**. Do **not** teach Classic `[key](map)` index or “bold is return of print”. Classic migration only: [migration-classic.md](migration-classic.md).
1. **File suffix** `.mq.md`. Print with `> print text=…` / `> 打印 内容=…` — never use `**…**` to print.
2. **`**…**` = code / return expression.** `*…*` = statement (bind / expr). Closing markers touch the last token (no trailing space).
2. **`**…**` = statement** (assign / call / expression). `*…*` = return. Closing markers touch the last token (no trailing space).
3. **`[描述](payload)` = occlusion:** `[]` is prose (never executed); `()` is the only executed part. Empty `()` → `None` stub.
4. **Read collections with `access(coll, key)`** (aliases `取` / `取值`). **Never** use `[key](map)` as index under default Occlusion.
5. **Foreach:** `- [描述](遍历 coll 为 item)` (`遍历`/`for`, `为`/`as`/`in`). Prefer explicit traversal when generating new code.
@@ -34,8 +34,8 @@ These rules are the **current default**. Do **not** teach Classic `[key](map)` i
| Marker | Meaning (v2.1) |
|--------|----------------|
| `#` / `##`+ | Object/type · function/method |
| `*…*` | Statement |
| `**…**` | Return / code expression |
| `**…**` | Statement (assign / call / expression) |
| `*…*` | Return |
| `[描述](payload)` | Occlusion step (payload executes) |
| `access(c, k)` | Collection read |
| `> fn …` | Call |
@@ -57,6 +57,24 @@ Executable-knowledge gate: [`tests/executable_knowledge/`](../../tests/executabl
8. Browser: Marqdo on WASM + `lib/browser` tables; do not hand-write business JS.
## Call one function
```markdown
# 计算器
## 加法
计算两个数字`a`和`b`的和。
*[计算结果](a + b)*
```
```
marqdo call calculator.mq.md 计算器.加法 a=2 b=3
```
prints `5`. The same command works on any `.mq.md`; the file does not need a special skill name.
## Minimal program
```markdown
@@ -96,13 +114,13 @@ Chinese builtins (same functions, no import):
## Statements, returns, branches
```markdown
*x = 1*
*y = x + 2*
**x = 1**
**y = x + 2**
## add_one
+ `n`
**n + 1**
*n + 1*
```
```markdown
@@ -130,9 +148,9 @@ import:
# main
*xs = > text.split value="a,b,c" sep=","*
**xs = > text.split value="a,b,c" sep=","**
> print text=`xs`
*t = > clock.now_unix*
**t = > clock.now_unix**
```
- Import **English** or **Chinese** library file; call that file’s API names via `lib.member` (do not mix languages).
@@ -191,8 +209,8 @@ Install: `marqdo ext add quantum` (ZH id: `量子`). Build: `cargo build --relea
**Q8 themes (circuit / probs / bloch):** `theme="dark"|"light"|"bw"` (ZH `主题=`; **default `dark`** — slate lab look, gate-family colors, label chips so wires never cross `q0` text). View plot chrome follows `data-theme` on the SVG. Advanced kinds (hinton/…) still Q8c pending full re-skin.
```markdown
*_ = > `qc`.draw kind="circuit" theme="dark"*
*_ = > `qc`.draw kind="probs" theme="light"*
**_ = > `qc`.draw kind="circuit" theme="dark"**
**_ = > `qc`.draw kind="probs" theme="light"**
```
Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7.md](../../doc/design/ext-quantum-q7.md) · Q8 viz: [ext-quantum-viz-style.md](../../doc/design/ext-quantum-viz-style.md) · examples: [quantum-bell](../../examples/quantum-bell/) · [quantum-entanglement](../../examples/quantum-entanglement/).
@@ -223,8 +241,8 @@ Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7
| `# main` then prose then code with no blank line | Blank line before code |
| `if x > 0:` | `1. `x` > 0` |
| `* > print text=hi *` | `> print text=hi` |
| `` *`a` = 1 * `` (trailing space + backticks) | `*a = 1*` — bare bind, no trailing space before `*` / `**` |
| `*xs = > text.split value=a,b sep=,*` (bare text args) | `*xs = > text.split value="a,b" sep=","*` — quote text literals; bare words are variables |
| `` *`a` = 1 * `` (trailing space + backticks) | `**a = 1**` — bare bind, no trailing space before `*` / `**` |
| `*xs = > text.split value=a,b sep=,*` (italic is return, and bare text args) | `**xs = > text.split value="a,b" sep=","**` — statement; quote text literals |
| Forgetting empty return after nested `##` helper when outer continues | Add `*None*` / `**` / `****` (not `---`) |
| Import `lib/text` then call bare `split` | `> text.split …` (qualified) |
| Import `lib/text` then call `拆分` | Match file language (`text.split`, not 文本) |
+293
View File
@@ -0,0 +1,293 @@
//! Application compositions with precomputed independent SQL/Floyd oracles.
use crate::host::{
foreign_doc::{packing, DocumentSession},
json::{json_to_value, value_to_json},
};
use crate::value::Value;
use serde_json::{json, Value as Json};
use sha2::{Digest, Sha256};
use std::path::{Path, PathBuf};
fn root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
}
fn cases() -> Vec<Json> {
let path = std::env::var_os("MARQDO_APPLICATION_CASES")
.map(PathBuf::from)
.unwrap_or_else(|| root().join("tests/applications/CASES.json"));
let rows: Vec<Json> = serde_json::from_slice(&std::fs::read(path).unwrap()).unwrap();
assert!(!rows.is_empty());
rows
}
fn out() -> PathBuf {
let path = std::env::var_os("MARQDO_APPLICATION_ARTIFACT_DIR")
.map(PathBuf::from)
.unwrap_or_else(|| root().join("target/applications"));
std::fs::create_dir_all(&path).unwrap();
path
}
fn code(case: &Json, stage: &str) -> String {
let lang = if stage == "producer" {
case["producer"].as_str().unwrap()
} else {
case["consumer"].as_str().unwrap()
};
let ext = if lang == "python" { "py" } else { "js" };
std::fs::read_to_string(
root()
.join("tests/applications")
.join(case["family"].as_str().unwrap())
.join(format!("{stage}.{ext}")),
)
.unwrap()
}
fn failure(lang: &str) -> &'static str {
if lang == "python" {
"retained = answer\nraise RuntimeError('application_expected_failure')"
} else {
"globalThis.retained = answer; throw new Error('application_expected_failure');"
}
}
fn mutate(lang: &str) -> &'static str {
if lang == "python" {
"bridge = None\nNone"
} else {
"bridge = null; null;"
}
}
fn field<'a>(value: &'a Value, name: &str) -> &'a Value {
let Value::Map(fields) = value else {
panic!("expected record: {value:?}")
};
&fields.iter().find(|(k, _)| k == name).unwrap().1
}
fn save(path: &Path, value: &Json) {
std::fs::write(path, serde_json::to_vec_pretty(value).unwrap()).unwrap();
}
#[test]
fn compositions_preserve_data_failure_recovery_and_copy_isolation_on_both_backends() {
let out = out();
let rows = cases();
for case in &rows {
let id = case["id"].as_str().unwrap();
let q = |s: &str| serde_json::to_string(s).unwrap();
let input = q(&case["input"].to_string());
let family = root().join("lib");
let consumer = case["consumer"].as_str().unwrap();
let source = format!(
r#"---
markup: occlusion
import:
foreign: {foreign}
json: {json}
---
# main
**data = > json.parse text={input}**
**p = > foreign.session_open lang={producer}**
**c = > foreign.session_open lang={consumer}**
**> foreign.session_set id=`p` name="data" value=`data`**
**first = > evaluate target=`p` source={produce}**
**bridge = > foreign.session_get id=`p` name="bridge"**
**> foreign.session_set id=`c` name="bridge" value=`bridge`**
**second = > evaluate target=`c` source={consume}**
**failure = > evaluate target=`c` source={failure}**
**kept = > foreign.session_get id=`c` name="retained"**
**cleared = > evaluate target=`c` source={mutate}**
**empty = > foreign.session_get id=`c` name="bridge"**
**original = > foreign.session_get id=`p` name="bridge"**
**answer = > foreign.session_get id=`c` name="answer"**
**> foreign.session_close id=`p`**
**> foreign.session_close id=`c`**
*answer*
## evaluate
+ `target`
+ `source`
**record = > foreign.session_eval id=`target` source=`source`**
*record*
"#,
foreign = family.join("foreign.mq.md").display(),
json = family.join("json.mq.md").display(),
producer = q(case["producer"].as_str().unwrap()),
consumer = q(consumer),
produce = q(&code(case, "producer")),
consume = q(&code(case, "consumer")),
failure = q(failure(consumer)),
mutate = q(mutate(consumer))
);
let path = out.join(format!("{id}.mq.md"));
std::fs::write(&path, &source).unwrap();
let module = crate::load::load_module(&path).unwrap();
let mut executions = Vec::new();
for backend in ["tree", "bytecode"] {
let (result, stdout, (journal, protocol, live)) = if backend == "tree" {
let mut tree = crate::interp::Interpreter::with_capture(Some(&path), false);
let result = tree.run_module(&module);
(
result,
tree.captured_stdout.clone(),
tree.application_evidence(),
)
} else {
let program = crate::bytecode::compile_module(Some(&path), &module).unwrap();
let mut vm = crate::bytecode::Vm::with_capture(Some(&path));
let result = vm.run(&program);
(
result,
vm.captured_stdout.clone(),
vm.application_evidence(),
)
};
let error = result.as_ref().err().map(|e| e.to_string());
let actual = result.ok().map(|v| value_to_json(&v).unwrap());
executions.push(json!({"backend":backend,"ok":error.is_none(),"error":error,"result":actual,"stdout":stdout,"live_sessions":live,
"journal":value_to_json(&Value::List(journal)).unwrap(),"protocol":protocol.iter().map(|p|json!({"session":p.session,"request":p.request,"response":p.response})).collect::<Vec<_>>()}));
}
let result = json!({"version":1,"kind":"document","case":case,"source":source,"source_sha256":format!("{:x}",Sha256::digest(source.as_bytes())),"executions":executions});
save(&out.join(format!("{id}-document.json")), &result);
for execution in result["executions"].as_array().unwrap() {
assert_eq!(execution["result"], case["answer"], "{id}: {execution}");
assert_eq!(execution["ok"], true, "{id}: {execution}");
assert_eq!(execution["stdout"], "");
assert_eq!(execution["live_sessions"], 0);
}
}
println!(
"APPLICATION_METRIC:{}",
json!({"cases":rows.len(),"executions":rows.len()*2})
);
}
#[test]
fn application_action_histories_replay_raw_and_packed_after_failure() {
let out = out();
let rows = cases();
for case in &rows {
let id = case["id"].as_str().unwrap();
let consumer = case["consumer"].as_str().unwrap();
let source=format!("---\nimport:\n foreign: {}\n---\n# main\n\n`会话` = > foreign.session\n\n`produce` =\n```{}\n{}\n```\n\n`consume` =\n```{}\n{}\n```\n\n`failure` =\n```{}\n{}\n```\n\n`mutate` =\n```{}\n{}\n```\n",root().join("lib/foreign.mq.md").display(),case["producer"].as_str().unwrap(),code(case,"producer"),consumer,code(case,"consumer"),consumer,failure(consumer),consumer,mutate(consumer));
let path = out.join(format!("{id}-interactive.mq.md"));
std::fs::write(&path, &source).unwrap();
let mut session = DocumentSession::open(&path, &root()).unwrap();
let p = session
.cells
.iter()
.find(|c| c.name == "produce")
.unwrap()
.kernel
.clone();
let c = session
.cells
.iter()
.find(|c| c.name == "consume")
.unwrap()
.kernel
.clone();
let expected = json_to_value(&case["answer"]).unwrap();
session
.set_value(&p, "data", json_to_value(&case["input"]).unwrap())
.unwrap();
let first = session.run_cell("produce", None).unwrap();
assert_eq!(field(&first, "ok"), &Value::Bool(true), "{id}: {first:?}");
let bridge = session.pull_value(&p, "bridge").unwrap();
assert_eq!(value_to_json(&bridge).unwrap(), case["bridge"]);
session.set_value(&c, "bridge", bridge.clone()).unwrap();
let second = session.run_cell("consume", None).unwrap();
assert_eq!(field(&second, "result"), &expected, "{id}: {second:?}");
let failed = session.run_cell("failure", None).unwrap();
assert_eq!(field(&failed, "ok"), &Value::Bool(false));
assert_eq!(
session
.cells
.iter()
.find(|c| c.name == "failure")
.unwrap()
.last_result,
Value::None
);
assert_eq!(session.pull_value(&c, "retained").unwrap(), expected);
session.run_cell("mutate", None).unwrap();
assert_eq!(
session
.cells
.iter()
.find(|c| c.name == "mutate")
.unwrap()
.last_result,
Value::None
);
assert_eq!(session.pull_value(&c, "bridge").unwrap(), Value::None);
assert_eq!(session.pull_value(&p, "bridge").unwrap(), bridge);
assert_eq!(session.pull_value(&c, "answer").unwrap(), expected);
let snapshot = session.export_snapshot().unwrap();
session.close();
let packed = packing::pack(&snapshot).unwrap();
assert_eq!(
value_to_json(&packing::unpack(&packed).unwrap()).unwrap(),
value_to_json(&snapshot).unwrap()
);
let raw_check =
DocumentSession::check_snapshot_isolated(&path, &root(), &snapshot).unwrap();
let packed_check =
DocumentSession::check_snapshot_isolated(&path, &root(), &packed).unwrap();
save(
&out.join(format!("{id}-interactive.json")),
&json!({"version":1,"kind":"interactive","case":case,"source":source,"source_sha256":format!("{:x}",Sha256::digest(source.as_bytes())),"snapshot":value_to_json(&snapshot).unwrap(),"packed":value_to_json(&packed).unwrap(),"raw_check":value_to_json(&raw_check).unwrap(),"packed_check":value_to_json(&packed_check).unwrap()}),
);
assert_eq!(
field(&raw_check, "ok"),
&Value::Bool(true),
"{id}: {raw_check:?}"
);
assert_eq!(
field(&packed_check, "ok"),
&Value::Bool(true),
"{id}: {packed_check:?}"
);
}
println!(
"APPLICATION_REPLAY_METRIC:{}",
json!({"cases":rows.len(),"strict_replays":rows.len()*2})
);
}
#[test]
fn explicit_null_and_stdout_remain_distinct_and_only_success_can_persist() {
for lang in ["python", "node"] {
let (good, bad) = if lang == "python" {
(
"print('audit')\nNone",
"print('failed audit')\nraise RuntimeError('expected_failure')",
)
} else {
(
"console.log('audit'); null;",
"console.log('failed audit'); throw Error('expected_failure');",
)
};
let path = out().join(format!("null-persist-{lang}-{}.mq.md", std::process::id()));
std::fs::write(&path,format!("---\nimport:\n foreign: {}\n---\n# main\n\n`会话` = > foreign.session\n\n`good` =\n```{lang}\n{good}\n```\n\n`bad` =\n```{lang}\n{bad}\n```\n",root().join("lib/foreign.mq.md").display())).unwrap();
let mut session = DocumentSession::open(&path, &root()).unwrap();
assert!(session.persist_cell("good", "empty").is_err());
session.run_cell("good", None).unwrap();
let cell = session.cells.iter().find(|c| c.name == "good").unwrap();
assert_eq!(cell.last_result, Value::None);
assert_eq!(cell.last_stdout, "audit\n");
assert_eq!(session.oracle_journal.last().unwrap().value, Value::None);
assert_eq!(session.oracle_journal.last().unwrap().result, Value::None);
session.persist_cell("good", "empty").unwrap();
session.run_cell("bad", None).unwrap();
assert_eq!(
session
.cells
.iter()
.find(|c| c.name == "bad")
.unwrap()
.last_result,
Value::None
);
assert!(session.persist_cell("bad", "must_not_write").is_err());
session.close();
}
}
+122
View File
@@ -0,0 +1,122 @@
//! Value-independent argument binding. Effects finish before this plan is used.
use std::collections::HashMap;
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct ArgumentPlan {
pub parameters: Vec<Option<usize>>,
pub assignments: Vec<(String, usize)>,
}
pub(crate) fn plan(
params: &[String],
names: &[Option<String>],
allow_missing: bool,
) -> Result<ArgumentPlan, String> {
plan_with(params, names, |_| allow_missing)
}
pub(crate) fn plan_with_omissions(
params: &[String],
names: &[Option<String>],
omissions: &[bool],
) -> Result<ArgumentPlan, String> {
if params.len() != omissions.len() {
return Err("argument omission mask count mismatch".into());
}
plan_with(params, names, |index| omissions[index])
}
fn plan_with(
params: &[String],
names: &[Option<String>],
may_omit: impl Fn(usize) -> bool,
) -> Result<ArgumentPlan, String> {
let mut named = HashMap::new();
let mut positionals = Vec::new();
for (index, name) in names.iter().enumerate() {
match name {
Some(name) => {
if named.insert(name.clone(), index).is_some() {
return Err(format!("duplicate named argument `{name}`"));
}
}
None => positionals.push(index),
}
}
let mut pos = 0;
let mut parameters = Vec::new();
let mut assignments = Vec::new();
for (parameter_index, param) in params.iter().enumerate() {
let index = if let Some(index) = named.get(param) {
Some(*index)
} else if let Some(index) = positionals.get(pos) {
pos += 1;
Some(*index)
} else if may_omit(parameter_index) {
None
} else {
return Err(format!("missing argument for parameter `{param}`"));
};
parameters.push(index);
if let Some(index) = index {
assignments.push((param.clone(), index));
}
}
if pos < positionals.len() {
return Err(format!(
"too many positional arguments ({} extra)",
positionals.len() - pos
));
}
for (index, name) in names.iter().enumerate() {
if let Some(name) = name {
if !params.contains(name) {
assignments.push((name.clone(), index));
}
}
}
Ok(ArgumentPlan {
parameters,
assignments,
})
}
pub(crate) fn bind<T: Clone>(
params: &[String],
names: &[Option<String>],
values: &[T],
allow_missing: bool,
) -> Result<HashMap<String, T>, String> {
if names.len() != values.len() {
return Err("argument layout/value count mismatch".into());
}
let plan = plan(params, names, allow_missing)?;
Ok(plan
.assignments
.into_iter()
.map(|(name, index)| (name, values[index].clone()))
.collect())
}
pub(crate) fn plan_evidence(
params: &[String],
names: &[Option<String>],
omissions: &[bool],
) -> serde_json::Value {
let result = match plan_with_omissions(params, names, omissions) {
Ok(plan) => {
serde_json::json!({"ok":true,"parameters":plan.parameters,"assignments":plan.assignments})
}
Err(error) => serde_json::json!({"ok":false,"error":error}),
};
serde_json::json!({"params":params,"arguments":names,"omissions":omissions,"result":result})
}
pub(crate) fn builtin_signature(name: &str) -> Option<(&'static [&'static str], bool)> {
Some(match name {
"print" => (&["text"], false),
"input" => (&["prompt"], true),
"len" | "str" | "int" | "type" | "trim" => (&["value"], false),
"split" | "join" => (&["value", "sep"], false),
"at" => (&["value", "index"], false),
"call_fn" => (&["name"], false),
_ => return None,
})
}
+47 -13
View File
@@ -22,6 +22,7 @@ pub struct Use {
#[derive(Debug, Clone)]
pub struct Module {
pub source: crate::diagnostics::SourceIdentity,
pub imports: Vec<Import>,
pub uses: Vec<Use>,
pub functions: Vec<Function>,
@@ -36,6 +37,7 @@ pub struct Module {
impl Default for Module {
fn default() -> Self {
Self {
source: Default::default(),
imports: Vec::new(),
uses: Vec::new(),
functions: Vec::new(),
@@ -46,9 +48,31 @@ impl Default for Module {
}
}
/// Parameter names have distinct slots. Function lookup retains its existing
/// first-match/ambiguity rules rather than narrowing accepted declaration trees.
pub(crate) fn validate_parameters(module:&Module) -> Result<(),crate::diagnostics::Diagnostic> {
fn siblings(module:&Module,functions:&[Function]) -> Result<(),crate::diagnostics::Diagnostic> {
for fun in functions {
let mut params=std::collections::HashSet::new();
for param in &fun.params {
if !params.insert(&param.name) {
return Err(crate::diagnostics::Diagnostic::new(module.source.path.as_deref(),param.span,format!("duplicate parameter `{}`",param.name)).with_source(&module.source));
}
}
siblings(module,&fun.children)?;
}
Ok(())
}
siblings(module,&module.functions)?;
let mut imports=module.import_modules.iter().collect::<Vec<_>>();imports.sort_by_key(|(name,_)|*name);
for (_,child) in imports { validate_parameters(child)?; }
Ok(())
}
#[derive(Debug, Clone)]
pub struct Param {
pub name: String,
pub span: Span,
pub default: Option<Expr>,
/// True when collected by v0.3 body/prose inference (not an explicit `+` line).
pub inferred: bool,
@@ -91,6 +115,16 @@ pub fn function_uses_receiver(fun: &Function) -> bool {
matches!(name, "self" | "自")
}
fn call_uses_receiver(call: &CallExpr) -> bool {
call.receiver.as_deref().is_some_and(is_receiver)
|| call.path.as_ref().is_some_and(|path| {
path.first().is_some_and(|name| is_receiver(name))
})
|| call.args.iter().any(|arg| match arg {
Arg::Positional(value) | Arg::Named { value, .. } => expr_uses_receiver(value),
})
}
fn expr_uses_receiver(expr: &Expr) -> bool {
match expr {
Expr::Var(name) => is_receiver(name),
@@ -102,12 +136,7 @@ pub fn function_uses_receiver(fun: &Function) -> bool {
Expr::Binary { left, right, .. } => {
expr_uses_receiver(left) || expr_uses_receiver(right)
}
Expr::Call(call) => {
call.receiver.as_deref().is_some_and(is_receiver)
|| call.args.iter().any(|arg| match arg {
Arg::Positional(value) | Arg::Named { value, .. } => expr_uses_receiver(value),
})
}
Expr::Call(call) => call_uses_receiver(call),
Expr::List(items) => items.iter().any(expr_uses_receiver),
Expr::Map(entries) => entries.iter().any(|(_, value)| expr_uses_receiver(value)),
Expr::Index { base, label } => {
@@ -124,12 +153,7 @@ pub fn function_uses_receiver(fun: &Function) -> bool {
Stmt::Assign { value, .. } | Stmt::Return { value, .. } | Stmt::Expr { value, .. } => {
expr_uses_receiver(value)
}
Stmt::Call { call, .. } => {
call.receiver.as_deref().is_some_and(is_receiver)
|| call.args.iter().any(|arg| match arg {
Arg::Positional(value) | Arg::Named { value, .. } => expr_uses_receiver(value),
})
}
Stmt::Call { call, .. } => call_uses_receiver(call),
Stmt::Branch { arms, .. } => arms.iter().any(|arm| {
arm.condition.as_ref().is_some_and(expr_uses_receiver)
|| arm.body.iter().any(stmt_uses_receiver)
@@ -143,7 +167,8 @@ pub fn function_uses_receiver(fun: &Function) -> bool {
}
}
fun.body.iter().any(stmt_uses_receiver)
fun.params.iter().any(|param| param.default.as_ref().is_some_and(expr_uses_receiver))
|| fun.body.iter().any(stmt_uses_receiver)
}
/// Source line of a statement (every variant carries a span).
@@ -257,6 +282,15 @@ pub struct CallExpr {
}
impl CallExpr {
/// Shared execution preparation: normalize bare aliases before testing
/// modifier conflicts, matching the source evaluator's binding order.
pub fn for_evaluation(&self) -> Result<CallExpr,String> {
let mut call=self.clone();
if call.path.is_none() && call.receiver.is_none(){
call.callee=crate::aliases::normalize_call_callee_and_args(&call.callee,&mut call.args);
}
call.with_modifiers_expanded()
}
/// Fold `pre_modifiers` into leading `name=True` named args (conflicts → Err).
pub fn with_modifiers_expanded(&self) -> Result<CallExpr, String> {
let mut call = self.clone();
+30 -1
View File
@@ -1,7 +1,29 @@
//! Built-in helpers shared by tree-walk and bytecode.
mod primitives;
pub use primitives::{eval_binary,eval_unary};
use crate::value::Value;
/// Machine integer arithmetic has one profile-independent failure contract.
/// Overflow must remain a language diagnostic in debug, release and Wasm.
pub fn checked_int_binary(op: crate::ast::BinaryOp, left: i64, right: i64) -> Result<i64, String> {
use crate::ast::BinaryOp;
if matches!(op, BinaryOp::Div) && right == 0 { return Err("division by zero".into()); }
let result = match op {
BinaryOp::Add => left.checked_add(right),
BinaryOp::Sub => left.checked_sub(right),
BinaryOp::Mul => left.checked_mul(right),
BinaryOp::Div => left.checked_div(right),
_ => return Err("not an integer arithmetic operation".into()),
};
result.ok_or_else(|| "integer overflow".into())
}
pub fn checked_int_neg(value: i64) -> Result<i64, String> {
value.checked_neg().ok_or_else(|| "integer overflow".into())
}
/// `len`: text → Unicode scalar count; list → element count.
pub fn builtin_len(v: &Value) -> Result<i64, String> {
match v {
@@ -20,7 +42,13 @@ pub fn builtin_str(v: &Value) -> Value {
pub fn builtin_int(v: &Value) -> Result<i64, String> {
match v {
Value::Int(n) => Ok(*n),
Value::Num(n) => Ok(*n as i64),
Value::Num(n) => {
// The upper bound is exclusive: i64::MAX rounds to 2^63 as f64.
if !n.is_finite() || *n < -9223372036854775808.0 || *n >= 9223372036854775808.0 {
return Err("int conversion outside signed 64-bit domain".into());
}
Ok(*n as i64)
}
Value::Bool(true) => Ok(1),
Value::Bool(false) => Ok(0),
Value::Text(s) => {
@@ -42,6 +70,7 @@ pub fn builtin_type(v: &Value) -> Value {
Value::Num(_) => "num",
Value::Text(_) => "text",
Value::Secret(_) => "secret",
Value::Handle(_) => "handle",
Value::List(_) => "list",
Value::Map(_) => "map",
Value::Formula(_) => "formula",
+88
View File
@@ -0,0 +1,88 @@
//! Scalar operator semantics shared by source evaluation and VM.
use crate::{ast::BinaryOp, value::Value};
pub fn eval_binary(op: BinaryOp, l: &Value, r: &Value) -> Result<Value, String> {
match op {
BinaryOp::Add => match (l, r) {
(Value::Int(a), Value::Int(b)) => {
crate::builtin::checked_int_binary(op, *a, *b).map(Value::Int)
}
(Value::Num(a), Value::Num(b)) => Ok(Value::Num(a + b)),
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 + b)),
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a + *b as f64)),
(Value::Text(a), Value::Text(b)) => Ok(Value::Text(format!("{a}{b}"))),
(Value::Text(a), Value::Secret(b)) => Ok(Value::Text(format!("{a}{b}"))),
(Value::Secret(a), Value::Text(b)) => Ok(Value::Text(format!("{a}{b}"))),
(Value::Secret(a), Value::Secret(b)) => Ok(Value::Text(format!("{a}{b}"))),
(Value::Text(a), b) => Ok(Value::Text(format!("{a}{}", b.as_display()))),
(a, Value::Text(b)) => Ok(Value::Text(format!("{}{b}", a.as_display()))),
_ => Err("`+` needs ints, nums, or text".into()),
},
BinaryOp::Sub => match (l, r) {
(Value::Int(a), Value::Int(b)) => {
crate::builtin::checked_int_binary(op, *a, *b).map(Value::Int)
}
(Value::Num(a), Value::Num(b)) => Ok(Value::Num(a - b)),
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 - b)),
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a - *b as f64)),
_ => Err("`-` needs ints or nums".into()),
},
BinaryOp::Mul => match (l, r) {
(Value::Int(a), Value::Int(b)) => {
crate::builtin::checked_int_binary(op, *a, *b).map(Value::Int)
}
(Value::Num(a), Value::Num(b)) => Ok(Value::Num(a * b)),
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 * b)),
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a * *b as f64)),
_ => Err("`*` needs ints or nums".into()),
},
BinaryOp::Div => match (l, r) {
(Value::Int(_), Value::Int(0)) => Err("division by zero".into()),
(Value::Int(a), Value::Int(b)) => {
crate::builtin::checked_int_binary(op, *a, *b).map(Value::Int)
}
(Value::Num(_), Value::Num(b)) if *b == 0.0 => Err("division by zero".into()),
(Value::Int(_), Value::Num(b)) if *b == 0.0 => Err("division by zero".into()),
(Value::Num(_), Value::Int(0)) => Err("division by zero".into()),
(Value::Num(a), Value::Num(b)) => Ok(Value::Num(a / b)),
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 / b)),
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a / *b as f64)),
_ => Err("`/` needs ints or nums".into()),
},
BinaryOp::Eq => Ok(Value::Bool(l == r)),
BinaryOp::Ne => Ok(Value::Bool(l != r)),
BinaryOp::Lt => cmp_ord(l, r, |o| o.is_lt()),
BinaryOp::Le => cmp_ord(l, r, |o| o.is_le()),
BinaryOp::Gt => cmp_ord(l, r, |o| o.is_gt()),
BinaryOp::Ge => cmp_ord(l, r, |o| o.is_ge()),
BinaryOp::And | BinaryOp::Or => unreachable!("handled above"),
}
}
fn cmp_ord(l: &Value, r: &Value, f: impl Fn(std::cmp::Ordering) -> bool) -> Result<Value, String> {
match (l, r) {
(Value::Int(a), Value::Int(b)) => Ok(Value::Bool(f(a.cmp(b)))),
(Value::Num(a), Value::Num(b)) => Ok(Value::Bool(f(a
.partial_cmp(b)
.unwrap_or(std::cmp::Ordering::Equal)))),
(Value::Int(a), Value::Num(b)) => Ok(Value::Bool(f((*a as f64)
.partial_cmp(b)
.unwrap_or(std::cmp::Ordering::Equal)))),
(Value::Num(a), Value::Int(b)) => Ok(Value::Bool(f(a
.partial_cmp(&(*b as f64))
.unwrap_or(std::cmp::Ordering::Equal)))),
(Value::Text(a), Value::Text(b)) => Ok(Value::Bool(f(a.cmp(b)))),
_ => Err("comparison needs two ints/nums or two texts".into()),
}
}
/// Scalar unary semantics shared by source and target execution.
pub fn eval_unary(op: crate::ast::UnaryOp, value: &Value) -> Result<Value, String> {
match op {
crate::ast::UnaryOp::Not => Ok(Value::Bool(!value.truthy())),
crate::ast::UnaryOp::Neg => match value {
Value::Int(n) => crate::builtin::checked_int_neg(*n).map(Value::Int),
Value::Num(n) => Ok(Value::Num(-n)),
_ => Err("unary `-` needs int or num".into()),
},
}
}
+327
View File
@@ -0,0 +1,327 @@
//! Check actual staging, scratch separation and post-evaluation call selection.
//! Resolution and dispatch implementations remain semantic interfaces.
use super::{expression::Cursor, ExpressionRule, FnChunk, Op, Program};
use crate::{
ast::{Arg, CallExpr},
value::Value,
};
use std::collections::HashSet;
#[derive(Debug, Clone)]
pub struct CallStage {
pub expression: usize,
pub slot: usize,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum PathFallback {
User { target: usize, tag_instance: bool },
Error { message: String },
}
impl PathFallback {
fn evidence(&self) -> serde_json::Value {
match self {
Self::User { target, tag_instance } => serde_json::json!({"kind":"user","target":target,"tag_instance":tag_instance}),
Self::Error { message } => serde_json::json!({"kind":"error","message":message}),
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CallDispatch {
User { target: usize, tag_instance: bool },
Method { receiver: String },
Builtin,
Host { function: u16 },
Plugin,
RejectDuplicate { name: String },
Path { receiver: String, check_receiver: bool, fallback: PathFallback },
ResolutionFault { message: String },
}
#[derive(Debug, Clone)]
pub struct CallEmission {
pub expression: usize,
/// Expanded and normalized source, captured before argument replacement.
pub source: CallExpr,
pub stages: Vec<CallStage>,
pub dispatch: Option<CallDispatch>,
pub as_statement: bool,
}
impl CallEmission {
pub fn binding_evidence(&self,program:&Program)->serde_json::Value {
let signature=match self.dispatch.as_ref() {
Some(CallDispatch::User{target,..})|Some(CallDispatch::Path{fallback:PathFallback::User{target,..},..})=>{
program.functions.get(*target).map(|callee|(callee.params.clone(),callee.defaults.clone()))
},
Some(CallDispatch::Builtin)=>crate::arguments::builtin_signature(&self.source.callee).map(|(params,optional)|
(params.iter().map(|p|p.to_string()).collect::<Vec<_>>(),vec![optional;params.len()])),
Some(CallDispatch::Host{function})=>crate::host::HostFn::from_u16(*function).map(|h|{
let params=h.all_params();let len=params.len();(params.into_iter().map(str::to_string).collect(),vec![true;len])
}),
_=>None,
};
match signature {
Some((params,omissions))=>crate::arguments::plan_evidence(&params,&names(self),&omissions),
None=>serde_json::Value::Null,
}
}
pub fn evidence(&self) -> serde_json::Value {
use serde_json::json;
let dispatch = match &self.dispatch {
Some(CallDispatch::User {
target,
tag_instance,
}) => json!({"kind":"user","target":target,"tag_instance":tag_instance}),
Some(CallDispatch::Method { receiver }) => json!({"kind":"method","receiver":receiver}),
Some(CallDispatch::Builtin) => json!({"kind":"builtin"}),
Some(CallDispatch::Host { function }) => {
let host = crate::host::HostFn::from_u16(*function);
json!({"kind":"host","function":function,"params":host.map(|h|h.all_params()),"required":host.map(|h|h.required_params().len())})
}
Some(CallDispatch::Plugin) => json!({"kind":"plugin"}),
Some(CallDispatch::RejectDuplicate { name }) => json!({"kind":"duplicate","name":name}),
Some(CallDispatch::Path { receiver, check_receiver, fallback }) => json!({"kind":"path","receiver":receiver,"check_receiver":check_receiver,"fallback":fallback.evidence()}),
Some(CallDispatch::ResolutionFault { message }) => json!({"kind":"resolution_fault","message":message}),
None => serde_json::Value::Null,
};
json!({"expression":self.expression,"callee":self.source.callee,"receiver":self.source.receiver,"path":self.source.path,
"arguments":self.source.args.iter().map(|a|match a {Arg::Positional(_)=>None,Arg::Named{name,..}=>Some(name)}).collect::<Vec<_>>(),
"source_rules":self.source.args.iter().map(|a|match a {Arg::Positional(e)|Arg::Named{value:e,..}=>ExpressionRule::from_source(e).evidence()}).collect::<Vec<_>>(),
"stages":self.stages.iter().map(|s|json!({"expression":s.expression,"slot":s.slot})).collect::<Vec<_>>(),
"dispatch":dispatch,"as_statement":self.as_statement})
}
}
fn names(call: &CallEmission) -> Vec<Option<String>> {
call.source
.args
.iter()
.map(|a| match a {
Arg::Positional(_) => None,
Arg::Named { name, .. } => Some(name.clone()),
})
.collect()
}
fn read(c: &mut Cursor<'_>, slot: usize) -> Result<(), String> {
c.op(Op::GetLocal(slot))
}
fn read_child(
c: &mut Cursor<'_>,
ids: &mut std::slice::Iter<'_, usize>,
slot: usize,
) -> Result<(), String> {
let id = *ids.next().ok_or("missing staged argument reload witness")?;
let expr = c
.fun
.expressions
.get(id)
.ok_or("missing reload expression")?;
if expr.start != c.position
|| expr.end != c.position + 1
|| !matches!(&expr.rule,ExpressionRule::Read(name) if c.fun.locals.get(slot)==Some(name))
|| !expr.children.is_empty()
{
return Err("staged argument reload source differs".into());
}
read(c, slot)
}
fn layout(c: &mut Cursor<'_>, arguments: &[Option<String>]) -> Result<usize, String> {
let id = match c.fun.code.get(c.position) {
Some(Op::CallArgs(_, id) | Op::MethodCallArgs(_, id)) => *id,
_ => return Err("missing argument-layout call".into()),
};
if c.fun.call_layouts.get(id).map(Vec::as_slice) != Some(arguments) {
return Err("call argument layout differs from source".into());
}
Ok(id)
}
pub fn verify(program: &Program, fun: &FnChunk) -> Result<(), String> {
let caller=program.functions.iter().position(|f|std::ptr::eq(f,fun)).ok_or("missing call owner")?;
if fun.source_local_count < fun.params.len() || fun.source_local_count > fun.locals.len() {
return Err("invalid source/scratch local boundary".into());
}
let mut owners = HashSet::new();
let loops = fun
.statements
.iter()
.flat_map(|s| s.auxiliary.iter().copied())
.collect::<HashSet<_>>();
let mut active: Vec<(&super::ExpressionEmission, HashSet<usize>)> = Vec::new();
for call in &fun.calls {
let node = fun
.expressions
.get(call.expression)
.ok_or("missing call source expression")?;
if !matches!(node.rule, ExpressionRule::CallBoundary(_))
|| !owners.insert(call.expression)
|| call.as_statement
{
return Err("call source owner differs".into());
}
let arguments = names(call);
if call.stages.len() != arguments.len() || node.children.len() < arguments.len() {
return Err("call authored-actual inventory".into());
}
let slots = call.stages.iter().map(|s| s.slot).collect::<HashSet<_>>();
if slots.len() != arguments.len()
|| slots
.iter()
.any(|s| *s < fun.source_local_count || *s >= fun.locals.len() || loops.contains(s))
{
return Err("call scratch aliases source/loop/another actual".into());
}
while active
.last()
.is_some_and(|(parent, _)| parent.end <= node.start)
{
active.pop();
}
if active.iter().any(|(parent, other)| {
node.start < parent.start || node.end > parent.end || !slots.is_disjoint(other)
}) {
return Err("nested call scratch overlap or interval order".into());
}
let mut c = Cursor {
fun,
position: node.start,
end: node.end,
};
for (i, stage) in call.stages.iter().enumerate() {
if node.children[i] != stage.expression {
return Err("authored actual order differs".into());
}
let source = match &call.source.args[i] {
Arg::Positional(e) | Arg::Named { value: e, .. } => e,
};
if ExpressionRule::from_source(source).evidence()
!= fun.expressions[stage.expression].rule.evidence()
{
return Err("authored argument source rule differs".into());
}
c.child(stage.expression)?;
c.op(Op::SetLocal(stage.slot))?;
c.op(Op::Pop)?;
}
let mut reloads = node.children[arguments.len()..].iter();
let dispatch=call.dispatch.as_ref().ok_or("missing call dispatch obligation")?;
use crate::sema::{DynamicReason, resolution::Target};
let target=super::names::resolve(program,caller,&call.source);
let user=match &target { Target::Local(id)|Target::Import{fun:id,..}=>Some(*id),_=>None };
let user_dispatch=|id:usize|CallDispatch::User{target:id,tag_instance:program.parents[id].is_none()&&program.functions[id].source_ast.level==1};
let expected=if let Some(receiver)=&call.source.receiver {
CallDispatch::Method{receiver:receiver.clone()}
} else if let Some(path)=call.source.path.as_ref().filter(|p|p.len()==2) {
let fallback=match user.map(user_dispatch) {
Some(CallDispatch::User{target,tag_instance})=>PathFallback::User{target,tag_instance},
_=>PathFallback::Error{message:match target {Target::Unresolved{message,..}=>message,_=>format!("unresolved dynamic call `{}`",crate::sema::call_display(&call.source))}},
};
CallDispatch::Path{receiver:path[0].clone(),check_receiver:!fun.source_ast.imports.contains(&path[0]),fallback}
} else if let Some(id)=user {user_dispatch(id)} else {match target {
Target::Builtin(_)|Target::Dynamic{reason:DynamicReason::CallFn,..}=>CallDispatch::Builtin,
Target::Host(h)=>CallDispatch::Host{function:h.as_u16()},
Target::Dynamic{reason:DynamicReason::Plugin,..}=>CallDispatch::Plugin,
Target::Unresolved{message,..}=>CallDispatch::ResolutionFault{message},
_=>return Err("unmodeled source call resolution".into()),
}};
if *dispatch!=expected{return Err("call resolution differs from source declaration graph".into());}
match dispatch {
CallDispatch::Path{..}=>{
for stage in &call.stages{read_child(&mut c,&mut reloads,stage.slot)?;}
let id=match fun.code.get(c.position){Some(Op::PathCallArgs(owner,id)) if fun.calls.get(*owner).is_some_and(|other|std::ptr::eq(other,call))=>*id,_=>return Err("path call owner differs".into())};
if fun.call_layouts.get(id)!=Some(&arguments){return Err("path call layout differs".into());}
let owner=fun.calls.iter().position(|other|std::ptr::eq(other,call)).unwrap();
c.op(Op::PathCallArgs(owner,id))?;
},
CallDispatch::ResolutionFault{message}=>{
let index=match fun.code.get(c.position){Some(Op::RejectCallResolution(index))=>*index,_=>return Err("missing call resolution fault".into())};
if fun.constants.get(index)!=Some(&Value::Text(message.clone())){return Err("call resolution fault payload differs".into());}
c.op(Op::RejectCallResolution(index))?;
},
CallDispatch::RejectDuplicate { name } => {
let index = match fun.code.get(c.position) {
Some(Op::RejectDuplicateArgument(i)) => *i,
_ => return Err("duplicate argument refusal missing".into()),
};
if fun.constants.get(index) != Some(&Value::Text(name.clone())) {
return Err("duplicate argument name differs".into());
}
c.op(Op::RejectDuplicateArgument(index))?;
}
CallDispatch::User {
target,
tag_instance,
} => {
if *target >= program.functions.len()
|| *tag_instance != program.objects.iter().any(|(_, id)| id == target)
{
return Err("resolved call target/tag differs".into());
}
for stage in &call.stages {
read_child(&mut c, &mut reloads, stage.slot)?;
}
let id = layout(&mut c, &arguments)?;
c.op(Op::CallArgs(*target, id))?;
if *tag_instance {
c.op(Op::TagInstance(*target))?;
}
}
CallDispatch::Method { receiver } => {
if call.source.receiver.as_ref() != Some(receiver)
&& call.source.path.as_ref().and_then(|p| p.first()) != Some(receiver)
{
return Err("method receiver differs from source".into());
}
for stage in &call.stages {
read_child(&mut c, &mut reloads, stage.slot)?;
}
let slot = fun
.locals
.iter()
.position(|n| n == receiver)
.ok_or("method receiver slot")?;
read(&mut c, slot)?;
let index = match fun.code.get(c.position) {
Some(Op::MethodCallArgs(i, _)) => *i,
_ => return Err("method dispatch missing".into()),
};
if fun.constants.get(index) != Some(&Value::Text(call.source.callee.clone())) {
return Err("method name differs from source".into());
}
let id = layout(&mut c, &arguments)?;
c.op(Op::MethodCallArgs(index, id))?;
}
CallDispatch::Builtin|CallDispatch::Plugin|CallDispatch::Host{..}=>{
for stage in &call.stages{read_child(&mut c,&mut reloads,stage.slot)?;}
let (first,id)=match fun.code.get(c.position){Some(Op::BuiltinCallArgs(n,l)|Op::PluginCallArgs(n,l))=>(*n,*l),Some(Op::HostCallArgs(n,l))=>(*n as usize,*l),_=>return Err("missing native argument dispatch".into())};
if fun.call_layouts.get(id)!=Some(&arguments){return Err("native argument layout differs".into());}
match dispatch {
CallDispatch::Builtin=>{
if crate::arguments::builtin_signature(&call.source.callee).is_none() || fun.constants.get(first)!=Some(&Value::Text(call.source.callee.clone())){return Err("builtin identity differs".into());}
c.op(Op::BuiltinCallArgs(first,id))?;
},
CallDispatch::Plugin=>{
if fun.constants.get(first)!=Some(&Value::Text(call.source.callee.clone())){return Err("plugin identity differs".into());}
c.op(Op::PluginCallArgs(first,id))?;
},
CallDispatch::Host{function}=>{
let hf=crate::host::HostFn::from_name(&call.source.callee).ok_or("source host name")?;
if hf.as_u16()!=*function || first!=*function as usize{return Err("host primitive identity differs".into());}
c.op(Op::HostCallArgs(*function,id))?;
},_=>unreachable!(),
}
}
}
if reloads.next().is_some() || c.position != node.end {
return Err("call instruction/expression coverage differs".into());
}
active.push((node, slots));
}
if fun
.expressions
.iter()
.filter(|e| matches!(e.rule, ExpressionRule::CallBoundary(_)))
.count()
!= owners.len()
{
return Err("uncovered call source expression".into());
}
Ok(())
}
+402 -481
View File
File diff suppressed because it is too large. Load diff
+439
View File
@@ -0,0 +1,439 @@
//! Source-directed expression emission obligations. Call staging and selection
//! are checked separately in call.rs; dispatch semantics remain a proof boundary.
use super::{FnChunk, Op};
use crate::{
ast::{BinaryOp, Expr, IndexKey, InterpPart, Literal, UnaryOp},
value::Value,
};
#[derive(Debug, Clone)]
pub enum ExpressionRule {
Literal(Value),
Read(String),
Unary(UnaryOp),
Binary(BinaryOp),
Interpolation(Vec<InterpPart>),
List(usize),
Map(Vec<String>),
Index {
label: IndexKey,
base_variable: Option<String>,
},
Occluded(bool),
CallBoundary(Vec<String>),
CallFault(String),
}
impl ExpressionRule {
pub fn from_source(expr: &Expr) -> Self {
match expr {
Expr::Literal(value) => Self::Literal(match value {
Literal::None => Value::None,
Literal::Bool(v) => Value::Bool(*v),
Literal::Int(v) => Value::Int(*v),
Literal::Num(v) => Value::Num(*v),
Literal::Text(v) => Value::Text(v.clone()),
}),
Expr::Var(name) => Self::Read(name.clone()),
Expr::Unary { op, .. } => Self::Unary(*op),
Expr::Binary { op, .. } => Self::Binary(*op),
Expr::Interp(parts) => Self::Interpolation(parts.clone()),
Expr::List(items) => Self::List(items.len()),
Expr::Map(items) => Self::Map(items.iter().map(|(k, _)| k.clone()).collect()),
Expr::Index { base, label } => Self::Index {
label: label.clone(),
base_variable: if let Expr::Var(name) = base.as_ref() {
Some(name.clone())
} else {
None
},
},
Expr::Formula(value) => Self::Literal(Value::Formula(value.clone())),
Expr::Code(value) => Self::Literal(Value::Code(value.clone())),
Expr::Occluded { payload, .. } => Self::Occluded(payload.is_some()),
Expr::Call(call) if call.for_evaluation().is_err() => {
Self::CallFault(call.for_evaluation().unwrap_err())
}
Expr::Call(call) => Self::CallBoundary(
call.receiver
.iter()
.cloned()
.chain(call.path.iter().filter_map(|path| path.first().cloned()))
.collect(),
),
}
}
}
#[derive(Debug, Clone)]
pub struct ExpressionEmission {
pub rule: ExpressionRule,
pub start: usize,
pub end: usize,
pub children: Vec<usize>,
}
pub(super) fn formula_equal(a: &crate::formula::Expr, b: &crate::formula::Expr) -> bool {
use crate::formula::Expr::*;
match (a, b) {
(Num(x), Num(y)) => x.to_bits() == y.to_bits(),
(Var(x), Var(y)) => x == y,
(UnaryNeg(x), UnaryNeg(y)) => formula_equal(x, y),
(
Bin {
op: a,
left: x,
right: y,
},
Bin {
op: b,
left: u,
right: v,
},
) => a == b && formula_equal(x, u) && formula_equal(y, v),
(Call { name: a, args: x }, Call { name: b, args: y }) => {
a == b && x.len() == y.len() && x.iter().zip(y).all(|(a, b)| formula_equal(a, b))
}
(Matrix { source: a, rows: x }, Matrix { source: b, rows: y }) => {
a == b
&& x.len() == y.len()
&& x.iter().zip(y).all(|(a, b)| {
a.len() == b.len() && a.iter().zip(b).all(|(x, y)| x.to_bits() == y.to_bits())
})
}
_ => false,
}
}
fn value_equal(a: &Value, b: &Value) -> bool {
match (a, b) {
(Value::Num(x), Value::Num(y)) => x.to_bits() == y.to_bits(),
(Value::Formula(x), Value::Formula(y)) => formula_equal(x, y),
_ => a == b,
}
}
pub(super) struct Cursor<'a> {
pub(super) fun: &'a FnChunk,
pub(super) position: usize,
pub(super) end: usize,
}
impl Cursor<'_> {
pub(super) fn op(&mut self, expected: Op) -> Result<(), String> {
if self.position >= self.end || self.fun.code.get(self.position) != Some(&expected) {
return Err(format!(
"source expression opcode differs at {}: expected {expected:?}",
self.position
));
}
self.position += 1;
Ok(())
}
pub(super) fn literal(&mut self, value: &Value) -> Result<(), String> {
match value {
Value::None => self.op(Op::None_),
Value::Bool(true) => self.op(Op::True),
Value::Bool(false) => self.op(Op::False),
_ => {
let Some(Op::Constant(index)) = self.fun.code.get(self.position) else {
return Err("source literal is not a constant instruction".into());
};
if !self
.fun
.constants
.get(*index)
.is_some_and(|actual| value_equal(actual, value))
{
return Err("source literal value differs".into());
}
self.op(Op::Constant(*index))
}
}
}
fn read(&mut self, name: &str) -> Result<(), String> {
let slot = self
.fun
.locals
.iter()
.position(|local| local == name)
.ok_or("source variable has no slot")?;
self.op(Op::GetLocal(slot))
}
pub(super) fn child(&mut self, id: usize) -> Result<(), String> {
let child = &self.fun.expressions[id];
if child.start != self.position {
return Err("source operand order/range differs".into());
}
self.position = child.end;
Ok(())
}
}
pub fn verify(fun: &FnChunk) -> Result<usize, String> {
let mut boundaries = 0;
for (id, node) in fun.expressions.iter().enumerate() {
if node.start >= node.end || node.end > fun.code.len() {
return Err("invalid source expression interval".into());
}
for child in &node.children {
if *child <= id || *child >= fun.expressions.len() {
return Err("invalid source expression child".into());
}
let c = &fun.expressions[*child];
if c.start < node.start || c.end > node.end {
return Err("expression child outside parent".into());
}
}
let mut c = Cursor {
fun,
position: node.start,
end: node.end,
};
let expected_children = match &node.rule {
ExpressionRule::Literal(v) => {
c.literal(v)?;
0
}
ExpressionRule::Read(name) => {
c.read(name)?;
0
}
ExpressionRule::Unary(op) => {
if node.children.len() != 1 {
return Err("unary source arity".into());
}
c.child(node.children[0])?;
c.op(match op {
UnaryOp::Not => Op::Not,
UnaryOp::Neg => Op::Negate,
})?;
1
}
ExpressionRule::Binary(op) => {
if node.children.len() != 2 {
return Err("binary source arity".into());
}
let left = &fun.expressions[node.children[0]];
let right = &fun.expressions[node.children[1]];
c.child(node.children[0])?;
match op {
BinaryOp::And => {
c.op(Op::JumpIfFalse(right.end + 5))?;
c.child(node.children[1])?;
for op in [
Op::JumpIfFalse(right.end + 3),
Op::True,
Op::Jump(node.end),
Op::False,
Op::Jump(node.end),
Op::False,
] {
c.op(op)?;
}
}
BinaryOp::Or => {
c.op(Op::JumpIfFalse(right.start))?;
c.op(Op::True)?;
c.op(Op::Jump(node.end))?;
c.child(node.children[1])?;
for op in [
Op::JumpIfFalse(right.end + 3),
Op::True,
Op::Jump(node.end),
Op::False,
] {
c.op(op)?;
}
}
_ => {
let _ = left;
c.child(node.children[1])?;
c.op(match op {
BinaryOp::Add => Op::Add,
BinaryOp::Sub => Op::Sub,
BinaryOp::Mul => Op::Mul,
BinaryOp::Div => Op::Div,
BinaryOp::Eq => Op::Equal,
BinaryOp::Ne => Op::NotEqual,
BinaryOp::Lt => Op::Less,
BinaryOp::Le => Op::Lessequal,
BinaryOp::Gt => Op::Greater,
BinaryOp::Ge => Op::GreaterEqual,
_ => unreachable!(),
})?;
}
}
2
}
ExpressionRule::List(count) => {
for child in &node.children {
c.child(*child)?;
}
c.op(Op::BuildList(*count))?;
*count
}
ExpressionRule::Map(keys) => {
if node.children.len() != keys.len() {
return Err("map source arity".into());
}
for (key, child) in keys.iter().zip(&node.children) {
c.literal(&Value::Text(key.clone()))?;
c.child(*child)?;
}
c.op(Op::BuildMap(keys.len()))?;
keys.len()
}
ExpressionRule::Occluded(payload) => {
if *payload {
if node.children.len() != 1 {
return Err("occluded source arity".into());
}
c.child(node.children[0])?;
1
} else {
c.literal(&Value::None)?;
0
}
}
ExpressionRule::Index {
label,
base_variable,
} => {
if node.children.len() != 1 {
return Err("index source arity".into());
}
c.child(node.children[0])?;
match label {
IndexKey::Lit(key) => c.literal(&Value::Text(key.clone()))?,
IndexKey::Var(name) => {
c.read(name)?;
c.op(Op::Str)?;
}
}
if let Some(name) = base_variable
.as_ref()
.filter(|name| fun.var_contracts.contains_key(*name))
{
let slot = fun
.locals
.iter()
.position(|local| local == name)
.ok_or("index source slot")?;
c.op(Op::ContractFootnoteGet(slot))?;
} else {
c.op(Op::FootnoteGet)?;
}
1
}
ExpressionRule::Interpolation(parts) => {
if parts.is_empty() {
c.literal(&Value::Text(String::new()))?;
}
for (i, part) in parts.iter().enumerate() {
match part {
InterpPart::Lit(value) => c.literal(&Value::Text(value.clone()))?,
InterpPart::Var(name) => c.read(name)?,
InterpPart::Index { base, labels } => {
c.read(base)?;
for label in labels {
c.literal(&Value::Text(label.clone()))?;
c.op(Op::FootnoteGet)?;
}
}
}
c.op(Op::Str)?;
if i > 0 {
c.op(Op::Add)?;
}
}
0
}
ExpressionRule::CallFault(message) => {
let Some(Op::RejectArgumentExpansion(index)) = fun.code.get(c.position) else {
return Err("missing argument preparation fault".into());
};
if fun.constants.get(*index) != Some(&Value::Text(message.clone())) {
return Err("argument preparation message differs".into());
}
c.op(Op::RejectArgumentExpansion(*index))?;
0
}
ExpressionRule::CallBoundary(_) => {
boundaries += 1;
continue;
}
};
if node.children.len() != expected_children || c.position != node.end {
return Err("source expression coverage differs".into());
}
}
Ok(boundaries)
}
/// Local evidence preserves machine values, not the narrower foreign copy domain.
pub(super) fn interpolation_evidence(part: &InterpPart) -> serde_json::Value {
use serde_json::json;
match part {
InterpPart::Lit(value) => json!({"kind":"text","value":value}),
InterpPart::Var(name) => json!({"kind":"read","name":name}),
InterpPart::Index { base, labels } => json!({"kind":"index","base":base,"labels":labels}),
}
}
pub(super) fn index_evidence(key: &IndexKey) -> serde_json::Value {
use serde_json::json;
match key {
IndexKey::Lit(value) => json!({"kind":"literal","value":value}),
IndexKey::Var(name) => json!({"kind":"read","name":name}),
}
}
pub(super) fn formula_evidence(value: &crate::formula::Expr) -> serde_json::Value {
use crate::formula::Expr::*;
use serde_json::json;
match value {
Num(n) => json!({"kind":"number","bits":n.to_bits()}),
Var(name) => json!({"kind":"variable","name":name}),
UnaryNeg(value) => json!({"kind":"negative","value":formula_evidence(value)}),
Bin { op, left, right } => {
json!({"kind":"binary","op":format!("{op:?}"),"left":formula_evidence(left),"right":formula_evidence(right)})
}
Call { name, args } => {
json!({"kind":"call","name":name,"args":args.iter().map(formula_evidence).collect::<Vec<_>>()})
}
Matrix { source, rows } => {
json!({"kind":"matrix","source":source,"rows":rows.iter().map(|row|row.iter().map(|n|n.to_bits()).collect::<Vec<_>>()).collect::<Vec<_>>()})
}
}
}
pub fn value_evidence(value: &Value) -> serde_json::Value {
use serde_json::json;
match value {
Value::None => json!({"tag":"none"}),
Value::Bool(v) => json!({"tag":"bool","value":v}),
Value::Int(v) => json!({"tag":"int","value":v}),
Value::Num(v) => json!({"tag":"num","bits":v.to_bits()}),
Value::Text(v) => json!({"tag":"text","value":v}),
Value::Code(v) => json!({"tag":"code","lang":v.lang,"source":v.source}),
Value::Formula(v) => json!({"tag":"formula","value":formula_evidence(v)}),
_ => json!({"tag":"nonliteral","value":format!("{value:?}")}),
}
}
impl ExpressionRule {
pub fn evidence(&self) -> serde_json::Value {
use serde_json::json;
match self {
Self::Literal(v) => json!({"kind":"literal","value":value_evidence(v)}),
Self::Read(name) => json!({"kind":"read","name":name}),
Self::Unary(op) => json!({"kind":"unary","op":format!("{op:?}")}),
Self::Binary(op) => json!({"kind":"binary","op":format!("{op:?}")}),
Self::CallBoundary(names) => json!({"kind":"call_boundary","names":names}),
Self::CallFault(message) => json!({"kind":"call_expansion_fault","message":message}),
Self::Interpolation(parts) => {
json!({"kind":"interpolation_boundary","parts":parts.iter().map(interpolation_evidence).collect::<Vec<_>>()})
}
Self::List(n) => json!({"kind":"list","arity":n}),
Self::Map(keys) => json!({"kind":"map","keys":keys}),
Self::Index {
label,
base_variable,
} => {
json!({"kind":"index_boundary","label":index_evidence(label),"base_variable":base_variable})
}
Self::Occluded(payload) => json!({"kind":"occluded","payload":payload}),
}
}
}
+79 -14
View File
@@ -2,22 +2,35 @@
mod compile;
mod vm;
mod verify;
mod expression;
mod statement;
mod call;
mod source;
mod names;
pub use source::SourceFunction;
pub use call::{CallEmission, CallStage, CallDispatch, PathFallback};
pub use statement::{StatementRule,StatementEmission};
pub use expression::{ExpressionRule,ExpressionEmission,value_evidence as expression_value_evidence};
pub use verify::StackCertificate;
pub use compile::compile_module;
pub use compile::{compile_for_call, compile_module};
pub use vm::Vm;
use std::collections::HashMap;
use crate::diagnostics::Span;
use crate::value::Value;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Op {
Constant(u16),
Constant(usize),
True,
False,
None_,
Pop,
GetLocal(u8),
SetLocal(u8),
GetLocal(usize),
SetLocal(usize),
Add,
Sub,
Mul,
@@ -31,14 +44,14 @@ pub enum Op {
Less,
Lessequal,
/// Absolute jump to instruction index.
Jump(u16),
JumpIfFalse(u16),
Jump(usize),
JumpIfFalse(usize),
Print,
/// Pop prompt text, print it (no newline), read one stdin line → push Text.
Input,
BuildList(u16),
BuildList(usize),
/// Stack (bottom→top): k0, v0, k1, v1, … (n pairs); push Map.
BuildMap(u16),
BuildMap(usize),
/// Pop value → push `len` (text/list).
Len,
/// Pop value → push display text.
@@ -59,25 +72,73 @@ pub enum Op {
/// Pop footnote label text, then base → push indexed value (1-based list / map key).
FootnoteGet,
/// Call function by index; argc values are on the stack (param order).
Call(u16, u8),
Call(usize, usize),
/// Evaluated actuals in authored order; layout preserves optional names.
CallArgs(usize, usize),
/// Runtime binding fault after all explicit actual effects.
RejectDuplicateArgument(usize),
/// Modifier/argument preparation failed before any actual is evaluated.
RejectArgumentExpansion(usize),
/// Skip a callee default when the parameter was explicitly supplied.
JumpIfProvided(usize, usize),
/// Entry/raw-call parameters without a value retain language None after defaults.
FillMissingParameters,
ResetLoop(usize),
CheckLoop(usize),
CheckArguments,
RestoreReceiver,
CheckFrameArguments,
CheckReturn,
/// Check the stack top against the named local's field contract.
CheckBinding(usize),
/// Checked footnote access; operand identifies the contracted base local.
ContractFootnoteGet(usize),
/// Pop function name text → call zero-arg top-level or lexically visible function.
CallFn,
/// After a constructor Call: tag top-of-stack map with `_type` = constants[idx].
TagInstance(u16),
/// After a constructor call: stamp name and module of functions[idx].
TagInstance(usize),
/// Method call: stack has argc args (param order) then receiver on top.
/// constants[name_idx] is method name text; resolve via receiver `_type`.
MethodCall(u16, u8),
MethodCall(usize, usize),
/// Resolve the receiver first, then bind against that method's signature.
MethodCallArgs(usize, usize),
/// Plugin call: stack has argc args in registered param order; name in constants.
PluginCall(u16, u8),
PluginCall(usize, usize),
/// Host primitive: pop `argc` args (param order), push result.
HostCall(u16, u8),
HostCall(u16, usize),
/// Native dispatch with authored argument labels; binding happens at runtime.
BuiltinCallArgs(usize, usize),
HostCallArgs(u16, usize),
PluginCallArgs(usize, usize),
/// Two-component paths inspect actual receiver presence before static fallback.
PathCallArgs(usize, usize),
RejectCallResolution(usize),
Return,
}
#[derive(Debug, Clone)]
pub struct FnChunk {
/// Captured directly from the loaded declaration before emission.
pub source_ast: SourceFunction,
pub module_scope: u32,
pub module_path: Vec<String>,
pub source: crate::diagnostics::SourceIdentity,
pub declaration_span: Span,
pub name: String,
pub params: Vec<String>,
pub contract: Option<crate::contract::Contract>,
pub var_contracts: HashMap<String, crate::contract::Contract>,
pub receiver_aliases: bool,
pub defaults: Vec<bool>,
pub default_expressions: Vec<Option<usize>>,
pub call_layouts: Vec<Vec<Option<String>>>,
pub calls: Vec<CallEmission>,
/// Source names are reserved before synthetic slots are allocated.
pub source_local_count: usize,
pub expressions: Vec<ExpressionEmission>,
pub statements: Vec<StatementEmission>,
pub statement_roots: Vec<usize>,
pub body_start: usize,
pub code: Vec<Op>,
/// Parallel to `code`: source location for diagnostics / debug.
pub spans: Vec<Span>,
@@ -90,6 +151,8 @@ pub struct FnChunk {
pub struct Program {
pub functions: Vec<FnChunk>,
pub entry: usize,
/// Applied by run(), not by named or nested calls to the entry function.
pub entry_metadata: Vec<(String, Value)>,
/// Top-level object name → function index (level-1).
pub objects: Vec<(String, usize)>,
/// Parallel to `objects`: base object flat index (`# Child = > Parent`).
@@ -100,6 +163,8 @@ pub struct Program {
pub parents: Vec<Option<usize>>,
/// Parallel to `functions`: child fn indices.
pub children: Vec<Vec<usize>>,
/// Dotted path (`计算器.加法`, `inc.加一`) → function index. Entry module exports plus imports.
pub path_index: HashMap<Vec<String>, usize>,
}
impl Program {
+172
View File
@@ -0,0 +1,172 @@
//! Source-grounded declaration identities and the flat view of name resolution.
use super::Program;
use crate::sema::resolution::{NameView, Target};
use std::collections::{HashMap, HashSet};
pub(crate) struct FlatView<'a>(pub &'a Program);
impl NameView for FlatView<'_> {
type Function = usize;
type Module = Vec<String>;
fn roots(&self, m: &Self::Module) -> Vec<usize> {
self.0
.functions
.iter()
.enumerate()
.filter(|(i, f)| &f.module_path == m && self.0.parents[*i].is_none())
.map(|(i, _)| i)
.collect()
}
fn children(&self, f: usize) -> Vec<usize> {
self.0.children[f].clone()
}
fn parent(&self, _: &Self::Module, f: usize) -> Option<usize> {
self.0.parents[f]
}
fn same(&self, a: usize, b: usize) -> bool {
a == b
}
fn name(&self, f: usize) -> String {
self.0.functions[f].source_ast.name.clone()
}
fn level(&self, f: usize) -> u8 {
self.0.functions[f].source_ast.level
}
fn is_object(&self, f: usize) -> bool {
self.level(f) == 1
}
fn uses_receiver(&self, f: usize) -> bool {
self.0.functions[f].source_ast.receiver_aliases
}
fn import(&self, m: &Self::Module, name: &str) -> Option<Self::Module> {
let owner = self.0.functions.iter().find(|f| &f.module_path == m)?;
if !owner.source_ast.imports.iter().any(|n| n == name) {
return None;
}
let mut path = m.clone();
path.push(name.into());
Some(path)
}
fn used_path(&self, m: &Self::Module, name: &str) -> Option<Vec<String>> {
self.0
.functions
.iter()
.find(|f| &f.module_path == m)?
.source_ast
.uses
.iter()
.find(|(n, _)| n == name)
.map(|(_, p)| p.clone())
}
}
pub(crate) fn resolve(
program: &Program,
caller: usize,
call: &crate::ast::CallExpr,
) -> Target<usize, Vec<String>> {
crate::sema::resolution::resolve(
&FlatView(program),
&program.functions[caller].module_path,
caller,
call,
)
}
pub(crate) fn lookup(program: &Program, caller: usize, name: &str) -> Option<usize> {
crate::sema::resolution::lookup(
&FlatView(program),
&program.functions[caller].module_path,
caller,
name,
)
}
pub(crate) fn verify(program: &Program) -> Result<(), String> {
let objects = program
.functions
.iter()
.enumerate()
.filter(|(id, f)| program.parents[*id].is_none() && f.source_ast.level == 1)
.map(|(id, f)| (f.source_ast.name.clone(), id))
.collect::<Vec<_>>();
if program.objects != objects {
return Err("object names differ from source declarations".into());
}
let methods = objects
.iter()
.flat_map(|(_, owner)| {
program.children[*owner].iter().map(|child| {
(
*owner,
program.functions[*child].source_ast.name.clone(),
*child,
)
})
})
.collect::<Vec<_>>();
if program.methods != methods {
return Err("method names differ from source declarations".into());
}
let mut identities = HashSet::new();
let mut scopes = HashMap::new();
let mut namespaces = HashMap::new();
for f in &program.functions {
let source = &f.source_ast;
if source.declaration_path.is_empty()
|| !identities.insert((f.module_path.clone(), source.declaration_path.clone()))
{
return Err("duplicate or missing source declaration identity".into());
}
if scopes
.insert(f.module_scope, f.module_path.clone())
.is_some_and(|p| p != f.module_path)
|| namespaces
.insert(f.module_path.clone(), f.module_scope)
.is_some_and(|s| s != f.module_scope)
|| (f.module_scope == 0) != f.module_path.is_empty()
{
return Err("source module scope mapping differs".into());
}
let owner = program
.functions
.iter()
.find(|g| g.module_path == f.module_path)
.unwrap();
if source.module_root_count != owner.source_ast.module_root_count
|| source.imports != owner.source_ast.imports
|| source.uses != owner.source_ast.uses
{
return Err("inconsistent source module header".into());
}
}
for (id, f) in program.functions.iter().enumerate() {
let source = &f.source_ast;
let parent_path = &source.declaration_path[..source.declaration_path.len() - 1];
match program.parents[id] {
Some(parent)
if program.functions[parent].source_ast.declaration_path == parent_path =>
{
()
}
None if parent_path.is_empty() => (),
_ => return Err("flat parent differs from source declaration".into()),
}
if program.children[id].len() != source.child_count {
return Err("source child inventory differs".into());
}
for (ordinal, child) in program.children[id].iter().enumerate() {
let mut expected = source.declaration_path.clone();
expected.push(ordinal);
if program.functions[*child].source_ast.declaration_path != expected {
return Err("source child order differs".into());
}
}
let roots = FlatView(program).roots(&f.module_path);
if roots.len() != source.module_root_count
|| roots
.iter()
.enumerate()
.any(|(i, r)| program.functions[*r].source_ast.declaration_path != [i])
{
return Err("source root inventory/order differs".into());
}
}
Ok(())
}
+366
View File
@@ -0,0 +1,366 @@
//! Structural correspondence from a separately captured loaded AST to emission
//! witnesses. Parsing and capture of the source snapshot remain trusted inputs.
use super::{ExpressionRule, FnChunk};
use crate::ast::{Arg, CallExpr, Expr, Function, Literal, Param, Stmt};
use serde_json::{json, Value as Json};
#[derive(Debug, Clone)]
pub struct SourceFunction {
pub declaration_path: Vec<usize>,
pub level: u8,
pub child_count: usize,
pub module_root_count: usize,
pub imports: Vec<String>,
pub uses: Vec<(String, Vec<String>)>,
pub name: String,
pub line: u32,
pub params: Vec<Param>,
pub body: Vec<Stmt>,
pub receiver_aliases: bool,
pub has_contract: bool,
pub binding_contracts: Vec<String>,
pub contract: Option<crate::contract::Contract>,
pub var_contracts: std::collections::HashMap<String, crate::contract::Contract>,
}
impl SourceFunction {
pub fn from_function(fun: &Function) -> Self {
let mut bindings = fun.var_contracts.keys().cloned().collect::<Vec<_>>();
bindings.sort();
Self {
declaration_path: Vec::new(),
level: fun.level,
child_count: fun.children.len(),
module_root_count: 0,
imports: Vec::new(),
uses: Vec::new(),
name: fun.name.clone(),
line: fun.span.line,
params: fun.params.clone(),
body: fun.body.clone(),
receiver_aliases: crate::ast::function_uses_receiver(fun),
has_contract: fun.contract.is_some(),
binding_contracts: bindings,
contract: fun.contract.clone(),
var_contracts: fun.var_contracts.clone(),
}
}
pub fn from_declaration(module: &crate::ast::Module, fun: &Function) -> Self {
fn locate(nodes: &[Function], target: &Function, path: &mut Vec<usize>) -> bool {
for (index, node) in nodes.iter().enumerate() {
path.push(index);
if std::ptr::eq(node, target) || locate(&node.children, target, path) { return true; }
path.pop();
}
false
}
let mut result = Self::from_function(fun);
assert!(locate(&module.functions, fun, &mut result.declaration_path), "source declaration belongs to its module");
result.module_root_count = module.functions.len();
result.imports = module.import_modules.keys().cloned().collect();
result.imports.sort();
result.uses = module.uses.iter().map(|u| (u.bind.clone(), u.path.clone())).collect();
result
}
pub fn evidence(&self) -> Result<Json, String> {
Ok(
json!({"declaration_path":self.declaration_path,"level":self.level,"child_count":self.child_count,"module_root_count":self.module_root_count,"imports":self.imports,"uses":self.uses,
"name":self.name,"line":self.line,"params":self.params.iter().map(|p|Ok(json!({"name":p.name,"default":p.default.as_ref().map(expression_tree).transpose()?}))).collect::<Result<Vec<_>,String>>()?,
"body":self.body.iter().map(statement_tree).collect::<Result<Vec<_>,_>>()?,"receiver_aliases":self.receiver_aliases,"has_contract":self.has_contract,"binding_contracts":self.binding_contracts,"contracts":Self::contract_metadata(self.contract.as_ref(),&self.var_contracts)}),
)
}
pub fn contract_metadata(
contract: Option<&crate::contract::Contract>,
bindings: &std::collections::HashMap<String, crate::contract::Contract>,
) -> Json {
fn ty(ty: &crate::contract::Ty) -> Json {
match ty {
crate::contract::Ty::Object(name) => json!({"kind":"object","name":name}),
other => json!({"kind":other.name()}),
}
}
fn encode(c: &crate::contract::Contract) -> Json {
json!({"params":c.params.iter().map(|p|json!({"name":p.name,"type":ty(&p.ty),"description":p.desc})).collect::<Vec<_>>(),
"returns":c.returns.as_ref().map(|r|json!({"type":ty(&r.ty),"description":r.desc})),
"fields":c.fields.iter().map(|f|json!({"name":f.name,"type":ty(&f.ty),"nullable":f.nullable,"description":f.desc})).collect::<Vec<_>>(),
"start_line":c.start_line,"end_line":c.end_line,"quote":c.quote})
}
let bindings = bindings
.iter()
.map(|(name, c)| (name, encode(c)))
.collect::<std::collections::BTreeMap<_, _>>();
json!({"function":contract.map(encode),"bindings":bindings})
}
}
fn normalized(call: &CallExpr) -> Result<CallExpr, String> {
call.for_evaluation()
}
fn call_shape(call: &CallExpr) -> Json {
json!({"callee":call.callee,"receiver":call.receiver,"path":call.path,"arguments":call.args.iter().map(|a|match a {Arg::Positional(_)=>None,Arg::Named{name,..}=>Some(name)}).collect::<Vec<_>>()})
}
fn arg_expr(arg: &Arg) -> &Expr {
match arg {
Arg::Positional(e) | Arg::Named { value: e, .. } => e,
}
}
// Deliberately match the loaded AST directly, rather than invoking the emitter's
// ExpressionRule::from_source or StatementRule::from_source recorder.
fn rule(expr: &Expr) -> Json {
use crate::value::Value;
match expr {
Expr::Literal(lit) => {
let value = match lit {
Literal::None => json!({"tag":"none"}),
Literal::Bool(v) => json!({"tag":"bool","value":v}),
Literal::Int(v) => json!({"tag":"int","value":v}),
Literal::Num(v) => json!({"tag":"num","bits":v.to_bits()}),
Literal::Text(v) => json!({"tag":"text","value":v}),
};
json!({"kind":"literal","value":value})
}
Expr::Var(name) => json!({"kind":"read","name":name}),
Expr::Unary { op, .. } => json!({"kind":"unary","op":format!("{op:?}")}),
Expr::Binary { op, .. } => json!({"kind":"binary","op":format!("{op:?}")}),
Expr::Call(call) => {
if let Err(message) = normalized(call) {
return json!({"kind":"call_expansion_fault","message":message});
}
json!({"kind":"call_boundary","names":call.receiver.iter().cloned().chain(call.path.iter().filter_map(|p|p.first().cloned())).collect::<Vec<_>>()})
}
Expr::List(items) => json!({"kind":"list","arity":items.len()}),
Expr::Map(items) => {
json!({"kind":"map","keys":items.iter().map(|(name,_)|name).collect::<Vec<_>>()})
}
Expr::Interp(parts) => {
json!({"kind":"interpolation_boundary","parts":parts.iter().map(super::expression::interpolation_evidence).collect::<Vec<_>>()})
}
Expr::Index { base, label } => {
json!({"kind":"index_boundary","label":super::expression::index_evidence(label),"base_variable":if let Expr::Var(name)=base.as_ref(){Some(name)}else{None}})
}
Expr::Occluded { payload, .. } => json!({"kind":"occluded","payload":payload.is_some()}),
Expr::Formula(value) => {
json!({"kind":"literal","value":super::expression::value_evidence(&Value::Formula(value.clone()))})
}
Expr::Code(value) => {
json!({"kind":"literal","value":{"tag":"code","lang":value.lang,"source":value.source}})
}
}
}
fn children(expr: &Expr) -> Vec<&Expr> {
match expr {
Expr::Unary { expr, .. } => vec![expr],
Expr::Binary { left, right, .. } => vec![left, right],
Expr::List(items) => items.iter().collect(),
Expr::Map(items) => items.iter().map(|(_, e)| e).collect(),
Expr::Index { base, .. } => vec![base],
Expr::Occluded { payload, .. } => payload.iter().map(|e| e.as_ref()).collect(),
_ => Vec::new(),
}
}
fn expression_tree(expr: &Expr) -> Result<Json, String> {
if matches!(expr,Expr::Call(call) if normalized(call).is_err()) {
return Ok(json!({"rule":rule(expr),"children":[],"call":null}));
}
let (children, call) = if let Expr::Call(call) = expr {
let expanded = normalized(call)?;
(
expanded
.args
.iter()
.map(|a| expression_tree(arg_expr(a)))
.collect::<Result<Vec<_>, _>>()?,
Some(call_shape(&expanded)),
)
} else {
(
children(expr)
.iter()
.map(|e| expression_tree(e))
.collect::<Result<Vec<_>, _>>()?,
None,
)
};
Ok(json!({"rule":rule(expr),"children":children,"call":call}))
}
fn statement_rule(stmt: &Stmt) -> Json {
match stmt {
Stmt::Assign { name, .. } => json!({"kind":"assign","name":name}),
Stmt::Return { .. } => json!({"kind":"return"}),
Stmt::Call { .. } => json!({"kind":"call"}),
Stmt::Expr { .. } => json!({"kind":"expression"}),
Stmt::Branch { arms, .. } => {
json!({"kind":"branch","arms":arms.iter().map(|a|(a.condition.is_some(),a.body.len())).collect::<Vec<_>>()})
}
Stmt::While { body, .. } => json!({"kind":"while","body":body.len()}),
Stmt::ForEach {
item,
collection,
body,
..
} => json!({"kind":"foreach","item":item,"collection":collection,"body":body.len()}),
}
}
fn statement_children(stmt: &Stmt) -> Vec<&Stmt> {
match stmt {
Stmt::Branch { arms, .. } => arms.iter().flat_map(|a| a.body.iter()).collect(),
Stmt::While { body, .. } | Stmt::ForEach { body, .. } => body.iter().collect(),
_ => Vec::new(),
}
}
fn statement_expressions(stmt: &Stmt) -> Vec<Expr> {
match stmt {
Stmt::Assign { value, .. } | Stmt::Return { value, .. } | Stmt::Expr { value, .. } => {
vec![value.clone()]
}
Stmt::Call { call, .. } => vec![Expr::Call(call.clone())],
Stmt::Branch { arms, .. } => arms.iter().filter_map(|a| a.condition.clone()).collect(),
Stmt::While { condition, .. } => vec![condition.clone()],
Stmt::ForEach { collection, .. } => vec![Expr::Var(collection.clone())],
}
}
fn statement_tree(stmt: &Stmt) -> Result<Json, String> {
Ok(
json!({"rule":statement_rule(stmt),"expressions":statement_expressions(stmt).iter().map(expression_tree).collect::<Result<Vec<_>,_>>()?,"statements":statement_children(stmt).iter().map(|s|statement_tree(s)).collect::<Result<Vec<_>,_>>()?}),
)
}
struct Walk<'a> {
fun: &'a FnChunk,
expressions: Vec<bool>,
statements: Vec<bool>,
}
impl Walk<'_> {
fn expression(&mut self, source: &Expr, id: usize) -> Result<(), String> {
let node = self
.fun
.expressions
.get(id)
.ok_or("AST expression reference")?;
if self.expressions[id] {
return Err("AST expression has multiple owners".into());
}
self.expressions[id] = true;
if node.rule.evidence() != rule(source) {
return Err("loaded AST expression rule differs from witness".into());
}
if let Expr::Call(call) = source {
let call = match normalized(call) {
Ok(call) => call,
Err(_) => {
if !node.children.is_empty()
|| self.fun.calls.iter().any(|c| c.expression == id)
{
return Err("AST preparation fault evaluated actuals".into());
}
return Ok(());
}
};
let record = self
.fun
.calls
.iter()
.find(|c| c.expression == id)
.ok_or("AST call obligation missing")?;
if call_shape(&call) != call_shape(&record.source)
|| node.children.len() < call.args.len()
{
return Err("loaded AST call differs from witness".into());
}
for (arg, child) in call.args.iter().zip(&node.children) {
self.expression(arg_expr(arg), *child)?;
}
for &generated in &node.children[call.args.len()..] {
let expr = self
.fun
.expressions
.get(generated)
.ok_or("generated reload reference")?;
if self.expressions[generated]
|| !expr.children.is_empty()
|| !matches!(expr.rule, ExpressionRule::Read(_))
{
return Err("generated reload is shared or not a read".into());
}
self.expressions[generated] = true;
}
} else {
let expected = children(source);
if expected.len() != node.children.len() {
return Err("loaded AST expression child inventory differs".into());
}
for (source, child) in expected.iter().zip(&node.children) {
self.expression(source, *child)?;
}
}
Ok(())
}
fn statement(&mut self, source: &Stmt, id: usize) -> Result<(), String> {
let node = self
.fun
.statements
.get(id)
.ok_or("AST statement reference")?;
if self.statements[id] {
return Err("AST statement has multiple owners".into());
}
self.statements[id] = true;
if node.rule.evidence() != statement_rule(source) {
return Err("loaded AST statement rule differs from witness".into());
}
let exprs = statement_expressions(source);
let stmts = statement_children(source);
if node.expressions.len() != exprs.len() || node.statements.len() != stmts.len() {
return Err("loaded AST statement child inventory differs".into());
}
for (expr, id) in exprs.iter().zip(&node.expressions) {
self.expression(expr, *id)?;
}
for (stmt, id) in stmts.iter().zip(&node.statements) {
self.statement(stmt, *id)?;
}
Ok(())
}
}
pub fn verify(fun: &FnChunk) -> Result<(), String> {
let source = &fun.source_ast;
if source.contract != fun.contract || source.var_contracts != fun.var_contracts {
return Err("loaded AST contract metadata differs".into());
}
if source.name != fun.name
|| source.line != fun.declaration_span.line
|| source.params.iter().map(|p| &p.name).ne(fun.params.iter())
|| source.receiver_aliases != fun.receiver_aliases
|| source.has_contract != fun.contract.is_some()
|| source.binding_contracts.len() != fun.var_contracts.len()
|| source
.binding_contracts
.iter()
.any(|name| !fun.var_contracts.contains_key(name))
{
return Err("loaded AST function metadata differs".into());
}
let mut walk = Walk {
fun,
expressions: vec![false; fun.expressions.len()],
statements: vec![false; fun.statements.len()],
};
if source.params.len() != fun.default_expressions.len()
|| source.body.len() != fun.statement_roots.len()
{
return Err("loaded AST function root inventory differs".into());
}
for (param, id) in source.params.iter().zip(&fun.default_expressions) {
match (&param.default, id) {
(Some(expr), Some(id)) => walk.expression(expr, *id)?,
(None, None) => {}
_ => return Err("loaded AST default presence differs".into()),
}
}
for (stmt, id) in source.body.iter().zip(&fun.statement_roots) {
walk.statement(stmt, *id)?;
}
if walk.expressions.iter().any(|seen| !seen) || walk.statements.iter().any(|seen| !seen) {
return Err("emission witness has no loaded AST owner".into());
}
Ok(())
}
+365
View File
@@ -0,0 +1,365 @@
//! Source-directed statement and control-flow templates. These checks validate
//! emitted templates; they do not prove the parser or source-witness exporter.
use super::expression::Cursor;
use super::{FnChunk, Op};
use crate::ast::Stmt;
#[derive(Debug, Clone)]
pub enum StatementRule {
Assign(String),
Return,
Call,
Expression,
Branch(Vec<(bool, usize)>),
While(usize),
ForEach {
item: String,
collection: String,
body: usize,
},
}
impl StatementRule {
pub fn from_source(stmt: &Stmt) -> Self {
match stmt {
Stmt::Assign { name, .. } => Self::Assign(name.clone()),
Stmt::Return { .. } => Self::Return,
Stmt::Call { .. } => Self::Call,
Stmt::Expr { .. } => Self::Expression,
Stmt::Branch { arms, .. } => Self::Branch(
arms.iter()
.map(|a| (a.condition.is_some(), a.body.len()))
.collect(),
),
Stmt::While { body, .. } => Self::While(body.len()),
Stmt::ForEach {
item,
collection,
body,
..
} => Self::ForEach {
item: item.clone(),
collection: collection.clone(),
body: body.len(),
},
}
}
}
#[derive(Debug, Clone)]
pub struct StatementEmission {
pub rule: StatementRule,
pub start: usize,
pub end: usize,
pub statements: Vec<usize>,
pub expressions: Vec<usize>,
pub auxiliary: Vec<usize>,
}
fn statement(c: &mut Cursor<'_>, id: usize) -> Result<(), String> {
let child = c.fun.statements.get(id).ok_or("missing statement child")?;
if child.start != c.position {
return Err("source statement order/range differs".into());
}
c.position = child.end;
Ok(())
}
pub fn verify(fun: &FnChunk) -> Result<(), String> {
if fun.defaults.len() != fun.params.len() || fun.default_expressions.len() != fun.params.len() {
return Err("default expression inventory".into());
}
let mut prefix = 0;
for (parameter, default) in fun.default_expressions.iter().enumerate() {
if fun.defaults[parameter] != default.is_some() {
return Err("default source presence differs".into());
}
if let Some(id) = default {
let expr = fun
.expressions
.get(*id)
.ok_or("default source expression missing")?;
if expr.start != prefix + 1
|| fun.code.get(prefix) != Some(&Op::JumpIfProvided(parameter, expr.end + 2))
|| fun.code.get(expr.end..expr.end + 2)
!= Some([Op::SetLocal(parameter), Op::Pop].as_slice())
{
return Err("source default prologue differs".into());
}
prefix = expr.end + 2;
}
}
if fun.code.get(prefix) != Some(&Op::FillMissingParameters) {
return Err("missing parameter-fill boundary".into());
}
prefix += 1;
if fun.contract.is_some() {
if fun.code.get(prefix) != Some(&Op::CheckArguments) {
return Err("missing input contract boundary".into());
}
prefix += 1;
}
if fun.receiver_aliases {
if fun.code.get(prefix)!=Some(&Op::RestoreReceiver){return Err("missing receiver restoration".into());}prefix+=1;
if fun.contract.is_some(){if fun.code.get(prefix)!=Some(&Op::CheckFrameArguments){return Err("missing frame contract boundary".into());}prefix+=1;}
}
if prefix != fun.body_start {
return Err("function prologue/body boundary differs".into());
}
let mut protected = fun
.params
.iter()
.cloned()
.collect::<std::collections::HashSet<_>>();
protected.extend(["self".to_string(), "自".to_string()]);
for expression in &fun.expressions {
use super::ExpressionRule;
match &expression.rule {
ExpressionRule::Read(name) => {
protected.insert(name.clone());
}
ExpressionRule::CallBoundary(names) => protected.extend(names.iter().cloned()),
ExpressionRule::Interpolation(parts) => {
for part in parts {
match part {
crate::ast::InterpPart::Var(name)
| crate::ast::InterpPart::Index { base: name, .. } => {
protected.insert(name.clone());
}
_ => {}
}
}
}
ExpressionRule::Index {
label,
base_variable,
} => {
if let Some(name) = base_variable {
protected.insert(name.clone());
}
if let crate::ast::IndexKey::Var(name) = label {
protected.insert(name.clone());
}
}
_ => {}
}
}
for statement in &fun.statements {
match &statement.rule {
StatementRule::Assign(name) => {
protected.insert(name.clone());
}
StatementRule::ForEach {
item, collection, ..
} => {
protected.insert(item.clone());
protected.insert(collection.clone());
}
_ => {}
}
}
let mut auxiliary_slots = std::collections::HashSet::new();
let mut parent_counts = vec![0usize; fun.statements.len()];
for (id, node) in fun.statements.iter().enumerate() {
if node.start > node.end || node.end > fun.code.len() {
return Err("invalid statement interval".into());
}
for &child in &node.statements {
if child <= id || child >= fun.statements.len() {
return Err("invalid statement child".into());
}
parent_counts[child] += 1;
let child = &fun.statements[child];
if child.start < node.start || child.end > node.end {
return Err("statement child outside parent".into());
}
}
for &expr in &node.expressions {
let expr = fun
.expressions
.get(expr)
.ok_or("invalid statement expression")?;
if expr.start < node.start || expr.end > node.end {
return Err("expression outside statement".into());
}
}
let mut c = Cursor {
fun,
position: node.start,
end: node.end,
};
match &node.rule {
StatementRule::Assign(_)
| StatementRule::Return
| StatementRule::Call
| StatementRule::Expression => {
if !node.statements.is_empty() || node.expressions.len() != 1 {
return Err("statement source arity".into());
}
c.child(node.expressions[0])?;
match &node.rule {
StatementRule::Assign(name) => {
let slot = fun
.locals
.iter()
.position(|n| n == name)
.ok_or("assignment slot")?;
if fun.var_contracts.contains_key(name) {
c.op(Op::CheckBinding(slot))?;
}
c.op(Op::SetLocal(slot))?;
c.op(Op::Pop)?;
}
StatementRule::Return => {
if fun.contract.is_some() {
c.op(Op::CheckReturn)?;
}
c.op(Op::Return)?;
}
StatementRule::Call | StatementRule::Expression => c.op(Op::Pop)?,
_ => unreachable!(),
}
}
StatementRule::While(count) => {
if node.expressions.len() != 1 || node.statements.len() != *count {
return Err("while source arity".into());
}
c.op(Op::ResetLoop(id))?;
let head = c.position;
c.child(node.expressions[0])?;
c.op(Op::JumpIfFalse(node.end))?;
c.op(Op::CheckLoop(id))?;
for &child in &node.statements {
statement(&mut c, child)?;
}
c.op(Op::Jump(head))?;
}
StatementRule::Branch(arms) => {
if node.expressions.len() != arms.iter().filter(|(condition, _)| *condition).count()
|| node.statements.len() != arms.iter().map(|(_, n)| n).sum::<usize>()
{
return Err("branch source arity".into());
}
let (mut expr, mut stmt) = (0, 0);
for (index, (condition, count)) in arms.iter().enumerate() {
let guard = if *condition {
c.child(node.expressions[expr])?;
expr += 1;
let pc = c.position;
c.position += 1;
Some(pc)
} else {
None
};
for _ in 0..*count {
statement(&mut c, node.statements[stmt])?;
stmt += 1;
}
if index + 1 < arms.len() {
c.op(Op::Jump(node.end))?;
}
if let Some(guard) = guard {
if fun.code.get(guard) != Some(&Op::JumpIfFalse(c.position)) {
return Err("source branch false target differs".into());
}
}
}
}
StatementRule::ForEach {
item,
collection,
body,
} => {
if node.expressions.len() != 1
|| node.statements.len() != *body
|| node.auxiliary.len() != 2
{
return Err("foreach source arity".into());
}
let (coll, index) = (node.auxiliary[0], node.auxiliary[1]);
if coll == index || coll >= fun.locals.len() || index >= fun.locals.len() {
return Err("foreach auxiliary slots".into());
}
for slot in [coll, index] {
if protected.contains(&fun.locals[slot]) || !auxiliary_slots.insert(slot) {
return Err("loop temporary aliases source or another loop".into());
}
}
// The initial expression must read the authored collection.
if !matches!(&fun.expressions[node.expressions[0]].rule,super::ExpressionRule::Read(name) if name==collection)
{
return Err("foreach source collection differs".into());
}
c.child(node.expressions[0])?;
c.op(Op::IterItems)?;
c.op(Op::SetLocal(coll))?;
c.op(Op::Pop)?;
c.literal(&crate::value::Value::Int(0))?;
c.op(Op::SetLocal(index))?;
c.op(Op::Pop)?;
let head = c.position;
for op in [
Op::GetLocal(index),
Op::GetLocal(coll),
Op::Len,
Op::Less,
Op::JumpIfFalse(node.end),
Op::GetLocal(coll),
Op::GetLocal(index),
Op::GetIndex,
] {
c.op(op)?;
}
let item = fun
.locals
.iter()
.position(|n| n == item)
.ok_or("foreach item slot")?;
c.op(Op::SetLocal(item))?;
c.op(Op::Pop)?;
for &child in &node.statements {
statement(&mut c, child)?;
}
c.op(Op::GetLocal(index))?;
c.literal(&crate::value::Value::Int(1))?;
for op in [Op::Add, Op::SetLocal(index), Op::Pop, Op::Jump(head)] {
c.op(op)?;
}
}
}
if c.position != node.end {
return Err("statement instruction coverage differs".into());
}
}
let mut cursor = fun.body_start;
for &root in &fun.statement_roots {
let node = fun.statements.get(root).ok_or("statement root missing")?;
if parent_counts[root] != 0 || node.start != cursor {
return Err("source body root order differs".into());
}
parent_counts[root] += 1;
cursor = node.end;
}
if parent_counts.iter().any(|n| *n != 1) {
return Err("statement forest coverage differs".into());
}
if fun.code.get(cursor..) != Some([Op::None_, Op::Return].as_slice()) {
return Err("implicit function return differs".into());
}
Ok(())
}
impl StatementRule {
pub fn evidence(&self) -> serde_json::Value {
use serde_json::json;
match self {
Self::Assign(name) => json!({"kind":"assign","name":name}),
Self::Return => json!({"kind":"return"}),
Self::Call => json!({"kind":"call"}),
Self::Expression => json!({"kind":"expression"}),
Self::Branch(arms) => json!({"kind":"branch","arms":arms}),
Self::While(body) => json!({"kind":"while","body":body}),
Self::ForEach {
item,
collection,
body,
} => json!({"kind":"foreach","item":item,"collection":collection,"body":body}),
}
}
}
+331
View File
@@ -0,0 +1,331 @@
//! Checks every emitted instruction and a control-flow stack-height certificate.
//! This is structural validation, not source/bytecode semantic equivalence.
use super::{FnChunk, Op, Program};
use crate::value::Value;
use std::collections::{HashSet, VecDeque};
#[derive(Debug, Clone)]
pub struct StackCertificate {
pub heights: Vec<Option<usize>>,
}
impl Program {
pub fn verify(&self) -> Result<Vec<StackCertificate>, String> {
let n = self.functions.len();
if self.entry >= n || self.parents.len() != n || self.children.len() != n {
return Err("invalid program function tables".into());
}
for (i, parent) in self.parents.iter().enumerate() {
if parent.is_some_and(|p| p >= n || p == i) {
return Err("invalid function parent".into());
}
for &child in &self.children[i] {
if child >= n || self.parents[child] != Some(i) {
return Err("inconsistent function child".into());
}
}
}
check_forest(&self.parents)?;
for (child, parent) in self.parents.iter().enumerate() {
if let Some(parent) = parent {
if !self.children[*parent].contains(&child)
|| self.functions[*parent].module_path != self.functions[child].module_path
{
return Err("inconsistent parent owner".into());
}
}
}
super::names::verify(self)?;
if self.objects.len() != self.object_bases.len() {
return Err("invalid object tables".into());
}
for (i, (_, fid)) in self.objects.iter().enumerate() {
if *fid >= n
|| self.object_bases[i]
.is_some_and(|base| base >= n || !self.objects.iter().any(|(_, f)| *f == base))
{
return Err("invalid object reference".into());
}
}
if self.methods.iter().any(|(owner, _, fid)| {
*owner >= n || *fid >= n || !self.objects.iter().any(|(_, f)| f == owner)
}) || self.path_index.values().any(|id| *id >= n)
{
return Err("invalid method/export reference".into());
}
let object_ids = self
.objects
.iter()
.enumerate()
.map(|(i, (_, fid))| (*fid, i))
.collect::<std::collections::HashMap<_, _>>();
if object_ids.len() != self.objects.len() {
return Err("duplicate object reference".into());
}
let bases = self
.object_bases
.iter()
.map(|base| base.map(|fid| object_ids[&fid]))
.collect::<Vec<_>>();
check_forest(&bases)?;
if self
.methods
.iter()
.any(|(owner, _, fid)| self.parents[*fid] != Some(*owner))
{
return Err("inconsistent method owner".into());
}
self.functions
.iter()
.enumerate()
.map(|(i, f)| {
verify_chunk(self, i, f)
.map_err(|e| format!("bytecode verification in {}: {e}", f.name))
})
.collect()
}
}
fn verify_chunk(program: &Program, _fid: usize, fun: &FnChunk) -> Result<StackCertificate, String> {
super::expression::verify(fun)?;
super::statement::verify(fun)?;
super::call::verify(program,fun)?;
super::source::verify(fun)?;
let size = fun.code.len();
if size == 0 || fun.spans.len() != size {
return Err("invalid code/span table".into());
}
if fun.defaults.len() != fun.params.len()
|| fun.locals.get(..fun.params.len()) != Some(fun.params.as_slice())
{
return Err("parameters must be a local-slot prefix".into());
}
if fun.locals.iter().collect::<HashSet<_>>().len() != fun.locals.len() {
return Err("duplicate local slot name".into());
}
let local = |i: usize| {
if i < fun.locals.len() {
Ok(())
} else {
Err("invalid local reference".to_string())
}
};
let target = |i: usize| {
if i < size {
Ok(())
} else {
Err("invalid jump target".to_string())
}
};
let function = |i: usize| {
if i < program.functions.len() {
Ok(())
} else {
Err("invalid call target".to_string())
}
};
let text = |i: usize| {
if matches!(fun.constants.get(i), Some(Value::Text(_))) {
Ok(())
} else {
Err("invalid text constant reference".to_string())
}
};
let mut shapes = Vec::with_capacity(size);
for (ip, op) in fun.code.iter().copied().enumerate() {
// pop count, push count, successor PCs. All opcodes are handled.
let next = vec![ip + 1];
let shape = match op {
Op::Constant(i) => {
if i >= fun.constants.len() {
return Err("invalid constant reference".into());
}
(0, 1, next)
}
Op::True | Op::False | Op::None_ => (0, 1, next),
Op::GetLocal(i) => {
local(i)?;
(0, 1, next)
}
Op::SetLocal(i) | Op::CheckBinding(i) => {
local(i)?;
(1, 1, next)
}
Op::Pop | Op::Print => (1, 0, next),
Op::Add
| Op::Sub
| Op::Mul
| Op::Div
| Op::Equal
| Op::NotEqual
| Op::Greater
| Op::GreaterEqual
| Op::Less
| Op::Lessequal
| Op::Split
| Op::Join
| Op::GetIndex
| Op::FootnoteGet => (2, 1, next),
Op::ContractFootnoteGet(i) => {
local(i)?;
(2, 1, next)
}
Op::Negate
| Op::Not
| Op::Input
| Op::Len
| Op::Str
| Op::Int
| Op::TypeOf
| Op::Trim
| Op::IterItems
| Op::CallFn => (1, 1, next),
Op::BuildList(n) => (n, 1, next),
Op::BuildMap(n) => (n.checked_mul(2).ok_or("map arity overflow")?, 1, next),
Op::Jump(t) => {
target(t)?;
(0, 0, vec![t])
}
Op::JumpIfFalse(t) => {
target(t)?;
(1, 0, vec![ip + 1, t])
}
Op::JumpIfProvided(p, t) => {
target(t)?;
if p >= fun.params.len() {
return Err("invalid presence reference".into());
}
(0, 0, vec![ip + 1, t])
}
Op::ResetLoop(id)|Op::CheckLoop(id)=>{
if !matches!(fun.statements.get(id).map(|s|&s.rule),Some(super::StatementRule::While(_))){return Err("invalid source loop reference".into());}
(0,0,next)
},
Op::FillMissingParameters | Op::CheckArguments | Op::RestoreReceiver | Op::CheckFrameArguments => (0, 0, next),
Op::CheckReturn => (1, 1, next),
Op::Call(f, n) => {
function(f)?;
(n, 1, next)
}
Op::CallArgs(f, l) => {
function(f)?;
let layout = fun.call_layouts.get(l).ok_or("invalid call layout")?;
(layout.len(), 1, next)
}
Op::BuiltinCallArgs(name,l)|Op::PluginCallArgs(name,l)=>{
text(name)?;
if matches!(op,Op::BuiltinCallArgs(_,_)) && !matches!(&fun.constants[name],Value::Text(name) if crate::arguments::builtin_signature(name).is_some()){return Err("unknown builtin signature".into());}
let layout=fun.call_layouts.get(l).ok_or("invalid call layout")?;(layout.len(),1,next)
},
Op::HostCallArgs(id,l)=>{
if crate::host::HostFn::from_u16(id).is_none(){return Err("unknown host id".into());}
let layout=fun.call_layouts.get(l).ok_or("invalid call layout")?;(layout.len(),1,next)
},
Op::PathCallArgs(id,l)=>{
if !matches!(fun.calls.get(id).and_then(|c|c.dispatch.as_ref()),Some(super::CallDispatch::Path{..})){return Err("invalid path call record".into());}
let layout=fun.call_layouts.get(l).ok_or("invalid path call layout")?;(layout.len(),1,next)
},
Op::RejectCallResolution(index)=>{text(index)?;(0,0,vec![])},
Op::MethodCall(name, n) => {
text(name)?;
(n.checked_add(1).ok_or("method arity overflow")?, 1, next)
}
Op::MethodCallArgs(name, l) => {
text(name)?;
let layout = fun.call_layouts.get(l).ok_or("invalid method layout")?;
(
layout.len().checked_add(1).ok_or("method arity overflow")?,
1,
next,
)
}
Op::PluginCall(name, n) => {
text(name)?;
if let Value::Text(s) = &fun.constants[name] {
if s.starts_with('@') && n % 2 != 0 {
return Err("odd named plugin arity".into());
}
}
(n, 1, next)
}
Op::HostCall(id, n) => {
if crate::host::HostFn::from_u16(id).is_none() {
return Err("invalid host reference".into());
}
(n, 1, next)
}
Op::TagInstance(f) => {
function(f)?;
if !program.objects.iter().any(|(_, id)| *id == f) {
return Err("invalid constructor reference".into());
}
(1, 1, next)
}
Op::RejectDuplicateArgument(name) | Op::RejectArgumentExpansion(name) => {
text(name)?;
(0, 0, vec![])
}
Op::Return => (1, 0, vec![]),
};
shapes.push(shape);
}
let mut heights = vec![None; size];
heights[0] = Some(0);
let mut queue = VecDeque::from([0]);
while let Some(ip) = queue.pop_front() {
let height = heights[ip].unwrap();
let (pop, push, edges) = &shapes[ip];
if height < *pop {
return Err(format!("stack underflow at {ip}"));
}
if matches!(fun.code[ip], Op::Return) && height != 1 {
return Err(format!("return leaks stack values at {ip}"));
}
let after = (height - pop)
.checked_add(*push)
.ok_or("stack height overflow")?;
for &to in edges {
target(to)?;
match heights[to] {
Some(known) if known != after => {
return Err(format!("inconsistent stack join at {to}"))
}
Some(_) => {}
None => {
heights[to] = Some(after);
queue.push_back(to);
}
}
}
}
Ok(StackCertificate { heights })
}
fn check_forest(parents: &[Option<usize>]) -> Result<(), String> {
let mut colors = vec![0u8; parents.len()];
for start in 0..parents.len() {
if colors[start] != 0 {
continue;
}
let mut cursor = Some(start);
let mut path = Vec::new();
while let Some(node) = cursor {
if node >= parents.len() {
return Err("invalid forest reference".into());
}
if colors[node] == 1 {
return Err("cyclic bytecode declaration graph".into());
}
if colors[node] == 2 {
break;
}
colors[node] = 1;
path.push(node);
cursor = parents[node];
}
for node in path {
colors[node] = 2;
}
}
Ok(())
}
+454 -287
View File
@@ -5,14 +5,13 @@ use std::io::Write;
use std::path::{Path, PathBuf};
use crate::builtin::{
builtin_at, builtin_footnote_get, builtin_int, builtin_iter_items, builtin_join,
builtin_len, builtin_split, builtin_str, builtin_trim,
builtin_type,
builtin_at, builtin_footnote_get, builtin_int, builtin_iter_items, builtin_join, builtin_len,
builtin_split, builtin_str, builtin_trim, builtin_type,
};
use crate::host::{call_host, HostContext, HostFn};
use crate::bytecode::{Op, Program};
use crate::bytecode::{FnChunk, Op, Program};
use crate::debug::emit_trace;
use crate::diagnostics::{bail_at, Span};
use crate::diagnostics::{Diagnostic, DiagnosticFrame, SourceIdentity, Span};
use crate::host::{call_host, HostContext, HostFn};
use crate::input_feed::InputFeed;
use crate::value::Value;
use std::collections::HashMap;
@@ -24,14 +23,32 @@ pub struct Vm {
input: InputFeed,
trace: bool,
host: HostContext,
current_source: SourceIdentity,
current_scope: Option<u32>,
diagnostic_frames: Vec<DiagnosticFrame>,
}
struct Frame {
fn_idx: usize,
ip: usize,
slots: Vec<Value>,
slots: Vec<Option<Value>>,
provided: Vec<bool>,
loop_counts: HashMap<usize,u32>,
receiver: Option<Value>,
defaults_active: bool,
/// When set, retag matrix-like return to this `_type` (method chaining).
retag_matrix: Option<String>,
retag_matrix: Option<(String, Option<Vec<String>>)>,
has_call_site: bool,
retag_instance: Option<usize>,
}
impl Frame {
fn read(&self, fun: &FnChunk, index: usize) -> Option<Value> {
let name=&fun.locals[index];
let overlay=self.defaults_active && self.receiver.is_some() && (name=="self" || name=="自") &&
(index>=fun.params.len() || self.provided.get(index).copied().unwrap_or(false) || self.slots[index].is_none());
if overlay {self.receiver.clone()} else {self.slots[index].clone()}
}
}
impl Vm {
@@ -43,6 +60,12 @@ impl Vm {
input: InputFeed::new(false, Vec::new()),
trace: false,
host: HostContext::for_run(path, Default::default(), Vec::new()),
current_source: SourceIdentity {
path: path.map(Path::to_path_buf),
sha256: None,
},
current_scope: None,
diagnostic_frames: Vec::new(),
}
}
@@ -54,6 +77,12 @@ impl Vm {
input: InputFeed::new(true, Vec::new()),
trace: false,
host: HostContext::for_capture(path, Default::default()),
current_source: SourceIdentity {
path: path.map(Path::to_path_buf),
sha256: None,
},
current_scope: None,
diagnostic_frames: Vec::new(),
}
}
@@ -76,6 +105,50 @@ impl Vm {
std::mem::take(&mut self.host.plots)
}
#[cfg(all(test, feature = "exec-host"))]
pub(crate) fn foreign_source_evidence(
&mut self,
) -> (
Vec<Value>,
Vec<Value>,
Vec<crate::host::foreign_session::ProtocolStep>,
) {
crate::source_refinement_tests::boundary_evidence(&mut self.host)
}
#[cfg(test)]
pub(crate) fn diagnostic_scope_state(
&self,
) -> (SourceIdentity, usize, Vec<String>, Vec<u32>, Option<u32>) {
(
self.current_source.clone(),
self.diagnostic_frames.len(),
self.host.call_stack.clone(),
self.host.call_site_lines.clone(),
self.current_scope,
)
}
#[cfg(all(test, feature = "exec-host"))]
pub(crate) fn causal_replay_evidence(&mut self) -> serde_json::Value {
crate::causal_replay_tests::evidence(&mut self.host)
}
#[cfg(all(test, feature = "exec-host"))]
pub(crate) fn application_evidence(
&self,
) -> (
Vec<Value>,
Vec<crate::host::foreign_session::ProtocolStep>,
usize,
) {
(
self.host.oracle_observations.clone(),
self.host.foreign_protocol_steps.clone(),
self.host.foreign_sessions.len(),
)
}
fn emit_line(&mut self, text: &str) {
if self.capture {
self.captured_stdout.push_str(text);
@@ -98,56 +171,177 @@ impl Vm {
}
fn err_at(&self, span: Span, message: impl Into<String>) -> anyhow::Error {
bail_at(self.path.as_deref(), span, message)
Diagnostic::new(self.current_source.path.as_deref(), span, message)
.with_source(&self.current_source)
.with_frames(&self.diagnostic_frames)
.into()
}
fn push_source_frame(&mut self, callee: &FnChunk, call_span: Option<Span>) {
let mut source = callee.source.clone();
if source.path.is_none() {
if self.diagnostic_frames.is_empty() {
source.path = self.path.clone();
} else if self.current_scope == Some(callee.module_scope) {
source.path = self.current_source.path.clone();
}
}
self.diagnostic_frames.push(DiagnosticFrame {
function: callee.name.clone(),
declaration: source.at(callee.declaration_span),
call_site: call_span.map(|span| self.current_source.at(span)),
});
}
pub fn run(&mut self, program: &Program) -> Result<Value> {
// Entry execution, like Interpreter::run_function, seeds unbound locals
// with None. Inferred prose parameters do not require a program caller.
let mut seen=std::collections::HashSet::new();
let metadata=program.entry_metadata.iter().filter(|(name,_)|seen.insert(name.clone())).cloned().collect::<Vec<_>>();
self.execute(program, program.entry, &metadata)
}
/// Run `fn_idx` with named arguments already bound to local slots.
/// Does not fall back to [`Program::entry`].
pub fn call_function(
&mut self,
program: &Program,
fn_idx: usize,
args: &[(String, Value)],
) -> Result<Value> {
if fn_idx >= program.functions.len() {
return Err(self.err_at(
Span::new(1, 1),
format!("bytecode: function index {fn_idx} is out of range"),
));
}
self.execute(program, fn_idx, args)
}
fn execute(
&mut self,
program: &Program,
fn_idx: usize,
args: &[(String, Value)],
) -> Result<Value> {
program.verify().map_err(|error|self.err_at(Span::new(1,1),error))?;
let previous_source = self.current_source.clone();
let previous_scope = self.current_scope;
let previous_depth = self.diagnostic_frames.len();
let previous_calls = self.host.call_stack.clone();
let previous_sites = self.host.call_site_lines.clone();
let previous_line = self.host.current_line;
// Scope numbers belong to one compiled program; never carry them into
// a new external invocation, even when it preserves an outer prefix.
self.current_scope = None;
let result = self.execute_inner(program, fn_idx, args);
self.diagnostic_frames.truncate(previous_depth);
self.current_source = previous_source;
self.current_scope = previous_scope;
self.host.call_stack = previous_calls;
self.host.call_site_lines = previous_sites;
self.host.current_line = previous_line;
result
}
fn execute_inner(
&mut self,
program: &Program,
fn_idx: usize,
args: &[(String, Value)],
) -> Result<Value> {
let mut stack: Vec<Value> = Vec::new();
let entry = program.entry;
let arity = program.functions[entry].params.len();
if arity != 0 {
let span = program.functions[entry]
.spans
.first()
.copied()
.unwrap_or(Span::new(1, 1));
return Err(self.err_at(span, "bytecode: entry function needs arguments"));
let mut slots = vec![None; program.functions[fn_idx].locals.len().max(1)];
for (name, value) in args {
if let Some(i) = program.functions[fn_idx]
.locals
.iter()
.position(|local| local == name)
{
if i < slots.len() {
slots[i] = Some(value.clone());
}
}
}
let mut frames = vec![Frame {
fn_idx: entry,
fn_idx,
loop_counts: HashMap::new(),
receiver: None, defaults_active: true,
ip: 0,
slots: vec![Value::None; program.functions[entry].locals.len().max(1)],
provided: initial_presence(&slots,program.functions[fn_idx].params.len()),
slots,
retag_matrix: None,
has_call_site: false,
retag_instance: None,
}];
self.push_source_frame(&program.functions[fn_idx], None);
self.host
.push_call_frame(&program.functions[entry].name.clone());
.push_call_frame(&program.functions[fn_idx].name.clone());
let mut source_owner = None;
loop {
let frame = frames.last_mut().unwrap();
let fun = &program.functions[frame.fn_idx];
if source_owner != Some(frame.fn_idx) {
self.current_source = self
.diagnostic_frames
.last()
.expect("active source frame")
.declaration
.source
.clone();
self.current_scope = Some(fun.module_scope);
source_owner = Some(frame.fn_idx);
}
if frame.ip >= fun.code.len() {
let span = fun.spans.last().copied().unwrap_or(Span::new(1, 1));
return Err(self.err_at(span, format!("bytecode: ip out of range in {}", fun.name)));
}
let ip = frame.ip;
let op = fun.code[ip];
let mut op = fun.code[ip];
let span = fun.spans.get(ip).copied().unwrap_or(Span::new(1, 1));
frame.ip += 1;
self.host.current_line = span.line;
let mut print_returns_none=false;
let mut method_name_override=None;
let mut constructor_tag=None;
if let Op::PathCallArgs(owner,layout)=op {
let call=&fun.calls[owner];
let Some(super::CallDispatch::Path{receiver,check_receiver,fallback})=&call.dispatch else{unreachable!("verified path dispatch")};
let value=if *check_receiver {fun.locals.iter().position(|n|n==receiver).and_then(|slot|frame.read(fun,slot))}else{None};
if let Some(value)=value {
stack.push(value);method_name_override=Some(call.source.callee.clone());op=Op::MethodCallArgs(0,layout);
} else {match fallback {
super::PathFallback::User{target,tag_instance}=>{op=Op::CallArgs(*target,layout);if *tag_instance{constructor_tag=Some(*target);}},
super::PathFallback::Error{message}=>return Err(self.err_at(span,message)),
}}
}
if let Op::BuiltinCallArgs(name,layout)=op {
let name=fun.constants[name].as_display();let names=&fun.call_layouts[layout];
let values=take_actuals(&mut stack,names.len()).map_err(|m|self.err_at(span,m))?;
let (params,allow_missing)=crate::arguments::builtin_signature(&name).ok_or_else(||self.err_at(span,"unknown builtin signature"))?;
let parameters=params.iter().map(|p|p.to_string()).collect::<Vec<_>>();
let bound=crate::arguments::bind(&parameters,names,&values,allow_missing).map_err(|m|self.err_at(span,m))?;
for param in params {
stack.push(bound.get(*param).cloned().unwrap_or_else(||Value::Text(String::new())));
}
op=match name.as_str(){"print"=>{print_returns_none=true;Op::Print},"input"=>Op::Input,"len"=>Op::Len,"str"=>Op::Str,"int"=>Op::Int,"type"=>Op::TypeOf,"trim"=>Op::Trim,"split"=>Op::Split,"join"=>Op::Join,"at"=>Op::GetIndex,"call_fn"=>Op::CallFn,_=>unreachable!()};
}
if self.trace {
match op {
Op::Print | Op::Input | Op::Call(_, _) | Op::Return => {
Op::Print | Op::Input | Op::Call(_, _) | Op::CallArgs(_, _) | Op::Return => {
let kind = match op {
Op::Print => "print",
Op::Input => "input",
Op::Call(_, _) => "call",
Op::Call(_, _) | Op::CallArgs(_, _) => "call",
Op::Return => "return",
_ => unreachable!(),
};
let ip_s = ip.to_string();
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(span),
"op",
&[
@@ -162,6 +356,61 @@ impl Vm {
}
match op {
Op::CheckArguments => {
if let Some(contract)=&fun.contract {
let bound=fun.locals.iter().zip(&frame.slots).filter_map(|(name,value)|
value.clone().map(|value|(name.clone(),value))).collect();
// Internal call errors identify the authored call site, like Tree.
let site=self.diagnostic_frames.last().and_then(|f|f.call_site.as_ref());
let call_site=site.cloned().unwrap_or_else(||self.current_source.at(span));
if let Some(diag)=crate::contract::call_arg_diag_in(contract,&fun.name,&bound,&call_site,&fun.source) {
let depth=self.diagnostic_frames.len()-usize::from(site.is_some());
return Err(diag.with_frames(&self.diagnostic_frames[..depth]).into());
}
}
}
Op::CheckReturn => {
if let Some(contract)=&fun.contract {
let value=stack.last().cloned().unwrap_or(Value::None);
if let Some(diag)=crate::contract::return_diag(contract,&fun.name,&value,self.current_source.path.as_deref(),span) {
return Err(diag.into());
}
}
}
Op::CheckBinding(slot) => {
let name=&fun.locals[slot];
if let Some(contract)=fun.var_contracts.get(name) {
let value=stack.last().cloned().unwrap_or(Value::None);
if let Some(diag)=crate::contract::fields_diag(contract,name,&value,self.current_source.path.as_deref(),span) {
return Err(diag.into());
}
}
}
Op::RestoreReceiver => {
if let Some(receiver)=&frame.receiver {
for (index,name) in fun.locals.iter().enumerate() {
if name=="self" || name=="自" {frame.slots[index]=Some(receiver.clone());}
}
}
frame.defaults_active=false;
}
Op::CheckFrameArguments => {
if let Some(contract)=&fun.contract {
let bound=fun.locals.iter().zip(&frame.slots).filter_map(|(name,value)|value.clone().map(|value|(name.clone(),value))).collect();
if let Some(diag)=crate::contract::call_arg_diag(contract,&fun.name,&bound,self.current_source.path.as_deref(),fun.declaration_span){return Err(diag.with_source(&self.current_source).with_frames(&self.diagnostic_frames).into());}
}
}
Op::ResetLoop(id) => {frame.loop_counts.insert(id,0);}
Op::CheckLoop(id) => {
let counter=frame.loop_counts.get_mut(&id).ok_or_else(||self.err_at(span,"uninitialized loop counter"))?;
if *counter>=1_000_000 {return Err(self.err_at(span,"while loop exceeded iteration limit"));}
*counter+=1;
}
Op::FillMissingParameters => {
for index in 0..fun.params.len() {
if frame.slots[index].is_none() { frame.slots[index]=Some(Value::None); }
}
}
Op::Constant(i) => stack.push(fun.constants[i as usize].clone()),
Op::True => stack.push(Value::Bool(true)),
Op::False => stack.push(Value::Bool(false)),
@@ -170,7 +419,10 @@ impl Vm {
stack.pop();
}
Op::GetLocal(i) => {
let v = frames.last().unwrap().slots[i as usize].clone();
let current=frames.last().unwrap();
let index=i as usize;let name=&fun.locals[index];
let value=current.read(fun,index);
let v=value.ok_or_else(||self.err_at(span,format!("undefined variable `{name}`")))?;
stack.push(v);
}
Op::SetLocal(i) => {
@@ -178,130 +430,32 @@ impl Vm {
let slot = i as usize;
let fr = frames.last_mut().unwrap();
if fr.slots.len() <= slot {
fr.slots.resize(slot + 1, Value::None);
fr.slots.resize(slot + 1, None);
}
fr.slots[slot] = v;
fr.slots[slot] = Some(v);
}
Op::Add => {
let b = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let a = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
if let Some(v) = crate::linalg_ops::try_matrix_binary(
&mut self.host,
crate::ast::BinaryOp::Add,
&a,
&b,
)
.map_err(|m| self.err_at(span, m))?
{
stack.push(v);
} else {
stack.push(add(a, b).map_err(|m| self.err_at(span, m))?);
}
Op::Add | Op::Sub | Op::Mul | Op::Div | Op::Equal | Op::NotEqual |
Op::Greater | Op::GreaterEqual | Op::Less | Op::Lessequal => {
use crate::ast::BinaryOp;
let operation=match op {
Op::Add=>BinaryOp::Add,Op::Sub=>BinaryOp::Sub,Op::Mul=>BinaryOp::Mul,Op::Div=>BinaryOp::Div,
Op::Equal=>BinaryOp::Eq,Op::NotEqual=>BinaryOp::Ne,Op::Greater=>BinaryOp::Gt,
Op::GreaterEqual=>BinaryOp::Ge,Op::Less=>BinaryOp::Lt,Op::Lessequal=>BinaryOp::Le,
_=>unreachable!(),
};
let right=pop(&mut stack).map_err(|m|self.err_at(span,m))?;
let left=pop(&mut stack).map_err(|m|self.err_at(span,m))?;
let value=if let Some(value)=crate::linalg_ops::try_matrix_binary(&mut self.host,operation,&left,&right)
.map_err(|m|self.err_at(span,m))? {value} else {
crate::builtin::eval_binary(operation,&left,&right).map_err(|m|self.err_at(span,m))?
};
stack.push(value);
}
Op::Sub => {
let b = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let a = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
if let Some(v) = crate::linalg_ops::try_matrix_binary(
&mut self.host,
crate::ast::BinaryOp::Sub,
&a,
&b,
)
.map_err(|m| self.err_at(span, m))?
{
stack.push(v);
} else {
match (a, b) {
(Value::Int(x), Value::Int(y)) => stack.push(Value::Int(x - y)),
(Value::Num(x), Value::Num(y)) => stack.push(Value::Num(x - y)),
(Value::Int(x), Value::Num(y)) => {
stack.push(Value::Num(x as f64 - y))
}
(Value::Num(x), Value::Int(y)) => {
stack.push(Value::Num(x - y as f64))
}
_ => return Err(self.err_at(span, "`-` needs ints or nums")),
}
}
Op::Negate | Op::Not => {
let value=pop(&mut stack).map_err(|m|self.err_at(span,m))?;
let operation=if matches!(op,Op::Negate){crate::ast::UnaryOp::Neg}else{crate::ast::UnaryOp::Not};
stack.push(crate::builtin::eval_unary(operation,&value).map_err(|m|self.err_at(span,m))?);
}
Op::Mul => {
let b = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let a = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
if let Some(v) = crate::linalg_ops::try_matrix_binary(
&mut self.host,
crate::ast::BinaryOp::Mul,
&a,
&b,
)
.map_err(|m| self.err_at(span, m))?
{
stack.push(v);
} else {
match (a, b) {
(Value::Int(x), Value::Int(y)) => stack.push(Value::Int(x * y)),
(Value::Num(x), Value::Num(y)) => stack.push(Value::Num(x * y)),
(Value::Int(x), Value::Num(y)) => {
stack.push(Value::Num(x as f64 * y))
}
(Value::Num(x), Value::Int(y)) => {
stack.push(Value::Num(x * y as f64))
}
_ => return Err(self.err_at(span, "`*` needs ints or nums")),
}
}
}
Op::Div => {
let b = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let a = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
match (a, b) {
(Value::Int(_), Value::Int(0)) => {
return Err(self.err_at(span, "division by zero"));
}
(Value::Int(x), Value::Int(y)) => stack.push(Value::Int(x / y)),
(Value::Num(_), Value::Num(y)) if y == 0.0 => {
return Err(self.err_at(span, "division by zero"));
}
(Value::Int(_), Value::Num(y)) if y == 0.0 => {
return Err(self.err_at(span, "division by zero"));
}
(Value::Num(_), Value::Int(0)) => {
return Err(self.err_at(span, "division by zero"));
}
(Value::Num(x), Value::Num(y)) => stack.push(Value::Num(x / y)),
(Value::Int(x), Value::Num(y)) => {
stack.push(Value::Num(x as f64 / y))
}
(Value::Num(x), Value::Int(y)) => {
stack.push(Value::Num(x / y as f64))
}
_ => return Err(self.err_at(span, "`/` needs ints or nums")),
}
}
Op::Negate => match pop(&mut stack).map_err(|m| self.err_at(span, m))? {
Value::Int(n) => stack.push(Value::Int(-n)),
Value::Num(n) => stack.push(Value::Num(-n)),
_ => return Err(self.err_at(span, "unary `-` needs int or num")),
},
Op::Not => {
let v = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
stack.push(Value::Bool(!v.truthy()));
}
Op::Equal => {
let b = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let a = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
stack.push(Value::Bool(a == b));
}
Op::NotEqual => {
let b = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let a = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
stack.push(Value::Bool(a != b));
}
Op::Greater => cmp(&mut stack, |o| o.is_gt()).map_err(|m| self.err_at(span, m))?,
Op::GreaterEqual => {
cmp(&mut stack, |o| o.is_ge()).map_err(|m| self.err_at(span, m))?
}
Op::Less => cmp(&mut stack, |o| o.is_lt()).map_err(|m| self.err_at(span, m))?,
Op::Lessequal => cmp(&mut stack, |o| o.is_le()).map_err(|m| self.err_at(span, m))?,
Op::Jump(t) => {
frames.last_mut().unwrap().ip = t as usize;
}
@@ -406,7 +560,7 @@ impl Vm {
let v = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
stack.push(builtin_iter_items(&v).map_err(|m| self.err_at(span, m))?);
}
Op::FootnoteGet => {
Op::FootnoteGet | Op::ContractFootnoteGet(_) => {
let label = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let base = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let label = match label {
@@ -415,10 +569,27 @@ impl Vm {
return Err(self.err_at(span, "footnote label must be text"));
}
};
stack.push(
builtin_footnote_get(&base, &label).map_err(|m| self.err_at(span, m))?,
);
let contract=if let Op::ContractFootnoteGet(slot)=op {
let name=&fun.locals[slot];fun.var_contracts.get(name).map(|c|(name,c))
} else {None};
if let Some((name,c))=contract {
if let Some(diag)=crate::contract::index_diag(c,name,&label,&Value::None,self.current_source.path.as_deref(),span) {return Err(diag.into());}
}
let value=builtin_footnote_get(&base,&label).map_err(|m|self.err_at(span,m))?;
if let Some((name,c))=contract {
if let Some(diag)=crate::contract::index_diag(c,name,&label,&value,self.current_source.path.as_deref(),span) {return Err(diag.into());}
}
stack.push(value);
}
Op::HostCallArgs(id, layout) => {
let hf=HostFn::from_u16(id).ok_or_else(||self.err_at(span,"unknown host id"))?;
let names=&fun.call_layouts[layout];let values=take_actuals(&mut stack,names.len()).map_err(|m|self.err_at(span,m))?;
let params=hf.all_params().iter().map(|p|p.to_string()).collect::<Vec<_>>();
let bound=crate::arguments::bind(&params,names,&values,true).map_err(|m|self.err_at(span,m))?;
for req in hf.required_params(){if !bound.contains_key(*req){return Err(self.err_at(span,format!("{} requires `{req}`",hf.name())));}}
stack.push(call_host(&mut self.host,hf,&bound).map_err(|m|self.err_at(span,m))?);
}
Op::BuiltinCallArgs(_,_)=>unreachable!("builtin dispatch was prepared above"),
Op::HostCall(id, argc) => {
let hf = HostFn::from_u16(id)
.ok_or_else(|| self.err_at(span, format!("unknown host id {id}")))?;
@@ -438,71 +609,105 @@ impl Vm {
}
for req in hf.required_params() {
if !bound.contains_key(*req) {
return Err(self.err_at(
span,
format!("{} requires `{req}`", hf.name()),
));
return Err(
self.err_at(span, format!("{} requires `{req}`", hf.name()))
);
}
}
let result =
call_host(&mut self.host, hf, &bound).map_err(|m| self.err_at(span, m))?;
stack.push(result);
}
Op::Call(fid, argc) => {
Op::RejectDuplicateArgument(index) => {
let name=fun.constants[index].as_display();
return Err(self.err_at(span,format!("duplicate named argument `{name}`")));
}
Op::RejectArgumentExpansion(index) => {
return Err(self.err_at(span,fun.constants[index].as_display()));
}
Op::RejectCallResolution(index)=>return Err(self.err_at(span,fun.constants[index].as_display())),
Op::PathCallArgs(_,_)=>unreachable!("path dispatch was selected above"),
Op::JumpIfProvided(index, target) => {
if frame.provided.get(index as usize).copied().unwrap_or(false) {
frame.ip = target as usize;
}
}
Op::Call(_, _) | Op::CallArgs(_, _) => {
let (fid, argc) = match op {
Op::Call(fid, argc) => (fid, argc as usize),
Op::CallArgs(fid, layout) => (fid, layout),
_ => unreachable!(),
};
let fid = fid as usize;
let argc = argc as usize;
let callee = &program.functions[fid];
if callee.params.len() != argc {
return Err(self.err_at(span, "call argument count mismatch"));
}
let mut slots = vec![Value::None; callee.locals.len().max(argc).max(1)];
for i in (0..argc).rev() {
slots[i] = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
}
let layout = if matches!(op, Op::CallArgs(_, _)) {
fun.call_layouts[argc as usize].clone()
} else { vec![None; argc as usize] };
let mut values = Vec::new();
for _ in &layout { values.push(pop(&mut stack).map_err(|m| self.err_at(span, m))?); }
values.reverse();
let (slots, provided) = bind_call_values(callee, &layout, values).map_err(|m| self.err_at(span, m))?;
let fname = program.functions[fid].name.clone();
frames.push(Frame {
fn_idx: fid,
loop_counts: HashMap::new(),
receiver: None, defaults_active: true,
ip: 0,
provided,
slots,
retag_matrix: None,
has_call_site: true,
retag_instance: constructor_tag,
});
self.push_source_frame(callee, Some(span));
self.host.push_call_frame(&fname);
self.host.push_call_site_line(span.line);
}
Op::TagInstance(idx) => {
let type_name = match fun.constants.get(idx as usize) {
Some(Value::Text(s)) => s.clone(),
_ => {
return Err(self.err_at(span, "TagInstance needs type name constant"));
}
};
let class = program.functions.get(idx).ok_or_else(|| self.err_at(span,"TagInstance needs an object function"))?;
let v = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
stack.push(tag_instance_value(&type_name, v));
stack.push(crate::value::tag_instance(&class.name, &class.module_path, v));
}
Op::MethodCall(name_idx, argc) => {
let argc = argc as usize;
let method_name = match fun.constants.get(name_idx as usize) {
Op::MethodCall(_, _) | Op::MethodCallArgs(_, _) => {
let (name_idx, argc) = match op {
Op::MethodCall(name, argc) => (name, argc as usize),
Op::MethodCallArgs(name, layout) => (name, layout),
_ => unreachable!(),
};
let layout = if matches!(op, Op::MethodCallArgs(_, _)) {
fun.call_layouts[argc as usize].clone()
} else { vec![None; argc as usize] };
let argc = layout.len();
let method_name = if let Some(name)=method_name_override {name} else {match fun.constants.get(name_idx as usize) {
Some(Value::Text(s)) => s.clone(),
_ => {
return Err(self.err_at(span, "MethodCall needs method name constant"));
}
};
}};
let recv = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let mut vals = Vec::with_capacity(argc);
for _ in 0..argc {
vals.push(pop(&mut stack).map_err(|m| self.err_at(span, m))?);
}
vals.reverse();
let type_name = instance_type_name_bc(&recv).map_err(|m| self.err_at(span, m))?;
let start_oi = program
.objects
.iter()
.position(|(n, _)| n == &type_name)
let type_name =
instance_type_name_bc(&recv).map_err(|m| self.err_at(span, m))?;
let namespace=crate::value::instance_module(&recv).map_err(|m| self.err_at(span,m))?;
let start_oi = if let Some(namespace)=&namespace {
program.objects.iter().position(|(name,id)| name==&type_name && &program.functions[*id].module_path==namespace)
} else {
let local=program.objects.iter().position(|(name,id)| name==&type_name && program.functions[*id].module_scope==fun.module_scope);
if local.is_some() { local } else {
let candidates=program.objects.iter().enumerate().filter(|(_, (name,_))| name==&type_name).map(|(index,_)|index).collect::<Vec<_>>();
if candidates.len()>1 { return Err(self.err_at(span,format!("ambiguous object type `{type_name}`: a module tag is required"))); }
candidates.first().copied()
}
}
.ok_or_else(|| {
self.err_at(span, format!("unknown object type `{type_name}`"))
})?;
let method_fid = resolve_method_fid(program, start_oi, &method_name).ok_or_else(
|| {
let method_fid = resolve_method_fid(program, start_oi, &method_name)
.ok_or_else(|| {
let mut chain = Vec::new();
let mut oi = Some(start_oi);
let mut seen = std::collections::HashSet::new();
@@ -524,36 +729,37 @@ impl Vm {
chain.join(" → ")
),
)
},
)?;
})?;
let callee = &program.functions[method_fid];
if callee.params.len() != argc {
return Err(self.err_at(span, "method argument count mismatch"));
}
let mut slots = vec![Value::None; callee.locals.len().max(1)];
for (i, v) in vals.into_iter().enumerate() {
slots[i] = v;
}
for (i, name) in callee.locals.iter().enumerate() {
if name == "自" || name == "self" {
slots[i] = recv.clone();
}
}
let (slots, provided) = bind_call_values(callee, &layout, vals).map_err(|m| self.err_at(span, m))?;
let retag_matrix = if crate::linalg_ops::is_matrix_value(&recv) {
Some(type_name.clone())
Some((type_name.clone(), namespace))
} else {
None
};
let fname = callee.name.clone();
frames.push(Frame {
fn_idx: method_fid,
loop_counts: HashMap::new(),
receiver: Some(recv.clone()), defaults_active: true,
ip: 0,
provided,
slots,
retag_matrix,
has_call_site: true,
retag_instance: None,
});
self.push_source_frame(callee, Some(span));
self.host.push_call_frame(&fname);
self.host.push_call_site_line(span.line);
}
Op::PluginCallArgs(name_idx,layout) => {
let name=fun.constants[name_idx].as_display();let names=&fun.call_layouts[layout];
let values=take_actuals(&mut stack,names.len()).map_err(|m|self.err_at(span,m))?;
let reg=self.host.plugins.get(&name).cloned().ok_or_else(||self.err_at(span,format!("unknown function `{name}`")))?;
let bound=crate::arguments::bind(&reg.params,names,&values,false).map_err(|m|self.err_at(span,m))?;
stack.push(crate::host::plugin::call_registered(&mut self.host,&name,&bound).map_err(|m|self.err_at(span,m))?);
}
Op::PluginCall(name_idx, argc) => {
let argc = argc as usize;
let raw_name = match fun.constants.get(name_idx as usize) {
@@ -590,14 +796,9 @@ impl Vm {
vals.push(pop(&mut stack).map_err(|m| self.err_at(span, m))?);
}
vals.reverse();
let reg = self
.host
.plugins
.get(&raw_name)
.cloned()
.ok_or_else(|| {
self.err_at(span, format!("unknown function `{raw_name}`"))
})?;
let reg = self.host.plugins.get(&raw_name).cloned().ok_or_else(|| {
self.err_at(span, format!("unknown function `{raw_name}`"))
})?;
if reg.params.len() != vals.len() {
return Err(self.err_at(
span,
@@ -628,41 +829,60 @@ impl Vm {
.map_err(|m| self.err_at(span, m))?;
let callee = &program.functions[fid];
if !callee.params.is_empty() {
return Err(self.err_at(
span,
format!("call_fn: `{name}` must have no parameters"),
));
return Err(
self.err_at(span, format!("call_fn: `{name}` must have no parameters"))
);
}
let slots = vec![Value::None; callee.locals.len().max(1)];
let slots = vec![None; callee.locals.len().max(1)];
let fname = callee.name.clone();
frames.push(Frame {
fn_idx: fid,
loop_counts: HashMap::new(),
receiver: None, defaults_active: true,
ip: 0,
provided: Vec::new(),
slots,
retag_matrix: None,
has_call_site: true,
retag_instance: None,
});
self.push_source_frame(callee, Some(span));
self.host.push_call_frame(&fname);
self.host.push_call_site_line(span.line);
}
Op::Return => {
let ret = pop(&mut stack).unwrap_or(Value::None);
let frame = frames.pop();
let constructor=frame.as_ref().and_then(|f|f.retag_instance);
self.diagnostic_frames.pop();
self.host.pop_call_frame();
self.host.pop_call_site_line();
if frame.as_ref().is_some_and(|f| f.has_call_site) {
self.host.pop_call_site_line();
}
let ret = match frame.and_then(|f| f.retag_matrix) {
Some(ty) => crate::linalg_ops::retag_matrix_like(&ty, ret),
Some((ty,namespace)) => {
let result=crate::linalg_ops::retag_matrix_like(&ty,ret);
if let Some(namespace)=namespace { crate::value::tag_instance(&ty,&namespace,result) } else { result }
},
None => ret,
};
let ret=if let Some(id)=constructor {let class=&program.functions[id];crate::value::tag_instance(&class.name,&class.module_path,ret)}else{ret};
if frames.is_empty() {
return Ok(ret);
}
stack.push(ret);
}
}
if print_returns_none{stack.push(Value::None);}
}
}
}
fn take_actuals(stack:&mut Vec<Value>,count:usize)->Result<Vec<Value>,String>{
if stack.len()<count{return Err("bytecode argument stack underflow".into());}
Ok(stack.split_off(stack.len()-count))
}
fn resolve_method_fid(program: &Program, start_oi: usize, method_name: &str) -> Option<usize> {
let mut oi = Some(start_oi);
let mut seen = std::collections::HashSet::new();
@@ -678,9 +898,12 @@ fn resolve_method_fid(program: &Program, start_oi: usize, method_name: &str) ->
{
return Some(*fid);
}
oi = program.object_bases.get(i).copied().flatten().and_then(|base_flat| {
program.objects.iter().position(|(_, f)| *f == base_flat)
});
oi = program
.object_bases
.get(i)
.copied()
.flatten()
.and_then(|base_flat| program.objects.iter().position(|(_, f)| *f == base_flat));
}
None
}
@@ -691,44 +914,6 @@ fn pop(stack: &mut Vec<Value>) -> Result<Value, String> {
.ok_or_else(|| "bytecode: stack underflow".to_string())
}
fn add(a: Value, b: Value) -> Result<Value, String> {
match (a, b) {
(Value::Int(x), Value::Int(y)) => Ok(Value::Int(x + y)),
(Value::Num(x), Value::Num(y)) => Ok(Value::Num(x + y)),
(Value::Int(x), Value::Num(y)) => Ok(Value::Num(x as f64 + y)),
(Value::Num(x), Value::Int(y)) => Ok(Value::Num(x + y as f64)),
(Value::Text(x), Value::Text(y)) => Ok(Value::Text(format!("{x}{y}"))),
(Value::Text(x), Value::Secret(y)) => Ok(Value::Text(format!("{x}{y}"))),
(Value::Secret(x), Value::Text(y)) => Ok(Value::Text(format!("{x}{y}"))),
(Value::Secret(x), Value::Secret(y)) => Ok(Value::Text(format!("{x}{y}"))),
(Value::Text(x), y) => Ok(Value::Text(format!("{x}{}", y.as_display()))),
(x, Value::Text(y)) => Ok(Value::Text(format!("{}{y}", x.as_display()))),
(Value::Int(x), y) => Ok(Value::Text(format!("{}{}", x, y.as_display()))),
(x, Value::Int(y)) => Ok(Value::Text(format!("{}{y}", x.as_display()))),
_ => Err("`+` needs ints, nums, or text".into()),
}
}
fn cmp(stack: &mut Vec<Value>, pred: impl Fn(std::cmp::Ordering) -> bool) -> Result<(), String> {
let b = pop(stack)?;
let a = pop(stack)?;
let ord = match (&a, &b) {
(Value::Int(x), Value::Int(y)) => x.cmp(y),
(Value::Num(x), Value::Num(y)) => x.partial_cmp(y).unwrap_or(std::cmp::Ordering::Equal),
(Value::Int(x), Value::Num(y)) => {
(*x as f64).partial_cmp(y).unwrap_or(std::cmp::Ordering::Equal)
}
(Value::Num(x), Value::Int(y)) => {
x.partial_cmp(&(*y as f64)).unwrap_or(std::cmp::Ordering::Equal)
}
(Value::Text(x), Value::Text(y)) => x.cmp(y),
_ => return Err("comparison needs two ints/nums or two texts".into()),
};
stack.push(Value::Bool(pred(ord)));
Ok(())
}
fn instance_type_name_bc(v: &Value) -> Result<String, String> {
match v {
Value::Map(entries) => entries
@@ -743,42 +928,24 @@ fn instance_type_name_bc(v: &Value) -> Result<String, String> {
}
}
fn tag_instance_value(type_name: &str, value: Value) -> Value {
match value {
Value::Map(mut entries) => {
// Always stamp the constructed type so an explicit `` `self` = > Parent ``
// inside a child constructor still yields the most specific `_type`.
entries.retain(|(k, _)| k != "_type");
entries.insert(0, ("_type".into(), Value::Text(type_name.into())));
Value::Map(entries)
}
Value::None => Value::Map(vec![("_type".into(), Value::Text(type_name.into()))]),
other => Value::Map(vec![
("_type".into(), Value::Text(type_name.into())),
("value".into(), other),
]),
}
fn resolve_fn_dynamic(program: &Program, current: usize, name: &str) -> Result<usize, String> {
super::names::lookup(program,current,name).ok_or_else(||format!("call_fn: unknown function `{name}`"))
}
fn resolve_fn_dynamic(program: &Program, current: usize, name: &str) -> Result<usize, String> {
for &cid in &program.children[current] {
if program.functions[cid].name == name {
return Ok(cid);
}
/// Bind already evaluated actuals; no expression or foreign effect runs here.
fn bind_call_values(callee: &FnChunk, layout: &[Option<String>], values: Vec<Value>)
-> std::result::Result<(Vec<Option<Value>>, Vec<bool>), String> {
if layout.len()!=values.len(){return Err("argument layout/value count mismatch".into());}
let plan=crate::arguments::plan_with_omissions(&callee.params,layout,&callee.defaults)?;
let mut slots = vec![None; callee.locals.len().max(1)];
for (name,actual) in plan.assignments {
if let Some(index)=callee.locals.iter().position(|local|local==&name){slots[index]=Some(values[actual].clone());}
}
let mut parent = program.parents[current];
while let Some(pid) = parent {
for &cid in &program.children[pid] {
if program.functions[cid].name == name {
return Ok(cid);
}
}
parent = program.parents[pid];
}
for (i, f) in program.functions.iter().enumerate() {
if program.parents[i].is_none() && f.name == name {
return Ok(i);
}
}
Err(format!("call_fn: unknown function `{name}`"))
let provided=initial_presence(&slots,callee.params.len());
Ok((slots, provided))
}
/// Presence is derived once, before defaults, rather than independently rebound.
fn initial_presence(slots:&[Option<Value>],parameters:usize)->Vec<bool> {
slots.iter().take(parameters).map(Option::is_some).collect()
}
+294
View File
@@ -0,0 +1,294 @@
//! `marqdo call`: run one already-bound function in a `.mq.md` file.
//!
//! Location, effects, and execution go through the existing compiler and runtime.
//! The command does not scan headings, and it does not run the program entry when
//! the named function cannot be compiled.
use std::path::Path;
use anyhow::Result;
use crate::ast::Function;
use crate::bytecode::{compile_for_call, Vm};
use crate::diagnostics::{Diagnostic, Span};
use crate::host::{HostCaps, HostContext};
use crate::interp::Interpreter;
use crate::load::load_module;
use crate::sema::{compile_file_closure, SemanticArtifact, SemanticProject};
use crate::value::Value;
use crate::Backend;
/// Call `function` in `path` with named arguments.
///
/// Non-pure effects are refused before the function runs. Host write, exec, net,
/// and plugin capabilities stay off for the call.
pub fn call_file(
path: &Path,
function: &str,
args: &[(String, Value)],
backend: Backend,
) -> Result<Value> {
let path = path
.canonicalize()
.map_err(|e| call_diag(None, 1, "call.compile", format!("failed to read {}: {e}", path.display())))?;
let compilation = compile_file_closure(&path).map_err(|e| preserve_diag(&path, e))?;
let artifact = locate_artifact(&compilation.semantics, &path, function)?;
deny_unsafe(&path, artifact)?;
execute_source_function(
&path,
artifact.source.line,
&artifact.display_name,
args,
tight_caps(),
backend,
true,
)
.map(|(value, _stdout)| value)
}
/// Run the function declared at `line` in `path`.
///
/// `display_name` is the declaration path (`计算器.加法`). The query the caller
/// typed is not used as a heading search and is not used as the bytecode path,
/// so a unique short name or SymbolId still executes that function.
/// `caps` are the host capabilities for this run. `call` passes them all off.
pub(crate) fn execute_source_function(
path: &Path,
line: u32,
display_name: &str,
args: &[(String, Value)],
caps: HostCaps,
backend: Backend,
strict_args: bool,
) -> Result<(Value, String)> {
let module = load_module(path)?;
let fun = find_function_at_line(&module.functions, line).ok_or_else(|| {
call_diag(
Some(path),
line,
"artifact.unresolved",
format!("无法在 {}:{} 定位函数 {display_name}", path.display(), line),
)
})?;
let expected = display_name
.rsplit('.')
.next()
.unwrap_or(display_name);
if fun.name != expected {
return Err(call_diag(
Some(path),
line,
"artifact.unresolved",
format!("源码行 {line} 上的函数是 `{}`,与 `{display_name}` 不一致", fun.name),
));
}
let mut interp = Interpreter::with_capture(Some(path), false)
.with_host(HostContext::for_capture(Some(path), caps));
// `call` checks the signature. `artifact.execute` keeps its previous
// environment injection so an input the body reads is not rejected as unknown.
let bound;
let invoke_args: &[(String, Value)] = if strict_args {
bound = interp.bind_external_args(&module, fun, args)?;
&bound
} else {
args
};
match backend {
Backend::Tree => {
let value = interp.invoke_bound_function(&module, fun, invoke_args)?;
Ok((value, interp.captured_stdout))
}
Backend::Bytecode => {
let program = compile_for_call(Some(path), &module)?;
// Source binding is authoritative even when declaration paths repeat.
let idx = program.functions.iter().position(|chunk|
chunk.module_scope == 0 && chunk.declaration_span == fun.span
&& chunk.name == fun.name && chunk.source.sha256 == module.source.sha256
).ok_or_else(|| {
call_diag(
Some(path),
line,
"artifact.unresolved",
format!("bytecode 没有编译出 `{display_name}`;拒绝改去执行程序入口"),
)
})?;
let mut vm = Vm::with_capture(Some(path))
.with_host(HostContext::for_capture(Some(path), caps));
let value = vm.call_function(&program, idx, invoke_args)?;
Ok((value, vm.captured_stdout))
}
}
}
/// Parse `name=value` tokens. Values are integers, decimals, quoted text, or True/False/None.
pub fn parse_call_args(tokens: &[String]) -> Result<Vec<(String, Value)>> {
let mut out = Vec::new();
for token in tokens {
let Some((name, raw)) = token.split_once('=') else {
return Err(call_diag(
None,
0,
"call.bad_arg",
format!("参数 `{token}` 需要写成 名字=值"),
));
};
if name.is_empty() {
return Err(call_diag(
None,
0,
"call.bad_arg",
format!("参数 `{token}` 缺少名字"),
));
}
let value = parse_literal(raw).map_err(|message| {
call_diag(None, 0, "call.bad_value", format!("参数 `{name}`: {message}"))
})?;
out.push((name.to_string(), value));
}
Ok(out)
}
fn tight_caps() -> HostCaps {
HostCaps {
fs_write: false,
exec: false,
net: false,
plugin: false,
}
}
fn locate_artifact<'a>(
project: &'a SemanticProject,
entry: &Path,
query: &str,
) -> Result<&'a SemanticArtifact> {
let in_entry: Vec<&SemanticArtifact> = project
.artifacts
.iter()
.filter(|artifact| same_file(&project.root, &artifact.source.file, entry))
.collect();
let exact: Vec<&SemanticArtifact> = in_entry
.iter()
.copied()
.filter(|artifact| {
artifact.display_name == query
|| artifact
.symbol_id
.as_ref()
.is_some_and(|id| id.as_str() == query)
|| artifact.artifact_id.0 == query
})
.collect();
let hits = if !exact.is_empty() {
exact
} else if !query.contains('.') {
in_entry
.iter()
.copied()
.filter(|artifact| {
artifact.kind == "function"
&& artifact.display_name.rsplit('.').next() == Some(query)
})
.collect()
} else {
Vec::new()
};
match hits.len() {
1 => Ok(hits[0]),
0 => Err(call_diag(
Some(entry),
1,
"artifact.unresolved",
format!("无法唯一绑定 `{query}`"),
)),
_ => Err(call_diag(
Some(entry),
1,
"artifact.ambiguous",
format!("无法唯一绑定 `{query}`"),
)),
}
}
fn deny_unsafe(path: &Path, artifact: &SemanticArtifact) -> Result<()> {
if artifact.kind != "function" || artifact.callable.as_deref() == Some("instance_method") {
return Err(call_diag(
Some(path),
artifact.source.line,
"artifact.not_executable",
"只有非实例函数可由 call 直接执行".to_string(),
));
}
if !artifact.effects.is_pure() {
return Err(call_diag(
Some(path),
artifact.source.line,
"artifact.effect_denied",
format!(
"call 默认只执行纯函数,拒绝效应 {}",
artifact.effects.values().join(", ")
),
));
}
Ok(())
}
fn same_file(root: &Path, rel: &str, entry: &Path) -> bool {
let abs = root.join(rel);
match (abs.canonicalize(), entry.canonicalize()) {
(Ok(left), Ok(right)) => left == right,
_ => false,
}
}
fn find_function_at_line(functions: &[Function], line: u32) -> Option<&Function> {
for fun in functions {
if fun.span.line == line {
return Some(fun);
}
if let Some(found) = find_function_at_line(&fun.children, line) {
return Some(found);
}
}
None
}
fn parse_literal(raw: &str) -> Result<Value, String> {
match raw {
"True" | "true" => return Ok(Value::Bool(true)),
"False" | "false" => return Ok(Value::Bool(false)),
"None" | "none" => return Ok(Value::None),
_ => {}
}
if raw.len() >= 2 {
let bytes = raw.as_bytes();
if (bytes[0] == b'"' && bytes[raw.len() - 1] == b'"')
|| (bytes[0] == b'\'' && bytes[raw.len() - 1] == b'\'')
{
return Ok(Value::Text(raw[1..raw.len() - 1].to_string()));
}
}
if let Ok(n) = raw.parse::<i64>() {
return Ok(Value::Int(n));
}
if raw.contains('.') {
if let Ok(n) = raw.parse::<f64>() {
return Ok(Value::Num(n));
}
}
Err(format!(
"无法解析 `{raw}`(整数、小数、引号文本、True/False/None)"
))
}
fn preserve_diag(path: &Path, err: anyhow::Error) -> anyhow::Error {
if Diagnostic::find(&err).is_some() {
err
} else {
call_diag(Some(path), 1, "call.compile", format!("{err:#}"))
}
}
fn call_diag(path: Option<&Path>, line: u32, code: &str, message: String) -> anyhow::Error {
anyhow::Error::new(Diagnostic::new(path, Span::new(line, 1), message).with_code(code))
}
+235
View File
@@ -0,0 +1,235 @@
use crate::host::{foreign_session as session, json::value_to_json, oracle, HostContext};
use crate::value::Value;
use serde_json::{json, Value as Json};
use sha2::{Digest, Sha256};
use std::path::PathBuf;
fn root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
}
fn directory() -> PathBuf {
let p = std::env::var_os("MARQDO_CAUSAL_DIR")
.map(PathBuf::from)
.unwrap_or_else(|| root().join("target/causal-replay"));
std::fs::create_dir_all(&p).unwrap();
p
}
fn steps(protocol: &[session::ProtocolStep]) -> Vec<Json> {
protocol
.iter()
.map(|p| json!({"session":p.session,"request":p.request,"response":p.response}))
.collect()
}
pub(crate) fn evidence(ctx: &mut HostContext) -> Json {
let journal = value_to_json(&Value::List(ctx.oracle_observations.clone())).unwrap();
let protocol = steps(&ctx.foreign_protocol_steps);
let divergence = ctx.foreign_replay_divergence.clone();
let live = ctx.foreign_sessions.len();
// Inspection is explicitly outside replay and outside the admitted journal.
ctx.foreign_replay_expected = None;
ctx.foreign_replay_divergence = None;
let n = if live == 1 {
session::get(ctx, &Value::Handle("s1".into()), &Value::Text("n".into()))
.ok()
.map(|v| value_to_json(&v).unwrap())
} else {
None
};
json!({"journal":journal,"protocol":protocol,"divergence":divergence,"live":live,"n":n})
}
#[test]
fn observation_journal_erases_writes_but_a_context_distinguishes_them() {
for lang in ["python", "node"] {
let out = directory();
let path = out.join(format!("journal-only-{lang}.marker"));
let literal = serde_json::to_string(&path.display().to_string()).unwrap();
let code = if lang == "python" {
format!("from pathlib import Path\nPath({literal}).write_text(str(ticket))\nNone")
} else {
format!("require('fs').writeFileSync({literal},String(ticket)); null;")
};
let mut journals = Vec::new();
let mut histories = Vec::new();
let mut contents = Vec::new();
let mut reads = Vec::new();
let mut first_history = Vec::new();
for ticket in [7, 8] {
let mut ctx = HostContext::default();
let id =
session::open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
session::set(
&mut ctx,
&id,
&Value::Text("ticket".into()),
&Value::Int(ticket),
)
.unwrap();
let reply = session::eval(
&mut ctx,
&id,
None,
Some(&Value::Text(code.clone())),
None,
None,
)
.unwrap();
let observed = oracle::eval_observation(&reply).unwrap();
oracle::append(&mut ctx, "eval", &observed).unwrap();
journals.push(Value::List(ctx.oracle_observations.clone()));
histories.push(steps(&ctx.foreign_protocol_steps));
if ticket == 7 {
first_history = ctx.foreign_protocol_steps.clone();
}
contents.push(std::fs::read_to_string(&path).unwrap());
reads.push(session::get(&mut ctx, &id, &Value::Text("ticket".into())).unwrap());
session::close(&mut ctx, &id).unwrap();
}
let checked = oracle::replay_check(&journals[0], &journals[1]).unwrap();
assert_eq!(journals[0], journals[1]);
assert_eq!(contents, ["7", "8"]);
assert_ne!(reads[0], reads[1]);
assert_eq!(value_to_json(&checked).unwrap()["ok"], true);
// Control requests use a different wrapper than eval. A mismatching
// setter must not retire the live kernel or overwrite its prior value.
let mut ctx = HostContext::default();
let id =
session::open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
session::set(
&mut ctx,
&id,
&Value::Text("ticket".into()),
&Value::Int(99),
)
.unwrap();
ctx.foreign_protocol_steps.clear();
ctx.foreign_replay_expected = Some(first_history.into());
let error =
session::set(&mut ctx, &id, &Value::Text("ticket".into()), &Value::Int(8)).unwrap_err();
let dispatched = ctx.foreign_protocol_steps.len();
let live = ctx.foreign_sessions.len();
ctx.foreign_replay_expected = None;
ctx.foreign_replay_divergence = None;
let retained = session::get(&mut ctx, &id, &Value::Text("ticket".into())).unwrap();
assert_eq!(dispatched, 0);
assert_eq!(live, 1);
assert_eq!(retained, Value::Int(99));
session::close(&mut ctx, &id).unwrap();
std::fs::write(out.join(format!("journal-only-{lang}.json")),serde_json::to_vec_pretty(&json!({"kind":"journal-erasure","language":lang,"journals":journals.iter().map(|j|value_to_json(j).unwrap()).collect::<Vec<_>>(),"histories":histories,"contents":contents,"reads":reads.iter().map(|v|value_to_json(v).unwrap()).collect::<Vec<_>>(),"journal_check":value_to_json(&checked).unwrap(),"setter_guard":{"error":error,"dispatched":dispatched,"live":live,"retained":value_to_json(&retained).unwrap()}})).unwrap()).unwrap();
}
}
#[test]
fn first_protocol_divergence_stops_host_suffix_and_preserves_current_effects() {
let lengths = std::env::var("MARQDO_CAUSAL_PREFIXES")
.unwrap_or("0,4".into())
.split(',')
.map(|s| s.parse::<usize>().unwrap())
.collect::<Vec<_>>();
let seeds = std::env::var("MARQDO_CAUSAL_SEEDS")
.unwrap_or("3".into())
.split(',')
.map(|s| s.parse::<i64>().unwrap())
.collect::<Vec<_>>();
let mut count = 0;
for lang in ["python", "node"] {
for &length in &lengths {
for &seed in &seeds {
for backend in ["tree", "bytecode"] {
for mode in ["reply", "request"] {
let out = directory();
let id = format!("{lang}-{length}-{seed}-{backend}-{mode}");
let path = out.join(format!("{id}.mq.md"));
let input = out.join(format!("{id}.input"));
let marker = out.join(format!("{id}.suffix"));
let q = |s: &str| serde_json::to_string(s).unwrap();
let code = if lang == "python" {
format!(
"from pathlib import Path\nn = 99\nint(Path({}).read_text())",
q(&input.display().to_string())
)
} else {
format!(
"n = 99; Number(require('fs').readFileSync({},'utf8'));",
q(&input.display().to_string())
)
};
let prefix=(0..length).map(|i|format!("**p{i} = > foreign.session_eval id=`s` source=\"n += 1; n\"**\n")).collect::<String>();
let source=format!("---\nmarkup: occlusion\nimport:\n foreign: {}\n fs: {}\n---\n# main\n**s = > foreign.session_open lang={}**\n**> foreign.session_set id=`s` name=\"n\" value={seed}**\n{prefix}**r = > foreign.session_eval id=`s` source={}**\n**> fs.write_text path={} text=\"suffix\"**\n*r*\n",root().join("lib/foreign.mq.md").display(),root().join("lib/fs.mq.md").display(),q(lang),q(&code),q(&marker.display().to_string()));
std::fs::write(&path, &source).unwrap();
std::fs::write(&input, "1").unwrap();
let module = crate::load::load_module(&path).unwrap();
let expected = if backend == "tree" {
let mut tree =
crate::interp::Interpreter::with_capture(Some(&path), false);
tree.run_module(&module).unwrap();
tree.application_evidence().1
} else {
let program =
crate::bytecode::compile_module(Some(&path), &module).unwrap();
let mut vm = crate::bytecode::Vm::with_capture(Some(&path));
vm.run(&program).unwrap();
vm.application_evidence().1
};
assert_eq!(std::fs::read_to_string(&marker).unwrap(), "suffix");
std::fs::remove_file(&marker).unwrap();
let actual_source = if mode == "request" {
source.replace(&q(&code), &q(&code.replace("n = 99", "n = 77")))
} else {
std::fs::write(&input, "2").unwrap();
source.clone()
};
std::fs::write(&path, &actual_source).unwrap();
let actual = crate::load::load_module(&path).unwrap();
let mut host = HostContext::for_capture(Some(&path), Default::default());
host.foreign_replay_expected = Some(expected.clone().into());
let (error, observed) = if backend == "tree" {
let mut tree =
crate::interp::Interpreter::with_capture(Some(&path), false)
.with_host(host);
let error = tree.run_module(&actual).err().map(|e| e.to_string());
(error, tree.causal_replay_evidence())
} else {
let program =
crate::bytecode::compile_module(Some(&path), &actual).unwrap();
let mut vm =
crate::bytecode::Vm::with_capture(Some(&path)).with_host(host);
let error = vm.run(&program).err().map(|e| e.to_string());
(error, vm.causal_replay_evidence())
};
let record = json!({"kind":"causal-stop","id":id,"language":lang,"prefix":length,"seed":seed,"backend":backend,"mode":mode,"expected":steps(&expected),"observed":observed,"error":error,"suffix_exists":marker.exists(),"source":actual_source,"source_sha256":format!("{:x}",Sha256::digest(actual_source.as_bytes()))});
std::fs::write(
out.join(format!("{id}.json")),
serde_json::to_vec_pretty(&record).unwrap(),
)
.unwrap();
assert!(
error.is_some(),
"{id}: replay divergence returned a soft result: {record}"
);
assert!(!marker.exists(), "{id}: host suffix ran after divergence");
assert_eq!(
observed["n"],
if mode == "reply" {
json!(99)
} else {
json!(seed + length as i64)
}
);
assert_eq!(observed["live"], 1);
assert_eq!(observed["journal"].as_array().unwrap().len(), length);
assert_eq!(
observed["protocol"].as_array().unwrap().len(),
length + if mode == "reply" { 2 } else { 1 }
);
count += 1;
}
}
}
}
}
println!(
"CAUSAL_METRIC:{}",
json!({"cases":count,"document_runs":count*2})
);
}
+21 -4
View File
@@ -109,6 +109,7 @@ pub fn type_of(v: &Value) -> &'static str {
Value::Num(_) => "num",
Value::Text(_) => "text",
Value::Secret(_) => "secret",
Value::Handle(_) => "handle",
Value::List(_) => "list",
Value::Map(_) => "map",
Value::Formula(_) => "formula",
@@ -127,20 +128,20 @@ fn instance_type_name(v: &Value) -> Option<String> {
})
}
#[derive(Debug, Clone)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ParamContract {
pub name: String,
pub ty: Ty,
pub desc: String,
}
#[derive(Debug, Clone)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ReturnContract {
pub ty: Ty,
pub desc: String,
}
#[derive(Debug, Clone)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct FieldContract {
pub name: String,
pub ty: Ty,
@@ -149,7 +150,7 @@ pub struct FieldContract {
}
/// 一份契约(一个契约表块;同单元多表由 [`merge_contract`] 合并)。
#[derive(Debug, Clone)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Contract {
pub params: Vec<ParamContract>,
pub returns: Option<ReturnContract>,
@@ -504,6 +505,22 @@ pub fn call_arg_diag(
None
}
/// Argument blame belongs to the caller; contract evidence belongs to the
/// declaration. Import aliases must not relocate a contract into the caller.
pub fn call_arg_diag_in(
c: &Contract, unit: &str, bound: &HashMap<String, Value>,
call_site: &crate::diagnostics::SourceLocation,
declaration: &crate::diagnostics::SourceIdentity,
) -> Option<Diagnostic> {
let mut diag=call_arg_diag(c,unit,bound,call_site.source.path.as_deref(),call_site.span)?
.with_source(&call_site.source);
let old=c.anchor(call_site.source.path.as_deref());
let new=c.anchor(declaration.path.as_deref());
diag.doc_anchor=Some(new.clone());
diag.suggestion=diag.suggestion.map(|text|text.replace(&old,&new));
Some(diag)
}
/// 边界 2:返回值 vs 返回契约。
pub fn return_diag(
c: &Contract,
+86 -1
View File
@@ -23,6 +23,58 @@ impl fmt::Display for Span {
}
}
/// Identity of the exact text that was parsed, including recorded revisions.
/// A path is a label; the digest is never recomputed from a later disk version.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct SourceIdentity {
pub path: Option<PathBuf>,
pub sha256: Option<String>,
}
impl SourceIdentity {
pub fn from_source(path: Option<&Path>, text: &str) -> Self {
use sha2::{Digest, Sha256};
Self {
path: path.map(Path::to_path_buf),
sha256: Some(format!("{:x}", Sha256::digest(text.as_bytes()))),
}
}
pub fn at(&self, span: Span) -> SourceLocation {
SourceLocation {
source: self.clone(),
span,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SourceLocation {
pub source: SourceIdentity,
pub span: Span,
}
impl SourceLocation {
pub fn to_json(&self) -> serde_json::Value {
serde_json::json!({"file": self.source.path.as_ref().map(|p| display_path(p)),
"source_sha256": self.source.sha256, "line": self.span.line, "col": self.span.col})
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DiagnosticFrame {
pub function: String,
pub declaration: SourceLocation,
pub call_site: Option<SourceLocation>,
}
impl DiagnosticFrame {
pub fn to_json(&self) -> serde_json::Value {
serde_json::json!({"function": self.function, "declaration": self.declaration.to_json(),
"call_site": self.call_site.as_ref().map(SourceLocation::to_json)})
}
}
/// Format a path for user-facing diagnostics.
///
/// Strips the Windows extended-length prefix (`\\?\` / `\\?\UNC\`) that
@@ -75,6 +127,9 @@ pub struct Diagnostic {
pub doc_anchor: Option<String>,
pub doc_quote: Option<String>,
pub contract_ref: Option<ContractRef>,
pub source_sha256: Option<String>,
/// Ordered outermost to innermost; every site retains its own source.
pub frames: Vec<DiagnosticFrame>,
}
impl Diagnostic {
@@ -89,6 +144,8 @@ impl Diagnostic {
doc_anchor: None,
doc_quote: None,
contract_ref: None,
source_sha256: None,
frames: Vec::new(),
}
}
@@ -101,6 +158,17 @@ impl Diagnostic {
self
}
pub fn with_source(mut self, source: &SourceIdentity) -> Self {
self.path = source.path.clone();
self.source_sha256 = source.sha256.clone();
self
}
pub fn with_frames(mut self, frames: &[DiagnosticFrame]) -> Self {
self.frames = frames.to_vec();
self
}
pub fn with_severity(mut self, severity: Severity) -> Self {
self.severity = severity;
self
@@ -132,10 +200,25 @@ impl Diagnostic {
}
pub fn format_message(&self) -> String {
match &self.path {
let mut message = match &self.path {
Some(p) => format!("{}:{}: {}", display_path(p), self.span, self.message),
None => format!("{}: {}", self.span, self.message),
};
for frame in self.frames.iter().rev() {
if let Some(site) = &frame.call_site {
let file = site
.source
.path
.as_ref()
.map(|p| display_path(p))
.unwrap_or_else(|| "<unknown>".into());
message.push_str(&format!(
"\n called {} from {}:{}",
frame.function, file, site.span
));
}
}
message
}
/// Machine-readable form(AI/MLSP 面;`run --json` 出口)。
@@ -153,6 +236,8 @@ impl Diagnostic {
"line": self.span.line,
"col": self.span.col,
},
"source_sha256": self.source_sha256,
"frames": self.frames.iter().map(DiagnosticFrame::to_json).collect::<Vec<_>>(),
"doc_anchor": self.doc_anchor,
"doc_quote": self.doc_quote,
"contract_ref": self.contract_ref.as_ref().map(|c| json!({
File diff suppressed because it is too large. Load diff
+25 -6
View File
@@ -220,6 +220,8 @@ pub enum HostFn {
ForeignSessionInterrupt = 211,
/// `<TAB>` M7: `access(集合, 键)` — polymorphic collection access (legacy index).
Access = 212,
OracleJournal = 213,
OracleReplayCheck = 214,
}
/// Single source of truth for every host function. `<TAB>` slimming
@@ -438,6 +440,8 @@ pub const HOST_FN_TABLE: &[HostFnDef] = &[
HostFnDef { id: 210, f: HostFn::ForeignSessionList, name: "host_foreign_session_list", required: &[], optional: &[] },
HostFnDef { id: 211, f: HostFn::ForeignSessionInterrupt, name: "host_foreign_session_interrupt", required: &["id"], optional: &[] },
HostFnDef { id: 212, f: HostFn::Access, name: "host_access", required: &["value", "key"], optional: &[] },
HostFnDef { id: 213, f: HostFn::OracleJournal, name: "host_oracle_journal", required: &[], optional: &[] },
HostFnDef { id: 214, f: HostFn::OracleReplayCheck, name: "host_oracle_replay_check", required: &["expected", "observed"], optional: &[] },
];
@@ -466,6 +470,8 @@ impl HostFn {
"host_cwd" => Self::Cwd,
"host_exit" => Self::Exit,
"host_exec" => Self::Exec,
"host_oracle_journal" => Self::OracleJournal,
"host_oracle_replay_check" => Self::OracleReplayCheck,
"host_json_parse" => Self::JsonParse,
"host_json_stringify" => Self::JsonStringify,
"host_json_get" => Self::JsonGet,
@@ -748,7 +754,13 @@ pub fn call_host(
bound.get("args"),
bound.get("capture"),
),
HostFn::JsonParse => json::parse(require(bound, "text")?),
HostFn::OracleJournal => Ok(Value::List(ctx.oracle_observations.clone())),
HostFn::OracleReplayCheck => super::oracle::replay_check(require(bound, "expected")?, require(bound, "observed")?),
HostFn::JsonParse => {
let value = json::parse(require(bound, "text")?)?;
super::oracle::append(ctx, "decode", &value)?;
Ok(value)
},
HostFn::JsonStringify => {
json::stringify(require(bound, "value")?, bound.get("indent"))
}
@@ -1127,7 +1139,7 @@ pub fn call_host(
HostFn::ForeignSessionEval => {
#[cfg(feature = "exec-host")]
{
crate::host::foreign_session::eval_with_stdin(
let record = crate::host::foreign_session::eval_with_stdin(
ctx,
require(bound, "id")?,
bound.get("code"),
@@ -1135,7 +1147,10 @@ pub fn call_host(
bound.get("input"),
bound.get("stdin").and_then(|v| v.reveal_text()),
bound.get("timeout"),
)
)?;
let value = super::oracle::eval_observation(&record)?;
super::oracle::append(ctx, "eval", &value)?;
Ok(record)
}
#[cfg(not(feature = "exec-host"))]
{
@@ -1162,11 +1177,13 @@ pub fn call_host(
HostFn::ForeignSessionGet => {
#[cfg(feature = "exec-host")]
{
crate::host::foreign_session::get(
let value = crate::host::foreign_session::get(
ctx,
require(bound, "id")?,
require(bound, "name")?,
)
)?;
super::oracle::append(ctx, "get", &value)?;
Ok(value)
}
#[cfg(not(feature = "exec-host"))]
{
@@ -1177,7 +1194,9 @@ pub fn call_host(
HostFn::ForeignSessionNames => {
#[cfg(feature = "exec-host")]
{
crate::host::foreign_session::names(ctx, require(bound, "id")?)
let value = crate::host::foreign_session::names(ctx, require(bound, "id")?)?;
super::oracle::append(ctx, "decode", &value)?;
Ok(value)
}
#[cfg(not(feature = "exec-host"))]
{
+54 -55
View File
@@ -15,7 +15,7 @@ use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use crate::foreign::registry::{self, LangSpec};
use crate::foreign::split_cmd_line;
use crate::host::json::{json_to_value, value_to_json};
use crate::host::json::value_to_json;
use crate::host::{HostCaps, HostContext};
use crate::value::{CodeBlock, Value};
@@ -158,6 +158,7 @@ pub(crate) struct Execution {
pub lang: String,
pub cmd: String,
pub script: String,
pub diagnostic_source: String,
}
impl Execution {
@@ -180,6 +181,7 @@ impl Execution {
("result_error".into(), Value::Text(result_error)),
("error".into(), Value::Text(self.error.clone())),
("script".into(), Value::Text(self.script.clone())),
("diagnostic_source".into(), Value::Text(self.diagnostic_source.clone())),
("cached".into(), Value::Bool(false)),
])
}
@@ -588,10 +590,11 @@ fn prepare_invoke(args: &InvokeArgs<'_>) -> Result<PreparedInvoke, String> {
let stdin = opt_text(args.stdin)?;
let input_json = match args.input {
None | Some(Value::None) => None,
Some(v) => Some(
serde_json::to_string(&value_to_json(v)?)
.map_err(|e| format!("input JSON encode: {e}"))?,
),
Some(v) => {
super::oracle::admit(v)?;
Some(serde_json::to_string(&value_to_json(v)?)
.map_err(|e| format!("input JSON encode: {e}"))?)
},
};
if stdin.is_some() && input_json.is_some() {
return Err("stdin and input are mutually exclusive".into());
@@ -620,7 +623,7 @@ pub fn invoke(ctx: &mut HostContext, args: InvokeArgs<'_>) -> Result<Value, Stri
ensure_config(ctx);
let prepared = prepare_invoke(&args)?;
let exec = execute_job(ctx, &prepared.job)?;
let (result, result_error) = parse_output(&exec.stdout, &prepared.output);
let (result, result_error) = parse_output(&exec.stdout, &prepared.output)?;
Ok(exec.record(result, result_error))
}
@@ -636,13 +639,10 @@ pub fn call(ctx: &mut HostContext, args: InvokeArgs<'_>) -> Result<Value, String
if exec.stdout.trim().is_empty() {
return Ok(Value::None);
}
match serde_json::from_str::<serde_json::Value>(&exec.stdout) {
Ok(j) => json_to_value(&j),
Err(e) => Err(format!(
"foreign call: stdout is not JSON: {e}; stdout: {}",
tail(exec.stdout.trim(), 512)
)),
}
super::json::decode_text(&exec.stdout).map_err(|e| match e {
super::json::DecodeError::Domain(e) => e,
super::json::DecodeError::Syntax(e) => format!("foreign call: stdout is not JSON: {e}; stdout: {}", tail(exec.stdout.trim(), 512)),
})
}
fn present(v: Option<&Value>) -> Option<&Value> {
@@ -752,17 +752,18 @@ fn error_record(ctx: &HostContext, job: &Job, error: String) -> Value {
lang: job.lang.clone(),
cmd: display_for_job(ctx, job, None),
script: String::new(),
diagnostic_source: String::new(),
}
.record(Value::None, String::new())
}
fn run_prepared(env: &ForeignEnv, items: Vec<PreparedMany>, concurrency: usize) -> Vec<(usize, Value)> {
fn run_prepared(env: &ForeignEnv, items: Vec<PreparedMany>, concurrency: usize) -> Result<Vec<(usize, Value)>, String> {
if items.is_empty() {
return Vec::new();
return Ok(Vec::new());
}
let workers = concurrency.max(1).min(items.len());
let queue = Arc::new(Mutex::new(VecDeque::from(items)));
let results: Arc<Mutex<Vec<(usize, Value)>>> = Arc::new(Mutex::new(Vec::new()));
let results: Arc<Mutex<Vec<(usize, Result<Value, String>)>>> = Arc::new(Mutex::new(Vec::new()));
thread::scope(|scope| {
for _ in 0..workers {
let queue = Arc::clone(&queue);
@@ -782,11 +783,10 @@ fn run_prepared(env: &ForeignEnv, items: Vec<PreparedMany>, concurrency: usize)
execute_job(&ctx, &item.job)
})) {
Ok(Ok(exec)) => {
let (result, result_error) = parse_output(&exec.stdout, &item.output);
exec.record(result, result_error)
parse_output(&exec.stdout, &item.output).map(|(result, error)| exec.record(result, error))
}
Ok(Err(e)) => error_record(&ctx, &item.job, e),
Err(_) => error_record(&ctx, &item.job, "foreign worker panicked".into()),
Ok(Err(e)) => Ok(error_record(&ctx, &item.job, e)),
Err(_) => Ok(error_record(&ctx, &item.job, "foreign worker panicked".into())),
};
if let Ok(mut out) = results.lock() {
out.push((item.idx, value));
@@ -799,7 +799,7 @@ fn run_prepared(env: &ForeignEnv, items: Vec<PreparedMany>, concurrency: usize)
Err(shared) => shared.lock().map(|m| m.clone()).unwrap_or_default(),
};
out.sort_by_key(|(i, _)| *i);
out
out.into_iter().map(|(i, result)| result.map(|value| (i, value)).map_err(|e| format!("foreign[{i}]: {e}"))).collect()
}
/// Build a prepared job for one `runs` item (code value or option map).
@@ -851,7 +851,7 @@ pub fn run_many(
requested
};
let env = ForeignEnv::from_ctx(ctx);
let results = run_prepared(&env, prepared, concurrency);
let results = run_prepared(&env, prepared, concurrency)?;
Ok(Value::List(results.into_iter().map(|(_, v)| v).collect()))
}
@@ -879,6 +879,7 @@ pub fn call_many(
};
let mut prepared = Vec::with_capacity(n);
for (idx, input) in list.iter().enumerate() {
super::oracle::admit(input)?;
let json = serde_json::to_string(&value_to_json(input)?)
.map_err(|e| format!("call_many input JSON encode: {e}"))?;
let mut job = base.job.clone();
@@ -891,7 +892,7 @@ pub fn call_many(
});
}
let env = ForeignEnv::from_ctx(ctx);
let records = run_prepared(&env, prepared, concurrency);
let records = run_prepared(&env, prepared, concurrency)?;
let mut results = Vec::with_capacity(records.len());
let mut errors = Vec::new();
for (idx, record) in records {
@@ -936,34 +937,15 @@ fn resolve_invoke_source(args: &InvokeArgs<'_>) -> Result<(String, String), Stri
))
}
fn parse_output(stdout: &str, mode: &str) -> (Value, String) {
match mode {
"text" => (Value::None, String::new()),
"lines" => (
Value::List(stdout.lines().map(|l| Value::Text(l.to_string())).collect()),
String::new(),
),
"json" => {
if stdout.trim().is_empty() {
return (Value::None, String::new());
}
match serde_json::from_str::<serde_json::Value>(stdout) {
Ok(j) => match json_to_value(&j) {
Ok(v) => (v, String::new()),
Err(e) => (Value::None, e),
},
Err(e) => (Value::None, format!("stdout is not JSON: {e}")),
}
}
_ => {
if stdout.trim().is_empty() {
return (Value::None, String::new());
}
match serde_json::from_str::<serde_json::Value>(stdout) {
Ok(j) => (json_to_value(&j).unwrap_or(Value::None), String::new()),
Err(_) => (Value::None, String::new()),
}
}
fn parse_output(stdout: &str, mode: &str) -> Result<(Value, String), String> {
if mode == "text" { return Ok((Value::None, String::new())); }
if mode == "lines" { return Ok((Value::List(stdout.lines().map(|s| Value::Text(s.into())).collect()), String::new())); }
if stdout.trim().is_empty() { return Ok((Value::None, String::new())); }
match super::json::decode_text(stdout) {
Ok(v) => Ok((v, String::new())),
Err(super::json::DecodeError::Domain(e)) => Err(e),
Err(super::json::DecodeError::Syntax(e)) if mode == "json" => Ok((Value::None, format!("stdout is not JSON: {e}"))),
Err(super::json::DecodeError::Syntax(_)) => Ok((Value::None, String::new())),
}
}
@@ -1734,6 +1716,7 @@ fn execute_job(ctx: &HostContext, job: &Job) -> Result<Execution, String> {
lang: job.lang.clone(),
cmd: display_for_job(ctx, job, None),
script: String::new(),
diagnostic_source: String::new(),
});
}
};
@@ -1803,6 +1786,7 @@ fn execute_job(ctx: &HostContext, job: &Job) -> Result<Execution, String> {
lang: job.lang.clone(),
cmd,
script: script_keep,
diagnostic_source: script_path.display().to_string(),
})
}
@@ -1837,7 +1821,7 @@ pub fn invoke_for_view(
cmd_override: cmd_override.map(str::to_string),
};
let exec = execute_job(&ctx, &job)?;
let (result, result_error) = parse_output(&exec.stdout, "auto");
let (result, result_error) = parse_output(&exec.stdout, "auto")?;
Ok(exec.record(result, result_error))
}
@@ -1886,13 +1870,13 @@ mod tests {
#[test]
fn output_modes() {
let (v, e) = parse_output("42", "auto");
let (v, e) = parse_output("42", "auto").unwrap();
assert!(e.is_empty());
assert_eq!(v, Value::Int(42));
let (v, e) = parse_output("a\nb\n", "lines");
let (v, e) = parse_output("a\nb\n", "lines").unwrap();
assert!(e.is_empty());
assert_eq!(v, Value::List(vec![Value::Text("a".into()), Value::Text("b".into())]));
let (v, e) = parse_output("not json", "json");
let (v, e) = parse_output("not json", "json").unwrap();
assert!(v == Value::None);
assert!(!e.is_empty());
}
@@ -1965,4 +1949,19 @@ mod tests {
assert_eq!(map_get(&records[1], "ok"), Some(&Value::Bool(false)));
assert_eq!(map_get(&records[2], "stdout").and_then(|v| v.reveal_text()), Some("c\n"));
}
#[test]
fn structured_output_domain_errors_are_not_none_or_success_records() {
for mode in ["auto", "json"] {
for bad in ["9007199254740992", "9.007199254740992e15", "1e9999", r#"{"x":1,"x":2}"#, r#"{"a":[],"b":[1]}"#] {
assert!(parse_output(bad, mode).is_err(), "{mode}: {bad}");
}
}
let ctx = HostContext::default();
let env = ForeignEnv::from_ctx(&ctx);
let items = (0..2).map(|idx| PreparedMany {
idx, job: Job::simple("python", "print('{\"x\":1,\"x\":2}')", None), output: "auto".into(),
}).collect();
assert!(run_prepared(&env, items, 2).unwrap_err().contains("duplicate"));
}
}
+583 -100
View File
File diff suppressed because it is too large. Load diff
+174
View File
@@ -0,0 +1,174 @@
//! Concurrent control inputs are scoped to a single recorded document action.
//! Replay supplies their elapsed-time inputs, reobserves acceptance, and stops
//! on divergence; elapsed wall time is not a deterministic foreign-state oracle.
use crate::host::foreign_session::InterruptHandle;
use crate::value::Value;
use std::collections::BTreeMap;
use std::sync::{Arc, Condvar, Mutex};
use std::thread::JoinHandle;
use std::time::{Duration, Instant};
#[derive(Clone, Debug, PartialEq)]
pub(super) struct ControlEvent {
pub after_us: u64,
pub results: Vec<(String, Result<bool, String>)>,
}
impl ControlEvent {
pub fn value(&self) -> Value {
Value::Map(vec![
("after_us".into(), Value::Int(self.after_us as i64)),
(
"results".into(),
Value::List(
self.results
.iter()
.map(|(name, result)| {
Value::Map(vec![
("kernel".into(), Value::Text(name.clone())),
("accepted".into(), Value::Bool(result.is_ok())),
("active".into(), Value::Bool(matches!(result, Ok(true)))),
(
"error".into(),
Value::Text(result.as_ref().err().cloned().unwrap_or_default()),
),
])
})
.collect(),
),
),
])
}
}
#[derive(Default, Debug)]
struct State {
kernels: BTreeMap<String, InterruptHandle>,
generation: u64,
action: Option<(Instant, Vec<ControlEvent>)>,
}
#[derive(Default, Debug)]
struct Shared {
state: Mutex<State>,
changed: Condvar,
}
#[derive(Clone, Default, Debug)]
pub struct DocumentControl(Arc<Shared>);
impl DocumentControl {
pub(super) fn cancelled(&self) -> bool {
self.0
.state
.lock()
.unwrap()
.action
.as_ref()
.is_some_and(|(_, events)| !events.is_empty())
}
pub(super) fn register(&self, kernel: &str, handle: InterruptHandle) {
self.0
.state
.lock()
.unwrap()
.kernels
.insert(kernel.into(), handle);
}
pub(super) fn begin(&self, plan: Vec<ControlEvent>) -> Option<JoinHandle<()>> {
let generation = {
let mut state = self.0.state.lock().unwrap();
state.generation += 1;
state.action = Some((Instant::now(), Vec::new()));
state.generation
};
if plan.is_empty() {
return None;
}
let controller = self.clone();
Some(std::thread::spawn(move || {
for event in plan {
let mut state = controller.0.state.lock().unwrap();
loop {
if state.generation != generation || state.action.is_none() {
return;
}
let elapsed = state.action.as_ref().unwrap().0.elapsed();
let remaining = Duration::from_micros(event.after_us).saturating_sub(elapsed);
if remaining.is_zero() {
break;
}
state = controller
.0
.changed
.wait_timeout(state, remaining.min(Duration::from_millis(200)))
.unwrap()
.0;
}
// Keep the action-generation check and signal under one lock.
let _ = Self::request_locked(&mut state, Some(event.after_us));
}
}))
}
pub(super) fn finish(&self, scheduler: Option<JoinHandle<()>>) -> Vec<ControlEvent> {
let events = self
.0
.state
.lock()
.unwrap()
.action
.take()
.map(|(_, events)| events)
.unwrap_or_default();
self.0.changed.notify_all();
if let Some(scheduler) = scheduler {
let _ = scheduler.join();
}
events
}
pub fn interrupt(&self) -> Result<Value, String> {
let mut state = self
.0
.state
.lock()
.map_err(|_| "document control lock poisoned")?;
Self::request_locked(&mut state, None).map(|event| event.value())
}
fn request_locked(
state: &mut State,
scheduled_us: Option<u64>,
) -> Result<ControlEvent, String> {
let start = state
.action
.as_ref()
.ok_or("no document action in flight")?
.0;
let after_us = scheduled_us
.unwrap_or_else(|| start.elapsed().as_micros().min(i64::MAX as u128) as u64);
let results = state
.kernels
.iter()
.filter(|(_, handle)| handle.is_live())
.map(|(name, handle)| (name.clone(), handle.request()))
.collect();
let event = ControlEvent { after_us, results };
state.action.as_mut().unwrap().1.push(event.clone());
Ok(event)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn completed_action_cancels_pending_scheduler_before_next_generation() {
let controller = DocumentControl::default();
let scheduler = controller.begin(vec![ControlEvent {
after_us: 60_000_000,
results: Vec::new(),
}]);
let start = Instant::now();
assert!(controller.finish(scheduler).is_empty());
assert!(start.elapsed() < Duration::from_secs(1));
assert!(controller.interrupt().is_err());
let next = controller.begin(Vec::new());
assert!(!controller.cancelled());
assert!(controller.finish(next).is_empty());
}
}
+399
View File
@@ -0,0 +1,399 @@
//! Lossless, typed, bottom-up sharing for complete history snapshots.
//! References always point backwards. Expansion costs and depth are checked
//! before building the expanded value, so a small DAG cannot expand unboundedly.
use crate::host::{json::value_to_json, oracle::admit};
use crate::value::Value;
use sha2::{Digest, Sha256};
use std::collections::{HashMap, HashSet};
const FORMAT: &str = "marqdo.history.dag";
const MAX_NODES: usize = 1_000_000;
const MAX_VALUES: usize = 8_000_000;
const MAX_BYTES: usize = 512 * 1024 * 1024;
const MAX_DEPTH: usize = 128;
fn text(s: &str) -> Value {
Value::Text(s.into())
}
fn list(v: &Value) -> Result<&[Value], String> {
if let Value::List(v) = v {
Ok(v)
} else {
Err("snapshot DAG: expected list".into())
}
}
fn string(v: &Value) -> Result<&str, String> {
if let Value::Text(v) = v {
Ok(v)
} else {
Err("snapshot DAG: expected text".into())
}
}
fn field<'a>(v: &'a Value, key: &str) -> Result<&'a Value, String> {
if let Value::Map(fields) = v {
fields
.iter()
.find(|(k, _)| k == key)
.map(|(_, v)| v)
.ok_or_else(|| format!("snapshot DAG: missing {key}"))
} else {
Err("snapshot DAG: expected object".into())
}
}
fn wire(v: &Value) -> Result<String, String> {
serde_json::to_string(&value_to_json(v)?).map_err(|e| e.to_string())
}
fn checksum(root: &Value, nodes: &Value) -> Result<String, String> {
Ok(format!(
"{:x}",
Sha256::digest(wire(&Value::List(vec![root.clone(), nodes.clone()]))?.as_bytes())
))
}
pub fn is_packed(v: &Value) -> bool {
field(v, "format").ok() == Some(&text(FORMAT))
}
pub fn pack(snapshot: &Value) -> Result<Value, String> {
fn intern(
v: &Value,
nodes: &mut Vec<Value>,
seen: &mut HashMap<String, usize>,
depth: usize,
) -> Result<usize, String> {
if depth > MAX_DEPTH {
return Err("snapshot DAG: depth budget exceeded".into());
}
let row = match v {
Value::None => vec![text("null")],
Value::Bool(v) => vec![text("bool"), Value::Bool(*v)],
Value::Int(v) => vec![text("int"), text(&v.to_string())],
Value::Num(v) => vec![text("float"), text(&format!("{:016x}", v.to_bits()))],
Value::Text(v) => vec![text("text"), text(v)],
Value::List(values) => {
let refs = values
.iter()
.map(|v| intern(v, nodes, seen, depth + 1).map(|i| Value::Int(i as i64)))
.collect::<Result<Vec<_>, _>>()?;
vec![text("list"), Value::List(refs)]
}
Value::Map(fields) => {
let refs = fields
.iter()
.map(|(k, v)| {
intern(v, nodes, seen, depth + 1)
.map(|i| Value::List(vec![text(k), Value::Int(i as i64)]))
})
.collect::<Result<Vec<_>, _>>()?;
vec![text("map"), Value::List(refs)]
}
_ => return Err("snapshot DAG: value outside V_safe".into()),
};
let row = Value::List(row);
let key = wire(&row)?;
if let Some(index) = seen.get(&key) {
return Ok(*index);
}
if nodes.len() >= MAX_NODES {
return Err("snapshot DAG: node budget exceeded".into());
}
let index = nodes.len();
nodes.push(row);
seen.insert(key, index);
Ok(index)
}
let mut nodes = Vec::new();
let root = intern(snapshot, &mut nodes, &mut HashMap::new(), 0)?;
admit(snapshot)?;
let nodes = Value::List(nodes);
let root = Value::Int(root as i64);
let hash = checksum(&root, &nodes)?;
let envelope = Value::Map(vec![
("format".into(), text(FORMAT)),
("version".into(), Value::Int(1)),
("root".into(), root),
("nodes".into(), nodes),
("sha256".into(), text(&hash)),
]);
// Apply exactly the decoder's resource limits to locally produced envelopes.
validate(&envelope)?;
Ok(envelope)
}
enum Node {
Atom(Value),
List(Vec<usize>),
Map(Vec<(String, usize)>),
}
impl Node {
fn refs(&self) -> Vec<usize> {
match self {
Self::Atom(_) => Vec::new(),
Self::List(v) => v.clone(),
Self::Map(v) => v.iter().map(|(_, i)| *i).collect(),
}
}
}
fn index(v: &Value, before: usize) -> Result<usize, String> {
match v {
Value::Int(i) if *i >= 0 && (*i as u64) < before as u64 => Ok(*i as usize),
_ => Err("snapshot DAG: forward/cyclic/out-of-range reference".into()),
}
}
fn validate(envelope: &Value) -> Result<(Vec<Node>, usize), String> {
let Value::Map(fields) = envelope else {
return Err("snapshot DAG: expected envelope".into());
};
let names: HashSet<_> = fields.iter().map(|(k, _)| k.as_str()).collect();
if fields.len() != 5
|| names != HashSet::from(["format", "version", "root", "nodes", "sha256"])
|| string(field(envelope, "format")?)? != FORMAT
|| field(envelope, "version")? != &Value::Int(1)
{
return Err("snapshot DAG: invalid envelope schema".into());
}
let rows = list(field(envelope, "nodes")?)?;
if rows.is_empty() || rows.len() > MAX_NODES {
return Err("snapshot DAG: node budget exceeded".into());
}
let root = index(field(envelope, "root")?, rows.len())?;
if root + 1 != rows.len() {
return Err("snapshot DAG: root must be last".into());
}
let expected = string(field(envelope, "sha256")?)?;
if expected != checksum(field(envelope, "root")?, field(envelope, "nodes")?)? {
return Err("snapshot DAG: checksum differs".into());
}
let mut nodes = Vec::new();
let mut costs: Vec<(usize, usize, usize)> = Vec::new();
let mut unique = HashSet::new();
for (i, raw) in rows.iter().enumerate() {
if !unique.insert(wire(raw)?) {
return Err("snapshot DAG: duplicate dictionary node".into());
}
let row = list(raw)?;
let tag = string(row.first().ok_or("snapshot DAG: empty node")?)?;
let node = match (tag, row) {
("null", [_]) => Node::Atom(Value::None),
("bool", [_, Value::Bool(v)]) => Node::Atom(Value::Bool(*v)),
("text", [_, Value::Text(v)]) => Node::Atom(text(v)),
("int", [_, Value::Text(v)]) => {
let n = v
.parse::<i64>()
.map_err(|_| "snapshot DAG: invalid integer")?;
if n.to_string() != *v {
return Err("snapshot DAG: noncanonical integer".into());
}
Node::Atom(Value::Int(n))
}
("float", [_, Value::Text(v)]) if v.len() == 16 => {
let bits =
u64::from_str_radix(v, 16).map_err(|_| "snapshot DAG: invalid float bits")?;
if format!("{bits:016x}") != *v {
return Err("snapshot DAG: noncanonical float bits".into());
}
Node::Atom(Value::Num(f64::from_bits(bits)))
}
("list", [_, refs]) => Node::List(
list(refs)?
.iter()
.map(|v| index(v, i))
.collect::<Result<_, _>>()?,
),
("map", [_, refs]) => {
let mut keys = HashSet::new();
let mut fields = Vec::new();
for pair in list(refs)? {
let [key, value] = list(pair)? else {
return Err("snapshot DAG: invalid map pair".into());
};
let key = string(key)?;
if !keys.insert(key) {
return Err("snapshot DAG: duplicate map key".into());
}
fields.push((key.into(), index(value, i)?));
}
Node::Map(fields)
}
_ => return Err("snapshot DAG: invalid node tag/payload".into()),
};
let mut values = 1usize;
let mut bytes = match &node {
Node::Atom(v) => {
admit(v)?;
match v {
Value::Text(s) => s
.len()
.checked_mul(6)
.and_then(|n| n.checked_add(2))
.ok_or("snapshot DAG: size overflow")?,
_ => 32,
}
}
Node::Map(v) => v
.iter()
.try_fold(2usize, |n, (k, _)| {
n.checked_add(k.len().checked_mul(6)?.checked_add(4)?)
})
.ok_or("snapshot DAG: size overflow")?,
Node::List(_) => 2,
};
let mut depth = 1;
for reference in node.refs() {
let (count, size, height) = costs[reference];
values = values
.checked_add(count)
.ok_or("snapshot DAG: expansion overflow")?;
bytes = bytes
.checked_add(size)
.and_then(|n| n.checked_add(1))
.ok_or("snapshot DAG: expansion overflow")?;
depth = depth.max(height + 1);
}
if values > MAX_VALUES || bytes > MAX_BYTES || depth > MAX_DEPTH {
return Err("snapshot DAG: expansion budget exceeded".into());
}
costs.push((values, bytes, depth));
nodes.push(node);
}
let mut reachable = HashSet::new();
let mut todo = vec![root];
while let Some(i) = todo.pop() {
if reachable.insert(i) {
todo.extend(nodes[i].refs());
}
}
if reachable.len() != nodes.len() {
return Err("snapshot DAG: unreachable dictionary node".into());
}
Ok((nodes, root))
}
pub fn unpack(envelope: &Value) -> Result<Value, String> {
fn expand(nodes: &[Node], i: usize) -> Value {
match &nodes[i] {
Node::Atom(v) => v.clone(),
Node::List(refs) => Value::List(refs.iter().map(|i| expand(nodes, *i)).collect()),
Node::Map(fields) => Value::Map(
fields
.iter()
.map(|(k, i)| (k.clone(), expand(nodes, *i)))
.collect(),
),
}
}
let (nodes, root) = validate(envelope)?;
let value = expand(&nodes, root);
admit(&value)?;
Ok(value)
}
#[cfg(test)]
mod tests {
use super::*;
fn envelope(rows: Vec<Value>) -> Value {
let root = Value::Int(rows.len() as i64 - 1);
let nodes = Value::List(rows);
let hash = checksum(&root, &nodes).unwrap();
Value::Map(vec![
("format".into(), text(FORMAT)),
("version".into(), Value::Int(1)),
("root".into(), root),
("nodes".into(), nodes),
("sha256".into(), text(&hash)),
])
}
fn row(tag: &str, value: Value) -> Value {
Value::List(vec![text(tag), value])
}
#[test]
fn typed_roundtrip_preserves_order_float_bits_and_repeated_values() {
let shared = Value::Map(vec![
(
"z".into(),
Value::List(vec![
Value::None,
Value::Bool(true),
Value::Int(1),
Value::Num(-0.0),
Value::Num(1.25),
]),
),
("汉字".into(), text("🧪")),
]);
let input = Value::List(vec![shared; 128]);
let packed = pack(&input).unwrap();
let decoded_wire =
crate::host::json::parse(&crate::host::json::stringify(&packed, None).unwrap())
.unwrap();
let expanded = unpack(&decoded_wire).unwrap();
assert_eq!(input, expanded);
assert_eq!(wire(&input).unwrap(), wire(&expanded).unwrap());
let Value::List(ref xs) = expanded else {
panic!()
};
let Value::Map(ref fields) = xs[0] else {
panic!()
};
assert_eq!(fields[0].0, "z");
let Value::List(ref values) = fields[0].1 else {
panic!()
};
let Value::Num(number) = values[3] else {
panic!()
};
assert_eq!(number.to_bits(), (-0.0f64).to_bits());
assert!(list(field(&packed, "nodes").unwrap()).unwrap().len() < 20);
assert!(wire(&packed).unwrap().len() < wire(&input).unwrap().len());
}
#[test]
fn refuses_cycles_amplification_unreachable_duplicates_and_unsafe_atoms() {
let bad = [
envelope(vec![row("list", Value::List(vec![Value::Int(0)]))]),
envelope(vec![row("text", text("unused")), row("text", text("root"))]),
envelope(vec![row("text", text("same")), row("text", text("same"))]),
envelope(vec![row("int", text("9007199254740992"))]),
envelope(vec![row("float", text("7ff8000000000000"))]),
envelope(vec![row("int", text("00"))]),
];
for value in bad {
assert!(unpack(&value).is_err());
}
let mut rows = vec![row("text", text("x"))];
for n in 0..40 {
rows.push(row("list", Value::List(vec![Value::Int(n), Value::Int(n)])));
}
let error = unpack(&envelope(rows)).unwrap_err();
assert!(error.contains("budget"), "{error}");
let mut hash = pack(&Value::Int(1)).unwrap();
if let Value::Map(ref mut fields) = hash {
fields.iter_mut().find(|(k, _)| k == "sha256").unwrap().1 = text("bad");
}
assert!(unpack(&hash).unwrap_err().contains("checksum"));
}
#[test]
fn sharing_does_not_admit_ragged_columns_or_duplicate_keys() {
let ragged = Value::Map(vec![
("a".into(), Value::List(vec![Value::Int(1)])),
("b".into(), Value::List(Vec::new())),
]);
assert!(pack(&ragged).is_err());
let duplicate = Value::Map(vec![
("a".into(), Value::Int(1)),
("a".into(), Value::Int(2)),
]);
assert!(pack(&duplicate).is_err());
let value = envelope(vec![
row("int", text("1")),
row("list", Value::List(vec![Value::Int(0)])),
row("list", Value::List(Vec::new())),
row(
"map",
Value::List(vec![
Value::List(vec![text("a"), Value::Int(1)]),
Value::List(vec![text("b"), Value::Int(2)]),
]),
),
]);
assert!(unpack(&value).is_err());
}
}
File diff suppressed because it is too large. Load diff
+533 -42
View File
@@ -16,15 +16,59 @@ use crate::host::json::{json_to_value, value_to_json};
use crate::host::HostContext;
use crate::value::Value;
mod control;
pub mod validation;
pub use control::InterruptHandle;
const SESSION_TIMEOUT_SECS: f64 = 10.0;
const MAX_SESSION_TIMEOUT_SECS: f64 = 3600.0;
const MAX_PROCESS_STDERR: usize = 1_048_576;
const PY_BOOTSTRAP: &str = r#"import sys, json, io, ast, traceback, contextlib
const PY_BOOTSTRAP: &str = r#"import sys, json, io, ast, traceback, contextlib, math, signal
_out = sys.stdout
_MAX_SAFE = 9007199254740991
def _sanitize(o):
if o is None or isinstance(o, bool):
return o
if isinstance(o, int):
if abs(o) > _MAX_SAFE:
raise ValueError("json: integer outside V_safe (±(2^53-1)): %s" % o)
return o
if isinstance(o, float):
if not math.isfinite(o):
raise ValueError("json: non-finite number outside V_safe")
if o.is_integer() and abs(o) > _MAX_SAFE:
raise ValueError("json: integer outside V_safe (±(2^53-1)): %s" % o)
return o
if isinstance(o, str):
return o
if isinstance(o, (list, tuple)):
return [_sanitize(x) for x in o]
if isinstance(o, dict):
if any(not isinstance(k, str) for k in o):
raise TypeError("not JSON-safe: object keys must be strings")
return {k: _sanitize(v) for k, v in o.items()}
raise TypeError("not JSON-safe: %s" % type(o).__name__)
def _send(o):
_out.write(json.dumps(o, ensure_ascii=False, default=str) + "\n")
try:
line = json.dumps(_sanitize(o), ensure_ascii=False)
except Exception as e:
rid = o.get("id") if isinstance(o, dict) else None
line = json.dumps({
"id": rid, "ok": False,
"stdout": (o.get("stdout") if isinstance(o, dict) else "") or "",
"stderr": (o.get("stderr") if isinstance(o, dict) else "") or "",
"result": None, "error": str(e), "domain_error": True,
}, ensure_ascii=False)
_out.write(line + "\n")
_out.flush()
_busy = False
def _interrupt(signum, frame):
global _busy
if _busy:
_busy = False
raise KeyboardInterrupt
signal.signal(signal.SIGINT, _interrupt)
_g = {"__name__": "__main__", "marqdo_input": None}
for _line in sys.stdin:
_line = _line.strip()
@@ -60,6 +104,8 @@ for _line in sys.stdin:
_out_buf = io.StringIO()
_err_buf = io.StringIO()
try:
_busy = True
_send({"id": rid, "phase": "ready"})
_tree = ast.parse(_code, "<marqdo>")
_result = None
with contextlib.redirect_stdout(_out_buf), contextlib.redirect_stderr(_err_buf):
@@ -70,10 +116,13 @@ for _line in sys.stdin:
_result = eval(compile(ast.Expression(_last.value), "<marqdo>", "eval"), _g)
else:
exec(compile(_tree, "<marqdo>", "exec"), _g)
_busy = False
_send({"id": rid, "ok": True, "stdout": _out_buf.getvalue(), "stderr": _err_buf.getvalue(), "result": _result, "error": ""})
except BaseException:
_busy = False
_send({"id": rid, "ok": False, "stdout": _out_buf.getvalue(), "stderr": _err_buf.getvalue(), "result": None, "error": traceback.format_exc(limit=8)})
finally:
_busy = False
sys.stdin = _old_stdin
"#;
@@ -81,10 +130,67 @@ const JS_BOOTSTRAP: &str = r#"const readline = require("readline");
const util = require("util");
const vm = require("vm");
const out = process.stdout.write.bind(process.stdout);
process.on("SIGINT", () => {});
const MAX_SAFE = Number.MAX_SAFE_INTEGER;
globalThis.require = require;
const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });
function send(o) { out(JSON.stringify(o) + "\n"); }
function toJSON(v) { try { return JSON.parse(JSON.stringify(v)); } catch (e) { return String(v); } }
function sanitize(v, active = new Set()) {
if (v === null || typeof v === "boolean" || typeof v === "string") return v;
if (typeof v === "number") {
if (!Number.isFinite(v)) throw new Error("json: non-finite number outside V_safe");
if (Number.isInteger(v) && Math.abs(v) > MAX_SAFE)
throw new Error("json: integer outside V_safe (±(2^53-1)): " + v);
return v;
}
if (typeof v === "bigint") {
if (v > BigInt(MAX_SAFE) || v < -BigInt(MAX_SAFE))
throw new Error("json: integer outside V_safe (±(2^53-1)): " + v.toString());
return Number(v);
}
if (v && typeof v === "object") {
if (active.has(v)) throw new Error("not JSON-safe: cyclic value");
active.add(v);
try {
const keys = Reflect.ownKeys(v);
if (Array.isArray(v)) {
if (keys.length !== v.length + 1)
throw new Error("not JSON-safe: sparse array or extra array properties");
const rows = [];
for (let i = 0; i < v.length; i++) {
const d = Object.getOwnPropertyDescriptor(v, String(i));
if (!d || !d.enumerable || !("value" in d))
throw new Error("not JSON-safe: array accessor or missing element");
rows.push(sanitize(d.value, active));
}
return rows;
}
const prototype = Object.getPrototypeOf(v);
if (prototype !== Object.prototype && prototype !== null)
throw new Error("not JSON-safe: expected plain record");
const o = Object.create(null);
for (const k of keys) {
if (typeof k !== "string") throw new Error("not JSON-safe: symbol record key");
const d = Object.getOwnPropertyDescriptor(v, k);
if (!d || !d.enumerable || !("value" in d))
throw new Error("not JSON-safe: record accessor or hidden property");
o[k] = sanitize(d.value, active);
}
return o;
} finally { active.delete(v); }
}
throw new Error("not JSON-safe: " + (typeof v));
}
function send(o) {
try { out(JSON.stringify(sanitize(o)) + "\n"); }
catch (e) {
out(JSON.stringify({
id: o && o.id !== undefined ? o.id : null, ok: false,
stdout: (o && o.stdout) || "", stderr: (o && o.stderr) || "",
result: null, error: String(e && e.message ? e.message : e), domain_error: true
}) + "\n");
}
}
function toJSON(v) { return sanitize(v); }
async function capture(fn) {
const logs = [], errs = [];
const ow = process.stdout.write.bind(process.stdout);
@@ -109,7 +215,7 @@ rl.on("line", async (line) => {
let result = null;
try { result = vm.runInThisContext(String(req.name), { filename: "<marqdo-get>" }); }
catch (e) { result = globalThis[req.name] === undefined ? null : globalThis[req.name]; }
send({ id: rid, ok: true, stdout: "", stderr: "", result: toJSON(result), error: "" });
send({ id: rid, ok: true, stdout: "", stderr: "", result: result, error: "" });
return;
}
if (op === "names") {
@@ -130,15 +236,27 @@ rl.on("line", async (line) => {
stdinPatched = true;
} catch (e) {}
}
const cap = await capture(async () => vm.runInThisContext(req.code || "", { filename: "<marqdo>" }));
send({ id: rid, phase: "ready" });
const cap = await capture(async () => vm.runInThisContext(req.code || "", { filename: "<marqdo>", breakOnSigint: true }));
if (stdinPatched) {
try { Object.defineProperty(process, "stdin", { value: oldStdin, configurable: true, writable: true }); } catch (e) {}
}
send({ id: rid, ok: !cap.error, stdout: cap.logs, stderr: cap.errs, result: toJSON(cap.result), error: cap.error });
send({ id: rid, ok: !cap.error, stdout: cap.logs, stderr: cap.errs, result: cap.result, error: cap.error });
});
"#;
/// Internal protocol evidence, distinct from admitted language observations.
/// Includes failed requests and post-execution domain refusals, so set/input
/// dependencies cannot disappear from the recorded history.
#[derive(Debug, Clone)]
pub struct ProtocolStep {
pub session: String,
pub request: serde_json::Value,
pub response: Result<serde_json::Value, String>,
}
pub struct Session {
pub interrupt_handle: InterruptHandle,
pub id: String,
pub lang: String,
pub cmd: String,
@@ -167,7 +285,11 @@ impl std::fmt::Debug for Session {
impl Drop for Session {
fn drop(&mut self) {
let _ = self.child.kill();
if let Some(_pid) = self.interrupt_handle.retire() {
#[cfg(unix)]
unsafe { libc::kill(-(_pid as i32), libc::SIGKILL); }
let _ = self.child.kill();
}
let _ = self.child.wait();
}
}
@@ -182,6 +304,15 @@ fn opt_text(v: Option<&Value>) -> Result<Option<String>, String> {
}
}
/// Session ids are opaque handles — text fabrications are refused.
fn as_session_id<'a>(id: &'a Value) -> Result<&'a str, String> {
match id {
Value::Handle(s) => Ok(s.as_str()),
Value::Text(_) => Err("session id must be a handle (not text)".into()),
_ => Err("session id must be a handle".into()),
}
}
fn parse_timeout(v: Option<&Value>, default: f64) -> Result<Duration, String> {
let secs = match v {
None | Some(Value::None) => default,
@@ -402,6 +533,7 @@ pub fn open(
let started_ms = now_ms();
let cmd_display = argv[0].clone();
let session = Session {
interrupt_handle: InterruptHandle::new(child.id()),
id: id.clone(),
lang: canonical.clone(),
cmd: cmd_display.clone(),
@@ -418,7 +550,7 @@ pub fn open(
};
ctx.foreign_sessions.insert(id.clone(), session);
let _ = timeout;
Ok(Value::Text(id))
Ok(Value::Handle(id))
}
fn session_dead_stderr(ctx: &HostContext, id: &str) -> String {
@@ -453,7 +585,7 @@ pub fn eval_with_stdin(
timeout: Option<&Value>,
) -> Result<Value, String> {
ensure_config(ctx);
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
if !ctx.foreign_sessions.contains_key(&id) {
return Err(format!("foreign session `{id}` not found"));
}
@@ -479,7 +611,7 @@ pub fn eval_with_stdin(
};
let input_json = match input {
None | Some(Value::None) => serde_json::Value::Null,
Some(v) => value_to_json(v)?,
Some(v) => { super::oracle::admit(v)?; value_to_json(v)? },
};
let default_timeout = ctx.foreign_sessions[&id].timeout.as_secs_f64();
let dur = parse_timeout(timeout, default_timeout)?;
@@ -498,6 +630,9 @@ pub fn eval_with_stdin(
let reply = match send_and_wait(ctx, &id, &request, dur) {
Ok(r) => r,
Err(e) => {
// Replay divergence is a language-level stop, not process death or
// an ordinary failed-eval value. Keep the actual committed kernel.
if ctx.foreign_replay_divergence.is_some() { return Err(e); }
let timed_out = e.contains("timed out");
let stderr = session_dead_stderr(ctx, &id);
remove_session(ctx, &id);
@@ -544,11 +679,16 @@ pub fn eval_with_stdin(
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
// Ω ∉ V_safe is a language fault, not a soft cell failure.
if reply.get("domain_error").and_then(|v| v.as_bool()) == Some(true) {
return Err(error);
}
let result = reply
.get("result")
.map(json_to_value)
.transpose()?
.unwrap_or(Value::None);
super::oracle::admit(&result)?;
Ok(record(
Some(&id),
&lang,
@@ -577,13 +717,54 @@ fn send_and_wait(
request: &serde_json::Value,
timeout: Duration,
) -> Result<serde_json::Value, String> {
if ctx.foreign_replay_expected.is_some() {
if let Some(error) = &ctx.foreign_replay_divergence { return Err(error.clone()); }
}
let permit = validation::prepare(ctx.foreign_replay_expected.as_mut(), id, request)
.map_err(|failure| {
let error = failure.to_string();
ctx.foreign_replay_divergence = Some(error.clone());
error
})?;
let control = ctx.foreign_sessions.get(id).map(|s| s.interrupt_handle.clone());
if let Some(handle) = &control { handle.begin(request.get("op").and_then(|v| v.as_str()) == Some("eval")); }
let response = send_and_wait_inner(ctx, permit, timeout);
if let Some(handle) = control { handle.finish(); }
ctx.foreign_protocol_steps.push(ProtocolStep {
session: id.to_string(), request: request.clone(), response: response.clone(),
});
if let Some(steps) = &mut ctx.foreign_replay_expected {
if let Err(failure) = steps.accept_response(&response) {
let error = failure.to_string();
ctx.foreign_replay_divergence = Some(error.clone());
return Err(error);
}
}
response
}
fn send_control(
ctx: &mut HostContext, id: &str, request: &serde_json::Value,
) -> Result<serde_json::Value, String> {
let response = send_and_wait(ctx, id, request, Duration::from_secs(10));
if response.is_err() && ctx.foreign_replay_divergence.is_none() { remove_session(ctx, id); }
response
}
fn send_and_wait_inner(
ctx: &mut HostContext,
permit: validation::DispatchPermit,
timeout: Duration,
) -> Result<serde_json::Value, String> {
let id = permit.session();
let request = permit.request();
let line = serde_json::to_string(request).map_err(|e| format!("session request: {e}"))?;
{
let session = ctx
.foreign_sessions
.get_mut(id)
.ok_or_else(|| format!("foreign session `{id}` not found"))?;
if let Ok(Some(_)) = session.child.try_wait() {
if let Ok(Some(_)) = session.interrupt_handle.try_wait(&mut session.child) {
return Err("session process exited".into());
}
session
@@ -617,13 +798,18 @@ fn send_and_wait(
if rid != 1 {
continue;
}
if v.get("phase").and_then(|v| v.as_str()) == Some("ready") {
session.interrupt_handle.ready()?;
continue;
}
return Ok(v);
}
// Unexpected non-protocol output: keep waiting until the deadline.
continue;
}
Err(mpsc::RecvTimeoutError::Timeout) => {
if let Ok(Some(_)) = session.child.try_wait() {
session.interrupt_handle.retry()?;
if let Ok(Some(_)) = session.interrupt_handle.try_wait(&mut session.child) {
return Err("session process exited".into());
}
continue;
@@ -637,11 +823,13 @@ fn send_and_wait(
fn kill_session(ctx: &mut HostContext, id: &str) {
if let Some(mut s) = ctx.foreign_sessions.remove(id) {
#[cfg(unix)]
unsafe {
libc::kill(-(s.child.id() as i32), libc::SIGKILL);
if let Some(_pid) = s.interrupt_handle.retire() {
#[cfg(unix)]
unsafe {
libc::kill(-(_pid as i32), libc::SIGKILL);
}
let _ = s.child.kill();
}
let _ = s.child.kill();
let _ = s.child.wait();
s.alive = false;
}
@@ -650,22 +838,12 @@ fn kill_session(ctx: &mut HostContext, id: &str) {
/// Interrupt the current evaluation without destroying the kernel. Python
/// reports KeyboardInterrupt through the normal result frame.
pub fn interrupt(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
let session = ctx
.foreign_sessions
.get_mut(&id)
.ok_or_else(|| format!("foreign session `{id}` not found"))?;
#[cfg(unix)]
unsafe {
libc::kill(-(session.child.id() as i32), libc::SIGINT);
}
#[cfg(windows)]
{
let _ = Command::new("taskkill")
.args(["/PID", &session.child.id().to_string(), "/T", "/F"])
.output();
session.alive = false;
}
session.interrupt_handle.request()?;
Ok(Value::Bool(true))
}
@@ -675,14 +853,18 @@ pub fn set(
name: &Value,
value: &Value,
) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
if !ctx.foreign_sessions.contains_key(&id) {
return Err(format!("foreign session `{id}` not found"));
}
let name = as_text(name, "name")?.to_string();
if !valid_name(&name) {
return Err("session_set: name must be an identifier".into());
}
super::oracle::admit(value)?;
let json = value_to_json(value)?;
let request = serde_json::json!({"id": 1, "op": "set", "name": name, "value": json});
let reply = send_and_wait(ctx, &id, &request, Duration::from_secs(10))?;
let reply = send_control(ctx, &id, &request)?;
if reply.get("ok").and_then(|v| v.as_bool()).unwrap_or(false) {
Ok(Value::None)
} else {
@@ -695,13 +877,16 @@ pub fn set(
}
pub fn get(ctx: &mut HostContext, id: &Value, name: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
if !ctx.foreign_sessions.contains_key(&id) {
return Err(format!("foreign session `{id}` not found"));
}
let name = as_text(name, "name")?.to_string();
if !valid_name(&name) {
return Err("session_get: name must be an identifier".into());
}
let request = serde_json::json!({"id": 1, "op": "get", "name": name});
let reply = send_and_wait(ctx, &id, &request, Duration::from_secs(10))?;
let reply = send_control(ctx, &id, &request)?;
if !reply.get("ok").and_then(|v| v.as_bool()).unwrap_or(false) {
return Err(reply
.get("error")
@@ -709,16 +894,21 @@ pub fn get(ctx: &mut HostContext, id: &Value, name: &Value) -> Result<Value, Str
.unwrap_or("session_get failed")
.to_string());
}
match reply.get("result") {
let value = match reply.get("result") {
Some(v) => json_to_value(v),
None => Ok(Value::None),
}
}?;
super::oracle::admit(&value)?;
Ok(value)
}
pub fn names(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
if !ctx.foreign_sessions.contains_key(&id) {
return Err(format!("foreign session `{id}` not found"));
}
let request = serde_json::json!({"id": 1, "op": "names"});
let reply = send_and_wait(ctx, &id, &request, Duration::from_secs(10))?;
let reply = send_control(ctx, &id, &request)?;
if !reply.get("ok").and_then(|v| v.as_bool()).unwrap_or(false) {
return Err(reply
.get("error")
@@ -733,10 +923,14 @@ pub fn names(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
}
pub fn close(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let existed = ctx.foreign_sessions.contains_key(&id);
let id = as_session_id(id)?.to_string();
// Affine close: consuming an absent / already-closed handle is a fault
// (same surface as eval/set/get after close).
if !ctx.foreign_sessions.contains_key(&id) {
return Err(format!("foreign session `{id}` not found"));
}
kill_session(ctx, &id);
Ok(Value::Bool(existed))
Ok(Value::Bool(true))
}
pub fn list(ctx: &HostContext) -> Result<Value, String> {
@@ -745,7 +939,7 @@ pub fn list(ctx: &HostContext) -> Result<Value, String> {
let mut out = Vec::new();
for s in sessions {
out.push(Value::Map(vec![
("id".into(), Value::Text(s.id.clone())),
("id".into(), Value::Handle(s.id.clone())),
("lang".into(), Value::Text(s.lang.clone())),
("cmd".into(), Value::Text(s.cmd.clone())),
("evals".into(), Value::Int(s.evals as i64)),
@@ -763,7 +957,7 @@ pub fn shutdown(ctx: &mut HostContext) {
}
}
fn valid_name(name: &str) -> bool {
pub(crate) fn valid_name(name: &str) -> bool {
let mut chars = name.chars();
match chars.next() {
Some(c) if c == '_' || c.is_ascii_alphabetic() => {}
@@ -783,4 +977,301 @@ mod tests {
assert!(!valid_name("1x"));
assert!(!valid_name("a-b"));
}
#[test]
fn bridge_live_boundary_and_refusal_preserves_input_store() {
let edge = 9_007_199_254_740_991i64;
for lang in ["python", "node"] {
let mut ctx = HostContext::default();
let id = open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
let name = Value::Text("payload".into());
for n in [i64::MIN, -edge-1, -edge, -edge+1, -1, 0, 1, edge-1, edge, edge+1, i64::MAX] {
set(&mut ctx, &id, &name, &Value::Int(17)).unwrap();
let input = Value::List(vec![Value::Text("中文🧪".into()), Value::List(vec![Value::Int(n)])]);
let result = set(&mut ctx, &id, &name, &input);
if (-edge..=edge).contains(&n) {
result.unwrap();
assert_eq!(get(&mut ctx, &id, &name).unwrap(), input, "{lang} {n}");
} else {
assert!(result.unwrap_err().contains("outside V_safe"));
assert_eq!(get(&mut ctx, &id, &name).unwrap(), Value::Int(17));
}
println!("E4 {lang} integer {n}: verified");
}
for source in if lang == "python" {
["2**53", "float('nan')", "float('inf')"]
} else { ["2**53", "NaN", "Infinity"] } {
let error = eval(&mut ctx, &id, None, Some(&Value::Text(source.into())), None, None).unwrap_err();
assert!(error.contains("outside V_safe"), "{lang} {source}: {error}");
// A refused response must remain a protocol response, not kill Node.
assert_eq!(get(&mut ctx, &id, &name).unwrap(), Value::Int(17));
println!("E4 {lang} response {source}: refused, session responsive");
}
let alias = id.clone();
close(&mut ctx, &id).unwrap();
// Missing handle wins over an invalid name and an OOD payload.
let err = set(&mut ctx, &alias, &Value::Text("bad-name".into()), &Value::Int(i64::MAX)).unwrap_err();
assert!(err.contains("not found"), "{err}");
assert!(get(&mut ctx, &alias, &name).is_err());
assert!(close(&mut ctx, &alias).is_err());
assert!(ctx.foreign_sessions.is_empty());
}
}
#[test]
fn generated_two_session_model_traces() {
for lang in ["python", "node"] {
let mut ctx = HostContext::default();
let mut ids = Vec::new();
let mut live = [true; 2];
let mut expected = [0i64; 2];
let name = Value::Text("counter".into());
for _ in 0..2 {
let id = open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
set(&mut ctx, &id, &name, &Value::Int(0)).unwrap();
ids.push(id);
}
let mut seed = 0x52455155495245u64;
for step in 0..256 {
seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1);
let slot = ((seed >> 32) & 1) as usize;
let op = (seed >> 40) % 5;
match op {
0 => {
let value = step as i64 - 128;
let result = set(&mut ctx, &ids[slot], &name, &Value::Int(value));
assert_eq!(result.is_ok(), live[slot]);
if live[slot] { expected[slot] = value; }
}
1 => {
let result = get(&mut ctx, &ids[slot], &name);
if live[slot] { assert_eq!(result.unwrap(), Value::Int(expected[slot])); }
else { assert!(result.unwrap_err().contains("not found")); }
}
2 => {
let src = if lang == "python" { "counter += 1\ncounter" } else { "counter += 1; counter" };
let result = eval(&mut ctx, &ids[slot], None, Some(&Value::Text(src.into())), None, None);
assert_eq!(result.is_ok(), live[slot]);
if live[slot] { expected[slot] += 1; }
}
3 => {
assert_eq!(close(&mut ctx, &ids[slot]).is_ok(), live[slot]);
live[slot] = false;
}
_ => {
if live[slot] { close(&mut ctx, &ids[slot]).unwrap(); }
let stale = ids[slot].clone();
ids[slot] = open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
assert_ne!(stale, ids[slot]);
assert!(get(&mut ctx, &stale, &name).is_err());
set(&mut ctx, &ids[slot], &name, &Value::Int(0)).unwrap();
expected[slot] = 0;
live[slot] = true;
}
}
// Independent model predicts every remaining store after every command.
for j in 0..2 {
assert_eq!(ctx.foreign_sessions.contains_key(as_session_id(&ids[j]).unwrap()), live[j]);
if live[j] { assert_eq!(get(&mut ctx, &ids[j], &name).unwrap(), Value::Int(expected[j]), "{lang} step {step} slot {j}"); }
}
}
shutdown(&mut ctx);
assert!(ctx.foreign_sessions.is_empty());
println!("E3 {lang}: seed 0x52455155495245, 256 commands verified");
}
}
#[test]
fn failure_process_exit_timeout_and_reopen() {
for lang in ["python", "node"] {
let mut ctx = HostContext::default();
for source in if lang == "python" { ["import os; os._exit(7)", "while True: pass"] }
else { ["process.exit(7)", "while(true) {}"] } {
let id = open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
let result = eval(&mut ctx, &id, None, Some(&Value::Text(source.into())), None, Some(&Value::Num(0.1)));
assert!(result.is_ok(), "process failure should be a failure record: {result:?}");
let Value::Map(m) = result.unwrap() else { panic!() };
assert!(m.iter().any(|(k,v)| k == "ok" && v == &Value::Bool(false)));
assert!(ctx.foreign_sessions.is_empty(), "dead session must be retired");
assert!(get(&mut ctx, &id, &Value::Text("x".into())).is_err());
println!("E5 {lang} {source}: recorded and retired");
}
let id = open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
set(&mut ctx, &id, &Value::Text("x".into()), &Value::Int(41)).unwrap();
assert_eq!(get(&mut ctx, &id, &Value::Text("x".into())).unwrap(), Value::Int(41));
shutdown(&mut ctx);
}
}
#[test]
fn input_history_and_post_refusal_effects_are_preserved() {
for lang in ["python", "node"] {
let mut ctx = HostContext::default();
let id = open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
let name = Value::Text("n".into());
set(&mut ctx, &id, &name, &Value::Int(1)).unwrap();
let code = if lang == "python" { "n += marqdo_input\nn" } else { "n += marqdo_input; n" };
let result = eval(&mut ctx, &id, None, Some(&Value::Text(code.into())), Some(&Value::Int(3)), None).unwrap();
let Value::Map(fields) = result else { panic!() };
assert_eq!(fields.iter().find(|(k,_)| k == "result").unwrap().1, Value::Int(4));
let sent = ctx.foreign_protocol_steps.last().unwrap();
assert_eq!(sent.session, as_session_id(&id).unwrap());
assert_eq!(sent.request["op"], "eval");
assert_eq!(sent.request["input"], 3);
assert_eq!(sent.request["code"], code);
let before = ctx.foreign_protocol_steps.len();
assert!(set(&mut ctx, &id, &name, &Value::Int(i64::MAX)).is_err());
assert_eq!(ctx.foreign_protocol_steps.len(), before, "invalid input must not query oracle");
assert_eq!(get(&mut ctx, &id, &name).unwrap(), Value::Int(4));
let bad = if lang == "python" { "n = 99\n2**53" } else { "n = 99; 2**53" };
assert!(eval(&mut ctx, &id, None, Some(&Value::Text(bad.into())), None, None).unwrap_err().contains("outside V_safe"));
let fault = ctx.foreign_protocol_steps.last().unwrap();
assert_eq!(fault.response.as_ref().unwrap()["domain_error"], true);
assert_eq!(get(&mut ctx, &id, &name).unwrap(), Value::Int(99), "foreign effects must not be claimed to roll back");
let ordinary_error = if lang == "python" { "raise ValueError('outside V_safe')" }
else { "throw new Error('outside V_safe')" };
let record = eval(&mut ctx, &id, None, Some(&Value::Text(ordinary_error.into())), None, None).unwrap();
let Value::Map(fields) = record else { panic!() };
assert_eq!(fields.iter().find(|(k,_)| k == "ok").unwrap().1, Value::Bool(false));
let unsupported = if lang == "python" { "object()" } else { "(function(){})" };
assert!(eval(&mut ctx, &id, None, Some(&Value::Text(unsupported.into())), None, None).unwrap_err().contains("not JSON-safe"));
assert_eq!(get(&mut ctx, &id, &name).unwrap(), Value::Int(99));
close(&mut ctx, &id).unwrap();
println!("E6 {lang}: complete inputs recorded; pre-send refusal has no request; post-send refusal retains effects; structured domain classification verified");
}
}
#[test]
fn numeric_extension_policy_agrees_on_python_and_node() {
for lang in ["python", "node"] {
let mut ctx = HostContext::default();
let id = open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
let key = Value::Text("number".into());
for value in [1.5, -0.0, 9_007_199_254_740_991.0, -9_007_199_254_740_991.0] {
set(&mut ctx, &id, &key, &Value::Num(value)).unwrap();
match get(&mut ctx, &id, &key).unwrap() {
Value::Num(n) => assert_eq!(n, value),
Value::Int(n) => assert_eq!(n as f64, value),
other => panic!("{lang}: {other:?}"),
}
}
for value in [9_007_199_254_740_992.0, -9_007_199_254_740_992.0, 1e100] {
let before = ctx.foreign_protocol_steps.len();
assert!(set(&mut ctx, &id, &key, &Value::Num(value)).unwrap_err().contains("outside V_safe"));
assert_eq!(ctx.foreign_protocol_steps.len(), before);
}
let source = if lang == "python" { "float(2**53)" } else { "Number(2**53)" };
assert!(eval(&mut ctx, &id, None, Some(&Value::Text(source.into())), None, None).unwrap_err().contains("outside V_safe"));
close(&mut ctx, &id).unwrap();
println!("E7 {lang}: finite fractions admitted, integral float/scientific-notation bypass refused");
}
}
#[test]
fn node_non_json_containers_and_lossy_properties_are_refused_without_coercion() {
let mut ctx = HostContext::default();
let id = open(&mut ctx, &Value::Text("node".into()), None, None, None, None).unwrap();
let expressions = [
"new Map([['k',1]])", "new Set([1,2])", "new Date(0)", "/x/",
"new Uint8Array([1,2])", "new Number(7)",
"Object.assign(Object.create({inherited:1}),{own:2})",
"Object.defineProperty({},'hidden',{value:7})", "({[Symbol('k')]:7})",
"Object.defineProperty({},'data',{enumerable:true,get(){retained=123;return 7}})",
"new Array(2)", "Object.assign([1,2],{tag:3})", "[undefined]",
"(()=>{var x={};x.self=x;return x;})()",
];
for expression in expressions {
let code = Value::Text(format!("retained = 99; {expression}"));
let result = eval(&mut ctx,&id,None,Some(&code),None,None);
assert!(result.as_ref().is_err_and(|e|e.contains("not JSON-safe")),"silently coerced {expression}: {result:?}");
let reply=ctx.foreign_protocol_steps.last().unwrap().response.as_ref().unwrap().clone();
assert_eq!(reply["domain_error"],serde_json::json!(true));
assert_eq!(get(&mut ctx,&id,&Value::Text("retained".into())).unwrap(),Value::Int(99),"getter must not run during refusal");
println!("NODE_DOMAIN_REFUSAL:{}",serde_json::json!({"expression":expression,"reply":reply.clone(),"retained":99}));
}
// Null prototypes, special string keys and shared acyclic subvalues remain records/lists.
let code=Value::Text("var item = Object.assign(Object.create(null),{constructor:7}); [item,item]".into());
let result=eval(&mut ctx,&id,None,Some(&code),None,None).unwrap();
let Value::Map(fields)=result else {panic!()};
let observed=&fields.iter().find(|(k,_)|k=="result").unwrap().1;
assert_eq!(observed,&Value::List(vec![Value::Map(vec![("constructor".into(),Value::Int(7))]);2]));
close(&mut ctx,&id).unwrap();
}
#[test]
fn recursive_values_and_object_keys_cross_without_coercion() {
fn payload(seed: u64, depth: usize) -> Value {
if depth == 0 { return match seed % 4 {
0 => Value::None, 1 => Value::Bool(seed & 4 != 0),
2 => Value::Int((seed % 2001) as i64 - 1000),
_ => Value::Text(format!("中文🧪\\\"\n{}", seed % 10)),
}; }
match seed % 3 {
0 => Value::List(vec![payload(seed.wrapping_mul(17).wrapping_add(1), depth-1), payload(seed.wrapping_mul(31).wrapping_add(2), depth-1)]),
1 => Value::Map(vec![("data".into(), payload(seed.wrapping_mul(43).wrapping_add(3), depth-1)), ("tag".into(), Value::Text("record".into()))]),
_ => payload(seed, 0),
}
}
for lang in ["python", "node"] {
let mut ctx = HostContext::default();
let id = open(&mut ctx, &Value::Text(lang.into()), None, None, None, None).unwrap();
let name = Value::Text("payload".into());
for seed in 0..64 {
let v = payload(seed, 4);
set(&mut ctx, &id, &name, &v).unwrap();
assert_eq!(get(&mut ctx, &id, &name).unwrap(), v, "{lang} seed {seed}");
}
let special = Value::Map(vec![
("__proto__".into(), Value::Map(vec![("polluted".into(), Value::Bool(true))])),
("constructor".into(), Value::Text("data".into())),
]);
set(&mut ctx, &id, &name, &special).unwrap();
assert_eq!(get(&mut ctx, &id, &name).unwrap(), special, "{lang} valid special keys lost");
for bad in [Value::Map(vec![("a".into(), Value::Int(1)), ("a".into(), Value::Int(2))]),
Value::Map(vec![("a".into(), Value::List(vec![])), ("b".into(), Value::List(vec![Value::Int(1)]))])] {
let before = ctx.foreign_protocol_steps.len();
assert!(set(&mut ctx, &id, &name, &bad).is_err());
assert_eq!(ctx.foreign_protocol_steps.len(), before);
assert_eq!(get(&mut ctx, &id, &name).unwrap(), special);
}
if lang == "python" {
let code = Value::Text("{1: 'integer key', '1': 'string key'}".into());
assert!(eval(&mut ctx, &id, None, Some(&code), None, None).unwrap_err().contains("keys must be strings"));
} else {
let code = Value::Text("typeof ({}).polluted === 'undefined'".into());
let Value::Map(m) = eval(&mut ctx, &id, None, Some(&code), None, None).unwrap() else { panic!() };
assert_eq!(m.iter().find(|(k,_)| k == "result").unwrap().1, Value::Bool(true));
}
close(&mut ctx, &id).unwrap();
println!("E8 {lang}: 64 nested round-trips, special keys preserved, invalid key/shape inputs refused without requests");
}
}
#[cfg(unix)]
#[test]
fn bootstrap_interrupts_busy_eval_and_preserves_idle_state(){
for lang in ["python","node"]{
let mut ctx=HostContext::default();let id=open(&mut ctx,&Value::Text(lang.into()),None,None,None,None).unwrap();
let name=Value::Text("n".into());set(&mut ctx,&id,&name,&Value::Int(8)).unwrap();
interrupt(&mut ctx,&id).unwrap();assert_eq!(get(&mut ctx,&id,&name).unwrap(),Value::Int(8));
let marker=ctx.cwd.join(format!("target/interrupt-marker-{}-{lang}",std::process::id()));let _=std::fs::remove_file(&marker);
let literal=serde_json::to_string(&marker.display().to_string()).unwrap();
let code=if lang=="python"{format!("with open({literal}, 'w') as marker:\n marker.write('ready')\nwhile True: pass")}
else{format!("require('fs').writeFileSync({literal}, 'ready'); while(true) {{}}")};
let pid=ctx.foreign_sessions[as_session_id(&id).unwrap()].child.id();let child_id=id.clone();
let worker=std::thread::spawn(move||{let out=eval(&mut ctx,&child_id,None,Some(&Value::Text(code)),None,Some(&Value::Num(3.0)));(ctx,out)});
let deadline=Instant::now()+Duration::from_secs(2);
while !marker.exists()&&Instant::now()<deadline{std::thread::sleep(Duration::from_millis(1));}
let ready=marker.exists();
if ready{assert_eq!(unsafe{libc::kill(-(pid as i32),libc::SIGINT)},0);}
let (mut ctx,out)=worker.join().unwrap();assert!(ready,"{lang}: code did not start");
let Value::Map(record)=out.unwrap() else{panic!()};
assert_eq!(record.iter().find(|(k,_)|k=="ok").unwrap().1,Value::Bool(false));
assert_eq!(record.iter().find(|(k,_)|k=="alive").unwrap().1,Value::Bool(true));
let error=record.iter().find(|(k,_)|k=="error").unwrap().1.as_display();
assert!(error.contains("KeyboardInterrupt")||error.contains("interrupted"),"{lang}: {error}");
assert_eq!(get(&mut ctx,&id,&name).unwrap(),Value::Int(8));close(&mut ctx,&id).unwrap();std::fs::remove_file(marker).unwrap();
println!("E10 {lang}: idle interrupt is safe; busy synchronous eval interrupted with kernel and store alive");
}
}
}
+204
View File
@@ -0,0 +1,204 @@
//! An owned-process interrupt capability. Retirement precedes every child reap,
//! so a stale capability cannot signal a reused operating-system process id.
use std::sync::{Arc, Mutex};
#[derive(Debug, Default)]
struct State {
pid: Option<u32>,
evaluating: bool,
ready: bool,
requested: bool,
}
#[derive(Clone, Debug, Default)]
pub struct InterruptHandle(Arc<Mutex<State>>);
impl InterruptHandle {
pub(super) fn new(pid: u32) -> Self {
Self(Arc::new(Mutex::new(State {
pid: Some(pid),
..State::default()
})))
}
pub(super) fn retire(&self) -> Option<u32> {
if let Ok(mut state) = self.0.lock() {
let pid = state.pid.take();
state.evaluating = false;
pid
} else {
None
}
}
pub(super) fn try_wait(
&self,
child: &mut std::process::Child,
) -> std::io::Result<Option<std::process::ExitStatus>> {
let mut state = self.0.lock().unwrap();
let result = child.try_wait();
if matches!(result, Ok(Some(_))) {
state.pid = None;
state.evaluating = false;
}
result
}
pub(super) fn begin(&self, evaluating: bool) {
let mut state = self.0.lock().unwrap();
state.evaluating = evaluating;
state.ready = false;
state.requested = false;
}
pub(super) fn finish(&self) {
self.0.lock().unwrap().evaluating = false;
}
// SIGINT can arrive between a Node ready frame and installation of V8's
// breakOnSigint handler. Retry the same logical request while awaiting its
// response; never carry a retry into the next request or retired process.
pub(super) fn retry(&self) -> Result<(), String> {
let state = self.0.lock().map_err(|_| "interrupt lock poisoned")?;
if state.evaluating && state.ready && state.requested {
Self::signal(&state)?;
}
Ok(())
}
pub(super) fn ready(&self) -> Result<(), String> {
let mut state = self.0.lock().map_err(|_| "interrupt lock poisoned")?;
if state.ready {
return Ok(());
}
state.ready = true;
if state.requested {
Self::signal(&state)?;
}
Ok(())
}
pub fn is_live(&self) -> bool {
self.0.lock().is_ok_and(|s| s.pid.is_some())
}
/// False means idle; cancellation is coalesced within one evaluation.
/// Before the bootstrap's ready frame it is queued rather than sent into
/// interpreter startup or transport parsing.
pub fn request(&self) -> Result<bool, String> {
let mut state = self.0.lock().map_err(|_| "interrupt lock poisoned")?;
if state.pid.is_none() {
return Err("interrupt capability retired".into());
}
if !state.evaluating {
return Ok(false);
}
#[cfg(not(unix))]
return Err("non-destructive interrupt is unsupported on this platform".into());
if !state.requested {
if state.ready {
Self::signal(&state)?;
}
state.requested = true;
}
Ok(true)
}
fn signal(state: &State) -> Result<(), String> {
let pid = state.pid.ok_or("interrupt capability retired")?;
#[cfg(unix)]
{
if unsafe { libc::kill(-(pid as i32), libc::SIGINT) } != 0 {
return Err(format!(
"interrupt failed: {}",
std::io::Error::last_os_error()
));
}
Ok(())
}
#[cfg(not(unix))]
{
let _ = pid;
Err("non-destructive interrupt is unsupported on this platform".into())
}
}
}
#[cfg(all(test, unix))]
mod tests {
use crate::host::{foreign_session, HostContext};
use crate::value::Value;
#[test]
fn retired_interrupt_capability_cannot_reach_replacement_kernel() {
for language in ["python", "node"] {
let mut host = HostContext::default();
let lang = Value::Text(language.into());
let old = foreign_session::open(&mut host, &lang, None, None, None, None).unwrap();
let Value::Handle(id) = &old else { panic!() };
let control = host.foreign_sessions[id].interrupt_handle.clone();
assert!(!control.request().unwrap());
foreign_session::close(&mut host, &old).unwrap();
let new = foreign_session::open(&mut host, &lang, None, None, None, None).unwrap();
assert!(!control.is_live());
assert!(control.request().unwrap_err().contains("retired"));
foreign_session::set(&mut host, &new, &Value::Text("n".into()), &Value::Int(8))
.unwrap();
assert_eq!(
foreign_session::get(&mut host, &new, &Value::Text("n".into())).unwrap(),
Value::Int(8)
);
foreign_session::close(&mut host, &new).unwrap();
}
}
#[test]
fn cancellation_queued_during_bootstrap_reaches_busy_evaluation() {
use std::time::{Duration, Instant};
for language in ["python", "node"] {
let mut host = HostContext::default();
let id = foreign_session::open(
&mut host,
&Value::Text(language.into()),
None,
None,
None,
None,
)
.unwrap();
let Value::Handle(ref name) = id else {
panic!()
};
let control = host.foreign_sessions[name].interrupt_handle.clone();
let kernel = id.clone();
let code = if language == "python" {
"while True: pass"
} else {
"while(true) {}"
};
let worker = std::thread::spawn(move || {
let record = foreign_session::eval(
&mut host,
&kernel,
None,
Some(&Value::Text(code.into())),
None,
Some(&Value::Int(2)),
)
.unwrap();
(host, record)
});
let deadline = Instant::now() + Duration::from_secs(1);
while !control.0.lock().unwrap().evaluating && Instant::now() < deadline {
std::thread::yield_now();
}
assert!(control.request().unwrap());
assert!(
control.request().unwrap(),
"logical duplicates must coalesce"
);
let (mut host, record) = worker.join().unwrap();
let Value::Map(fields) = record else { panic!() };
assert_eq!(
fields.iter().find(|(k, _)| k == "timed_out").unwrap().1,
Value::Bool(false),
"{language}: {fields:?}"
);
assert_eq!(
fields.iter().find(|(k, _)| k == "alive").unwrap().1,
Value::Bool(true)
);
foreign_session::close(&mut host, &id).unwrap();
}
}
}
+173
View File
@@ -0,0 +1,173 @@
//! Production adapter to the executable Lean replay policy.
//! All replay decisions come from ReplayPolicy.Cursor, with no Rust fallback.
use super::ProtocolStep;
use serde_json::{json, Value};
use std::collections::VecDeque;
mod policy;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DivergenceKind { Missing, Request, Reply, Surplus, Phase, Policy }
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Divergence {
pub kind: DivergenceKind,
pub matched_prefix: usize,
pub detail: Option<String>,
}
impl std::fmt::Display for Divergence {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(match self.kind {
DivergenceKind::Missing | DivergenceKind::Request => "replay: protocol request differs before dispatch",
DivergenceKind::Reply => "replay: protocol response differs",
DivergenceKind::Surplus => "replay: recorded execution prefix was not consumed",
DivergenceKind::Phase => "replay: invalid protocol validation phase",
DivergenceKind::Policy => "replay: verified policy unavailable",
})?;
if let Some(detail) = &self.detail { write!(f, ": {detail}")? }
Ok(())
}
}
#[derive(Debug)]
pub struct ReplayCursor {
client: Option<policy::Client>,
progress: (usize, usize),
stopped: Option<Divergence>,
}
impl From<VecDeque<ProtocolStep>> for ReplayCursor {
fn from(expected: VecDeque<ProtocolStep>) -> Self {
let count = expected.len();
match policy::Client::start(expected) {
Ok((client, reply)) if reply.allow && reply.matched == 0 && reply.remaining == count =>
Self { client: Some(client), progress: (0, count), stopped: None },
result => {
let detail = match result { Err(e) => e, Ok(_) => "invalid policy initialization".into() };
Self { client: None, progress: (0,count), stopped: Some(Divergence {
kind: DivergenceKind::Policy, matched_prefix: 0, detail: Some(detail) }) }
}
}
}
}
impl From<Vec<ProtocolStep>> for ReplayCursor {
fn from(expected: Vec<ProtocolStep>) -> Self { VecDeque::from(expected).into() }
}
/// Private construction, no Clone: actual dispatch consumes this snapshot.
pub(super) struct DispatchPermit { session: String, request: Value }
impl DispatchPermit {
pub(super) fn session(&self) -> &str { &self.session }
pub(super) fn request(&self) -> &Value { &self.request }
}
impl ReplayCursor {
fn unavailable(&mut self, detail: String) -> Divergence {
self.client.take();
let error = Divergence { kind: DivergenceKind::Policy, matched_prefix: self.progress.0, detail: Some(detail) };
self.stopped = Some(error.clone());
error
}
fn command(&mut self, command: Value) -> Result<(), Divergence> {
if let Some(error) = &self.stopped { return Err(error.clone()) }
let result = self.client.as_mut().ok_or_else(|| "no policy process".to_string())
.and_then(|client| client.exchange(command));
let reply = result.map_err(|e| self.unavailable(e))?;
self.progress = (reply.matched, reply.remaining);
if reply.allow { return Ok(()) }
let kind = match reply.reason.as_deref() {
Some("missing") => DivergenceKind::Missing,
Some("request") => DivergenceKind::Request,
Some("reply") => DivergenceKind::Reply,
Some("surplus") => DivergenceKind::Surplus,
Some("phase") => DivergenceKind::Phase,
_ => return Err(self.unavailable("unknown policy stop reason".into())),
};
let error = Divergence { kind, matched_prefix: reply.matched, detail: None };
self.stopped = Some(error.clone());
Err(error)
}
fn prepare(&mut self, session: &str, request: &Value) -> Result<DispatchPermit, Divergence> {
self.command(json!({"op":"prepare", "value":policy::request_key(session,request)}))?;
Ok(DispatchPermit { session: session.into(), request: request.clone() })
}
pub(super) fn accept_response(&mut self, response: &Result<Value,String>) -> Result<(), Divergence> {
self.command(json!({"op":"receive", "value":policy::response_key(response)}))
}
pub fn progress(&self) -> (usize,usize) { self.progress }
/// Called only at the outer action boundary; consumes the cursor.
pub fn finish(mut self) -> Result<(),Divergence> { self.command(json!({"op":"finish"})) }
}
pub(super) fn prepare(cursor: Option<&mut ReplayCursor>, session: &str, request: &Value)
-> Result<DispatchPermit,Divergence> {
if let Some(cursor) = cursor { return cursor.prepare(session,request) }
Ok(DispatchPermit { session: session.into(), request: request.clone() })
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn step(reply: Result<Value, String>) -> ProtocolStep {
ProtocolStep {
session: "s".into(),
request: json!({"op":"get"}),
response: reply,
}
}
#[test]
fn local_returns_and_ordinary_failures_preserve_residual_history() {
let mut cursor: ReplayCursor =
vec![step(Err("foreign failure".into())), step(Ok(json!(7)))].into();
assert_eq!(cursor.progress(), (0, 2));
let permit = cursor.prepare("s", &json!({"op":"get"})).unwrap();
assert_eq!(permit.session(), "s");
cursor
.accept_response(&Err("foreign failure".into()))
.unwrap();
assert_eq!(cursor.progress(), (1, 1));
cursor.prepare("s", &json!({"op":"get"})).unwrap();
cursor.accept_response(&Ok(json!(7))).unwrap();
cursor.finish().unwrap();
}
#[test]
fn divergence_is_sticky_and_matching_reply_cannot_revive_it() {
let mut cursor: ReplayCursor = vec![step(Ok(json!(7)))].into();
cursor.prepare("s", &json!({"op":"get"})).unwrap();
let first = cursor.accept_response(&Ok(json!(8))).unwrap_err();
assert_eq!(first.kind, DivergenceKind::Reply);
assert_eq!(
cursor.prepare("s", &json!({"op":"get"})).err().unwrap(),
first
);
assert_eq!(cursor.accept_response(&Ok(json!(7))).unwrap_err(), first);
assert_eq!(cursor.finish().unwrap_err(), first);
}
#[test]
fn permit_is_a_snapshot_and_phase_errors_fail_closed() {
let mut request = json!({"op":"get"});
let mut cursor: ReplayCursor = vec![step(Ok(json!(7)))].into();
let permit = cursor.prepare("s", &request).unwrap();
request["op"] = json!("write");
assert_ne!(permit.request(), &request);
assert_eq!(
cursor.prepare("s", permit.request()).err().unwrap().kind,
DivergenceKind::Phase
);
assert!(cursor.accept_response(&Ok(json!(7))).is_err());
}
#[test]
fn mismatching_and_missing_requests_never_issue_a_permit() {
let mut cursor: ReplayCursor = vec![step(Ok(json!(7)))].into();
let error = cursor
.prepare("other-session", &json!({"op":"get"}))
.err()
.unwrap();
assert_eq!(error.kind, DivergenceKind::Request);
assert_eq!(cursor.progress(), (0, 1));
assert_eq!(
cursor.prepare("s", &json!({"op":"get"})).err().unwrap(),
error
);
let mut empty: ReplayCursor = Vec::<ProtocolStep>::new().into();
assert_eq!(
empty.prepare("s", &json!({"op":"get"})).err().unwrap().kind,
DivergenceKind::Missing
);
assert!(empty.finish().is_err());
}
}
@@ -0,0 +1,92 @@
//! Transport for the executable Lean policy. This module makes no replay
//! acceptance decisions; malformed/unavailable policy responses fail closed.
use super::super::ProtocolStep;
use serde::Deserialize;
use serde_json::{json, Value};
use std::io::{BufRead, BufReader, Write};
use std::path::PathBuf;
use std::process::{Child, ChildStdin, Command, Stdio};
use std::sync::mpsc::{self, Receiver};
use std::time::Duration;
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
pub(super) struct Reply {
pub policy: String,
pub allow: bool,
pub matched: usize,
pub remaining: usize,
pub reason: Option<String>,
}
#[derive(Debug)]
pub(super) struct Client {
child: Child,
input: ChildStdin,
replies: Receiver<Result<Reply,String>>,
}
pub(super) fn request_key(session: &str, request: &Value) -> String {
serde_json::to_string(&(session,request.to_string())).expect("strings serialize")
}
pub(super) fn response_key(response: &Result<Value,String>) -> String {
serde_json::to_string(response).expect("protocol responses serialize")
}
fn executable() -> PathBuf {
if let Some(path) = std::env::var_os("MARQDO_REPLAY_POLICY") { return path.into() }
if let Ok(exe) = std::env::current_exe() {
if let Some(parent) = exe.parent() {
let adjacent = parent.join("marqdo-replay-policy");
if adjacent.is_file() { return adjacent }
}
}
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("verification/replay-policy/.lake/build/bin/replayPolicy")
}
impl Client {
pub(super) fn start(steps: impl IntoIterator<Item=ProtocolStep>) -> Result<(Self,Reply),String> {
let path=executable();
let mut child=Command::new(&path).stdin(Stdio::piped()).stdout(Stdio::piped())
.stderr(Stdio::null()).spawn().map_err(|e|format!("{}: {e}",path.display()))?;
let input=child.stdin.take().ok_or("policy stdin unavailable")?;
let output=child.stdout.take().ok_or("policy stdout unavailable")?;
let (sender,replies)=mpsc::channel();
std::thread::spawn(move || {
for line in BufReader::new(output).lines() {
let reply=line.map_err(|e|e.to_string())
.and_then(|line|serde_json::from_str(&line).map_err(|e|e.to_string()));
let failed=reply.is_err();
if sender.send(reply).is_err() || failed { break }
}
});
let steps: Vec<_>=steps.into_iter().map(|step|json!({
"q":request_key(&step.session,&step.request),"r":response_key(&step.response)
})).collect();
let mut client=Self {child,input,replies};
let reply=client.exchange(json!({"op":"init","steps":steps}))?;
Ok((client,reply))
}
pub(super) fn exchange(&mut self, command: Value) -> Result<Reply,String> {
serde_json::to_writer(&mut self.input,&command).map_err(|e|e.to_string())?;
self.input.write_all(b"\n").map_err(|e|e.to_string())?;
self.input.flush().map_err(|e|e.to_string())?;
let reply=self.replies.recv_timeout(Duration::from_secs(30))
.map_err(|e|format!("policy response: {e}"))??;
if reply.policy != include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/verification/replay-policy/POLICY-SHA256")).trim() {
return Err("policy executable does not match the compiled source identity".into())
}
if reply.allow == reply.reason.is_some() {
return Err("policy response has inconsistent verdict and reason".into())
}
Ok(reply)
}
}
impl Drop for Client {
fn drop(&mut self) {
let _=self.child.kill();
let _=self.child.wait();
}
}
+269 -10
View File
@@ -1,15 +1,110 @@
//! JSON host primitives + Value↔JSON mapping.
//!
//! Integer gate matches Lean \(V_{\mathsf{safe}}\) (`doc/design/v-safe-json.md`):
//! JSON integers with `|n| > 2^53-1` refuse on encode and decode.
use crate::value::Value;
/// JavaScript `Number.MAX_SAFE_INTEGER` / Lean `ForeignJSON.maxSafe`.
pub const MAX_SAFE_INT: i64 = 9_007_199_254_740_991; // 2^53 - 1
pub fn in_safe_int(n: i64) -> bool {
(-MAX_SAFE_INT..=MAX_SAFE_INT).contains(&n)
}
fn ood_int_err(n: impl std::fmt::Display) -> String {
format!("json: integer outside V_safe (±(2^53-1)): {n}")
}
/// The integer boundary is semantic, not a property of JSON spelling or the
/// host's Int/Num tag. Exponent notation must not bypass it.
fn checked_number(n: f64) -> Result<serde_json::Value, String> {
if !n.is_finite() { return Err("json: non-finite number outside V_safe".into()); }
if n.fract() == 0.0 && n.abs() > MAX_SAFE_INT as f64 { return Err(ood_int_err(n)); }
Ok(serde_json::Value::Number(serde_json::Number::from_f64(n)
.ok_or("json: non-finite number outside V_safe")?))
}
#[derive(Debug)]
pub(crate) enum DecodeError { Syntax(String), Domain(String) }
struct BoundaryValue(Value);
impl<'de> serde::Deserialize<'de> for BoundaryValue {
fn deserialize<D: serde::Deserializer<'de>>(de: D) -> Result<Self, D::Error> {
struct BoundaryVisitor;
impl<'de> serde::de::Visitor<'de> for BoundaryVisitor {
type Value = BoundaryValue;
fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { f.write_str("a boundary JSON value") }
fn visit_unit<E: serde::de::Error>(self) -> Result<Self::Value,E> { Ok(BoundaryValue(Value::None)) }
fn visit_bool<E: serde::de::Error>(self, v: bool) -> Result<Self::Value,E> { Ok(BoundaryValue(Value::Bool(v))) }
fn visit_i64<E: serde::de::Error>(self, n: i64) -> Result<Self::Value,E> {
if !in_safe_int(n) { return Err(E::custom(ood_int_err(n))); }
Ok(BoundaryValue(Value::Int(n)))
}
fn visit_u64<E: serde::de::Error>(self, n: u64) -> Result<Self::Value,E> {
if n > MAX_SAFE_INT as u64 { return Err(E::custom(ood_int_err(n))); }
Ok(BoundaryValue(Value::Int(n as i64)))
}
fn visit_f64<E: serde::de::Error>(self, n: f64) -> Result<Self::Value,E> {
checked_number(n).map_err(E::custom)?;
Ok(BoundaryValue(Value::Num(n)))
}
fn visit_str<E: serde::de::Error>(self, v: &str) -> Result<Self::Value,E> { Ok(BoundaryValue(Value::Text(v.into()))) }
fn visit_string<E: serde::de::Error>(self, v: String) -> Result<Self::Value,E> { Ok(BoundaryValue(Value::Text(v))) }
fn visit_seq<A: serde::de::SeqAccess<'de>>(self, mut seq: A) -> Result<Self::Value,A::Error> {
let mut xs = Vec::new();
while let Some(BoundaryValue(v)) = seq.next_element()? { xs.push(v); }
Ok(BoundaryValue(Value::List(xs)))
}
fn visit_map<A: serde::de::MapAccess<'de>>(self, mut map: A) -> Result<Self::Value,A::Error> {
use serde::de::Error;
let mut fields = Vec::new();
let mut keys = std::collections::HashSet::new();
while let Some(key) = map.next_key::<String>()? {
if !keys.insert(key.clone()) { return Err(A::Error::custom("json: duplicate object key outside V_safe")); }
let BoundaryValue(v) = map.next_value()?;
fields.push((key, v));
}
// Preserve the pre-existing JSON object's canonical key order.
fields.sort_by(|a, b| a.0.cmp(&b.0));
let value = Value::Map(fields);
value_to_json(&value).map_err(A::Error::custom)?;
Ok(BoundaryValue(value))
}
}
de.deserialize_any(BoundaryVisitor)
}
}
pub(crate) fn decode_text(text: &str) -> Result<Value, DecodeError> {
use serde::Deserialize;
let classify = |error: serde_json::Error| {
let message = error.to_string();
// serde_json categorizes its own numeric-overflow diagnostic as Syntax.
// It is a domain refusal for a grammar-valid unbounded JSON number.
// Match this trusted decoder diagnostic, never foreign program output.
if message.starts_with("number out of range at line ") {
return DecodeError::Domain(format!("json: number outside V_safe: {message}"));
}
match error.classify() {
serde_json::error::Category::Data => DecodeError::Domain(message),
_ => DecodeError::Syntax(message),
}
};
let mut de = serde_json::Deserializer::from_str(text);
let value = BoundaryValue::deserialize(&mut de).map_err(classify)?;
de.end().map_err(classify)?;
Ok(value.0)
}
pub fn parse(text: &Value) -> Result<Value, String> {
let s = match text {
Value::Text(s) => s.as_str(),
_ => return Err("json parse needs text".into()),
};
let v: serde_json::Value =
serde_json::from_str(s).map_err(|e| format!("json parse: {e}"))?;
json_to_value(&v)
decode_text(s).map_err(|e| match e {
DecodeError::Syntax(e) | DecodeError::Domain(e) => format!("json parse: {e}"),
})
}
pub fn stringify(value: &Value, indent: Option<&Value>) -> Result<Value, String> {
@@ -70,11 +165,19 @@ pub(crate) fn json_to_value(v: &serde_json::Value) -> Result<Value, String> {
serde_json::Value::Bool(b) => Value::Bool(*b),
serde_json::Value::Number(n) => {
if let Some(i) = n.as_i64() {
if !in_safe_int(i) {
return Err(ood_int_err(i));
}
Value::Int(i)
} else if let Some(u) = n.as_u64() {
// Always > i64::MAX ⇒ outside V_safe; never coerce via f64/Text.
return Err(ood_int_err(u));
} else if let Some(f) = n.as_f64() {
// Host extension: non-integer JSON numbers → Num (not in Lean SafeVal).
checked_number(f)?;
Value::Num(f)
} else {
Value::Text(n.to_string())
return Err(format!("json: unsupported number: {n}"));
}
}
serde_json::Value::String(s) => Value::Text(s.clone()),
@@ -99,10 +202,13 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
Ok(match v {
Value::None => serde_json::Value::Null,
Value::Bool(b) => serde_json::Value::Bool(*b),
Value::Int(n) => serde_json::Value::Number((*n).into()),
Value::Num(n) => serde_json::Number::from_f64(*n)
.map(serde_json::Value::Number)
.unwrap_or(serde_json::Value::Null),
Value::Int(n) => {
if !in_safe_int(*n) {
return Err(ood_int_err(*n));
}
serde_json::Value::Number((*n).into())
}
Value::Num(n) => checked_number(*n)?,
Value::Text(s) => serde_json::Value::String(s.clone()),
Value::List(xs) => {
let mut arr = Vec::with_capacity(xs.len());
@@ -112,6 +218,22 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
serde_json::Value::Array(arr)
}
Value::Map(entries) => {
// Column-table maps (all fields lists): refuse unequal list lengths.
let list_lens: Vec<usize> = entries
.iter()
.filter_map(|(_, v)| match v {
Value::List(xs) => Some(xs.len()),
_ => None,
})
.collect();
if list_lens.len() == entries.len() && list_lens.len() >= 2 {
let n0 = list_lens[0];
if list_lens.iter().any(|&n| n != n0) {
return Err(
"json: table column arity mismatch (unequal list lengths)".into(),
);
}
}
let mut map = serde_json::Map::new();
for (k, v) in entries {
map.insert(k.clone(), value_to_json(v)?);
@@ -127,7 +249,7 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
cells.push(
serde_json::Number::from_f64(n)
.map(serde_json::Value::Number)
.unwrap_or(serde_json::Value::Null),
.ok_or_else(|| "json: non-finite number outside V_safe".to_string())?,
);
}
arr.push(serde_json::Value::Array(cells));
@@ -139,7 +261,7 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
if let crate::formula::Expr::Num(n) = s {
serde_json::Number::from_f64(n)
.map(serde_json::Value::Number)
.unwrap_or(serde_json::Value::Null)
.ok_or_else(|| "json: non-finite number outside V_safe".to_string())?
} else {
serde_json::Value::String(e.as_display())
}
@@ -147,6 +269,9 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
},
Value::Code(c) => serde_json::Value::String(c.source.clone()),
Value::Secret(_) => serde_json::Value::String("<secret>".into()),
Value::Handle(_) => {
return Err("json: session handle cannot be serialized".into());
}
})
}
@@ -159,4 +284,138 @@ mod tests {
let q = quote(&Value::Text("a\"b".into())).unwrap();
assert_eq!(q, Value::Text("\"a\\\"b\"".into()));
}
#[test]
fn roundtrip_max_safe() {
for n in [MAX_SAFE_INT, -MAX_SAFE_INT, 0_i64, 1, -1] {
let j = value_to_json(&Value::Int(n)).unwrap();
let v = json_to_value(&j).unwrap();
assert_eq!(v, Value::Int(n));
}
}
#[test]
fn refuse_ood_encode() {
for n in [
MAX_SAFE_INT + 1,
-MAX_SAFE_INT - 1,
1_i64 << 53, // 2^53
1_i64 << 60, // 2^60
(1_i64 << 60) + 1, // 2^60+1
] {
let err = value_to_json(&Value::Int(n)).unwrap_err();
assert!(
err.contains("outside V_safe"),
"encode {n}: {err}"
);
}
}
#[test]
fn refuse_ood_decode() {
for n in [
MAX_SAFE_INT + 1,
-MAX_SAFE_INT - 1,
1_i64 << 53,
1_i64 << 60,
(1_i64 << 60) + 1,
] {
let raw = format!("{n}");
let j: serde_json::Value = serde_json::from_str(&raw).unwrap();
let err = json_to_value(&j).unwrap_err();
assert!(
err.contains("outside V_safe"),
"decode {n}: {err}"
);
}
}
#[test]
fn refuse_nested_ood() {
let j: serde_json::Value =
serde_json::from_str("{\"a\":[1152921504606846977]}").unwrap();
let err = json_to_value(&j).unwrap_err();
assert!(err.contains("outside V_safe"), "{err}");
}
#[test]
fn decimal_string_ok() {
let j = serde_json::Value::String("1152921504606846977".into());
assert_eq!(
json_to_value(&j).unwrap(),
Value::Text("1152921504606846977".into())
);
}
#[test]
fn float_host_extension_ok() {
let j: serde_json::Value = serde_json::from_str("1.5").unwrap();
match json_to_value(&j).unwrap() {
Value::Num(f) => assert!((f - 1.5).abs() < 1e-12),
other => panic!("expected Num, got {other:?}"),
}
}
#[test]
fn refuse_u64_above_i64_max() {
// 2^63 as JSON integer: as_i64 fails, as_u64 succeeds.
let j: serde_json::Value = serde_json::from_str("9223372036854775808").unwrap();
let err = json_to_value(&j).unwrap_err();
assert!(err.contains("outside V_safe"), "{err}");
}
#[test]
fn refuse_session_handle_encode() {
let err = value_to_json(&Value::Handle("s1".into())).unwrap_err();
assert!(
err.contains("session handle cannot be serialized"),
"{err}"
);
}
#[test]
fn refuse_unequal_column_table_encode() {
let ragged = Value::Map(vec![
("a".into(), Value::List(vec![Value::Int(1), Value::Int(2)])),
("b".into(), Value::List(vec![Value::Int(3)])),
]);
let err = value_to_json(&ragged).unwrap_err();
assert!(err.contains("arity mismatch"), "{err}");
}
#[test]
fn refuse_nonfinite_in_nested_payloads() {
for n in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY] {
for v in [Value::Num(n), Value::List(vec![Value::Num(n)]),
Value::Map(vec![("x".into(), Value::Num(n))])] {
assert!(value_to_json(&v).unwrap_err().contains("outside V_safe"));
}
}
}
#[test]
fn exponent_and_float_tags_cannot_bypass_integer_domain() {
for text in ["9007199254740992.0", "9.007199254740992e15", "1e100", "1e9999", "-9.007199254740992e15"] {
assert!(parse(&Value::Text(text.into())).unwrap_err().contains("outside V_safe"), "{text}");
}
for n in [9_007_199_254_740_992.0, -9_007_199_254_740_992.0, 1e100] {
assert!(value_to_json(&Value::List(vec![Value::Num(n)])).unwrap_err().contains("outside V_safe"));
}
for text in ["1.5", "-0.0", "9.007199254740991e15"] {
assert!(parse(&Value::Text(text.into())).is_ok(), "valid extension {text}");
}
}
#[test]
fn strict_decode_refuses_duplicate_keys_and_ragged_columns() {
for text in [r#"{"a":1,"a":2}"#, r#"[{"x":null,"x":true}]"#,
r#"{"a":[1],"b":[2,3]}"#] {
assert!(matches!(decode_text(text), Err(DecodeError::Domain(_))), "{text}");
}
for text in [r#"{"a":[1],"b":"metadata"}"#, r#"{"a":[],"b":[]}"#,
r#"{"__proto__":{"safe":true},"constructor":"data"}"#] {
assert!(decode_text(text).is_ok(), "{text}");
}
assert!(matches!(decode_text("not JSON"), Err(DecodeError::Syntax(_))));
}
}
+16 -1
View File
@@ -23,6 +23,7 @@ mod hash;
mod html_ops;
pub(crate) mod json;
mod log;
pub(crate) mod oracle;
pub mod math;
mod net;
mod path;
@@ -67,7 +68,7 @@ use std::collections::HashMap;
use std::path::{Path, PathBuf};
/// Host capability flags. Default: all enabled.
#[derive(Debug, Clone)]
#[derive(Debug, Clone, Copy)]
pub struct HostCaps {
pub fs_write: bool,
pub exec: bool,
@@ -106,6 +107,7 @@ pub struct HostContext {
/// LCG state for `random` / `random_int`.
pub rng: u64,
pub plots: Vec<PlotArtifact>,
pub oracle_observations: Vec<crate::value::Value>,
/// lang → interpreter argv (from `set_cmd`).
pub foreign_cmds: HashMap<String, Vec<String>>,
/// Process-local language registrations (`register` / `set_cmd`).
@@ -120,6 +122,12 @@ pub struct HostContext {
/// Monotonic session id counter.
#[cfg(feature = "exec-host")]
pub foreign_session_seq: u64,
#[cfg(feature = "exec-host")]
pub foreign_protocol_steps: Vec<foreign_session::ProtocolStep>,
#[cfg(feature = "exec-host")]
pub foreign_replay_expected: Option<foreign_session::validation::ReplayCursor>,
#[cfg(feature = "exec-host")]
pub foreign_replay_divergence: Option<String>,
/// Loaded native plugins (ABI v1/v2).
pub plugins: PluginState,
/// Entry `.mq.md` source text (for ABI host_query / context injection).
@@ -154,6 +162,7 @@ impl Default for HostContext {
sleep_limit_ms: Some(30_000),
rng: 0xC0FF_EE42_CAFE_BABE,
plots: Vec::new(),
oracle_observations: Vec::new(),
foreign_cmds: HashMap::new(),
foreign_specs: HashMap::new(),
foreign_config_specs: HashMap::new(),
@@ -162,6 +171,12 @@ impl Default for HostContext {
foreign_sessions: HashMap::new(),
#[cfg(feature = "exec-host")]
foreign_session_seq: 0,
#[cfg(feature = "exec-host")]
foreign_protocol_steps: Vec::new(),
#[cfg(feature = "exec-host")]
foreign_replay_expected: None,
#[cfg(feature = "exec-host")]
foreign_replay_divergence: None,
plugins: PluginState::default(),
entry_source: None,
entry_path: None,
+136
View File
@@ -0,0 +1,136 @@
//! Document-level observations, separate from process timing and execution logs.
use crate::value::Value;
use super::HostContext;
pub(crate) fn admit(value: &Value) -> Result<(), String> {
match value {
Value::None | Value::Bool(_) | Value::Text(_) => Ok(()),
Value::Int(_) => super::json::value_to_json(value).map(|_| ()),
Value::Num(_) => super::json::value_to_json(value).map(|_| ()),
Value::List(xs) => { for x in xs { admit(x)?; } Ok(()) }
Value::Map(xs) => {
super::json::value_to_json(value)?;
let mut keys = std::collections::HashSet::new();
for (key, x) in xs {
if !keys.insert(key) { return Err("oracle: duplicate record field".into()); }
admit(x)?;
}
Ok(())
}
_ => Err("oracle: value outside V_safe".into()),
}
}
pub(crate) fn append(ctx: &mut HostContext, channel: &str, value: &Value) -> Result<(), String> {
admit(value)?;
ctx.oracle_observations.push(Value::Map(vec![
("index".into(), Value::Int(ctx.oracle_observations.len() as i64)),
("channel".into(), Value::Text(channel.into())),
("value".into(), value.clone()),
]));
Ok(())
}
/// Project a process result to its semantic observation. Success observes the
/// returned payload. Failure observes stable diagnostics, never elapsed time,
/// command paths or opaque handles. Both document and live runners use this.
pub(crate) fn eval_observation(record: &Value) -> Result<Value, String> {
let Value::Map(fields) = record else { return Err("oracle: expected eval record".into()) };
let get = |key: &str| fields.iter().find(|(k, _)| k == key).map(|(_,v)|v.clone()).unwrap_or(Value::None);
let value = match get("ok") {
Value::Bool(true) => get("result"),
Value::Bool(false) => {
let source = get("diagnostic_source");
Value::Map(["ok", "code", "stdout", "stderr", "error", "timed_out", "alive"]
.into_iter().map(|key| {
let value = get(key);
// Alpha-rename only this execution's allocated source path,
// only in diagnostics. Keep raw output and ordinary paths intact.
let value = match (key, value, &source) {
("stderr" | "error", Value::Text(text), Value::Text(path)) if !path.is_empty() =>
Value::Text(text.replace(path, "<foreign-source>")),
(_, value, _) => value,
};
(key.to_string(), value)
}).collect())
},
_ => return Err("oracle: eval record missing boolean ok".into()),
};
admit(&value)?;
Ok(value)
}
fn validate(journal: &Value) -> Result<&[Value], String> {
let Value::List(rows) = journal else { return Err("oracle journal must be a list".into()) };
for (i, row) in rows.iter().enumerate() {
let Value::Map(fields) = row else { return Err(format!("oracle step {i}: expected record")) };
if fields.len() != 3 { return Err(format!("oracle step {i}: expected index/channel/value")); }
let get = |k: &str| fields.iter().find(|(key,_)| key == k).map(|(_,v)|v);
if get("index") != Some(&Value::Int(i as i64)) { return Err(format!("oracle step {i}: invalid index")); }
if !matches!(get("channel"), Some(Value::Text(s)) if ["eval", "get", "decode"].contains(&s.as_str())) {
return Err(format!("oracle step {i}: invalid channel"));
}
admit(get("value").ok_or_else(|| format!("oracle step {i}: missing value"))?)?;
}
Ok(rows)
}
/// Pure structural equality on admitted journals; reports the least differing index.
pub(crate) fn replay_check(expected: &Value, observed: &Value) -> Result<Value, String> {
let a = validate(expected)?;
let b = validate(observed)?;
// Map field order is not part of record identity (matches JSON object semantics).
let canonical = |v: &Value| super::json::value_to_json(v);
let mut mismatch = None;
for i in 0..a.len().max(b.len()) {
if a.get(i).map(canonical).transpose()? != b.get(i).map(canonical).transpose()? {
mismatch = Some(i);
break;
}
}
let index = mismatch.map(|i| Value::Int(i as i64)).unwrap_or(Value::None);
Ok(Value::Map(vec![
("ok".into(), Value::Bool(mismatch.is_none())),
("index".into(), index),
("expected".into(), mismatch.and_then(|i| a.get(i)).cloned().unwrap_or(Value::None)),
("observed".into(), mismatch.and_then(|i| b.get(i)).cloned().unwrap_or(Value::None)),
]))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn oracle_journal_structural_mismatch_and_length() {
let mut ctx = HostContext::default();
append(&mut ctx, "eval", &Value::Int(3)).unwrap();
append(&mut ctx, "get", &Value::Int(2)).unwrap();
let journal = Value::List(ctx.oracle_observations.clone());
let result = replay_check(&journal, &journal).unwrap();
assert_eq!(super::super::json::get(&result, &Value::Text("ok".into())).unwrap(), Value::Bool(true));
let short = Value::List(ctx.oracle_observations[..1].to_vec());
let result = replay_check(&journal, &short).unwrap();
assert_eq!(super::super::json::get(&result, &Value::Text("index".into())).unwrap(), Value::Int(1));
assert!(append(&mut ctx, "eval", &Value::List(vec![Value::Handle("x".into())])).is_err());
let code = Value::Code(crate::value::CodeBlock { lang: "python".into(), source: "1".into() });
assert!(admit(&Value::List(vec![code])).is_err());
assert_eq!(ctx.oracle_observations.len(), 2);
}
#[test]
fn diagnostic_projection_renames_only_allocated_source_identity() {
let make = |path: &str, message: &str| Value::Map(vec![
("ok".into(), Value::Bool(false)),
("stderr".into(), Value::Text(format!("File {path}: {message}; input /data/original.csv"))),
("diagnostic_source".into(), Value::Text(path.into())),
]);
let a = make("/tmp/allocated-a.py", "ValueError");
let b = make("/tmp/allocated-b.py", "ValueError");
assert_ne!(a, b, "raw diagnostics must survive");
assert_eq!(eval_observation(&a).unwrap(), eval_observation(&b).unwrap());
let c = make("/tmp/allocated-b.py", "TypeError");
assert_ne!(eval_observation(&a).unwrap(), eval_observation(&c).unwrap());
let d = make("/tmp/allocated-b.py", "ValueError /data/changed.csv");
assert_ne!(eval_observation(&a).unwrap(), eval_observation(&d).unwrap());
}
}
+11
View File
@@ -194,6 +194,17 @@ impl Clone for PluginState {
}
impl PluginState {
/// Exercise the real JSON/C ABI without loading an external shared library.
#[cfg(test)]
pub(crate) fn register_binding_probe(&mut self) {
unsafe extern "C" fn echo(args: *const c_char, out: *mut *mut c_char, _err: *mut *mut c_char) -> c_int {
if args.is_null() || out.is_null() { return 1; }
let Ok(input)=CStr::from_ptr(args).to_str() else { return 1; };
*out=host_strdup(input);
if (*out).is_null() {1} else {0}
}
self.fns.insert("binding_probe".into(),RegisteredFn{params:vec!["a".into(),"b".into()],fn_ptr:echo});
}
pub fn get(&self, name: &str) -> Option<&RegisteredFn> {
self.fns.get(name)
}
+79 -85
View File
@@ -1,107 +1,101 @@
//! Object inheritance: `# Child = > Parent` validation and helpers.
use std::collections::{HashMap, HashSet};
use std::collections::HashSet;
use anyhow::Result;
use crate::ast::{Function, Module};
use crate::diagnostics::{Diagnostic, Span};
use crate::diagnostics::Diagnostic;
/// Import binding paths identify module instances, including repeated imports
/// of the same file under different bindings. The tree remains immutable.
pub(crate) fn module_namespace(root:&Module, wanted:&Module) -> Option<Vec<String>> {
if std::ptr::eq(root,wanted) { return Some(Vec::new()); }
let mut imports=root.import_modules.iter().collect::<Vec<_>>();
imports.sort_by_key(|(name,_)| *name);
for (bind,module) in imports {
if let Some(mut path)=module_namespace(module,wanted) {
path.insert(0,bind.clone()); return Some(path);
}
}
None
}
pub(crate) fn object_at_namespace<'a>(root:&'a Module,path:&[String],name:&str) -> Option<(&'a Module,&'a Function)> {
let mut owner=root;
for segment in path { owner=owner.import_modules.get(segment)?; }
let object=owner.functions.iter().find(|fun|fun.is_object() && fun.name==name)?;
Some((owner,object))
}
/// Bare types prefer their defining module. An imported short-name binding
/// resolves its explicit path; a legacy imported bare name must be unique.
pub(crate) fn resolve_object<'a>(owner:&'a Module,name:&str) -> Result<(&'a Module,&'a Function),String> {
if let Some(alias)=owner.uses.iter().find(|binding|binding.bind==name) {
let Some((class,path))=alias.path.split_last() else { return Err("empty object alias".into()); };
return object_at_namespace(owner,path,class).ok_or_else(||format!("unknown object type `{name}`"));
}
if let Some(object)=owner.functions.iter().find(|fun|fun.is_object() && fun.name==name) { return Ok((owner,object)); }
fn imported<'a>(module:&'a Module,name:&str,out:&mut Vec<(&'a Module,&'a Function)>) {
let mut imports=module.import_modules.iter().collect::<Vec<_>>();imports.sort_by_key(|(name,_)|*name);
for (_,child) in imports {
out.extend(child.functions.iter().filter(|fun|fun.is_object() && fun.name==name).map(|fun|(child,fun)));
imported(child,name,out);
}
}
let mut candidates=Vec::new(); imported(owner,name,&mut candidates);
match candidates.as_slice() {
[only]=>Ok(*only),
[]=>Err(format!("unknown object type `{name}`")),
_=>Err(format!("ambiguous object type `{name}`; use an explicit import binding")),
}
}
/// Validate that every object's `base` resolves and that the inheritance graph is acyclic.
pub fn validate_inheritance(module: &Module) -> Result<()> {
let mut by_name: HashMap<String, (Span, Option<String>)> = HashMap::new();
collect_objects(module, &mut by_name);
// Deterministic order: source line, then name (stable gold diagnostics).
let mut names: Vec<String> = by_name.keys().cloned().collect();
names.sort_by(|a, b| {
let (sa, _) = &by_name[a];
let (sb, _) = &by_name[b];
sa.line.cmp(&sb.line).then_with(|| a.cmp(b))
});
for name in &names {
let (span, base) = &by_name[name];
let Some(base) = base else { continue };
if !by_name.contains_key(base) {
return Err(Diagnostic::new(
None,
*span,
format!("unknown base type `{base}` for object `{name}`"),
)
.into());
}
fn modules<'a>(module: &'a Module, out: &mut Vec<&'a Module>) {
out.push(module);
let mut imports=module.import_modules.iter().collect::<Vec<_>>();
imports.sort_by_key(|(name,_)| *name);
for (_, child) in imports { modules(child,out); }
}
for name in &names {
let (span, base) = &by_name[name];
if base.is_none() {
continue;
}
let mut path = Vec::new();
let mut cur = name.clone();
loop {
if path.iter().any(|p| p == &cur) {
path.push(cur);
return Err(Diagnostic::new(
None,
*span,
format!(
"cyclic inheritance involving `{name}` ({})",
path.join(" → ")
),
)
.into());
let mut all=Vec::new(); modules(module,&mut all);
for owner in all {
for object in owner.functions.iter().filter(|fun|fun.is_object()) {
let mut current_owner=owner;
let mut current=object;
let mut seen=HashSet::new();
let mut chain=Vec::new();
loop {
chain.push(current.name.clone());
if !seen.insert((current_owner as *const Module,current as *const Function)) {
return Err(Diagnostic::new(owner.source.path.as_deref(),object.span,
format!("cyclic inheritance involving `{}` ({})",object.name,chain.join(" → ")))
.with_source(&owner.source).into());
}
let Some(base)=&current.base else { break; };
(current_owner,current)=resolve_object(current_owner,base).map_err(|message| {
Diagnostic::new(current_owner.source.path.as_deref(),current.span,
format!("{} base type `{base}` for object `{}`: {message}",if message.starts_with("ambiguous") { "ambiguous" } else { "unknown" },current.name))
.with_source(&current_owner.source)
})?;
}
path.push(cur.clone());
let Some((_, Some(next))) = by_name.get(&cur) else {
break;
};
cur = next.clone();
}
}
Ok(())
}
fn collect_objects(module: &Module, out: &mut HashMap<String, (Span, Option<String>)>) {
for fun in &module.functions {
if fun.is_object() {
out.entry(fun.name.clone())
.or_insert((fun.span, fun.base.clone()));
}
}
for lib in module.import_modules.values() {
collect_objects(lib, out);
}
}
/// Walk the base chain starting at `type_name` (inclusive).
pub fn inheritance_chain<'a>(module: &'a Module, type_name: &str) -> Vec<&'a Function> {
let mut out = Vec::new();
let mut cur = type_name.to_string();
let mut seen = HashSet::new();
while seen.insert(cur.clone()) {
let Some(obj) = find_object(module, &cur) else {
break;
};
out.push(obj);
match &obj.base {
Some(b) => cur = b.clone(),
None => break,
}
let mut out=Vec::new();
let Ok((mut owner,mut object))=resolve_object(module,type_name) else { return out; };
let mut seen=HashSet::new();
while seen.insert((owner as *const Module,object as *const Function)) {
out.push(object);
let Some(base)=&object.base else { break; };
let Ok(next)=resolve_object(owner,base) else { break; };
(owner,object)=next;
}
out
}
fn find_object<'a>(module: &'a Module, name: &str) -> Option<&'a Function> {
module
.functions
.iter()
.find(|f| f.name == name && f.is_object())
.or_else(|| {
module.import_modules.values().find_map(|lib| {
lib.functions
.iter()
.find(|f| f.name == name && f.is_object())
})
})
}
+339 -259
View File
@@ -6,7 +6,6 @@ use std::path::{Path, PathBuf};
use crate::ast::{
Arg, BinaryOp, CallExpr, Expr, Function, IndexKey, InterpPart, Literal, Module, Param, Stmt,
UnaryOp,
};
use crate::builtin::{
builtin_at, builtin_footnote_get, builtin_int, builtin_iter_items, builtin_join,
@@ -14,7 +13,7 @@ use crate::builtin::{
builtin_type,
};
use crate::debug::{emit_trace, DebugController};
use crate::diagnostics::{bail_at, Span};
use crate::diagnostics::{Diagnostic, DiagnosticFrame, SourceIdentity, Span};
use crate::host::{call_host, HostContext};
use crate::input_feed::InputFeed;
use crate::capture::{
@@ -39,6 +38,9 @@ pub struct Interpreter {
/// Active loops: `(loop_id, iter_index_0based)`.
loop_stack: Vec<(u32, u32)>,
current_span: Span,
current_source: SourceIdentity,
current_module: Option<*const Module>,
diagnostic_frames: Vec<DiagnosticFrame>,
input: InputFeed,
host: HostContext,
debug: Option<Arc<DebugController>>,
@@ -100,6 +102,9 @@ impl Interpreter {
next_loop_id: 1,
loop_stack: Vec::new(),
current_span: Span::new(1, 1),
current_source: SourceIdentity { path: path.map(Path::to_path_buf), sha256: None },
current_module: None,
diagnostic_frames: Vec::new(),
input: InputFeed::new(false, Vec::new()),
host: HostContext::for_run(path, Default::default(), Vec::new()),
debug: None,
@@ -121,6 +126,9 @@ impl Interpreter {
next_loop_id: 1,
loop_stack: Vec::new(),
current_span: Span::new(1, 1),
current_source: SourceIdentity { path: path.map(Path::to_path_buf), sha256: None },
current_module: None,
diagnostic_frames: Vec::new(),
input: InputFeed::new(true, Vec::new()),
host: HostContext::for_capture(path, Default::default()),
debug: None,
@@ -147,8 +155,37 @@ impl Interpreter {
std::mem::take(&mut self.host.plots)
}
#[cfg(all(test, feature = "exec-host"))]
pub(crate) fn foreign_source_evidence(&mut self) -> (Vec<Value>, Vec<Value>, Vec<crate::host::foreign_session::ProtocolStep>) {
crate::source_refinement_tests::boundary_evidence(&mut self.host)
}
#[cfg(all(test, feature = "exec-host"))]
pub(crate) fn application_evidence(&self) -> (Vec<Value>, Vec<crate::host::foreign_session::ProtocolStep>, usize) {
(self.host.oracle_observations.clone(), self.host.foreign_protocol_steps.clone(), self.host.foreign_sessions.len())
}
#[cfg(all(test, feature = "exec-host"))]
pub(crate) fn causal_replay_evidence(&mut self) -> serde_json::Value {
crate::causal_replay_tests::evidence(&mut self.host)
}
#[cfg(test)]
pub(crate) fn diagnostic_scope_state(&self) -> (SourceIdentity, usize, Vec<String>, Vec<u32>, bool) {
(self.current_source.clone(), self.diagnostic_frames.len(), self.host.call_stack.clone(),
self.host.call_site_lines.clone(), self.current_module.is_some())
}
fn err(&self, message: impl Into<String>) -> anyhow::Error {
bail_at(self.path.as_deref(), self.current_span, message)
Diagnostic::new(self.current_source.path.as_deref(), self.current_span, message)
.with_source(&self.current_source).with_frames(&self.diagnostic_frames).into()
}
fn err_code(&self, code: &str, message: impl Into<String>) -> anyhow::Error {
anyhow::Error::new(
Diagnostic::new(self.current_source.path.as_deref(), self.current_span, message)
.with_source(&self.current_source).with_frames(&self.diagnostic_frames).with_code(code),
)
}
fn emit_line(&mut self, text: &str) {
@@ -360,7 +397,92 @@ impl Interpreter {
fun: &Function,
args: &[(String, Value)],
) -> Result<Value> {
self.run_function(module, fun, Env::new(), args, false)
let previous=self.site_module;
self.site_module=Some(module as *const Module);
let result=self.run_function(module, fun, Env::new(), args, false);
self.site_module=previous;
result
}
/// Bind named arguments for an external call. Unknown names, missing
/// required parameters, and contract mismatches are errors. Defaults run.
pub fn bind_external_args(
&mut self,
module: &Module,
fun: &Function,
args: &[(String, Value)],
) -> Result<Vec<(String, Value)>> {
let previous=self.site_module;
self.site_module=Some(module as *const Module);
let result=self.in_function_source(module, fun, |this| this.bind_external_args_inner(module, fun, args));
self.site_module=previous;
result
}
fn bind_external_args_inner(
&mut self,
module: &Module,
fun: &Function,
args: &[(String, Value)],
) -> Result<Vec<(String, Value)>> {
self.current_span = fun.span;
self.host.current_line = fun.span.line;
let mut named: HashMap<String, Value> = HashMap::new();
for (k, v) in args {
if named.insert(k.clone(), v.clone()).is_some() {
return Err(self.err_code(
"call.duplicate_arg",
format!("duplicate named argument `{k}`"),
));
}
}
for k in named.keys() {
if !fun.params.iter().any(|p| &p.name == k) {
return Err(self.err_code(
"call.unknown_arg",
format!("unknown argument `{k}`"),
));
}
}
let ev: Vec<EvArg> = named
.into_iter()
.map(|(k, v)| EvArg::Named(k, v))
.collect();
let mut env = Env::new();
let bound = bind_function_args(self, module, fun, &mut env, &fun.params, &ev, false)
.map_err(|error| {
if Diagnostic::find(&error).is_some() { return error; }
let m = error.to_string();
if m.starts_with("missing argument") {
self.err_code("call.missing_arg", m)
} else if m.starts_with("duplicate named argument") {
self.err_code("call.duplicate_arg", m)
} else {
self.err(m)
}
})?;
if let Some(c) = &fun.contract {
if let Some(d) = crate::contract::call_arg_diag(
c,
&fun.name,
&bound,
self.current_source.path.as_deref(),
fun.span,
) {
return Err(d.into());
}
}
let mut out = Vec::new();
for p in &fun.params {
if let Some(v) = bound.get(&p.name) {
out.push((p.name.clone(), v.clone()));
}
}
Ok(out)
}
fn argument_error(&self, error: anyhow::Error) -> anyhow::Error {
if Diagnostic::find(&error).is_some() { error } else { self.err(error.to_string()) }
}
/// Call a function using (and updating) [`Self::entry_bindings`] from `# main`.
@@ -619,6 +741,59 @@ impl Interpreter {
}
fn run_function(
&mut self,
module: &Module,
fun: &Function,
env: Env,
args: &[(String, Value)],
capture_entry_env: bool,
) -> Result<Value> {
self.in_function_source(module, fun, |this| {
this.run_function_inner(module, fun, env, args, capture_entry_env)
})
}
fn in_function_source<T>(
&mut self,
module: &Module,
fun: &Function,
run: impl FnOnce(&mut Self) -> Result<T>,
) -> Result<T> {
let mut source = module.source.clone();
if source.path.is_none() {
if self.diagnostic_frames.is_empty() {
source.path = self.path.clone();
} else if self.current_module == Some(module as *const Module) {
source.path = self.current_source.path.clone();
}
}
let caller = (!self.diagnostic_frames.is_empty()).then(|| self.current_source.at(self.current_span));
let previous_source = std::mem::replace(&mut self.current_source, source);
let previous_module = self.current_module.replace(module as *const Module);
let previous_span = std::mem::replace(&mut self.current_span, fun.span);
let previous_line = self.host.current_line;
self.diagnostic_frames.push(DiagnosticFrame {
function: fun.name.clone(), declaration: self.current_source.at(fun.span), call_site: caller,
});
let result = run(self)
.map_err(|error| {
if let Some(mut diagnostic) = Diagnostic::find(&error).cloned() {
if diagnostic.source_sha256.is_none() && diagnostic.path == self.current_source.path {
diagnostic.source_sha256 = self.current_source.sha256.clone();
}
if diagnostic.frames.is_empty() { diagnostic.frames = self.diagnostic_frames.clone(); }
diagnostic.into()
} else { self.err(error.to_string()) }
});
self.diagnostic_frames.pop();
self.current_source = previous_source;
self.current_module = previous_module;
self.current_span = previous_span;
self.host.current_line = previous_line;
result
}
fn run_function_inner(
&mut self,
module: &Module,
fun: &Function,
@@ -628,7 +803,7 @@ impl Interpreter {
) -> Result<Value> {
if self.trace {
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(fun.span),
"enter_fn",
&[("fn", fun.name.as_str())],
@@ -637,12 +812,23 @@ impl Interpreter {
for (k, v) in args {
env.set(k.clone(), v.clone());
}
// Raw/named entry APIs and document metadata must use the same default
// semantics as ordinary calls. Already bound defaults are supplied
// values here and therefore never execute a second time.
let supplied=env.vars.iter().map(|(name,value)|EvArg::Named(name.clone(),value.clone())).collect::<Vec<_>>();
let completed=bind_function_args(self,module,fun,&mut env,&fun.params,&supplied,true)?;
for (name,value) in completed { env.set(name,value); }
for p in &fun.params {
if env.get(&p.name).is_none() {
env.set(p.name.clone(), Value::None);
}
}
if let Some(contract)=&fun.contract {
if let Some(diag)=crate::contract::call_arg_diag(contract,&fun.name,&env.vars,self.current_source.path.as_deref(),fun.span) {
return Err(diag.into());
}
}
let ret = Value::None;
self.push_view_call(fun, &env);
self.host.push_call_frame(&fun.name);
@@ -663,7 +849,7 @@ impl Interpreter {
if self.trace {
let display = ret.as_display();
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(fun.span),
"leave_fn",
&[("fn", fun.name.as_str()), ("value", display.as_str())],
@@ -693,7 +879,7 @@ impl Interpreter {
// 契约边界 3(T2.2):表绑定 vs 字段契约。
if let Some(c) = fun.var_contracts.get(name) {
if let Some(d) =
crate::contract::fields_diag(c, name, &v, self.path.as_deref(), *span)
crate::contract::fields_diag(c, name, &v, self.current_source.path.as_deref(), *span)
{
return Err(d.into());
}
@@ -701,7 +887,7 @@ impl Interpreter {
if self.trace {
let display = v.as_display();
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(*span),
"stmt",
&[
@@ -732,7 +918,7 @@ impl Interpreter {
// 契约边界 2(T2.2):返回值 vs 返回契约。
if let Some(c) = &fun.contract {
if let Some(d) =
crate::contract::return_diag(c, &fun.name, &v, self.path.as_deref(), *span)
crate::contract::return_diag(c, &fun.name, &v, self.current_source.path.as_deref(), *span)
{
return Err(d.into());
}
@@ -740,7 +926,7 @@ impl Interpreter {
if self.trace {
let display = v.as_display();
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(*span),
"stmt",
&[("kind", "return"), ("value", display.as_str())],
@@ -764,7 +950,7 @@ impl Interpreter {
self.current_span = *span;
if self.trace {
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(*span),
"stmt",
&[("kind", "call"), ("callee", call.callee.as_str())],
@@ -796,7 +982,7 @@ impl Interpreter {
self.current_span = *span;
if self.trace {
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(*span),
"stmt",
&[("kind", "expr")],
@@ -832,7 +1018,7 @@ impl Interpreter {
self.current_span = *span;
if self.trace {
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(*span),
"stmt",
&[("kind", "branch")],
@@ -865,7 +1051,7 @@ impl Interpreter {
self.current_span = *span;
if self.trace {
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(*span),
"stmt",
&[("kind", "while")],
@@ -877,7 +1063,10 @@ impl Interpreter {
None
};
let mut guard = 0u32;
while self.eval_expr(module, fun, env, condition)?.truthy() {
loop {
self.current_span=*span;
self.host.current_line=span.line;
if !self.eval_expr(module,fun,env,condition)?.truthy(){break;}
guard += 1;
if guard > 1_000_000 {
return Err(self.err("while loop exceeded iteration limit"));
@@ -915,7 +1104,7 @@ impl Interpreter {
self.current_span = *span;
if self.trace {
emit_trace(
self.path.as_deref(),
self.current_source.path.as_deref(),
Some(*span),
"stmt",
&[
@@ -1044,15 +1233,8 @@ impl Interpreter {
Ok(Value::Text(s))
}
Expr::Unary { op, expr } => {
let v = self.eval_expr(module, fun, env, expr)?;
match op {
UnaryOp::Not => Ok(Value::Bool(!v.truthy())),
UnaryOp::Neg => match v {
Value::Int(n) => Ok(Value::Int(-n)),
Value::Num(n) => Ok(Value::Num(-n)),
_ => Err(self.err("unary `-` needs int or num")),
},
}
let value=self.eval_expr(module,fun,env,expr)?;
crate::builtin::eval_unary(*op,&value).map_err(|message|self.err(message))
}
Expr::Binary { op, left, right } => {
let l = self.eval_expr(module, fun, env, left)?;
@@ -1084,7 +1266,7 @@ impl Interpreter {
{
return Ok(v);
}
eval_binary(*op, &l, &r).map_err(|e| self.err(e.to_string()))
crate::builtin::eval_binary(*op, &l, &r).map_err(|e| self.err(e.to_string()))
}
Expr::Call(call) => self.eval_call(module, fun, env, call),
Expr::List(items) => {
@@ -1113,7 +1295,7 @@ impl Interpreter {
name,
&key,
&Value::None,
self.path.as_deref(),
self.current_source.path.as_deref(),
self.current_span,
) {
return Err(d.into());
@@ -1128,7 +1310,7 @@ impl Interpreter {
name,
&key,
&out,
self.path.as_deref(),
self.current_source.path.as_deref(),
self.current_span,
) {
return Err(d.into());
@@ -1290,21 +1472,21 @@ impl Interpreter {
fn eval_bound_function_call(
&mut self,
module: &Module,
caller: &Function,
env: &mut Env,
_caller: &Function,
_env: &mut Env,
target: &Function,
ev_args: &[EvArg],
) -> Result<Value> {
let bound =
bind_function_args(self, module, caller, env, &target.params, ev_args, false)
.map_err(|m| self.err(m))?;
bind_function_args(self, module, target, &mut Env::new(), &target.params, ev_args, false)
.map_err(|e| self.argument_error(e))?;
if let Some(c) = &target.contract {
if let Some(d) = crate::contract::call_arg_diag(
if let Some(d) = crate::contract::call_arg_diag_in(
c,
&target.name,
&bound,
self.path.as_deref(),
self.current_span,
&self.current_source.at(self.current_span),
&module.source,
) {
return Err(d.into());
}
@@ -1318,7 +1500,7 @@ impl Interpreter {
self.host.pop_call_site_line();
let result = result?;
if target.is_object() {
Ok(tag_instance(&target.name, result))
Ok(crate::value::tag_instance(&target.name, &self.object_module_path(module), result))
} else {
Ok(result)
}
@@ -1450,36 +1632,7 @@ impl Interpreter {
path: &[String],
ev_args: &[EvArg],
) -> Result<Value> {
let display = path.join(".");
if path.len() < 2 {
return Err(self.err(format!("library path `{display}`: need at least `lib.member`")));
}
let lib_name = &path[0];
let lib = module.import_modules.get(lib_name).ok_or_else(|| {
self.err(format!("unknown library `{lib_name}` (import it in frontmatter)"))
})?;
let mut parent_is_object = false;
let mut node = lib
.functions
.iter()
.find(|f| f.name == path[1])
.ok_or_else(|| self.err(format!("unknown `{display}`")))?;
for seg in &path[2..] {
parent_is_object = node.is_object();
node = node
.children
.iter()
.find(|c| c.name == *seg)
.ok_or_else(|| self.err(format!("unknown `{display}`")))?;
}
if parent_is_object && !node.is_object() && crate::ast::function_uses_receiver(node) {
return Err(self.err(format!(
"`{display}` is an instance method; call it as `var`.{}",
node.name
)));
}
let (lib, node) = locate_import_function(module, path).map_err(|m| self.err(m))?;
let bound = {
let mut dummy_env = Env::new();
@@ -1492,16 +1645,16 @@ impl Interpreter {
ev_args,
false,
)
.map_err(|m| self.err(m))?
.map_err(|e| self.argument_error(e))?
};
// 契约边界 1(T2.2):库路径调用的实参 vs 形参契约。
if let Some(c) = &node.contract {
if let Some(d) = crate::contract::call_arg_diag(
if let Some(d) = crate::contract::call_arg_diag_in(
c,
&display,
&node.name,
&bound,
self.path.as_deref(),
self.current_span,
&self.current_source.at(self.current_span),
&lib.source,
) {
return Err(d.into());
}
@@ -1516,7 +1669,7 @@ impl Interpreter {
self.host.pop_call_site_line();
let result = result?;
if node.is_object() {
Ok(tag_instance(&node.name, result))
Ok(crate::value::tag_instance(&node.name, &self.object_module_path(lib), result))
} else {
Ok(result)
}
@@ -1525,7 +1678,7 @@ impl Interpreter {
fn eval_method_call(
&mut self,
module: &Module,
fun: &Function,
_fun: &Function,
env: &mut Env,
recv_name: &str,
method: &str,
@@ -1536,12 +1689,19 @@ impl Interpreter {
.cloned()
.ok_or_else(|| self.err(format!("undefined variable `{recv_name}`")))?;
let type_name = instance_type_name(&recv).map_err(|m| self.err(m))?;
if find_object_type(module, &type_name).is_none() {
return Err(self.err(format!("unknown object type `{type_name}`")));
}
let (owner, _obj, target) = find_method_along_bases(module, &type_name, method)
// The active root is immutable for this invocation; site_module is
// installed/restored by the public run/call entry points.
let root=unsafe { self.site_module.map(|root| &*root) }.unwrap_or(module);
let namespace=crate::value::instance_module(&recv).map_err(|m|self.err(m))?;
let (class_owner,class)=if let Some(namespace)=namespace {
crate::inherit::object_at_namespace(root,&namespace,&type_name)
.ok_or_else(||self.err(format!("unknown object type `{type_name}` in module `{}`",namespace.join("."))))?
} else {
crate::inherit::resolve_object(module,&type_name).map_err(|m|self.err(m))?
};
let (owner, _obj, target) = find_method_along_bases(class_owner, class, method)
.ok_or_else(|| {
let chain = crate::inherit::inheritance_chain(module, &type_name)
let chain = crate::inherit::inheritance_chain(class_owner, &type_name)
.into_iter()
.map(|o| o.name.as_str())
.collect::<Vec<_>>()
@@ -1550,16 +1710,19 @@ impl Interpreter {
"unknown method `{type_name}.{method}` (searched {chain})"
))
})?;
let bound = bind_function_args(self, module, fun, env, &target.params, ev_args, false)
.map_err(|m| self.err(m))?;
let mut receiver_env = Env::new();
receiver_env.set("self".into(), recv.clone());
receiver_env.set("自".into(), recv.clone());
let bound = bind_function_args(self, owner, target, &mut receiver_env, &target.params, ev_args, false)
.map_err(|e| self.argument_error(e))?;
// 契约边界 1(T2.2):方法调用的实参 vs 形参契约。
if let Some(c) = &target.contract {
if let Some(d) = crate::contract::call_arg_diag(
if let Some(d) = crate::contract::call_arg_diag_in(
c,
&target.name,
&bound,
self.path.as_deref(),
self.current_span,
&self.current_source.at(self.current_span),
&owner.source,
) {
return Err(d.into());
}
@@ -1575,11 +1738,16 @@ impl Interpreter {
self.host.pop_call_site_line();
let result = result?;
if crate::linalg_ops::is_matrix_value(&recv) {
Ok(crate::linalg_ops::retag_matrix_like(&type_name, result))
Ok(crate::value::preserve_instance_module(&recv,crate::linalg_ops::retag_matrix_like(&type_name, result)))
} else {
Ok(result)
}
}
fn object_module_path(&self, module:&Module) -> Vec<String> {
let root=unsafe { self.site_module.map(|root| &*root) }.unwrap_or(module);
crate::inherit::module_namespace(root,module).unwrap_or_default()
}
}
fn instance_type_name(v: &Value) -> std::result::Result<String, String> {
@@ -1596,33 +1764,54 @@ fn instance_type_name(v: &Value) -> std::result::Result<String, String> {
}
}
fn tag_instance(type_name: &str, value: Value) -> Value {
match value {
Value::Map(mut entries) => {
// Always stamp the constructed type so explicit `` `self` = > Parent ``
// in a child constructor still yields the most specific `_type`.
entries.retain(|(k, _)| k != "_type");
entries.insert(0, ("_type".into(), Value::Text(type_name.into())));
Value::Map(entries)
}
Value::None => Value::Map(vec![("_type".into(), Value::Text(type_name.into()))]),
other => Value::Map(vec![
("_type".into(), Value::Text(type_name.into())),
("value".into(), other),
]),
/// `lib.member` / `lib.Type.member` against frontmatter imports. Does not execute.
pub(crate) fn locate_import_function<'a>(
module: &'a Module,
path: &[String],
) -> Result<(&'a Module, &'a Function), String> {
let display = path.join(".");
if path.len() < 2 {
return Err(format!(
"library path `{display}`: need at least `lib.member`"
));
}
let lib = module
.import_modules
.get(&path[0])
.ok_or_else(|| format!("unknown library `{}` (import it in frontmatter)", path[0]))?;
let mut parent_is_object = false;
let mut node = lib
.functions
.iter()
.find(|f| f.name == path[1])
.ok_or_else(|| format!("unknown `{display}`"))?;
for seg in &path[2..] {
parent_is_object = node.is_object();
node = node
.children
.iter()
.find(|c| c.name == *seg)
.ok_or_else(|| format!("unknown `{display}`"))?;
}
if parent_is_object && !node.is_object() && crate::ast::function_uses_receiver(node) {
return Err(format!(
"`{display}` is an instance method; call it as `var`.{}",
node.name
));
}
Ok((lib, node))
}
/// Bind evaluated args onto parameter names. Err is a plain message (caller adds span/path).
fn bind_function_args(
interp: &mut Interpreter,
module: &Module,
caller: &Function,
caller_env: &mut Env,
target: &Function,
initial_env: &mut Env,
params: &[Param],
args: &[EvArg],
allow_missing: bool,
) -> std::result::Result<HashMap<String, Value>, String> {
) -> Result<HashMap<String, Value>> {
let mut named: HashMap<String, Value> = HashMap::new();
let mut positionals = Vec::new();
for a in args {
@@ -1630,7 +1819,7 @@ fn bind_function_args(
EvArg::Positional(v) => positionals.push(v.clone()),
EvArg::Named(k, v) => {
if named.contains_key(k) {
return Err(format!("duplicate named argument `{k}`"));
return Err(anyhow::anyhow!("duplicate named argument `{k}`"));
}
named.insert(k.clone(), v.clone());
}
@@ -1640,6 +1829,13 @@ fn bind_function_args(
let mut out: HashMap<String, Value> = HashMap::new();
let mut used_named = HashSet::new();
let mut pos_i = 0usize;
let mut defaults_env = Env::new();
// Only an explicit receiver crosses into a method's default environment.
for name in ["self", "自"] {
if let Some(value) = initial_env.get(name) {
defaults_env.set(name.into(), value.clone());
}
}
for p in params {
if let Some(v) = named.get(&p.name) {
@@ -1648,26 +1844,44 @@ fn bind_function_args(
} else if pos_i < positionals.len() {
out.insert(p.name.clone(), positionals[pos_i].clone());
pos_i += 1;
} else if let Some(def) = &p.default {
let v = interp
.eval_expr(module, caller, caller_env, def)
.map_err(|e| e.to_string())?;
out.insert(p.name.clone(), v);
} else if !allow_missing {
return Err(format!("missing argument for parameter `{}`", p.name));
} else if p.default.is_none() && !allow_missing {
return Err(anyhow::anyhow!("missing argument for parameter `{}`", p.name));
}
}
if pos_i < positionals.len() {
return Err(format!(
"too many positional arguments ({} extra)",
positionals.len() - pos_i
));
return Err(anyhow::anyhow!("too many positional arguments ({} extra)", positionals.len() - pos_i));
}
for (k, v) in named {
if !used_named.contains(&k) && !out.contains_key(&k) {
out.insert(k, v);
if !used_named.contains(&k) && !out.contains_key(&k) { out.insert(k, v); }
}
for (name, value) in &out { defaults_env.set(name.clone(), value.clone()); }
for name in ["self", "自"] {
if let Some(value)=initial_env.get(name) { defaults_env.set(name.into(),value.clone()); }
}
for p in params {
if out.contains_key(&p.name) { continue; }
if let Some(def) = &p.default {
let mut evaluate=|this:&mut Interpreter| {
let previous_span=this.current_span;
let previous_line=this.host.current_line;
this.current_span=p.span;
this.host.current_line=p.span.line;
let result=this.eval_expr(module,target,&mut defaults_env,def);
this.current_span=previous_span;
this.host.current_line=previous_line;
result
};
let v = if interp.current_module == Some(module as *const Module)
&& interp.diagnostic_frames.last().is_some_and(|frame| frame.declaration.span == target.span)
{
evaluate(interp)?
} else {
interp.in_function_source(module, target, evaluate)?
};
out.insert(p.name.clone(), v);
}
if let Some(value) = out.get(&p.name) {
defaults_env.set(p.name.clone(), value.clone());
}
}
@@ -1680,50 +1894,9 @@ fn bind_args(
args: &[EvArg],
allow_missing: bool,
) -> std::result::Result<HashMap<String, Value>, String> {
let mut named: HashMap<String, Value> = HashMap::new();
let mut positionals = Vec::new();
for a in args {
match a {
EvArg::Positional(v) => positionals.push(v.clone()),
EvArg::Named(k, v) => {
if named.contains_key(k) {
return Err(format!("duplicate named argument `{k}`"));
}
named.insert(k.clone(), v.clone());
}
}
}
let mut out: HashMap<String, Value> = HashMap::new();
let mut used_named = HashSet::new();
let mut pos_i = 0usize;
for p in params {
if let Some(v) = named.get(p) {
out.insert(p.clone(), v.clone());
used_named.insert(p.clone());
} else if pos_i < positionals.len() {
out.insert(p.clone(), positionals[pos_i].clone());
pos_i += 1;
} else if !allow_missing {
return Err(format!("missing argument for parameter `{p}`"));
}
}
if pos_i < positionals.len() {
return Err(format!(
"too many positional arguments ({} extra)",
positionals.len() - pos_i
));
}
for (k, v) in named {
if !used_named.contains(&k) && !out.contains_key(&k) {
out.insert(k, v);
}
}
Ok(out)
let names=args.iter().map(|a|match a{EvArg::Positional(_)=>None,EvArg::Named(name,_)=>Some(name.clone())}).collect::<Vec<_>>();
let values=args.iter().map(|a|match a{EvArg::Positional(value)|EvArg::Named(_,value)=>value.clone()}).collect::<Vec<_>>();
crate::arguments::bind(params,&names,&values,allow_missing)
}
fn collect_decl_reads(expr: &Expr, out: &mut std::collections::HashSet<String>) {
@@ -1791,49 +1964,21 @@ fn find_top<'a>(module: &'a Module, name: &str) -> Option<&'a Function> {
}
/// Resolve `#` object type by name in the entry module or any imported library.
fn find_object_type<'a>(module: &'a Module, name: &str) -> Option<(&'a Module, &'a Function)> {
fn search<'a>(
module: &'a Module,
name: &str,
seen: &mut std::collections::HashSet<*const Module>,
) -> Option<(&'a Module, &'a Function)> {
let key = module as *const Module;
if !seen.insert(key) {
return None;
}
if let Some(f) = module
.functions
.iter()
.find(|f| f.name == name && f.is_object())
{
return Some((module, f));
}
for lib in module.import_modules.values() {
if let Some(hit) = search(lib, name, seen) {
return Some(hit);
}
}
None
}
let mut seen = std::collections::HashSet::new();
search(module, name, &mut seen)
}
/// Resolve `` `obj`.method `` along the inheritance chain (`_type` first, then bases).
fn find_method_along_bases<'a>(
module: &'a Module,
type_name: &str,
object: &'a Function,
method: &str,
) -> Option<(&'a Module, &'a Function, &'a Function)> {
let mut cur = type_name.to_string();
let mut owner=module;
let mut obj=object;
let mut seen = std::collections::HashSet::new();
while seen.insert(cur.clone()) {
let (owner, obj) = find_object_type(module, &cur)?;
while seen.insert((owner as *const Module,obj as *const Function)) {
if let Some(target) = obj.children.iter().find(|c| c.name == method) {
return Some((owner, obj, target));
}
match &obj.base {
Some(b) => cur = b.clone(),
Some(base) => { (owner,obj)=crate::inherit::resolve_object(owner,base).ok()?; },
None => break,
}
}
@@ -1885,71 +2030,6 @@ fn lit_to_value(lit: &Literal) -> Value {
}
}
fn eval_binary(op: BinaryOp, l: &Value, r: &Value) -> Result<Value> {
match op {
BinaryOp::Add => match (l, r) {
(Value::Int(a), Value::Int(b)) => Ok(Value::Int(a + b)),
(Value::Num(a), Value::Num(b)) => Ok(Value::Num(a + b)),
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 + b)),
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a + *b as f64)),
(Value::Text(a), Value::Text(b)) => Ok(Value::Text(format!("{a}{b}"))),
(Value::Text(a), Value::Secret(b)) => Ok(Value::Text(format!("{a}{b}"))),
(Value::Secret(a), Value::Text(b)) => Ok(Value::Text(format!("{a}{b}"))),
(Value::Secret(a), Value::Secret(b)) => Ok(Value::Text(format!("{a}{b}"))),
(Value::Text(a), b) => Ok(Value::Text(format!("{a}{}", b.as_display()))),
(a, Value::Text(b)) => Ok(Value::Text(format!("{}{b}", a.as_display()))),
_ => bail!("`+` needs ints, nums, or text"),
},
BinaryOp::Sub => match (l, r) {
(Value::Int(a), Value::Int(b)) => Ok(Value::Int(a - b)),
(Value::Num(a), Value::Num(b)) => Ok(Value::Num(a - b)),
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 - b)),
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a - *b as f64)),
_ => bail!("`-` needs ints or nums"),
},
BinaryOp::Mul => match (l, r) {
(Value::Int(a), Value::Int(b)) => Ok(Value::Int(a * b)),
(Value::Num(a), Value::Num(b)) => Ok(Value::Num(a * b)),
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 * b)),
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a * *b as f64)),
_ => bail!("`*` needs ints or nums"),
},
BinaryOp::Div => match (l, r) {
(Value::Int(_), Value::Int(0)) => bail!("division by zero"),
(Value::Int(a), Value::Int(b)) => Ok(Value::Int(a / b)),
(Value::Num(_), Value::Num(b)) if *b == 0.0 => bail!("division by zero"),
(Value::Int(_), Value::Num(b)) if *b == 0.0 => bail!("division by zero"),
(Value::Num(_), Value::Int(0)) => bail!("division by zero"),
(Value::Num(a), Value::Num(b)) => Ok(Value::Num(a / b)),
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 / b)),
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a / *b as f64)),
_ => bail!("`/` needs ints or nums"),
},
BinaryOp::Eq => Ok(Value::Bool(l == r)),
BinaryOp::Ne => Ok(Value::Bool(l != r)),
BinaryOp::Lt => cmp_ord(l, r, |o| o.is_lt()),
BinaryOp::Le => cmp_ord(l, r, |o| o.is_le()),
BinaryOp::Gt => cmp_ord(l, r, |o| o.is_gt()),
BinaryOp::Ge => cmp_ord(l, r, |o| o.is_ge()),
BinaryOp::And | BinaryOp::Or => unreachable!("handled above"),
}
}
fn cmp_ord(l: &Value, r: &Value, f: impl Fn(std::cmp::Ordering) -> bool) -> Result<Value> {
match (l, r) {
(Value::Int(a), Value::Int(b)) => Ok(Value::Bool(f(a.cmp(b)))),
(Value::Num(a), Value::Num(b)) => Ok(Value::Bool(f(a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal)))),
(Value::Int(a), Value::Num(b)) => {
Ok(Value::Bool(f((*a as f64).partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal))))
}
(Value::Num(a), Value::Int(b)) => {
Ok(Value::Bool(f(a.partial_cmp(&(*b as f64)).unwrap_or(std::cmp::Ordering::Equal))))
}
(Value::Text(a), Value::Text(b)) => Ok(Value::Bool(f(a.cmp(b)))),
_ => bail!("comparison needs two ints/nums or two texts"),
}
}
fn brief_trace_val(v: &Value) -> String {
let s = v.as_display();
if s.chars().count() > 24 {
+54 -1
View File
@@ -4,6 +4,7 @@
//! CLI also offers `view` for AST-backed browsing.
pub mod aliases;
mod arguments;
pub mod artifact;
pub mod label;
pub mod migrate;
@@ -12,6 +13,7 @@ pub mod binding;
pub mod browser_session;
pub mod builtin;
pub mod bytecode;
pub mod call;
pub mod capture;
#[cfg(feature = "cli")]
pub mod catalog;
@@ -44,6 +46,16 @@ pub mod project;
pub mod repair;
pub mod sema;
pub mod value;
#[cfg(all(test, feature = "exec-host"))]
mod source_refinement_tests;
#[cfg(all(test, feature = "exec-host"))]
mod application_tests;
#[cfg(all(test, feature = "exec-host"))]
mod causal_replay_tests;
#[cfg(test)]
mod source_map_tests;
#[cfg(test)]
mod frontend_elaboration_tests;
#[cfg(feature = "net-host")]
pub mod version_check;
#[cfg(feature = "view")]
@@ -260,6 +272,23 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
host.fs_root = Some(root.clone());
host.cwd = root.clone();
}
let bound_call = if let Some(lib_path) = &opts.call_lib_path {
let named = named_call_args(opts)?;
let segments: Vec<String> = lib_path
.split('.')
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
.collect();
let (lib, fun) = crate::interp::locate_import_function(&module, &segments)
.map_err(|m| anyhow::anyhow!(m))?;
let mut binder = Interpreter::with_capture(Some(path), false).with_host(
HostContext::for_capture(Some(path), opts.host_caps()),
);
let bound = binder.bind_external_args(lib, fun, &named)?;
Some((lib_path.clone(), bound))
} else {
None
};
let mut vm = Vm::new(Some(path))
.with_stdin(stdin_lines)
.with_trace(opts.trace_eval)
@@ -267,7 +296,21 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
if opts.trace_eval {
eprintln!("=== marqdo: trace-eval ({path_label}) ===");
}
let value = vm.run(&program)?;
let value = if let Some((lib_path, bound)) = bound_call {
let key: Vec<String> = lib_path
.split('.')
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
.collect();
let idx = program.path_index.get(&key).copied().ok_or_else(|| {
anyhow::anyhow!(
"bytecode: `{lib_path}` did not compile to a function; refusing to run the program entry"
)
})?;
vm.call_function(&program, idx, &bound)?
} else {
vm.run(&program)?
};
if opts.trace_eval {
eprintln!("=== marqdo: end trace-eval ===");
}
@@ -301,6 +344,16 @@ fn run_call_lib_path(
Ok(value)
}
fn named_call_args(opts: &RunOptions) -> Result<Vec<(String, crate::value::Value)>> {
let Some(value) = load_args_json(opts)? else {
return Ok(Vec::new());
};
match value {
crate::value::Value::Map(pairs) => Ok(pairs),
_ => bail!("args-json must be a JSON object"),
}
}
fn load_args_json(opts: &RunOptions) -> Result<Option<crate::value::Value>> {
let Some(path) = &opts.args_json else {
return Ok(None);
+36 -2
View File
@@ -26,10 +26,22 @@ pub fn load_module_with_ctx(path: &Path, ctx: &BindingContext) -> Result<Module>
load_module_inner(path, &mut visited, ctx)
}
/// Load recorded entry text at its original path. Imports and metadata retain
/// normal file-relative resolution; the user's source file is never rewritten.
pub(crate) fn load_module_with_source(path: &Path, source: &str, ctx: &BindingContext) -> Result<Module> {
load_module_inner_source(path, &mut HashSet::new(), ctx, Some(source))
}
fn load_module_inner(
path: &Path,
visited: &mut HashSet<PathBuf>,
ctx: &BindingContext,
) -> Result<Module> {
load_module_inner_source(path, visited, ctx, None)
}
fn load_module_inner_source(
path: &Path, visited: &mut HashSet<PathBuf>, ctx: &BindingContext, source_override: Option<&str>,
) -> Result<Module> {
let canon = path.canonicalize().unwrap_or_else(|_| path.to_path_buf());
if !visited.insert(canon.clone()) {
@@ -37,11 +49,14 @@ fn load_module_inner(
}
let result = (|| {
let source = read_module_source(path)
.with_context(|| format!("failed to read {}", path.display()))?;
let source = match source_override {
Some(source) => source.to_string(),
None => read_module_source(path).with_context(|| format!("failed to read {}", path.display()))?,
};
let markup = frontmatter_markup(&source);
let mut module = crate::parse::expr::with_markup(markup, || parse_source(&source))
.map_err(|e| attach_path(path, e))?;
module.source.path = Some(path.to_path_buf());
// Resolve Artifact Metadata bindings (entry and imports share the same ctx).
let file_ctx = BindingContext {
@@ -343,6 +358,7 @@ fn load_module_from_source_inner(
let markup = frontmatter_markup(source);
let mut module = crate::parse::expr::with_markup(markup, || parse_source(source))
.map_err(|e| label_error(label, e))?;
module.source.path = Some(PathBuf::from(label));
module.metadata = resolve_metadata(&module.metadata_raw, ctx, Span::new(1, 1))
.map_err(|e| label_error(label, e))?;
let imports = module.imports.clone();
@@ -429,4 +445,22 @@ mod tests {
"{msg}"
);
}
#[test]
fn recorded_source_resolves_relative_imports_without_rewriting_files() {
let root=std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let dir=root.join(format!("target/load-recorded-{}-{}",std::process::id(),std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos()));
std::fs::create_dir_all(&dir).unwrap();let entry=dir.join("main.mq.md");
let original="# main\n\n> print text=original\n";
std::fs::write(&entry,original).unwrap();std::fs::write(dir.join("dep.mq.md"),"# helper\n\n> print text=dependency\n").unwrap();
let recorded="---\nimport:\n dep: dep.mq.md\n---\n# main\n\n> print text=recorded\n";
let ctx=BindingContext::from_binds(&[],cwd_for_path(Some(&entry)));
let module=load_module_with_source(&entry,recorded,&ctx).unwrap();
assert!(module.import_modules.contains_key("dep"));
assert_eq!(std::fs::read_to_string(&entry).unwrap(),original);
let cyclic="---\nimport:\n again: main.mq.md\n---\n# main\n";
assert!(load_module_with_source(&entry,cyclic,&ctx).unwrap_err().to_string().contains("circular import"));
assert_eq!(std::fs::read_to_string(&entry).unwrap(),original);
std::fs::remove_dir_all(dir).unwrap();
}
}
+61
View File
@@ -137,6 +137,26 @@ enum Commands {
#[arg(long)]
legacy_markup: bool,
},
/// Call one function in a `.mq.md` file: `marqdo call <file> <function> name=value`.
Call {
/// Source file.
file: PathBuf,
/// Function display path, for example `计算器.加法`.
function: String,
/// Named arguments `name=value` (integer, decimal, quoted text, True/False/None).
/// A value may be negative (`a=-2`). `--json` and `--backend` remain flags.
args: Vec<String>,
/// Print one JSON object: `{"ok":true,"result":...}` or `{"ok":false,"error":...}`.
#[arg(long)]
json: bool,
/// Runtime. Both backends execute the named function.
#[arg(long, value_enum, default_value_t = BackendCli::Tree)]
backend: BackendCli,
},
/// Static contract cross-check (渐进式契约): drift, unknown types, misplaced tables
Check {
/// Program path (.mq.md)
@@ -784,6 +804,47 @@ fn try_main(cli: Cli) -> Result<i32> {
}
Ok(0)
}
Commands::Call {
file,
function,
args,
json,
backend,
} => {
let outcome = (|| -> Result<marqdo::value::Value> {
let parsed = marqdo::call::parse_call_args(&args)?;
marqdo::call::call_file(&file, &function, &parsed, backend.into())
})();
match outcome {
Ok(value) => {
if json {
let result = marqdo::value_as_json(&value)
.map_err(|e| anyhow::anyhow!(e))?;
println!(
"{}",
serde_json::json!({"ok": true, "result": result})
);
} else {
println!("{}", value.as_display());
}
Ok(0)
}
Err(err) => {
let diag = Diagnostic::find(&err).cloned().unwrap_or_else(|| {
Diagnostic::new(None, Span::new(0, 0), format!("{err:#}")).with_code("call.failed")
});
if json {
println!(
"{}",
serde_json::json!({"ok": false, "error": diag.to_json()})
);
} else {
eprintln!("error: {diag}");
}
Ok(1)
}
}
}
Commands::Check {
file,
json,
+11 -32
View File
@@ -833,28 +833,22 @@ fn artifact_execute(params: &Value) -> Result<Value, Value> {
let denied: Vec<String> = artifact.effects.values().into_iter().filter(|effect| effect != "pure" && !allowed.contains(&effect.as_str())).collect();
if !denied.is_empty() { return Err(json!({"code":"artifact.effect_denied","message":"效果策略拒绝执行","denied":denied,"effects":artifact.effects.values()})); }
let path = compilation.semantics.root.join(&artifact.source.file);
let module = load::load_module(&path).map_err(|e| diagnostic_err("artifact.load", &e))?;
let input = params.get("inputs").cloned().unwrap_or_else(|| json!({}));
let object = input.as_object().ok_or_else(|| err("mlsp.bad_request", "inputs 必须是 JSON 对象"))?;
let mut args = Vec::new();
for (name, value) in object { args.push((name.clone(), crate::host::json::json_to_value(value).map_err(|e| err("mlsp.bad_input", e))?)); }
let caps = crate::host::HostCaps { fs_write: allowed.contains(&"fs.write"), exec: allowed.iter().any(|e| matches!(*e, "process.exec" | "plugin.call" | "llm.call")), net: allowed.iter().any(|e| matches!(*e, "network.read" | "network.write" | "llm.call")), plugin: allowed.contains(&"plugin.call") };
let host = crate::host::HostContext::for_capture(Some(&path), caps);
let mut interpreter = crate::interp::Interpreter::with_capture(Some(&path), false).with_host(host);
// Bind by source line / declaration identity — never by bare display_name suffix (G07).
let fun = find_function_at_line(&module.functions, artifact.source.line).ok_or_else(|| {
err(
"artifact.unresolved",
format!(
"无法在 {}:{} 定位绑定函数 {}",
artifact.source.file, artifact.source.line, artifact.display_name
),
)
})?;
let value = interpreter
.invoke_bound_function(&module, fun, &args)
.map_err(|e| diagnostic_err("artifact.execution_failed", &e))?;
let stdout = std::mem::take(&mut interpreter.captured_stdout);
// Same line identity, argument binding, and runtime as `marqdo call`.
let (value, stdout) = crate::call::execute_source_function(
&path,
artifact.source.line,
&artifact.display_name,
&args,
caps,
crate::Backend::Tree,
false,
)
.map_err(|e| diagnostic_err("artifact.execution_failed", &e))?;
Ok(json!({
"schema_version": 1,
"artifact_id": artifact.artifact_id.0,
@@ -866,21 +860,6 @@ fn artifact_execute(params: &Value) -> Result<Value, Value> {
}))
}
fn find_function_at_line<'a>(
functions: &'a [crate::ast::Function],
line: u32,
) -> Option<&'a crate::ast::Function> {
for fun in functions {
if fun.span.line == line {
return Some(fun);
}
if let Some(found) = find_function_at_line(&fun.children, line) {
return Some(found);
}
}
None
}
fn err(code: &str, message: impl Into<String>) -> Value {
json!({"code": code, "message": message.into()})
}
+14
View File
@@ -508,6 +508,20 @@ impl<'a> Parser<'a> {
});
}
if self.eat("-") {
// The magnitude of i64::MIN is not itself a positive i64 literal.
// Recognize its signed token before recursively parsing a unary
// operand; decimal literals keep their floating-point type.
let start = self.i;
self.skip_ws();
let digits = self.rest().bytes().take_while(u8::is_ascii_digit).count();
let end = self.i + digits;
if &self.src[self.i..end] == "9223372036854775808"
&& !self.src[end..].starts_with(['.', 'e', 'E'])
{
self.i = end;
return Ok(Expr::Literal(Literal::Int(i64::MIN)));
}
self.i = start;
let expr = self.parse_unary()?;
return Ok(Expr::Unary {
op: UnaryOp::Neg,
+42 -6
View File
@@ -26,7 +26,9 @@ use crate::parse::expr::parse_expr as parse_expr_str;
/// Parse full source (with optional frontmatter) into a module AST.
pub fn parse_source(source: &str) -> Result<Module> {
let lines = classify_source(source);
parse_classified(&lines)
let mut module = parse_classified(&lines)?;
module.source = crate::diagnostics::SourceIdentity::from_source(None, source);
Ok(module)
}
pub fn parse_classified(lines: &[ClassifiedLine]) -> Result<Module> {
@@ -63,14 +65,17 @@ pub fn parse_classified(lines: &[ClassifiedLine]) -> Result<Module> {
bail!("import bind `{}` conflicts with top-level name", u.bind);
}
}
Ok(Module {
let module=Module {
source: Default::default(),
imports,
uses,
functions,
import_modules: std::collections::HashMap::new(),
metadata_raw,
metadata: Vec::new(),
})
};
crate::ast::validate_parameters(&module)?;
Ok(module)
}
struct Cursor<'a> {
@@ -320,7 +325,7 @@ impl<'a> Cursor<'a> {
if self.peek_is_prose() {
break;
}
if let Some(param) = parse_param_line(trimmed) {
if let Some(param) = parse_param_line(trimmed, Span::new(l.line_no,1)) {
if fun.body.is_empty() {
fun.params.push(param);
self.bump();
@@ -891,6 +896,24 @@ impl<'a> Cursor<'a> {
if !header_done {
bail!("expected GFM table after empty assignment");
}
// Strong / Surface: unequal row arity is stuck — no silent pad/truncate.
for (line_no, cells) in &data_rows {
if cells.len() != header.len() {
return Err(Diagnostic::new(
None,
Span {
line: *line_no,
col: 1,
},
format!(
"table row arity mismatch: expected {} columns, got {}",
header.len(),
cells.len()
),
)
.into());
}
}
let row_oriented = header
.first()
.map(|h| is_row_table_marker(h))
@@ -1260,7 +1283,7 @@ fn parse_backtick_ident(s: &str) -> Option<String> {
Some(name.to_string())
}
fn parse_param_line(trimmed: &str) -> Option<crate::ast::Param> {
fn parse_param_line(trimmed: &str, span:Span) -> Option<crate::ast::Param> {
use crate::ast::Param;
let rest = trimmed.strip_prefix('+')?.trim();
if rest.is_empty() || !rest.starts_with('`') {
@@ -1285,6 +1308,7 @@ fn parse_param_line(trimmed: &str) -> Option<crate::ast::Param> {
};
Some(Param {
name,
span,
default,
inferred: false,
})
@@ -2086,6 +2110,17 @@ fn merge_inferred_params(
let mut needs_input = HashSet::new();
let mut bound = HashSet::new();
let mut assign_decls = Vec::new();
// Defaults execute before the body. Reads used only there still require
// declared inputs; later body assignments cannot satisfy those reads.
for param in &fun.params {
if let Some(default)=&param.default { collect_reads_expr(default,&mut needs_input); }
}
for (_,default,_) in prose_decls {
if let Some(default)=default {
let parsed=parse_expr_prefer_var(default).or_else(|_|parse_value_or_interp(default))?;
collect_reads_expr(&parsed,&mut needs_input);
}
}
collect_names_needing_input(
&fun.body,
&mut bound,
@@ -2113,7 +2148,7 @@ fn merge_inferred_params(
}
events.sort_by_key(|e| (e.0, e.1));
for (_line, _ord, name, default_txt) in events {
for (line, _ord, name, default_txt) in events {
if name == "self" || name == "自" {
continue;
}
@@ -2135,6 +2170,7 @@ fn merge_inferred_params(
};
fun.params.push(Param {
name,
span:Span::new(line,1),
default,
inferred: true,
});
+6 -192
View File
@@ -5,8 +5,6 @@
//! dynamics are marked separately.
use crate::ast::{CallExpr, Function, Module};
use crate::diagnostics::Span;
use crate::host::HostFn;
/// Why a call site is not a static BoundCall.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -26,31 +24,7 @@ impl DynamicReason {
}
}
/// Runtime resolution shared by Tree (and mirrored by Bytecode rules).
#[derive(Debug)]
pub enum ExecTarget<'a> {
/// Function in the calling module (or nested children).
Local(&'a Function),
/// `bind.member…` through `module.import_modules`.
Import {
bind: String,
lib: &'a Module,
fun: &'a Function,
path: Vec<String>,
},
Builtin(&'static str),
Host(HostFn),
/// Plugin / method / call_fn — not a static BoundCall.
Dynamic {
reason: DynamicReason,
name: String,
},
Unresolved {
message: String,
#[allow(dead_code)]
span_hint: Option<Span>,
},
}
pub type ExecTarget<'a> = super::resolution::Target<&'a Function, &'a Module>;
pub fn call_display(call: &CallExpr) -> String {
if let Some(path) = &call.path {
@@ -62,172 +36,12 @@ pub fn call_display(call: &CallExpr) -> String {
call.callee.clone()
}
fn find_in_children<'a>(fun: &'a Function, name: &str) -> Option<&'a Function> {
for child in &fun.children {
if child.name == name {
return Some(child);
}
if let Some(f) = find_in_children(child, name) {
return Some(f);
}
}
None
/// Same-module bare names never search imports.
pub fn lookup_local<'a>(module: &'a Module, current: &'a Function, name: &str) -> Option<&'a Function> {
super::resolution::lookup(&super::resolution::AstView::default(), &module, current, name)
}
/// Same-module bare-name lookup (never searches imports).
pub fn lookup_local<'a>(
module: &'a Module,
current: &'a Function,
name: &str,
) -> Option<&'a Function> {
if let Some(f) = find_in_children(current, name) {
return Some(f);
}
module.functions.iter().find(|f| f.name == name)
}
fn walk_path<'a>(lib: &'a Module, path: &[String]) -> Result<&'a Function, String> {
let display = path.join(".");
if path.is_empty() {
return Err("empty knowledge path".into());
}
let mut parent_is_object = false;
let mut node = lib
.functions
.iter()
.find(|f| f.name == path[0])
.ok_or_else(|| format!("unknown `{display}`"))?;
for seg in &path[1..] {
parent_is_object = node.is_object();
node = node
.children
.iter()
.find(|c| c.name == *seg)
.ok_or_else(|| format!("unknown `{display}`"))?;
}
if parent_is_object && node.level != 1 && crate::ast::function_uses_receiver(node) {
return Err(format!(
"`{display}` is an instance method; call it as `var`.{}",
node.name
));
}
Ok(node)
}
fn resolve_import_path<'a>(module: &'a Module, path: &[String]) -> ExecTarget<'a> {
let display = path.join(".");
if path.len() < 2 {
return ExecTarget::Unresolved {
message: format!("library path `{display}`: need at least `lib.member`"),
span_hint: None,
};
}
let bind = &path[0];
let Some(lib) = module.import_modules.get(bind) else {
return ExecTarget::Unresolved {
message: format!("unknown library `{bind}` (import it in frontmatter)"),
span_hint: None,
};
};
match walk_path(lib, &path[1..]) {
Ok(fun) => ExecTarget::Import {
bind: bind.clone(),
lib,
fun,
path: path.to_vec(),
},
Err(message) => ExecTarget::Unresolved {
message,
span_hint: None,
},
}
}
/// Resolve a call for execution. Import bare-name leakage is forbidden (G02).
pub fn resolve_for_exec<'a>(
module: &'a Module,
current: &'a Function,
call: &CallExpr,
) -> ExecTarget<'a> {
if call.receiver.is_some() {
return ExecTarget::Dynamic {
reason: DynamicReason::MethodReceiver,
name: call_display(call),
};
}
if let Some(path) = &call.path {
if module.import_modules.contains_key(&path[0]) {
return resolve_import_path(module, path);
}
// Same-module definition-tree path (`Type.member` / nested), BoundCall.
if path.len() >= 2 {
match walk_path(module, path) {
Ok(fun) => return ExecTarget::Local(fun),
Err(_) if path.len() == 2 => {
// Possible local-variable method; caller checks env.
return ExecTarget::Dynamic {
reason: DynamicReason::MethodReceiver,
name: call_display(call),
};
}
Err(message) => {
return ExecTarget::Unresolved {
message,
span_hint: None,
};
}
}
}
return ExecTarget::Unresolved {
message: format!(
"library path `{}`: need at least `lib.member`",
path.join(".")
),
span_hint: None,
};
}
if let Some(u) = module.uses.iter().find(|u| u.bind == call.callee) {
return resolve_import_path(module, &u.path);
}
match call.callee.as_str() {
"print" | "input" | "len" | "str" | "int" | "type" | "trim" | "split" | "join" | "at" => {
return ExecTarget::Builtin(match call.callee.as_str() {
"print" => "print",
"input" => "input",
"len" => "len",
"str" => "str",
"int" => "int",
"type" => "type",
"trim" => "trim",
"split" => "split",
"join" => "join",
"at" => "at",
_ => unreachable!(),
});
}
"call_fn" => {
return ExecTarget::Dynamic {
reason: DynamicReason::CallFn,
name: "call_fn".into(),
};
}
other if HostFn::from_name(other).is_some() => {
return ExecTarget::Host(HostFn::from_name(other).unwrap());
}
_ => {}
}
if let Some(fun) = lookup_local(module, current, &call.callee) {
return ExecTarget::Local(fun);
}
ExecTarget::Dynamic {
reason: DynamicReason::Plugin,
name: call.callee.clone(),
}
pub fn resolve_for_exec<'a>(module: &'a Module, current: &'a Function, call: &CallExpr) -> ExecTarget<'a> {
super::resolution::resolve(&super::resolution::AstView::default(), &module, current, call)
}
pub fn is_static_bound(target: &ExecTarget<'_>) -> bool {
+69 -6
View File
@@ -8,7 +8,7 @@ use std::collections::{BTreeMap, BTreeSet};
use std::fs;
use std::path::{Path, PathBuf};
use anyhow::{Context, Result};
use anyhow::{bail, Context, Result};
use serde_json::{json, Value};
use sha2::{Digest as _, Sha256};
@@ -240,10 +240,20 @@ pub struct Compilation {
/// side-effect free: it never loads plugins, evaluates expressions or fetches
/// sources.
pub fn compile_project(options: &CompileOptions) -> Result<Compilation> {
let t_index = std::time::Instant::now();
let root = ProjectRoot::new(&options.project_root)?;
let index = ProjectIndex::build(root.clone())?;
let index_ms = t_index.elapsed().as_millis();
compile_index(ProjectIndex::build(root)?)
}
/// Compile `entry` and the modules it imports. Sibling files in the directory are not included.
pub fn compile_file_closure(entry: &Path) -> Result<Compilation> {
let files = import_closure(entry)?;
let ancestor = common_ancestor(&files)?;
let root = ProjectRoot::new(&ancestor)?;
compile_index(ProjectIndex::from_files(root, files)?)
}
fn compile_index(index: ProjectIndex) -> Result<Compilation> {
let root = index.root().clone();
// Reuse parsed modules from ProjectIndex — same input snapshot, no second parse (M-R5b).
let mut files = BTreeMap::new();
let mut diagnostics = Vec::new();
@@ -257,8 +267,6 @@ pub fn compile_project(options: &CompileOptions) -> Result<Compilation> {
),
);
}
let _ = index_ms; // ProjectIndex build duration (M-R5b span)
// Wire import_modules from already-parsed project files so BoundCall path
// resolve shares `resolve_for_exec` (no plugin load / no re-parse).
attach_project_imports(&mut files);
@@ -349,6 +357,56 @@ pub fn compile_project(options: &CompileOptions) -> Result<Compilation> {
})
}
fn import_closure(entry: &Path) -> Result<Vec<PathBuf>> {
let mut out = Vec::new();
let mut seen = std::collections::HashSet::new();
fn walk(path: &Path, out: &mut Vec<PathBuf>, seen: &mut std::collections::HashSet<PathBuf>) -> Result<()> {
if !path.is_file() {
return Ok(());
}
let canon = path.canonicalize().with_context(|| format!("resolve {}", path.display()))?;
if !seen.insert(canon.clone()) {
return Ok(());
}
out.push(canon.clone());
let source = fs::read_to_string(&canon).with_context(|| format!("read {}", canon.display()))?;
let module = crate::load::parse_source_auto(&source)
.with_context(|| format!("parse {}", canon.display()))?;
let base = canon.parent().unwrap_or_else(|| Path::new("."));
for imp in &module.imports {
let dep = crate::load::resolve_import(base, &imp.path)
.with_context(|| format!("import `{}` from {}", imp.path, canon.display()))?;
walk(&dep, out, seen)?;
}
Ok(())
}
if !entry.is_file() {
bail!("call target {} is not a file", entry.display());
}
walk(entry, &mut out, &mut seen)?;
out.sort();
Ok(out)
}
fn common_ancestor(files: &[PathBuf]) -> Result<PathBuf> {
let Some(first) = files.first() else {
bail!("call target has no source files");
};
let mut ancestor = first
.parent()
.map(|p| p.to_path_buf())
.unwrap_or_else(|| PathBuf::from("."));
for file in files.iter().skip(1) {
while !file.starts_with(&ancestor) {
ancestor = ancestor
.parent()
.map(|p| p.to_path_buf())
.ok_or_else(|| anyhow::anyhow!("source files have no common directory"))?;
}
}
Ok(ancestor)
}
fn module_artifact(index: &ProjectIndex, module: &ModuleId, path: &Path, text: &str) -> SemanticArtifact {
let implementation = digest(normalize_source(text));
let interface = digest(module.as_str());
@@ -799,6 +857,11 @@ fn collect_call_sites(function: &Function) -> Vec<(usize, CallExpr, u32)> {
}
}
}
// Omitted arguments execute these expressions before the body. Leaving
// them out lets the pure-call policy miss both direct and transitive effects.
for param in &function.params {
if let Some(default) = &param.default { expr(default, function.span.line, &mut calls); }
}
statements(&function.body, &mut calls);
calls
}
+3 -1
View File
@@ -5,6 +5,7 @@
//! the meaning of the same source text.
pub mod bind;
pub(crate) mod resolution;
pub mod extract;
pub mod ir;
pub mod lock;
@@ -16,7 +17,8 @@ pub use bind::{
};
pub use extract::{plan_pure_extract, ExtractError, ExtractPlan};
pub use ir::{
artifact_json, compile_project, semantic_generation_dir, write_semantic_projections, ArtifactId,
artifact_json, compile_file_closure, compile_project, semantic_generation_dir,
write_semantic_projections, ArtifactId,
BoundCall, CompileOptions, Compilation, Digest, DynamicCall, EffectSet, LockMode,
SemanticArtifact, SemanticProject,
};
+254
View File
@@ -0,0 +1,254 @@
//! Ordered declaration lookup, independent of the declaration representation.
use super::bind::{call_display, DynamicReason};
use crate::{ast::CallExpr, diagnostics::Span, host::HostFn};
pub(crate) trait NameView {
type Function: Copy;
type Module: Clone;
fn roots(&self, module: &Self::Module) -> Vec<Self::Function>;
fn children(&self, function: Self::Function) -> Vec<Self::Function>;
fn parent(&self, module: &Self::Module, function: Self::Function) -> Option<Self::Function>;
fn same(&self, left: Self::Function, right: Self::Function) -> bool;
fn name(&self, function: Self::Function) -> String;
fn level(&self, function: Self::Function) -> u8;
fn is_object(&self, function: Self::Function) -> bool;
fn uses_receiver(&self, function: Self::Function) -> bool;
fn import(&self, module: &Self::Module, name: &str) -> Option<Self::Module>;
fn used_path(&self, module: &Self::Module, name: &str) -> Option<Vec<String>>;
}
#[derive(Debug)]
pub enum Target<F, M> {
Local(F),
Import {
bind: String,
lib: M,
fun: F,
path: Vec<String>,
},
Builtin(&'static str),
Host(HostFn),
Dynamic {
reason: DynamicReason,
name: String,
},
Unresolved {
message: String,
span_hint: Option<Span>,
},
}
fn ancestors<V: NameView>(
view: &V,
nodes: Vec<V::Function>,
current: V::Function,
path: &mut Vec<V::Function>,
) -> bool {
for node in nodes {
if view.same(node, current) {
return true;
}
path.push(node);
if ancestors(view, view.children(node), current, path) {
return true;
}
path.pop();
}
false
}
pub(crate) fn lookup<V: NameView>(
view: &V,
module: &V::Module,
current: V::Function,
name: &str,
) -> Option<V::Function> {
let mut cursor = Some(current);
while let Some(node) = cursor {
if let Some(f) = view
.children(node)
.into_iter()
.find(|f| view.name(*f) == name)
{
return Some(f);
}
cursor = view.parent(module, node);
}
view.roots(module)
.into_iter()
.find(|f| view.name(*f) == name)
}
fn walk<V: NameView>(view: &V, module: &V::Module, path: &[String]) -> Result<V::Function, String> {
let display = path.join(".");
let first = path.first().ok_or("empty knowledge path")?;
let mut node = view
.roots(module)
.into_iter()
.find(|f| view.name(*f) == *first)
.ok_or_else(|| format!("unknown `{display}`"))?;
let mut parent_is_object = false;
for segment in &path[1..] {
parent_is_object = view.is_object(node);
node = view
.children(node)
.into_iter()
.find(|f| view.name(*f) == *segment)
.ok_or_else(|| format!("unknown `{display}`"))?;
}
if parent_is_object && view.level(node) != 1 && view.uses_receiver(node) {
return Err(format!(
"`{display}` is an instance method; call it as `var`.{}",
view.name(node)
));
}
Ok(node)
}
fn imported<V: NameView>(
view: &V,
module: &V::Module,
path: &[String],
) -> Target<V::Function, V::Module> {
let unresolved = |message| Target::Unresolved {
message,
span_hint: None,
};
if path.len() < 2 {
return unresolved(format!(
"library path `{}`: need at least `lib.member`",
path.join(".")
));
}
let bind = &path[0];
let Some(lib) = view.import(module, bind) else {
return unresolved(format!(
"unknown library `{bind}` (import it in frontmatter)"
));
};
match walk(view, &lib, &path[1..]) {
Ok(fun) => Target::Import {
bind: bind.clone(),
lib,
fun,
path: path.to_vec(),
},
Err(message) => unresolved(message),
}
}
pub(crate) fn resolve<V: NameView>(
view: &V,
module: &V::Module,
current: V::Function,
call: &CallExpr,
) -> Target<V::Function, V::Module> {
if call.receiver.is_some() {
return Target::Dynamic {
reason: DynamicReason::MethodReceiver,
name: call_display(call),
};
}
if let Some(path) = &call.path {
if path
.first()
.is_some_and(|p| view.import(module, p).is_some())
{
return imported(view, module, path);
}
if path.len() >= 2 {
return match walk(view, module, path) {
Ok(fun) => Target::Local(fun),
Err(_) if path.len() == 2 => Target::Dynamic {
reason: DynamicReason::MethodReceiver,
name: call_display(call),
},
Err(message) => Target::Unresolved {
message,
span_hint: None,
},
};
}
return Target::Unresolved {
message: format!(
"library path `{}`: need at least `lib.member`",
path.join(".")
),
span_hint: None,
};
}
if let Some(path) = view.used_path(module, &call.callee) {
return imported(view, module, &path);
}
if crate::arguments::builtin_signature(&call.callee).is_some() {
if call.callee == "call_fn" {
return Target::Dynamic {
reason: DynamicReason::CallFn,
name: "call_fn".into(),
};
}
let name = match call.callee.as_str() {
"print" => "print",
"input" => "input",
"len" => "len",
"str" => "str",
"int" => "int",
"type" => "type",
"trim" => "trim",
"split" => "split",
"join" => "join",
"at" => "at",
_ => unreachable!(),
};
return Target::Builtin(name);
}
if let Some(host) = HostFn::from_name(&call.callee) {
return Target::Host(host);
}
if let Some(fun) = lookup(view, module, current, &call.callee) {
return Target::Local(fun);
}
Target::Dynamic {
reason: DynamicReason::Plugin,
name: call.callee.clone(),
}
}
#[derive(Default)]
pub(crate) struct AstView<'a>(std::marker::PhantomData<&'a crate::ast::Module>);
impl<'a> NameView for AstView<'a> {
type Function = &'a crate::ast::Function;
type Module = &'a crate::ast::Module;
fn roots(&self, m: &Self::Module) -> Vec<Self::Function> {
m.functions.iter().collect()
}
fn children(&self, f: Self::Function) -> Vec<Self::Function> {
f.children.iter().collect()
}
fn parent(&self, m: &Self::Module, f: Self::Function) -> Option<Self::Function> {
let mut path = Vec::new();
if ancestors(self, self.roots(m), f, &mut path) {
path.last().copied()
} else {
None
}
}
fn same(&self, a: Self::Function, b: Self::Function) -> bool {
std::ptr::eq(a, b)
}
fn name(&self, f: Self::Function) -> String {
f.name.clone()
}
fn level(&self, f: Self::Function) -> u8 {
f.level
}
fn is_object(&self, f: Self::Function) -> bool {
f.is_object()
}
fn uses_receiver(&self, f: Self::Function) -> bool {
crate::ast::function_uses_receiver(f)
}
fn import(&self, m: &Self::Module, name: &str) -> Option<Self::Module> {
m.import_modules.get(name)
}
fn used_path(&self, m: &Self::Module, name: &str) -> Option<Vec<String>> {
m.uses
.iter()
.find(|u| u.bind == name)
.map(|u| u.path.clone())
}
}
+6 -1
View File
@@ -119,8 +119,13 @@ pub struct ProjectIndex {
impl ProjectIndex {
pub fn build(root: ProjectRoot) -> Result<Self> {
let project_id = project_id(root.path());
let files = root.source_files()?;
Self::from_files(root, files)
}
/// Index exactly `files` (already chosen by the caller). Paths must sit under `root`.
pub fn from_files(root: ProjectRoot, files: Vec<PathBuf>) -> Result<Self> {
let project_id = project_id(root.path());
let mut modules = BTreeMap::new();
let mut path_to_module = BTreeMap::new();
let mut symbols = Vec::new();
+577
View File
@@ -0,0 +1,577 @@
use crate::diagnostics::{Diagnostic, Span};
use crate::{
bytecode::{compile_module, Vm},
interp::Interpreter,
load::load_module,
value::Value,
};
use serde_json::json;
use sha2::{Digest, Sha256};
use std::path::{Path, PathBuf};
fn digest(text: &str) -> String {
format!("{:x}", Sha256::digest(text.as_bytes()))
}
fn line(text: &str, marker: &str) -> u32 {
text.lines()
.position(|s| s.contains(marker))
.expect("oracle marker") as u32
+ 1
}
fn root(label: &str) -> PathBuf {
let dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("target/source-map")
.join(format!("{label}-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
dir
}
fn save(path: &Path, text: &str) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, text).unwrap();
}
fn source(key: &str, path: &Path, text: &str) -> serde_json::Value {
json!({"key":key,"path":path,"text":text,"virtual":!path.is_absolute()})
}
fn site(key: &str, marker: &str) -> serde_json::Value {
json!({"source":key,"marker":marker})
}
fn frame(
name: &str,
declaration: serde_json::Value,
call_site: Option<serde_json::Value>,
) -> serde_json::Value {
json!({"function":name,"declaration":declaration,"call_site":call_site})
}
fn record(
id: &str,
backend: &str,
diagnostic: &Diagnostic,
sources: Vec<serde_json::Value>,
origin: serde_json::Value,
frames: Vec<serde_json::Value>,
) {
if let Some(directory) = std::env::var_os("MARQDO_SOURCE_MAP_DIR") {
let directory = PathBuf::from(directory);
std::fs::create_dir_all(&directory).unwrap();
std::fs::write(
directory.join(format!("{id}-{backend}.json")),
serde_json::to_vec_pretty(
&json!({"version":1,"id":id,"backend":backend,"sources":sources,
"expected":{"origin":origin,"frames":frames},"diagnostic":diagnostic.to_json()}),
)
.unwrap(),
)
.unwrap();
}
}
fn assert_site(site: &crate::diagnostics::SourceLocation, path: &Path, text: &str, expected: u32) {
assert_eq!(
site.source.path.as_deref().unwrap().canonicalize().unwrap(),
path.canonicalize().unwrap()
);
assert_eq!(site.source.sha256.as_deref(), Some(digest(text).as_str()));
assert_eq!(site.span, Span::new(expected, 1));
}
fn assert_origin(d: &Diagnostic, path: &Path, text: &str, expected: u32) {
assert_eq!(
d.path.as_deref().unwrap().canonicalize().unwrap(),
path.canonicalize().unwrap()
);
assert_eq!(d.source_sha256.as_deref(), Some(digest(text).as_str()));
assert_eq!(d.span, Span::new(expected, 1));
assert!(d.message.contains("division by zero"), "{d}");
assert_eq!(d.to_json()["source_sha256"], json!(digest(text)));
}
#[test]
fn external_argument_binding_uses_target_declaration_source() {
let dir = root("external-binding");
let entry = dir.join("entry.mq.md");
let target = dir.join("target.mq.md");
let leaf = dir.join("leaf.mq.md");
let main = "# main\n*7*\n";
let text = "---\nimport:\n leaf: leaf.mq.md\n---\n\n\n## required\n + `x`\n*x*\n\n## defaulted\n + `x`=> leaf.explode\n*x*\n";
let leaf_text = "\n\n\n## explode\n**bad = 1 / 0**\n*bad*\n";
save(&entry, main);
save(&target, text);
save(&leaf, leaf_text);
let module = load_module(&target).unwrap();
let mut tree = Interpreter::with_capture(Some(&entry), false);
let before = tree.diagnostic_scope_state();
for name in ["required", "defaulted"] {
let fun = module.functions.iter().find(|f| f.name == name).unwrap();
let error = tree.bind_external_args(&module, fun, &[]).unwrap_err();
let d = Diagnostic::find(&error).unwrap();
assert_eq!(tree.diagnostic_scope_state(), before);
let expected_frames = if name == "required" {
assert_eq!(d.code, "call.missing_arg");
assert_eq!(d.path.as_deref(), Some(target.as_path()));
assert_eq!(d.source_sha256.as_deref(), Some(digest(text).as_str()));
assert_eq!(d.span, fun.span);
vec![frame("required", site("target", "## required"), None)]
} else {
assert_origin(d, &leaf, leaf_text, line(leaf_text, "**bad"));
vec![
frame("defaulted", site("target", "## defaulted"), None),
frame(
"explode",
site("leaf", "## explode"),
Some(site("target", "+ `x`=> leaf.explode")),
),
]
};
record(
&format!("external-binding-{name}"),
"tree",
d,
vec![
source("entry", &entry, main),
source("target", &target, text),
source("leaf", &leaf, leaf_text),
],
if name == "required" {
site("target", "## required")
} else {
site("leaf", "**bad")
},
expected_frames,
);
let first = d.to_json();
let bound = tree
.bind_external_args(&module, fun, &[("x".into(), Value::Int(7))])
.unwrap();
assert_eq!(bound, vec![("x".into(), Value::Int(7))]);
assert_eq!(tree.diagnostic_scope_state(), before);
let again = tree.bind_external_args(&module, fun, &[]).unwrap_err();
assert_eq!(Diagnostic::find(&again).unwrap().to_json(), first);
assert_eq!(tree.diagnostic_scope_state(), before);
}
}
#[test]
fn imported_source_identity_frames_and_reuse_match_on_both_backends() {
let root = root("imports");
let mut count = 0;
for pad in [0, 4, 17] {
for crlf in [false, true] {
for side in ["left", "right"] {
let dir = root.join(format!("{pad}-{crlf}-{side}"));
let basename = if pad == 17 {
"共同源.mq.md"
} else {
"shared.mq.md"
};
let left = dir.join("left").join(basename);
let right = dir.join("right").join(basename);
let middle = dir.join("middle.mq.md");
let entry = dir.join("entry.mq.md");
let lf = |s: String| if crlf { s.replace('\n', "\r\n") } else { s };
let leaf_a = lf(format!(
"源位置验证 🧪\n{}## explode\n**bad = 1 / 0**\n*bad*\n\n## good\n*7*\n",
"\n".repeat(pad)
));
let leaf_b = if pad == 4 {
leaf_a.clone()
} else {
lf(format!(
"{}## explode\n\n**bad = 1 / 0**\n*bad*\n\n## good\n*7*\n",
"\n".repeat(pad + 3)
))
};
save(&left, &leaf_a);
save(&right, &leaf_b);
let mid=lf(format!("---\nmarkup: occlusion\nimport:\n leaf: {side}/{basename}\n---\n## relay\n**answer = > leaf.explode**\n*answer*\n"));
let main=lf("---\nmarkup: occlusion\nimport:\n mid: middle.mq.md\n---\n# main\n**answer = > mid.relay**\n*answer*\n".into());
save(&middle, &mid);
save(&entry, &main);
let module = load_module(&entry).unwrap();
let (leaf_path, leaf) = if side == "left" {
(&left, &leaf_a)
} else {
(&right, &leaf_b)
};
let leaf_module = &module.import_modules["mid"].import_modules["leaf"];
let program = compile_module(Some(&entry), &module).unwrap();
// Dependencies are private, so API calls compile that module
// directly rather than assuming transitive re-exports.
let good_code =
crate::bytecode::compile_for_call(Some(leaf_path), leaf_module).unwrap();
let good_fid = good_code.path_index[&vec!["good".into()]];
for backend in ["tree", "bytecode"] {
let host = || {
let mut h =
crate::host::HostContext::for_capture(Some(&entry), Default::default());
h.call_stack.push("outside-source-entry".into());
h.call_site_lines.push(99);
h
};
let mut tree = Interpreter::with_capture(Some(&entry), false).with_host(host());
let mut vm = Vm::with_capture(Some(&entry)).with_host(host());
let before_tree = tree.diagnostic_scope_state();
let before_vm = vm.diagnostic_scope_state();
let err = if backend == "tree" {
tree.run_module(&module)
} else {
vm.run(&program)
}
.unwrap_err();
let diagnostic = Diagnostic::find(&err).unwrap();
assert_origin(diagnostic, leaf_path, leaf, line(leaf, "**bad"));
assert_eq!(diagnostic.frames.len(), 3);
assert_eq!(
diagnostic
.frames
.iter()
.map(|f| f.function.as_str())
.collect::<Vec<_>>(),
["main", "relay", "explode"]
);
assert!(diagnostic.frames[0].call_site.is_none());
assert_site(
diagnostic.frames[1].call_site.as_ref().unwrap(),
&entry,
&main,
line(&main, "**answer"),
);
assert_site(
diagnostic.frames[2].call_site.as_ref().unwrap(),
&middle,
&mid,
line(&mid, "**answer"),
);
assert_site(
&diagnostic.frames[2].declaration,
leaf_path,
leaf,
line(leaf, "## explode"),
);
assert_eq!(tree.diagnostic_scope_state(), before_tree);
assert_eq!(vm.diagnostic_scope_state(), before_vm);
record(
&format!("imports-{pad}-{crlf}-{side}"),
backend,
diagnostic,
vec![
source("entry", &entry, &main),
source("middle", &middle, &mid),
source("leaf", leaf_path, leaf),
],
site("leaf", "**bad"),
vec![
frame("main", site("entry", "# main"), None),
frame(
"relay",
site("middle", "## relay"),
Some(site("entry", "**answer")),
),
frame(
"explode",
site("leaf", "## explode"),
Some(site("middle", "**answer")),
),
],
);
let expected = diagnostic.to_json();
let good = if backend == "tree" {
tree.invoke_function(leaf_module, "good", &[])
} else {
vm.call_function(&good_code, good_fid, &[])
}
.unwrap();
assert_eq!(good, Value::Int(7));
let again = if backend == "tree" {
tree.run_module(&module)
} else {
vm.run(&program)
}
.unwrap_err();
assert_eq!(
Diagnostic::find(&again).unwrap().to_json(),
expected,
"stale caller leaked after reuse"
);
assert_eq!(tree.diagnostic_scope_state(), before_tree);
assert_eq!(vm.diagnostic_scope_state(), before_vm);
count += 1;
}
}
}
}
println!(
"SOURCE_MAP_METRIC:{}",
json!({"kind":"imports","cases":count,"correct":true})
);
}
#[test]
fn source_revision_is_loaded_text_not_later_disk_contents() {
let root = root("revision");
let lib = root.join("shared.mq.md");
let entry = root.join("entry.mq.md");
let first = "## explode\n**bad = 1 / 0**\n*bad*\n";
let second = "\n\n\n\n\n## explode\n**bad = 1 / 0**\n*bad*\n";
let main="---\nmarkup: occlusion\nimport:\n lib: shared.mq.md\n---\n# main\n**answer = > lib.explode**\n*answer*\n";
save(&entry, main);
save(&lib, first);
let old = load_module(&entry).unwrap();
let old_code = compile_module(Some(&entry), &old).unwrap();
save(&lib, second);
let new = load_module(&entry).unwrap();
let new_code = compile_module(Some(&entry), &new).unwrap();
for (revision, module, code, text) in [
("old", &old, &old_code, first),
("new", &new, &new_code, second),
] {
let mut tree = Interpreter::with_capture(Some(&entry), false);
let mut vm = Vm::with_capture(Some(&entry));
for (backend, error) in [
("tree", tree.run_module(module).unwrap_err()),
("bytecode", vm.run(code).unwrap_err()),
] {
let d = Diagnostic::find(&error).unwrap();
assert_origin(d, &lib, text, line(text, "**bad"));
assert_site(
d.frames[1].call_site.as_ref().unwrap(),
&entry,
main,
line(main, "**answer"),
);
record(
&format!("revision-{revision}"),
backend,
d,
vec![source("entry", &entry, main), source("leaf", &lib, text)],
site("leaf", "**bad"),
vec![
frame("main", site("entry", "# main"), None),
frame(
"explode",
site("leaf", "## explode"),
Some(site("entry", "**answer")),
),
],
);
}
}
}
#[test]
#[cfg(feature = "exec-host")]
fn foreign_boundary_diagnostics_keep_library_origin_and_real_user_call_site() {
let root = root("foreign");
let lib = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("lib/foreign.mq.md");
let library = std::fs::read_to_string(&lib).unwrap();
let mut count = 0;
for lang in ["python", "node"] {
for output in [false, true] {
let entry = root.join(format!("{lang}-{output}.mq.md"));
let action = if output {
"**> foreign.session_eval id=`s` source=\"n = 99; 9007199254740992\"**"
} else {
"**> foreign.session_set id=`s` name=\"n\" value=`s`**"
};
let main=format!("---\nmarkup: occlusion\nimport:\n foreign: {}\n---\n# main\n**s = > foreign.session_open lang=\"{lang}\"**\n**> foreign.session_set id=`s` name=\"n\" value=1**\n{action}\n",lib.display());
save(&entry, &main);
let module = load_module(&entry).unwrap();
let program = compile_module(Some(&entry), &module).unwrap();
let expected_function = if output {
"session_eval"
} else {
"session_set"
};
let expected_primitive = if output {
"host_foreign_session_eval"
} else {
"host_foreign_session_set"
};
for backend in ["tree", "bytecode"] {
let mut tree = Interpreter::with_capture(Some(&entry), false);
let mut vm = Vm::with_capture(Some(&entry));
let error = if backend == "tree" {
tree.run_module(&module)
} else {
vm.run(&program)
}
.unwrap_err();
let d = Diagnostic::find(&error).unwrap();
assert!(d.message.contains("outside V_safe"));
assert_eq!(
d.path.as_ref().unwrap().canonicalize().unwrap(),
lib.canonicalize().unwrap()
);
assert_eq!(d.source_sha256, Some(digest(&library)));
assert_eq!(d.span.line, line(&library, expected_primitive));
let last_frame = d.frames.last().unwrap();
assert_eq!(last_frame.function, expected_function);
assert_site(
last_frame.call_site.as_ref().unwrap(),
&entry,
&main,
line(&main, action),
);
assert!(d.format_message().contains(&format!(
"{}:{}:1",
entry.display(),
line(&main, action)
)));
let (_, values, _) = if backend == "tree" {
tree.foreign_source_evidence()
} else {
vm.foreign_source_evidence()
};
assert_eq!(values, vec![Value::Int(if output { 99 } else { 1 })]);
record(
&format!("foreign-{lang}-{output}"),
backend,
d,
vec![
source("entry", &entry, &main),
source("foreign", &lib, &library),
],
site("foreign", expected_primitive),
vec![
frame("main", site("entry", "# main"), None),
frame(
expected_function,
site("foreign", &format!("## {expected_function}")),
Some(site("entry", action)),
),
],
);
count += 1;
}
}
}
println!(
"SOURCE_MAP_METRIC:{}",
json!({"kind":"foreign","cases":count,"correct":true})
);
}
#[test]
fn memory_dynamic_and_method_calls_keep_their_own_sources() {
let directory = root("entry-kinds");
let memory="---\nmarkup: occlusion\nimport:\n json: lib/json.mq.md\n---\n# main\n**bad = > json.parse text=not-json**\n";
let module = crate::load::load_module_from_source(memory).unwrap();
let json_text = crate::embedded_lib::read_file("json.mq.md").unwrap();
let code = compile_module(None, &module).unwrap();
let dynamic="# main\n**name = \"explode\"**\n> call_fn name=`name`\n\n## explode\n**bad = 1 / 0**\n*bad*\n";
let dynamic_path = directory.join("dynamic.mq.md");
save(&dynamic_path, dynamic);
let dynamic_module = load_module(&dynamic_path).unwrap();
let dynamic_code = compile_module(Some(&dynamic_path), &dynamic_module).unwrap();
let object_path = directory.join("object.mq.md");
let json_path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("lib/json.mq.md");
let object=format!("---\nmarkup: occlusion\nimport:\n json: {}\n---\n# Counter\n**state = > json.parse text={{\"n\":1}}**\n*state*\n\n## fail\n**saved = `self`**\n**bad = 1 / 0**\n*bad*\n",json_path.display());
save(&object_path, &object);
let entry_path = directory.join("method.mq.md");
let method="---\nmarkup: occlusion\nimport:\n obj: object.mq.md\n---\n# main\n**c = > obj.Counter**\n> `c`.fail\n";
save(&entry_path, method);
let method_module = load_module(&entry_path).unwrap();
let method_code = compile_module(Some(&entry_path), &method_module).unwrap();
for backend in ["tree", "bytecode"] {
let error = if backend == "tree" {
Interpreter::with_capture(None, false).run_module(&module)
} else {
Vm::with_capture(None).run(&code)
}
.unwrap_err();
let d = Diagnostic::find(&error).unwrap();
assert_eq!(d.path.as_deref(), Some(Path::new("lib/json.mq.md")));
assert_eq!(d.source_sha256, Some(digest(&json_text)));
assert_eq!(d.span.line, line(&json_text, "host_json_parse"));
assert_eq!(
d.frames
.last()
.unwrap()
.call_site
.as_ref()
.unwrap()
.source
.path
.as_deref(),
Some(Path::new("<memory>"))
);
record(
"memory",
backend,
d,
vec![
source("entry", Path::new("<memory>"), memory),
source("json", Path::new("lib/json.mq.md"), &json_text),
],
site("json", "host_json_parse"),
vec![
frame("main", site("entry", "# main"), None),
frame(
"parse",
site("json", "## parse"),
Some(site("entry", "**bad")),
),
],
);
let error = if backend == "tree" {
Interpreter::with_capture(Some(&dynamic_path), false).run_module(&dynamic_module)
} else {
Vm::with_capture(Some(&dynamic_path)).run(&dynamic_code)
}
.unwrap_err();
let d = Diagnostic::find(&error).unwrap();
assert_origin(d, &dynamic_path, dynamic, line(dynamic, "**bad"));
assert_eq!(d.frames.last().unwrap().function, "explode");
assert_site(
d.frames.last().unwrap().call_site.as_ref().unwrap(),
&dynamic_path,
dynamic,
line(dynamic, "> call_fn"),
);
record(
"dynamic",
backend,
d,
vec![source("entry", &dynamic_path, dynamic)],
site("entry", "**bad"),
vec![
frame("main", site("entry", "# main"), None),
frame(
"explode",
site("entry", "## explode"),
Some(site("entry", "> call_fn")),
),
],
);
let error = if backend == "tree" {
Interpreter::with_capture(Some(&entry_path), false).run_module(&method_module)
} else {
Vm::with_capture(Some(&entry_path)).run(&method_code)
}
.unwrap_err();
let d = Diagnostic::find(&error).unwrap();
assert_origin(d, &object_path, &object, line(&object, "**bad"));
assert_eq!(d.frames.last().unwrap().function, "fail");
assert_site(
d.frames.last().unwrap().call_site.as_ref().unwrap(),
&entry_path,
method,
line(method, "> `c`.fail"),
);
record(
"method",
backend,
d,
vec![
source("entry", &entry_path, method),
source("object", &object_path, &object),
],
site("object", "**bad"),
vec![
frame("main", site("entry", "# main"), None),
frame(
"fail",
site("object", "## fail"),
Some(site("entry", "> `c`.fail")),
),
],
);
}
}
+452
View File
@@ -0,0 +1,452 @@
//! Export the actual loaded AST and independently observe both real backends.
//! Unsupported AST constructs are errors, never omitted or normalized to no-op.
use crate::ast::{Arg, BinaryOp, CallExpr, Expr, Function, Literal, Module, Stmt};
use crate::host::{foreign_session, json::value_to_json, HostContext};
use crate::value::Value;
use serde_json::{json, Value as Json};
use sha2::{Digest, Sha256};
use std::path::PathBuf;
pub(crate) fn boundary_evidence(
ctx: &mut HostContext,
) -> (Vec<Value>, Vec<Value>, Vec<foreign_session::ProtocolStep>) {
let observations = ctx.oracle_observations.clone();
let protocol = ctx.foreign_protocol_steps.clone();
let mut ids = ctx.foreign_sessions.keys().cloned().collect::<Vec<_>>();
ids.sort();
// These verification reads bypass the document observation API. They occur
// only after its journal was captured, and never alter that journal.
let values = ids
.iter()
.map(|id| {
foreign_session::get(ctx, &Value::Handle(id.clone()), &Value::Text("n".into())).unwrap()
})
.collect();
assert_eq!(observations, ctx.oracle_observations);
(observations, values, protocol)
}
fn expr(e: &Expr) -> Result<Json, String> {
Ok(match e {
Expr::Literal(Literal::None) => json!(["literal", null]),
Expr::Literal(Literal::Bool(b)) => json!(["literal", b]),
Expr::Literal(Literal::Int(n)) => json!(["literal", n]),
Expr::Literal(Literal::Text(s)) => json!(["literal", s]),
Expr::Var(name) => json!(["var", name]),
Expr::Binary { op, left, right } => {
let tag = match op {
BinaryOp::Add => "add",
BinaryOp::Sub => "sub",
BinaryOp::Lt => "lt",
_ => return Err(format!("unsupported source binary op {op:?}")),
};
json!([tag, expr(left)?, expr(right)?])
}
_ => return Err(format!("unsupported source expression {e:?}")),
})
}
fn call(c: &CallExpr, dest: Option<&str>, defs: &[&Function]) -> Result<Json, String> {
if c.receiver.is_some() || !c.pre_modifiers.is_empty() {
return Err("unsupported receiver/modifiers".into());
}
if let Some(path) = &c.path {
if path.len() != 2 || path[0] != "foreign" {
return Err(format!("unsupported library path {path:?}"));
}
let mut args = std::collections::BTreeMap::new();
for a in &c.args {
let Arg::Named { name, value } = a else {
return Err("foreign primitive needs named arguments".into());
};
if args.insert(name.as_str(), value).is_some() {
return Err("duplicate argument".into());
}
}
let allowed: &[&str] = match path[1].as_str() {
"session_open" => &["lang"],
"session_set" => &["id", "name", "value"],
"session_get" => &["id", "name"],
"session_eval" => &["id", "source", "input", "stdin"],
"session_close" => &["id"],
_ => return Err(format!("unsupported primitive {path:?}")),
};
if args.keys().any(|k| !allowed.contains(k)) {
return Err("unsupported primitive option".into());
}
let arg = |name: &str| {
args.get(name)
.copied()
.ok_or_else(|| format!("missing {name}"))
};
let name = || match arg("name")? {
Expr::Literal(Literal::Text(s)) => Ok(s.clone()),
_ => Err("dynamic binding name unsupported".to_string()),
};
match path[1].as_str() {
"session_open" => Ok(json!([
"open",
dest.ok_or("open destination required")?,
expr(arg("lang")?)?
])),
"session_get" => Ok(json!([
"get",
dest.ok_or("get destination required")?,
expr(arg("id")?)?,
name()?
])),
_ if dest.is_some() => {
Err("assigned set/eval/close record requires a richer adapter".into())
}
"session_set" => Ok(json!([
"set",
expr(arg("id")?)?,
name()?,
expr(arg("value")?)?
])),
"session_eval" => Ok(json!([
"eval",
expr(arg("id")?)?,
expr(arg("source")?)?,
args.get("input")
.map(|v| expr(v))
.transpose()?
.unwrap_or(json!(["literal", null])),
args.get("stdin").map(|v| expr(v)).transpose()?
])),
"session_close" => Ok(json!(["close", expr(arg("id")?)?])),
_ => unreachable!(),
}
} else {
let f = defs
.iter()
.find(|f| f.name == c.callee)
.ok_or_else(|| format!("unknown function {}", c.callee))?;
let mut values = std::collections::BTreeMap::new();
let mut positional = 0;
for a in &c.args {
let (name, value) = match a {
Arg::Named { name, value } => (name.as_str(), value),
Arg::Positional(value) => {
let p = f.params.get(positional).ok_or("too many arguments")?;
positional += 1;
(p.name.as_str(), value)
}
};
if !f.params.iter().any(|p| p.name == name) || values.insert(name, value).is_some() {
return Err("invalid call argument".into());
}
}
let ordered = f
.params
.iter()
.map(|p| expr(values.get(p.name.as_str()).ok_or("missing parameter")?))
.collect::<Result<Vec<_>, _>>()?;
Ok(json!(["call", dest, c.callee, ordered]))
}
}
fn block(body: &[Stmt], defs: &[&Function]) -> Result<Vec<Json>, String> {
body.iter()
.map(|s| {
Ok(match s {
Stmt::Assign {
name,
value: Expr::Call(c),
..
} => call(c, Some(name), defs)?,
Stmt::Assign { name, value, .. } => json!(["assign", name, expr(value)?]),
Stmt::Return { value, .. } => json!(["return", expr(value)?]),
Stmt::Call { call: c, .. } => call(c, None, defs)?,
Stmt::While {
condition, body, ..
} => json!(["while", expr(condition)?, block(body, defs)?]),
Stmt::Branch { arms, .. } => {
let mut rest = Vec::new();
for (i, arm) in arms.iter().enumerate().rev() {
let body = block(&arm.body, defs)?;
if let Some(condition) = &arm.condition {
rest = vec![json!(["if", expr(condition)?, body, rest])];
} else {
if i != arms.len() - 1 {
return Err("nonterminal else".into());
}
rest = body;
}
}
if rest.len() != 1 || rest[0][0] != "if" {
return Err("branch needs condition".into());
}
rest.pop().unwrap()
}
_ => return Err(format!("unsupported source statement {s:?}")),
})
})
.collect()
}
fn source_ir(module: &Module) -> Result<Json, String> {
if module.functions.len() != 1 || !module.uses.is_empty() {
return Err("module needs the declared single-entry adapter".into());
}
let main = module
.functions
.iter()
.find(|f| f.name == "main")
.ok_or("missing main")?;
let defs = main.children.iter().collect::<Vec<_>>();
if !main.params.is_empty() {
return Err("entry parameters need an explicit environment adapter".into());
}
let mut seen = std::collections::HashSet::new();
for f in &defs {
if !seen.insert(&f.name)
|| !f.children.is_empty()
|| f.params.iter().any(|p| p.default.is_some())
{
return Err("unsupported function scope/default".into());
}
}
let functions = defs.iter().map(|f| Ok(json!({"name":f.name,"params":f.params.iter().map(|p|p.name.clone()).collect::<Vec<_>>(),"body":block(&f.body,&defs)?}))).collect::<Result<Vec<_>,String>>()?;
let initial = module
.metadata
.iter()
.map(|(name, value)| {
if !matches!(
value,
Value::None | Value::Bool(_) | Value::Int(_) | Value::Text(_)
) {
return Err("non-scalar metadata needs an explicit initial-value adapter".into());
}
Ok(json!([name, value_to_json(value)?]))
})
.collect::<Result<Vec<_>, String>>()?;
Ok(json!({"initial":initial,"entry":block(&main.body,&defs)?,"functions":functions}))
}
fn source(lang: &str, seed: i64, rounds: usize, scenario: &str) -> String {
let increment = "n += 1; n";
let bad = "n = 99; 9007199254740992";
let stdin = if scenario == "explicit_empty_stdin" {
" stdin=\"\""
} else {
""
};
let condition = if scenario == "else_branch" {
seed - 1
} else {
seed + 1
};
let function = match scenario {
"recursive" => "advance_rec",
"early_return" => "early",
_ => "adjust",
};
let failure = match scenario {
"closed_get" => "**> foreign.session_close id=`alias`**\n**lost = > foreign.session_get id=`alias` name=\"n\"**".into(),
"copy_handle" => "**> foreign.session_set id=`a` name=\"n\" value=`b`**".into(),
"closed_copy" => "**> foreign.session_close id=`alias`**\n**> foreign.session_set id=`alias` name=\"n\" value=`b`**".into(),
"wide_input" => "**> foreign.session_set id=`a` name=\"n\" value=9007199254740992**".into(),
"wide_output" => format!("**> foreign.session_eval id=`a` source=\"{bad}\"**"),
_ => String::new(),
};
format!(
r#"---
markup: occlusion
import:
foreign: lib/foreign.mq.md
---
# main
**a = > foreign.session_open lang="{lang}"**
**b = > foreign.session_open lang="{lang}"**
**> foreign.session_set id=`a` name="n" value={seed}**
**> foreign.session_set id=`b` name="n" value=100**
**alias = `a`**
**counter = 777**
**before = > foreign.session_get id=`a` name="n"**
{failure}
1. `before` < {condition}
**answer = > {function} s=`alias` rounds={rounds}**
2. *
**> foreign.session_set id=`a` name="n" value={else_value}**
**after = > foreign.session_get id=`a` name="n"**
**peer = > foreign.session_get id=`b` name="n"**
**> foreign.session_close id=`alias`**
**c = > foreign.session_open lang="{lang}"**
**> foreign.session_set id=`c` name="n" value=7**
**fresh = > foreign.session_get id=`c` name="n"**
**> foreign.session_close id=`b`**
**> foreign.session_close id=`c`**
**total = after + peer + fresh + counter**
*total*
## adjust
+ `s`
+ `rounds`
**counter = 0**
- `counter` < `rounds`
> bump s=`s`
**counter = counter + 1**
**answer = > foreign.session_get id=`s` name="n"**
*answer*
## bump
+ `s`
> foreign.session_eval id=`s` source="{increment}"{stdin}
## advance_rec
+ `s`
+ `rounds`
1. 0 < `rounds`
> bump s=`s`
**remaining = rounds - 1**
**answer = > advance_rec s=`s` rounds=`remaining`**
*answer*
2. *
**answer = > foreign.session_get id=`s` name="n"**
*answer*
## early
+ `s`
+ `rounds`
**counter = 0**
- `counter` < `rounds`
**answer = > foreign.session_get id=`s` name="n"**
*answer*
**answer = > foreign.session_get id=`s` name="n"**
*answer*
"#,
else_value = seed + 20
)
}
#[test]
fn parsed_source_control_matches_independent_oracles_on_both_backends() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let out = std::env::var_os("MARQDO_SOURCE_ARTIFACT_DIR")
.map(PathBuf::from)
.unwrap_or_else(|| root.join("target/source-refinement"));
std::fs::create_dir_all(&out).unwrap();
let seeds = std::env::var("MARQDO_SOURCE_SEEDS")
.unwrap_or("3".into())
.split(',')
.map(|x| x.parse::<i64>().unwrap())
.collect::<Vec<_>>();
let lengths = std::env::var("MARQDO_SOURCE_ROUNDS")
.unwrap_or("0,3".into())
.split(',')
.map(|x| x.parse::<usize>().unwrap())
.collect::<Vec<_>>();
let scenarios = [
"normal",
"recursive",
"early_return",
"else_branch",
"closed_get",
"copy_handle",
"closed_copy",
"wide_input",
"wide_output",
"explicit_empty_stdin",
];
let mut count = 0;
for lang in ["python", "node"] {
for &seed in &seeds {
for &rounds in &lengths {
for scenario in scenarios {
let id = format!("{lang}-{seed}-{rounds}-{scenario}");
let text = source(lang, seed, rounds, scenario);
// Absolute import is only a fixture location; the actual module is loaded.
let text = text.replace(
"foreign: lib/foreign.mq.md",
&format!("foreign: {}", root.join("lib/foreign.mq.md").display()),
);
let path = out.join(format!("{id}.mq.md"));
std::fs::write(&path, &text).unwrap();
let module = crate::load::load_module(&path).unwrap();
let ir = source_ir(&module).unwrap_or_else(|e| panic!("{id}: AST export: {e}"));
let mut executions = Vec::new();
for backend in ["tree", "bytecode"] {
let (result, journal, values, protocol) = if backend == "tree" {
let mut interpreter =
crate::interp::Interpreter::with_capture(Some(&path), false);
let result = interpreter.run_module(&module);
let (journal, values, protocol) = interpreter.foreign_source_evidence();
assert!(interpreter.captured_stdout.is_empty(), "unexpected stdout");
(result, journal, values, protocol)
} else {
let code =
crate::bytecode::compile_module(Some(&path), &module).unwrap();
let mut vm = crate::bytecode::Vm::with_capture(Some(&path));
let result = vm.run(&code);
let (journal, values, protocol) = vm.foreign_source_evidence();
assert!(vm.captured_stdout.is_empty(), "unexpected stdout");
(result, journal, values, protocol)
};
let failure = matches!(
scenario,
"closed_get"
| "copy_handle"
| "closed_copy"
| "wide_input"
| "wide_output"
);
assert_eq!(result.is_err(), failure, "{id} {backend}: {result:?}");
if !failure {
let expected = seed
+ if scenario == "early_return" {
0
} else if scenario == "else_branch" {
20
} else {
rounds as i64
};
assert_eq!(
result.as_ref().unwrap(),
&Value::Int(expected + 884),
"{id} {backend}"
);
}
let error = result.as_ref().err().map(|e| e.to_string());
let diagnostic = result.as_ref().err().and_then(crate::diagnostics::Diagnostic::find).map(crate::diagnostics::Diagnostic::to_json);
let fault = if let Some(e) = &error {
if e.contains("not found") {
"unknown"
} else if e.contains("outside V_safe")
|| e.contains("cannot be serialized")
{
if protocol.last().is_some_and(|p| {
p.response.as_ref().is_ok_and(|reply| {
reply.get("domain_error").and_then(Json::as_bool)
== Some(true)
})
}) {
"outputDomain"
} else {
"inputDomain"
}
} else {
panic!("{id} {backend}: unclassified error {e}")
}
} else {
"none"
};
let protocol = protocol.iter().map(|p| json!({"session":p.session,"request":p.request,"response":p.response})).collect::<Vec<_>>();
executions.push(json!({"backend":backend,"ok":result.is_ok(),"result":result.ok().map(|v|value_to_json(&v).unwrap()),"error":error,"fault":fault,"protocol":protocol,"diagnostic":diagnostic,
"journal":value_to_json(&Value::List(journal)).unwrap(),"live_values":value_to_json(&Value::List(values)).unwrap()}));
}
std::fs::write(out.join(format!("{id}.json")),serde_json::to_vec_pretty(&json!({"version":1,"id":id,"language":lang,"seed":seed,"rounds":rounds,"scenario":scenario,"source_sha256":format!("{:x}",Sha256::digest(text.as_bytes())),"ast":ir,"executions":executions})).unwrap()).unwrap();
count += 1;
}
}
}
}
println!(
"SOURCE_METRIC:{}",
json!({"cases":count,"executions":count*2,"scenarios":scenarios,"correct":true})
);
}
+43
View File
@@ -4,6 +4,44 @@ use std::fmt;
use crate::formula::Expr as FormulaExpr;
/// An object's defining module instance, expressed as import binding segments.
/// This is ordinary copied data, not an unforgeable session capability.
pub(crate) const OBJECT_MODULE_KEY: &str = "_type_module";
pub(crate) fn instance_module(value: &Value) -> Result<Option<Vec<String>>, String> {
let Value::Map(fields) = value else { return Ok(None); };
let Some((_, module)) = fields.iter().find(|(key, _)| key == OBJECT_MODULE_KEY) else { return Ok(None); };
let Value::List(segments) = module else { return Err("object module tag must be a list of text segments".into()); };
segments.iter().map(|segment| match segment {
Value::Text(name) => Ok(name.clone()),
_ => Err("object module tag must be a list of text segments".into()),
}).collect::<Result<Vec<_>,_>>().map(Some)
}
pub(crate) fn tag_instance(type_name: &str, module: &[String], value: Value) -> Value {
let mut fields = match value {
Value::Map(fields) => fields,
Value::None => Vec::new(),
other => vec![("value".into(),other)],
};
fields.retain(|(key,_)| key != "_type" && key != OBJECT_MODULE_KEY);
fields.insert(0,("_type".into(),Value::Text(type_name.into())));
fields.insert(1,(OBJECT_MODULE_KEY.into(),Value::List(module.iter().cloned().map(Value::Text).collect())));
Value::Map(fields)
}
pub(crate) fn preserve_instance_module(receiver: &Value, value: Value) -> Value {
let Ok(Some(module)) = instance_module(receiver) else { return value; };
match value {
Value::Map(mut fields) => {
fields.retain(|(key,_)| key != OBJECT_MODULE_KEY);
fields.insert(1.min(fields.len()),(OBJECT_MODULE_KEY.into(),Value::List(module.into_iter().map(Value::Text).collect())));
Value::Map(fields)
}
other => other,
}
}
/// Bound Markdown code fence (`type` → `code`).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CodeBlock {
@@ -29,6 +67,8 @@ pub enum Value {
Code(CodeBlock),
/// Sensitive text from `${secret.*}` — Display shows `<secret>`; reveal on Text concat.
Secret(String),
/// Opaque foreign-session handle. Not JSON-serializable; not printable structure.
Handle(String),
}
impl Value {
@@ -40,6 +80,7 @@ impl Value {
Value::Num(n) => *n != 0.0 && !n.is_nan(),
Value::Text(s) => !s.is_empty(),
Value::Secret(s) => !s.is_empty(),
Value::Handle(s) => !s.is_empty(),
Value::List(xs) => !xs.is_empty(),
Value::Map(xs) => !xs.is_empty(),
Value::Formula(_) => true,
@@ -64,6 +105,7 @@ impl Value {
Value::Num(n) => crate::formula::format_num(*n),
Value::Text(s) => s.clone(),
Value::Secret(_) => "<secret>".into(),
Value::Handle(_) => "<handle>".into(),
Value::List(xs) => {
let parts: Vec<String> = xs.iter().map(|v| v.as_display()).collect();
format!("[{}]", parts.join(", "))
@@ -71,6 +113,7 @@ impl Value {
Value::Map(pairs) => {
let parts: Vec<String> = pairs
.iter()
.filter(|(key,_)| key != OBJECT_MODULE_KEY)
.map(|(k, v)| format!("{k}: {}", v.as_display()))
.collect();
format!("{{{}}}", parts.join(", "))
+57
View File
@@ -916,6 +916,17 @@ fn render_line_block(
if let Some(body) = writeback_body {
s.push_str(&crate::view::render::render_output_card(line_no, body));
}
// Must close `.doc-block` — an early return left the wrapper open and
// nested the Functions/Variables outline under the document surface.
let band_annots: Vec<ViewAnnot> = annots
.iter()
.filter(|a| !is_noisy_result_chip(a))
.map(|a| (*a).clone())
.collect();
if !band_annots.is_empty() {
s.push_str(&render_annot_band(&band_annots, MAX_VISIBLE_ANNOTS));
}
s.push_str("</div>");
return s;
}
if bits.is_empty() {
@@ -1545,6 +1556,52 @@ When we meet `who`, we **print text="Hi, `who`!"** and close with*None*.
}
}
#[cfg(test)]
mod occl_render_tests {
use super::render_module_document;
fn div_balance(html: &str) -> i32 {
let mut depth = 0i32;
let b = html.as_bytes();
let mut i = 0usize;
while i + 3 < b.len() {
if b[i] == b'<' {
if i + 4 <= b.len() && &b[i..i + 4] == b"<div" && (b.get(i + 4) == Some(&b' ') || b.get(i + 4) == Some(&b'>')) {
depth += 1;
} else if i + 6 <= b.len() && &b[i..i + 6] == b"</div>" {
depth -= 1;
}
}
i += 1;
}
depth
}
#[test]
fn occlusion_line_closes_doc_block() {
let src = r#"---
markup: occlusion
---
# main
`v` = [描述](1)
"#;
let module = crate::parse::parse_source(src).unwrap();
let html = render_module_document(&module, src, &[]);
assert!(html.contains("doc-occl"), "{html}");
assert_eq!(
div_balance(&html),
0,
"occlusion must not leave .doc-block open: {html}"
);
assert!(
html.contains("doc-occl") && html.contains("</div></div>"),
"doc-block around occlusion must close: {html}"
);
}
}
#[cfg(test)]
mod occl_split_tests {
use super::split_occlusion;
+34 -2
View File
@@ -237,6 +237,11 @@ a:focus-visible { outline: 2px solid var(--focus); outline-offset: 2px; }
min-width: 0;
overflow: auto;
background: var(--bg);
/* Content-pane width (excludes file nav) drives outline stacking — not the
* viewport. A 960px viewport media query wrongly put Functions/Variables
* under a tall Structure while the left nav was still visible. */
container-type: inline-size;
container-name: mq-panel;
}
main.panel {
padding: 2rem 2.25rem 3.5rem;
@@ -719,9 +724,17 @@ body.notebook-page .exec-clear:hover { border-color: var(--nb-accent); color: va
gap: 0.75rem;
align-items: start;
}
@media (max-width: 960px) {
/* Stack only when the *panel* is too narrow for Structure + outline rail.
* Prefer keeping the rail beside the doc at tablet widths (matches ≤800px
* shell collapse in doc/design/view.md). When stacked, pull the rail above
* so it is not buried under a long document. */
@container mq-panel (max-width: 32rem) {
.struct-wrap { grid-template-columns: 1fr; }
.outline-rail, .outline-panel { position: static; }
.outline-rail {
order: -1;
position: static;
}
.outline-panel { position: static; }
}
.outline-rail {
position: sticky;
@@ -4527,6 +4540,25 @@ mod plan_card_tests {
mod tests {
use super::*;
#[test]
fn outline_rail_stacks_by_panel_container_not_960_viewport() {
let css = stylesheet();
assert!(
css.contains("container-name: mq-panel"),
"panel-wrap should be a size container for outline layout"
);
assert!(
css.contains("@container mq-panel (max-width: 32rem)"),
"outline must stack from panel width, not a mid-width viewport media query"
);
assert!(
!css.contains("@media (max-width: 960px) {\n .struct-wrap"),
"960px viewport media wrongly buried the outline under Structure"
);
let html = layout("t", "<p>nav</p>", "<p>main</p>");
assert!(html.contains("container-name: mq-panel"), "{html}");
}
#[test]
fn notebook_ui_has_controls() {
let ui = notebook_ui();
+2
View File
@@ -199,6 +199,7 @@ const API_ROUTES: &[(&str, ApiRoute)] = &[
("/api/session/snapshot/export", ApiRoute::Notebook(notebook_api::api_export_snapshot)),
("/api/session/snapshot/import", ApiRoute::Notebook(notebook_api::api_import_snapshot)),
("/api/session/snapshot/replay", ApiRoute::Notebook(notebook_api::api_replay_snapshot)),
("/api/session/snapshot/replay_check", ApiRoute::Notebook(notebook_api::api_replay_check)),
("/api/session/plan/assign", ApiRoute::Notebook(notebook_api::api_plan_assign)),
("/api/session/plan/add", ApiRoute::Notebook(notebook_api::api_plan_add)),
("/api/session/plan/remove", ApiRoute::Notebook(notebook_api::api_plan_remove)),
@@ -216,6 +217,7 @@ const API_ROUTES: &[(&str, ApiRoute)] = &[
("/api/session/state/push", ApiRoute::Notebook(notebook_api::api_state_push)),
("/api/session/state/pull", ApiRoute::Notebook(notebook_api::api_state_pull)),
("/api/session/journal", ApiRoute::Notebook(notebook_api::api_journal)),
("/api/session/oracle_journal", ApiRoute::Notebook(notebook_api::api_oracle_journal)),
("/api/debug/start", ApiRoute::Notebook(debug_api::api_debug_start)),
("/api/debug/continue", ApiRoute::Action(crate::debug::DebugAction::Continue)),
("/api/debug/step", ApiRoute::Action(crate::debug::DebugAction::Step)),
+199 -15
View File
@@ -1,15 +1,40 @@
//! Live-view notebook session APIs.
#[cfg(all(test, unix))]
mod interrupt_http_tests;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, OnceLock};
use std::sync::{Arc, Mutex, OnceLock, MutexGuard, TryLockError};
use std::sync::atomic::{AtomicBool, Ordering};
use crate::host::json::{json_to_value, value_to_json};
use crate::host::foreign_doc::DocumentSession;
use crate::value::Value;
fn sessions() -> &'static Mutex<HashMap<String, Arc<Mutex<DocumentSession>>>> {
static S: OnceLock<Mutex<HashMap<String, Arc<Mutex<DocumentSession>>>>> = OnceLock::new();
struct ApiSession {
closed: AtomicBool,
document: OnceLock<Result<Mutex<DocumentSession>, String>>,
control: crate::host::foreign_doc::DocumentControl,
}
impl ApiSession {
fn mutex(&self) -> Result<&Mutex<DocumentSession>, String> {
// `OnceLock::wait` needs Rust ≥1.84; MSRV is 1.81 — poll until init.
loop {
match self.document.get() {
Some(slot) => return slot.as_ref().map_err(Clone::clone),
None => std::thread::yield_now(),
}
}
}
fn lock(&self) -> Result<MutexGuard<'_, DocumentSession>, String> {
let guard = self.mutex()?.lock().map_err(|_| "session lock poisoned".to_string())?;
if self.closed.load(Ordering::Acquire) { return Err("session closed".into()); }
Ok(guard)
}
}
fn sessions() -> &'static Mutex<HashMap<String, Arc<ApiSession>>> {
static S: OnceLock<Mutex<HashMap<String, Arc<ApiSession>>>> = OnceLock::new();
S.get_or_init(|| Mutex::new(HashMap::new()))
}
@@ -17,7 +42,7 @@ fn body_value(body: &str) -> serde_json::Value {
serde_json::from_str(body).unwrap_or(serde_json::Value::Null)
}
fn path_of(root: &Path, v: &serde_json::Value) -> Result<(String, PathBuf), String> {
fn requested_path(root: &Path, v: &serde_json::Value) -> Result<(String, PathBuf), String> {
let rel = v
.get("path")
.and_then(|p| p.as_str())
@@ -28,21 +53,37 @@ fn path_of(root: &Path, v: &serde_json::Value) -> Result<(String, PathBuf), Stri
return Err("missing path".into());
}
let abs = root.join(rel.replace('/', std::path::MAIN_SEPARATOR_STR));
Ok((rel, abs))
}
fn path_of(root: &Path, v: &serde_json::Value) -> Result<(String, PathBuf), String> {
let (rel, abs) = requested_path(root, v)?;
if !abs.is_file() {
return Err(format!("file not found: {rel}"));
}
Ok((rel, abs))
}
fn get_or_open(root: &Path, abs: &Path) -> Result<Arc<Mutex<DocumentSession>>, String> {
fn get_or_open(root: &Path, abs: &Path) -> Result<Arc<ApiSession>, String> {
let key = abs.display().to_string();
let mut map = sessions().lock().map_err(|_| "session lock poisoned".to_string())?;
if let Some(s) = map.get(&key) {
return Ok(Arc::clone(s));
let arc = {
let mut map = sessions().lock().map_err(|_| "session lock poisoned".to_string())?;
Arc::clone(map.entry(key.clone()).or_insert_with(|| Arc::new(ApiSession {
closed: AtomicBool::new(false),
control: Default::default(), document: OnceLock::new(),
})))
};
// Kernel startup may block. It must never hold the registry lock needed by
// interrupt requests for an unrelated already-running document.
let initialized = arc.document.get_or_init(|| DocumentSession::open(abs, root).map(|mut session| {
session.adopt_control(arc.control.clone());
Mutex::new(session)
}));
if let Err(error) = initialized {
let mut map = sessions().lock().map_err(|_| "session lock poisoned".to_string())?;
if map.get(&key).is_some_and(|old| Arc::ptr_eq(old, &arc)) { map.remove(&key); }
return Err(error.clone());
}
let session = DocumentSession::open(abs, root)?;
let arc = Arc::new(Mutex::new(session));
map.insert(key, Arc::clone(&arc));
Ok(arc)
}
@@ -137,6 +178,7 @@ pub fn api_close(root: &Path, body: &str) -> serde_json::Value {
.and_then(|mut m| m.remove(&key));
if let Some(arc) = removed {
if let Ok(mut s) = arc.lock() {
arc.closed.store(true, Ordering::Release);
s.close();
}
}
@@ -181,17 +223,54 @@ pub fn api_persist_cell(root: &Path, body: &str) -> serde_json::Value {
}
pub fn api_interrupt(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "interrupted", |s| s.interrupt())
let v = body_value(body);
// A control request addresses an existing loaded session. It neither reads
// the source again nor creates a process as a side effect of cancellation.
let (_, abs) = match requested_path(root, &v) { Ok(p) => p, Err(e) => return err(e) };
let session = match sessions().lock() {
Ok(map) => match map.get(&abs.display().to_string()) { Some(s) => s.clone(), None => return err("no open session".into()) },
Err(_) => return err("session lock poisoned".into()),
};
if session.closed.load(Ordering::Acquire) { return err("session closed".into()); }
let mutex = match session.document.get() {
Some(Ok(mutex)) => mutex, Some(Err(error)) => return err(error.clone()),
None => return err("session is still opening; no evaluation is active".into()),
};
let result = match mutex.try_lock() {
Ok(mut document) => if session.closed.load(Ordering::Acquire) { err("session closed".into()) } else { match document.interrupt() {
Ok(value) => ok_session(&document, serde_json::json!({ "interrupted": json_value(&value), "in_flight": false })),
Err(error) => err(error),
} },
Err(TryLockError::WouldBlock) => match session.control.interrupt() {
Ok(value) => {
let receipt = json_value(&value);
let accepted = receipt["results"].as_array().is_some_and(|rows| rows.iter().all(|row| row["accepted"] == true));
if accepted { serde_json::json!({ "ok": true, "interrupted": receipt, "in_flight": true }) }
else { serde_json::json!({ "ok": false, "error": "interrupt delivery failed", "interrupted": receipt, "in_flight": true }) }
},
Err(error) => err(error),
},
Err(TryLockError::Poisoned(_)) => err("session lock poisoned".into()),
};
result
}
pub fn api_export_snapshot(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "snapshot", |s| s.export_snapshot())
let v = body_value(body);
let compact = match v.get("compact") { None | Some(serde_json::Value::Bool(false)) => false,
Some(serde_json::Value::Bool(true)) => true, _ => return err("compact must be boolean".into()) };
with_session(root, body, "snapshot", |s| {
let snapshot = s.export_snapshot()?;
if compact { crate::host::foreign_doc::packing::pack(&snapshot) } else { Ok(snapshot) }
})
}
pub fn api_import_snapshot(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let snapshot = v.get("snapshot").cloned().unwrap_or(serde_json::Value::Null);
let snapshot = match json_to_value(&snapshot) {
let snapshot = match json_to_value(&snapshot).and_then(|value| if crate::host::foreign_doc::packing::is_packed(&value) {
crate::host::foreign_doc::packing::unpack(&value)
} else { Ok(value) }) {
Ok(v) => v,
Err(e) => return err(e),
};
@@ -201,13 +280,40 @@ pub fn api_import_snapshot(root: &Path, body: &str) -> serde_json::Value {
pub fn api_replay_snapshot(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let snapshot = v.get("snapshot").cloned().unwrap_or(serde_json::Value::Null);
let snapshot = match json_to_value(&snapshot) {
let snapshot = match json_to_value(&snapshot).and_then(|value| if crate::host::foreign_doc::packing::is_packed(&value) {
crate::host::foreign_doc::packing::unpack(&value)
} else { Ok(value) }) {
Ok(v) => v,
Err(e) => return err(e),
};
with_session(root, body, "replay", |s| s.replay_snapshot(&snapshot))
}
pub fn api_replay_check(root: &Path, body: &str) -> serde_json::Value {
// Decode before serde_json::Value can overwrite duplicate recording fields.
let request = match crate::host::json::parse(&Value::Text(body.to_string())) {
Ok(value) => value,
Err(error) => return err(error),
};
let v = match value_to_json(&request) { Ok(v) => v, Err(e) => return err(e) };
let (_, abs) = match path_of(root, &v) { Ok(path) => path, Err(e) => return err(e) };
let Value::Map(fields) = request else { return err("request must be a map".into()) };
let Some((_, snapshot)) = fields.iter().find(|(k,_)| k == "snapshot") else { return err("missing snapshot".into()) };
let check = match DocumentSession::check_snapshot_isolated(&abs, root, snapshot) {
Ok(check) => check, Err(error) => return err(error),
};
let existing = match sessions().lock() {
Ok(map) => map.get(&abs.display().to_string()).cloned(),
Err(_) => return err("session lock poisoned".into()),
};
let extra = serde_json::json!({"replay_check": json_value(&check)});
if let Some(arc) = existing {
match arc.lock() { Ok(session) => ok_session(&session, extra), Err(_) => err("session lock poisoned".into()) }
} else {
serde_json::json!({"ok": true, "replay_check": json_value(&check)})
}
}
pub fn api_clear_cell(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
@@ -352,6 +458,10 @@ pub fn api_journal(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "journal", |s| Ok(s.journal_json()))
}
pub fn api_oracle_journal(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "oracle_journal", |s| Ok(s.oracle_journal_json()))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -429,3 +539,77 @@ mod tests {
let _ = api_close(&root, open_body());
}
}
#[cfg(test)]
mod replay_api_tests {
use super::*;
#[test]
fn closed_api_session_cannot_be_reactivated_after_path_reopen() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let relative = format!("target/api-close-capability-{}.mq.md", std::process::id());
let path = root.join(&relative);
std::fs::write(&path, "---\nimport:\n foreign: lib/foreign.mq.md\n---\n# main\n\n`会话` = > foreign.session\n\n`a` =\n```python\nn = 1\n```\n").unwrap();
let old = get_or_open(&root, &path).unwrap();
old.lock().unwrap().run_all(false).unwrap();
let body = serde_json::json!({"path": relative}).to_string();
assert_eq!(api_close(&root, &body)["ok"], true);
let fresh = get_or_open(&root, &path).unwrap();
assert!(!Arc::ptr_eq(&old, &fresh));
assert!(old.lock().is_err());
fresh.lock().unwrap().run_all(false).unwrap();
assert_eq!(api_close(&root, &body)["ok"], true);
std::fs::remove_file(path).unwrap();
}
#[test]
fn strict_replay_endpoint_uses_isolation_and_preserves_warm_session() {
let root=PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let relative=format!("target/api-replay-isolated-{}.mq.md",std::process::id());let path=root.join(&relative);
std::fs::write(&path,"---\nimport:\n foreign: lib/foreign.mq.md\n---\n# main\n\n`会话` = > foreign.session\n\n`a` =\n```python\nstate = globals().get('state',0) + 1\nprint(state)\n```\n").unwrap();
let arc=get_or_open(&root,&path).unwrap();
let (snapshot,kernel)={let mut live=arc.lock().unwrap();live.run_all(false).unwrap();let snapshot=live.export_snapshot().unwrap();let kernel=live.cells[0].kernel.clone();live.set_value(&kernel,"state",Value::Int(99)).unwrap();(snapshot,kernel)};
let before={let live=arc.lock().unwrap();live.summary()};
let body=serde_json::json!({"path":relative,"snapshot":value_to_json(&snapshot).unwrap()}).to_string();
let response=api_replay_check(&root,&body);
assert_eq!(response["ok"],true,"{response}");assert_eq!(response["replay_check"]["ok"],true,"{response}");
assert_eq!(arc.lock().unwrap().summary(),before,"verification mutated live state");
let duplicate=body.replacen("\"snapshot\":", "\"snapshot\":null,\"snapshot\":",1);
let bad=api_replay_check(&root,&duplicate);assert_eq!(bad["ok"],false);assert!(bad["error"].as_str().unwrap().contains("duplicate"));
assert_eq!(arc.lock().unwrap().summary(),before);
assert_eq!(arc.lock().unwrap().pull_value(&kernel,"state").unwrap(),Value::Int(99),"isolated verification reset the user's actual kernel");
sessions().lock().unwrap().remove(&path.display().to_string());arc.lock().unwrap().close();std::fs::remove_file(path).unwrap();
}
#[cfg(unix)]
#[test]
fn malformed_replay_does_not_start_declared_seed_kernel() {
use std::os::unix::fs::PermissionsExt;
let root=PathBuf::from(env!("CARGO_MANIFEST_DIR"));let pid=std::process::id();
let relative=format!("target/api-replay-seeded-{pid}.mq.md");let path=root.join(&relative);
let wrapper=root.join(format!("target/api-replay-wrapper-{pid}.py"));let marker=root.join(format!("target/api-replay-marker-{pid}"));
let marker_literal=serde_json::to_string(&marker.display().to_string()).unwrap();
let script=format!("#!/usr/bin/python3\nimport os,sys\nfrom pathlib import Path\nPath({marker_literal}).write_text('started')\nos.execv(sys.executable,[sys.executable]+sys.argv[1:])\n");
std::fs::write(&wrapper,script).unwrap();std::fs::set_permissions(&wrapper,std::fs::Permissions::from_mode(0o755)).unwrap();
let command=serde_json::to_string(&wrapper.display().to_string()).unwrap();
let source=format!("---\nimport:\n foreign: lib/foreign.mq.md\n---\n# main\n\n`kernels` =\n| 名称 | 语言 | 命令 |\n|---|---|---|\n| py | python | {command} |\n\n`units` =\n| 单元 | 内核 |\n|---|---|\n| a | py |\n\n`initial` =\n| 内核 | 名称 | 值 |\n|---|---|---|\n| py | n | 1 |\n\n`会话` = > foreign.session kernels=`kernels` cells=`units` values=`initial`\n\n`a` =\n```python\nprint(n)\n```\n");
std::fs::write(&path,source).unwrap();let mut recording=DocumentSession::open(&path,&root).unwrap();
recording.run_all(false).unwrap();let snapshot=recording.export_snapshot().unwrap();recording.close();assert!(marker.exists());std::fs::remove_file(&marker).unwrap();
let valid=value_to_json(&snapshot).unwrap();let mut malformed=valid.clone();malformed["actions"].as_array_mut().unwrap().pop();
let cold = api_interrupt(&root, &serde_json::json!({"path":relative}).to_string());
assert_eq!(cold["ok"], false); assert!(!marker.exists(), "cold interrupt started a kernel");
let body=serde_json::json!({"path":relative,"snapshot":malformed}).to_string();let response=api_replay_check(&root,&body);
assert_eq!(response["ok"],false,"{response}");assert!(!marker.exists(),"malformed suffix started a seed kernel before preflight");
assert!(!sessions().lock().unwrap().contains_key(&path.display().to_string()));
let packed = crate::host::foreign_doc::packing::pack(&snapshot).unwrap();
let mut corrupt = value_to_json(&packed).unwrap(); corrupt["sha256"] = "bad".into();
let response = api_replay_check(&root, &serde_json::json!({"path":relative,"snapshot":corrupt}).to_string());
assert_eq!(response["ok"], false); assert!(!marker.exists(), "corrupt DAG started a seed kernel");
let mut controls = valid.clone();
controls["actions"][0]["controls"] = serde_json::json!([{"after_us":-1,"results":[]}]);
let response = api_replay_check(&root, &serde_json::json!({"path":relative,"snapshot":controls}).to_string());
assert_eq!(response["ok"], false); assert!(!marker.exists(), "invalid control started a seed kernel");
let response=api_replay_check(&root,&serde_json::json!({"path":relative,"snapshot":valid}).to_string());
assert_eq!(response["replay_check"]["ok"],true,"{response}");assert!(marker.exists());
assert!(!sessions().lock().unwrap().contains_key(&path.display().to_string()));
std::fs::remove_file(path).unwrap();std::fs::remove_file(wrapper).unwrap();std::fs::remove_file(marker).unwrap();
}
}
@@ -0,0 +1,296 @@
use super::*;
use std::io::{Read, Write};
use std::net::{SocketAddr, TcpStream};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
struct HttpServer {
addr: SocketAddr,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl HttpServer {
fn start(root: &Path) -> Self {
let root = Arc::new(crate::view::build_root(root).unwrap());
let server = tiny_http::Server::http("127.0.0.1:0").unwrap();
let addr = server.server_addr().to_ip().unwrap();
let stop = Arc::new(AtomicBool::new(false));
let stopping = stop.clone();
let thread = std::thread::spawn(move || {
let mut workers = Vec::new();
while !stopping.load(Ordering::SeqCst) {
if let Some(request) = server.recv_timeout(Duration::from_millis(50)).unwrap() {
let root = root.clone();
workers.push(std::thread::spawn(move || {
crate::view::handle(&root, request)
}));
}
}
for worker in workers {
let _ = worker.join();
}
});
Self {
addr,
stop,
thread: Some(thread),
}
}
}
impl Drop for HttpServer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(thread) = self.thread.take() {
let _ = thread.join();
}
}
}
fn post(addr: SocketAddr, endpoint: &str, body: &serde_json::Value) -> serde_json::Value {
let bytes = body.to_string();
let mut stream = TcpStream::connect(addr).unwrap();
stream
.set_read_timeout(Some(Duration::from_secs(20)))
.unwrap();
write!(stream, "POST /api/session/{endpoint} HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{bytes}", bytes.len()).unwrap();
let mut response = String::new();
stream.read_to_string(&mut response).unwrap();
let (_, body) = response.split_once("\r\n\r\n").unwrap();
serde_json::from_str(body).unwrap_or_else(|e| panic!("{endpoint}: {e}: {response}"))
}
#[test]
fn real_http_interrupt_preserves_store_stops_batch_and_replays_control_input() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let server = HttpServer::start(&root);
let repeats: usize = std::env::var("MARQDO_CANCEL_REPEATS")
.map(|s| s.parse().unwrap())
.unwrap_or(1);
assert!(repeats > 0);
for language in ["python", "node"] {
for mode in ["cell/run", "cell/step", "run_all"] {
for trial in 0..repeats {
let id = format!(
"{}-{}-{language}",
std::process::id(),
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos()
);
let relative = format!("target/http-cancel-{id}.mq.md");
let path = root.join(&relative);
let marker = root.join(format!("target/http-cancel-started-{id}"));
let suffix = root.join(format!("target/http-cancel-suffix-{id}"));
let marker_literal = serde_json::to_string(&marker.display().to_string()).unwrap();
let suffix_literal = serde_json::to_string(&suffix.display().to_string()).unwrap();
let (busy, after) = if language == "python" {
(format!("n = 99\nopen({marker_literal}, 'w').write('started')\nwhile True: pass"),
format!("open({suffix_literal}, 'w').write('unexpected')\nn = 100"))
} else {
(format!("n = 99; require('fs').writeFileSync({marker_literal}, 'started'); while (true) {{}}"),
format!("require('fs').writeFileSync({suffix_literal}, 'unexpected'); n = 100;"))
};
let source = format!("---\nimport:\n foreign: lib/foreign.mq.md\n---\n# main\n\n`kernels` =\n| 名称 | 语言 |\n|---|---|\n| worker | {language} |\n\n`units` =\n| 单元 | 内核 |\n|---|---|\n| busy | worker |\n| after | worker |\n\n`initial` =\n| 内核 | 名称 | 值 |\n|---|---|---|\n| worker | n | 1 |\n\n`会话` = > foreign.session kernels=`kernels` cells=`units` values=`initial`\n\n`busy` =\n```{language}\n{busy}\n```\n\n`after` =\n```{language}\n{after}\n```\n");
std::fs::write(&path, &source).unwrap();
let body = serde_json::json!({"path": relative});
assert_eq!(post(server.addr, "open", &body)["ok"], true);
let addr = server.addr;
let mut running_body = body.clone();
if mode == "cell/run" {
running_body["cell"] = "busy".into();
}
let runner = std::thread::spawn(move || post(addr, mode, &running_body));
let deadline = Instant::now() + Duration::from_secs(3);
while !marker.exists() && Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(2));
}
assert!(
marker.exists(),
"{language}: evaluation never reached its side effect"
);
let slow_open = if language == "python" && mode == "run_all" && trial == 0 {
let slow_relative = format!("target/http-slow-open-{id}.mq.md");
let slow_path = root.join(&slow_relative);
let wrapper = root.join(format!("target/http-slow-wrapper-{id}.py"));
let started = root.join(format!("target/http-slow-started-{id}"));
let literal = serde_json::to_string(&started.display().to_string()).unwrap();
std::fs::write(&wrapper, format!("import os,sys,time\nfrom pathlib import Path\nPath({literal}).write_text('opening')\ntime.sleep(2)\nos.execv(sys.executable,[sys.executable]+sys.argv[1:])\n")).unwrap();
let command =
serde_json::to_string(&format!("python3 {}", wrapper.display())).unwrap();
std::fs::write(&slow_path, format!("---\nimport:\n foreign: lib/foreign.mq.md\n---\n# main\n\n`kernels` =\n| 名称 | 语言 | 命令 |\n|---|---|---|\n| slow | python | {command} |\n\n`initial` =\n| 内核 | 名称 | 值 |\n|---|---|---|\n| slow | n | 1 |\n\n`会话` = > foreign.session kernels=`kernels` values=`initial`\n")).unwrap();
let slow_body = serde_json::json!({"path":slow_relative});
let opening_body = slow_body.clone();
let opener = std::thread::spawn(move || post(addr, "open", &opening_body));
let deadline = Instant::now() + Duration::from_secs(3);
while !started.exists() && Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(2));
}
assert!(started.exists(), "unrelated slow kernel never started");
let start = Instant::now();
let pending = post(addr, "interrupt", &slow_body);
assert_eq!(pending["ok"], false, "{pending}");
assert!(
start.elapsed() < Duration::from_secs(1),
"control waited for initialization"
);
Some((opener, slow_body, slow_path, wrapper, started))
} else {
None
};
std::thread::sleep(Duration::from_millis([0, 5, 25, 50][trial % 4]));
let removed_source = language == "node" && mode == "cell/run" && trial == 0;
if removed_source {
std::fs::remove_file(&path).unwrap();
}
let start = Instant::now();
let interrupted = post(server.addr, "interrupt", &body);
let latency = start.elapsed();
assert!(
latency < Duration::from_secs(1),
"interrupt queued behind execution: {latency:?}"
);
assert_eq!(interrupted["ok"], true, "{interrupted}");
assert_eq!(interrupted["in_flight"], true, "{interrupted}");
assert_eq!(
interrupted["interrupted"]["results"][0]["active"], true,
"{interrupted}"
);
let run = runner.join().unwrap();
let completion_latency = start.elapsed();
assert!(
completion_latency < Duration::from_secs(1),
"cancel did not finish execution promptly: {completion_latency:?}"
);
if removed_source {
std::fs::write(&path, &source).unwrap();
}
if let Some((opener, slow_body, slow_path, wrapper, started)) = slow_open {
assert_eq!(opener.join().unwrap()["ok"], true);
assert_eq!(post(server.addr, "close", &slow_body)["ok"], true);
for file in [slow_path, wrapper, started] {
std::fs::remove_file(file).unwrap();
}
}
let record = if mode == "run_all" {
assert_eq!(run["summary"]["cancelled"], true, "{run}");
assert_eq!(run["summary"]["ran"], 1, "{run}");
assert_eq!(run["summary"]["remaining"], 1, "{run}");
&run["summary"]["records"][0]
} else {
&run["record"]
};
assert_eq!(record["ok"], false, "{run}");
assert_eq!(record["timed_out"], false, "{run}");
assert_eq!(record["alive"], true, "{run}");
assert!(
!suffix.exists(),
"cancelled batch executed its dependent suffix"
);
let pull = serde_json::json!({"path":relative,"kernel":"worker","name":"n"});
assert_eq!(post(server.addr, "state/pull", &pull)["value"], 99);
let exported = post(server.addr, "snapshot/export", &body);
let snapshot = &exported["snapshot"];
assert_eq!(snapshot["version"], 5);
let controls: Vec<_> = snapshot["actions"]
.as_array()
.unwrap()
.iter()
.filter(|a| a.get("controls").is_some())
.collect();
assert_eq!(controls.len(), 1);
assert_eq!(
controls[0]["kind"],
match mode {
"cell/run" => "cell",
"cell/step" => "step",
_ => "run_all",
}
);
assert_eq!(controls[0]["controls"].as_array().unwrap().len(), 1);
let replay = post(
server.addr,
"snapshot/replay_check",
&serde_json::json!({"path":relative,"snapshot":snapshot}),
);
assert_eq!(replay["replay_check"]["ok"], true,
"{language}: expected snapshot={snapshot}; actual replay={replay}");
assert!(!suffix.exists(), "replay executed a cancelled suffix");
let compact = post(
server.addr,
"snapshot/export",
&serde_json::json!({"path":relative,"compact":true}),
);
assert_eq!(
compact["snapshot"]["format"], "marqdo.history.dag",
"{compact}"
);
let replay_compact = post(
server.addr,
"snapshot/replay_check",
&serde_json::json!({"path":relative,"snapshot":compact["snapshot"]}),
);
assert_eq!(
replay_compact["replay_check"]["ok"], true,
"expected snapshot={snapshot}; actual compact replay={replay_compact}"
);
let mut corrupt = compact["snapshot"].clone();
corrupt["sha256"] = "bad".into();
let rejected = post(
server.addr,
"snapshot/replay_check",
&serde_json::json!({"path":relative,"snapshot":corrupt}),
);
assert_eq!(rejected["ok"], false, "{rejected}");
// A malformed cancellation suffix must fail before any replay executes.
for mutation in ["time", "receipt", "duplicate", "version"] {
let mut bad = snapshot.clone();
let index = bad["actions"]
.as_array()
.unwrap()
.iter()
.position(|a| a.get("controls").is_some())
.unwrap();
match mutation {
"time" => bad["actions"][index]["controls"][0]["after_us"] = (-1).into(),
"receipt" => {
bad["actions"][index]["controls"][0]["results"][0]["accepted"] =
false.into()
}
"duplicate" => {
let row = bad["actions"][index]["controls"][0]["results"][0].clone();
bad["actions"][index]["controls"][0]["results"]
.as_array_mut()
.unwrap()
.push(row);
}
_ => bad["version"] = 4.into(),
}
let rejected = post(
server.addr,
"snapshot/replay_check",
&serde_json::json!({"path":relative,"snapshot":bad}),
);
assert_eq!(rejected["ok"], false, "{mutation}: {rejected}");
}
assert_eq!(
post(server.addr, "state/pull", &pull)["value"],
99,
"checker disturbed warm state"
);
assert_eq!(post(server.addr, "close", &body)["ok"], true);
println!("E11 {language} {mode} trial {trial}: real HTTP interrupt {:?}, retained n=99, skipped suffix, control-input replay and four malformed histories verified", latency);
println!(
"CONTROL_METRIC:{}",
serde_json::json!({
"language": language, "mode": mode, "trial": trial,
"ack_ns": latency.as_nanos(), "completion_ns": completion_latency.as_nanos(),
"expected_store": 99, "plain_replay": true, "compact_replay": true,
"malformed_histories": 5, "correct": true
})
);
std::fs::remove_file(path).unwrap();
std::fs::remove_file(marker).unwrap();
}
}
}
}
+1
View File
@@ -542,6 +542,7 @@ fn value_kind_label(v: &crate::value::Value) -> String {
Value::Num(_) => "num".into(),
Value::Text(_) => "text".into(),
Value::Secret(_) => "secret".into(),
Value::Handle(_) => "handle".into(),
Value::Formula(crate::formula::Expr::Matrix { rows, .. }) => {
format!("formula · {}×{}", rows.len(), rows.first().map(|r| r.len()).unwrap_or(0))
}
File diff suppressed because it is too large. Load diff
+34
View File
@@ -0,0 +1,34 @@
# Application composition corpus
Four authored tasks exercise the actual document loader, tree/bytecode evaluators
and interactive history API. Each has separate Python and Node implementations
of the producer and consumer. `CASES.json` is the eight-case regression subset;
the reviewer driver generates 80 size/seed/direction instances before execution.
They are four application families, not 80 independent real-user applications.
- Invoice ledger: discard void rows, first-posted invoice deduplication, integer
cents/discount accounting, signed refunds with floor division, regional totals.
- Routes: minimum nonnegative parallel edge, zero-cost edges, disconnected nodes
and weighted single-source distances. Python uses heap-based Dijkstra, Node a
linear frontier; independent reference uses SQL and all-pairs Floyd-Warshall.
- Concordance: NFC plus **ASCII** lowercase, explicit space/tab/LF/CR/NBSP
separators, occurrence/document frequencies, UTF-8 byte order. This is not
Unicode full case folding. Wide IDs remain strings; special object-key text,
combining marks, BMP and astral characters are included.
- Wine: UCI red-wine CSV, exact decimal density/pH scaling to integers and
quality-group count/sum/min/max. Full input contains 1599 rows; no predictor
accuracy or general machine-learning performance is claimed.
Independent SQL/Fraction/Floyd references live in the reviewer experiment
`application_cases.py`; no foreign program output supplies expected answers.
The raw/public-data digest, citation, license and retrieval source are preserved
in `data/PROVENANCE.json`. Wine Quality is by Cortez, Cerdeira, Almeida, Matos and
Reis (2009), UCI, DOI 10.24432/C56S3T, CC BY 4.0. CSV text fixture generation
normalizes physical line endings and uses the requested first N data rows.
The composition copies input into a producer, gets a normalized value, copies it
into the consumer, records an expected ordinary exception after saving `answer`,
recovers it, replaces the consumer's bridge with null, verifies the producer's
copy and answer remain, and closes both sessions. Interactive recordings repeat
these actions and strictly replay both raw and losslessly packed history.
Native and independent checks distinguish explicit null from display stdout.
@@ -0,0 +1,10 @@
var index = new Map();
for (var row of bridge) {
for (var word of row.tokens) {
var value = index.get(word) || {count:0,docs:new Set()};
value.count++; value.docs.add(row.id); index.set(word,value);
}
}
var answer = [...index].sort((a,b)=>Buffer.compare(Buffer.from(a[0]),Buffer.from(b[0])))
.map(([word,v])=>({word,occurrences:v.count,documents:v.docs.size}));
answer;
@@ -0,0 +1,10 @@
from collections import Counter, defaultdict
occurrences = Counter()
documents = defaultdict(set)
for row in bridge:
for word in row['tokens']:
occurrences[word] += 1
documents[word].add(row['id'])
answer = [{'word':word,'occurrences':occurrences[word],'documents':len(documents[word])}
for word in sorted(occurrences,key=lambda s:s.encode('utf-8'))]
answer
@@ -0,0 +1,3 @@
var bridge = data.map(row=>({id:row.id,tokens:row.text.normalize('NFC').replace(/[A-Z]/g,c=>c.toLowerCase()).split(/[ \n\t\r\u00a0]+/).filter(t=>t.length)}));
bridge.sort((a,b)=>Buffer.compare(Buffer.from(a.id),Buffer.from(b.id)));
bridge;
@@ -0,0 +1,6 @@
import re, unicodedata
ascii_lower = str.maketrans('ABCDEFGHIJKLMNOPQRSTUVWXYZ','abcdefghijklmnopqrstuvwxyz')
bridge = [{'id': row['id'], 'tokens': [t for t in re.split('[ \n\t\r\u00a0]+',unicodedata.normalize('NFC',row['text']).translate(ascii_lower)) if t]}
for row in data]
bridge.sort(key=lambda row:row['id'].encode('utf-8'))
bridge
+21
View File
@@ -0,0 +1,21 @@
{
"source": "https://archive.ics.uci.edu/static/public/186/wine+quality.zip",
"landing_page": "https://archive.ics.uci.edu/dataset/186/wine%2Bquality",
"doi": "10.24432/C56S3T",
"authors": [
"Paulo Cortez",
"A. Cerdeira",
"F. Almeida",
"T. Matos",
"J. Reis"
],
"year": 2009,
"license": "CC BY 4.0",
"license_url": "https://creativecommons.org/licenses/by/4.0/",
"retrieved": "2026-11-03",
"files": {
"winequality.names": "15d215e73b39105952380fd487e4ce1bf5a6b83a425abb285c2ec55193d3ca92",
"winequality-red.csv": "4a402cf041b025d4566d954c3b9ba8635a3a8a01e039005d97d6a710278cf05e",
"wine-quality.zip": "3ed56667f4b828242bd732d7d1dd7f2861e54432239d7fa63877014cbb0304d4"
}
}
Binary file not shown.
File diff suppressed because it is too large. Load diff
+72
View File
@@ -0,0 +1,72 @@
Citation Request:
This dataset is public available for research. The details are described in [Cortez et al., 2009].
Please include this citation if you plan to use this database:
P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis.
Modeling wine preferences by data mining from physicochemical properties.
In Decision Support Systems, Elsevier, 47(4):547-553. ISSN: 0167-9236.
Available at: [@Elsevier] http://dx.doi.org/10.1016/j.dss.2009.05.016
[Pre-press (pdf)] http://www3.dsi.uminho.pt/pcortez/winequality09.pdf
[bib] http://www3.dsi.uminho.pt/pcortez/dss09.bib
1. Title: Wine Quality
2. Sources
Created by: Paulo Cortez (Univ. Minho), Antonio Cerdeira, Fernando Almeida, Telmo Matos and Jose Reis (CVRVV) @ 2009
3. Past Usage:
P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis.
Modeling wine preferences by data mining from physicochemical properties.
In Decision Support Systems, Elsevier, 47(4):547-553. ISSN: 0167-9236.
In the above reference, two datasets were created, using red and white wine samples.
The inputs include objective tests (e.g. PH values) and the output is based on sensory data
(median of at least 3 evaluations made by wine experts). Each expert graded the wine quality
between 0 (very bad) and 10 (very excellent). Several data mining methods were applied to model
these datasets under a regression approach. The support vector machine model achieved the
best results. Several metrics were computed: MAD, confusion matrix for a fixed error tolerance (T),
etc. Also, we plot the relative importances of the input variables (as measured by a sensitivity
analysis procedure).
4. Relevant Information:
The two datasets are related to red and white variants of the Portuguese "Vinho Verde" wine.
For more details, consult: http://www.vinhoverde.pt/en/ or the reference [Cortez et al., 2009].
Due to privacy and logistic issues, only physicochemical (inputs) and sensory (the output) variables
are available (e.g. there is no data about grape types, wine brand, wine selling price, etc.).
These datasets can be viewed as classification or regression tasks.
The classes are ordered and not balanced (e.g. there are munch more normal wines than
excellent or poor ones). Outlier detection algorithms could be used to detect the few excellent
or poor wines. Also, we are not sure if all input variables are relevant. So
it could be interesting to test feature selection methods.
5. Number of Instances: red wine - 1599; white wine - 4898.
6. Number of Attributes: 11 + output attribute
Note: several of the attributes may be correlated, thus it makes sense to apply some sort of
feature selection.
7. Attribute information:
For more information, read [Cortez et al., 2009].
Input variables (based on physicochemical tests):
1 - fixed acidity
2 - volatile acidity
3 - citric acid
4 - residual sugar
5 - chlorides
6 - free sulfur dioxide
7 - total sulfur dioxide
8 - density
9 - pH
10 - sulphates
11 - alcohol
Output variable (based on sensory data):
12 - quality (score between 0 and 10)
8. Missing Attribute Values: None
+10
View File
@@ -0,0 +1,10 @@
var totals = new Map();
for (var row of bridge) {
var value = totals.get(row.region) || [0, 0];
value[0] += 1;
value[1] += Math.floor(row.units * row.cents * (10000-row.discount) / 10000);
totals.set(row.region, value);
}
var answer = [...totals].sort((a,b) => Buffer.compare(Buffer.from(a[0]),Buffer.from(b[0])))
.map(([region, value]) => ({region, invoices: value[0], net_cents: value[1]}));
answer;
+8
View File
@@ -0,0 +1,8 @@
from collections import defaultdict
totals = defaultdict(lambda: [0, 0])
for row in bridge:
totals[row['region']][0] += 1
totals[row['region']][1] += row['units'] * row['cents'] * (10000-row['discount']) // 10000
answer = [{'region': k, 'invoices': v[0], 'net_cents': v[1]}
for k, v in sorted(totals.items(), key=lambda kv: kv[0].encode('utf-8'))]
answer
+10
View File
@@ -0,0 +1,10 @@
var seen = new Set();
var bridge = [];
for (var row of data) {
if (row.status === 'posted' && !seen.has(row.id)) {
seen.add(row.id);
bridge.push({id: row.id, region: row.region, units: row.units, cents: row.cents, discount: row.discount});
}
}
bridge.sort((a, b) => Buffer.compare(Buffer.from(a.id), Buffer.from(b.id)));
bridge;
+8
View File
@@ -0,0 +1,8 @@
seen = set()
bridge = []
for row in data:
if row['status'] == 'posted' and row['id'] not in seen:
seen.add(row['id'])
bridge.append({k: row[k] for k in ('id', 'region', 'units', 'cents', 'discount')})
bridge.sort(key=lambda r: r['id'].encode('utf-8'))
bridge
Loaded 100 of 159 files, more files were not shown because too many files have changed in this diff. Show more