Compare commits

..
2 Commits
Author SHA1 Message Date
cflmyandCursor 142ab523d6 release: v2.3.0 Foreign Strong session/handle/replay/arity
Opaque session handles, affine close, replay_check vs oracle journal,
table arity refuse, and Ω∈V_safe gate for the Strong foreign narrative.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-01 14:46:08 +08:00
cflmyandCursor 82c8d78fd3 release: v2.2.2 V_safe JSON integer refuse on foreign bridge
Align host json encode/decode with Lean ForeignJSON: refuse OOD integers
instead of silent Node distortion; keep decimal-string BigInt author path.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-01 09:43:51 +08:00
40 changed files with 1261 additions and 59 deletions

No files matched your search

+1 -1
View File
@@ -221,7 +221,7 @@ Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7
- `invoke` returns a record (`ok`, `code`, `stdout`, `stderr`, `ms`, `timed_out`, `truncated`, `result`); pass `input` for JSON stdin and `output="json"` for a parsed result.
- `call` is JSON-in / JSON-out; `call_many` fans out one program over a list with ordered results.
- `session_open` starts a persistent Python/Node kernel; `session_eval` evaluates source; `session_set` / `session_get` bridge values explicitly as JSON. Close it with `session_close`.
- `session_open` returns an opaque `handle` (not text; not JSON-serializable). `session_eval` / `set` / `get` borrow the handle; `session_close` consumes it (double-close / use-after-close fault). Prefer `replay_check` against the oracle journal when comparing recordings (`replay_snapshot` only re-runs). JSON integers outside \(\pm(2^{53}-1)\) and ragged table columns are refused (`doc/design/v-safe-json.md`, `foreign-strong-correspondence.md`).
- A view document opts into kernel state with `会话 = > foreign.session kernels=... cells=...`; the table names are free. `view` consumes only the `会话` handle and never scans source lines or reserved table names. Without `会话`, cells are stateless one-shot runs; no kernel is created. Clearing outputs calls `/api/session/clear` and keeps persistent kernel state. There is no separate notebook CLI mode or command, and no new Markdown marker exists.
- The same source remains the authority; state crossing kernels is explicit, never implicitly mirrored.
+3
View File
@@ -126,6 +126,9 @@ jobs:
## Marqdo ${{ github.ref_name }}
### 亮点 / Highlights
- **v2.3.0**:Foreign Strong——不透明 `handle`、仿射 close、`replay_check`/oracle journal、表 arity refuse、Ω∈V_safe
- **v2.3.0**: Foreign Strong — opaque handle, affine close, replay_check/oracle journal, table arity refuse, Ω∈V_safe
- **v2.3.0 ext**:扩展包同步 2.3.0 / Ext pack 2.3.0
- **v2.2.1**:view / 静态导出——修复 occlusion 与中等宽度下大纲侧栏掉到正文下方
- **v2.2.1**: view / static export — fix outline rail nested under document (occlusion + mid-width layout)
- **v2.2.1 ext**:扩展包同步 2.2.1 / Ext pack 2.2.1
+49
View File
@@ -8,6 +8,55 @@
### Changed
## v2.3.0 — 2026-11-01
### Highlights
**Foreign Strong(session / value / replay)**:会话句柄改为不透明 `handle`(不可 JSON 过桥、禁止 text 伪造);`close` 仿射消费(双 close / use-after-close 故障);`replay_check` 对照 oracle journal(旧 `replay_snapshot` 仅重跑保留);GFM 表行 arity 与列长映射拒绝静默补空;Ω 返回域外整数故障。对齐 PLDI Strong 叙事与 Lean SessionReg / \(V_{\mathsf{safe}}\)。CLI · wasm · ext **2.3.0**。
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——破坏面在宿主 foreign/session/JSON 与表解析纪律,不增 Markdown 标记。
### Added
- **`Value::Handle`**:`session_open` 返回句柄;session 操作拒绝 text 伪造。
- **`DocumentSession::replay_check`** + API `/api/session/snapshot/replay_check`;snapshot 录音字段 + **`oracle_journal`**(与 execution journal 分名)。
- **API** `/api/session/oracle_journal`。
- **文档**:`doc/design/foreign-strong-correspondence.md`;`v-safe-json.md` 补 Handle / 列 arity / Ω 门。
- **回归**:`foreign-session-{use-before-open,use-after-close,double-close,cold-replay-empty,isolation,handle-no-serialize,result-vsafe-refuse}`;`tests/errors/table-arity.mq.md`。
### Fixed
- 不等宽 GFM 表不再静默补空单元格;解析报 `table row arity mismatch`。
- Py/Node session bootstrap:Ω 结果 OOD 整数不再 `default=str` 偷渡,宿主升级为故障。
### Changed
- **`session_close`**:未知 / 已关闭句柄 → 故障(不再返回 `False`)。
- **`session_list`**:`id` 字段为 `handle`。
- Snapshot export 含 `ok`/`code`/`stdout`/`result`/`source_hash` 与 `oracle_journal`。
## v2.2.2 — 2026-11-01
### Highlights
**\(V_{\mathsf{safe}}\) JSON 整数契约落地**:外联 / `json.parse` / `json.stringify` / `session_set` 等共用映射对齐 Lean ForeignJSON——`|n| > 2^{53}-1` 的 JSON 整数显式拒绝(错误前缀 `json: integer outside V_safe`),禁止 Node 路径静默扭曲;宽整数仍用十进制字符串 + 对端 `BigInt`。CLI · wasm · ext **2.2.2**。
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——宿主 JSON 契约与测试,不改 Markdown 标记。
### Added
- **`doc/design/v-safe-json.md`**:宿主 ↔ Lean \(V_{\mathsf{safe}}\) 契约锁定(`MAX_SAFE`、encode/decode refuse、嵌套、float 为宿主扩展)。
- **回归**:`tests/lib/json-vsafe-*`、`foreign-vsafe-*`、`foreign-session-vsafe-refuse`(含算术后越界 refuse)。
### Fixed
- **`src/host/json.rs`**:对域外整数与 `u64 > i64::MAX` 显式失败;不再经 `f64`/`Text` 静默降级。
- **`doc/design/foreign-kernel.md`**:Int→number 仅当 `|n| ≤ 2^{53}-1`;宽整数走 string 桥。
### Changed
- 外联 JSON 桥与 `lib/json` 行为:OOD 整数从「可能静默失真」改为「诊断失败」。
## v2.2.1 — 2026-10-30
### Highlights
Generated
+2 -2
View File
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "marqdo"
version = "2.2.1"
version = "2.3.0"
dependencies = [
"anyhow",
"assert_cmd",
@@ -611,7 +611,7 @@ dependencies = [
[[package]]
name = "marqdo-wasm"
version = "2.2.1"
version = "2.3.0"
dependencies = [
"marqdo",
"serde_json",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo"
version = "2.2.1"
version = "2.3.0"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo — Markdown markers as an interpreted programming language"
+6 -4
View File
@@ -284,8 +284,10 @@ marqdo catalog public -o .marqdo
---
## 现状(v2.2.1)
## 现状(v2.3.0)
- **Foreign Strong(v2.3.0)**:会话 `handle`(不可过桥)· 仿射 close · `replay_check`/oracle journal · 表 arity refuse · Ω∈\(V_{\mathsf{safe}}\)。扩展包同步 **2.3.0**
- **\(V_{\mathsf{safe}}\) JSON 整数(v2.2.2)**:外联与 `lib/json` 拒绝 `|n| > 2^{53}-1` 的 JSON 整数(对齐 Lean ForeignJSON);宽整数用十进制字符串 + Node `BigInt`。
- **view 大纲侧栏(v2.2.1)**:修复 occlusion 文档与中等宽度下 Functions / Variables 掉到正文下方的问题;静态 `view output` 同源。
- **函数级调用(v2.2.0)**:`marqdo call <文件.mq.md> <函数> 名字=值`。同一份文档给人读,也按函数执行;Tree 与 Bytecode 都跑被点名的函数,不回退到 `# main`。默认只执行纯函数。`--json` 给 Agent 读取结果或诊断。说明与价值见 [22-函数调用](public/features/22-函数调用.mq.md)。示例:`marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3` → `5`。扩展包同步 **2.2.1**
- **Agent 可靠工作台(v2.1.5)**:ResourceView(JSON/CSV/PDF/image)· ToolSpec 授权先于调用 · CheckpointV2 / EffectLedger / ContextPackV2 · EKC `ir.json` 暖机与 `--refresh` · Web 短读/控制与 execute 槽隔离;扩展包同步 **2.1.5**
@@ -330,7 +332,7 @@ marqdo catalog public -o .marqdo
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
### 如何使用最新 Marqdo(v2.2.1)
### 如何使用最新 Marqdo(v2.3.0)
**安装解释器(任选其一)**
@@ -345,7 +347,7 @@ marqdo catalog public -o .marqdo
```bash
# 1) 源码安装(跟 tag 或 main)
git clone https://github.com/cflmy/marqdo.git && cd marqdo
git checkout v2.2.1 # 或留在 main
git checkout v2.3.0 # 或留在 main
cargo build --release
export PATH="$PWD/target/release:$PATH"
marqdo version
@@ -386,7 +388,7 @@ marqdo wasm build
# → dist/wasm/ … 见 examples/browser-hello/
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
# https://github.com/cflmy/marqdo/releases/tag/v2.2.1
# https://github.com/cflmy/marqdo/releases/tag/v2.3.0
```
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo-wasm"
version = "2.2.1"
version = "2.3.0"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
+14
View File
@@ -1,3 +1,17 @@
marqdo (2.3.0-1ppa1) noble; urgency=medium
* Marqdo 2.3.0: Strong foreign session — opaque Handle, affine close,
replay_check vs oracle journal, table arity refuse, Omega V_safe gate.
-- Marqdo Maintainers <cflmy@users.noreply.github.com> Sat, 01 Nov 2026 14:45:00 +0800
marqdo (2.2.2-1ppa1) noble; urgency=medium
* Marqdo 2.2.2: V_safe JSON integer contract on foreign/json bridge —
refuse integers outside +/- (2^53-1); wide ints via decimal string.
-- Marqdo Maintainers <noreply@marqdo.com> Sat, 01 Nov 2026 09:45:00 +0800
marqdo (2.2.1-1ppa1) noble; urgency=medium
* Marqdo 2.2.1: fix view/static-export outline rail under document
+9 -5
View File
@@ -214,19 +214,23 @@ template = ["php", "-r", "{inline}"]
### 4.1 值映射
Canonical integer contract: [`v-safe-json.md`](v-safe-json.md) (Lean \(V_{\mathsf{safe}}\): `|n| ≤ 2^{53}-1`).
| Marqdo | JSON |
|---|---|
| None | null |
| Bool | bool |
| Int | number |
| Num | number(NaN/Inf → null) |
| Text | string |
| List | array |
| Map | object |
| Int | number **仅当** `|n| ≤ 2^{53}-1`;越界 → 显式失败(禁止静默扭曲) |
| Num | number(NaN/Inf → null);float 为宿主扩展,∉ mechanized \(V_{\mathsf{safe}}\) |
| Text | string(宽整数用十进制字符串过桥,对端 `BigInt` / Python `int`) |
| List | array(子树递归;任一越界整数使整体失败) |
| Map | object(同上) |
| Formula | 矩阵 → 二维数组;其它 → 数字或字符串 |
| Code | 源码字符串 |
| Secret | JSON 中 `<secret>`;`env` 显式交付时取真实值 |
错误前缀:`json: integer outside V_safe (±(2^53-1)): <n>`。
### 4.2 输入
- `input=` → JSON;写 stdin;同时提供 `MARQDO_INPUT`(≤8 KiB)与 `MARQDO_INPUT_FILE`(任意大小)。
@@ -0,0 +1,23 @@
# Foreign Strong: Lean ↔ host correspondence
Pinned with PLDI Strong narrative (`FOREIGN-NARRATIVE-ASSESSMENT-2026-11-01.md`).
**Not** verified compilation — conformance evidence only.
| Lean / paper rule | Host surface | Evidence |
|---|---|---|
| `SessionReg` + `sessionOpen` / `Close` / `sessionEval` | `HostContext.foreign_sessions` + `Value::Handle` | `DocCalc.lean`; `foreign_session.rs` |
| `Ω(s, τ, c)` | Persistent kernel + process order | Oracle journal records `(cell, source_hash, stdout, result)` |
| `SafeVal` / Encode / Decode | `src/host/json.rs` + session bootstrap sanitize | `ForeignJSON.lean`; `v-safe-json.md`; gold `foreign-*-vsafe-*` |
| Handle ∉ \(V_{\mathsf{safe}}\) | `value_to_json(Handle) → Err` | `foreign-session-handle-no-serialize.mq.md` |
| Closed / Closed-Close | use-after-close / double-close `Err` | gold misuse suite |
| Oracle-Domain | Ω OOD integer → host `Err` | `foreign-session-result-vsafe-refuse.mq.md` |
| Table arity stuck | Parse refuse unequal GFM row width; JSON refuse unequal column lists | `Surface.lean` `table_arity_*`; `tests/errors/table-arity.mq.md` |
| Snapshot hash match/mismatch | `view/output.rs` `snapshot_page` | unit `snapshot_page_rejects_source_hash_mismatch` |
| `replay_check` vs journal \(J\) | `DocumentSession::replay_check` | unit + `defense/fate_matrix/run_replay_pack.py` |
| Dual-backend differential | `tests/gold.rs` tree + bytecode | 308 tests × 2; foreign subset includes session/V_safe |
## Intentional non-claims
- No static affine/linear session types (host death unsound).
- Floats are host extension outside Lean `SafeVal`.
- `replay_check` is host/document-session API, not a second interpreter backend.
+78
View File
@@ -0,0 +1,78 @@
# \(V_{\mathsf{safe}}\) JSON integer contract (host ↔ Lean)
Canonical Lean model: `MarqdoThesis/PLDI/proofs/MarqdoProof/MarqdoProof/ForeignJSON.lean`.
Host mapper: `src/host/json.rs` (`json_to_value` / `value_to_json`).
This document freezes **runtime conformance** of the Marqdo JSON bridge to the
mechanized \(V_{\mathsf{safe}}\) integer domain. It is **not** a verified
compilation of the Rust interpreter.
## Domain
```
MAX_SAFE = 9007199254740991 # 2^53 - 1
```
\(V_{\mathsf{safe}}\) values (mechanized):
| JSON | Marqdo |
|---|---|
| `null` | `None` |
| `bool` | `Bool` |
| integer with `\|n\| ≤ MAX_SAFE` | `Int` |
| `string` | `Text` |
| `array` | `List` |
| `object` | `Map` |
**Host extension (not in Lean `SafeVal`):** JSON non-integer numbers → `Value::Num(f64)` (NaN/Inf encode as `null`). Floats are documented host behavior; they are **not** claimed as mechanized \(V_{\mathsf{safe}}\) fidelity.
## Encode (`value_to_json`)
- `Value::Int(n)` with `|n| ≤ MAX_SAFE` → JSON number.
- `Value::Int(n)` with `|n| > MAX_SAFE` → **refuse** (error).
- `Value::Handle(_)` → **refuse** (`json: session handle cannot be serialized`).
- Column-table `Map` whose fields are all `List` with **unequal lengths** →
**refuse** (`json: table column arity mismatch`).
- Nested `List` / `Map`: any child refuse fails the whole tree.
- GFM table **parse** also refuses unequal row width
(`table row arity mismatch`) before a value is built.
- Wide integers that must cross a language boundary: author converts to a
**decimal string** (∈ \(V_{\mathsf{safe}}\)); the foreign program may use
`BigInt` / Python `int` on that string.
## Foreign oracle (`session_eval` / `session_get`)
Python/Node session bootstraps sanitize results with the same integer gate
before JSON emission. An Ω response containing an OOD integer is a **fault**
(host `Err`), not a soft `ok:false` cell record. Stdout text that authors
`json.parse` themselves is still gated by host `json_to_value`.
## Decode (`json_to_value`)
- JSON integer with `|n| ≤ MAX_SAFE` → `Int`.
- JSON integer with `|n| > MAX_SAFE` → **refuse**.
- JSON unsigned integer `> i64::MAX` → **refuse** (do **not** coerce via `f64` or `Text`).
- Nested array/object: any child refuse fails the whole tree.
- Decimal-string encodings of wide integers → `Text` (in domain).
## Error prefix
All refuse messages use a fixed prefix for probes and diagnostics:
```
json: integer outside V_safe (±(2^53-1)): <n>
```
## In-domain roundtrip
`±MAX_SAFE`, bool, null, string, array, and object roundtrips through
`encode → decode` must succeed. Integers `MAX_SAFE+1`, `-MAX_SAFE-1`, `2^53`,
`2^60`, `2^60+1` (and nested forms) must refuse.
## Non-claims
- Not verified Rust ↔ Lean compilation.
- Not arbitrary host-object fidelity (DataFrame, functions, heap sharing).
- Not automatic BigInt promotion inside Node/Python bootstraps.
- Document-level `Value::Int` may still hold i64 values that are OOD for JSON;
the gate fires when those values enter JSON encode/decode.
+1 -1
View File
@@ -1 +1 @@
2.2.1
2.3.0
+6 -3
View File
@@ -177,8 +177,9 @@ view has no persistent kernel state.
+ `cwd`=None
+ `timeout`=None
Start a persistent Python/Node kernel; returns a session id. `timeout` is the
default per-eval timeout.
Start a persistent Python/Node kernel; returns an opaque `handle` value.
Text fabrications are refused by session ops; handles cannot cross the JSON
bridge. `timeout` is the default per-eval timeout.
*> host_foreign_session_open lang=`lang` cmd=`cmd` env=`env` cwd=`cwd` timeout=`timeout`*
@@ -223,7 +224,9 @@ List kernel variable names.
## session_close
+ `id`
Kill and remove a session; returns whether it existed.
Consume (affine close) a live session handle: kill and remove it. Returns
`True` on success. Closing an unknown or already-closed id is a fault
(`foreign session … not found`), matching use-after-close on eval/set/get.
*> host_foreign_session_close id=`id`*
+1
View File
@@ -42,6 +42,7 @@ pub fn builtin_type(v: &Value) -> Value {
Value::Num(_) => "num",
Value::Text(_) => "text",
Value::Secret(_) => "secret",
Value::Handle(_) => "handle",
Value::List(_) => "list",
Value::Map(_) => "map",
Value::Formula(_) => "formula",
+1
View File
@@ -109,6 +109,7 @@ pub fn type_of(v: &Value) -> &'static str {
Value::Num(_) => "num",
Value::Text(_) => "text",
Value::Secret(_) => "secret",
Value::Handle(_) => "handle",
Value::List(_) => "list",
Value::Map(_) => "map",
Value::Formula(_) => "formula",
+372 -26
View File
@@ -1,4 +1,9 @@
//! Document session: cells, kernels, execution journal, state store.
//! Document session: cells, kernels, journals, state store.
//!
//! Two journals (do not mix names):
//! - **execution journal** (`JournalRecord`): UI timing / status / errors.
//! - **oracle journal** (`OracleRecord`): Ω step recordings for `replay_check`
//! (source hash, stdout, result, ok, code).
//!
//! Stateful cells live here, owned by the foreign runtime. The view reads a
//! `会话 = > foreign.session ...` handle; no table names are reserved, and a
@@ -56,6 +61,9 @@ pub struct Cell {
pub last_ms: u64,
pub last_ok: bool,
pub last_error: String,
/// Captured stdout from the last evaluation (recording surface for replay_check).
pub last_stdout: String,
pub last_code: i64,
/// Value returned by the last successful evaluation, used by explicit persist.
pub last_result: Value,
}
@@ -81,6 +89,20 @@ pub struct JournalRecord {
pub error: String,
}
/// One recorded Ω step: what a later `replay_check` compares against.
#[derive(Debug, Clone)]
pub struct OracleRecord {
pub seq: u64,
pub cell: String,
pub kernel: String,
pub lang: String,
pub source_hash: String,
pub ok: bool,
pub code: i64,
pub stdout: String,
pub result: Value,
}
pub struct DocumentSession {
pub path: PathBuf,
root: PathBuf,
@@ -89,6 +111,7 @@ pub struct DocumentSession {
pub kernels: Vec<KernelDecl>,
pub order: Vec<String>,
pub journal: Vec<JournalRecord>,
pub oracle_journal: Vec<OracleRecord>,
pub value_store: HashMap<String, Value>,
pub cursor: usize,
pub generation: u64,
@@ -289,6 +312,8 @@ fn collect_cells(stmts: &[crate::ast::Stmt], out: &mut Vec<Cell>, seen: &mut Has
last_ms: 0,
last_ok: false,
last_error: String::new(),
last_stdout: String::new(),
last_code: 0,
last_result: Value::None,
});
}
@@ -464,6 +489,7 @@ impl DocumentSession {
kernels,
order,
journal: Vec::new(),
oracle_journal: Vec::new(),
value_store: HashMap::new(),
cursor: 0,
generation: 1,
@@ -484,7 +510,7 @@ impl DocumentSession {
let id = session.ensure_kernel_session(&kernel)?;
foreign_session::set(
&mut session.host,
&Value::Text(id),
&Value::Handle(id),
&Value::Text(name),
&value,
)?;
@@ -545,7 +571,7 @@ impl DocumentSession {
.collect();
for k in orphans {
if let Some(id) = fresh.kernel_sessions.remove(&k) {
let _ = foreign_session::close(&mut fresh.host, &Value::Text(id));
let _ = foreign_session::close(&mut fresh.host, &Value::Handle(id));
}
}
*self = fresh;
@@ -590,8 +616,8 @@ impl DocumentSession {
None,
)?;
let id = match id {
Value::Text(s) => s,
_ => return Err("session_open returned non-text id".into()),
Value::Handle(s) => s,
_ => return Err("session_open returned non-handle id".into()),
};
self.kernel_sessions
.insert(kernel_name.to_string(), id.clone());
@@ -618,23 +644,34 @@ impl DocumentSession {
Some(Value::Int(n)) => n,
_ => 0,
};
let lang = self
.cell_index(cell)
.map(|i| self.cells[i].lang.clone())
let stdout = field("stdout")
.and_then(|v| v.reveal_text().map(|s| s.to_string()))
.unwrap_or_default();
let (lang, source_hash) = self
.cell_index(cell)
.map(|i| {
(
self.cells[i].lang.clone(),
self.cells[i].source_hash.clone(),
)
})
.unwrap_or_default();
let last_result = field("result")
.filter(|v| !matches!(v, Value::None))
.unwrap_or_else(|| {
field("stdout")
.and_then(|v| v.reveal_text().map(|s| Value::Text(s.to_string())))
.unwrap_or(Value::None)
});
if let Some(i) = self.cell_index(cell) {
let c = &mut self.cells[i];
c.runs += 1;
c.last_ms = ms;
c.last_ok = ok;
c.last_error = error.clone();
c.last_result = field("result")
.filter(|v| !matches!(v, Value::None))
.unwrap_or_else(|| {
field("stdout")
.and_then(|v| v.reveal_text().map(|s| Value::Text(s.to_string())))
.unwrap_or(Value::None)
});
c.last_stdout = stdout.clone();
c.last_code = code;
c.last_result = last_result.clone();
c.status = if ok { CellStatus::Clean } else { CellStatus::Error };
}
self.seq += 1;
@@ -642,13 +679,24 @@ impl DocumentSession {
seq: self.seq,
cell: cell.to_string(),
kernel: kernel.to_string(),
lang,
lang: lang.clone(),
ok,
code,
ms,
timestamp_ms: now_ms(),
error,
});
self.oracle_journal.push(OracleRecord {
seq: self.seq,
cell: cell.to_string(),
kernel: kernel.to_string(),
lang,
source_hash,
ok,
code,
stdout,
result: last_result,
});
Ok(())
}
@@ -727,7 +775,7 @@ impl DocumentSession {
let source = Value::Text(self.cells[ci].source.clone());
let record = foreign_session::eval_with_stdin(
&mut self.host,
&Value::Text(id),
&Value::Handle(id),
None,
Some(&source),
None,
@@ -847,7 +895,7 @@ impl DocumentSession {
let mut out = Vec::new();
for name in ids {
if let Some(id) = self.kernel_sessions.get(&name) {
match foreign_session::interrupt(&mut self.host, &Value::Text(id.clone())) {
match foreign_session::interrupt(&mut self.host, &Value::Handle(id.clone())) {
Ok(_) => out.push(Value::Text(name)),
Err(e) => out.push(Value::Text(format!("{name}: {e}"))),
}
@@ -858,7 +906,7 @@ impl DocumentSession {
pub fn restart_kernel(&mut self, name: &str) -> Result<Value, String> {
if let Some(id) = self.kernel_sessions.remove(name) {
let _ = foreign_session::close(&mut self.host, &Value::Text(id));
let _ = foreign_session::close(&mut self.host, &Value::Handle(id));
}
for c in &mut self.cells {
if c.kernel == name {
@@ -938,6 +986,8 @@ impl DocumentSession {
last_ms: 0,
last_ok: false,
last_error: String::new(),
last_stdout: String::new(),
last_code: 0,
last_result: Value::None,
});
self.order.push(cell.to_string());
@@ -974,7 +1024,7 @@ impl DocumentSession {
let mut vars = Vec::new();
if let Some(id) = live {
if self.host.foreign_sessions.contains_key(&id) {
if let Ok(names) = foreign_session::names(&mut self.host, &Value::Text(id)) {
if let Ok(names) = foreign_session::names(&mut self.host, &Value::Handle(id)) {
if let Value::List(xs) = names {
vars = xs;
}
@@ -997,7 +1047,7 @@ impl DocumentSession {
let id = self.ensure_kernel_session(kernel)?;
foreign_session::set(
&mut self.host,
&Value::Text(id),
&Value::Handle(id),
&Value::Text(name.to_string()),
&value,
)?;
@@ -1009,7 +1059,7 @@ impl DocumentSession {
let id = self.ensure_kernel_session(kernel)?;
let value = foreign_session::get(
&mut self.host,
&Value::Text(id),
&Value::Handle(id),
&Value::Text(name.to_string()),
)?;
self.value_store
@@ -1095,10 +1145,14 @@ impl DocumentSession {
self.cells[i].last_ms = 0;
self.cells[i].last_ok = false;
self.cells[i].last_error.clear();
self.cells[i].last_stdout.clear();
self.cells[i].last_code = 0;
self.cells[i].last_result = Value::None;
if self.cells[i].listed {
self.cells[i].status = CellStatus::Never;
}
self.journal.retain(|j| j.cell != cell);
self.oracle_journal.retain(|j| j.cell != cell);
Ok(self.summary())
}
@@ -1110,13 +1164,13 @@ impl DocumentSession {
let mut values = Vec::new();
if let Some(id) = self.kernel_sessions.get(&decl.name).cloned() {
if let Ok(Value::List(names)) =
foreign_session::names(&mut self.host, &Value::Text(id.clone()))
foreign_session::names(&mut self.host, &Value::Handle(id.clone()))
{
for name in names {
if let Value::Text(n) = &name {
if let Ok(v) = foreign_session::get(
&mut self.host,
&Value::Text(id.clone()),
&Value::Handle(id.clone()),
&Value::Text(n.clone()),
) {
values.push(Value::Map(vec![
@@ -1144,10 +1198,16 @@ impl DocumentSession {
("lang".into(), Value::Text(c.lang.clone())),
("kernel".into(), Value::Text(c.kernel.clone())),
("source".into(), Value::Text(c.source.clone())),
("source_hash".into(), Value::Text(c.source_hash.clone())),
("stdin".into(), Value::Text(c.stdin.clone())),
("status".into(), Value::Text(c.status.as_str().into())),
("runs".into(), Value::Int(c.runs as i64)),
("last_ok".into(), Value::Bool(c.last_ok)),
// Recording fields: replay_check compares a later run against these.
("ok".into(), Value::Bool(c.last_ok)),
("code".into(), Value::Int(c.last_code)),
("stdout".into(), Value::Text(c.last_stdout.clone())),
("result".into(), c.last_result.clone()),
])
})
.collect();
@@ -1163,6 +1223,7 @@ impl DocumentSession {
self.order.iter().cloned().map(Value::Text).collect(),
)),
("journal".into(), journal),
("oracle_journal".into(), self.oracle_journal_json()),
]))
}
@@ -1208,7 +1269,7 @@ impl DocumentSession {
let Some(value) = getv("value") else { continue };
foreign_session::set(
&mut self.host,
&Value::Text(id.clone()),
&Value::Handle(id.clone()),
&Value::Text(vname),
&value,
)?;
@@ -1243,6 +1304,8 @@ impl DocumentSession {
}
/// Replay recorded cells in order, using the snapshot's per-cell stdin.
/// Re-runs only; does not compare against the recording. Prefer
/// [`Self::replay_check`] when equivalence against a recording matters.
pub fn replay_snapshot(&mut self, snapshot: &Value) -> Result<Value, String> {
let Value::Map(fields) = snapshot else {
return Err("snapshot must be a map".into());
@@ -1287,6 +1350,157 @@ impl DocumentSession {
]))
}
/// Replay cells and compare each step against the snapshot recording.
///
/// Expected fields per cell (when present): `ok`, `code`, `stdout`, `result`.
/// A difference is localized to the first mismatched field on that cell.
/// Cells without any recording fields are skipped for comparison but still run.
pub fn replay_check(&mut self, snapshot: &Value) -> Result<Value, String> {
let Value::Map(fields) = snapshot else {
return Err("snapshot must be a map".into());
};
let cells = fields
.iter()
.find(|(k, _)| k == "cells")
.and_then(|(_, v)| match v {
Value::List(xs) => Some(xs.clone()),
_ => None,
})
.ok_or_else(|| "snapshot: missing cells".to_string())?;
let mut checks = Vec::new();
let mut matched = 0i64;
let mut mismatched = 0i64;
let mut skipped = 0i64;
for item in cells {
let Value::Map(cfields) = item else { continue };
let getc = |name: &str| {
cfields
.iter()
.find(|(k, _)| k == name)
.map(|(_, v)| v.clone())
};
let Some(Value::Text(name)) = getc("name") else { continue };
if self.cell_index(&name).is_none() {
continue;
}
let has_recording = getc("ok").is_some()
|| getc("code").is_some()
|| getc("stdout").is_some()
|| getc("result").is_some();
let stdin = getc("stdin").and_then(|v| match v {
Value::Text(s) if !s.is_empty() => Some(s),
_ => None,
});
let record = self.run_cell_with_stdin(&name, None, stdin.as_deref())?;
let observed_ok = match &record {
Value::Map(m) => matches!(
m.iter().find(|(k, _)| k == "ok").map(|(_, v)| v),
Some(Value::Bool(true))
),
_ => false,
};
let observed_code = match &record {
Value::Map(m) => match m.iter().find(|(k, _)| k == "code").map(|(_, v)| v) {
Some(Value::Int(n)) => *n,
_ => 0,
},
_ => 0,
};
let observed_stdout = match &record {
Value::Map(m) => m
.iter()
.find(|(k, _)| k == "stdout")
.and_then(|(_, v)| v.reveal_text().map(|s| s.to_string()))
.unwrap_or_default(),
_ => String::new(),
};
// Match record_result: prefer explicit result, else stdout as text.
let observed_result = match &record {
Value::Map(m) => {
let explicit = m
.iter()
.find(|(k, _)| k == "result")
.map(|(_, v)| v.clone())
.filter(|v| !matches!(v, Value::None));
explicit.unwrap_or_else(|| {
m.iter()
.find(|(k, _)| k == "stdout")
.and_then(|(_, v)| v.reveal_text().map(|s| Value::Text(s.to_string())))
.unwrap_or(Value::None)
})
}
_ => Value::None,
};
if !has_recording {
skipped += 1;
checks.push(Value::Map(vec![
("cell".into(), Value::Text(name)),
("match".into(), Value::Bool(true)),
("compared".into(), Value::Bool(false)),
("diff".into(), Value::Text(String::new())),
]));
continue;
}
let mut diff = String::new();
if let Some(Value::Bool(exp_ok)) = getc("ok") {
if exp_ok != observed_ok {
diff = format!("ok: expected {exp_ok}, got {observed_ok}");
}
}
if diff.is_empty() {
if let Some(Value::Int(exp_code)) = getc("code") {
if exp_code != observed_code {
diff = format!("code: expected {exp_code}, got {observed_code}");
}
}
}
if diff.is_empty() {
if let Some(Value::Text(exp_stdout)) = getc("stdout") {
if exp_stdout != observed_stdout {
diff = format!(
"stdout: expected {:?}, got {:?}",
exp_stdout, observed_stdout
);
}
}
}
if diff.is_empty() {
if let Some(exp_result) = getc("result") {
if exp_result != observed_result {
diff = format!(
"result: expected {}, got {}",
exp_result.as_display(),
observed_result.as_display()
);
}
}
}
let is_match = diff.is_empty();
if is_match {
matched += 1;
} else {
mismatched += 1;
}
checks.push(Value::Map(vec![
("cell".into(), Value::Text(name)),
("match".into(), Value::Bool(is_match)),
("compared".into(), Value::Bool(true)),
("diff".into(), Value::Text(diff)),
("observed_ok".into(), Value::Bool(observed_ok)),
("observed_code".into(), Value::Int(observed_code)),
("observed_stdout".into(), Value::Text(observed_stdout)),
("observed_result".into(), observed_result),
]));
}
Ok(Value::Map(vec![
("ok".into(), Value::Bool(mismatched == 0)),
("matched".into(), Value::Int(matched)),
("mismatched".into(), Value::Int(mismatched)),
("skipped".into(), Value::Int(skipped)),
("checks".into(), Value::List(checks)),
]))
}
/// Explicitly persist the last cell result through lib/writeback.
pub fn persist_cell(&mut self, cell: &str, key: &str) -> Result<Value, String> {
let i = self
@@ -1315,11 +1529,15 @@ impl DocumentSession {
cell.last_ms = 0;
cell.last_ok = false;
cell.last_error.clear();
cell.last_stdout.clear();
cell.last_code = 0;
cell.last_result = Value::None;
if cell.listed {
cell.status = CellStatus::Never;
}
}
self.journal.clear();
self.oracle_journal.clear();
self.cursor = 0;
self.summary()
}
@@ -1345,10 +1563,31 @@ impl DocumentSession {
)
}
pub fn oracle_journal_json(&self) -> Value {
Value::List(
self.oracle_journal
.iter()
.map(|j| {
Value::Map(vec![
("seq".into(), Value::Int(j.seq as i64)),
("cell".into(), Value::Text(j.cell.clone())),
("kernel".into(), Value::Text(j.kernel.clone())),
("lang".into(), Value::Text(j.lang.clone())),
("source_hash".into(), Value::Text(j.source_hash.clone())),
("ok".into(), Value::Bool(j.ok)),
("code".into(), Value::Int(j.code)),
("stdout".into(), Value::Text(j.stdout.clone())),
("result".into(), j.result.clone()),
])
})
.collect(),
)
}
pub fn close(&mut self) {
let ids: Vec<String> = self.kernel_sessions.values().cloned().collect();
for id in ids {
let _ = foreign_session::close(&mut self.host, &Value::Text(id));
let _ = foreign_session::close(&mut self.host, &Value::Handle(id));
}
self.kernel_sessions.clear();
}
@@ -2066,4 +2305,111 @@ console.log("after error");
s.close();
let _ = std::fs::remove_file(&path);
}
#[test]
fn oracle_journal_records_stdout_separate_from_execution_journal() {
if !node_available() {
return;
}
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let path = root.join("tests/notebook/shared.mq.md");
let mut s = DocumentSession::open(&path, &root).unwrap();
s.run_all(true).unwrap();
assert!(!s.journal.is_empty());
assert_eq!(s.oracle_journal.len(), s.journal.len());
let read = s
.oracle_journal
.iter()
.find(|j| j.cell == "读取")
.expect("oracle entry for 读取");
assert!(read.ok);
assert!(
read.stdout.contains('4') || read.stdout.contains("42"),
"oracle must capture stdout, got {:?}",
read.stdout
);
assert!(!read.source_hash.is_empty());
// Execution journal must not pretend to be the oracle (no stdout field).
let exec = s.journal.iter().find(|j| j.cell == "读取").unwrap();
assert_eq!(exec.ok, read.ok);
s.close();
}
#[test]
fn replay_check_matches_exported_recording() {
if !node_available() {
return;
}
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let path = root.join("tests/notebook/shared.mq.md");
let mut a = DocumentSession::open(&path, &root).unwrap();
a.run_all(true).unwrap();
let snap = a.export_snapshot().unwrap();
a.close();
let mut b = DocumentSession::open(&path, &root).unwrap();
let check = b.replay_check(&snap).unwrap();
assert_eq!(field(&check, "ok"), Some(Value::Bool(true)));
assert_eq!(field(&check, "mismatched"), Some(Value::Int(0)));
let matched = field(&check, "matched").and_then(|v| match v {
Value::Int(n) => Some(n),
_ => None,
});
assert!(matched.unwrap_or(0) >= 1, "expected at least one compared cell");
b.close();
}
#[test]
fn replay_check_localizes_stdout_mismatch() {
if !node_available() {
return;
}
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let path = root.join("tests/notebook/shared.mq.md");
let mut a = DocumentSession::open(&path, &root).unwrap();
a.run_all(true).unwrap();
let mut snap = a.export_snapshot().unwrap();
a.close();
// Tamper with the recording for cell 读取.
if let Value::Map(ref mut fields) = snap {
if let Some((_, Value::List(cells))) =
fields.iter_mut().find(|(k, _)| k == "cells")
{
for cell in cells.iter_mut() {
let Value::Map(cfields) = cell else { continue };
let name = cfields
.iter()
.find(|(k, _)| k == "name")
.and_then(|(_, v)| v.reveal_text().map(|s| s.to_string()));
if name.as_deref() == Some("读取") {
if let Some((_, v)) = cfields.iter_mut().find(|(k, _)| k == "stdout") {
*v = Value::Text("wrong-recording\n".into());
}
}
}
}
}
let mut b = DocumentSession::open(&path, &root).unwrap();
let check = b.replay_check(&snap).unwrap();
assert_eq!(field(&check, "ok"), Some(Value::Bool(false)));
assert_eq!(field(&check, "mismatched"), Some(Value::Int(1)));
let Some(Value::List(checks)) = field(&check, "checks") else {
panic!("missing checks");
};
let bad = checks.iter().find(|c| {
field(c, "cell")
.and_then(|v| v.reveal_text().map(|s| s.to_string()))
.as_deref()
== Some("读取")
});
let Some(bad) = bad else { panic!("missing 读取 check") };
assert_eq!(field(bad, "match"), Some(Value::Bool(false)));
let diff = field(bad, "diff")
.and_then(|v| v.reveal_text().map(|s| s.to_string()))
.unwrap_or_default();
assert!(diff.contains("stdout"), "diff should localize to stdout: {diff}");
b.close();
}
}
+87 -13
View File
@@ -22,8 +22,35 @@ const MAX_PROCESS_STDERR: usize = 1_048_576;
const PY_BOOTSTRAP: &str = r#"import sys, json, io, ast, traceback, contextlib
_out = sys.stdout
_MAX_SAFE = 9007199254740991
def _sanitize(o):
if o is None or isinstance(o, bool):
return o
if isinstance(o, int):
if abs(o) > _MAX_SAFE:
raise ValueError("json: integer outside V_safe (±(2^53-1)): %s" % o)
return o
if isinstance(o, float):
return o
if isinstance(o, str):
return o
if isinstance(o, (list, tuple)):
return [_sanitize(x) for x in o]
if isinstance(o, dict):
return {str(k): _sanitize(v) for k, v in o.items()}
raise TypeError("not JSON-safe: %s" % type(o).__name__)
def _send(o):
_out.write(json.dumps(o, ensure_ascii=False, default=str) + "\n")
try:
line = json.dumps(_sanitize(o), ensure_ascii=False)
except Exception as e:
rid = o.get("id") if isinstance(o, dict) else None
line = json.dumps({
"id": rid, "ok": False,
"stdout": (o.get("stdout") if isinstance(o, dict) else "") or "",
"stderr": (o.get("stderr") if isinstance(o, dict) else "") or "",
"result": None, "error": str(e),
}, ensure_ascii=False)
_out.write(line + "\n")
_out.flush()
_g = {"__name__": "__main__", "marqdo_input": None}
for _line in sys.stdin:
@@ -81,10 +108,40 @@ const JS_BOOTSTRAP: &str = r#"const readline = require("readline");
const util = require("util");
const vm = require("vm");
const out = process.stdout.write.bind(process.stdout);
const MAX_SAFE = Number.MAX_SAFE_INTEGER;
globalThis.require = require;
const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });
function send(o) { out(JSON.stringify(o) + "\n"); }
function toJSON(v) { try { return JSON.parse(JSON.stringify(v)); } catch (e) { return String(v); } }
function sanitize(v) {
if (v === null || typeof v === "boolean" || typeof v === "string") return v;
if (typeof v === "number") {
if (Number.isInteger(v) && Math.abs(v) > MAX_SAFE)
throw new Error("json: integer outside V_safe (±(2^53-1)): " + v);
return v;
}
if (typeof v === "bigint") {
if (v > BigInt(MAX_SAFE) || v < -BigInt(MAX_SAFE))
throw new Error("json: integer outside V_safe (±(2^53-1)): " + v.toString());
return Number(v);
}
if (Array.isArray(v)) return v.map(sanitize);
if (v && typeof v === "object") {
const o = {};
for (const k of Object.keys(v)) o[k] = sanitize(v[k]);
return o;
}
throw new Error("not JSON-safe: " + (typeof v));
}
function send(o) {
try { out(JSON.stringify(sanitize(o)) + "\n"); }
catch (e) {
out(JSON.stringify({
id: o && o.id !== undefined ? o.id : null, ok: false,
stdout: (o && o.stdout) || "", stderr: (o && o.stderr) || "",
result: null, error: String(e && e.message ? e.message : e)
}) + "\n");
}
}
function toJSON(v) { return sanitize(v); }
async function capture(fn) {
const logs = [], errs = [];
const ow = process.stdout.write.bind(process.stdout);
@@ -182,6 +239,15 @@ fn opt_text(v: Option<&Value>) -> Result<Option<String>, String> {
}
}
/// Session ids are opaque handles — text fabrications are refused.
fn as_session_id<'a>(id: &'a Value) -> Result<&'a str, String> {
match id {
Value::Handle(s) => Ok(s.as_str()),
Value::Text(_) => Err("session id must be a handle (not text)".into()),
_ => Err("session id must be a handle".into()),
}
}
fn parse_timeout(v: Option<&Value>, default: f64) -> Result<Duration, String> {
let secs = match v {
None | Some(Value::None) => default,
@@ -418,7 +484,7 @@ pub fn open(
};
ctx.foreign_sessions.insert(id.clone(), session);
let _ = timeout;
Ok(Value::Text(id))
Ok(Value::Handle(id))
}
fn session_dead_stderr(ctx: &HostContext, id: &str) -> String {
@@ -453,7 +519,7 @@ pub fn eval_with_stdin(
timeout: Option<&Value>,
) -> Result<Value, String> {
ensure_config(ctx);
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
if !ctx.foreign_sessions.contains_key(&id) {
return Err(format!("foreign session `{id}` not found"));
}
@@ -544,6 +610,10 @@ pub fn eval_with_stdin(
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
// Ω ∉ V_safe is a language fault, not a soft cell failure.
if error.contains("outside V_safe") {
return Err(error);
}
let result = reply
.get("result")
.map(json_to_value)
@@ -650,7 +720,7 @@ fn kill_session(ctx: &mut HostContext, id: &str) {
/// Interrupt the current evaluation without destroying the kernel. Python
/// reports KeyboardInterrupt through the normal result frame.
pub fn interrupt(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
let session = ctx
.foreign_sessions
.get_mut(&id)
@@ -675,7 +745,7 @@ pub fn set(
name: &Value,
value: &Value,
) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
let name = as_text(name, "name")?.to_string();
if !valid_name(&name) {
return Err("session_set: name must be an identifier".into());
@@ -695,7 +765,7 @@ pub fn set(
}
pub fn get(ctx: &mut HostContext, id: &Value, name: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
let name = as_text(name, "name")?.to_string();
if !valid_name(&name) {
return Err("session_get: name must be an identifier".into());
@@ -716,7 +786,7 @@ pub fn get(ctx: &mut HostContext, id: &Value, name: &Value) -> Result<Value, Str
}
pub fn names(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let id = as_session_id(id)?.to_string();
let request = serde_json::json!({"id": 1, "op": "names"});
let reply = send_and_wait(ctx, &id, &request, Duration::from_secs(10))?;
if !reply.get("ok").and_then(|v| v.as_bool()).unwrap_or(false) {
@@ -733,10 +803,14 @@ pub fn names(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
}
pub fn close(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let existed = ctx.foreign_sessions.contains_key(&id);
let id = as_session_id(id)?.to_string();
// Affine close: consuming an absent / already-closed handle is a fault
// (same surface as eval/set/get after close).
if !ctx.foreign_sessions.contains_key(&id) {
return Err(format!("foreign session `{id}` not found"));
}
kill_session(ctx, &id);
Ok(Value::Bool(existed))
Ok(Value::Bool(true))
}
pub fn list(ctx: &HostContext) -> Result<Value, String> {
@@ -745,7 +819,7 @@ pub fn list(ctx: &HostContext) -> Result<Value, String> {
let mut out = Vec::new();
for s in sessions {
out.push(Value::Map(vec![
("id".into(), Value::Text(s.id.clone())),
("id".into(), Value::Handle(s.id.clone())),
("lang".into(), Value::Text(s.lang.clone())),
("cmd".into(), Value::Text(s.cmd.clone())),
("evals".into(), Value::Int(s.evals as i64)),
+145 -2
View File
@@ -1,7 +1,21 @@
//! JSON host primitives + Value↔JSON mapping.
//!
//! Integer gate matches Lean \(V_{\mathsf{safe}}\) (`doc/design/v-safe-json.md`):
//! JSON integers with `|n| > 2^53-1` refuse on encode and decode.
use crate::value::Value;
/// JavaScript `Number.MAX_SAFE_INTEGER` / Lean `ForeignJSON.maxSafe`.
pub const MAX_SAFE_INT: i64 = 9_007_199_254_740_991; // 2^53 - 1
pub fn in_safe_int(n: i64) -> bool {
(-MAX_SAFE_INT..=MAX_SAFE_INT).contains(&n)
}
fn ood_int_err(n: impl std::fmt::Display) -> String {
format!("json: integer outside V_safe (±(2^53-1)): {n}")
}
pub fn parse(text: &Value) -> Result<Value, String> {
let s = match text {
Value::Text(s) => s.as_str(),
@@ -70,11 +84,18 @@ pub(crate) fn json_to_value(v: &serde_json::Value) -> Result<Value, String> {
serde_json::Value::Bool(b) => Value::Bool(*b),
serde_json::Value::Number(n) => {
if let Some(i) = n.as_i64() {
if !in_safe_int(i) {
return Err(ood_int_err(i));
}
Value::Int(i)
} else if let Some(u) = n.as_u64() {
// Always > i64::MAX ⇒ outside V_safe; never coerce via f64/Text.
return Err(ood_int_err(u));
} else if let Some(f) = n.as_f64() {
// Host extension: non-integer JSON numbers → Num (not in Lean SafeVal).
Value::Num(f)
} else {
Value::Text(n.to_string())
return Err(format!("json: unsupported number: {n}"));
}
}
serde_json::Value::String(s) => Value::Text(s.clone()),
@@ -99,7 +120,12 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
Ok(match v {
Value::None => serde_json::Value::Null,
Value::Bool(b) => serde_json::Value::Bool(*b),
Value::Int(n) => serde_json::Value::Number((*n).into()),
Value::Int(n) => {
if !in_safe_int(*n) {
return Err(ood_int_err(*n));
}
serde_json::Value::Number((*n).into())
}
Value::Num(n) => serde_json::Number::from_f64(*n)
.map(serde_json::Value::Number)
.unwrap_or(serde_json::Value::Null),
@@ -112,6 +138,22 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
serde_json::Value::Array(arr)
}
Value::Map(entries) => {
// Column-table maps (all fields lists): refuse unequal list lengths.
let list_lens: Vec<usize> = entries
.iter()
.filter_map(|(_, v)| match v {
Value::List(xs) => Some(xs.len()),
_ => None,
})
.collect();
if list_lens.len() == entries.len() && list_lens.len() >= 2 {
let n0 = list_lens[0];
if list_lens.iter().any(|&n| n != n0) {
return Err(
"json: table column arity mismatch (unequal list lengths)".into(),
);
}
}
let mut map = serde_json::Map::new();
for (k, v) in entries {
map.insert(k.clone(), value_to_json(v)?);
@@ -147,6 +189,9 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
},
Value::Code(c) => serde_json::Value::String(c.source.clone()),
Value::Secret(_) => serde_json::Value::String("<secret>".into()),
Value::Handle(_) => {
return Err("json: session handle cannot be serialized".into());
}
})
}
@@ -159,4 +204,102 @@ mod tests {
let q = quote(&Value::Text("a\"b".into())).unwrap();
assert_eq!(q, Value::Text("\"a\\\"b\"".into()));
}
#[test]
fn roundtrip_max_safe() {
for n in [MAX_SAFE_INT, -MAX_SAFE_INT, 0_i64, 1, -1] {
let j = value_to_json(&Value::Int(n)).unwrap();
let v = json_to_value(&j).unwrap();
assert_eq!(v, Value::Int(n));
}
}
#[test]
fn refuse_ood_encode() {
for n in [
MAX_SAFE_INT + 1,
-MAX_SAFE_INT - 1,
1_i64 << 53, // 2^53
1_i64 << 60, // 2^60
(1_i64 << 60) + 1, // 2^60+1
] {
let err = value_to_json(&Value::Int(n)).unwrap_err();
assert!(
err.contains("outside V_safe"),
"encode {n}: {err}"
);
}
}
#[test]
fn refuse_ood_decode() {
for n in [
MAX_SAFE_INT + 1,
-MAX_SAFE_INT - 1,
1_i64 << 53,
1_i64 << 60,
(1_i64 << 60) + 1,
] {
let raw = format!("{n}");
let j: serde_json::Value = serde_json::from_str(&raw).unwrap();
let err = json_to_value(&j).unwrap_err();
assert!(
err.contains("outside V_safe"),
"decode {n}: {err}"
);
}
}
#[test]
fn refuse_nested_ood() {
let j: serde_json::Value =
serde_json::from_str("{\"a\":[1152921504606846977]}").unwrap();
let err = json_to_value(&j).unwrap_err();
assert!(err.contains("outside V_safe"), "{err}");
}
#[test]
fn decimal_string_ok() {
let j = serde_json::Value::String("1152921504606846977".into());
assert_eq!(
json_to_value(&j).unwrap(),
Value::Text("1152921504606846977".into())
);
}
#[test]
fn float_host_extension_ok() {
let j: serde_json::Value = serde_json::from_str("1.5").unwrap();
match json_to_value(&j).unwrap() {
Value::Num(f) => assert!((f - 1.5).abs() < 1e-12),
other => panic!("expected Num, got {other:?}"),
}
}
#[test]
fn refuse_u64_above_i64_max() {
// 2^63 as JSON integer: as_i64 fails, as_u64 succeeds.
let j: serde_json::Value = serde_json::from_str("9223372036854775808").unwrap();
let err = json_to_value(&j).unwrap_err();
assert!(err.contains("outside V_safe"), "{err}");
}
#[test]
fn refuse_session_handle_encode() {
let err = value_to_json(&Value::Handle("s1".into())).unwrap_err();
assert!(
err.contains("session handle cannot be serialized"),
"{err}"
);
}
#[test]
fn refuse_unequal_column_table_encode() {
let ragged = Value::Map(vec![
("a".into(), Value::List(vec![Value::Int(1), Value::Int(2)])),
("b".into(), Value::List(vec![Value::Int(3)])),
]);
let err = value_to_json(&ragged).unwrap_err();
assert!(err.contains("arity mismatch"), "{err}");
}
}
+18
View File
@@ -891,6 +891,24 @@ impl<'a> Cursor<'a> {
if !header_done {
bail!("expected GFM table after empty assignment");
}
// Strong / Surface: unequal row arity is stuck — no silent pad/truncate.
for (line_no, cells) in &data_rows {
if cells.len() != header.len() {
return Err(Diagnostic::new(
None,
Span {
line: *line_no,
col: 1,
},
format!(
"table row arity mismatch: expected {} columns, got {}",
header.len(),
cells.len()
),
)
.into());
}
}
let row_oriented = header
.first()
.map(|h| is_row_table_marker(h))
+4
View File
@@ -29,6 +29,8 @@ pub enum Value {
Code(CodeBlock),
/// Sensitive text from `${secret.*}` — Display shows `<secret>`; reveal on Text concat.
Secret(String),
/// Opaque foreign-session handle. Not JSON-serializable; not printable structure.
Handle(String),
}
impl Value {
@@ -40,6 +42,7 @@ impl Value {
Value::Num(n) => *n != 0.0 && !n.is_nan(),
Value::Text(s) => !s.is_empty(),
Value::Secret(s) => !s.is_empty(),
Value::Handle(s) => !s.is_empty(),
Value::List(xs) => !xs.is_empty(),
Value::Map(xs) => !xs.is_empty(),
Value::Formula(_) => true,
@@ -64,6 +67,7 @@ impl Value {
Value::Num(n) => crate::formula::format_num(*n),
Value::Text(s) => s.clone(),
Value::Secret(_) => "<secret>".into(),
Value::Handle(_) => "<handle>".into(),
Value::List(xs) => {
let parts: Vec<String> = xs.iter().map(|v| v.as_display()).collect();
format!("[{}]", parts.join(", "))
+2
View File
@@ -199,6 +199,7 @@ const API_ROUTES: &[(&str, ApiRoute)] = &[
("/api/session/snapshot/export", ApiRoute::Notebook(notebook_api::api_export_snapshot)),
("/api/session/snapshot/import", ApiRoute::Notebook(notebook_api::api_import_snapshot)),
("/api/session/snapshot/replay", ApiRoute::Notebook(notebook_api::api_replay_snapshot)),
("/api/session/snapshot/replay_check", ApiRoute::Notebook(notebook_api::api_replay_check)),
("/api/session/plan/assign", ApiRoute::Notebook(notebook_api::api_plan_assign)),
("/api/session/plan/add", ApiRoute::Notebook(notebook_api::api_plan_add)),
("/api/session/plan/remove", ApiRoute::Notebook(notebook_api::api_plan_remove)),
@@ -216,6 +217,7 @@ const API_ROUTES: &[(&str, ApiRoute)] = &[
("/api/session/state/push", ApiRoute::Notebook(notebook_api::api_state_push)),
("/api/session/state/pull", ApiRoute::Notebook(notebook_api::api_state_pull)),
("/api/session/journal", ApiRoute::Notebook(notebook_api::api_journal)),
("/api/session/oracle_journal", ApiRoute::Notebook(notebook_api::api_oracle_journal)),
("/api/debug/start", ApiRoute::Notebook(debug_api::api_debug_start)),
("/api/debug/continue", ApiRoute::Action(crate::debug::DebugAction::Continue)),
("/api/debug/step", ApiRoute::Action(crate::debug::DebugAction::Step)),
+14
View File
@@ -208,6 +208,16 @@ pub fn api_replay_snapshot(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "replay", |s| s.replay_snapshot(&snapshot))
}
pub fn api_replay_check(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let snapshot = v.get("snapshot").cloned().unwrap_or(serde_json::Value::Null);
let snapshot = match json_to_value(&snapshot) {
Ok(v) => v,
Err(e) => return err(e),
};
with_session(root, body, "replay_check", |s| s.replay_check(&snapshot))
}
pub fn api_clear_cell(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
@@ -352,6 +362,10 @@ pub fn api_journal(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "journal", |s| Ok(s.journal_json()))
}
pub fn api_oracle_journal(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "oracle_journal", |s| Ok(s.oracle_journal_json()))
}
#[cfg(test)]
mod tests {
use super::*;
+1
View File
@@ -542,6 +542,7 @@ fn value_kind_label(v: &crate::value::Value) -> String {
Value::Num(_) => "num".into(),
Value::Text(_) => "text".into(),
Value::Secret(_) => "secret".into(),
Value::Handle(_) => "handle".into(),
Value::Formula(crate::formula::Expr::Matrix { rows, .. }) => {
format!("formula · {}×{}", rows.len(), rows.first().map(|r| r.len()).unwrap_or(0))
}
+15
View File
@@ -0,0 +1,15 @@
---
markup: occlusion
title: table row arity mismatch is a fault
---
# main
`t` =
| a | b |
|---|---|
| 1 | 2 |
| 3 |
> print text=`t`
+180
View File
@@ -827,6 +827,15 @@ fn error_bad_arity() {
);
}
#[test]
fn error_table_arity() {
assert_err(
"tests/errors/table-arity.mq.md",
"13:1",
"table row arity mismatch",
);
}
#[test]
fn error_syntax_bad_line() {
assert_err(
@@ -1857,6 +1866,41 @@ fn lib_json_roundtrip() {
);
}
#[test]
fn lib_json_vsafe_max_safe_ok() {
assert_out(
"tests/lib/json-vsafe-max-safe-ok.mq.md",
"9007199254740991\n{\"n\":9007199254740991}",
);
}
#[test]
fn lib_json_vsafe_string_ok() {
assert_out(
"tests/lib/json-vsafe-string-ok.mq.md",
"1152921504606846977",
);
}
#[test]
fn lib_json_vsafe_refuse() {
// Span points at lib/json.mq.md `parse` body while path stays the caller file.
assert_err(
"tests/lib/json-vsafe-refuse.mq.md",
"16:1",
"json: integer outside V_safe",
);
}
#[test]
fn lib_json_vsafe_stringify_refuse() {
assert_err(
"tests/lib/json-vsafe-stringify-refuse.mq.md",
"24:1",
"json: integer outside V_safe",
);
}
#[test]
fn lib_table_collections() {
assert_out(
@@ -3579,6 +3623,142 @@ fn lib_foreign_session_python() {
assert_out("tests/lib/foreign-session-python.mq.md", "42\n5");
}
#[test]
fn lib_foreign_session_vsafe_refuse() {
if !node_available() {
eprintln!("skip lib_foreign_session_vsafe_refuse: no node on PATH");
return;
}
// Span points at lib/foreign.mq.md session_set body.
assert_err(
"tests/lib/foreign-session-vsafe-refuse.mq.md",
"207:1",
"json: integer outside V_safe",
);
}
#[test]
fn lib_foreign_session_use_after_close() {
if !node_available() {
eprintln!("skip lib_foreign_session_use_after_close: no node on PATH");
return;
}
assert_err(
"tests/lib/foreign-session-use-after-close.mq.md",
"198:1",
"foreign session",
);
}
#[test]
fn lib_foreign_session_double_close() {
if !node_available() {
eprintln!("skip lib_foreign_session_double_close: no node on PATH");
return;
}
assert_err(
"tests/lib/foreign-session-double-close.mq.md",
"231:1",
"foreign session",
);
}
#[test]
fn lib_foreign_session_use_before_open() {
// Text fabrication is refused before registry lookup (opaque handle discipline).
assert_err(
"tests/lib/foreign-session-use-before-open.mq.md",
"198:1",
"session id must be a handle",
);
}
#[test]
fn lib_foreign_session_handle_no_serialize() {
if !node_available() {
eprintln!("skip lib_foreign_session_handle_no_serialize: no node on PATH");
return;
}
assert_err(
"tests/lib/foreign-session-handle-no-serialize.mq.md",
"24:1",
"session handle cannot be serialized",
);
}
#[test]
fn lib_foreign_session_result_vsafe_refuse() {
if !python_available() {
eprintln!("skip lib_foreign_session_result_vsafe_refuse: no python on PATH");
return;
}
assert_err(
"tests/lib/foreign-session-result-vsafe-refuse.mq.md",
"198:1",
"json: integer outside V_safe",
);
}
#[test]
fn lib_foreign_session_isolation() {
if !node_available() {
eprintln!("skip lib_foreign_session_isolation: no node on PATH");
return;
}
assert_out("tests/lib/foreign-session-isolation.mq.md", "41\nFalse");
}
#[test]
fn lib_foreign_session_cold_replay_empty() {
if !node_available() {
eprintln!("skip lib_foreign_session_cold_replay_empty: no node on PATH");
return;
}
assert_err(
"tests/lib/foreign-session-cold-replay-empty.mq.md",
"198:1",
"foreign session",
);
}
#[test]
fn lib_foreign_vsafe_refuse() {
if !python_available() {
eprintln!("skip lib_foreign_vsafe_refuse: no python on PATH");
return;
}
assert_err(
"tests/lib/foreign-vsafe-refuse.mq.md",
"16:1",
"json: integer outside V_safe",
);
}
#[test]
fn lib_foreign_vsafe_postarith_refuse() {
if !python_available() {
eprintln!("skip lib_foreign_vsafe_postarith_refuse: no python on PATH");
return;
}
assert_err(
"tests/lib/foreign-vsafe-postarith-refuse.mq.md",
"16:1",
"json: integer outside V_safe",
);
}
#[test]
fn lib_foreign_vsafe_string_ok() {
if !node_available() {
eprintln!("skip lib_foreign_vsafe_string_ok: no node on PATH");
return;
}
assert_out(
"tests/lib/foreign-vsafe-string-ok.mq.md",
"{\"out\":\"1152921504606846978\"}",
);
}
#[test]
fn bytecode_lib_math_formula() {
assert_out_backend(
@@ -0,0 +1,13 @@
---
markup: occlusion
title: closed handle cannot be reused across a later step
import:
foreign: lib/foreign.mq.md
---
# main
**会话 = > foreign.session_open lang="node" timeout=30**
**> foreign.session_eval id=`会话` source="var x = 1"**
**> foreign.session_close id=`会话`**
**> foreign.session_eval id=`会话` source="x"**
@@ -0,0 +1,12 @@
---
markup: occlusion
title: foreign session double-close is a fault
import:
foreign: lib/foreign.mq.md
---
# main
**会话 = > foreign.session_open lang="node" timeout=30**
**> foreign.session_close id=`会话`**
**> foreign.session_close id=`会话`**
@@ -0,0 +1,13 @@
---
markup: occlusion
title: foreign session handle cannot cross JSON bridge
import:
foreign: lib/foreign.mq.md
json: lib/json.mq.md
---
# main
**会话 = > foreign.session_open lang="node" timeout=30**
**> json.stringify value=`会话`**
**> foreign.session_close id=`会话`**
+18
View File
@@ -0,0 +1,18 @@
---
markup: occlusion
title: two same-lang sessions do not share bindings
import:
foreign: lib/foreign.mq.md
---
# main
**a = > foreign.session_open lang="node" timeout=30**
**b = > foreign.session_open lang="node" timeout=30**
**> foreign.session_eval id=`a` source="var shared = 41"**
**ra = > foreign.session_eval id=`a` source="shared"**
**rb = > foreign.session_eval id=`b` source="shared"**
**> print text=[result](access(`ra`, "result"))**
**> print text=[ok](access(`rb`, "ok"))**
**> foreign.session_close id=`a`**
**> foreign.session_close id=`b`**
@@ -0,0 +1,12 @@
---
markup: occlusion
title: foreign session_eval result outside V_safe is a fault
import:
foreign: lib/foreign.mq.md
---
# main
**py = > foreign.session_open lang="python" timeout=30**
**> foreign.session_eval id=`py` source="2**60 + 1"**
**> foreign.session_close id=`py`**
@@ -0,0 +1,12 @@
---
markup: occlusion
title: foreign session use-after-close is a fault
import:
foreign: lib/foreign.mq.md
---
# main
**会话 = > foreign.session_open lang="node" timeout=30**
**> foreign.session_close id=`会话`**
**> foreign.session_eval id=`会话` source="1 + 1"**
@@ -0,0 +1,10 @@
---
markup: occlusion
title: foreign session text id fabrication is a fault
import:
foreign: lib/foreign.mq.md
---
# main
**> foreign.session_eval id="no-such-session" source="1 + 1"**
@@ -0,0 +1,13 @@
---
markup: occlusion
title: foreign session_set V_safe refuse
import:
foreign: lib/foreign.mq.md
---
# main
**node = > foreign.session_open lang="node" timeout=30**
**n = 1152921504606846977**
**> foreign.session_set id=`node` name="__n" value=`n`**
**> foreign.session_close id=`node`**
@@ -0,0 +1,24 @@
---
markup: occlusion
title: foreign V_safe refuse after Python arithmetic
import:
foreign: lib/foreign.mq.md
json: lib/json.mq.md
---
# main
**py = > foreign.session_open lang="python" timeout=30**
`emit` =
```python
import json
n = 2**53 - 1
print(json.dumps({"n": 2 * n}))
```
**raw = > foreign.session_eval id=`py` code=`emit`**
**stdout = > json.get value=`raw` key="stdout"**
**payload = > json.parse text=`stdout`**
**> print text=`payload`**
**> foreign.session_close id=`py`**
+23
View File
@@ -0,0 +1,23 @@
---
markup: occlusion
title: foreign Python JSON number V_safe refuse
import:
foreign: lib/foreign.mq.md
json: lib/json.mq.md
---
# main
**py = > foreign.session_open lang="python" timeout=30**
`emit` =
```python
import json
print(json.dumps({"n": 2**60 + 1}))
```
**raw = > foreign.session_eval id=`py` code=`emit`**
**stdout = > json.get value=`raw` key="stdout"**
**payload = > json.parse text=`stdout`**
**> print text=`payload`**
**> foreign.session_close id=`py`**
+24
View File
@@ -0,0 +1,24 @@
---
markup: occlusion
title: foreign V_safe string to Node BigInt
import:
foreign: lib/foreign.mq.md
json: lib/json.mq.md
---
# main
**node = > foreign.session_open lang="node" timeout=30**
**s = "1152921504606846977"**
`inc` =
```javascript
const out = (BigInt(globalThis.__s) + 1n).toString();
console.log(JSON.stringify({out}));
```
**> foreign.session_set id=`node` name="__s" value=`s`**
**js = > foreign.session_eval id=`node` code=`inc`**
**js_out = > json.get value=`js` key="stdout"**
**> print text=`js_out`**
**> foreign.session_close id=`node`**
+15
View File
@@ -0,0 +1,15 @@
---
markup: occlusion
title: json V_safe MAX_SAFE roundtrip
import:
json: lib/json.mq.md
---
# main
**obj = > json.parse text={"n":9007199254740991}**
**n = > json.get value=`obj` key="n"**
**out = > json.stringify value=`obj`**
> print text=`n`
> print text=`out`
+12
View File
@@ -0,0 +1,12 @@
---
markup: occlusion
title: json V_safe refuse OOD integer
import:
json: lib/json.mq.md
---
# main
**bad = > json.parse text={"n":1152921504606846977}**
> print text=`bad`
+13
View File
@@ -0,0 +1,13 @@
---
markup: occlusion
title: json V_safe decimal string ok
import:
json: lib/json.mq.md
---
# main
**obj = > json.parse text={"s":"1152921504606846977"}**
**s = > json.get value=`obj` key="s"**
> print text=`s`
@@ -0,0 +1,13 @@
---
markup: occlusion
title: json V_safe refuse OOD Int stringify
import:
json: lib/json.mq.md
---
# main
**n = 1152921504606846977**
**out = > json.stringify value=`n`**
> print text=`out`