Compare commits

...
41 Commits
Author SHA1 Message Date
cflmy 33a14d272c release: v1.5.2 — foreign v3 P0/P1 (stdin, plan API, snapshot, capture, persist) 2026-10-18 14:57:15 +08:00
cflmy 6ed70fd8b8 docs(design): defer G7 and G8 from the v3 target
Keep the cross-document scope and non-Python/Node adapter designs in the document for future work, but remove them from the current implementation target and priorities.
2026-10-18 14:51:12 +08:00
cflmy 115d1f6c0b feat(view): explicit persist and nested-plan boundary
Add per-cell Persist backed by lib/writeback and /api/session/cell/persist. Reject nested foreign.session plans with a top-level diagnostic and tests.
2026-10-18 14:47:54 +08:00
cflmy 7dc5e56850 feat(foreign): reject nested view plans
Enforce the v3 plan boundary: foreign.session must be a top-level binding. Nested plans are diagnosed instead of silently becoming stateless. Tests cover the AST helper and DocumentSession open.
2026-10-18 14:46:57 +08:00
cflmy 514caa55ae docs(design): mark G9 snapshot capture implemented
view capture and view output --snapshot are implemented with source-hash validation; remaining gaps are G3/G7/G8/G10.
2026-10-18 14:46:20 +08:00
cflmy 74908ebf04 feat(view): capture live snapshots for static export
Add view capture to record per-cell outputs and session summary with source hashes. view output --snapshot consumes the capture without re-executing and rejects source drift. Includes snapshot parsing tests.
2026-10-18 14:45:26 +08:00
cflmy 0f852593bc docs(design): record foreign v3 P0/P1 implementation status
Mark stdin/per-cell inputs/interrupt and plan/snapshot APIs as implemented; keep G3/G7/G8/G9/G10 as pending design items.
2026-10-18 14:44:29 +08:00
cflmy 34e2fb4f74 feat(foreign): snapshot export, import and replay
Export JSON-safe kernel values, cells and journal; restore values into a matching plan; replay cells with recorded stdin. Add /api/session/snapshot/{export,import,replay}.
2026-10-18 14:43:53 +08:00
cflmy 0e407b2bd3 feat(foreign): runtime plan updates and generation
Add explicit add/remove cell and set-value operations with a plan generation counter. Expose /api/session/plan/{assign,add,remove,value}.
2026-10-18 14:43:26 +08:00
cflmy e5e532d83f fix(foreign): keep session_list arity empty
ForeignSessionList accepts no id; only ForeignSessionInterrupt takes id.
2026-10-18 14:38:27 +08:00
cflmy 064522a338 feat(foreign): raw stdin, per-cell inputs and interrupt
Persistent Python/Node kernels receive raw program stdin per eval; foreign.session accepts an inputs table and view exposes per-cell stdin. Add session interrupt host API and view control.
2026-10-18 14:37:45 +08:00
cflmy c85f1a2829 feat(foreign): support raw stdin and per-cell inputs
Session eval now carries raw stdin, and the Python/Node workers temporarily replace program stdin during evaluation. foreign.session accepts an inputs table; view exposes per-cell stdin and Run All sends a per-cell input map. Stateless and persistent cells share the same input semantics.
2026-10-18 14:36:47 +08:00
cflmy a7266185b2 docs(design): propose foreign v3 capability plan
Document the remaining foreign runtime gaps: persistent raw stdin, per-cell input, dynamic plan boundaries, runtime plan updates, interrupt, state export/replay, cross-document scope, non-Python/Node adapters, live-to-static snapshots and explicit writeback policy.
2026-10-18 14:31:44 +08:00
cflmy 4d7c0bea1b fix: sync Cargo.lock with v1.5.1 version bump 2026-10-18 14:23:20 +08:00
cflmy 53758ebd1c debian: changelog entry for 1.5.1-1ppa1 2026-10-18 14:20:24 +08:00
cflmy dff0b700ae release: v1.5.1 — view static/live unified + foreign value passing 2026-10-18 14:05:19 +08:00
cflmy 4da07c1b26 fix(view): do not turn documentation tables into call chips
Keep descriptive table labels non-callable and suppress call annotations on table blocks. This removes the collapsed call {步骤...} chip while preserving the table and the return 950 chips at the actual foreign/writeback calls.
2026-10-18 13:59:24 +08:00
cflmy 78922102e4 fix(view): anchor imported-function returns at their call site
Track the caller line on CallFrame and use it for return annotations instead of the callee library line. Return 950 stays visible on the actual foreign/writeback calls; the Markdown table remains intact and no longer receives stray return chips.
2026-10-18 13:56:50 +08:00
cflmy 064bae2c65 docs(view): keep writeback table labels as inline code
Wrap foreign.run/session and writeback.record table labels in code spans so the prose table is not traced as executable calls; this removes the stray return 950 chips.
2026-10-18 13:52:51 +08:00
cflmy 92b91ffc81 fix(view): group and theme writeback blocks
Render contiguous writeback comment lines as one block, and map output-card/writeback tokens to the View dark theme. Static and live pages no longer show three dashed fragments or light output cards on dark.
2026-10-18 13:51:11 +08:00
cflmy ab6648ae0e docs(view): make the writeback page itself the runnable example
Move the foreign + writeback demo into the user persistence page. The page runs a Python code fence, records a keyed total slot, and the slot is visible in both source and static rendering. Remove the separate example document.
2026-10-18 13:48:05 +08:00
cflmy 5acf026e0c docs(view): add a runnable foreign writeback example
Move persistence guidance out of the walkthrough into separate static pages. Add examples/foreign-writeback/index.mq.md, which executes a foreign code fence and writes a keyed total slot into its own source.
2026-10-18 13:45:27 +08:00
cflmy 8dcd69412f docs(view): recommend static snapshots and writeback persistence
Add a persistence recommendation to the user-facing foreign walkthrough: static export for published snapshots, and lib/writeback for opt-in result slots in the .mq.md source. Show a readable code-fence based writeback example.
2026-10-18 13:40:44 +08:00
cflmy 2a13e52981 docs(view): show low-level session evaluation as code fences
Replace inline source strings with readable Python code fences bound as Marqdo code values. The low-level session evaluates those values; no long escaped source argument remains.
2026-10-18 13:35:51 +08:00
cflmy 9b269f6447 feat(foreign): pass Marqdo values into multi-kernel view sessions
foreign.session accepts a values table and initializes each kernel from ordinary Marqdo values. The static walkthrough now uses two kernels and a shared 订单 value instead of redefining orders inside Python. Static pages embed the session summary so kernel controls and Clean stay visible.
2026-10-18 13:32:51 +08:00
cflmy 15362cb048 fix(view): restore per-cell kernel switch and Clean
Code cards again show a kernel selector and a Clean button. Clean calls /api/session/cell/clear for live sessions and clears local output in static snapshots, without restarting kernels.
2026-10-18 13:27:15 +08:00
cflmy 05d48f68a4 docs(view): show kernel state and Marqdo value passing
Rewrite the static foreign walkthrough without stdin-driven examples. It now demonstrates session_set from an ordinary Marqdo table, persistent kernel state across evaluations, and a foreign.session view plan sharing state across cells. Static export executes those cells and embeds the real outputs.
2026-10-18 13:23:03 +08:00
cflmy e4387259ba fix(view): make static examples self-contained
Foreign cell examples read stdin with a deterministic fallback, so static exports show real output without tracebacks. Low-level session bindings named 会话 are no longer mistaken for a view plan; static export runs all cells again.
2026-10-18 13:19:41 +08:00
cflmy 518f8e386b fix(view): avoid stdin tracebacks in static snapshots
Static export skips cells that read stdin when the document has no preset stdin, and shows a short explanation instead of a traceback. Preset stdin still executes normally.
2026-10-18 13:14:17 +08:00
cflmy 9da12c9093 feat(view): add cell stdin for live and static runs
View gets a Cell stdin/input field wired to cell Run, Step and Run All. Stateless cells receive raw stdin; persistent kernels receive marqdo_input. Static export uses the document preset stdin and the same run_all path.
2026-10-18 13:10:08 +08:00
cflmy e0d34669e0 fix(view): use the same run path for static and live cells
Run All now records per-cell failures and continues; static export uses the same foreign_doc run_all and injects those records. A cell that succeeds in view therefore succeeds in the static snapshot, and failures match.
2026-10-18 13:03:46 +08:00
cflmy 0ba4d3d2d7 feat(view): render real cell outputs in static exports
Static export opens the foreign document session, runs cells once, and injects stdout/stderr/error into the code cards. Failures are shown as real errors; --no-exec still skips execution.
2026-10-18 13:00:41 +08:00
cflmy ad3ebf243a fix(view): keep static buttons with a no-live notice
Static pages now render the same toolbar and Execution Clear button as live view. Clicking any execution control shows a static-export notice instead of hiding controls or pretending to run.
2026-10-18 12:55:07 +08:00
cflmy 142d30ce08 fix(view): share the same shell between static and live output
Static exports now include the View shell and dark theme; interactive toolbar and Execution Clear stay hidden, while dynamic pages show them.
2026-10-18 12:52:48 +08:00
cflmy 4757b82edd fix(view): run stateless documents and clear Execution output
No-session view Run All now runs the document and renders stdout into Execution; Clear outputs clears Execution output, plots and plan cards as well as cell outputs.
2026-10-18 12:51:13 +08:00
cflmy bff2f53e52 fix(view): clear Execution output from the view surface
Add an Execution Clear button and clear the Execution out/plots/plan card in addition to code-cell outputs. The same clearOutputs path calls /api/session/clear, so ordinary stateless pages and foreign sessions are both wired.
2026-10-18 12:47:56 +08:00
cflmy fb28eee62f feat(foreign): own view sessions and clear outputs
foreign.session returns a handle from ordinary kernel/cell tables; view consumes only the 会话 binding. Sessions live in host foreign_doc. Without 会话, view cells run stateless one-shot. Add /api/session/clear and a Clear outputs button for every view page; clearing keeps persistent kernel state.
2026-10-18 12:42:15 +08:00
cflmy 01468ad976 test(view): add foreign and non-foreign preview fixtures
Add a pure Marqdo control page and make the Python view-cells example explicitly import lib/foreign.
2026-10-18 12:22:36 +08:00
cflmy f3791d11b4 refactor(view): evaluate cell plans through the interpreter
Delete the source-line table scanner. 内核表/会话表 are now evaluated as ordinary Marqdo bindings (with simple assignment dependencies) via Interpreter::capture_declarations. Every view page uses the same cell toolbar; run/watch semantics are unchanged; no new CLI or Markdown marker.
2026-10-18 12:17:42 +08:00
cflmy 7fef690167 refactor(view): unify cells into the view surface
Cell controls now appear only when a document declares 内核表/会话表. Remove Notebook branding and Jupyter In/Out prompts; ordinary view pages stay classic. There is no new CLI command or second interactive mode; run remains canonical execution and view is the only interactive surface.
2026-10-18 12:03:17 +08:00
cflmy 024cdee82a debian: add 1.5.0 PPA changelog 2026-10-18 03:36:53 +08:00
49 changed files with 4603 additions and 1523 deletions

No files matched your search

+1 -1
View File
@@ -210,7 +210,7 @@ Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7
- `invoke` returns a record (`ok`, `code`, `stdout`, `stderr`, `ms`, `timed_out`, `truncated`, `result`); pass `input` for JSON stdin and `output="json"` for a parsed result.
- `call` is JSON-in / JSON-out; `call_many` fans out one program over a list with ordered results.
- `session_open` starts a persistent Python/Node kernel; `session_eval` evaluates source; `session_set` / `session_get` bridge values explicitly as JSON. Close it with `session_close`.
- A notebook is still a document: bind `内核表` and `会话表` as ordinary tables inside `# main`. A `view` session adds cell stepping, Run All, kernel switching, journal and reload. No new Markdown marker exists.
- A view document opts into kernel state with `会话 = > foreign.session kernels=... cells=...`; the table names are free. `view` consumes only the `会话` handle and never scans source lines or reserved table names. Without `会话`, cells are stateless one-shot runs; no kernel is created. Clearing outputs calls `/api/session/clear` and keeps persistent kernel state. There is no separate notebook CLI mode or command, and no new Markdown marker exists.
- The same source remains the authority; state crossing kernels is explicit, never implicitly mirrored.
## AI authoring workflow
+5
View File
@@ -126,6 +126,11 @@ jobs:
## Marqdo ${{ github.ref_name }}
### 亮点 / Highlights
- **v1.5.2**:外联 v3 P0/P1——raw stdin + 每 cell 输入 · plan API · snapshot · `view capture` · Persist
- **v1.5.2**: Foreign v3 P0/P1 — raw stdin + per-cell inputs, plan API, snapshot export/import/replay, `view capture`, Persist
- **v1.5.2 ext**:扩展包同步 1.5.2 / Ext pack 1.5.2
- **v1.5.1**:view 静态/动态统一 + 外联值传递——共用执行路径 · 构建时运行注入 stdout · `foreign.session` 值传递 · cell stdin
- **v1.5.1**: View static/live unified + foreign value passing — shared execution path, build-time run with stdout injection, `foreign.session` value passing, cell stdin
- **v1.5.0**:外联 v2 + 文档原生 Notebook——invoke/JSON call/并行/持久 Python·Node/状态桥 · cell 单步 · 高对比 view
- **v1.5.0**: Foreign v2 + document-native notebooks — invoke, JSON calls, fan-out, persistent Python/Node, state bridge, cell stepping, high-contrast view
- **v1.5.0 ext**:扩展包同步 1.5.0 / Ext pack 1.5.0
+67 -1
View File
@@ -8,6 +8,72 @@
### Changed
## v1.5.2 — 2026-10-18
### Highlights
**外联 v3 P0/P1**:session eval 支持 raw stdin(Python/Node 持久内核);每 cell 独立 stdin 与 Interrupt;plan generation/add/remove/value API;snapshot export/import/replay;`view capture` / `view output --snapshot` 捕获实时快照;显式 Persist 按钮。CLI · wasm · ext **1.5.2**。
```bash
git checkout v1.5.2 && cargo build --release
marqdo ext add web # CDN → 1.5.2
```
**核心表面 Δ(Core surface Δ)**:**无增删**——外联 v3 扩展 `lib/foreign` 库 API;不改 MQ 关键词 / 表几何。
### Added
- **raw stdin + 每 cell 输入**:session eval 支持 raw stdin;`foreign.session inputs=` 每 cell 独立 stdin;view 增加每卡 stdin 字段与 Interrupt 按钮。
- **plan API**:plan generation/add/remove/value;运行时 plan 更新与 generation。
- **snapshot**:snapshot export/import/replay;`view capture` / `view output --snapshot` 捕获 live 快照用于静态导出。
- **显式 Persist**:Persist 按钮 + `/api/session/cell/persist`。
- **嵌套 plan 边界**:显式 persist 与嵌套 plan 边界;foreign 拒绝嵌套 view plans。
### Fixed
- **`session_list` arity**:保持空 arity。
### Changed
- CLI / wasm / `ext/VERSION` **同步 1.5.2**。
## v1.5.1 — 2026-10-18
### Highlights
**view 静态/动态统一 + 外联值传递**:静态导出与 live view 共用同一套 `foreign_doc` 执行路径;构建时实际运行代码单元并注入 stdout;`foreign.session` 可把 Marqdo 值传入多内核会话;cell stdin 支持 live/static/持久内核三种模式。CLI · wasm · ext **1.5.1**。
```bash
git checkout v1.5.1 && cargo build --release
marqdo ext add web # CDN → 1.5.1
```
**核心表面 Δ(Core surface Δ)**:**无增删**——view 执行路径统一 + 外联值传递;不改 MQ 关键词 / 表几何。
### Added
- **外联值传递**:`foreign.session kernels= cells=` 可把 Marqdo 值传入多内核 view 会话;`view` 只消费 `会话` 绑定,不再识别 `内核表` / `会话表` 保留名。
- **Cell stdin**:view 增加 `Cell stdin / input` 输入框;Run/Step/Run All 都会传给外联 cells;无状态 cell 使用 raw stdin,持久内核使用 `marqdo_input`;静态导出使用 frontmatter `stdin` 作为构建时输入。
- **静态导出真实执行**:构建时实际执行代码单元并把 stdout/stderr 注入代码卡;静态示例通过代码 fallback 或 frontmatter `stdin` 提供输入。
- **可运行外联写回示例**:新增 `examples/foreign-writeback/index.mq.md`,用 `foreign.run` + `writeback.record` 将结果写入 `.mq.md` 的 keyed 槽位。
- **写回块分组与主题**:view 对写回块按目标分组并统一主题样式。
### Fixed
- **code 卡控制恢复**:恢复每个代码卡的内核切换下拉与 Clean 按钮;Clean 清除单个 cell 输出/记录,不重启内核。
- **导入函数返回锚定**:导入函数的返回值锚定在调用位置。
- **静态导出自包含**:静态示例不再依赖空 stdin 或外部内核状态。
- **Execution 输出清除**:view 清除 Execution 输出从 view 表面。
- **文档表不再变 call chips**:view 不再把文档表转为 call chips。
- **写回表标签**:写回表标签保持为行内代码。
### Changed
- **静态/动态统一执行路径**:static 与 live cells 使用同一 `foreign_doc` `run_all` 路径;移除 `In []` / `Out []` 与 Notebook 品牌。
- **静态按钮行为**:静态页面执行按钮点击显示"静态页面无法真实执行",不伪造成功。
- **低层会话评估可见**:文档展示低层 session evaluation 为代码围栏。
- CLI / wasm / `ext/VERSION` **同步 1.5.1**。
## v1.5.0 — 2026-10-18
### Highlights
@@ -22,7 +88,7 @@ git checkout v1.5.0 && cargo build --release
### Added
- **用户静态站(外联 / OKF / EKC)**:新增中英双语 `features/19-foreign-notebook` 与 `features/20-okf-ekc`,重写 `stdlib/foreign`,首页加入「一份文件三个面」、外联 Python 运行、Notebook 表、OKF/EKC 与 preflight 说明。
- **用户静态站(外联 / OKF / EKC)**:新增中英双语 `features/19-foreign-view-cells` 与 `features/20-okf-ekc`,重写 `stdlib/foreign`,首页加入「一份文件三个面」、外联 Python 运行、Notebook 表、OKF/EKC 与 preflight 说明。
- **`lib/foreign` F0/F1**:`invoke` 结构化记录(`ok`/`code`/`stdout`/`stderr`/`ms`/`timed_out`/`truncated`/`result`);`run`/`run_lang` 新增 `env`/`cwd`/`timeout`/`args`/`max_output`;15 个语言预设;`probe`/`register`/`unset`/`load_config`;`.marqdo/foreign.toml`。
- **`lib/foreign` F2**:JSON 进 / JSON 出 `call`;并行 `run_many` / `call_many`(有序、单项隔离、并发上限);`invoke` 的 `input` / `output` / `result` / `result_error`。
- **`lib/foreign` F3/F3.5**:Python / Node 持久会话 `session_open` / `session_eval` / `session_close` / `session_list`;状态桥 `session_set` / `session_get` / `session_names`;会话超时即杀、崩溃隔离、drop 清理。
Generated
+2 -2
View File
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "marqdo"
version = "1.5.0"
version = "1.5.2"
dependencies = [
"anyhow",
"assert_cmd",
@@ -603,7 +603,7 @@ dependencies = [
[[package]]
name = "marqdo-wasm"
version = "1.5.0"
version = "1.5.2"
dependencies = [
"marqdo",
"serde_json",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo"
version = "1.5.0"
version = "1.5.2"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo — Markdown markers as an interpreted programming language"
+6 -5
View File
@@ -149,9 +149,10 @@ marqdo catalog public -o .marqdo
---
## 现状(v1.5.0)
## 现状(v1.5.2)
- **外联 v2 / 文档原生 Notebook(v1.5.0)**:结构化 invoke · JSON call/call_many · 并行扇出 · Python/Node 持久内核 · 显式状态桥 · cell 单步/Run All/内核切换 · 高对比浅深主题;扩展包同步 **1.5.0**
- **外联 v3 P0/P1(v1.5.2)**:raw stdin + 每 cell 独立输入 · plan API · snapshot export/import/replay · `view capture` · Persist 按钮 · 扩展包同步 **1.5.2**
- **view 静态/动态统一 + 外联值传递(v1.5.1)**:静态导出与 live view 共用 `foreign_doc` 执行路径 · 构建时运行注入 stdout · `foreign.session` 值传递 · cell stdin;扩展包同步 **1.5.0**
- **裸自增升参(v1.4.6)**:首次粗体赋值左值即声明;`**n = n + 1**` 无反引号也升为必选形参。扩展包仍为 **1.4.5**
- **Streamable HTTP MCP + web invoke 鉴权(v1.4.5)**:`agent_mcp_serve transport=http` · Bearer/`X-Api-Key` 注入 · `unauthorized`→401 · openai/agent `json.get` 硬化
- **LLM 传输软失败 + `ext/ai` 表优先(v1.4.4)**:上游不可达 → `status=error`(不 abort);`agent`/`llm` 去 `json.set` 胶水 · [agent-llm-transport-soft-2026-10.md](doc/research/agent-llm-transport-soft-2026-10.md)
@@ -185,7 +186,7 @@ marqdo catalog public -o .marqdo
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
### 如何使用最新 Marqdo(v1.5.0)
### 如何使用最新 Marqdo(v1.5.2)
**安装解释器(任选其一)**
@@ -200,7 +201,7 @@ marqdo catalog public -o .marqdo
```bash
# 1) 源码安装(跟 tag 或 main)
git clone https://github.com/cflmy/marqdo.git && cd marqdo
git checkout v1.5.0 # 或留在 main
git checkout v1.5.2 # 或留在 main
cargo build --release
export PATH="$PWD/target/release:$PATH"
marqdo version
@@ -239,7 +240,7 @@ marqdo wasm build
# → dist/wasm/ … 见 examples/browser-hello/
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
# https://github.com/cflmy/marqdo/releases/tag/v1.5.0
# https://github.com/cflmy/marqdo/releases/tag/v1.5.2
```
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo-wasm"
version = "1.5.0"
version = "1.5.2"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
+29
View File
@@ -1,3 +1,32 @@
marqdo (1.5.2-1ppa1) noble; urgency=medium
* Marqdo 1.5.2: Foreign v3 P0/P1 — raw stdin with per-cell inputs and
interrupt, plan generation/add/remove/value API, snapshot
export/import/replay, view capture for static export, explicit Persist
button, nested plan boundary.
-- cflmy <pingan@cflmy.cn> Sun, 18 Oct 2026 15:00:00 +0800
marqdo (1.5.1-1ppa1) noble; urgency=medium
* Marqdo 1.5.1: View static/live execution path unified; foreign value
passing into multi-kernel sessions; cell stdin for live/static/persistent
kernels; build-time execution with stdout injection in static exports;
runnable foreign writeback example; writeback block grouping.
-- cflmy <pingan@cflmy.cn> Sun, 18 Oct 2026 22:00:00 +0800
marqdo (1.5.0-1ppa1) noble; urgency=medium
* Marqdo 1.5.0: Foreign kernels v2 (structured invoke, JSON calls,
parallel fan-out, persistent Python/Node kernels, explicit state
bridge) and document-native notebooks in view; high-contrast
light/dark view themes, resizable code/output panes, OKF/EKC and
Python examples on the user site. Fixes branch-local variables
leaking into required constructor parameters.
-- cflmy <pingan@cflmy.cn> Sun, 18 Oct 2026 03:30:00 +0800
marqdo (1.4.6-1ppa1) noble; urgency=medium
* Marqdo 1.4.6: Promote a bare self-increment to a required parameter.
+654
View File
@@ -0,0 +1,654 @@
# 外联运行时 v3 — 能力缺口与深化设计(Proposed)
| | |
|---|---|
| 状态 | **Proposed;尚未实现** |
| 日期 | 2026-10-18 |
| 层 | Runtime(foreign)· Document(`foreign.session`)· View(投影) |
| 相关 | [foreign-kernel.md](foreign-kernel.md) · [notebook-session.md](notebook-session.md) · [foreign-v2.md](../roadmap/foreign-v2.md) · [stdlib-writeback.md](stdlib-writeback.md) · [view-static-interactive.md](view-static-interactive.md) |
| 核心表面 Δ | **无增删**:不新增 Markdown 标记、不新增 notebook CLI |
> 本文只定义设计,不承诺已实现。当前基线是 v1.5.0 之后的本地工作:`foreign.session` 句柄、多内核、values 初始化、view 控制、静态执行与显式 writeback 已可用;本文处理仍然缺失的能力。
---
## 1. 设计约束
v3 必须继续满足以下约束,否则不进入实现:
1. **外联库拥有内核与状态**:`foreign` 管理内核、会话、cell 计划、输入、输出、journal 和生命周期。`view` 只消费句柄,不拥有第二套运行时。
2. **表与围栏是值**:`内核表`、`单元表`、`输入表`、`值表` 都是普通 Marqdo 值;没有保留表名,没有新增标记。
3. **单一执行语义**:`run` 是规范执行;`view` 是同一文档的交互投影;静态导出是同一会话的快照,不另写解释器。
4. **没有 Notebook 模式**:没有 `marqdo notebook`,没有 `In/Out`,没有“普通 view / notebook view”两套布局。
5. **显式 I/O**:Marqdo 值通过 `values` / `session_set` 传递;程序 stdin 是单独概念,不能伪装成 `marqdo_input`。
6. **显式持久化**:默认不改源码;writeback 必须由文档显式调用,或由用户显式选择策略。
7. **安全默认值**:执行受 capability 控制;输入/输出有界;不反序列化任意对象;跨文档共享必须显式开启。
---
## 2. 基线:现在已经有什么
| 能力 | 状态 |
|---|---|
| `foreign.run` / `run_lang` / `invoke` / `probe` | 已实现 |
| `foreign.call` / `call_many` / JSON 进出 / 并行 | 已实现 |
| `session_open` / `eval` / `set` / `get` / `names` / `close` / `list` | 已实现 |
| `foreign.session kernels= cells= values=` | 已实现 |
| 多内核 view 会话、cell 顺序、内核归属 | 已实现 |
| Marqdo 表通过 `values` 传入多个内核 | 已实现 |
| 无 `会话` 时无状态 cells | 已实现 |
| view 每 cell 内核选择 / Clean / Run | 已实现 |
| 静态导出执行 cells 并嵌入真实输出 | 已实现 |
| 静态页 session summary、内核归属、多内核展示 | 已实现 |
| 显式 `writeback.record` | 已实现(库能力) |
| 静态页按钮的“不可执行”提示 | 已实现 |
---
## 2.1 实施状态(2026-10-18)
| ID | 状态 | 备注 |
|---|---|---|
| G1/G2 | ✅ 已实现 | session eval raw stdin;`foreign.session inputs=`;per-cell UI;Python/Node 测试 |
| G5 | ✅ 基本实现 | `session_interrupt` + view Interrupt;Python SIGINT 保留会话;Node 进程可能被终止 |
| G4 | ✅ 已实现(API) | add/remove/set-value + generation;view 尚未提供 add/remove UI |
| G6 | ✅ 已实现(API) | export/import/replay JSON snapshot;view/CLI 尚未做文件封装 |
| G9 | ✅ 已实现(CLI/API 基础) | `view capture` + `view output --snapshot`;source hash 校验 |
| G3 | ✅ 边界已实现 | 顶层 plan;嵌套 plan 诊断 |
| G10 | ✅ 显式 persist | `/api/session/cell/persist` + view Persist;不自动写源码 |
| G7/G8 | 📌 已移出当前目标 | 设计保留,后续版本再开发 |
## 3. 缺口清单
| ID | 缺口 | 优先级 | 当前行为 | 目标 |
|---|---|---|---|---|
| G1 | 持久内核 raw stdin | P0 | 持久内核只有 `marqdo_input` | 程序 stdin 与控制协议分离 |
| G2 | 每 cell 独立 stdin | P0 | 页面级 Cell stdin,Run All 共用一个输入 | cell 级输入表 + UI |
| G3 | 动态 plan | P2 | 只读顶层 `会话` | 明确边界,或受控动态 |
| G4 | 运行时更新 plan | P1 | values 只在打开时初始化 | 显式 session 变更 API |
| G5 | kernel interrupt | P0 | 只能等超时/杀进程组 | 可中断当前 eval |
| G6 | 状态持久化 / replay | P1 | journal/dirty 只在内存 | 导出/恢复/replay |
| G7 | 跨文档内核 | P2 | 每文档独立 | 显式 scope/空间 |
| G8 | 非 Python/Node 持久内核 | P3 | Go 等一次性执行 | LangSpec 会话适配器 |
| G9 | live → static 输入捕获 | P0 | 静态构建不捕获临时 stdin | 显式 snapshot 捕获 |
| G10 | 自动 writeback | P2 | 必须显式调用 | 显式策略下的“运行后写回” |
---
## 4. 统一模型
### 4.1 会话句柄
`foreign.session` 返回的句柄是一个普通 Marqdo 值,建议稳定形状:
```text
{
kind: "foreign-session",
id: "<session-id>",
document: "<entry-source-hash>",
status: "created | ready | running | interrupted | dirty | closed",
kernels: [...],
cells: [...],
inputs: [...],
values: [...],
generation: 1
}
```
`view` 只读这些字段;任何执行操作都通过 `foreign` 的 host API。句柄不是第二套运行时。
### 4.2 三种输入
必须严格区分:
| 输入 | 目标 | 形态 | 当前 |
|---|---|---|---|
| `values` | 内核变量 | Marqdo 值 → JSON copy | 已实现 |
| `input` | `marqdo_input` | JSON 值 | 已实现(低层) |
| `stdin` | 程序 stdin | 原始文本/字节 | 仅无状态 cell 有 |
v3 不允许把 raw stdin 偷偷塞进 `marqdo_input`;两者语义不同。
### 4.3 每 cell 输入表
不新增标记,仍然使用普通表:
```mq.md
`单元输入` =
| 单元 | stdin | input |
|------|-------|-------|
| 计算 | hello | {rate: 0.08} |
| 复核 | | {check: true} |
```
`foreign.session` 增加可选参数:
```text
foreign.session kernels=... cells=... values=... inputs=`单元输入`
```
规则:
- `stdin` 只给该 cell 的程序 stdin;
- `input` 只给该 cell 的 `marqdo_input`;
- 空值表示不提供;
- 输入大小有上限,超限显式报错;
- 静态导出使用同一张表,因此 view 与静态快照可以一致。
### 4.4 执行流水线
```text
evaluate 会话 handle
→ open kernels
→ apply values
→ apply cell inputs
→ run cell
→ record {stdout, stderr, input_echo, result}
→ next cell
```
失败不终止整个 Run All;错误写入 cell record,journal 保留。
---
## 5. G1/G2:持久内核 stdin 与每 cell 输入
### 5.1 问题
当前持久内核的 worker 用 stdin 传输 JSON 控制协议,因此程序自己的 stdin 不可用;`marqdo_input` 只是 JSON 变量,不是 stdin。
### 5.2 方案 A(推荐):控制通道与程序 stdin 分离
- Unix:父进程创建 **socketpair**(或两个 pipe),worker 通过 fd 3/4 读写控制帧;程序 stdin 使用单独的 pipe。
- Windows:named pipe 或可继承的匿名 pipe 作为控制通道;程序 stdin 单独 pipe。
- worker 协议升级为 `protocol: 2`:
- 控制帧:`eval` / `set` / `get` / `interrupt` / `close`
- eval 帧可带 `stdin`(UTF-8)或 `stdin_b64`(二进制,后续)
- Python worker:每次 eval 前把 `sys.stdin` 替换为 `io.TextIOWrapper`/`StringIO`,eval 后恢复控制通道;用户代码读到的是真正的 stdin。
- Node worker:控制 readline 移到控制 pipe;每次 eval 把 `process.stdin` 替换为 `PassThrough`/`Readable`,eval 后恢复。
- 超时/中断关闭该次 stdin,不污染下一次 eval。
### 5.3 方案 B(降级):只支持 `input`
如果控制通道暂不实现,持久内核只支持 `input`,并显式拒绝 raw stdin:
```text
persistent kernel stdin is unavailable; use input= or stateless cells
```
不能静默把 stdin 变成 `marqdo_input`。
### 5.4 方案 C:stdin cell 降级为一次性
持久会话中遇到 raw stdin 时,用一次性进程执行该 cell;明确标记 `state: none`,不假装内核态连续。
### 5.5 UI 与静态
- `view`:每个代码卡提供 **Cell stdin** 小输入框,或者页面级输入表;推荐每 cell,因为它们经常不同。
- 静态:读取 `inputs` 表;没有输入时显示“requires stdin”说明,而不是 traceback。
- Run All 使用每 cell 的输入;不再把一个页面级 stdin 复制给所有 cell。
### 5.6 验收
- Python 与 Node 持久内核都能读取真实 stdin;
- 同一个 cell 连续 eval 不串输入;
- 超时/中断后下一次 eval 仍可读 stdin;
- 无状态 cell 行为不变;
- 静态快照使用相同输入表,结果与 live 一致(在相同输入下)。
---
## 6. G3:动态 plan 的边界
### 6.1 问题
`foreign.session` 目前只识别 entry 顶层的 `会话` 绑定。分支、函数、运行时循环中定义的 plan 不会被 view 发现。
### 6.2 设计决定(建议)
**v3 不支持任意控制流中的 plan。** 这是有意的语义边界:
- plan 是文档的声明,不是程序运行时的副作用;
- view 不应为了找 plan 而执行 `# main`;
- 避免 view 再次长出一套“运行程序再猜计划”的隐式逻辑。
允许的动态形式:
- 顶层赋值右侧的纯表达式,包括引入的纯函数;
- `foreign.session` 的 tables/values 可以由顶层表函数计算;
- 运行时变化通过 G4 的显式 session API 完成。
**不支持**:
- `##` 函数内部的 plan;
- `if`/`while` 分支里的 plan;
- 依赖 `# main` 副作用才能得到的 plan。
### 6.3 诊断
如果源码中出现非顶层 `foreign.session`,给出:
```text
foreign.session: plan must be a top-level binding (会话 = > foreign.session ...)
```
而不是静默降级为无状态。
---
## 7. G4:运行时更新 plan
### 7.1 目标
已打开的 view session 允许用户显式修改 cell 归属、增加/移除 cell、更新内核变量;所有修改都要可追踪。
### 7.2 API
```text
foreign.session_assign session=<handle> cell="计算" kernel="复核"
foreign.session_add_cell session=<handle> cell="新单元" kernel="分析"
foreign.session_remove_cell session=<handle> cell="旧单元"
foreign.session_set_value session=<handle> kernel="分析" name="rate" value=0.08
```
所有操作:
- 校验 cell/kernel 语言;
- 递增 `generation`;
- 写入 journal(kind=plan_change / value_change);
- 标记受影响 cell 为 `dirty`;
- 失败不改变 handle。
### 7.3 View
- 内核下拉复用 `session_assign`;
- 增加/删除 cell 放在显式菜单,不做隐式拖拽写入源码;
- UI 显示 generation / dirty 状态。
### 7.4 静态
静态快照不执行变更;它记录变更后的 plan,或在构建时执行一次并固化。
---
## 8. G5:kernel interrupt
### 8.1 API
```text
foreign.session_interrupt id=<session-id>
```
### 8.2 行为
- 控制通道发送 `interrupt`;
- worker 取消当前 eval,返回 `{ok:false, interrupted:true}`;
- 保留已完成的值和 journal;
- 状态从 `running` → `interrupted`;
- 下一次 eval 可以继续使用内核(除非进程已损坏);
- 超时继续作为最后一道防线。
### 8.3 View
- 工具栏增加 Interrupt;执行中的 cell 显示 interrupted;
- 不自动重跑;
- 静态导出没有 interrupt(快照不执行)。
### 8.4 跨平台
Unix:进程组信号 + 控制帧;Windows:控制帧 + `taskkill /T` fallback。两者都必须有测试。
---
## 9. G6:状态持久化与 replay
### 9.1 不做的事
- 不做 Python pickle / Node 堆快照;
- 不做任意对象反序列化;
- 不把内核内存默认写进 `.mq.md`。
### 9.2 显式快照
```text
foreign.session_export session=<handle> path=".marqdo/sessions/doc-1.json"
foreign.session_import path=".marqdo/sessions/doc-1.json"
```
快照只包含:
```text
{
version: 1,
document_hash: "...",
plan_hash: "...",
kernels: [{name, lang, cmd, values: {...}}],
cells: [{name, kernel, status, last_ok, last_error}],
journal: [...],
inputs: [...]
}
```
值只允许 JSON 可表达的 Marqdo 值;不可序列化的值显式失败。
### 9.3 replay
```text
foreign.session_replay path="..." verify=True
```
按 journal 顺序重放 cell + inputs,比较 stdout/status;差异写入 report。replay 不覆盖原 journal,除非显式 `append`。
### 9.4 存储
- 默认目录 `.marqdo/sessions/`(gitignore 可选);
- 用户可通过 path 放到受控目录;
- 不默认提交,不默认写源码。
---
## 10. G7:跨文档内核(Deferred / future)
> **状态:已移出 v3 当前实现目标。** 本节设计保留,作为后续版本(v4 或独立 session-scope 工作)的输入;当前不承诺 API 或兼容性。
### 10.1 原则
- 默认每文档独立;
- 跨文档必须显式 `scope=` / `space=`;
- 不自动按变量名或路径共享。
### 10.2 API
```text
foreign.session_open lang="python" scope="shared:analytics"
foreign.session_list scope="shared:analytics"
foreign.session_close id=... scope=...
```
- scope 注册表由 foreign runtime 管理;
- 同名冲突报错,不覆盖;
- view 只能打开当前文档声明允许的 scope;
- 生命周期由 owner/refcount 管理,最后一个引用关闭时清理。
### 10.3 安全
- 跨文档共享可能违反文档隔离,必须 capability + 显式配置;
- journal 标记 scope;
- 静态导出默认不打开跨文档 scope。
---
## 11. G8:非 Python/Node 持久内核(Deferred / future)
> **状态:已移出 v3 当前实现目标。** 本节设计保留,等待目标语言与 worker 适配器确定后再实现;当前不承诺 API。
### 11.1 模型
不是所有语言都需要持久内核。`LangSpec` 增加能力标志:
```text
session: none | protocol_v2
```
- `none`:只能 `run` / `invoke` 一次性;
- `protocol_v2`:实现控制通道与 eval 协议,可以 `session_open`。
### 11.2 适配器
Go 等语言通过一个薄 worker 实现:
```text
control frame: eval {code, stdin, input}
result frame: {ok, stdout, stderr, result, error}
```
不允许“看起来像持久内核、实际每次重启”的假实现。
### 11.3 失败语义
- `session_open` 对 `session: none` 的语言明确报错;
- `foreign.session` 的 plan 只能使用支持 session 的语言;
- 静态/无状态仍可使用一次性执行。
---
## 12. G9:live → static 输入捕获
### 12.1 问题
静态构建在构建环境执行,无法知道用户在 live view 中临时输入的 stdin;同一文档的结果可能不同。
### 12.2 设计
增加显式的“捕获快照”:
```text
view → Capture snapshot → .marqdo/snapshots/<doc>.view.json
```
快照内容:
- document source hash;
- session plan(kernels/cells/values);
- 每 cell 的 stdin/input;
- 每 cell 的 stdout/stderr/status;
- journal 摘要、语言/解释器版本;
- 捕获时间。
静态导出:
```text
marqdo view output --snapshot .marqdo/snapshots/doc.view.json
```
优先使用快照中的记录,校验 source hash,不重新执行;若 hash 不匹配,显式失败或 `--execute` 重跑。
### 12.3 安全/隐私
- 快照可能包含 secrets;默认不提交、不发布;
- 提供 `redact` 字段/正则;
- 静态页面只发布脱敏后的快照。
### 12.4 验收
- live 中成功的结果可以在静态页面原样显示;
- 不捕获内核内存,只捕获输入/输出/状态;
- 没有快照时回退到构建时执行,行为明确。
---
## 13. G10:自动 writeback
### 13.1 原则
**“自动”必须是用户显式选择的策略,不是默认副作用。**
### 13.2 API
```text
foreign.session persist=`写回策略`
```
策略表:
```mq.md
`写回策略` =
| 单元 | key |
|------|-----|
| 计算 | total |
| 复核 | check |
```
或 view 的 **Persist results** 按钮:将当前 cell 结果写到 `writeback.record key=...`。
### 13.3 语义
- 默认不写源码;
- 显式策略只在 **用户点击 Persist** 或 `marqdo run --writeback` 时执行;
- 静态构建默认不写回;`--writeback` 必须显式;
- writeback 只支持 JSON 可表达值;
- 写入位置/格式仍由 `writeback` 库决定。
### 13.4 与静态快照的关系
- 静态快照是只读发布;
- writeback 是源码内的活结果;
- 两者可以共存,但不能互相隐式触发。
---
## 14. API / 端点汇总
### 14.1 新增 foreign 函数
```text
foreign.session inputs=
foreign.session_assign
foreign.session_add_cell
foreign.session_remove_cell
foreign.session_set_value
foreign.session_interrupt
foreign.session_export
foreign.session_import
foreign.session_replay
foreign.session_open scope=
foreign.session_list scope=
```
### 14.2 View API
```text
POST /api/session/cell/run { cell, stdin }
POST /api/session/run_all { restart, inputs? }
POST /api/session/cell/clear
POST /api/session/plan/assign
POST /api/session/plan/add
POST /api/session/plan/remove
POST /api/session/plan/value
POST /api/session/interrupt
POST /api/session/snapshot/capture
```
所有端点只操作 foreign 句柄;view 不直接读写内核。
### 14.3 CLI
```text
marqdo view output --snapshot FILE
marqdo view output --execute
marqdo view output --no-exec
marqdo run --writeback
```
不新增 `marqdo notebook`。
---
## 15. 语义矩阵
| 能力 | run | view | static |
|---|---|---|---|
| `foreign.session` plan | 作为普通值执行 | 消费句柄 | 可执行或快照 |
| raw stdin | 支持(一次性/session) | 每 cell | inputs 表/快照 |
| interrupt | 无 UI | 可调用 | 不可用 |
| state export | 可调用 | 可显式导出 | 不导出或快照 |
| cross-doc(deferred) | 设计保留 | 设计保留 | 设计保留 |
| writeback | `--writeback` | 显式按钮 | 默认不写 |
---
## 16. 实施顺序
### P0(先做,直接影响真实使用)
1. **G1/G2 stdin**:控制通道 + 每 cell 输入表 + UI。
2. **G5 interrupt**:避免只能等超时。
3. **G9 snapshot capture**:解决 live/static 不一致。
### P1
4. **G4 runtime plan updates**。
5. **G6 export/replay**。
### P2
6. **G3 dynamic plan 边界**:按第 6 节,只做顶层纯表达式;任意控制流不承诺。
7. **G10 explicit writeback policy**。
### Deferred(不在当前目标)
- **G7 cross-document scope**:设计保留,后续版本再开发。
- **G8 non-Python/Node persistent adapters**:设计保留,等待目标语言与 worker。
---
## 17. 测试设计
### 单元
- `stdin` 协议:空输入、大输入、超时、中断、连续 eval;
- 每 cell 输入隔离;
- 返回值锚定在调用点(已修复的 CallFrame.call_line);
- 表格不产生 call chip(已修复);
- plan top-level 诊断;
- session generation / dirty;
- export/import/replay 校验。
### 集成
- Python 持久内核读 `sys.stdin`;
- Node 持久内核读 `process.stdin`;
- view Run All 使用每 cell 输入;
- interrupt 后下一次 eval 正常;
- static snapshot 与 live 结果一致;
- snapshot hash 不匹配显式失败;
- cross-doc scope 冲突/清理;
- Go 一次性 vs 持久能力诊断;
- writeback 只显式写入。
### 静态
- 输出无 traceback;
- 没有输入时显示说明而非伪造;
- 多内核 controls 显示;
- writeback slot 单块、暗色主题;
- 静态按钮提示。
---
## 18. 兼容与迁移
- 现有 `foreign.session kernels= cells= values=` 保持兼容;
- 新 `inputs=` 可选;
- `session_open` 旧协议继续可用;v2 控制通道通过 handshake 协商;
- 旧页面没有 `inputs` 时仍按空 stdin 执行/显示说明;
- 不修改已发布的 v1.5.0 tag;v3 以新版本发布。
---
## 19. 需要拍板的决策
1. **G1**:是否接受控制通道与程序 stdin 分离的协议改造?否则只能保留 `input`,不能宣称 raw stdin。
2. **G3**:是否接受“plan 必须是顶层声明”的边界?若不接受,需要定义 view 可安全调用的 plan 入口,风险更高。
3. **G6**:状态快照只允许 JSON 值,是否可接受?不做任意对象序列化。
4. **G9**:快照文件是否默认放在 `.marqdo/snapshots/` 且不提交?涉及 secrets。
5. **G10**:自动 writeback 是否只允许显式按钮/flag,而不是每次运行自动写源码?
---
## 20. 当前目标范围
v3 当前实现目标为:**G1、G2、G3、G4、G5、G6、G9、G10**。
**G7、G8 已移出当前目标**,但设计文档保留在本文件第 10、11 节,供后续开发;它们不计入 v3 完成条件,也不影响现有 API 的兼容性承诺。
## 21. 非目标
- 不新增 Markdown 标记;
- 不新增 notebook CLI/模式;
- 不把任意内核内存序列化;
- 不默认修改 `.mq.md`;
- 不自动镜像变量;
- 不把 view 变成第二套解释器;
- 不把“未实现”伪装成支持。
+23 -22
View File
@@ -1,14 +1,14 @@
# 文档会话(Document Session)
# 文档会话(View Cells)
| | |
|---|---|
| 状态 | **D1–D3 首版已实现;D4/D5 待做** |
| 日期 | 2026-10-17 |
| 层 | Runtime + Document(view 服务端 / notebook 交互) |
| 层 | Runtime + Document(view 服务端 / 交互单元) |
| 相关 | [notebook-syntax.md](notebook-syntax.md) · [foreign-kernel.md](foreign-kernel.md) · [foreign-notebook-decisions.md](../research/foreign-notebook-decisions.md) · [view.md](view.md) · [view-debug.md](view-debug.md) · [stdlib-subtask.md](stdlib-subtask.md) |
| 核心表面 Δ | 无(读取既有 AST 与表格) |
> 文档会话是 view 背后的“notebook 运行时”:它记住哪些 cell、在哪个内核、按什么顺序执行、状态是什么、结果是否过期。
> 文档会话是 view 背后的交互单元运行时:它记住哪些 cell、在哪个内核、按什么顺序执行、状态是什么、结果是否过期。它不是第二个 CLI 模式——`marqdo view` 是唯一的交互面,`marqdo run` 是规范执行。
---
@@ -23,11 +23,12 @@
---
## 1.1 实施状态(2026-10-17)
## 1.1 实施状态(2026-10-18)
| 能力 | 状态 | 位置 |
|---|---|---|
| `内核表` / `会话表` 解析(中英列名) | ✅ | `src/notebook/mod.rs` |
| `foreign.session` 句柄(kernels/cells 表参数) | ✅ | `lib/foreign.mq.md` · `src/host/foreign_doc.rs` |
| cell plan 求值 | ✅ 解释器求值 `会话` 绑定;不扫描源码行、不认保留表名 | `src/interp.rs` · `src/host/foreign_doc.rs` |
| cell index(绑定名 / 行号 / source hash) | ✅ | 同上 |
| 命名内核 + 默认内核 + 语言校验 | ✅ | 同上 |
| 同内核共享 / 跨内核隔离 | ✅ | Node 金样例 + 单测 |
@@ -39,8 +40,8 @@
| view 工具栏 / 状态徽标 / 内核下拉 / 状态表单 | ✅ | `src/view/html.rs` |
| `POST /api/session/*` | ✅(含 `reload`) | `src/view/notebook_api.rs` |
| cell step | ✅ | 同上 |
| Marqdo statement step | ✅(notebook 工具栏 Debug → 同一 view 服务的 `/debug`;`/api/debug/*`) | `src/debug` · `src/view/debug_api.rs` |
| dirty/stale 自动检测 | 部分(reload → dirty;assign → dirty;stale 传播待做) | `src/notebook/mod.rs` |
| Marqdo statement step | ✅(view 工具栏 Debug → 同一 view 服务的 `/debug`;`/api/debug/*`) | `src/debug` · `src/view/debug_api.rs` |
| dirty/stale 自动检测 | 部分(reload → dirty;assign → dirty;stale 传播待做) | `src/host/foreign_doc.rs` |
| writeback 集成 | 待做 | `writeback` |
| replay / 持久化 | 待做 | — |
| Python 会话 | ✅(本机无 python,CI/用户机条件测试) | `src/host/foreign_session.rs` |
@@ -50,18 +51,18 @@
---
## 2. 两种执行模式
## 2. 两个执行面(不是两个 CLI 模式)
### 2.1 Program mode(规范)
### 2.1 Program execution(规范)
- 入口:`marqdo run doc.mq.md` / CI / `view` 的 `Run document`。
- 执行:`# main` 程序语义;外联通过显式 `foreign.session_open/eval/call` 编排。
- 状态:一次运行内有效;运行结束进程退出(除非显式持久内核)。
- 结果:可复现;是文档的“真理”。
### 2.2 Notebook mode(交互)
### 2.2 View cells(交互)
- 入口:`marqdo view doc.mq.md` → Session 面板。
- 入口:`marqdo view doc.mq.md` → 文档单元(cell)控制。
- 执行:按 `会话表`(或自动顺序)逐 cell 执行。
- 状态:内核持久;可切换、重启、单步。
- 结果:默认临时显示;可显式 writeback。
@@ -72,7 +73,7 @@
```text
.mq.md
/ \
Program mode Notebook mode
Program run View cells
(canonical) (interactive)
\ /
Document Session
@@ -83,8 +84,8 @@
```
- 共享 cell index 与 kernel registry。
- Program mode 可以不使用 notebook 表;Notebook mode 可以使用。
- 冲突时以 Program mode 为准;Notebook mode 不改变文档语义。
- `run` 不消费视图会话;`view` 只在文档绑定 `会话 = > foreign.session ...` 时提供持久 cell 控制;无 `会话` 时 cells 无状态一次性执行。
- 冲突时以 `run` 的程序语义为准;view cells 不改变文档语义。
---
@@ -107,7 +108,7 @@ DocumentSession
├── value_store: HashMap<String, Value> // Marqdo 侧会话值
├── journal: Vec<ExecutionRecord>
├── created_ms / last_active_ms
└── mode: program | notebook
└── surface: program | view
```
### 3.1 Cell
@@ -130,7 +131,7 @@ DocumentSession
1. 读取/解析文档(现有 `parse_source` / `load_module`)。
2. 遍历 AST,找 `Stmt::Assign { name, value: Expr::Code(c) }`。
3. 记录 name、lang、source、span、source hash。
4. 读取 `内核表` / `会话表` 绑定(见 [notebook-syntax.md](notebook-syntax.md))。
4. 通过解释器求值 `会话` 绑定(`foreign.session` 返回的句柄,含简单依赖);不扫描源码行、不认保留表名(见 [notebook-syntax.md](notebook-syntax.md))。
5. 解析 kernel 声明与 cell 归属;校验;生成诊断。
6. 生成执行顺序:
- 有会话表 → 行顺序;
@@ -233,7 +234,7 @@ resolve cell -> resolve kernel
| **Marqdo statement step** | 执行下一条 Marqdo 语句;遇到 `foreign.session_eval` 时整个外联调用原子完成 |
| **Foreign line step** | 不做;未来 DAP |
- Cell step 是 Notebook mode 的主单步。
- Cell step 是 view cells 的主单步。
- Statement step 复用 `DebugController`;需要把 debug session 与 DocumentSession 关联。
- 两者可以互相切换:cell step 到某个 Marqdo cell 后可 statement step;语句执行完回到 cell 边界。
@@ -333,7 +334,7 @@ resolve cell -> resolve kernel
### 7.1 默认
- Notebook mode 输出只在 view 中显示;不写 `.mq.md`。
- View cells 输出只在 view 中显示;不写 `.mq.md`。
- 关闭/刷新后可通过 journal 恢复最近结果(若 session 持久化)。
- 不写 execution_count 到文档。
@@ -463,7 +464,7 @@ Kernel: [分析 ▾] Status: ready [Restart] [Interrupt]
### 12.1 服务端
- cell index:绑定名/行号/lang/source hash;未绑定围栏排除。
- 表解析:内核表/会话表/自动模式/重复/未知/语言不匹配。
- 会话解析:foreign.session 句柄/自动模式/重复/未知/语言不匹配。
- 会话:open/close/reload;kernel 启动/重启/中断。
- cell run:状态机 never→running→clean/error;dirty/stale。
- step:cell step 顺序;statement step 与 debug 联动。
@@ -474,7 +475,7 @@ Kernel: [分析 ▾] Status: ready [Restart] [Interrupt]
### 12.2 金样例
- `tests/notebook/kernels.mq.md`:内核表/会话表。
- `tests/notebook/kernels.mq.md`:foreign.session 句柄。
- `tests/notebook/shared-state.mq.md`:两个 Python cell 共享 `x`。
- `tests/notebook/step.mq.md`:cell step 顺序。
- `tests/notebook/state-bridge.mq.md`:Marqdo ↔ Python 推拉。
@@ -495,7 +496,7 @@ Kernel: [分析 ▾] Status: ready [Restart] [Interrupt]
| 源 | `.ipynb` | `.mq.md` |
| 执行顺序 | 隐藏的 `In[n]` | 会话表 / 文档顺序 / journal |
| 状态 | 隐藏 kernel | kernel + value store + 状态面板 |
| 多语言 | 一 notebook 一 kernel | 命名内核 + 每 cell 归属 |
| 多语言 | 一文档一 kernel | 命名内核 + 每 cell 归属 |
| 单步 | cell | cell + statement |
| 复现 | 手动 | Restart & Run All + journal + replay |
| Git | 输出/计数噪声 | 源码干净;输出显式 |
@@ -509,7 +510,7 @@ Kernel: [分析 ▾] Status: ready [Restart] [Interrupt]
| 阶段 | 内容 |
|---|---|
| D0 | 设计/语法/决策文档(本文件) |
| D1 | `src/notebook/`:cell index、表解析、session 结构、诊断 |
| D1 | `src/host/foreign_doc.rs`:cell index、foreign.session 句柄、session 结构、诊断 |
| D2 | 服务端 API:open/close/cell run/step/run_all/restart/interrupt |
| D3 | view UI:工具栏、cell 徽标、kernel 下拉、状态面板、journal |
| D4 | statement step 接入;dirty/stale;writeback |
+10 -10
View File
@@ -1,4 +1,4 @@
# 文档原生 Notebook 语法
# 文档原生 view cells 语法
| | |
|---|---|
@@ -110,7 +110,7 @@ Marqdo 的选择:
````markdown
# main
这份文档演示一个最小 notebook:两个 Python 单元共享 `分析` 内核,
这份文档演示一个最小 view cells 文档:两个 Python 单元共享 `分析` 内核,
一个绘图单元在隔离的 `绘图` 内核里运行。
`内核表` =
@@ -211,7 +211,7 @@ print(x + 1)
## 5. 可读性规则(语法评审标准)
新增/修改任何 notebook 语法,必须满足:
新增/修改任何 cell 表约定,必须满足:
1. **先读人话**:去掉执行语义后,表格仍像目录/清单。
2. **不使用属性串**:不写 `kernel=…`、`class=…`、JSON/YAML 元数据。
@@ -231,7 +231,7 @@ print(x + 1)
| 情况 | 诊断(中文示例) | 修复建议 |
|---|---|---|
| 内核名重复 | `内核表: 内核名「分析」重复(第 12、20 行)` | 改名或合并 |
| 会话引用未知内核 | `会话表: 单元「汇总」引用了未声明的内核「分析」` | 在 `内核表` 声明,或改用语言 id |
| 会话引用未知内核 | `foreign.session: cell「汇总」引用了未声明的内核「分析」` | 在 kernels 表声明,或改用语言 id |
| 会话引用未知单元 | `会话表: 找不到名为「清洗」的代码单元` | 检查 `` `清洗` = `` 绑定 |
| 单元重复 | `会话表: 单元「汇总」重复出现` | 每个单元只列一次 |
| 语言不匹配 | `会话表: 单元「前端」(node) 不能使用 python 内核「分析」` | 换内核或换语言 |
@@ -243,15 +243,15 @@ print(x + 1)
---
## 7. 与 Program mode 的关系
## 7. 与 program execution 的关系
- `内核表` / `会话表` 是**声明**,不是执行指令。
- `内核表` / `会话表` 只是传递给 `foreign.session` 的普通表变量,不是保留名。
- `marqdo run` 仍然执行 `# main` 的程序语义;这两个表会被求值为普通数据。
- Program mode 的外联编排用显式 API(`foreign.session_open` / `session_eval` / `call`);表只服务 view 的 notebook 会话与一键 Run All。
- 两种模式可以并存:
- `run` 的外联编排用显式 API(`foreign.session_open` / `session_eval` / `call`);`foreign.session` 句柄服务 `view` 的 cells 与一键 Run All。
- 两者可以并存:
- 文档用表描述“交互时怎么跑”;
- `# main` 用显式调用描述“CI 时怎么跑”。
- 如果两者冲突,以 Program mode 的程序语义为准;Notebook mode 的表只影响 view。
- 如果两者冲突,以 `run` 的程序语义为准;cell 表只影响 `view`。
这样避免把“执行计划”变成第二套程序语义。
@@ -295,7 +295,7 @@ v2 **不**为状态交换新增表格/注释。原因:
| 逻辑 | 位置 |
|---|---|
| 识别 `内核表` / `会话表` 绑定 | `src/notebook/syntax.rs`(新) |
| 识别 `会话` 的 foreign.session 句柄 | `src/host/foreign_doc.rs` |
| 读取绑定值 | AST `Stmt::Assign` + `Expr::Table` / `Expr::List`;或最近一次 run 的 bindings |
| 校验名称/语言/重复/未入会话 | 同上;错误进入现有 Diagnostic |
| cell index(绑定名/行号/语言/source hash) | 解析后的 `Module` 遍历 |
+3 -3
View File
@@ -54,8 +54,8 @@
### 结论 C:Hybrid 执行模型
- **Program mode(规范模式)**:`marqdo run` / CI / 可复现执行。文档是程序;外联通过显式 `foreign.session_*` / `call` 编排;结果可重复。
- **Notebook mode(交互模式)**:`view` 中按 cell 执行、单步、切换内核、查看状态。适合探索,不默认写入文档。
- 两者共享同一个内核运行时与同一套 cell 索引;Notebook mode 是 Program mode 的**交互镜头**,不是替代品。
- **View cells(交互面)**:`view` 中按 cell 执行、单步、切换内核、查看状态。适合探索,不默认写入文档;不是第二个 CLI 模式。
- 两者共享同一个内核运行时与同一套 cell 索引;view cells 是 program execution 的**交互镜头**,不是替代品。
理由:如果让 cell 执行取代文档程序语义,Marqdo 就退化成 Jupyter;如果只有 Program mode,就失去 Jupyter 的即时反馈。Hybrid 同时保留两者。
@@ -99,7 +99,7 @@
### 结论 H:输出默认临时,writeback 显式;状态不写进 `.mq.md`
- Notebook mode 的输出默认只在 view 中显示,不修改文档。
- View cells 的输出默认只在 view 中显示,不修改文档。
- 需要持久化时显式 writeback(沿用 `<!-- marqdo-out … -->`),可 Git 跟踪、可忽略。
- 执行日志放 `.marqdo/sessions/…`(默认 git 忽略)。
- 不把 execution_count / kernel 状态 / base64 输出写进 `.mq.md`。
+1 -1
View File
@@ -1 +1 @@
1.5.0
1.5.2
+32 -2
View File
@@ -1,6 +1,7 @@
---
title: lib/foreign
description: Run bound code fences and inline sources via local interpreters
import table:lib/table.mq.md
---
Run bound code fences and inline sources via local interpreters.
@@ -148,6 +149,25 @@ in input order. Raises with the failing index after all workers finish.
*> host_foreign_call_many inputs=`inputs` code=`code` lang=`lang` source=`source` cmd=`cmd` env=`env` cwd=`cwd` timeout=`timeout` args=`args` max_output=`max_output` concurrency=`concurrency`*
## session
+ `kernels`=None
+ `cells`=None
+ `values`=None
+ `inputs`=None
Declare a document view session. `kernels` and `cells` are ordinary tables:
kernels are `|名称|语言|命令|` (ZH or EN), cells are `|单元|内核|`.
Returns a foreign session handle consumed by `view`; without this handle the
view has no persistent kernel state.
**p = > table.empty_map**
**p = > table.put in=`p` at="kind" value="foreign-session"**
**p = > table.put in=`p` at="kernels" value=`kernels`**
**p = > table.put in=`p` at="cells" value=`cells`**
**p = > table.put in=`p` at="values" value=`values`**
**p = > table.put in=`p` at="inputs" value=`inputs`**
*p*
## session_open
+ `lang`
+ `cmd`=None
@@ -165,12 +185,14 @@ default per-eval timeout.
+ `code`=None
+ `source`=None
+ `input`=None
+ `stdin`=None
+ `timeout`=None
Evaluate code (a code value) or source text in session id; returns a record
with id/alive. `input` is JSON-encoded for the kernel.
with id/alive. `input` is JSON-encoded for the kernel; `stdin` is raw program
stdin for this evaluation.
*> host_foreign_session_eval id=`id` code=`code` source=`source` input=`input` timeout=`timeout`*
*> host_foreign_session_eval id=`id` code=`code` source=`source` input=`input` stdin=`stdin` timeout=`timeout`*
## session_set
+ `id`
@@ -203,6 +225,14 @@ Kill and remove a session; returns whether it existed.
*> host_foreign_session_close id=`id`*
## session_interrupt
+ `id`
Interrupt the current evaluation of a persistent kernel. The session remains
usable for the next evaluation when the worker supports interruption.
*> host_foreign_session_interrupt id=`id`*
## session_list
List live sessions with id/lang/cmd/evals/alive/started_ms.
+27 -1
View File
@@ -1,6 +1,7 @@
---
title: lib/外联
description: 通过本机解释器运行绑定的代码围栏与内联源码
import table:lib/table.mq.md
---
通过本机解释器运行绑定的代码围栏与内联源码。
@@ -146,6 +147,23 @@ JSON 进 / JSON 出:把 `输入` 编码到 stdin,解析 stdout JSON 返回
*> host_foreign_call_many inputs=`输入表` code=`代码` lang=`语言` source=`源码` cmd=`命令` env=`环境` cwd=`工作目录` timeout=`超时` args=`参数` max_output=`最大输出` concurrency=`并发`*
## 会话
+ `内核表`=None
+ `单元表`=None
+ `值表`=None
+ `输入表`=None
声明文档视图会话。内核表和单元表都是普通表格;返回一个 foreign 会话句柄,
供 view 消费。没有这个句柄时,view 的 cells 是无状态的。
**p = > table.empty_map**
**p = > table.put in=`p` at="kind" value="foreign-session"**
**p = > table.put in=`p` at="kernels" value=`内核表`**
**p = > table.put in=`p` at="cells" value=`单元表`**
**p = > table.put in=`p` at="values" value=`值表`**
**p = > table.put in=`p` at="inputs" value=`输入表`**
*p*
## 会话开启
+ `语言`
+ `命令`=None
@@ -162,12 +180,13 @@ JSON 进 / JSON 出:把 `输入` 编码到 stdin,解析 stdout JSON 返回
+ `代码`=None
+ `源码`=None
+ `输入`=None
+ `标准输入`=None
+ `超时`=None
在会话中求值代码(code 值)或源码文本;返回带 id/alive 的记录。
`输入` 会编码为 JSON 交给内核。
*> host_foreign_session_eval id=`标识` code=`代码` source=`源码` input=`输入` timeout=`超时`*
*> host_foreign_session_eval id=`标识` code=`代码` source=`源码` input=`输入` stdin=`标准输入` timeout=`超时`*
## 会话设值
+ `标识`
@@ -186,6 +205,13 @@ JSON 进 / JSON 出:把 `输入` 编码到 stdin,解析 stdout JSON 返回
*> host_foreign_session_get id=`标识` name=`名称`*
## 会话中断
+ `标识`
中断持久内核当前求值;worker 支持时,会话仍可用于下一次求值。
*> host_foreign_session_interrupt id=`标识`*
## 会话变量
+ `标识`
+7 -5
View File
@@ -57,28 +57,30 @@ Foreign v2 is a good example. Probe Python, then run the bound fence:
2. *
> print text="(python3 not found — install Python to run this cell)"
The same module also supports JSON calls, parallel fan-out, persistent kernels and an explicit state bridge. See [stdlib/foreign.mq.md](stdlib/foreign.mq.md) and the full Python walkthrough in [features/19-foreign-notebook.mq.md](features/19-foreign-notebook.mq.md).
The same module also supports JSON calls, parallel fan-out, persistent kernels and an explicit state bridge. See [stdlib/foreign.mq.md](stdlib/foreign.mq.md) and the full Python walkthrough in [features/19-foreign-view-cells.mq.md](features/19-foreign-view-cells.mq.md).
A notebook is still a document. Named kernels and a cell order are ordinary bound tables:
A view document is still a document. Named kernels and a cell order are ordinary bound tables:
```text
`内核表` =
`分析内核` =
| 名称 | 语言 | 说明 |
|------|------|------|
| 分析 | python | shared state |
| 复核 | python | isolated state |
`会话表` =
`分析单元` =
| 单元 | 内核 | 说明 |
|------|------|------|
| 准备数据 | 分析 | write |
| 计算指标 | 分析 | read |
| 独立复核 | 复核 | validate |
`会话` = > foreign.session kernels=`分析内核` cells=`分析单元`
```
Live view turns that into Run All, Step, per-cell kernels, a journal and a state panel. Program mode still runs the file as one program.
`marqdo view` consumes the foreign-owned `会话` handle and turns it into Run All, Step, per-cell kernels, a journal and a state panel. Without `会话`, cells are stateless one-shot runs. There is no separate interactive mode or command; `marqdo run` still executes the file as one program.
Documentation becomes knowledge when it can be discovered, verified and reused. OKF supplies the portable Markdown + YAML shape; EKC projects the repository into catalog layers L0–L4, a graph and an engineering preflight. The source `.mq.md` remains the only authority; generated artifacts are marked and rebuildable.
+7 -5
View File
@@ -57,28 +57,30 @@ print("hello-from-python")
2. *
> 打印 内容="(未找到 python3 — 安装 Python 后可运行此单元)"
同一个模块还支持 JSON 调用、并行扇出、持久内核与显式状态桥。见 [stdlib/外联.mq.md](stdlib/外联.mq.md) 与完整 Python 示例 [features/19-外联Notebook.mq.md](features/19-外联Notebook.mq.md)。
同一个模块还支持 JSON 调用、并行扇出、持久内核与显式状态桥。见 [stdlib/外联.mq.md](stdlib/外联.mq.md) 与完整 Python 示例 [features/19-外联视图单元.mq.md](features/19-外联视图单元.mq.md)。
Notebook 仍然是一篇文档。命名内核与单元顺序就是普通绑定表:
view 仍然是一篇文档。命名内核与单元顺序就是普通绑定表:
```text
`内核表` =
`分析内核` =
| 名称 | 语言 | 说明 |
|------|------|------|
| 分析 | python | 共享状态 |
| 复核 | python | 隔离状态 |
`会话表` =
`分析单元` =
| 单元 | 内核 | 说明 |
|------|------|------|
| 准备数据 | 分析 | 写入 |
| 计算指标 | 分析 | 读取 |
| 独立复核 | 复核 | 校验 |
`会话` = > foreign.session kernels=`分析内核` cells=`分析单元`
```
live view 会把它变成 Run All、Step、每单元内核、journal 与状态面板;Program 模式仍然把文件当单个程序运行。
`marqdo view` 消费 foreign 拥有的 `会话` 句柄,变成 Run All、Step、每单元内核、journal 与状态面板。没有 `会话` 时,单元是无状态的一次性运行。没有单独的交互模式或命令;`marqdo run` 仍然把文件当单个程序运行。
文档能被发现、验证和复用时,才成为知识。OKF 提供可移植的 Markdown + YAML 形态;EKC 把仓库投影成 catalog L0–L4、图与工程 preflight。源 `.mq.md` 始终是唯一权威,生成物有标记、可重建。
+2 -1
View File
@@ -11,8 +11,9 @@
| [`keywords/`](keywords/) | 关键字与 L0 内置 |
| [`stdlib/`](stdlib/) | **标准库**(导入方式 + text/表/文件/时间/系统/json/网络/数学/外联/插件/**自写回**/**子任务**;v0.1.2+ 内置于 CLI) |
| [`features/`](features/) | 特性迭代(含对象、官方 `ext/`) |
| [`features/19-foreign-notebook.mq.md`](features/19-foreign-notebook.mq.md) / [`19-外联Notebook.mq.md`](features/19-外联Notebook.mq.md) | 外联 v2 完整 Python 示例:probe / JSON call / 并行 / 持久内核 / 状态桥 / notebook |
| [`features/19-foreign-view-cells.mq.md`](features/19-foreign-view-cells.mq.md) / [`19-外联视图单元.mq.md`](features/19-外联视图单元.mq.md) | 外联 v2 完整 Python 示例:probe / JSON call / 并行 / 持久内核 / 状态桥 / view 单元 |
| [`features/20-okf-ekc.mq.md`](features/20-okf-ekc.mq.md) / [`20-OKF与EKC.mq.md`](features/20-OKF与EKC.mq.md) | 代码即文档、文档即知识:OKF 形态与 EKC 工程知识编译器 |
| [`features/21-persistence.mq.md`](features/21-persistence.mq.md) / [`21-持久化.mq.md`](features/21-持久化.mq.md) | 外联结果持久化:静态快照与 writeback |
官方扩展(非 stdlib):`ext/llm`、`ext/agent`、`ext/web`、`ext/quantum`、`ext/linalg` — 见 `features/05-*.mq.md`、`06-*.mq.md`、`07-linalg.mq.md` / `07-线性代数.mq.md` 与 `doc/design/ext-*.md`。
-183
View File
@@ -1,183 +0,0 @@
---
title: Foreign kernels v2 — a Python analysis notebook
description: A complete Python data walkthrough with probe, JSON calls, parallel fan-out, persistent state, an explicit bridge and document-native notebook cells
import foreign:lib/foreign.mq.md
---
# main
This page is both a readable document and a runnable program. It uses the same file to show the foreign-kernel features added in v2: language probing, structured records, JSON in/out, parallel fan-out, a persistent Python kernel, and an explicit state bridge.
The data is intentionally small and deterministic, so every number below can be reproduced by running the page.
---
The input.
`orders` =
| @ | region | amount |
|---|--------|--------|
| 1 | 华东 | 120 |
| 2 | 华东 | 300 |
| 3 | 华南 | 90 |
| 4 | 华南 | 140 |
| 5 | 华北 | 220 |
| 6 | 西部 | 80 |
---
Probe Python before using it.
**python = > foreign.probe lang="python"**
> print text=[ok](`python`)
> print text=[version](`python`)
The probe record also carries `cmd`, `path` and `error`, so a site can explain a missing interpreter instead of producing a blank cell.
---
A JSON call. Python reads the table as JSON, groups it, and writes one JSON result.
`analyze` =
```python
import json
import statistics
import sys
rows = json.load(sys.stdin)
by_region = {}
for row in rows:
by_region.setdefault(row["region"], []).append(float(row["amount"]))
summary = []
for region, values in sorted(by_region.items()):
mean = statistics.mean(values)
sd = statistics.pstdev(values) or 1.0
summary.append({
"region": region,
"orders": len(values),
"total": round(sum(values), 2),
"mean": round(mean, 2),
"z_span": round((max(values) - mean) / sd, 3),
})
print(json.dumps({
"regions": summary,
"total": round(sum(float(row["amount"]) for row in rows), 2),
"count": len(rows),
}, ensure_ascii=False))
```
**result = > foreign.call code=`analyze` input=`orders`**
> print text=[count](`result`)
> print text=[total](`result`)
> print text=[region]([1]([regions](`result`)))
> print text=[mean]([1]([regions](`result`)))
A call is JSON-in / JSON-out: the table is encoded on stdin, and the final stdout value is parsed into Marqdo values.
---
The same work as a structured record. `invoke` keeps stdout, stderr, exit code and timing even when the program fails.
**record = > foreign.invoke code=`analyze` input=`orders` output="json"**
> print text=[ok](`record`)
> print text=[code](`record`)
> print text=[ms](`record`)
Here `result` is already a parsed value, while `stdout` remains available for logging.
---
Persistent Python state. The first evaluation imports a module and creates a list; the second evaluation sees both.
**session = > foreign.session_open lang="python"**
**first = > foreign.session_eval id=`session` source="import math\nvalues = [3, 4, 5]"**
**second = > foreign.session_eval id=`session` source="round(math.sqrt(sum(value * value for value in values)), 3)"**
> print text=[result](`second`)
The kernel is strict and serial: evaluations do not interleave, and a timeout kills the process group.
---
The explicit state bridge. Values cross as JSON copies, never as shared object references.
**> foreign.session_set id=`session` name="threshold" value=2.5**
**threshold = > foreign.session_get id=`session` name="threshold"**
> print text=`threshold`
**> foreign.session_close id=`session`**
No implicit mirroring: if a variable is not pushed or pulled, the other side does not see it.
---
Parallel fan-out. Every input gets its own process; failures are isolated and order is preserved.
`increment` =
```python
import json
import sys
value = json.load(sys.stdin)
print(json.dumps(value + 1))
```
`numbers` =
| value |
|-------|
| 1 |
| 2 |
| 3 |
| 4 |
**bumped = > foreign.call_many code=`increment` inputs=`numbers` concurrency=4**
> print text=`bumped`
---
A document-native notebook. Named kernels and a session table are ordinary bound data tables, not fence metadata or YAML config.
```text
`内核表` =
| 名称 | 语言 | 命令 | 说明 |
|------|------|------|------|
| 分析 | python | | shared analysis |
| 复核 | python | | isolated review |
`会话表` =
| 单元 | 内核 | 说明 |
|------|------|------|
| 准备数据 | 分析 | generate orders |
| 计算指标 | 分析 | regions and totals |
| 独立复核 | 复核 | read the JSON report |
```
Live `view` renders these as cells with Run All, Step, per-cell kernel switching, a journal, dirty reload and a state panel. The same file still runs as a single program.
---
Why this stays readable.
| Layer | What the reader sees |
|---|---|
| Markdown | headings, prose, tables and fenced code |
| Program | assignments, calls and return values |
| Knowledge | frontmatter, links, task pages and generated catalog entries |
The foreign layer does not add syntax to the language. A fence is still a fence; a table is still a table.
For the lower-level API and safety defaults, read [`../../lib/foreign.mq.md`](../../lib/foreign.mq.md) and [`../../doc/design/foreign-kernel.md`](../../doc/design/foreign-kernel.md).
+152
View File
@@ -0,0 +1,152 @@
---
title: Foreign kernels v2 — Python analysis in view
description: Kernel state, Marqdo-to-kernel values and view cells — without a stdin-only demo
import foreign:lib/foreign.mq.md
---
# main
This page shows the foreign runtime as a language feature: a persistent kernel, explicit state from Marqdo values, and view cells that share kernel state. It never asks the reader to type JSON into stdin.
---
Probe the interpreter first. The result is ordinary Marqdo data.
**python = > foreign.probe lang="python"**
> print text=[ok](`python`)
> print text=[version](`python`)
---
Marqdo values into a kernel.
The `订单` table is a normal Marqdo value. `foreign.session_set` copies it into the Python kernel as JSON; no shell and no stdin.
`订单` =
| @ | 区域 | 金额 |
|---|------|------|
| 1 | 华东 | 120 |
| 2 | 华东 | 300 |
| 3 | 华南 | 90 |
| 4 | 华南 | 140 |
| 5 | 华北 | 220 |
| 6 | 西部 | 80 |
**内核会话 = > foreign.session_open lang="python"**
**> foreign.session_set id=`内核会话` name="orders" value=`订单`**
`total_code` =
```python
round(sum(float(row["金额"]) for row in orders), 2)
```
**total = > foreign.session_eval id=`内核会话` code=`total_code`**
> print text=[result](`total`)
The low-level kernel session is managed entirely by `foreign`; the table crossed the boundary as an explicit JSON copy.
---
Kernel state persists.
Two evaluations share the same kernel. The second sees the first variable.
`set_values_code` =
```python
values = [3, 4, 5]
```
**> foreign.session_eval id=`内核会话` code=`set_values_code`**
`square_sum_code` =
```python
sum(value * value for value in values)
```
**square_sum = > foreign.session_eval id=`内核会话` code=`square_sum_code`**
> print text=[result](`square_sum`)
**> foreign.session_close id=`内核会话`**
This is kernel state, not a one-shot process. It is created, queried and closed through `foreign`.
---
View cells share state.
`foreign.session` declares a view session from ordinary tables. The table names are free; this document uses `分析内核` and `分析单元`.
`分析内核` =
| @ | 名称 | 语言 |
|---|------|------|
| 1 | 分析 | python |
| 2 | 复核 | python |
`分析单元` =
| @ | 单元 | 内核 |
|---|------|------|
| 1 | 计算 | 分析 |
| 2 | 复核 | 复核 |
`初始值` =
| 内核 | 名称 | 值 |
|------|------|----|
| 分析 | orders | `订单` |
| 复核 | orders | `订单` |
`会话` = > foreign.session kernels=`分析内核` cells=`分析单元` values=`初始值`
`计算` =
```python
import json
import statistics
by_region = {}
for row in orders:
by_region.setdefault(row["区域"], []).append(float(row["金额"]))
summary = []
for region, values in sorted(by_region.items()):
mean = statistics.mean(values)
sd = statistics.pstdev(values) or 1.0
summary.append({
"region": region,
"orders": len(values),
"total": round(sum(values), 2),
"mean": round(mean, 2),
"z_span": round((max(values) - mean) / sd, 3),
})
print(json.dumps({
"regions": summary,
"total": round(sum(float(row["金额"]) for row in orders), 2),
"count": len(orders),
}, ensure_ascii=False))
```
`复核` =
```python
import json
print(json.dumps({
"kernel": "复核",
"count": len(orders),
"total": round(sum(float(row["金额"]) for row in orders), 2),
}, ensure_ascii=False))
```
`查看` consumes the `会话` handle. Run All runs `计算` in the `分析` kernel and `复核` in the `复核` kernel; the static export runs the same cells and embeds their real outputs.
The important language point:
The `初始值` table passes the same Marqdo `订单` value into both kernels. Kernel state lives in `foreign`. View only renders the handle and its cells. `run` still executes the same document as a program.
There is no separate notebook CLI, no stdin requirement in this example, and no hidden language switch.
-183
View File
@@ -1,183 +0,0 @@
---
title: 外联内核 v2 — Python 分析 Notebook
description: 从 probe、JSON 调用、并行扇出、持久状态到显式桥与文档原生 notebook 单元的完整 Python 示例
导入 外联:lib/外联.mq.md
---
# main
这一页既是可读文档,也是可运行程序。它用同一份文件展示 v2 的外联内核能力:语言探测、结构化记录、JSON 进出、并行扇出、持久 Python 内核,以及显式状态桥。
数据故意保持小而确定,因此下面每个数字都可以通过运行本页复现。
---
输入数据。
`订单` =
| @ | 区域 | 金额 |
|---|------|------|
| 1 | 华东 | 120 |
| 2 | 华东 | 300 |
| 3 | 华南 | 90 |
| 4 | 华南 | 140 |
| 5 | 华北 | 220 |
| 6 | 西部 | 80 |
---
先探测 Python。
**python = > 外联.探测 语言="python"**
> 打印 内容=[ok](`python`)
> 打印 内容=[version](`python`)
探测记录还带 `cmd`、`path` 与 `error`,所以站点可以解释“解释器缺失”,而不是显示一个空白单元。
---
JSON 调用。Python 把表读成 JSON,按区域聚合,再写出一个 JSON 结果。
`分析` =
```python
import json
import statistics
import sys
rows = json.load(sys.stdin)
by_region = {}
for row in rows:
by_region.setdefault(row["区域"], []).append(float(row["金额"]))
summary = []
for region, values in sorted(by_region.items()):
mean = statistics.mean(values)
sd = statistics.pstdev(values) or 1.0
summary.append({
"region": region,
"orders": len(values),
"total": round(sum(values), 2),
"mean": round(mean, 2),
"z_span": round((max(values) - mean) / sd, 3),
})
print(json.dumps({
"regions": summary,
"total": round(sum(float(row["金额"]) for row in rows), 2),
"count": len(rows),
}, ensure_ascii=False))
```
**结果 = > 外联.调用 代码=`分析` 输入=`订单`**
> 打印 内容=[count](`结果`)
> 打印 内容=[total](`结果`)
> 打印 内容=[region]([1]([regions](`结果`)))
> 打印 内容=[mean]([1]([regions](`结果`)))
调用是 JSON 进 / JSON 出:表编码到 stdin,stdout 的最后一个值再解析成 Marqdo 值。
---
同样的工作也可以保留成结构化记录。`执行` 即使进程失败也保留 stdout、stderr、退出码与耗时。
**记录 = > 外联.执行 代码=`分析` 输入=`订单` 输出="json"**
> 打印 内容=[ok](`记录`)
> 打印 内容=[code](`记录`)
> 打印 内容=[ms](`记录`)
这里 `result` 已经是解析后的值,同时 `stdout` 仍可用于日志。
---
持久 Python 状态。第一次求值导入模块并创建列表;第二次求值仍能看到两者。
**会话 = > 外联.会话开启 语言="python"**
**第一次 = > 外联.会话求值 标识=`会话` 源码="import math\nvalues = [3, 4, 5]"**
**第二次 = > 外联.会话求值 标识=`会话` 源码="round(math.sqrt(sum(value * value for value in values)), 3)"**
> 打印 内容=[result](`第二次`)
内核严格串行:求值不会交错,超时会杀掉进程组。
---
显式状态桥。值以 JSON 拷贝跨越边界,永远不是共享对象引用。
**> 外联.会话设值 标识=`会话` 名称="threshold" 值=2.5**
**阈值 = > 外联.会话取值 标识=`会话` 名称="threshold"**
> 打印 内容=`阈值`
**> 外联.会话关闭 标识=`会话`**
没有隐式镜像:没有 push / pull 的变量,另一边看不到。
---
并行扇出。每个输入都有独立进程;失败互相隔离,顺序保持不变。
`加一` =
```python
import json
import sys
value = json.load(sys.stdin)
print(json.dumps(value + 1))
```
`数值` =
| value |
|-------|
| 1 |
| 2 |
| 3 |
| 4 |
**加一后 = > 外联.批量调用 代码=`加一` 输入表=`数值` 并发=4**
> 打印 内容=`加一后`
---
文档原生 Notebook。命名内核和会话表都是普通绑定数据表,不是 fence 元数据或 YAML 配置。
```text
`内核表` =
| 名称 | 语言 | 命令 | 说明 |
|------|------|------|------|
| 分析 | python | | 共享分析 |
| 复核 | python | | 独立复核 |
`会话表` =
| 单元 | 内核 | 说明 |
|------|------|------|
| 准备数据 | 分析 | 生成订单 |
| 计算指标 | 分析 | 区域与总额 |
| 独立复核 | 复核 | 读取 JSON 报告 |
```
live `view` 会把这些渲染成带 Run All、Step、切换内核、journal、dirty reload 和状态面板的单元。同一份文件仍可作为单个程序运行。
---
为什么它仍然可读。
| 层 | 读者看到什么 |
|---|---|
| Markdown | 标题、叙述、表格与围栏代码 |
| 程序 | 赋值、调用与返回值 |
| 知识 | frontmatter、链接、任务页与生成的 catalog 条目 |
外联层没有给语言新增语法。围栏还是围栏,表还是表。
底层 API 与安全默认值见 [`../../lib/外联.mq.md`](../../lib/外联.mq.md) 与 [`../../doc/design/foreign-kernel.md`](../../doc/design/foreign-kernel.md)。
+152
View File
@@ -0,0 +1,152 @@
---
title: 外联内核 v2 — Python 分析(view 单元)
description: 内核态、Marqdo 值传入与 view 单元 —— 不依赖 stdin 的完整示例
导入 外联:lib/外联.mq.md
---
# main
这一页把外联运行时当作语言能力来展示:持久内核、从 Marqdo 值显式传入状态,以及共享内核状态的 view 单元。读者不需要向 stdin 输入 JSON。
---
先探测解释器。结果仍然是普通 Marqdo 数据。
**python = > 外联.探测 语言="python"**
> 打印 内容=[ok](`python`)
> 打印 内容=[version](`python`)
---
Marqdo 值进入内核。
`订单` 表是普通 Marqdo 值。`外联.会话设值` 把它作为 JSON 复制进 Python 内核;没有 shell,也没有 stdin。
`订单` =
| @ | 区域 | 金额 |
|---|------|------|
| 1 | 华东 | 120 |
| 2 | 华东 | 300 |
| 3 | 华南 | 90 |
| 4 | 华南 | 140 |
| 5 | 华北 | 220 |
| 6 | 西部 | 80 |
**内核会话 = > 外联.会话开启 语言="python"**
**> 外联.会话设值 标识=`内核会话` 名称="orders" 值=`订单`**
`合计代码` =
```python
round(sum(float(row["金额"]) for row in orders), 2)
```
**合计 = > 外联.会话求值 标识=`内核会话` 代码=`合计代码`**
> 打印 内容=[result](`合计`)
低层内核会话完全由 `外联` 管理;表格以显式 JSON 拷贝跨过边界。
---
内核态是持续的。
两次求值共享同一个内核。第二次能看到第一次创建的变量。
`设值代码` =
```python
values = [3, 4, 5]
```
**> 外联.会话求值 标识=`内核会话` 代码=`设值代码`**
`平方和代码` =
```python
sum(value * value for value in values)
```
**平方和 = > 外联.会话求值 标识=`内核会话` 代码=`平方和代码`**
> 打印 内容=[result](`平方和`)
**> 外联.会话关闭 标识=`内核会话`**
这是内核态,不是一次性进程。它由 `外联` 创建、查询和关闭。
---
view 单元共享状态。
`外联.会话` 从普通表格声明一个 view 会话。表名完全自由;本页用 `分析内核` 和 `分析单元`。
`分析内核` =
| @ | 名称 | 语言 |
|---|------|------|
| 1 | 分析 | python |
| 2 | 复核 | python |
`分析单元` =
| @ | 单元 | 内核 |
|---|------|------|
| 1 | 计算 | 分析 |
| 2 | 复核 | 复核 |
`初始值` =
| 内核 | 名称 | 值 |
|------|------|----|
| 分析 | orders | `订单` |
| 复核 | orders | `订单` |
`会话` = > 外联.会话 内核表=`分析内核` 单元表=`分析单元` 值表=`初始值`
`计算` =
```python
import json
import statistics
by_region = {}
for row in orders:
by_region.setdefault(row["区域"], []).append(float(row["金额"]))
summary = []
for region, values in sorted(by_region.items()):
mean = statistics.mean(values)
sd = statistics.pstdev(values) or 1.0
summary.append({
"region": region,
"orders": len(values),
"total": round(sum(values), 2),
"mean": round(mean, 2),
"z_span": round((max(values) - mean) / sd, 3),
})
print(json.dumps({
"regions": summary,
"total": round(sum(float(row["金额"]) for row in orders), 2),
"count": len(orders),
}, ensure_ascii=False))
```
`复核` =
```python
import json
print(json.dumps({
"kernel": "复核",
"count": len(orders),
"total": round(sum(float(row["金额"]) for row in orders), 2),
}, ensure_ascii=False))
```
`查看` 消费 `会话` 句柄。Run All 在 `分析` 内核执行 `计算`,在 `复核` 内核执行 `复核`;静态导出走同一批单元,并把真实输出嵌入页面。
语言上的重点:
`初始值` 表把同一个 Marqdo `订单` 值传给两个内核。内核态归 `外联` 管理。view 只渲染句柄和单元。`run` 仍然把同一份文档当程序执行。
没有单独的 notebook CLI,这个示例不依赖 stdin,也没有隐藏的语言切换。
+4 -4
View File
@@ -21,13 +21,13 @@ OKF(Open Knowledge Format)用 Markdown + YAML frontmatter 表示知识:一
type: Capability
title: 持久外联内核
description: 在命名内核中运行 Python 或 Node 并保留状态
resource: public/features/19-外联Notebook.mq.md
resource: public/features/19-外联视图单元.mq.md
status: stable
verified:
by: marqdo-knowledge/verify
tags:
- foreign
- notebook
- view-cells
```
同一页可以在编辑器里读、从 catalog 链接、被 Agent 查询,也可以直接运行。未知字段保留;未知类型降级为可读正文。概念页不是源文件的复制品。
@@ -104,7 +104,7 @@ marqdo find "外联 会话"
marqdo reuse "实现 JSON 并行扇出"
# 看一次修改会影响什么
marqdo impact public/features/19-外联Notebook.mq.md
marqdo impact public/features/19-外联视图单元.mq.md
# 校验知识投影
marqdo knowledge verify
@@ -132,4 +132,4 @@ Catalog CLI:[Catalog CLI](../../doc/design/catalog-cli.md)
OKF 与 Marqdo 调研:[OKF 与 Marqdo 调研](../../doc/research/okf-and-marqdo.md)
外联 Notebook 示例:[外联 Notebook 示例](19-外联Notebook.mq.md)
外联 view 单元示例:[外联 view 单元示例](19-外联视图单元.mq.md)
+4 -4
View File
@@ -21,13 +21,13 @@ A concept page can therefore point at the exact file that proves it:
type: Capability
title: Persistent foreign kernels
description: Run Python or Node in a named, stateful kernel
resource: public/features/19-foreign-notebook.mq.md
resource: public/features/19-foreign-view-cells.mq.md
status: stable
verified:
by: marqdo-knowledge/verify
tags:
- foreign
- notebook
- view-cells
```
The same page can be opened in an editor, linked from a catalog, queried by an agent, and executed. Unknown fields are preserved; unknown types degrade to readable prose. The page is a concept, not a copy of one.
@@ -104,7 +104,7 @@ marqdo find "foreign session"
marqdo reuse "implement JSON fan-out"
# See what a change would touch
marqdo impact public/features/19-foreign-notebook.mq.md
marqdo impact public/features/19-foreign-view-cells.mq.md
# Validate the knowledge projection
marqdo knowledge verify
@@ -132,4 +132,4 @@ Catalog CLI: [Catalog CLI](../../doc/design/catalog-cli.md)
OKF and Marqdo research: [OKF and Marqdo research](../../doc/research/okf-and-marqdo.md)
Foreign notebook example: [Foreign notebook example](19-foreign-notebook.mq.md)
Foreign view-cells example: [Foreign view-cells example](19-foreign-view-cells.mq.md)
+56
View File
@@ -0,0 +1,56 @@
---
title: Persisting foreign execution results
description: Static snapshots and writeback — how foreign results survive after the process exits
import foreign:lib/foreign.mq.md
import writeback:lib/writeback.mq.md
---
# main
Foreign code has two useful lives: as a published snapshot, and as a living document whose latest result is kept in source. They answer different questions and should not be confused.
---
Static snapshot: publish the result.
`marqdo view output` executes the document's cells and embeds their real output in the generated HTML. The `.mq.md` source is not modified. This is the default choice for user documentation, reviews and stable links: a reader sees the result without running anything, and the source stays clean.
The static snapshot is a point-in-time artifact. If an external service changes, rebuild it; do not treat it as a live dashboard.
---
Writeback: keep the result in the source.
For an exploratory or audit document, pair `foreign` with `lib/writeback`. `writeback.record` writes a keyed output slot next to the call. The slot is visible in the Markdown source, can be reviewed in a diff and can be committed with the document.
| Step | What it does |
|------|--------------|
| Foreign run / Foreign session | Execute the foreign code and return a Marqdo value |
| Writeback record | Persist that value in a named slot |
| git diff | Review the changed result like any other document change |
A real writeback in this page.
This page is itself the runnable example. The code fence below is ordinary Marqdo data; `foreign.run` executes it, then `writeback.record` writes the `total` slot into this source file.
`写回代码` =
```python
print(950)
```
**结果 = > foreign.run code=`写回代码`**
**> writeback.record value=`结果` key="total"**
<!-- marqdo-out total
950
-->
The generated slot is directly below that call. Re-running the page replaces only the `total` slot; it does not duplicate it.
Writeback is explicit. It is not hidden kernel serialization, does not persist arbitrary kernel memory, and does not touch a file unless the document calls it.
---
Choosing.
For published user docs, use the static snapshot. For a living report that must keep the latest run result in the `.mq.md` itself, use writeback. Both can coexist in one repository: static pages for readers, writeback examples for authors and audits.
+56
View File
@@ -0,0 +1,56 @@
---
title: 外联执行结果的持久化
description: 静态快照与自写回 —— 外联结果在进程退出后如何保留
导入 外联:lib/外联.mq.md
import writeback:lib/writeback.mq.md
---
# main
外联代码有两种有用的形态:发布快照,以及把最新结果保留在源码里的活文档。两者回答不同的问题,不应混为一谈。
---
静态快照:发布结果。
`marqdo view output` 执行文档单元,并把真实输出嵌入生成的 HTML。`.mq.md` 源码不会被修改。这是用户文档、评审和稳定链接的默认选择:读者不需要运行任何东西就能看到结果,源码保持干净。
静态快照是某个时间点的产物。外部服务变化后重新构建即可,不要把它当成实时看板。
---
自写回:把结果留在源码里。
对于探索性或审计型文档,把 `外联` 和 `lib/writeback` 搭配使用。`writeback.record` 会在调用点附近写入带 key 的输出槽位。槽位在 Markdown 源码中可见,可以在 diff 中审阅,也可以随文档提交。
| 步骤 | 作用 |
|------|------|
| 外联执行 / 外联会话 | 执行外联代码并返回 Marqdo 值 |
| 写回记录 | 把该值持久化到命名槽位 |
| git diff | 像审阅普通文档改动一样审阅结果 |
这一页本身就是完整示例。
这一页就是可运行的示例。下面的代码围栏是普通 Marqdo 值;`外联.运行` 执行它,然后 `writeback.record` 把 `total` 槽位写进本页源码。
`写回代码` =
```python
print(950)
```
**结果 = > 外联.运行 代码=`写回代码`**
**> writeback.record value=`结果` key="total"**
<!-- marqdo-out total
950
-->
生成的槽位就在这次调用下方。再次运行只替换 `total` 槽位,不会重复添加。
自写回是显式动作。它不是隐藏的内核序列化,不持久化任意内核内存,也不会在文档没有调用时修改文件。
---
如何选择。
面向用户的发布文档用静态快照;需要把最新执行结果留在 `.mq.md` 本身里的活文档用自写回。两者可以共存于同一个仓库:静态页面给读者,自写回示例给作者和审计。
+1 -1
View File
@@ -11,7 +11,7 @@ English-named files export English APIs; Chinese-named files export Chinese APIs
No extra allow flags: importing a module means you intend to use it. Under marqdo view or static export, exit soft-fails and long sleeps are clamped so the console stays up.
Modules: text (text / 文本), table (table / 表), files (fs / 文件), time (time / 时间), system (sys / 系统), json (json only), net (net / 网络), math (math / 数学), foreign (foreign / 外联), plugin (plugin / 插件), writeback (writeback / 自写回), subtask (subtask / 子任务). Foreign v2 adds structured invoke records, JSON calls, parallel fan-out, persistent Python/Node kernels, an explicit state bridge and document-native notebook sessions — see stdlib/foreign.mq.md and features/19-foreign-notebook.mq.md.
Modules: text (text / 文本), table (table / 表), files (fs / 文件), time (time / 时间), system (sys / 系统), json (json only), net (net / 网络), math (math / 数学), foreign (foreign / 外联), plugin (plugin / 插件), writeback (writeback / 自写回), subtask (subtask / 子任务). Foreign v2 adds structured invoke records, JSON calls, parallel fan-out, persistent Python/Node kernels, an explicit state bridge and document-native view cells — see stdlib/foreign.mq.md and features/19-foreign-view-cells.mq.md.
From v0.1.2, official `lib/*.mq.md` is embedded in the `marqdo` binary (disk `lib/` or `MARQDO_LIB` still overrides). Standalone `.exe` works without a separate stdlib zip.
+1 -1
View File
@@ -11,7 +11,7 @@ description: 如何导入官方 lib 模块
导入即表示要用对应能力,无需再加允许旗标。marqdo view 或静态导出里,退出会软失败、长睡眠会被钳制,以免拖垮控制台。
模块:文本(text / 文本)、表(table / 表)、文件(fs / 文件)、时间(time / 时间)、系统(sys / 系统)、json(仅 json)、网络(net / 网络)、数学(math / 数学)、外联(foreign / 外联)、插件(plugin / 插件)、自写回(writeback / 自写回)、子任务(subtask / 子任务)。外联 v2 新增结构化执行记录、JSON 调用、并行扇出、Python/Node 持久内核、显式状态桥与文档原生 notebook 会话 — 见 stdlib/外联.mq.md 与 features/19-外联Notebook.mq.md。
模块:文本(text / 文本)、表(table / 表)、文件(fs / 文件)、时间(time / 时间)、系统(sys / 系统)、json(仅 json)、网络(net / 网络)、数学(math / 数学)、外联(foreign / 外联)、插件(plugin / 插件)、自写回(writeback / 自写回)、子任务(subtask / 子任务)。外联 v2 新增结构化执行记录、JSON 调用、并行扇出、Python/Node 持久内核、显式状态桥与文档原生 view 单元 — 见 stdlib/外联.mq.md 与 features/19-外联视图单元.mq.md。
自 v0.1.2 起,官方 `lib/*.mq.md` 内置于 `marqdo` 二进制(磁盘 `lib/` 或 `MARQDO_LIB` 仍可覆盖)。单独下载 `.exe` 即可导入标准库,无需再解压 stdlib 包。
+8 -5
View File
@@ -2,6 +2,7 @@
title: lib/foreign — foreign kernels v2
description: Structured execution, JSON calls, parallel fan-out, persistent kernels and an explicit state bridge
import foreign:lib/foreign.mq.md
stdin: 21
---
# main
@@ -106,28 +107,30 @@ This is a JSON copy, not shared memory. The kernel receives a value; Marqdo rece
---
Notebook sessions.
Cells in view.
A document can declare named kernels and a cell order without new Markdown markers:
`foreign.session` owns the document's kernel state. The kernel and cell order are ordinary tables passed as arguments; the names are free:
```text
`内核表` =
`分析内核` =
| 名称 | 语言 | 命令 | 说明 |
|------|------|------|------|
| 分析 | python | | shared analysis |
| 复核 | python | | isolated review |
`会话表` =
`分析单元` =
| 单元 | 内核 | 说明 |
|------|------|------|
| 准备数据 | 分析 | write state |
| 计算指标 | 分析 | read state |
| 独立复核 | 复核 | isolated kernel |
`会话` = > foreign.session kernels=`分析内核` cells=`分析单元`
```
Live `view` then adds Run All, Step, kernel switching, a journal and per-cell state. See the complete Python walkthrough in [`features/19-foreign-notebook.mq.md`](../features/19-foreign-notebook.mq.md).
`view` consumes the `会话` handle and adds Run All, Step, kernel switching, a journal and per-cell state. Without `会话`, view cells are stateless one-shot runs: same shell, no persistent kernel. There is no second CLI mode. See the complete Python walkthrough in [`features/19-foreign-view-cells.mq.md`](../features/19-foreign-view-cells.mq.md).
---
+8 -5
View File
@@ -2,6 +2,7 @@
title: lib/外联 — 外联内核 v2
description: 结构化执行、JSON 调用、并行扇出、持久内核与显式状态桥
导入 外联:lib/外联.mq.md
stdin: 21
---
# main
@@ -106,28 +107,30 @@ process.stdin.on("end", () => {
---
Notebook 会话。
view 中的单元。
文档可以直接声明命名内核和单元顺序,不需要新增 Markdown 标记:
`foreign.session` 拥有文档的内核状态。内核和单元顺序都是作为参数传入的普通表格,名字可以自由选择:
```text
`内核表` =
`分析内核` =
| 名称 | 语言 | 命令 | 说明 |
|------|------|------|------|
| 分析 | python | | 共享分析 |
| 复核 | python | | 独立复核 |
`会话表` =
`分析单元` =
| 单元 | 内核 | 说明 |
|------|------|------|
| 准备数据 | 分析 | 写入状态 |
| 计算指标 | 分析 | 读取状态 |
| 独立复核 | 复核 | 隔离内核 |
`会话` = > foreign.session kernels=`分析内核` cells=`分析单元`
```
live `view` 会提供 Run All、Step、切换内核、journal 和单元状态。完整 Python 示例见 [`features/19-外联Notebook.mq.md`](../features/19-外联Notebook.mq.md)。
`view` 消费 `会话` 句柄,提供 Run All、Step、切换内核、journal 和单元状态。没有 `会话` 时,view 的 cells 是无状态的一次性运行:同一套外壳,但没有持久内核。没有第二个 CLI 模式。完整 Python 示例见 [`features/19-外联视图单元.mq.md`](../features/19-外联视图单元.mq.md)。
---
+3
View File
@@ -89,7 +89,10 @@ impl StmtEventKind {
pub struct CallFrame {
pub call_id: u32,
pub fn_name: String,
/// Definition line of the callee.
pub fn_line: u32,
/// Call site line in the executing document (0 for the entry call).
pub call_line: u32,
pub ordinal: u32,
pub args_brief: String,
}
+19 -2
View File
@@ -217,6 +217,7 @@ pub enum HostFn {
ForeignSessionNames = 208,
ForeignSessionClose = 209,
ForeignSessionList = 210,
ForeignSessionInterrupt = 211,
}
@@ -424,6 +425,7 @@ impl HostFn {
208 => Self::ForeignSessionNames,
209 => Self::ForeignSessionClose,
210 => Self::ForeignSessionList,
211 => Self::ForeignSessionInterrupt,
_ => return None,
})
}
@@ -515,6 +517,7 @@ impl HostFn {
"host_foreign_session_names" | "foreign_session_names" => Self::ForeignSessionNames,
"host_foreign_session_close" | "foreign_session_close" => Self::ForeignSessionClose,
"host_foreign_session_list" | "foreign_session_list" => Self::ForeignSessionList,
"host_foreign_session_interrupt" | "foreign_session_interrupt" => Self::ForeignSessionInterrupt,
"host_foreign_langs" => Self::ForeignLangs,
"host_dotenv_load" | "load_dotenv" => Self::DotenvLoad,
"host_http_request" | "http_request" => Self::HttpRequest,
@@ -726,6 +729,7 @@ impl HostFn {
Self::ForeignSessionNames => "host_foreign_session_names",
Self::ForeignSessionClose => "host_foreign_session_close",
Self::ForeignSessionList => "host_foreign_session_list",
Self::ForeignSessionInterrupt => "host_foreign_session_interrupt",
Self::ForeignLangs => "host_foreign_langs",
Self::DotenvLoad => "host_dotenv_load",
Self::HttpRequest => "host_http_request",
@@ -904,6 +908,7 @@ impl HostFn {
Self::ForeignSessionNames => &["id"],
Self::ForeignSessionClose => &["id"],
Self::ForeignSessionList => &[],
Self::ForeignSessionInterrupt => &["id"],
Self::ForeignLangs => &[],
Self::PluginLoad => &["path"],
Self::PluginUnload | Self::PluginList => &[],
@@ -1053,7 +1058,7 @@ impl HostFn {
"concurrency",
],
Self::ForeignSessionOpen => &["cmd", "env", "cwd", "timeout"],
Self::ForeignSessionEval => &["code", "source", "input", "timeout"],
Self::ForeignSessionEval => &["code", "source", "input", "stdin", "timeout"],
Self::ForeignSessionSet
| Self::ForeignSessionGet
| Self::ForeignSessionNames
@@ -1512,12 +1517,13 @@ pub fn call_host(
HostFn::ForeignSessionEval => {
#[cfg(feature = "exec-host")]
{
crate::host::foreign_session::eval(
crate::host::foreign_session::eval_with_stdin(
ctx,
require(bound, "id")?,
bound.get("code"),
bound.get("source"),
bound.get("input"),
bound.get("stdin").and_then(|v| v.reveal_text()),
bound.get("timeout"),
)
}
@@ -1580,6 +1586,17 @@ pub fn call_host(
Err("foreign unavailable in browser wasm".into())
}
}
HostFn::ForeignSessionInterrupt => {
#[cfg(feature = "exec-host")]
{
crate::host::foreign_session::interrupt(ctx, require(bound, "id")?)
}
#[cfg(not(feature = "exec-host"))]
{
let _ = (ctx, bound);
Err("foreign unavailable in browser wasm".into())
}
}
HostFn::ForeignSessionList => {
#[cfg(feature = "exec-host")]
{
File diff suppressed because it is too large. Load diff
+54
View File
@@ -53,6 +53,9 @@ for _line in sys.stdin:
continue
_code = req.get("code", "")
_g["marqdo_input"] = req.get("input")
_old_stdin = sys.stdin
if req.get("stdin") is not None:
sys.stdin = io.StringIO(str(req.get("stdin")))
_out_buf = io.StringIO()
_err_buf = io.StringIO()
try:
@@ -69,6 +72,8 @@ for _line in sys.stdin:
_send({"id": rid, "ok": True, "stdout": _out_buf.getvalue(), "stderr": _err_buf.getvalue(), "result": _result, "error": ""})
except BaseException:
_send({"id": rid, "ok": False, "stdout": _out_buf.getvalue(), "stderr": _err_buf.getvalue(), "result": None, "error": traceback.format_exc(limit=8)})
finally:
sys.stdin = _old_stdin
"#;
const JS_BOOTSTRAP: &str = r#"const readline = require("readline");
@@ -113,7 +118,21 @@ rl.on("line", async (line) => {
return;
}
globalThis.marqdo_input = req.input === undefined ? null : req.input;
const oldStdin = process.stdin;
let stdinPatched = false;
if (req.stdin !== undefined && req.stdin !== null) {
try {
const { PassThrough } = require("stream");
const ps = new PassThrough();
ps.end(String(req.stdin));
Object.defineProperty(process, "stdin", { value: ps, configurable: true, writable: true });
stdinPatched = true;
} catch (e) {}
}
const cap = await capture(async () => vm.runInThisContext(req.code || "", { filename: "<marqdo>" }));
if (stdinPatched) {
try { Object.defineProperty(process, "stdin", { value: oldStdin, configurable: true, writable: true }); } catch (e) {}
}
send({ id: rid, ok: !cap.error, stdout: cap.logs, stderr: cap.errs, result: toJSON(cap.result), error: cap.error });
});
"#;
@@ -425,6 +444,18 @@ pub fn eval(
source: Option<&Value>,
input: Option<&Value>,
timeout: Option<&Value>,
) -> Result<Value, String> {
eval_with_stdin(ctx, id, code, source, input, None, timeout)
}
pub fn eval_with_stdin(
ctx: &mut HostContext,
id: &Value,
code: Option<&Value>,
source: Option<&Value>,
input: Option<&Value>,
stdin: Option<&str>,
timeout: Option<&Value>,
) -> Result<Value, String> {
ensure_config(ctx);
let id = as_text(id, "id")?.to_string();
@@ -462,6 +493,7 @@ pub fn eval(
"op": "eval",
"code": code,
"input": input_json,
"stdin": stdin,
});
let start = Instant::now();
let (lang, cmd) = {
@@ -620,6 +652,28 @@ fn kill_session(ctx: &mut HostContext, id: &str) {
}
}
/// Interrupt the current evaluation without destroying the kernel. Python
/// reports KeyboardInterrupt through the normal result frame.
pub fn interrupt(ctx: &mut HostContext, id: &Value) -> Result<Value, String> {
let id = as_text(id, "id")?.to_string();
let session = ctx
.foreign_sessions
.get_mut(&id)
.ok_or_else(|| format!("foreign session `{id}` not found"))?;
#[cfg(unix)]
unsafe {
libc::kill(-(session.child.id() as i32), libc::SIGINT);
}
#[cfg(windows)]
{
let _ = Command::new("taskkill")
.args(["/PID", &session.child.id().to_string(), "/T", "/F"])
.output();
session.alive = false;
}
Ok(Value::Bool(true))
}
pub fn set(
ctx: &mut HostContext,
id: &Value,
+2
View File
@@ -15,6 +15,8 @@ pub mod event_bus;
#[cfg(feature = "exec-host")]
pub mod foreign;
#[cfg(feature = "exec-host")]
pub mod foreign_doc;
#[cfg(feature = "exec-host")]
pub mod foreign_session;
mod fs;
mod hash;
+148
View File
@@ -208,6 +208,7 @@ impl Interpreter {
call_id,
fn_name: fun.name.clone(),
fn_line: fun.span.line,
call_line: self.current_span.line,
ordinal,
args_brief,
});
@@ -372,6 +373,98 @@ impl Interpreter {
result
}
/// Evaluate only the entry object's top-level assignments, without running
/// calls/returns/control flow. This is the view plan's language-native
/// projection: cell tables are ordinary Marqdo values, not source scans.
/// Evaluate selected entry bindings plus their assignment dependencies.
/// This is the view plan's language-native projection: cell tables are
/// ordinary Marqdo values, not source scans.
pub fn capture_declarations(
&mut self,
module: &Module,
names: &[&str],
) -> Result<std::collections::HashMap<String, Value>> {
let prev = self.site_module;
self.site_module = Some(module as *const Module);
let result = (|| {
let entry = find_top(module, "main").or_else(|| {
let entries: Vec<&Function> = module
.functions
.iter()
.filter(|f| f.is_object() && f.params.is_empty())
.collect();
if entries.len() == 1 {
Some(entries[0])
} else {
None
}
});
let entry = entry.ok_or_else(|| self.err("no `# main` object to inspect"))?;
let assignments: std::collections::HashMap<&str, &Expr> = entry
.body
.iter()
.filter_map(|stmt| match stmt {
Stmt::Assign { name, value, .. } => Some((name.as_str(), value)),
_ => None,
})
.collect();
let mut env = Env::with_metadata(module);
let mut done = std::collections::HashSet::new();
let mut visiting = std::collections::HashSet::new();
fn resolve(
this: &mut Interpreter,
module: &Module,
fun: &Function,
assignments: &std::collections::HashMap<&str, &Expr>,
name: &str,
env: &mut Env,
done: &mut std::collections::HashSet<String>,
visiting: &mut std::collections::HashSet<String>,
) -> Result<()> {
if done.contains(name) {
return Ok(());
}
if !visiting.insert(name.to_string()) {
return Err(this.err(format!("cyclic view-plan binding `{name}`")));
}
let Some(expr) = assignments.get(name) else {
visiting.remove(name);
return Ok(());
};
let mut deps = std::collections::HashSet::new();
collect_decl_reads(expr, &mut deps);
for dep in deps {
if assignments.contains_key(dep.as_str()) && env.get(&dep).is_none() {
resolve(this, module, fun, assignments, &dep, env, done, visiting)?;
}
}
let value = this.eval_expr(module, fun, env, expr)?;
env.set(name.to_string(), value);
done.insert(name.to_string());
visiting.remove(name);
Ok(())
}
for name in names {
resolve(
self,
module,
entry,
&assignments,
name,
&mut env,
&mut done,
&mut visiting,
)?;
}
self.entry_bindings = env.vars_clone();
Ok(self.entry_bindings.clone())
})();
self.site_module = prev;
result
}
pub fn run_module(&mut self, module: &Module) -> Result<Value> {
let prev = self.site_module;
self.site_module = Some(module as *const Module);
@@ -1557,6 +1650,61 @@ fn bind_args(
Ok(out)
}
fn collect_decl_reads(expr: &Expr, out: &mut std::collections::HashSet<String>) {
match expr {
Expr::Var(name) => {
out.insert(name.clone());
}
Expr::Interp(parts) => {
for part in parts {
match part {
InterpPart::Var(name) => {
out.insert(name.clone());
}
InterpPart::Index { base, .. } => {
out.insert(base.clone());
}
InterpPart::Lit(_) => {}
}
}
}
Expr::Index { base, label } => {
collect_decl_reads(base, out);
if let IndexKey::Var(name) = label {
out.insert(name.clone());
}
}
Expr::Binary { left, right, .. } => {
collect_decl_reads(left, out);
collect_decl_reads(right, out);
}
Expr::Unary { expr, .. } => collect_decl_reads(expr, out),
Expr::List(xs) => {
for x in xs {
collect_decl_reads(x, out);
}
}
Expr::Map(entries) => {
for (_, v) in entries {
collect_decl_reads(v, out);
}
}
Expr::Call(call) => {
if let Some(receiver) = &call.receiver {
out.insert(receiver.clone());
}
for arg in &call.args {
match arg {
Arg::Positional(e) | Arg::Named { value: e, .. } => {
collect_decl_reads(e, out);
}
}
}
}
Expr::Literal(_) | Expr::Formula(_) | Expr::Code(_) => {}
}
}
fn find_top<'a>(module: &'a Module, name: &str) -> Option<&'a Function> {
module.functions.iter().find(|f| f.name == name)
}
-2
View File
@@ -34,8 +34,6 @@ pub mod lex;
pub mod linalg_ops;
pub mod load;
pub mod mlsp;
#[cfg(feature = "exec-host")]
pub mod notebook;
pub mod parse;
pub mod repair;
pub mod value;
+23 -1
View File
@@ -16,7 +16,7 @@ use marqdo::knowledge::{
resolve, reuse_json, stale_json, verify, verify_json, write_knowledge, KnowledgeOptions,
LearnInput, ReuseBudget,
};
use marqdo::view::{serve, serve_debug, write_static, DebugOptions, OutputOptions, ViewOptions};
use marqdo::view::{serve, serve_debug, write_capture, write_static, CaptureOptions, DebugOptions, OutputOptions, ViewOptions};
use marqdo::{Backend, RunOptions};
#[derive(Parser, Debug)]
@@ -374,6 +374,17 @@ enum ViewAction {
#[arg(long)]
no_exec: bool,
#[arg(long, value_name = "FILE")]
snapshot: Option<PathBuf>,
},
/// Capture live cell results for `view output --snapshot`
Capture {
#[arg(value_name = "PATH")]
path: Option<PathBuf>,
#[arg(short = 'o', long = "out", required = true)]
out: PathBuf,
},
}
@@ -509,6 +520,7 @@ fn try_main(cli: Cli) -> Result<i32> {
path: out_path,
out,
no_exec,
snapshot,
}) => {
write_static(OutputOptions {
path: out_path
@@ -516,6 +528,16 @@ fn try_main(cli: Cli) -> Result<i32> {
.unwrap_or_else(|| PathBuf::from(".")),
out_dir: out,
no_exec,
snapshot,
})?;
Ok(0)
}
Some(ViewAction::Capture { path: out_path, out }) => {
write_capture(CaptureOptions {
path: out_path
.or(path)
.unwrap_or_else(|| PathBuf::from(".")),
out,
})?;
Ok(0)
}
-995
View File
@@ -1,995 +0,0 @@
//! Document session: cells, kernels, execution journal, state store.
//!
//! The notebook layer reads the conventional `内核表`/`会话表` bindings and the
//! bound code fences. It does not change the Markdown parser.
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use sha2::{Digest, Sha256};
use crate::foreign::registry;
use crate::host::foreign_session;
use crate::host::HostContext;
use crate::parse::parse_source;
use crate::value::Value;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CellStatus {
Never,
Running,
Clean,
Dirty,
Stale,
Error,
Unlisted,
}
impl CellStatus {
fn as_str(self) -> &'static str {
match self {
Self::Never => "never",
Self::Running => "running",
Self::Clean => "clean",
Self::Dirty => "dirty",
Self::Stale => "stale",
Self::Error => "error",
Self::Unlisted => "unlisted",
}
}
}
#[derive(Debug, Clone)]
pub struct Cell {
pub name: String,
pub lang: String,
pub source: String,
pub line_start: u32,
pub line_end: u32,
pub source_hash: String,
pub kernel: String,
pub listed: bool,
pub status: CellStatus,
pub runs: u64,
pub last_ms: u64,
pub last_ok: bool,
pub last_error: String,
}
#[derive(Debug, Clone)]
pub struct KernelDecl {
pub name: String,
pub lang: String,
pub cmd: Option<String>,
pub implicit: bool,
}
#[derive(Debug, Clone)]
pub struct JournalRecord {
pub seq: u64,
pub cell: String,
pub kernel: String,
pub lang: String,
pub ok: bool,
pub code: i64,
pub ms: u64,
pub timestamp_ms: i64,
pub error: String,
}
pub struct DocumentSession {
pub path: PathBuf,
pub cells: Vec<Cell>,
pub kernels: Vec<KernelDecl>,
pub order: Vec<String>,
pub journal: Vec<JournalRecord>,
pub value_store: HashMap<String, Value>,
pub cursor: usize,
pub opened_ms: i64,
host: HostContext,
kernel_sessions: HashMap<String, String>,
seq: u64,
}
fn hash_source(source: &str) -> String {
let mut h = Sha256::new();
h.update(source.as_bytes());
format!("{:x}", h.finalize())
}
fn now_ms() -> i64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as i64)
.unwrap_or(0)
}
/// Split a GFM row into trimmed cells.
fn split_row(line: &str) -> Vec<String> {
let t = line.trim();
let t = t.strip_prefix('|').unwrap_or(t);
let t = t.strip_suffix('|').unwrap_or(t);
t.split('|')
.map(|c| c.trim().trim_matches('`').trim().to_string())
.collect()
}
/// Find `` `name` = `` and the first contiguous table after it.
fn table_after(source: &str, name: &str) -> Result<Option<Vec<HashMap<String, String>>>, String> {
let lines: Vec<&str> = source.lines().collect();
let needle_a = format!("`{name}`");
let needle_b = format!("{name} =");
let mut start = None;
for (i, line) in lines.iter().enumerate() {
let t = line.trim();
if t.starts_with(&needle_a) && t.contains('=') || t.starts_with(&needle_b) {
start = Some(i + 1);
break;
}
}
let Some(mut i) = start else {
return Ok(None);
};
while i < lines.len() && !lines[i].trim().starts_with('|') {
if !lines[i].trim().is_empty() && !lines[i].trim().starts_with('|') {
// Prose between the binding and the table is allowed.
}
i += 1;
}
if i >= lines.len() {
return Ok(None);
}
let mut rows = Vec::new();
let mut header: Option<Vec<String>> = None;
while i < lines.len() {
let t = lines[i].trim();
if !t.starts_with('|') {
break;
}
i += 1;
if t.chars().all(|c| c == '|' || c == '-' || c == ':' || c.is_whitespace()) {
continue;
}
let cells = split_row(t);
if header.is_none() {
header = Some(cells);
continue;
}
let h = header.as_ref().unwrap();
if cells.len() != h.len() {
return Err(format!(
"table `{name}`: row has {} cells, header has {}",
cells.len(),
h.len()
));
}
let mut row = HashMap::new();
for (k, v) in h.iter().zip(cells) {
row.insert(k.clone(), v);
}
rows.push(row);
}
Ok(Some(rows))
}
fn table_any(
source: &str,
names: &[&str],
) -> Result<Option<Vec<HashMap<String, String>>>, String> {
for name in names {
if let Some(rows) = table_after(source, name)? {
return Ok(Some(rows));
}
}
Ok(None)
}
fn col<'a>(row: &'a HashMap<String, String>, names: &[&str]) -> Option<&'a String> {
for n in names {
if let Some(v) = row.get(*n) {
if !v.is_empty() {
return Some(v);
}
}
}
None
}
fn collect_cells(stmts: &[crate::ast::Stmt], out: &mut Vec<Cell>, seen: &mut HashSet<String>) -> Result<(), String> {
for stmt in stmts {
match stmt {
crate::ast::Stmt::Assign {
name,
value: crate::ast::Expr::Code(c),
span,
end_line,
} => {
if !seen.insert(name.clone()) {
return Err(format!(
"cell `{name}` is assigned more than once (line {})",
span.line
));
}
out.push(Cell {
name: name.clone(),
lang: c.lang.clone(),
source: c.source.clone(),
line_start: span.line,
line_end: *end_line,
source_hash: hash_source(&c.source),
kernel: String::new(),
listed: false,
status: CellStatus::Never,
runs: 0,
last_ms: 0,
last_ok: false,
last_error: String::new(),
});
}
crate::ast::Stmt::Branch { arms, .. } => {
for arm in arms {
collect_cells(&arm.body, out, seen)?;
}
}
crate::ast::Stmt::While { body, .. } | crate::ast::Stmt::ForEach { body, .. } => {
collect_cells(body, out, seen)?;
}
_ => {}
}
}
Ok(())
}
impl DocumentSession {
pub fn open(path: &Path, root: &Path) -> Result<Self, String> {
let source = std::fs::read_to_string(path)
.map_err(|e| format!("read {}: {e}", path.display()))?;
let module = parse_source(&source).map_err(|e| format!("parse {}: {e}", path.display()))?;
let entry = module
.functions
.iter()
.find(|f| f.name == "main" && f.is_object())
.or_else(|| {
let objs: Vec<_> = module.functions.iter().filter(|f| f.is_object()).collect();
if objs.len() == 1 {
Some(objs[0])
} else {
None
}
})
.ok_or_else(|| "notebook: no `# main` (or sole object) to index".to_string())?;
let mut cells = Vec::new();
let mut seen = HashSet::new();
collect_cells(&entry.body, &mut cells, &mut seen)?;
if cells.is_empty() {
return Err("notebook: no bound code fences (`name` = + ```lang)".into());
}
let mut kernels: Vec<KernelDecl> = Vec::new();
let mut kernel_map: HashMap<String, KernelDecl> = HashMap::new();
if let Some(rows) = table_any(&source, &["内核表", "kernels", "Kernels"])? {
if rows.is_empty() {
return Err("内核表: at least one kernel row is required".into());
}
for row in rows {
let name = col(&row, &["名称", "Name", "name"])
.ok_or_else(|| "内核表: missing 名称/Name".to_string())?
.clone();
let lang = col(&row, &["语言", "Language", "language", "lang"])
.ok_or_else(|| format!("内核表[{name}]: missing 语言/Language"))?
.clone();
let lang = registry::canonical(&lang);
let cmd = col(&row, &["命令", "Command", "command", "cmd"]).cloned();
if kernel_map.contains_key(&name) {
return Err(format!("内核表: kernel `{name}` is declared twice"));
}
let decl = KernelDecl {
name: name.clone(),
lang,
cmd,
implicit: false,
};
kernel_map.insert(name.clone(), decl.clone());
kernels.push(decl);
}
}
let mut order: Vec<String> = Vec::new();
let session_rows = table_any(&source, &["会话表", "session", "Session"])?;
if let Some(rows) = session_rows {
if rows.is_empty() {
return Err("会话表: at least one cell row is required".into());
}
let by_name: HashMap<String, usize> = cells
.iter()
.enumerate()
.map(|(i, c)| (c.name.clone(), i))
.collect();
let mut listed = HashSet::new();
for row in rows {
let cell_name = col(&row, &["单元", "Cell", "cell", "单元名"])
.ok_or_else(|| "会话表: missing 单元/Cell".to_string())?
.clone();
let kernel_name = col(&row, &["内核", "Kernel", "kernel"])
.ok_or_else(|| format!("会话表[{cell_name}]: missing 内核/Kernel"))?
.clone();
let Some(&ci) = by_name.get(&cell_name) else {
return Err(format!("会话表: unknown cell `{cell_name}`"));
};
if !listed.insert(cell_name.clone()) {
return Err(format!("会话表: cell `{cell_name}` is listed twice"));
}
let kernel = resolve_kernel(&mut kernels, &mut kernel_map, &kernel_name)?;
if registry::canonical(&cells[ci].lang) != registry::canonical(&kernel.lang) {
return Err(format!(
"会话表: cell `{cell_name}` ({}) cannot use kernel `{}` ({})",
cells[ci].lang, kernel.name, kernel.lang
));
}
cells[ci].kernel = kernel.name.clone();
cells[ci].listed = true;
order.push(cell_name);
}
for cell in &mut cells {
if !cell.listed {
cell.status = CellStatus::Unlisted;
}
}
} else {
for cell in &mut cells {
let kernel = ensure_default_kernel(
&mut kernels,
&mut kernel_map,
&cell.lang,
);
cell.kernel = kernel;
cell.listed = true;
order.push(cell.name.clone());
}
}
let mut host = HostContext::default();
let doc_dir = path.parent().unwrap_or_else(|| Path::new("."));
host.cwd = doc_dir.to_path_buf();
host.fs_root = Some(root.to_path_buf());
host.entry_path = Some(path.to_path_buf());
host.entry_source = Some(source);
Ok(Self {
path: path.to_path_buf(),
cells,
kernels,
order,
journal: Vec::new(),
value_store: HashMap::new(),
cursor: 0,
opened_ms: now_ms(),
host,
kernel_sessions: HashMap::new(),
seq: 0,
})
}
/// Re-read the document, preserving kernels/state/journal. Changed cells become dirty.
pub fn reload(&mut self, root: &Path) -> Result<(), String> {
let mut fresh = Self::open(&self.path, root)?;
let old: HashMap<String, Cell> = self
.cells
.iter()
.cloned()
.map(|c| (c.name.clone(), c))
.collect();
for c in &mut fresh.cells {
if let Some(o) = old.get(&c.name) {
c.runs = o.runs;
c.last_ms = o.last_ms;
c.last_ok = o.last_ok;
c.last_error = o.last_error.clone();
c.status = if o.source_hash != c.source_hash || o.kernel != c.kernel {
CellStatus::Dirty
} else if o.status == CellStatus::Running {
CellStatus::Dirty
} else {
o.status
};
}
}
fresh.host = std::mem::replace(&mut self.host, HostContext::default());
fresh.kernel_sessions = std::mem::take(&mut self.kernel_sessions);
fresh.value_store = std::mem::take(&mut self.value_store);
fresh.journal = std::mem::take(&mut self.journal);
fresh.seq = self.seq;
fresh.cursor = self.cursor.min(fresh.order.len());
fresh.opened_ms = self.opened_ms;
let known: HashSet<String> = fresh.kernels.iter().map(|k| k.name.clone()).collect();
let orphans: Vec<String> = fresh
.kernel_sessions
.keys()
.filter(|k| !known.contains(*k))
.cloned()
.collect();
for k in orphans {
if let Some(id) = fresh.kernel_sessions.remove(&k) {
let _ = foreign_session::close(&mut fresh.host, &Value::Text(id));
}
}
*self = fresh;
Ok(())
}
pub fn title(&self) -> String {
self.path
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or("session")
.trim_end_matches(".mq")
.to_string()
}
fn cell_index(&self, name: &str) -> Option<usize> {
self.cells.iter().position(|c| c.name == name)
}
fn kernel(&self, name: &str) -> Option<&KernelDecl> {
self.kernels.iter().find(|k| k.name == name)
}
fn ensure_kernel_session(&mut self, kernel_name: &str) -> Result<String, String> {
if let Some(id) = self.kernel_sessions.get(kernel_name) {
if self.host.foreign_sessions.contains_key(id) {
return Ok(id.clone());
}
}
let decl = self
.kernel(kernel_name)
.ok_or_else(|| format!("unknown kernel `{kernel_name}`"))?
.clone();
let cmd_value = decl.cmd.map(Value::Text);
let lang = Value::Text(decl.lang.clone());
let id = foreign_session::open(
&mut self.host,
&lang,
cmd_value.as_ref(),
None,
None,
None,
)?;
let id = match id {
Value::Text(s) => s,
_ => return Err("session_open returned non-text id".into()),
};
self.kernel_sessions
.insert(kernel_name.to_string(), id.clone());
Ok(id)
}
fn record_result(&mut self, cell: &str, kernel: &str, record: &Value) -> Result<(), String> {
let field = |k: &str| match record {
Value::Map(entries) => entries
.iter()
.find(|(kk, _)| kk == k)
.map(|(_, v)| v.clone()),
_ => None,
};
let ok = matches!(field("ok"), Some(Value::Bool(true)));
let ms = match field("ms") {
Some(Value::Int(n)) => n as u64,
_ => 0,
};
let error = field("error")
.and_then(|v| v.reveal_text().map(|s| s.to_string()))
.unwrap_or_default();
let code = match field("code") {
Some(Value::Int(n)) => n,
_ => 0,
};
let lang = self
.cell_index(cell)
.map(|i| self.cells[i].lang.clone())
.unwrap_or_default();
if let Some(i) = self.cell_index(cell) {
let c = &mut self.cells[i];
c.runs += 1;
c.last_ms = ms;
c.last_ok = ok;
c.last_error = error.clone();
c.status = if ok { CellStatus::Clean } else { CellStatus::Error };
}
self.seq += 1;
self.journal.push(JournalRecord {
seq: self.seq,
cell: cell.to_string(),
kernel: kernel.to_string(),
lang,
ok,
code,
ms,
timestamp_ms: now_ms(),
error,
});
Ok(())
}
pub fn run_cell(&mut self, cell_name: &str, kernel_override: Option<&str>) -> Result<Value, String> {
let ci = self
.cell_index(cell_name)
.ok_or_else(|| format!("unknown cell `{cell_name}`"))?;
if !self.cells[ci].listed && kernel_override.is_none() {
return Err(format!("cell `{cell_name}` is not in 会话表"));
}
let kernel = match kernel_override {
Some(k) => {
if self.kernel(k).is_none() {
return Err(format!("unknown kernel `{k}`"));
}
let ci_lang = registry::canonical(&self.cells[ci].lang);
let klang = registry::canonical(
&self.kernel(k).map(|d| d.lang.clone()).unwrap_or_default(),
);
if ci_lang != klang {
return Err(format!(
"cell `{cell_name}` ({}) cannot run in kernel `{k}` ({klang})",
self.cells[ci].lang
));
}
self.cells[ci].kernel = k.to_string();
self.cells[ci].listed = true;
k.to_string()
}
None => self.cells[ci].kernel.clone(),
};
self.cells[ci].status = CellStatus::Running;
let result = (|| -> Result<Value, String> {
let id = self.ensure_kernel_session(&kernel)?;
let source = Value::Text(self.cells[ci].source.clone());
let record = foreign_session::eval(
&mut self.host,
&Value::Text(id),
None,
Some(&source),
None,
None,
)?;
self.record_result(cell_name, &kernel, &record)?;
Ok(record)
})();
if result.is_err() {
self.cells[ci].status = CellStatus::Error;
}
result
}
pub fn step(&mut self) -> Result<Value, String> {
if self.cursor >= self.order.len() {
return Ok(Value::Map(vec![
("done".into(), Value::Bool(true)),
("cursor".into(), Value::Int(self.cursor as i64)),
]));
}
let name = self.order[self.cursor].clone();
let record = self.run_cell(&name, None)?;
let ci = self.cell_index(&name).unwrap_or(0);
let kernel = self.cells[ci].kernel.clone();
self.cursor += 1;
let mut fields = match record {
Value::Map(m) => m,
other => vec![("result".into(), other)],
};
fields.push(("cell".into(), Value::Text(name)));
fields.push(("kernel".into(), Value::Text(kernel)));
fields.push(("cursor".into(), Value::Int(self.cursor as i64)));
Ok(Value::Map(fields))
}
pub fn run_all(&mut self, restart: bool) -> Result<Value, String> {
if restart {
self.restart_all()?;
}
self.cursor = 0;
let mut records = Vec::new();
let mut failed = 0i64;
for name in self.order.clone() {
let record = self.run_cell(&name, None)?;
let ok = match &record {
Value::Map(m) => matches!(
m.iter().find(|(k, _)| k == "ok").map(|(_, v)| v),
Some(Value::Bool(true))
),
_ => false,
};
if !ok {
failed += 1;
}
let mut m = match record {
Value::Map(m) => m,
other => vec![("result".into(), other)],
};
m.push(("cell".into(), Value::Text(name)));
records.push(Value::Map(m));
self.cursor += 1;
}
Ok(Value::Map(vec![
("ok".into(), Value::Bool(failed == 0)),
("ran".into(), Value::Int(self.order.len() as i64)),
("failed".into(), Value::Int(failed)),
("records".into(), Value::List(records)),
]))
}
pub fn restart_kernel(&mut self, name: &str) -> Result<Value, String> {
if let Some(id) = self.kernel_sessions.remove(name) {
let _ = foreign_session::close(&mut self.host, &Value::Text(id));
}
for c in &mut self.cells {
if c.kernel == name {
c.status = CellStatus::Never;
c.last_error.clear();
}
}
Ok(Value::Bool(true))
}
pub fn restart_all(&mut self) -> Result<Value, String> {
let names: Vec<String> = self.kernel_sessions.keys().cloned().collect();
for name in names {
self.restart_kernel(&name)?;
}
self.cursor = 0;
for c in &mut self.cells {
c.status = if c.listed { CellStatus::Never } else { CellStatus::Unlisted };
}
Ok(Value::Bool(true))
}
pub fn assign_kernel(&mut self, cell: &str, kernel: &str) -> Result<Value, String> {
let ci = self
.cell_index(cell)
.ok_or_else(|| format!("unknown cell `{cell}`"))?;
let decl = self
.kernel(kernel)
.ok_or_else(|| format!("unknown kernel `{kernel}`"))?
.clone();
if registry::canonical(&self.cells[ci].lang) != registry::canonical(&decl.lang) {
return Err(format!(
"cell `{cell}` ({}) cannot use kernel `{kernel}` ({})",
self.cells[ci].lang, decl.lang
));
}
self.cells[ci].kernel = kernel.to_string();
self.cells[ci].listed = true;
self.cells[ci].status = CellStatus::Dirty;
Ok(Value::None)
}
pub fn state_list(&mut self) -> Result<Value, String> {
let marqdo = self
.value_store
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect::<Vec<_>>();
let mut kernels = Vec::new();
for k in &self.kernels {
let live = self.kernel_sessions.get(&k.name).cloned();
let mut vars = Vec::new();
if let Some(id) = live {
if self.host.foreign_sessions.contains_key(&id) {
if let Ok(names) = foreign_session::names(&mut self.host, &Value::Text(id)) {
if let Value::List(xs) = names {
vars = xs;
}
}
}
}
kernels.push(Value::Map(vec![
("name".into(), Value::Text(k.name.clone())),
("lang".into(), Value::Text(k.lang.clone())),
("vars".into(), Value::List(vars)),
]));
}
Ok(Value::Map(vec![
("marqdo".into(), Value::Map(marqdo)),
("kernels".into(), Value::List(kernels)),
]))
}
pub fn push_value(&mut self, kernel: &str, name: &str, value: Value) -> Result<Value, String> {
let id = self.ensure_kernel_session(kernel)?;
foreign_session::set(
&mut self.host,
&Value::Text(id),
&Value::Text(name.to_string()),
&value,
)?;
self.value_store.insert(name.to_string(), value);
Ok(Value::None)
}
pub fn pull_value(&mut self, kernel: &str, name: &str) -> Result<Value, String> {
let id = self.ensure_kernel_session(kernel)?;
let value = foreign_session::get(
&mut self.host,
&Value::Text(id),
&Value::Text(name.to_string()),
)?;
self.value_store
.insert(name.to_string(), value.clone());
Ok(value)
}
pub fn summary(&self) -> Value {
let cells = self
.cells
.iter()
.map(|c| {
Value::Map(vec![
("name".into(), Value::Text(c.name.clone())),
("lang".into(), Value::Text(c.lang.clone())),
("kernel".into(), Value::Text(c.kernel.clone())),
("listed".into(), Value::Bool(c.listed)),
("status".into(), Value::Text(c.status.as_str().into())),
("line_start".into(), Value::Int(c.line_start as i64)),
("line_end".into(), Value::Int(c.line_end as i64)),
("runs".into(), Value::Int(c.runs as i64)),
("last_ms".into(), Value::Int(c.last_ms as i64)),
("last_ok".into(), Value::Bool(c.last_ok)),
("last_error".into(), Value::Text(c.last_error.clone())),
])
})
.collect();
let kernels = self
.kernels
.iter()
.map(|k| {
Value::Map(vec![
("name".into(), Value::Text(k.name.clone())),
("lang".into(), Value::Text(k.lang.clone())),
("implicit".into(), Value::Bool(k.implicit)),
(
"live".into(),
Value::Bool(self.kernel_sessions.contains_key(&k.name)),
),
])
})
.collect();
Value::Map(vec![
("title".into(), Value::Text(self.title())),
("path".into(), Value::Text(self.path.display().to_string())),
("cells".into(), Value::List(cells)),
("kernels".into(), Value::List(kernels)),
("order".into(), Value::List(
self.order.iter().cloned().map(Value::Text).collect(),
)),
("cursor".into(), Value::Int(self.cursor as i64)),
("journal".into(), Value::List(
self.journal
.iter()
.map(|j| {
Value::Map(vec![
("seq".into(), Value::Int(j.seq as i64)),
("cell".into(), Value::Text(j.cell.clone())),
("kernel".into(), Value::Text(j.kernel.clone())),
("lang".into(), Value::Text(j.lang.clone())),
("ok".into(), Value::Bool(j.ok)),
("code".into(), Value::Int(j.code)),
("ms".into(), Value::Int(j.ms as i64)),
("timestamp_ms".into(), Value::Int(j.timestamp_ms)),
("error".into(), Value::Text(j.error.clone())),
])
})
.collect(),
)),
("opened_ms".into(), Value::Int(self.opened_ms)),
])
}
pub fn journal_json(&self) -> Value {
Value::List(
self.journal
.iter()
.map(|j| {
Value::Map(vec![
("seq".into(), Value::Int(j.seq as i64)),
("cell".into(), Value::Text(j.cell.clone())),
("kernel".into(), Value::Text(j.kernel.clone())),
("lang".into(), Value::Text(j.lang.clone())),
("ok".into(), Value::Bool(j.ok)),
("code".into(), Value::Int(j.code)),
("ms".into(), Value::Int(j.ms as i64)),
("timestamp_ms".into(), Value::Int(j.timestamp_ms)),
("error".into(), Value::Text(j.error.clone())),
])
})
.collect(),
)
}
pub fn close(&mut self) {
let ids: Vec<String> = self.kernel_sessions.values().cloned().collect();
for id in ids {
let _ = foreign_session::close(&mut self.host, &Value::Text(id));
}
self.kernel_sessions.clear();
}
}
fn resolve_kernel(
kernels: &mut Vec<KernelDecl>,
map: &mut HashMap<String, KernelDecl>,
name: &str,
) -> Result<KernelDecl, String> {
if let Some(k) = map.get(name) {
return Ok(k.clone());
}
if registry::preset_spec(name).is_some() {
let lang = registry::canonical(name);
let decl = KernelDecl {
name: lang.clone(),
lang,
cmd: None,
implicit: true,
};
map.insert(decl.name.clone(), decl.clone());
kernels.push(decl.clone());
return Ok(decl);
}
Err(format!(
"会话表: kernel `{name}` is not declared in 内核表"
))
}
fn ensure_default_kernel(
kernels: &mut Vec<KernelDecl>,
map: &mut HashMap<String, KernelDecl>,
lang: &str,
) -> String {
let lang = registry::canonical(lang);
let name = format!("default:{lang}");
if !map.contains_key(&name) {
let decl = KernelDecl {
name: name.clone(),
lang,
cmd: None,
implicit: true,
};
map.insert(name.clone(), decl.clone());
kernels.push(decl);
}
name
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn rows_parse() {
assert_eq!(split_row("| a | b |"), vec!["a", "b"]);
assert_eq!(split_row("| `a` | b |"), vec!["a", "b"]);
}
#[test]
fn table_after_finds_rows() {
let src = "# main\n\n`内核表` =\n\n| 名称 | 语言 |\n|---|---|\n| 分析 | python |\n";
let rows = table_after(src, "内核表").unwrap().unwrap();
assert_eq!(rows.len(), 1);
assert_eq!(rows[0].get("名称").map(|s| s.as_str()), Some("分析"));
}
fn node_available() -> bool {
std::process::Command::new("node")
.args(["-e", "process.exit(0)"])
.output()
.map(|o| o.status.success())
.unwrap_or(false)
}
fn field(record: &Value, key: &str) -> Option<Value> {
match record {
Value::Map(entries) => entries
.iter()
.find(|(k, _)| k == key)
.map(|(_, v)| v.clone()),
_ => None,
}
}
#[test]
fn shared_state_and_isolation() {
if !node_available() {
return;
}
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let path = root.join(format!(
"target/notebook-test-{}-{}.mq.md",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
let source = r#"# main
`内核表` =
| 名称 | 语言 | 命令 | 说明 |
|------|------|------|------|
| 分析 | node | | shared |
| 绘图 | node | | isolated |
`会话表` =
| 单元 | 内核 | 说明 |
|------|------|------|
| 设置 | 分析 | write |
| 读取 | 分析 | read |
| 隔离 | 绘图 | isolated |
`设置` =
```node
let shared = 41;
```
`读取` =
```node
console.log(shared + 1);
```
`隔离` =
```node
console.log(typeof shared === "undefined" ? "isolated" : "leaked");
```
"#;
std::fs::write(&path, source).unwrap();
let mut s = DocumentSession::open(&path, &root).unwrap();
assert_eq!(
s.order,
vec!["设置".to_string(), "读取".to_string(), "隔离".to_string()]
);
let r = s.run_cell("设置", None).unwrap();
assert_eq!(field(&r, "ok"), Some(Value::Bool(true)));
let r = s.run_cell("读取", None).unwrap();
assert_eq!(
field(&r, "stdout").and_then(|v| v.reveal_text().map(|s| s.to_string())),
Some("42\n".to_string())
);
let r = s.run_cell("隔离", None).unwrap();
assert_eq!(
field(&r, "stdout").and_then(|v| v.reveal_text().map(|s| s.to_string())),
Some("isolated\n".to_string())
);
s.restart_all().unwrap();
let r = s.run_cell("读取", None).unwrap();
assert_eq!(field(&r, "ok"), Some(Value::Bool(false)));
s.close();
let _ = std::fs::remove_file(&path);
}
#[test]
fn reload_marks_changed_cell_dirty() {
if !node_available() {
return;
}
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let path = root.join(format!(
"target/notebook-reload-{}-{}.mq.md",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
let src = "# main\n\n`a` =\n```node\nconsole.log(1);\n```\n";
std::fs::write(&path, src).unwrap();
let mut s = DocumentSession::open(&path, &root).unwrap();
let r = s.run_cell("a", None).unwrap();
assert_eq!(field(&r, "ok"), Some(Value::Bool(true)));
assert_eq!(s.cells[0].status, CellStatus::Clean);
let src2 = "# main\n\n`a` =\n```node\nconsole.log(2);\n```\n";
std::fs::write(&path, src2).unwrap();
s.reload(&root).unwrap();
assert_eq!(s.cells[0].status, CellStatus::Dirty);
assert!(s.cells[0].source.contains("console.log(2)"));
s.close();
let _ = std::fs::remove_file(&path);
}
}
+15 -2
View File
@@ -205,9 +205,22 @@ pub fn build_view_annots(module: &Module, source: &str, cap: &RunCapture) -> Vec
last_ret.insert((ev.line, ev.call_id, ev.loop_id, ev.iter), ev);
}
}
let call_sites: HashMap<u32, u32> = cap
.view_trace
.calls
.iter()
.map(|fr| (fr.call_id, fr.call_line))
.collect();
for ((line, _, _, _), ev) in last_ret {
// Return events from an imported library carry the callee's line
// number. Anchor the chip at the caller's line instead.
let anchor_line = call_sites
.get(&ev.call_id)
.copied()
.filter(|l| *l > 0)
.unwrap_or(line);
let text = lines
.get(line.saturating_sub(1) as usize)
.get(anchor_line.saturating_sub(1) as usize)
.copied()
.unwrap_or("");
let mut start = None;
@@ -222,7 +235,7 @@ pub fn build_view_annots(module: &Module, source: &str, cap: &RunCapture) -> Vec
out.push(ViewAnnot {
id: next_id("r"),
kind: AnnotKind::Return,
anchor_line: line,
anchor_line,
anchor_byte_start: start,
anchor_byte_end: end,
label: "return".into(),
+18 -5
View File
@@ -101,11 +101,18 @@ pub fn render_module_document(module: &Module, source: &str, annots: &[ViewAnnot
}
LineKind::Writeback => {
flush_para(&mut para, &mut out);
let start_line = l.line_no;
let mut block = String::new();
while i < lines.len() && matches!(lines[i].kind, LineKind::Writeback) {
if !block.is_empty() {
block.push('\n');
}
block.push_str(lines[i].text.trim_end_matches(['\r', '\n']));
i += 1;
}
out.push_str(&crate::view::render::render_writeback_source_line(
l.line_no,
l.text.trim_end_matches(['\r', '\n']),
start_line, &block,
));
i += 1;
}
LineKind::Comment => {
let text = l.text.trim_end_matches(['\r', '\n']);
@@ -335,8 +342,14 @@ fn render_table_block(
html.push_str("</table>");
// Annots on first table line (bind of collection).
if let Some(list) = by_line.get(&first_line) {
if !list.is_empty() {
let owned: Vec<ViewAnnot> = list.iter().map(|a| (*a).clone()).collect();
// A table is already its own readable value; a call chip would collapse
// the whole collection into an ellipsis. Keep only non-call chips.
let owned: Vec<ViewAnnot> = list
.iter()
.filter(|a| a.kind != crate::view::annot::AnnotKind::Call)
.map(|a| (*a).clone())
.collect();
if !owned.is_empty() {
html.push_str(&render_annot_band(&owned, MAX_VISIBLE_ANNOTS));
}
}
+372 -56
View File
@@ -625,6 +625,24 @@ ul.doc-loop, ol.doc-branch {
border-radius: 10px;
border: 1px solid transparent;
}
.exec-clear {
float: right;
font: inherit;
font-size: 0.72rem;
padding: 0.2rem 0.6rem;
border: 1px solid var(--line);
border-radius: 999px;
background: var(--surface);
color: var(--muted);
cursor: pointer;
}
.exec-clear:hover { color: var(--ink); border-color: var(--muted); }
body.notebook-page .exec-clear {
border-color: var(--nb-border);
background: var(--nb-surface-2);
color: var(--nb-text);
}
body.notebook-page .exec-clear:hover { border-color: var(--nb-accent); color: var(--nb-strong); }
.out, .source {
font-family: var(--mono);
font-size: 0.82rem;
@@ -1021,20 +1039,18 @@ ul.doc-loop, ol.doc-branch {
}
.doc-writeback {
margin: 0.35rem 0;
padding: 0.4rem 0.55rem;
padding: 0.45rem 0.6rem;
border-radius: 8px;
border: 1px dashed #94a3b8;
background: #f1f5f9;
display: flex;
flex-wrap: wrap;
gap: 0.4rem 0.65rem;
align-items: baseline;
}
.doc-writeback[data-wb-kind="image"] {
border-color: #0d9488;
background: #f0fdfa;
}
.wb-source-badge {
display: inline-block;
margin-bottom: 0.28rem;
font: 0.68rem/1.2 var(--sans);
letter-spacing: 0.04em;
text-transform: uppercase;
@@ -1048,10 +1064,11 @@ ul.doc-loop, ol.doc-branch {
color: #115e59;
}
.wb-source-text {
display: block;
font: 0.82rem/1.4 var(--mono);
color: #0f172a;
white-space: pre-wrap;
word-break: break-all;
word-break: break-word;
}
.wb-outline {
margin-top: 0.75rem;
@@ -1645,15 +1662,7 @@ pub fn layout(title: &str, nav: &str, main: &str) -> String {
var body = card.querySelector(".code-body code");
// Narrative ```text``` cards have no Run chrome — leave as-is.
if (!btn) return;
if (!live) {{
// Static: hide fake Run (view-static-interactive D5 / view-prose-annot-gaps P4).
if (foot) foot.hidden = true;
else {{
btn.hidden = true;
if (cmd) cmd.hidden = true;
}}
return;
}}
if (!live) return; // static actions are bound to the view-shell notice
if (!out || !body) return;
btn.addEventListener("click", function () {{
btn.disabled = true;
@@ -3050,6 +3059,33 @@ body.notebook-page #notebook-bar strong {
color: var(--nb-strong);
font: 600 0.76rem/1.6 var(--sans);
}
body.notebook-page .nb-stdin-bar {
display: flex;
align-items: center;
gap: 0.55rem;
margin: 0 0 0.75rem;
padding: 0.4rem 0.55rem;
border: 1px solid var(--nb-border);
border-radius: var(--nb-radius);
background: var(--nb-surface);
}
body.notebook-page .nb-stdin-bar label {
color: var(--nb-muted);
font: 0.7rem/1.2 var(--mono);
white-space: nowrap;
}
body.notebook-page .nb-stdin-bar textarea {
flex: 1;
min-height: 2.1rem;
max-height: 7rem;
resize: vertical;
border: 1px solid var(--nb-border);
border-radius: 6px;
padding: 0.3rem 0.5rem;
background: var(--nb-bg);
color: var(--nb-text);
font: 0.72rem/1.35 var(--mono);
}
body.notebook-page #notebook-bar button {
appearance: none;
padding: 0.34rem 0.58rem;
@@ -3233,16 +3269,6 @@ body.notebook-page .code-head {
background: var(--nb-surface-2);
}
body.notebook-page .code-head .badge { display: none; }
body.notebook-page .nb-prompt {
order: 0;
min-width: 3.2rem;
padding-right: 0.5rem;
border-right: 1px solid var(--nb-border);
color: var(--nb-muted);
font: 0.66rem/1.3 var(--mono);
text-align: right;
white-space: nowrap;
}
body.notebook-page .code-name {
order: 1;
color: var(--nb-strong);
@@ -3264,8 +3290,46 @@ body.notebook-page .code-head .nb-kernel {
font: 0.66rem/1.2 var(--sans);
cursor: pointer;
}
body.notebook-page .code-head .nb-badge {
body.notebook-page .code-head .nb-cell-stdin {
order: 4;
width: 8rem;
padding: 0.16rem 0.3rem;
border: 1px solid var(--nb-border);
border-radius: 3px;
background: var(--nb-bg);
color: var(--nb-text);
font: 0.62rem/1.2 var(--mono);
}
body.notebook-page .code-head .nb-persist {
order: 6;
padding: 0.14rem 0.4rem;
border: 1px solid var(--nb-border);
border-radius: 3px;
background: var(--nb-bg);
color: var(--nb-muted);
font: 0.62rem/1.2 var(--sans);
cursor: pointer;
}
body.notebook-page .code-head .nb-persist:hover {
border-color: var(--nb-accent);
color: var(--nb-strong);
}
body.notebook-page .code-head .nb-clean {
order: 5;
padding: 0.14rem 0.4rem;
border: 1px solid var(--nb-border);
border-radius: 3px;
background: var(--nb-bg);
color: var(--nb-muted);
font: 0.62rem/1.2 var(--sans);
cursor: pointer;
}
body.notebook-page .code-head .nb-clean:hover {
border-color: var(--nb-accent);
color: var(--nb-strong);
}
body.notebook-page .code-head .nb-badge {
order: 7;
padding: 0.14rem 0.34rem;
border: 1px solid var(--nb-border);
border-radius: 3px;
@@ -3277,7 +3341,7 @@ body.notebook-page .code-head .nb-badge {
body.notebook-page .nb-badge.ok { color: var(--nb-success); }
body.notebook-page .nb-badge.bad { color: var(--nb-error); }
body.notebook-page .code-head .code-actions {
order: 5;
order: 8;
display: flex;
align-items: center;
gap: 0.3rem;
@@ -3350,7 +3414,7 @@ body.notebook-page .code-out {
word-break: break-word;
}
body.notebook-page .code-out::before {
content: "Out [ ]";
content: "Output";
display: block;
margin-bottom: 0.38rem;
color: var(--nb-muted);
@@ -3457,6 +3521,34 @@ body.notebook-page .stdin-form button {
color: #fff;
}
body.notebook-page .output-card {
background: var(--nb-surface-2);
border-left-color: var(--nb-accent);
}
body.notebook-page .output-card.output-card-image { border-left-color: var(--nb-success); }
body.notebook-page .output-card .wb-badges { color: var(--nb-muted); }
body.notebook-page .output-card .output-body {
color: var(--nb-text);
background: var(--nb-bg);
}
body.notebook-page .wb-badge {
background: var(--nb-surface-3);
color: var(--nb-text);
border-color: var(--nb-border);
}
body.notebook-page .wb-badge.wb-ok { background: #143d2b; color: #86efac; }
body.notebook-page .wb-badge.wb-error { background: #4a1d1d; color: #fca5a5; }
body.notebook-page .wb-badge.wb-trace { background: #2e2a5a; color: #c7d2fe; }
body.notebook-page .doc-writeback {
background: var(--nb-surface-2);
border-color: var(--nb-border);
}
body.notebook-page .wb-source-badge {
background: var(--nb-surface-3);
color: var(--nb-muted);
}
body.notebook-page .wb-source-text { color: var(--nb-text); }
/* ── Narrow preview panes ──────────────────────────────────────────── */
@media (max-width: 760px) {
body.notebook-page main.panel { padding: 1rem 0.75rem 3.5rem; }
@@ -3470,17 +3562,23 @@ body.notebook-page .stdin-form button {
}
</style>
<div class="notebook-bar" id="notebook-bar" hidden>
<strong>Notebook</strong>
<strong>View</strong>
<button type="button" id="nb-run-all">Run All</button>
<button type="button" id="nb-restart">Restart &amp; Run All</button>
<button type="button" id="nb-step">Step</button>
<button type="button" id="nb-interrupt">Interrupt</button>
<button type="button" id="nb-state">State</button>
<button type="button" id="nb-journal">Journal</button>
<button type="button" id="nb-clear">Clear outputs</button>
<button type="button" id="nb-reload">Reload</button>
<button type="button" id="nb-debug">Debug</button>
<button type="button" id="nb-theme" title="切换浅色/深色主题">浅色</button>
<span id="nb-status">opening…</span>
</div>
<div class="nb-stdin-bar" id="nb-stdin-bar">
<label for="nb-stdin">Cell stdin / input</label>
<textarea id="nb-stdin" rows="1" placeholder="raw stdin for stateless cells; JSON/string for persistent kernels" spellcheck="false"></textarea>
</div>
<div class="notebook-bar" id="nb-state-actions" hidden>
<span>Bridge</span>
<input id="nb-kernel-input" placeholder="kernel" autocomplete="off"/>
@@ -3508,10 +3606,10 @@ body.notebook-page .stdin-form button {
var savedTheme = "dark";
try { savedTheme = localStorage.getItem(THEME_KEY) || "dark"; } catch (e) {}
applyTheme(savedTheme);
if (!window.MARQDO_VIEW_LIVE) return;
var live = !!window.MARQDO_VIEW_LIVE;
var path = window.MARQDO_FILE_PATH || "";
if (!path) return;
var state = { open: false, kernels: [], cells: {} };
var state = { open: false, persistent: false, kernels: [], cells: {} };
function post(url, body) {
return fetch(url, {
method: "POST",
@@ -3520,6 +3618,10 @@ body.notebook-page .stdin-form button {
}).then(function (r) { return r.json(); });
}
function status(t) { var e = document.getElementById("nb-status"); if (e) e.textContent = t; }
function cellStdin() {
var el = document.getElementById("nb-stdin");
return el ? el.value : "";
}
function card(name) {
var els = document.querySelectorAll(".code-card");
for (var i = 0; i < els.length; i++) {
@@ -3539,6 +3641,90 @@ body.notebook-page .stdin-form button {
b.textContent = text;
b.className = "nb-badge " + (ok ? "ok" : (text === "clean" || text === "never" ? "" : "bad"));
}
function ensureKernel(el, persistent) {
var sel = el.querySelector(".nb-kernel");
if (!sel) {
sel = document.createElement("select");
sel.className = "nb-kernel";
var head = el.querySelector(".code-head");
if (head) head.appendChild(sel);
}
if (!persistent) {
sel.disabled = true;
sel.title = "No live kernel; this cell is stateless";
sel.innerHTML = "";
var opt = document.createElement("option");
opt.textContent = "one-shot";
sel.appendChild(opt);
} else {
sel.disabled = false;
sel.title = "Switch kernel";
}
return sel;
}
function ensureCellStdin(el, c) {
var inp = el.querySelector(".nb-cell-stdin");
if (!inp) {
inp = document.createElement("input");
inp.className = "nb-cell-stdin";
inp.placeholder = "stdin";
inp.autocomplete = "off";
var head = el.querySelector(".code-head");
if (head) head.appendChild(inp);
}
if (c && inp.dataset.loaded !== "1") {
inp.value = c.stdin || "";
inp.dataset.loaded = "1";
}
return inp;
}
function clearCard(el) {
var out = el.querySelector(".code-out");
if (out) {
out.hidden = true;
out.textContent = "";
out.classList.remove("err");
}
var b = el.querySelector(".nb-badge");
if (b) { b.textContent = "never"; b.className = "nb-badge"; }
}
function bindClean(el) {
var btn = el.querySelector(".nb-clean");
if (!btn || btn.dataset.bound) return;
btn.dataset.bound = "1";
btn.addEventListener("click", function (e) {
e.preventDefault();
var name = el.getAttribute("data-code-name") || "";
if (!live) {
clearCard(el);
status("static export: local output cleared");
return;
}
if (!name) return;
post("api/session/cell/clear", { cell: name }).then(function (r) {
clearCard(el);
if (r.ok && r.session) renderSummary(r.session);
status(r.ok ? "clean" : (r.error || "clean failed"));
}).catch(function (e2) { status(String(e2)); });
});
}
function bindPersist(el) {
var btn = el.querySelector(".nb-persist");
if (!btn || btn.dataset.bound) return;
btn.dataset.bound = "1";
btn.addEventListener("click", function (e) {
e.preventDefault();
var name = el.getAttribute("data-code-name") || "";
if (!name) return;
if (!live) {
status("static export: persist is unavailable");
return;
}
post("api/session/cell/persist", { cell: name, key: name }).then(function (r) {
status(r.ok ? "persisted " + name : (r.error || "persist failed"));
}).catch(function (e2) { status(String(e2)); });
});
}
function renderRecord(name, rec) {
var el = card(name);
if (!el || !rec) return;
@@ -3562,27 +3748,18 @@ body.notebook-page .stdin-form button {
}
function renderSummary(s) {
if (!s) return;
state.persistent = !!s.persistent;
state.kernels = s.kernels || [];
state.cells = {};
(s.cells || []).forEach(function (c) { state.cells[c.name] = c; });
(s.cells || []).forEach(function (c) {
var el = card(c.name);
if (!el) return;
var head0 = el.querySelector(".code-head");
var prompt = el.querySelector(".nb-prompt");
if (!prompt && head0) {
prompt = document.createElement("span");
prompt.className = "nb-prompt";
head0.insertBefore(prompt, head0.firstChild);
}
if (prompt) prompt.textContent = "In [" + (c.runs ? c.runs : " ") + "]";
badge(el, c.status, c.status === "clean");
var sel = el.querySelector(".nb-kernel");
if (!sel) {
sel = document.createElement("select");
sel.className = "nb-kernel";
var head = el.querySelector(".code-head");
if (head) head.appendChild(sel);
var sel = ensureKernel(el, state.persistent);
if (!state.persistent) return;
if (!sel.dataset.bound) {
sel.dataset.bound = "1";
sel.addEventListener("change", function () {
post("api/session/kernel/assign", { cell: c.name, kernel: sel.value }).then(function (r) {
status(r.ok ? "kernel: " + sel.value : (r.error || "assign failed"));
@@ -3600,6 +3777,35 @@ body.notebook-page .stdin-form button {
});
});
}
function renderStaticSummary(s) {
if (!s) return;
state.persistent = !!s.persistent;
state.kernels = s.kernels || [];
state.cells = {};
(s.cells || []).forEach(function (c) { state.cells[c.name] = c; });
(s.cells || []).forEach(function (c) {
var el = card(c.name);
if (!el) return;
badge(el, c.status, c.status === "clean");
ensureCellStdin(el, c);
if (!state.persistent) {
ensureKernel(el, false);
return;
}
var sel = ensureKernel(el, true);
var opts = state.kernels.filter(function (k) { return k.lang === c.lang; }).map(function (k) { return k.name; });
if (!opts.length) opts = [c.kernel];
sel.innerHTML = "";
opts.forEach(function (n) {
var o = document.createElement("option");
o.value = n; o.textContent = n;
if (n === c.kernel) o.selected = true;
sel.appendChild(o);
});
sel.disabled = true;
sel.title = "Static export: kernel fixed to " + c.kernel;
});
}
function openSession() {
var bar = document.getElementById("notebook-bar");
if (bar) bar.hidden = false;
@@ -3611,14 +3817,39 @@ body.notebook-page .stdin-form button {
});
}
function runOne(name) {
return post("api/session/cell/run", { cell: name }).then(function (r) {
var inputEl = document.querySelector('.code-card[data-code-name="' + name + '"] .nb-cell-stdin');
var stdinValue = inputEl && inputEl.value ? inputEl.value : cellStdin();
return post("api/session/cell/run", { cell: name, stdin: stdinValue }).then(function (r) {
if (r.ok) { renderRecord(name, r.record); if (r.session) renderSummary(r.session); }
else status(r.error || "run failed");
});
}
function renderDocumentResult(data) {
var out = document.getElementById("exec-main");
if (out) {
out.hidden = false;
out.classList.toggle("err", !data.ok);
out.textContent = data.ok ? (data.stdout || "(no output)") : (data.error || "run failed");
}
var block = document.getElementById("exec-block");
if (block) block.hidden = false;
}
function runAll(restart) {
status(restart ? "restarting…" : "running all…");
post("api/session/run_all", { restart: !!restart }).then(function (r) {
if (!state.persistent && Object.keys(state.cells).length === 0) {
post("api/session/run_document", {}).then(function (r) {
renderDocumentResult(r);
status(r.ok ? "document run done" : (r.error || "document run failed"));
}).catch(function (e) { status(String(e)); });
return;
}
var perCell = {};
document.querySelectorAll(".code-card").forEach(function (el) {
var inp = el.querySelector(".nb-cell-stdin");
var name = el.getAttribute("data-code-name");
if (inp && name && inp.value) perCell[name] = inp.value;
});
post("api/session/run_all", { restart: !!restart, stdin: cellStdin(), inputs: perCell }).then(function (r) {
if (!r.ok) { status(r.error || "run all failed"); return; }
if (r.session) renderSummary(r.session);
var recs = (r.summary && r.summary.records) || [];
@@ -3626,6 +3857,37 @@ body.notebook-page .stdin-form button {
status("run all done");
});
}
function clearDom() {
document.querySelectorAll(".code-out").forEach(function (el) {
el.hidden = true;
el.textContent = "";
el.classList.remove("err");
});
document.querySelectorAll(".nb-badge").forEach(function (el) {
el.textContent = "never";
el.className = "nb-badge";
});
var exec = document.getElementById("exec-main");
if (exec) { exec.hidden = true; exec.textContent = ""; }
var execBlock = document.getElementById("exec-block");
if (execBlock) {
execBlock.querySelectorAll(".plots, .plan-card").forEach(function (el) {
el.hidden = true;
el.innerHTML = "";
});
}
var stateOut = document.getElementById("nb-state-out");
if (stateOut) { stateOut.hidden = true; stateOut.textContent = ""; }
var pullOut = document.getElementById("nb-pull-out");
if (pullOut) pullOut.textContent = "";
}
function clearOutputs() {
clearDom();
post("api/session/clear", {}).then(function (r) {
if (r.ok && r.session) renderSummary(r.session);
status(r.ok ? "outputs cleared" : (r.error || "clear failed"));
}).catch(function (e) { status(String(e)); });
}
function bind(id, fn) { var b = document.getElementById(id); if (b) b.addEventListener("click", fn); }
function init() {
var themeBtn = document.getElementById("nb-theme");
@@ -3646,10 +3908,55 @@ body.notebook-page .stdin-form button {
foot.remove();
}
});
if (!live) {
var staticBar = document.getElementById("notebook-bar");
if (staticBar) staticBar.hidden = false;
var staticStdin = document.getElementById("nb-stdin-bar");
if (staticStdin) staticStdin.hidden = false;
function staticNotice() {
status("静态页面无法真实执行 / static export: no live execution");
}
["nb-run-all", "nb-restart", "nb-step", "nb-interrupt", "nb-state", "nb-journal",
"nb-clear", "nb-reload", "nb-debug", "exec-clear"].forEach(function (id) {
bind(id, staticNotice);
});
document.querySelectorAll(".code-run").forEach(function (btn) {
btn.addEventListener("click", function (e) {
e.preventDefault();
staticNotice();
});
});
if (window.MARQDO_STATIC_SESSION) {
renderStaticSummary(window.MARQDO_STATIC_SESSION);
} else {
document.querySelectorAll(".code-card").forEach(function (el) {
ensureKernel(el, false);
});
}
document.querySelectorAll(".code-card").forEach(function (el) {
bindClean(el);
bindPersist(el);
});
status("static export · preview only");
return;
}
document.querySelectorAll(".code-card").forEach(function (el) {
bindClean(el);
bindPersist(el);
});
bind("nb-run-all", function () { runAll(false); });
bind("nb-restart", function () { runAll(true); });
bind("nb-interrupt", function () {
post("api/session/interrupt", {}).then(function (r) {
status(r.ok ? "interrupt sent" : (r.error || "interrupt failed"));
});
});
bind("nb-step", function () {
post("api/session/cell/step", {}).then(function (r) {
if (!state.persistent && Object.keys(state.cells).length === 0) {
status("no cells; Run All runs the document");
return;
}
post("api/session/cell/step", { stdin: cellStdin() }).then(function (r) {
if (!r.ok) { status(r.error || "step failed"); return; }
if (r.record) {
if (r.record.cell) renderRecord(r.record.cell, r.record);
@@ -3669,6 +3976,7 @@ body.notebook-page .stdin-form button {
pre.textContent = JSON.stringify(r.state || r, null, 2);
if (actions) actions.hidden = false;
if (ki && !ki.value && state.kernels.length) ki.value = state.kernels[0].name;
if (!state.persistent) status("stateless: no kernel state");
});
});
bind("nb-push", function () {
@@ -3700,11 +4008,14 @@ body.notebook-page .stdin-form button {
if (!pre) return;
pre.hidden = false;
pre.textContent = JSON.stringify(r.journal || r, null, 2);
if (!state.persistent) status("stateless: journal is per one-shot run");
});
});
bind("nb-debug", function () {
window.open("debug?path=" + encodeURIComponent(path), "_blank");
});
bind("nb-clear", clearOutputs);
bind("exec-clear", clearOutputs);
bind("nb-reload", function () {
post("api/session/reload", {}).then(function (r) {
if (!r.ok) { status(r.error || "reload failed"); return; }
@@ -3832,6 +4143,7 @@ pub fn page_file(files: &[PathBuf], rel: &str, vm: &FileViewModel, links: &LinkM
LinkMode::Live => "true",
LinkMode::Static { .. } => "false",
};
let exec_clear = r#"<button type="button" class="exec-clear" id="exec-clear">Clear</button>"#;
let outline_block = if vm.outline_html.is_empty() {
String::new()
} else {
@@ -3872,11 +4184,9 @@ pub fn page_file(files: &[PathBuf], rel: &str, vm: &FileViewModel, links: &LinkM
let trace_safe = vm
.trace_json
.replace("</", "<\\/");
let notebook = if matches!(links, LinkMode::Live) {
notebook_ui().to_string()
} else {
String::new()
};
// Static exports share the same view shell. The script sees
// MARQDO_VIEW_LIVE=false and leaves the interactive controls hidden.
let notebook = notebook_ui().to_string();
let main = format!(
r#"
<script>
@@ -3892,12 +4202,12 @@ window.MARQDO_AUTO_STREAM = {auto_stream};
<h2>Structure</h2>
{structure_block}
</section>
<section class="block exec-block">
<h2>Execution</h2>
<section class="block exec-block" id="exec-block">
<h2>Execution {exec_clear}</h2>
{stdin_panel}
{stream_panel}
{plan_card}
<div class="out {status}">{out}</div>
<div class="out {status}" id="exec-main">{out}</div>
{plots}
</section>
<section class="block source-block">
@@ -3918,6 +4228,7 @@ window.MARQDO_AUTO_STREAM = {auto_stream};
status_label = status_label,
rel = escape(rel),
notebook = notebook,
exec_clear = exec_clear,
structure_block = structure_block,
stdin_panel = stdin_panel,
stream_panel = stream_panel,
@@ -4140,5 +4451,10 @@ mod tests {
"notebook API URLs must stay relative for path-prefix proxy"
);
assert!(ui.contains("post(\"api/session/open\""));
assert!(!ui.contains("In ["), "view cells must not mimic Jupyter In[] prompts");
assert!(!ui.contains("Out ["), "view cells must not mimic Jupyter Out[] prompts");
assert!(ui.contains("id=\"nb-clear\""), "view must expose Clear outputs");
assert!(ui.contains("api/session/clear"), "Clear outputs must call the session API");
}
}
+61 -1
View File
@@ -10,7 +10,7 @@ mod notebook_api;
mod output;
mod render;
pub use output::{write_static, OutputOptions};
pub use output::{write_capture, write_static, CaptureOptions, OutputOptions};
use std::fs;
use std::io::{self, Read};
@@ -189,6 +189,66 @@ fn handle(root: &ViewRoot, request: Request) -> Result<()> {
notebook_api::api_close(&root.root, body)
})
}
"/api/session/run_document" => {
return api_debug_post(request, |body| {
notebook_api::api_run_document(&root.root, body)
})
}
"/api/session/snapshot/export" => {
return api_debug_post(request, |body| {
notebook_api::api_export_snapshot(&root.root, body)
})
}
"/api/session/snapshot/import" => {
return api_debug_post(request, |body| {
notebook_api::api_import_snapshot(&root.root, body)
})
}
"/api/session/snapshot/replay" => {
return api_debug_post(request, |body| {
notebook_api::api_replay_snapshot(&root.root, body)
})
}
"/api/session/plan/assign" => {
return api_debug_post(request, |body| {
notebook_api::api_plan_assign(&root.root, body)
})
}
"/api/session/plan/add" => {
return api_debug_post(request, |body| {
notebook_api::api_plan_add(&root.root, body)
})
}
"/api/session/plan/remove" => {
return api_debug_post(request, |body| {
notebook_api::api_plan_remove(&root.root, body)
})
}
"/api/session/plan/value" => {
return api_debug_post(request, |body| {
notebook_api::api_plan_value(&root.root, body)
})
}
"/api/session/cell/persist" => {
return api_debug_post(request, |body| {
notebook_api::api_persist_cell(&root.root, body)
})
}
"/api/session/interrupt" => {
return api_debug_post(request, |body| {
notebook_api::api_interrupt(&root.root, body)
})
}
"/api/session/cell/clear" => {
return api_debug_post(request, |body| {
notebook_api::api_clear_cell(&root.root, body)
})
}
"/api/session/clear" => {
return api_debug_post(request, |body| {
notebook_api::api_clear(&root.root, body)
})
}
"/api/session/cell/run" => {
return api_debug_post(request, |body| {
notebook_api::api_cell_run(&root.root, body)
+135 -4
View File
@@ -5,7 +5,7 @@ use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, OnceLock};
use crate::host::json::{json_to_value, value_to_json};
use crate::notebook::DocumentSession;
use crate::host::foreign_doc::DocumentSession;
use crate::value::Value;
fn sessions() -> &'static Mutex<HashMap<String, Arc<Mutex<DocumentSession>>>> {
@@ -67,6 +67,25 @@ fn err(error: String) -> serde_json::Value {
serde_json::json!({ "ok": false, "error": error })
}
pub fn api_run_document(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let (_rel, abs) = match path_of(root, &v) {
Ok(p) => p,
Err(e) => return err(e),
};
let mut opts = crate::RunOptions::default();
opts.fs_root = Some(root.to_path_buf());
opts.sleep_limit_ms = Some(30_000);
match crate::run_file_capture(&abs, &opts) {
Ok(cap) => serde_json::json!({
"ok": true,
"stdout": cap.stdout,
"value": json_value(&cap.value),
}),
Err(e) => serde_json::json!({ "ok": false, "error": format!("{e:#}") }),
}
}
pub fn api_open(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let (_rel, abs) = match path_of(root, &v) {
@@ -151,24 +170,84 @@ where
}
}
pub fn api_persist_cell(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
let key = v.get("key").and_then(|x| x.as_str()).unwrap_or("");
if cell.is_empty() {
return err("missing cell".into());
}
with_session(root, body, "persisted", |s| s.persist_cell(cell, key))
}
pub fn api_interrupt(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "interrupted", |s| s.interrupt())
}
pub fn api_export_snapshot(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "snapshot", |s| s.export_snapshot())
}
pub fn api_import_snapshot(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let snapshot = v.get("snapshot").cloned().unwrap_or(serde_json::Value::Null);
let snapshot = match json_to_value(&snapshot) {
Ok(v) => v,
Err(e) => return err(e),
};
with_session(root, body, "restored", |s| s.import_snapshot(&snapshot))
}
pub fn api_replay_snapshot(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let snapshot = v.get("snapshot").cloned().unwrap_or(serde_json::Value::Null);
let snapshot = match json_to_value(&snapshot) {
Ok(v) => v,
Err(e) => return err(e),
};
with_session(root, body, "replay", |s| s.replay_snapshot(&snapshot))
}
pub fn api_clear_cell(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
if cell.is_empty() {
return err("missing cell".into());
}
with_session(root, body, "cleared", |s| s.clear_cell(cell))
}
pub fn api_clear(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "cleared", |s| Ok(s.clear_outputs()))
}
pub fn api_cell_run(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
let kernel = v.get("kernel").and_then(|x| x.as_str()).filter(|s| !s.is_empty());
let stdin = v.get("stdin").and_then(|x| x.as_str());
if cell.is_empty() {
return err("missing cell".into());
}
with_session(root, body, "record", |s| s.run_cell(cell, kernel))
with_session(root, body, "record", |s| s.run_cell_with_stdin(cell, kernel, stdin))
}
pub fn api_cell_step(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "record", |s| s.step())
let v = body_value(body);
let stdin = v.get("stdin").and_then(|x| x.as_str());
with_session(root, body, "record", |s| s.step_with_stdin(stdin))
}
pub fn api_run_all(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let restart = v.get("restart").and_then(|x| x.as_bool()).unwrap_or(false);
with_session(root, body, "summary", |s| s.run_all(restart))
let stdin = v.get("stdin").and_then(|x| x.as_str());
let inputs: Option<HashMap<String, String>> = v.get("inputs").and_then(|x| x.as_object()).map(|m| {
m.iter()
.filter_map(|(k, val)| val.as_str().map(|s| (k.clone(), s.to_string())))
.collect()
});
with_session(root, body, "summary", |s| s.run_all_with_inputs(restart, stdin, inputs.as_ref()))
}
pub fn api_kernel_restart(root: &Path, body: &str) -> serde_json::Value {
@@ -180,6 +259,54 @@ pub fn api_kernel_restart(root: &Path, body: &str) -> serde_json::Value {
with_session(root, body, "value", |s| s.restart_kernel(kernel))
}
pub fn api_plan_assign(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
let kernel = v.get("kernel").and_then(|x| x.as_str()).unwrap_or("");
if cell.is_empty() || kernel.is_empty() {
return err("missing cell/kernel".into());
}
with_session(root, body, "value", |s| s.assign_kernel(cell, kernel))
}
pub fn api_plan_add(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
let lang = v.get("lang").and_then(|x| x.as_str()).unwrap_or("");
let kernel = v.get("kernel").and_then(|x| x.as_str()).unwrap_or("");
let source = v.get("source").and_then(|x| x.as_str()).unwrap_or("");
if cell.is_empty() || lang.is_empty() || kernel.is_empty() {
return err("missing cell/lang/kernel".into());
}
with_session(root, body, "value", |s| s.add_cell(cell, lang, source, kernel))
}
pub fn api_plan_remove(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
if cell.is_empty() {
return err("missing cell".into());
}
with_session(root, body, "value", |s| s.remove_cell(cell))
}
pub fn api_plan_value(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let kernel = v.get("kernel").and_then(|x| x.as_str()).unwrap_or("");
let name = v.get("name").and_then(|x| x.as_str()).unwrap_or("");
if kernel.is_empty() || name.is_empty() {
return err("missing kernel/name".into());
}
let value = match v.get("value") {
Some(v) => match json_to_value(v) {
Ok(v) => v,
Err(e) => return err(e),
},
None => Value::None,
};
with_session(root, body, "value", |s| s.set_value(kernel, name, value))
}
pub fn api_assign(root: &Path, body: &str) -> serde_json::Value {
let v = body_value(body);
let cell = v.get("cell").and_then(|x| x.as_str()).unwrap_or("");
@@ -281,6 +408,10 @@ mod tests {
Some(false),
"{failed:?}"
);
let clear = api_clear(&root, r#"{"path":"shared.mq.md"}"#);
assert_eq!(clear.get("ok").and_then(|v| v.as_bool()), Some(true), "{clear:?}");
let cleared = clear.get("session").and_then(|s| s.get("cells")).and_then(|c| c.as_array()).cloned().unwrap_or_default();
assert!(cleared.iter().all(|c| c.get("runs").and_then(|v| v.as_u64()) == Some(0)));
let push = api_state_push(
&root,
r#"{"path":"shared.mq.md","kernel":"分析","name":"z","value":9}"#,
+328 -7
View File
@@ -1,20 +1,275 @@
//! Static HTML export: `marqdo view output`.
use std::collections::HashMap;
use std::fs;
use std::path::PathBuf;
use anyhow::{Context, Result};
use sha2::{Digest, Sha256};
use crate::view::html::{page_file, page_index, LinkMode};
use crate::host::foreign_doc::DocumentSession;
use crate::host::json::json_to_value;
use crate::value::Value;
use crate::view::html::{escape, page_file, page_index, LinkMode};
use crate::view::{build_file_view, build_root, ViewRoot};
pub struct OutputOptions {
pub path: PathBuf,
pub out_dir: PathBuf,
pub no_exec: bool,
pub snapshot: Option<PathBuf>,
}
pub struct CaptureOptions {
pub path: PathBuf,
pub out: PathBuf,
}
#[derive(Debug, Clone)]
struct StaticCellOutput {
ok: bool,
text: String,
}
fn source_hash(source: &str) -> String {
let mut h = Sha256::new();
h.update(source.as_bytes());
format!("{:x}", h.finalize())
}
fn value_text(v: &Value) -> Option<String> {
match v {
Value::Text(s) => Some(s.clone()),
Value::None => None,
_ => v.reveal_text().map(|s| s.to_string()),
}
}
fn value_bool(v: &Value) -> Option<bool> {
match v {
Value::Bool(b) => Some(*b),
_ => None,
}
}
fn record_field(record: &Value, key: &str) -> Option<Value> {
match record {
Value::Map(entries) => entries
.iter()
.find(|(k, _)| k == key)
.map(|(_, v)| v.clone()),
_ => None,
}
}
/// Run the document's cells at export time and keep their real outputs. This
/// is what makes a static page show foreign results instead of just source.
fn static_page_data(
root: &ViewRoot,
abs: &std::path::Path,
stdin: &str,
) -> (HashMap<String, StaticCellOutput>, Option<Value>) {
let mut out = HashMap::new();
let Ok(mut session) = DocumentSession::open(abs, &root.root) else {
return (out, None);
};
let page_stdin = if stdin.trim().is_empty() { None } else { Some(stdin) };
let Ok(summary) = session.run_all_with_stdin(false, page_stdin) else {
session.close();
return (out, None);
};
let records = match &summary {
Value::Map(entries) => entries
.iter()
.find(|(k, _)| k == "records")
.map(|(_, v)| v.clone()),
_ => None,
};
if let Some(Value::List(records)) = records {
for rec in records {
let Some(cell) = record_field(&rec, "cell").and_then(|v| value_text(&v)) else {
continue;
};
let ok = record_field(&rec, "ok")
.and_then(|v| value_bool(&v))
.unwrap_or(false);
let mut parts = Vec::new();
if let Some(stdout) = record_field(&rec, "stdout").and_then(|v| value_text(&v)) {
if !stdout.is_empty() {
parts.push(stdout);
}
}
if let Some(stderr) = record_field(&rec, "stderr").and_then(|v| value_text(&v)) {
if !stderr.is_empty() {
parts.push(format!("[stderr]\n{stderr}"));
}
}
if let Some(error) = record_field(&rec, "error").and_then(|v| value_text(&v)) {
if !error.is_empty() {
parts.push(error);
}
}
let has_body = !parts.is_empty();
let mut meta = Vec::new();
if let Some(code) = record_field(&rec, "code").and_then(|v| match v {
Value::Int(n) => Some(n),
_ => None,
}) {
if code != 0 {
meta.push(format!("exit {code}"));
}
}
if let Some(ms) = record_field(&rec, "ms").and_then(|v| match v {
Value::Int(n) => Some(n),
_ => None,
}) {
if ms > 0 {
meta.push(format!("{ms}ms"));
}
}
if !meta.is_empty() {
parts.push(format!("— {}", meta.join(" · ")));
}
let text = if !has_body {
"(no output)".to_string()
} else {
parts.join("\n\n")
};
out.insert(cell, StaticCellOutput { ok, text });
}
}
let view_summary = session.summary();
session.close();
(out, Some(view_summary))
}
/// Inject static cell outputs into code cards without running client JS.
fn inject_cell_outputs(html: &str, outputs: &HashMap<String, StaticCellOutput>) -> String {
let mut html = html.to_string();
for (name, out) in outputs {
let marker = format!(r#"data-code-name="{name}""#);
let Some(pos) = html.find(&marker) else {
continue;
};
let tail = &html[pos..];
let Some(out_pos) = tail.find(r#"<pre class="code-out" hidden></pre>"#) else {
continue;
};
let abs_pos = pos + out_pos;
let replacement = if out.ok {
format!(r#"<pre class="code-out">{}</pre>"#, escape(&out.text))
} else {
format!(r#"<pre class="code-out err">{}</pre>"#, escape(&out.text))
};
html.replace_range(
abs_pos..abs_pos + r#"<pre class="code-out" hidden></pre>"#.len(),
&replacement,
);
}
html
}
fn inject_static_session(html: &str, summary: Option<&Value>) -> String {
let Some(summary) = summary else {
return html.to_string();
};
let Ok(json) = crate::host::json::value_to_json(summary) else {
return html.to_string();
};
let Ok(json) = serde_json::to_string(&json) else {
return html.to_string();
};
let line = format!(
"window.MARQDO_VIEW_LIVE = false;\nwindow.MARQDO_STATIC_SESSION = {json};"
);
html.replace("window.MARQDO_VIEW_LIVE = false;", &line)
}
/// Capture live-cell results into a snapshot consumed by `view output --snapshot`.
pub fn write_capture(opts: CaptureOptions) -> Result<()> {
let root = build_root(&opts.path)?;
let mut pages = serde_json::Map::new();
for rel_pb in &root.files {
let rel = rel_pb.to_string_lossy().replace('\\', "/");
let abs = resolve_abs(&root, &rel)?;
let vm = build_file_view(&root, &abs, &rel, &[], false)?;
let (outputs, summary) = static_page_data(&root, &abs, &vm.preset_stdin);
let mut out_map = serde_json::Map::new();
for (name, output) in outputs {
out_map.insert(
name,
serde_json::json!({ "ok": output.ok, "text": output.text }),
);
}
let summary_json = summary
.as_ref()
.and_then(|v| crate::host::json::value_to_json(v).ok())
.unwrap_or(serde_json::Value::Null);
pages.insert(
rel.clone(),
serde_json::json!({
"source_hash": source_hash(&vm.source),
"summary": summary_json,
"outputs": out_map,
}),
);
}
let doc = serde_json::json!({ "version": 1, "pages": pages });
if let Some(parent) = opts.out.parent() {
fs::create_dir_all(parent)?;
}
fs::write(&opts.out, serde_json::to_string_pretty(&doc)?)
.with_context(|| format!("write {}", opts.out.display()))?;
eprintln!("marqdo view capture: {} page(s) → {}", root.files.len(), opts.out.display());
Ok(())
}
fn snapshot_page(
snapshot: &serde_json::Value,
rel: &str,
source: &str,
) -> Result<(HashMap<String, StaticCellOutput>, Option<Value>)> {
let page = snapshot
.get("pages")
.and_then(|p| p.get(rel))
.ok_or_else(|| anyhow::anyhow!("snapshot has no page `{rel}`"))?;
let expected = page
.get("source_hash")
.and_then(|v| v.as_str())
.unwrap_or("");
if expected != source_hash(source) {
anyhow::bail!("snapshot source hash mismatch for `{rel}`");
}
let mut outputs = HashMap::new();
if let Some(map) = page.get("outputs").and_then(|v| v.as_object()) {
for (name, value) in map {
outputs.insert(
name.clone(),
StaticCellOutput {
ok: value.get("ok").and_then(|v| v.as_bool()).unwrap_or(false),
text: value
.get("text")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
},
);
}
}
let summary = page
.get("summary")
.filter(|v| !v.is_null())
.and_then(|v| json_to_value(v).ok());
Ok((outputs, summary))
}
pub fn write_static(opts: OutputOptions) -> Result<()> {
let snapshot = match &opts.snapshot {
Some(path) => Some(serde_json::from_str::<serde_json::Value>(
&fs::read_to_string(path).with_context(|| format!("read {}", path.display()))?,
)?),
None => None,
};
let root = build_root(&opts.path)?;
fs::create_dir_all(&opts.out_dir)
.with_context(|| format!("create {}", opts.out_dir.display()))?;
@@ -48,7 +303,20 @@ pub fn write_static(opts: OutputOptions) -> Result<()> {
let links = LinkMode::Static {
from: Some(rel.clone()),
};
let html = page_file(&root.files, &rel, &vm, &links);
let (cell_outputs, static_summary) = if let Some(snapshot) = &snapshot {
snapshot_page(snapshot, &rel, &vm.source)?
} else if opts.no_exec {
(HashMap::new(), None)
} else {
static_page_data(&root, &abs, &vm.preset_stdin)
};
let html = inject_static_session(
&inject_cell_outputs(
&page_file(&root.files, &rel, &vm, &links),
&cell_outputs,
),
static_summary.as_ref(),
);
let out_file = pages_root.join(format!("{rel}.html"));
if let Some(parent) = out_file.parent() {
fs::create_dir_all(parent)?;
@@ -57,11 +325,17 @@ pub fn write_static(opts: OutputOptions) -> Result<()> {
if first_page_html.is_none() {
// index.html = first file (same content, links rooted at OUT_DIR)
first_page_html = Some(page_file(
&root.files,
&rel,
&vm,
&LinkMode::Static { from: None },
first_page_html = Some(inject_static_session(
&inject_cell_outputs(
&page_file(
&root.files,
&rel,
&vm,
&LinkMode::Static { from: None },
),
&cell_outputs,
),
static_summary.as_ref(),
));
}
}
@@ -85,3 +359,50 @@ fn resolve_abs(root: &ViewRoot, rel: &str) -> Result<PathBuf> {
let abs = root.root.join(rel);
Ok(abs.canonicalize().unwrap_or(abs))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn snapshot_page_rejects_source_hash_mismatch() {
let snap = serde_json::json!({
"pages": {
"a.mq.md": {
"source_hash": "bad",
"summary": null,
"outputs": {}
}
}
});
assert!(snapshot_page(&snap, "a.mq.md", "# main\n").is_err());
}
#[test]
fn snapshot_page_reads_outputs_and_summary() {
let source = "# main\n";
let hash = source_hash(source);
let snap = serde_json::json!({
"pages": {
"a.mq.md": {
"source_hash": hash,
"summary": {"persistent": false, "cells": [], "kernels": []},
"outputs": {"a": {"ok": true, "text": "42"}}
}
}
});
let (outputs, summary) = snapshot_page(&snap, "a.mq.md", source).unwrap();
assert_eq!(outputs["a"].text, "42");
assert!(summary.is_some());
}
#[test]
fn injects_visible_static_cell_outputs() {
let html = r#"<div class="card" data-code-name="a"></div><pre class="code-out" hidden></pre>"#;
let mut outputs = HashMap::new();
outputs.insert("a".to_string(), StaticCellOutput { ok: true, text: "42".into() });
let html = inject_cell_outputs(html, &outputs);
assert!(html.contains(r#"<pre class="code-out">42</pre>"#), "{html}");
assert!(!html.contains("hidden"), "{html}");
}
}
+4
View File
@@ -1259,6 +1259,8 @@ fn render_stmt(
<span class="badge">code</span>
<code class="expr">`{name}` =</code>
<span class="code-lang">{lang}</span>
<button type="button" class="nb-clean" title="Clear this output">Clean</button>
<button type="button" class="nb-persist" title="Persist this result with writeback">Persist</button>
</div>
<pre class="code-body"><code class="language-{lang}">{src}</code></pre>
<div class="code-foot">
@@ -1537,6 +1539,8 @@ pub(crate) fn render_code_assign_card(name: &str, lang: &str, source: &str) -> S
<span class="badge">code</span>
<span class="code-name">{name}</span>
<span class="code-lang">{lang}</span>
<button type="button" class="nb-clean" title="Clear this output">Clean</button>
<button type="button" class="nb-persist" title="Persist this result with writeback">Persist</button>
</div>
<pre class="code-body"><code class="language-{lang}">{src}</code></pre>
<div class="code-foot">
+31
View File
@@ -0,0 +1,31 @@
---
title: View without foreign
description: A pure Marqdo document — formula, table, plot and prose, no external kernel
import math:lib/math.mq.md
---
# main
This page has no foreign import and no ` ```lang ` code cell. It is still opened by the same `view` surface, with the same toolbar and layout. The cell list is simply empty.
The formula is ordinary Marqdo data:
$$
% marqdo: var=f
x^2 - 2
$$
**_plot = math.plot formula=`f` var="x" min=-4 max=4**
| region | orders | revenue |
|--------|--------|---------|
| 华东 | 45 | 56253 |
| 华南 | 45 | 67187 |
| 华北 | 45 | 65272 |
| 西部 | 45 | 66884 |
**total = 56253 + 67187 + 65272 + 66884**
> print text=`total`
This is the control page: no foreign, no cells, no separate mode. The same view shell is used.
+4 -1
View File
@@ -1,6 +1,7 @@
---
title: Python 经营洞察 Notebook
title: Python 经营洞察 · View Cells
description: 多 cell 共享 Python 内核,做订单生成、区域/产品指标、异常检测、报告输出与独立复核。
import foreign:lib/foreign.mq.md
---
# main
@@ -22,6 +23,8 @@ description: 多 cell 共享 Python 内核,做订单生成、区域/产品指
| 生成报告 | 分析 | ASCII 图 + JSON 报告 |
| 独立复核 | 复核 | 另一个内核读取报告做校验 |
`会话` = > foreign.session kernels=`内核表` cells=`会话表`
`准备数据` =
```python
import math
+3
View File
@@ -1,5 +1,6 @@
---
title: notebook shared state
import foreign:lib/foreign.mq.md
---
# main
@@ -19,6 +20,8 @@ title: notebook shared state
| 读取 | 分析 | 读取共享变量 |
| 隔离 | 绘图 | 看不到 shared |
`会话` = > foreign.session kernels=`内核表` cells=`会话表`
`设置` =
```node
let shared = 41;