Files
marqdo/public/stdlib/foreign.mq.md
T
cflmy 4488a3cb70 feat(migrate): M7 迁移与版本门 — per-file markup + migrate labels + 全仓迁移
原计划的"翻转默认 + 旧形硬报错"不可行:`[键](集合)` 与 `[描述](实现)` 外形相同,
翻转会让 506 处取元静默变义,硬报错又拿不到证据。改为**按文件声明**。

一、per-file markup
- frontmatter `markup: occlusion`(或 `标记: 遮蔽`)决定该文件模式
- load.rs 先取模式,用 parse::expr::with_markup(线程局部压栈)包住解析
- 同一进程可同时加载两种模式的文件(imports);MARQDO_MARKUP 仍作进程默认
- 好处:迁移逐文件、可增量、可回滚,不存在半迁移的静默错义

二、`marqdo migrate labels [--write|--json]`
改写规则:
  [键](集合)      → [键](access(集合, "键"))
  [`名`](集合)    → [名](access(集合, `名`))
  - [项](集合)    → - [项](遍历 集合 为 项)

安全模型:改写 → 重解析 → 证明等价 → 才写
- 原文件按 classic 解析得 AST-A;改写后按 occlusion 解析得 AST-B
- 反归一化 B(标签与键同名且 payload 为 access 的遮蔽还原成 Index)
- Debug(A) == Debug(反归一化(B))(抹掉行号)才写入,否则原文件一字不动

只改代码行(LineKind::Code);表格行只改 **…** 单元格,散文 markdown 链接不碰;
链式索引由内向外改写,已迁移的有守卫不二次改。

三、access 提升为宿主函数 HostFn::Access(u16 id 212)
与旧取元逐字一致(Map 按键 / List 1-based / 缺键 missing map key);
进 HostFn 是为了两个后端共用同一实现,interp 里的特例已删除。

四、顺带修的两处真实语义缺陷
1. payload 继承外层"文本模式":`f text=[k](access(c,k))` 中 access 不被当调用
   → `()` 内永远允许 callee(args)
2. 遮蔽形状判定过宽:`[问候] ([格式化] "Bo")` 误判为遮蔽
   → 要求 `]` 后紧跟 `(`,否则落回方括号调用

五、全仓迁移结果
migrate labels: 256 index, 23 loop, 80 file written;0 文件被等价性拒绝;
迁移后 cargo test --release → 604 passed / 0 failed;再跑迁移幂等。

等价性检查先后抓出 4 类改写 bug(表格散文链接、payload 文本模式、链式索引、
遮蔽与方括号调用混淆),每类都是先拦下再修工具,未污染文件。

测试:6 个单测 + 3 个集成(干跑不写盘 / 写入后运行输出不变且幂等 /
等价失败文件不动);全量 604 passed / 0 failed(M3 后 595 + 9);双后端一致。

文档:doc/design/change-m7-migration.md
2026-10-22 15:33:31 +08:00

4.4 KiB


markup: occlusion title: lib/foreign — foreign kernels v2 description: Structured execution, JSON calls, parallel fan-out, persistent kernels and an explicit state bridge import foreign:lib/foreign.mq.md stdin: 21

main

lib/foreign runs code fences and inline sources with local interpreters. v2 keeps the Markdown surface unchanged while adding structured records, JSON in/out, parallel fan-out, persistent Python/Node kernels and an explicit state bridge.

The same file is still a document: a bound fence is readable Markdown, and foreign.* turns it into an executable unit.


Probe before you run.

node = > foreign.probe lang="node"

print text=ok) print text=version)

probe returns {lang, cmd, path, ok, version, error}. Omit lang to inspect every known language.


A structured record, not just stdout.

marker =

console.log("hello from a foreign kernel");

record = > foreign.invoke code=marker

print text=ok) print text=stdout) print text=code)

invoke never throws on a process failure. Read ok, code, stdout, stderr, ms, timed_out, truncated, result and error from one record.


JSON in, JSON out.

double =

let input = "";
process.stdin.on("data", chunk => { input += chunk; });
process.stdin.on("end", () => {
  console.log(JSON.stringify(JSON.parse(input) * 2));
});

single = > foreign.call code=double input=21

print text=single

The input is encoded as JSON on stdin; the last stdout value is parsed as JSON. Use output="json" with invoke when you also want the raw record.


Parallel fan-out.

numbers =

value
1
2
3
4

doubled = > foreign.call_many code=double inputs=numbers concurrency=4

print text=doubled

call_many applies one program to every input, runs independently, preserves input order, and returns parsed JSON results. run_many does the same for full invoke-option maps.


A persistent kernel.

session = > foreign.session_open lang="node"

first = > foreign.session_eval id=session source="let total = 0; for (let i = 1; i <= 5; i += 1) total += i;"

second = > foreign.session_eval id=session source="total * 2"

print text=result)

Variables created in the first evaluation survive into the second. The kernel is not a shell: use session_eval, session_get, session_names, then session_close.


The explicit state bridge.

> foreign.session_set id=session name="rate" value=0.08

rate = > foreign.session_get id=session name="rate"

print text=rate

> foreign.session_close id=session

This is a JSON copy, not shared memory. The kernel receives a value; Marqdo receives a value. There is no implicit mirroring and no hidden global namespace.


Cells in view.

foreign.session owns the document's kernel state. The kernel and cell order are ordinary tables passed as arguments; the names are free:

`分析内核` =

| 名称 | 语言 | 命令 | 说明 |
|------|------|------|------|
| 分析 | python | | shared analysis |
| 复核 | python | | isolated review |

`分析单元` =

| 单元 | 内核 | 说明 |
|------|------|------|
| 准备数据 | 分析 | write state |
| 计算指标 | 分析 | read state |
| 独立复核 | 复核 | isolated kernel |

`会话` = > foreign.session kernels=`分析内核` cells=`分析单元`

view consumes the 会话 handle and adds Run All, Step, kernel switching, a journal and per-cell state. Without 会话, view cells are stateless one-shot runs: same shell, no persistent kernel. There is no second CLI mode. See the complete Python walkthrough in features/19-foreign-view-cells.mq.md.


Safety defaults.

Boundary Default
Execution requires the host execution capability
Shell no shell; arguments are split explicitly
Output bounded, with truncated
Timeout process group killed on Unix / taskkill /T on Windows
CWD file sandbox root; no implicit escape
Parallelism capped fan-out; each item isolated

Read the full API in ../../lib/foreign.mq.md and the design in ../../doc/design/foreign-kernel.md.