Compare commits

...
4 Commits
Author SHA1 Message Date
cflmyandCursor 3cabe53da2 release: v2.1.4 isolated HTTP execute (ADR 0009)
Default execute/sse-live/stream to Supervisor worker pool; MR-01 gold;
/_runtime/*; bump CLI/wasm/ext to 2.1.4.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-26 18:33:33 +08:00
cflmy 58e13ee6d3 docs: add isolated web runtime development design 2026-10-26 05:53:02 +00:00
cflmyandCursor 1b11fe063a fix(web): fail-fast when invoke lock is busy; document single-interp limit
Long sse-live chats held INVOKE_LOCK indefinitely and starved short routes
(UI freeze). Timed try-lock returns interpreter busy → HTTP 503; ADR 0008
records that HTTP is multi-goroutine while Marqdo evaluation stays single-flight.
Also lands plan/delta streaming and soft map guards used by qdagent chat.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-26 13:27:53 +08:00
cflmyandCursor b792f44556 feat(web): auto sse-live when chat body has stream:true
Upgrade json invoke routes to the M02 EventBus forwarder for OpenAI-compatible
stream requests, and normalize LLM delta frames to message.delta.payload.delta
so tokens flush before run.completed.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-26 09:47:05 +08:00
63 changed files with 6175 additions and 457 deletions

No files matched your search

@@ -19,6 +19,12 @@ jobs:
run: cargo test --test workbench_contracts --locked
- name: Offline executable-knowledge gate
run: cargo test --test executable_knowledge --locked
- name: Isolated execute MR-01 (dual 60s SSE + short reads)
run: |
cargo build -p marqdo_plugin_llm --locked
bash scripts/build-web-plugin.sh
cargo test --test isolated_execute_mr01 --locked -- --nocapture
timeout-minutes: 5
- name: no ignored tests counted as success
run: |
if grep -n '#\[ignore\]' tests/executable_knowledge.rs; then
+3 -1
View File
@@ -126,9 +126,11 @@ jobs:
## Marqdo ${{ github.ref_name }}
### 亮点 / Highlights
- **v2.1.4**:默认隔离 HTTP execute(ADR 0009)——worker 池 / `--call-lib-path` · `/_runtime/*` · MR-01 双流金样例 · `runtime.isolated_execution=1`
- **v2.1.4**: Default isolated HTTP execute (ADR 0009) — worker pool / `--call-lib-path`, `/_runtime/*`, MR-01 dual-stream gold, `runtime.isolated_execution=1`
- **v2.1.4 ext**:扩展包同步 2.1.4 / Ext pack 2.1.4
- **v2.1.3**:知识工作台运行时——ModelSelection/RunContext/usage · view document 只读嵌入 · LLM selection/messages/cancel · agent selection e2e
- **v2.1.3**: Workbench runtime — ModelSelection/RunContext/usage, view document read-only embed, LLM selection/messages/cancel, agent selection e2e
- **v2.1.3 ext**:扩展包同步 2.1.3 / Ext pack 2.1.3
- **v2.1.1**:可执行知识主链——BoundCall · NarrativeRef · 来源/适用期 · patch/extract · 整代 CURRENT + closure · 发票示例 113
- **v2.1.1**: Executable-knowledge spine — BoundCall, NarrativeRef, sources, patch/extract, generation CURRENT + closure, invoice example 113
- **v1.5.2**:外联 v3 P0/P1——raw stdin + 每 cell 输入 · plan API · snapshot · `view capture` · Persist
+26
View File
@@ -8,6 +8,32 @@
### Changed
## v2.1.4 — 2026-10-26
### Highlights
**默认隔离 HTTP execute(ADR 0009)**:`class=execute` / `sse-live` / `stream:true` 经 Supervisor worker 池(`marqdo run --call-lib-path`),父进程不再 `CallLibPath` 占 `INVOKE_LOCK`;短读 / 未声明非流式仍 `legacy_serial`(ADR 0008)。`/_runtime/*` 控制面;`runtime.isolated_execution=1`;MR-01 双 60s 假 SSE + 短读金样例。CLI · wasm · ext **2.1.4**。
```bash
git checkout v2.1.4 && cargo build --release
marqdo ext add web # CDN / Releases → 2.1.4
```
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——隔离运行时与 Web execute 路径,不新增 Markdown 标记。
### Added
- **默认隔离 execute(ADR 0009)**:HTTP `execute` / `sse-live` / `stream:true` 经 Supervisor worker 池(`marqdo run --call-lib-path`),父进程不再 `CallLibPath` 占 `INVOKE_LOCK`;可选 route `class`;`/_runtime/*` 控制面;`runtime.isolated_execution=1`。
- **MR-01 金样例**:`tests/isolated_execute_mr01.rs` + `tests/ext/isolated-execute-site/`(双 60s 假 SSE + 短读并行;槽满拒绝);接入 `executable-knowledge` workflow。
- **Worker CLI**:`marqdo run --call-lib-path` / `--fs-root` / `--args-json` / `--emit-result`;host `runtime` admit/dispatch/cancel/events。
### Fixed
### Changed
- **compile 1000 门禁**:executable-knowledge / workbench 路径恢复与能力探测诚实(isolated 默认开启)。
- CLI / wasm / `ext/VERSION` **同步 2.1.4**。
## v2.1.3 — 2026-10-25
### Highlights
Generated
+3 -2
View File
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "marqdo"
version = "2.1.3"
version = "2.1.4"
dependencies = [
"anyhow",
"assert_cmd",
@@ -589,6 +589,7 @@ dependencies = [
"predicates",
"pulldown-cmark",
"regex",
"serde",
"serde_json",
"sha1",
"sha2",
@@ -610,7 +611,7 @@ dependencies = [
[[package]]
name = "marqdo-wasm"
version = "2.1.3"
version = "2.1.4"
dependencies = [
"marqdo",
"serde_json",
+2 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo"
version = "2.1.3"
version = "2.1.4"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo — Markdown markers as an interpreted programming language"
@@ -29,6 +29,7 @@ anyhow = "1"
chrono = { version = "0.4", default-features = false, features = ["clock", "std"] }
clap = { version = "=4.4.18", features = ["derive"], optional = true }
pulldown-cmark = "0.12"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "1"
tiny_http = { version = "0.12", optional = true }
+6 -5
View File
@@ -281,9 +281,10 @@ marqdo catalog public -o .marqdo
---
## 现状(v2.1.3)
## 现状(v2.1.4)
- **知识工作台运行时(v2.1.3)**:ModelSelection / RunContext / usage null≠0 · `view document` 只读嵌入 · LLM selection/messages/cancel · agent.step selection e2e · live smoke · 扩展包同步 **2.1.3**(设计 Partial:原生适配器 unavailable)
- **默认隔离 HTTP execute(v2.1.4 · ADR 0009)**:`execute` / `sse-live` / `stream:true` → Supervisor worker 池(`--call-lib-path`);父进程不再占 `INVOKE_LOCK`;`/_runtime/*`;`runtime.isolated_execution=1`;MR-01 金样例;扩展包同步 **2.1.4**
- **知识工作台运行时(v2.1.3)**:ModelSelection / RunContext / usage null≠0 · `view document` 只读嵌入 · LLM selection/messages/cancel · agent.step selection e2e · live smoke(设计 Partial:原生适配器 unavailable)
- **可执行知识生产链(v2.1.2)**:证据/闭包失效 · `verify`/`adjudicate` · `reuse_execute` / `eng_bind_execute`+`publish_gate` · `document.extract` · View `KnowledgePresentation`+SSE `page_token` · 上传限流 · 门禁 45 项(100/1000 单元)
- **可执行知识主链(v2.1.1)**:BoundCall 双后端 · NarrativeRef · 来源/适用期 · `artifact.patch` / `extract_pure` · 整代 `CURRENT` + closure · 发票示例 `113`
- **链接语法(v2.1.0)**:`[描述](目标)` 描述不执行 · `()` 与 `**代码**` 同权(`加(1)` / `加一 号(31)`)· `access` 取元与 `遍历` 链接循环 · `marqdo labels` 两级检索 / `--baseline` 债务棘轮 · `migrate labels --mark` · 诊断 `label.*` · view 描述正文 + 代码片 + 桩徽章 · **可执行知识 [link-syntax.md](doc/design/link-syntax.md)**(项目索引、LinkInfo、链接图与 lock)
@@ -322,7 +323,7 @@ marqdo catalog public -o .marqdo
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
### 如何使用最新 Marqdo(v2.1.3)
### 如何使用最新 Marqdo(v2.1.4)
**安装解释器(任选其一)**
@@ -337,7 +338,7 @@ marqdo catalog public -o .marqdo
```bash
# 1) 源码安装(跟 tag 或 main)
git clone https://github.com/cflmy/marqdo.git && cd marqdo
git checkout v2.1.3 # 或留在 main
git checkout v2.1.4 # 或留在 main
cargo build --release
export PATH="$PWD/target/release:$PATH"
marqdo version
@@ -376,7 +377,7 @@ marqdo wasm build
# → dist/wasm/ … 见 examples/browser-hello/
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
# https://github.com/cflmy/marqdo/releases/tag/v2.1.3
# https://github.com/cflmy/marqdo/releases/tag/v2.1.4
```
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo-wasm"
version = "2.1.3"
version = "2.1.4"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
+8
View File
@@ -1,3 +1,11 @@
marqdo (2.1.4-1ppa1) noble; urgency=medium
* Marqdo 2.1.4: default isolated HTTP execute (ADR 0009) — Supervisor
worker pool for execute/sse-live/stream, /_runtime/* control plane,
MR-01 dual 60s SSE gold sample, runtime.isolated_execution=1.
-- cflmy <pingan@cflmy.cn> Mon, 26 Oct 2026 10:00:00 +0000
marqdo (2.1.3-1ppa1) noble; urgency=medium
* Marqdo 2.1.3: workbench M00–M06 runtime contracts (ModelSelection,
+3
View File
@@ -19,6 +19,9 @@
| 文档 | 摘要 |
|------|------|
| [design/isolated-web-runtime-development-2026-10-26.md](design/isolated-web-runtime-development-2026-10-26.md) | **隔离执行与响应式 Web 运行时**:进程池、控制/读取分离、取消、事件恢复、发布一致性;实现见 `src/runtime/` |
| [adr/0009-isolated-web-runtime.md](adr/0009-isolated-web-runtime.md) | **ADR 0009**:隔离 Web 运行时决策;金样例通过后替代 ADR 0008 的并行性结论 |
| [跨库缺陷复核(2026-10-26)](../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md) | **最新源码与测试证据**:实际 HTTP 路由、知识发布/阅读断点、旧缺陷处置及 1000 单元编译性能失败;需同级 qdagent checkout |
| [design/qdagent-workbench-runtime-and-view-2026-10-25.md](design/qdagent-workbench-runtime-and-view-2026-10-25.md) | **qdagent 知识工作台框架支持**:M00–M05 合同落地中(`src/workbench/`、MLSP `capabilities` / `view.document` / `view.reference_resolve`、`tests/workbench_contracts.rs`);完整 verified 以切片验收证据为准 |
| [design/view-knowledge-reading-experience-2026-10-24.md](design/view-knowledge-reading-experience-2026-10-24.md) | **View 知识阅读体验重设计(建议)**:正文优先、结构导航、就地知识说明、来源与运行状态、响应式线框、实现契约与验收;含当前呈现和 no-exec 反例 |
| [roadmap/qdagent-knowledge-support-gaps-2026-10-24.md](roadmap/qdagent-knowledge-support-gaps-2026-10-24.md) | **求道知识沉淀与文档编译支撑缺口**:2.1.1 源码复核、来源/依赖失效反例、验证与 Agent 协议、语法提示词、导入底座及跨库分工 |
+68
View File
@@ -0,0 +1,68 @@
# ADR 0008:Web 热路径单解释器串行化(设计缺陷与缓解)
| | |
|---|---|
| Status | **Accepted**(缺陷已记录;缓解已落地;真并行见 [ADR 0009](0009-isolated-web-runtime.md)) |
| Date | 2026-10-26 |
| Related | [web-invoke-serialization.md](../design/web-invoke-serialization.md) · [ADR 0009](0009-isolated-web-runtime.md) · ADR [0004](0004-web-plugin-go.md) · ADR [0007](0007-web-document-endpoint.md) · [ext-abi.md](../design/ext-abi.md) · [ext-web-go-rewrite.md](../design/ext-web-go-rewrite.md) |
## Context
- Go `plugins/web` 的 HTTP 服务是**多 goroutine** 的:每个请求可并发进入 `handleInvoke` / `handleInvokeLiveSSE`。
- 每个 Endpoint 最终经 C ABI `host_query("call_lib_path", …)` 回到 Rust 宿主,在**同一个** `HostContext`(Marqdo 解释器状态)上执行作者 `.mq.md`。
- `HostContext` **不是** `Sync`:共享可变会话、库装载、写回、事件总线等状态不能安全地被多线程同时求值。
- 因此宿主用进程级 `INVOKE_LOCK`(`src/host/plugin.rs`)串行化所有来自 worker 线程的 `call_lib_path`。
### 常见误解
> 「web 都是单线程的吗?不能多线程吗?」
| 层 | 并发模型 |
|----|----------|
| Go HTTP / SSE flush / 取消 | **多线程(多 goroutine)** |
| Marqdo 解释器 `call_lib_path` | **单飞(一把全局锁)** |
| 浏览器 UI | 与后端锁无关;但若短 API 同步阻塞等待锁,会表现为「整页卡死」 |
**缺陷不在「HTTP 单线程」,而在「多 HTTP 工作线程共享一个非 Sync 解释器」。**
真并行需要多解释器实例(或可共享的隔离运行时),不是去掉 Go 侧并发。
### 事故形态(已观测)
长对话(`return=sse-live` / `stream:true`)在 `CallLibPath` 期间一直持有 `INVOKE_LOCK`。并发的短路由(如 `/v1/models`、设置页探测)在旧实现里对锁 **无限期** `lock()`,HTTP 挂起 → 前端 `fetch` 无超时 → 页面无响应。重启进程可恢复,但属于运维掩盖,不是正确错误处理。
## Decision
1. **承认并文档化**本限制为宿主架构缺陷(单 `HostContext` + 全局 `INVOKE_LOCK`),不得再被表述为「web 插件只能单线程」。
2. **缓解(已实现,非根治)**:
- `call_lib_path` 从 worker 进入时用 **限时 try-lock**(默认 `MARQDO_INVOKE_WAIT_MS=1500`),超时返回错误串 `interpreter busy: …`。
- Go `handleInvoke` 将含 `interpreter busy` 的错误映射为 **HTTP 503**,并设 `Retry-After: 2`。
- **禁止**在 SSE 期间把进程级 `MARQDO_INVOKE_WAIT_MS` 拉长:那会让并发短请求重新卡死。
- 长 SSE 在启动时用同一短预算争锁;一旦拿到则持有至调用结束(对话本来就要跑完)。
3. **客户端契约**:短 API 必须有超时与 503 文案;不得假设「任意时刻都能同步进入解释器」。
4. **根治(未决,另案)**:解释器池 / per-request isolate / 可 `Send` 的会话切片。未落地前,产品行为以本 ADR 的 503 语义为准。
## Consequences
### Positive
- 对话进行中,设置/模型列表等短请求约 1.5s 内失败并可读,UI 不再无限挂起。
- 缺陷有名有姓,后续多解释器工作不会被「修好了」的假象挡住。
### Negative / Risks
- 同一进程仍无法真正并行跑两个 Marqdo Endpoint 体;第二个长 SSE 也会在短预算后失败。
- 极短窗口内,短请求若正好撞上锁交接,可能偶发 503(可重试)。
- `MARQDO_INVOKE_WAIT_MS` 是进程环境变量;误设过大仍会复现卡顿。
### Neutral
- Go 多 goroutine、SSE 桥(`MARQDO_EVENT_FORWARD`)、取消令牌(`MARQDO_CANCEL_TOKEN`)保持不变。
- 不改变 Document / Endpoint 作者面(ADR 0007);只约束宿主与 web 插件错误语义。
## Implementation pointers
| 位置 | 职责 |
|------|------|
| `src/host/plugin.rs` → `acquire_invoke_lock` | 限时争锁;失败 `set_err("interpreter busy…")` |
| `plugins/web/.../routes_invoke.go` → `handleInvoke` | `interpreter busy` → 503 + `Retry-After` |
| 设计长文 | [web-invoke-serialization.md](../design/web-invoke-serialization.md) |
+31
View File
@@ -0,0 +1,31 @@
# ADR 0009: Isolated Web Runtime (worker process pool)
| 项目 | 约定 |
|---|---|
| 状态 | **Accepted**;HTTP **execute** 路径正式替代 [ADR 0008](0008-web-invoke-serialization.md) 的并行性结论。`legacy_serial`(未声明且非流式)仍引用 0008 单飞 / 父进程 `CallLibPath`。 |
| 日期 | 2026-10-26 |
| 设计 | [isolated-web-runtime-development-2026-10-26.md](../design/isolated-web-runtime-development-2026-10-26.md) |
| 金样例 | `tests/isolated_execute_mr01.rs`(MR-01 双 60s SSE + short reads;同文件 MR-02 槽满拒绝) |
## 背景
ADR 0008 确认:Go HTTP 多线程,但所有 Endpoint 共享一把解释器锁与进程级 SSE 桥环境变量。1500ms busy → 503 只是缓解,不是并行。
## 决策
1. **首选架构**:预热的一次性 Marqdo worker 进程池;每个租约独占一个进程与 HostContext。
2. **HTTP 默认**:`class=execute` 或 `return=sse-live` 或 body `stream:true` → `admit` → `dispatch`(`--call-lib-path` worker);禁止父进程 `CallLibPath` / `Setenv` / `legacySSEBridgeMu`。`short` / `read` / `control` 仍父进程短路径;未声明非流式 → `legacy_serial`。
3. **控制/读取分离**:取消与事件订阅不进入任务解释器锁;短业务与长执行有独立容量(execute 槽默认 2)。
4. **协议**:`InvokeRequestV2` / worker env + NDJSON `MARQDO_EVENT_FORWARD` + `--emit-result`(首期);`/_runtime/*` 控制面。
5. **能力探测**:`runtime.isolated_execution=1`(默认)、`runtime.task_control_v2`、`runtime.event_replay`、`view.analysis_cache_v2`;`runtime.read_lane` 仍 0。
6. **发布**:`expected_generation` CAS + publish journal;逻辑文档 ID 与物理 generation 路径解耦。
## 后果
- Execute / live SSE 不再抢进程级 SSE 桥;双流可并行(MR-01)。
- Legacy 未声明非流式路径仍可走父进程锁(ADR 0008)。
- 产品(qdagent)cancel / Last-Event-ID 接线仍为后续;不阻碍本框架发版。
## 验收(摘要)
MR-01 绿:双假上游 60s 流不混字、短读全 200、两 worker `--call-lib-path`、第三路槽满 429/503、`runtime.isolated_execution=1`。其余见设计文档 §12。
@@ -0,0 +1,326 @@
# 隔离执行与响应式 Web 运行时:详细开发设计
| 项目 | 约定 |
|---|---|
| 状态 | **Verified**(execute 路径);HTTP `execute` / `sse-live` / `stream:true` 默认走 Supervisor worker 池(不再经父进程 `CallLibPath` / `INVOKE_LOCK`);`runtime.isolated_execution=1`;MR-01 金样例 `tests/isolated_execute_mr01.rs`;未声明非流式仍 `legacy_serial`(ADR 0008) |
| 基线 | Marqdo `1b11fe0` / 2.1.3;qdagent `64b97e5`;2026-10-26 |
| 问题依据 | [单解释器缺陷](web-invoke-serialization.md)、[ADR 0008](../adr/0008-web-invoke-serialization.md)、[本轮跨库复核 R01–R14](../../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md) |
| 产品配套 | [qdagent 可靠工作台开发设计](../../../qdagent/doc/design/reliable-workbench-development-2026-10-26.md) |
| 继承合同 | [工作台 M00–M06](qdagent-workbench-runtime-and-view-2026-10-25.md) 中 ModelSelection、ModelSnapshot、Usage、DocumentView、Locator;保留文档原生作者面 |
## 1. 目标、边界与不变量
目标是让用户在长对话、导入和构建过程中持续打开模型列表、读取来源、修改下一轮设置并可靠停止任务。高并发容量按资源预算显式限制;不能让所有请求进入一把解释器锁再超时。
Marqdo 负责通用执行隔离、资源调度、事件/取消、语义分析和制品校验。qdagent 负责用户/工作区授权、业务状态与发布政策。产品仍通过 `.mq.md` 调用 Marqdo Agent,不增加第二套模型循环或知识编译器。
必须保持的系统不变量:
1. 同一个可变 HostContext 同时最多一个求值者;不删除锁、不增加虚假的 `unsafe impl Sync` 来获得并行。
2. 一个执行租约只属于一个 run/attempt;凭证、事件、取消、cwd、权限和模型选择不可流入另一请求。
3. 控制与读取具有独立容量,不被 LLM 等待占满;停止不需要重新进入正在执行的解释器。
4. 事件可以重投,外部副作用不默认重做;不宣称跨网络 exactly-once。
5. 已确认终态不能被晚到完成/取消覆盖;取消不回滚已经完成的外部写入。
6. 阅读不执行 `.mq.md`,缓存不绕过授权或版本校验;缓存命中必须实际复用结果。
7. 正文、依赖、证据和编译代属于同一不可变快照;不会把两个代的结果拼成一次阅读或执行。
8. 任何能力开关都以实际探测和测试为依据;新 API/schema 在落地前都属于本文提案。
## 2. 架构选择
| 方案 | 优点 | 当前代价 | 本轮选择 |
|---|---|---|---|
| 继续全局锁,只增加 Go handler | 改动小 | 长任务依然阻塞所有 Endpoint,串扰变量也未消失 | 只保留作兼容模式 |
| 同进程多 HostContext | 潜在内存复用较好 | GLOBAL_HOST、插件回调、环境、cwd、事件 sink 生命周期需系统重做 | 后续优化,不是首个交付门槛 |
| 独立 Marqdo worker 进程 + 控制/读取通道 | 隔离现有全局状态,可强制回收卡死任务,复用现有语言与 Agent | IPC、进程启动、内存、发布写协调 | **首选实现** |
| 引入外部通用队列和新业务 Runtime | 容易扩到集群 | 运维复杂度及第二套编排风险 | 单机阶段不引入 |
首期采用**预热的一次性执行进程池**:进程可预装运行时和非用户特定模块,分配任务后独占,结束即退出并异步补充。这样不用假设清空部分字段就能安全复用 HostContext。长寿命多租约 worker 只有通过状态泄漏、插件资源与 RSS 回归后才开启。只读分析服务可以长寿命运行,但不得加载有副作用的求值插件。
```mermaid
flowchart TD
UI[浏览器 / CLI / MCP] --> WEB[Go HTTP / 认证入口 / 限流]
WEB --> CTRL[任务控制与事件服务]
WEB --> SHORT[短业务请求容量]
WEB --> READ[只读分析服务]
SHORT --> POLICY[qdagent .mq.md 授权与业务规则]
POLICY --> CTRL
CTRL --> DB[(任务索引 / outbox / 事件日志)]
CTRL --> SUP[Rust Supervisor / 有界调度]
SUP --> A[Marqdo worker A / 一个租约]
SUP --> B[Marqdo worker B / 一个租约]
A --> AGENT[既有 Marqdo Agent / LLM / sandbox]
B --> AGENT
A --> STAGE[每任务暂存目录]
B --> STAGE
STAGE --> COMMIT[按资源串行提交与 revision CAS]
COMMIT --> SNAP[(不可变知识代 / 发布 manifest)]
READ --> SNAP
A --> CTRL
B --> CTRL
CTRL --> UI
```
“短业务请求容量”依旧可执行产品 `.mq.md`,但有独立 worker 预算和短 deadline。控制服务只做通用接纳、状态 CAS、取消与日志投递,不把模型选型、知识晋升等业务逻辑搬进 Go。读取服务接收经过授权的资源句柄;不得直接相信浏览器提交的绝对路径。
## 3. 路由分类与接纳
### 3.1 执行类别
在 Endpoint 装配中新增可选 metadata(字段名待 schema 固定):
| 类别 | 示例 | 资源预算与规则 |
|---|---|---|
| `control` | 任务查询、事件订阅、取消、health | 不执行用户代码;认证后访问任务存储/租约注册表 |
| `read` | view.document、引用解析、已发布模型列表投影 | 只读快照;不写用户源码;可缓存 |
| `short` | 保存设置、创建导入、接纳任务 | 无 LLM 长等待;默认 deadline 2s,较慢动作转为任务 |
| `execute` | ASK/BUILD/生成/验证 | 有界队列、专属进程、显式任务 deadline |
| `commit` | 切换知识代、合并受控文件变更 | 按 workspace/kb 的短临界区;执行前检查 revision |
旧 Endpoint 未声明时仍走 legacy_serial;不能自动猜测任意函数是纯读。纯度来自静态能力和注册配置,实际写权限仍由宿主限制。文件上传字节传输不占解释器,应先落隔离存储,再进入短接纳路径。
### 3.2 接纳步骤
1. HTTP 层解析受限请求,验证身份、格式、速率与连接数量。
2. 产品短入口解析 workspace/model/policy;授权版本与模型配置 revision 一并冻结。返回规范化任务请求,不执行 Agent。
3. 控制服务在同一数据库事务中登记作用域化 idempotency key、payload hash、task 和 dispatch outbox;重复请求读回已有 task,同键异内容返回 409。
4. 提交事务后才返回 202/Location。调度器消费 outbox 分配租约,租约带 attempt/epoch。
5. 对同步兼容 API,先完成接纳,再挂接结果;等待客户端超时不等于安全重做。
队列入库前容量不足返回 429 + Retry-After;服务不可用返回 503;接纳失败不发送 started。控制通道须保留槽位,不能因为数据流连接占满而无法取消。
## 4. Supervisor、生命周期与调度
### 4.1 Worker 生命周期
`starting → ready → leased → running → draining → exited`;启动失败/心跳丢失走 `unhealthy → terminating → exited`。任务状态由控制服务维护,进程状态不能直接当作业务成功。
- worker 启动握手报告 CLI/ext/native、protocol_version、能力、构建摘要;不匹配不进入 ready。
- 预热阶段不读取用户凭证、不挂载工作区写权限。分配租约后挂载当前任务的 capability roots、只读快照和独立 staging。
- 每任务具有随机 lease_id、run_id、attempt_id、epoch、绝对 deadline 和资源限制。
- Supervisor 持有进程句柄、子进程组、事件通道与取消句柄。worker 不能自行修改其他 worker 状态。
- 返回业务结果后仍须 drain 事件/用量并提交最终 checkpoint,控制服务持久化终态后确认 ACK,之后释放租约。
- 超时或断连清理不能遗留孤儿子任务;正常优雅退出后再删除临时文件。异常由独立 janitor 按租约过期回收。
### 4.2 有界、公平与资源预算
初始验证配置建议 4 vCPU/8 GiB、执行槽 2、短请求槽 1、读取服务 1。具体容量由每类任务的 RSS p95 和 CPU profile 决定,不能把这个数当所有机器的默认。执行槽上限:
`min(CPU约束, floor((可用内存 - 系统/控制/缓存预留) / worker_RSS_p95))`。
运行队列按 workspace 轮转,再按 session FIFO;一个 session 默认只允许一个活动 turn,跨 session 可并行。相同知识库的生成可并行,发布只占短提交锁。interactive 优先于后台索引,但后台使用老化提升或最低服务配额,防永久饥饿。
全局队列初始上限 64、单工作区 16、单会话待处理 4,均可配置;队列超限及时返回,不能悄悄丢任务。显示队列位置或“前方有任务”,没有可靠完成时间估计时不显示虚构倒计时。
### 4.3 副作用与重试
任务声明 `read_only / staged_write / external_effect`,由工具权限校验而非模型自报。read_only 可按政策重试;staged_write 丢弃未提交 staging 后重试;external_effect 出现超时/失联时标记 `effect_unknown`,等待核对,不自动重复支付、发信或写入。
稳定的操作键使用 `task_id + logical_step_id`,attempt_id 只标记传输尝试,不应导致同一外部动作生成新的去重键。工具不支持幂等查询时,只能明确报告不确定。工作簿记录已完成与未知副作用;“已停止”不意味着“全部撤销”。
## 5. IPC 与请求上下文
首期使用本机 Unix socket 或匿名管道,采用长度前缀 JSON 控制帧与独立事件通道。协议禁止无上限 JSON/单行日志;控制帧初始上限 1 MiB,大文档通过授权 blob handle 传递。普通 stdout/stderr 仅诊断,不与协议混流。当前 NDJSON 文件桥保留为 legacy fallback。
新增 `InvokeRequestV2` 示例(提案):
```json
{
"schema": "InvokeRequestV2",
"schema_version": 2,
"task_id": "task_01",
"run_id": "run_01",
"attempt_id": "attempt_01",
"lease": {"id": "lease_01", "epoch": 3},
"entry": {"module": "agent/qdagent.mq.md", "function": "run"},
"args_ref": "blob:sha256:...",
"context": {
"workspace_id": "ws_01",
"snapshot_id": "generation_01",
"selection_ref": "model-snapshot_01",
"capabilities_ref": "grant_01",
"deadline_at": "2026-10-26T08:10:00Z"
}
}
```
`grant_01` 是服务端验证的短寿命授权句柄,内容包括 principal、scope、根目录、允许效应、授权 revision、失效时间。worker 不使用浏览器任意传入的 fn/path。权限撤回后,新的工具调用必须重新验证 lease/grant,已开始的外部调用记录撤回时间与实际结果。
`RunContextV1` 保留原有字段与序列化兼容,内部增补显式 Context 对象;扩展 V2 由能力协商启用。event_sink、usage_sink、cancel_token 使用句柄,不使用外部可控路径。子任务通过 IPC/Command.env 显式继承,禁止父进程 `setenv`/`set_current_dir` 承担请求状态。
凭证在租约启动时通过受保护 IPC 或继承 FD 注入;不落任务 JSON、argv、日志或通用事件。模型快照可含 credential_ref,不含 api_key。即使选择相同 provider,不同请求的授权也不得合并。
## 6. 取消协议与状态 CAS
任务基础状态:`accepted → queued → running → finalizing → succeeded/failed`;queued/running 可进入 `cancel_requested → cancelled`;worker 失联进入 `interrupted`,副作用不明时进入 `needs_reconciliation`。这些是提案,产品可以将 succeeded 映射为 completed。
取消顺序:
1. 验证调用者对 task 的权限,CAS 写 cancel_requested、revision 与 requested_at;终态返回原状态,未知 ID 返回 404。
2. 向匹配 lease/epoch 的控制通道发送 Cancel;不经任务解释器锁。queued 任务原子移出待派发队列。
3. worker 在每次 LLM/工具调用、重试、子任务启动和提交之前检查 token;网络请求使用可中断句柄,不能仅在一行输出结束后才检查。
4. 收到 ACK 仍不等于任务停止。等待 worker 报 stopped、落未完成副作用和用量,再 CAS 到 cancelled。
5. 默认 2s 协作宽限、额外 3s 进程退出宽限,之后回收进程组。强制退出记 interrupted/needs_reconciliation,不能伪造干净 cancelled。
完成与取消竞争的线性化点是控制存储的 CAS:终态先提交则 cancel 返回 already_terminal;取消先提交则晚到 result 只可作为证据追加,不能重新变 succeeded。若外部动作已发生但响应晚到,记录 effect outcome,不撤销取消事实。
兼容文件 token 统一:不存在/空/`0` 为未取消,`1`/`cancelled` 为取消;读取失败返回诊断并按明确策略处理。删除 token 只能发生在持有者退出之后。移除全局 `MARQDO_CANCEL` 对多个任务的隐式作用域。
## 7. 事件、SSE 与背压
### 7.1 事件权威
生产者发出 `(run_id, attempt_id, local_seq)`,控制服务去重、持久化并分配 task 内单调 seq/event_id;客户端游标来自持久化日志,不是 handler 重连后从 1 开始计数。终态事件唯一,usage 可以在终态后以更高 usage revision 补录,但不能再次结束任务。
```json
{
"schema": "RuntimeEventV2",
"schema_version": 2,
"event_id": "task_01:42",
"seq": 42,
"task_id": "task_01",
"run_id": "run_01",
"attempt_id": "attempt_01",
"type": "message.delta",
"payload": {"message_id": "msg_01", "offset": 24, "text": "结果"}
}
```
不要直接破坏 RuntimeEventV1;新增字段/新版本由能力协商确定。普通用户可见 progress 只描述工具、阶段、文件等结构化事实;模型内部 reasoning 不作为完成证据,也不默认写入公开事件。
### 7.2 App SSE
新增任务 API 的事件订阅用 `GET /tasks/{id}/events`(最终产品前缀见配套设计),发送 SSE `id`/`event`/`data`;每个 data 是一个完整 JSON。客户端支持 Last-Event-ID、CRLF、多行 data、任意 UTF-8 分块和 EOF 未完成帧。重复事件按 ID 去重;message offset 检查缺口,缺口时恢复快照。
心跳 15s,客户端空闲探测建议 45s;二者可配置。事件保留建议 24h,最终结果/审计按项目政策更长保留。游标早于保留边界返回 410 `event.cursor_expired` 和快照入口,不能默默从最新开始丢字。
订阅断开默认只关闭订阅。App 显式停止通过 cancel API,切页/移动网络重连可恢复同一任务。传统同步 OpenAI 调用可保持 disconnect=cancel 的兼容策略,但必须显式区分,不能统一把所有浏览器断流当取消。
### 7.3 OpenAI 兼容流
`/v1/chat/completions stream=true` 输出 OpenAI chunk 与 `[DONE]`;App 运行事件由独立入口或协商的协议提供。禁止仅凭任意 JSON body 中 stream=true 将不相关路由自动改为 RuntimeEvent 协议。已发 header 后的错误使用该协议允许的错误帧;发 header 前可用 429/503/401 等正确状态码。
token delta 与最终完整答复分开:最终事件持有结果引用/摘要和 finish reason,不再次追加整篇正文。无 token 能力的适配器报告 `evented_batch`,界面用阶段状态,不把整包人为拆字伪装成实时。
### 7.4 背压和旧桥修复
worker→事件服务与服务→订阅者均有界。建议每订阅者 256 KiB 待发送内存,超出后断开慢消费者并允许按游标重连;事件正文已持久化所以不能静默丢 token。允许合并短时间 delta,必须保留 message offset;usage/终态/错误不丢弃。磁盘压力时停止接纳新任务,已有任务使用保留终态空间并按策略中断,不能无限写事件。
legacy NDJSON tailer 只消费换行已完整到达的字节;保留残片,offset 按真实消费字节移动。超长行、I/O error、文件旋转显式报协议错误。测试至少分三次写一个 JSON、中文多字节边界、恰好 EOF、超过上限、取消与尾帧竞争。禁止吞掉 Scanner.Err() 继续宣告成功。
## 8. 只读快照与真正有效的缓存
### 8.1 快照一致性
分析服务构建 SnapshotKey:规范化 project identity、允许扫描范围、源文件闭包摘要、依赖/来源/配置摘要、compiler/lib/ext 版本、policy/schema、解析选项。输入由编译器枚举,不另造“前 200 个根目录文件”规则。
读取一次快照内的正文、语义和来源,完成后复核输入 manifest;文件在分析中发生变化则重试一次或返回 `snapshot.changed`。对不可变发布代直接读其 manifest。mtime/size 只用于快速发现候选变更,不能代替内容摘要;watcher 丢事件后以完整扫描修复。
### 8.2 缓存结构
- `AnalysisCache<SnapshotKey, Arc<Compilation>>`:实际编译结果。
- `DocumentCache<(SnapshotKey, document_digest, renderer_version, view_options), DocumentView>`:安全 fragment、outline、nodes。
- 同 key single-flight 的 follower 等待同一个 Result,不再次 compile。
- RAII guard 在 error/panic/cancel 时清除 inflight 并唤醒 waiters;等待者有自己的 deadline。
- compile 失败只返回给当前等待者,短暂错误缓存需显式 TTL;不得调用“finish_success”标记失败结果可复用。
- LRU 按字节而非条数上限管理;初始预算按部署内存的 10% 且设硬上限,长文档超限不缓存或分页;记录淘汰原因。
- 每次读取先授权再查缓存;权限敏感的响应按 scope/policy 分区。全局缓存只共享不含用户秘密且访问已验证的不可变语义数据。
性能证据必须统计 `compile_invocations`、输入字节、缓存结果命中、leader/follower、wait_ms、evictions;现有 hit counter 不能证明减少编译。
### 8.3 HTML 与来源定位
沿用 DocumentViewV1,明确 `render.format=plain_text|sanitized_html`。HTML 通过解析器白名单清洗标签/属性/URL scheme,剥离脚本、事件属性、危险 CSS/嵌入与可执行 SVG;相对资源经受限资源端点解析。纯文本始终转义,不以 `<` 字符判断类型。
node_id、revision、generation 必须从定位到读取到浏览器滚动完整传递。不存在历史快照返回 version_unavailable;返回当前版是用户显式操作且标明版本变化。no_exec 测试分别验证“无语言副作用”和“浏览器无脚本执行”,二者不能互相替代。
### 8.4 首次编译的性能改造
本轮 `v3_corpus_1000_compile_smoke` 在 debug 下 243.012s,超过既有 180s 门禁;缓存不能缩短一个全新项目的第一次编译。先为 `knowledge::compile` 的语义编译、语义落盘、检索 extract、检索落盘、manifest 分别加 span,再细分 `ProjectIndex::build`、源枚举/解析、导入挂接、绑定、效应传播、闭包摘要。记录文件数/总字节、节点/边数、parse 次数与序列化字节量。
源码中 `sema::compile_project` 在每个文件查找 module path、每个 module 过滤全量 symbols,属于值得测量的重复扫描;未取得 profile 前不将其认定为 243s 的唯一根因。优化候选按证据选择:建立 path→module 与 module→symbols 索引;同一输入快照共享 parsed module;绑定/依赖图用稳定索引;效应传播用 worklist;闭包按 SCC DAG 计算并保留循环依赖语义;减少每制品重复序列化和小文件写入。
任何优化须保持排序确定性、SymbolId/ArtifactId 与证据摘要规则,禁止通过跳过来源校验、删掉复杂 fixture 或改大超时过关。用 100/1000/10000 单元、无依赖/链/菱形/循环四种图分别测时间与 RSS;给出随 V/E 增长的曲线。release 和 debug 分开;已有 debug 门禁恢复通过只是第一步,§12 的 release 预算仍须独立验收。
## 9. 写入、发布与数据库边界
执行 worker 只写任务 staging;需要写工程时通过沙箱产生文件 diff。发布以 kb 资源锁 + expected_generation CAS 提交。锁只覆盖校验当前版本和切换 manifest,不覆盖 LLM、全文编译或网络调用。
完整发布代先写临时目录、校验摘要、完成必要 fsync,再原子 rename 到不可变 generation 目录;最后在同一文件系统原子替换 CURRENT。源码、编译投影和证据在该 manifest 下对齐。SQLite 中的产品“published”是投影,不参与虚构的文件/数据库跨系统原子事务。通过 publish journal + outbox 恢复:CURRENT 已切换但 DB 未更新时重放;未切换时旧代继续有效。
project_id 与逻辑文档 ID 不随 staging/generation 的物理目录名变化。manifest 保存稳定 project identity、逻辑相对路径和实际 blob/generation 映射;导入引用按项目规则解析,不能因复制到发布目录而重新生成全部符号身份。框架若尚不能区分逻辑身份与物理根,先补该能力,再启用 generation 目录发布;迁移 fixture 必须断言未修改的知识 ID 和跨文档绑定保持稳定。
任务表/幂等/CAS/outbox 需要通用事务能力:先审计 `ext/data` 是否支持事务、条件更新、受影响行数、复合唯一键;缺少则在 Marqdo 数据层补齐,不能用 `.mq.md` 的先读后写模拟原子性。SQLite 使用短事务和有界 busy timeout;WAL/连接池是否开启需按现有插件适配测试确认。单主本地盘为首期部署范围,跨主机共享 SQLite/NFS 不在首期承诺内。
## 10. 临时缓解与迁移
### 10.1 进程池完成前
- 保留 1500ms busy 预算和 Retry-After;展示繁忙,而非无限 spinner。
- 在接纳任何 SSE 之前取得覆盖整个 bridge 生命周期的 legacy 单飞许可,第二条流在修改环境变量/发送 200 前失败;普通请求不得绕过许可观察被污染的状态。
- 断连只标记该任务取消,清理由监督者等待执行结束后负责;无法确认停止时报告 stopping/interrupted。
- 上述步骤只降低 R02/R03 风险,仍不算并行 Endpoint。
### 10.2 功能开关(新增提案)
`runtime.isolated_execution`、`runtime.task_control_v2`、`runtime.event_replay`、`view.analysis_cache_v2`、`runtime.read_lane`。探测返回 feature version 和限制。二进制版本相同不等于所有 native 插件都有该能力。
先离线 fixture → 本地假模型 → 新 workspace → 只读 ASK → staging BUILD → 真正发布。禁止对有副作用的请求做双执行 shadow;允许 shadow 只读分析并比较摘要。
回滚先停接新任务、drain 已接纳任务、保留任务/事件和新 schema;不能将活动任务交给不认识 lease 的旧进程。只读当前发布代可以保留,写入能力按版本协商关闭。ADR 0008 仅在并行金样例与回滚演练通过后由新 ADR 正式替代。
## 11. 开发包与依赖
下列新模块路径是建议,不代表文件已存在。
| 包 | 建议落点与改动 | 前置 | 验收 |
|---|---|---|---|
| M-R0 基线/缓解 | `host/plugin.rs`、`httpx/routes_invoke.go`;固定并发屏障、旧桥许可、取消清理 | 无 | 双流 busy 不改写 A 的桥;断连无提前清理 |
| M-R1 请求上下文 | `workbench/run_context.rs`、`host/subtask.rs`、LLM 插件;统一 token 与显式 sink | M-R0 | 父子运行模型/事件/取消不串;空 token 不取消 |
| M-R2 Worker/IPC | 新 `runtime/{supervisor,worker,protocol,lease}.rs`;独立进程与握手 | M-R1 | 两任务并行、超时回收、子进程无孤儿 |
| M-R3 接纳/控制 | 新 `runtime/{task_store,scheduler,control}.rs` 与 ext/data 事务 primitives | M-R2 | 原子幂等、队列公平、取消与结果 CAS |
| M-R4 事件协议 | `workbench/event_sink.rs`、Web stream/IPC、协议适配器 | M-R3 | 断点恢复/去重/半帧/慢消费/磁盘错误 |
| M-R5 读服务/缓存 | `analysis_cache.rs`、`document_view.rs`、`locator.rs`、MLSP transport | M-R1,可独立推进 | 100 同 key 真正只编译一次;嵌套变更正确失效 |
| M-R5b 首次编译性能 | `knowledge/compile.rs`、`sema/ir.rs`、ProjectIndex;分阶段 profile 与按热点优化 | 基线语义回归 | 现有 1000 单元门禁恢复;release 与规模曲线报告,身份和语义不变 |
| M-R6 提交底座 | `knowledge` generation/manifest、publish journal、条件更新 | M-R3 | 崩溃点恢复,旧代不被半发布污染 |
| M-R7 整合与发布 | 版本探测、CI、release、文档/ADR | M-R4–R6 + 产品接入 | §12 全部必要门禁与回滚演练 |
先以本地文件和单机进程解决明确缺陷;不要把集群、远程 worker、复杂 Actor 系统设为第一阶段依赖。
## 12. 验收、性能预算与可观测性
### 12.1 正确性矩阵
| 编号 | 场景 | 必须观察到的结果 |
|---|---|---|
| MR-01 | A/B 两个 60s 假模型流 + 20 次/s 模型/设置读取 | 两流不混字、不混模型;短请求 200;无 interpreter busy |
| MR-02 | B 在 A 运行时接纳失败/断连 | A 事件和取消句柄完全不变 |
| MR-03 | 同时取消 A、完成 B;取消重复/晚到 | A 正确停;B 成功;终态不回退;审计只结算一次 |
| MR-04 | 帧拆字节、多行、EOF 半帧、重连 | 无丢字/重字;协议错误可见;游标恢复精确 |
| MR-05 | 慢消费者、磁盘满、worker 崩溃 | 内存有界;事件不静默丢失;状态可恢复 |
| MR-06 | 100 个相同读取并发 | 实际编译一次;后续结果复用;失败 leader 不挂死 follower |
| MR-07 | 改子目录依赖、第 201 个文件、配置、来源 | SnapshotKey 改变;缓存失效;旧业务句柄拒绝 |
| MR-08 | 上传有写文件/网络代码、HTML 事件属性 | 阅读不求值;浏览器脚本不执行;正文可读 |
| MR-09 | 发布每个持久化边界 kill | 仅旧完整代或新完整代可见;DB 投影最终追上 |
| MR-10 | 并发幂等请求、取消队列、权限撤回 | 单一 task;同键异内容 409;无越权派发 |
| MR-11 | kill 后副作用未知 | 不自动重放,显示 needs_reconciliation |
| MR-12 | 降级/升级、混合 CLI/ext/native | 缺能力明确禁用,不报已支持 |
### 12.2 性能目标(待测验收预算)
基准机:Linux 4 vCPU/8 GiB SSD;release 构建;1000 个知识单元、文档总量 10 MiB、500 条历史 run;假上游固定 60s、每秒 20 个增量;30s 预热、5min 采样、3 轮,报告 p50/p95/p99、错误率、CPU、RSS、磁盘。另测 10k 单元和 100 MiB 以确定容量上限,不能直接沿用小数据阈值。
| 指标 | 初始预算 | 说明 |
|---|---|---|
| 短读/任务接纳 | p95 ≤200ms,p99 ≤500ms | 两个长任务运行时;过载拒绝另统计 |
| 取消受理 | p95 ≤200ms | 独立于任务停止耗时 |
| 协作取消收敛 | p95 ≤2s | 假上游/可中断工具;不可中断外部动作按不确定处理 |
| 已就绪 worker 调度 | p95 ≤100ms | 不含排队、模型首 token 时间 |
| 事件到订阅者新增延迟 | p95 ≤100ms | 从已收到 provider delta 计,不承诺上游 TTFT |
| 1000 单元首次分析 | p95 ≤2s | 候选目标,超标先剖析,不降低正确性 |
| 热 DocumentView | p95 ≤100ms | 真实复用结果;长文档另测分页 |
| 慢消费者缓冲 | 每订阅 ≤256 KiB | 总连接数也必须受限 |
| 压测结束内存 | 稳态可回收、无持续增长 | 记录每次重复轮次 RSS;不能只看一次峰值 |
新指标至少有 queue_depth/wait、worker_active/recycle、admission_rejected、cancel_ack/stop、orphan_count、event_lag/replay/duplicate、compile_invocations/cache_result_hit、publish_conflict/recovery。指标标签禁止 task_id、原始路径、prompt 等高基数/敏感值;trace 中可记录脱敏 task/attempt/lease 与构建摘要。
现有测试继续保留:`cargo test --offline --locked --test workbench_contracts --test executable_knowledge`、LLM 插件单测、Go httpx 测试。新增集成门禁必须启动真实 HTTP→IPC→worker 路径并使用本地假模型;不能以手写 fixture 或写 token 文件代替端到端行为。本轮执行结果统一记录在[跨库复核 §5](../../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md)。
+163
View File
@@ -0,0 +1,163 @@
# Web 热路径:单解释器串行化(设计缺陷说明)
| | |
|---|---|
| 状态 | **Accepted**(缺陷记录 + 缓解已落地) |
| 日期 | 2026-10-26 |
| 规范决策 | [ADR 0008](../adr/0008-web-invoke-serialization.md) |
| 相关 | [ext-web-go-rewrite.md](ext-web-go-rewrite.md) · [ext-abi.md](ext-abi.md) · [ADR 0004](../adr/0004-web-plugin-go.md) · [ADR 0007](../adr/0007-web-document-endpoint.md) |
本文把「页面卡死 / 像单线程」拆成可验证的架构事实、错误语义与演进边界。
**结论先行:HTTP 是多线程的;卡住是因为所有 Endpoint 共享一把解释器锁。**
增量实施提案见 [隔离执行与响应式 Web 运行时(2026-10-26)](isolated-web-runtime-development-2026-10-26.md):独立进程池、控制/读取容量、请求级事件与取消、缓存和恢复。提案通过验收前,本文件及 ADR 0008 的现行单飞约束不变。
---
## 1. 问题陈述
### 1.1 用户可见症状
1. 发起一次长对话(OpenAI 兼容 `POST …/v1/chat/completions` + `stream:true` / `sse-live`)。
2. 对话尚未结束时,打开设置、拉 `/v1/models`、或其它短 JSON API。
3. **旧行为**:浏览器请求一直 pending,整页像死机;只能杀 `marqdo run serve…` 恢复。
4. **目标行为**:短请求在约 1.5s 内得到 **HTTP 503**,错误体标明 `interpreter busy`;对话 SSE 继续;结束后短请求立刻恢复 200。
### 1.2 错误归因(不要再说错)
| 错误说法 | 正确说法 |
|----------|----------|
| 「web 插件是单线程的」 | Go `net/http` 与 SSE 刷写是多 goroutine |
| 「忘记启动进程所以卡死」 | 进程在跑;是 **锁争用** 导致短请求永不返回 |
| 「把 SSE 的等待时间调大就能修好」 | 进程级拉长 `MARQDO_INVOKE_WAIT_MS` 会让短请求一起卡死 |
| 「去掉 INVOKE_LOCK 即可并行」 | `HostContext` 非 Sync,无锁并行会数据竞争 / 未定义行为 |
---
## 2. 调用栈与锁边界
```
Browser
│ GET /v1/models POST /v1/chat/completions (SSE)
▼
Go plugins/web (多 goroutine)
handleInvoke / handleInvokeLiveSSE
│
▼ C ABI host_query("call_lib_path", {path, args})
Rust src/host/plugin.rs
┌─────────────────────────────┐
│ INVOKE_LOCK (进程级 Mutex) │ ← 同一时刻只允许一个 worker 进入
└─────────────────────────────┘
│
▼
HostContext:装载/求值作者 .mq.md、写回、事件总线、LLM 插件回调…
```
要点:
1. **锁在宿主,不在 Go 业务路由表。** 增加更多 Go handler 不会自动并行执行 Marqdo。
2. **SSE 持锁时间 ≈ 整次 Endpoint 求值时间**(含多次 LLM 往返),可达数十秒到数分钟。
3. **事件转发**(`MARQDO_EVENT_FORWARD` NDJSON)与 **取消**(`MARQDO_CANCEL_TOKEN`)在持锁期间仍可由其它 goroutine 写文件/读桥;它们不进入解释器,故不受 `INVOKE_LOCK` 保护,也不应误设长等待。
---
## 3. 为何不能简单「多线程跑 Marqdo」
`HostContext` 持有(非穷尽):
- 当前模块图、函数表、写回缓冲与 FS 根;
- 插件注册与 `CURRENT_HOST` / 全局 host 指针;
- Agent / LLM 运行时与 stream 发布钩子;
- 与文档写回、会话态相关的可变结构。
这些状态在实现上按**单线程求值**建模。把它标成 `Sync` 或去掉 `INVOKE_LOCK` 而没有隔离运行时,属于未定义行为,不是性能优化。
**真并行的合法方向(未实现):**
1. **解释器池**:每请求或每会话一个 `HostContext`(或可重置的 isolate),路由层选闲置实例;
2. **只读快照 + 写队列**:热路径只读克隆,写回串行合并(适合纯查询 Endpoint,不适合带写回的 agent);
3. **把长任务移出解释器**:Marqdo 只做编排,重活在独立进程/队列(与「代码即文档」主路径权衡后再定)。
在上述任一方案落地并经金样例验收前,**产品必须以「单飞解释器」为约束设计 UX**。
---
## 4. 缓解契约(已实现)
### 4.1 宿主:限时争锁
文件:`src/host/plugin.rs`
- 仅当从 worker 线程进入(`CURRENT_HOST` 为空、走全局 host)时争 `INVOKE_LOCK`。
- `acquire_invoke_lock`:`try_lock` 循环,预算默认 **1500ms**(`MARQDO_INVOKE_WAIT_MS`,最小 1)。
- 超时:`set_err("interpreter busy: another request holds the Marqdo invoke lock (waited …ms)")`,返回非 0。
- **不要**在 SSE 入口把该环境变量设成数分钟:它是进程级的,会污染并发短请求。
### 4.2 Web 插件:503
文件:`plugins/web/internal/httpx/routes_invoke.go`
- 普通 JSON invoke:错误信息含 `interpreter busy` → **503** + `Retry-After: 2` + `{ok:false, error:…}`。
- Live SSE:若启动时即 busy,以失败事件/错误结束该流,**不得**无限挂起握手。
### 4.3 客户端:超时与文案
短 API(模型列表、设置、registry 探测等)必须:
- 设置合理 `fetch` 超时(秒级,例如 8s);
- 将 503 / `interpreter busy` 展示为「服务正忙(对话进行中)」类可读提示,而不是空白转圈。
(qdagent 前端 `frontend/src/lib/api.ts` 已按此契约处理。)
### 4.4 验收探针
在本地 serve 上:
```bash
# 终端 A:启动长 SSE 对话(持锁)
curl -N -m 120 -H 'Authorization: Bearer …' -H 'Content-Type: application/json' \
-d '{"stream":true,…}' http://127.0.0.1:PORT/v1/chat/completions
# 终端 B:对话进行中
curl -sS -w 'http=%{http_code} time=%{time_total}\n' --max-time 5 \
-H 'Authorization: Bearer …' http://127.0.0.1:PORT/v1/models
# 期望:http=503,time≈1.5,body 含 interpreter busy
# 对话结束后再打同一 URL → http=200,time≪1s
```
---
## 5. 运维与调参
| 变量 | 默认 | 含义 |
|------|------|------|
| `MARQDO_INVOKE_WAIT_MS` | `1500` | worker 进入解释器前最长等待;超时 → busy |
建议:
- 开发机 / 产品默认保持 1–3s。
- **禁止**为「让第二个聊天排队」而把默认调到分钟级——那等于恢复卡死。
- 需要排队时,应在 **应用层**(会话队列、UI 禁用「再发一条」),而不是拉长宿主锁等待。
---
## 6. 与其它机制的边界
| 机制 | 与 INVOKE_LOCK 的关系 |
|------|----------------------|
| `MARQDO_EVENT_FORWARD` | SSE 期间写 NDJSON;不替代争锁;不延长等待 |
| `MARQDO_CANCEL_TOKEN` | 断开连接时可取消 LLM;释放锁仍取决于解释器何时回到 FFI |
| `web.serve` / 静态资源 | 通常不经 `call_lib_path`,不受本锁影响 |
| 插件内部 Tokio/Go 并发 | 可以;只要不再次 `call_lib_path` 进同一 HostContext |
---
## 7. 演进检查清单(根治前勿宣称「已支持并行 Endpoint」)
- [ ] 多 `HostContext` 或 isolate 的生命周期与 ABI(装载、shutdown、写回冲突)有设计稿与金样例。
- [ ] 池化后 `call_lib_path` 不再依赖进程级单锁,或锁粒度降到实例级。
- [ ] 两个并发长 SSE + 一个短 `/v1/models` 的压力测试:短请求稳定 200,长请求互不污染会话写回。
- [ ] 更新本文件与 ADR 0008 Status(例如改为 Superseded by ADR NNNN)。
在清单完成前,对外与文档中应明确:**Marqdo Web Endpoint 求值在单进程内串行;HTTP 层并发,解释器层单飞。**
+1 -1
View File
@@ -1 +1 @@
2.1.3
2.1.4
+307 -140
View File
@@ -8,6 +8,7 @@ import:
sys: lib/sys.mq.md
fs: lib/fs.mq.md
time: lib/time.mq.md
text: lib/text.mq.md
wb: lib/writeback.mq.md
subtask: lib/subtask.mq.md
plugin: lib/plugin.mq.md
@@ -58,30 +59,45 @@ The runbook `.mq.md` is ground truth (code as documentation). Default budgets ke
+ `prompt`
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"
One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{text,llm_calls,tokens}` for episode metrics; transport failures set `error` + `llm_calls=0` (never abort).
`stream_kind=plan` publishes protocol tokens onto the process trail (not the answer bubble).
1. `stream`
**evs = > model.stream prompt=`prompt` echo=`echo`**
**evs = > model.stream prompt=`prompt` echo=`echo` stream_kind=`stream_kind`**
**text = > llm.collect events=`evs`**
**tokens = None**
**tokens = 0**
**err = None**
**calls = 1**
1. not `text`
**e0 = > at value=`evs` index=0**
**err = > json.get value=`e0` key="message"**
1. `err`
**calls = 0**
**e0_s = > json.stringify value=`e0`**
**e0_map = > text.starts_with text=`e0_s` prefix="{"**
1. `e0_map`
**err = > json.get value=`e0` key="message"**
1. `err`
**calls = 0**
2. *
**_ = 1**
2. *
**r = > model.ask prompt=`prompt`**
**text = > json.get value=`r` key="text"**
**tokens = > json.get value=`r` key="tokens"**
**err = > json.get value=`r` key="error"**
**fin = > json.get value=`r` key="finish"**
1. `tokens`
**_ = 1**
**r_s = > json.stringify value=`r`**
**r_map = > text.starts_with text=`r_s` prefix="{"**
1. `r_map`
**text = > json.get value=`r` key="text"**
**tokens = > json.get value=`r` key="tokens"**
**err = > json.get value=`r` key="error"**
**fin = > json.get value=`r` key="finish"**
2. *
**tokens = None**
**text = > str value=`r`**
**tokens = 0**
**err = None**
**fin = None**
1. not `tokens`
**tokens = 0**
2. *
**_ = 1**
1. `err`
**calls = 0**
2. `fin` == "error"
@@ -183,7 +199,7 @@ Allowlist check, then invoke the runbook `##` via `lib/subtask` (`spawn fn=` →
Build a runnable workbook. `skeleton=single` (default): one worker `step`. `skeleton=dual`: research then write agents. On success the parent chooses DONE; runtime `agent_workbook_solidify` freezes the answer.
**q = > json.quote text=`goal`**
**esc = > json.parse text={"h":"---\ntitle: agent workbook\nimport:\n llm: ext/ai/llm.mq.md\n agent: ext/ai/agent.mq.md\n json: lib/json.mq.md\n sys: lib/sys.mq.md\n wb: lib/writeback.mq.md\n subtask: lib/subtask.mq.md\n---\n\n# Goal\n\nParent goal is embedded as JSON in # main.\n\n## Solidify\n\nWhen the child returns a good value, parent DECISION: DONE — runtime agent_workbook_solidify freezes the answer as **return** (do not invent long FIND/REPLACE).\n\n# main\n\n> llm.load_env\n\n","s1":"\u002a\u002a\u0060model\u0060 = > llm.create model=\u0060sys.env_get name=\"OPENAI_MODEL\"\u0060 base_url=\u0060sys.env_get name=\"OPENAI_BASE_URL\"\u0060 backend=openai name=workbook\u002a\u002a\n","s2":"\u002a\u002a\u0060tools\u0060 = > table.empty_list\u002a\u002a\n","s3":"\u002a\u002a\u0060worker\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You are a workbook worker. tools=[] means never CALL. Finish the task with a direct final answer; do not invent tools.\u002a\u002a\n","s4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","s5":"}\u002a\u002a\n","s6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","s7":"\u002a\u002a\u0060out\u0060 = > \u0060worker\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","s7b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","s8":"\u002a\u002a\u0060text\u0060\u002a\u002a\n","d3":"\u002a\u002a\u0060research\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You gather facts for the goal. Be concise. Do not invent tools.\u002a\u002a\n","d3b":"\u002a\u002a\u0060writer\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You write the final answer from research notes. Do not invent tools.\u002a\u002a\n","d4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","d5":"}\u002a\u002a\n","d6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","d7":"\u002a\u002a\u0060notes\u0060 = > \u0060research\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","d8":"\u002a\u002a\u0060ns\u0060 = > json.stringify value=\u0060notes\u0060\u002a\u002a\n","d9":"\u002a\u002a\u0060wt\u0060 = Write the final answer. Research notes:\u002a\u002a\n","d10":"\u002a\u002a\u0060wt\u0060 = \u0060wt\u0060 + \u0060ns\u0060\u002a\u002a\n","d11":"\u002a\u002a\u0060out\u0060 = > \u0060writer\u0060.step task=\u0060wt\u0060 stream=False\u002a\u002a\n","d11b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","d12":"\u002a\u002a\u0060text\u0060\u002a\u002a\n"}**
**esc = > json.parse text={"h":"---\ntitle: agent workbook\nimport:\n llm: ext/ai/llm.mq.md\n agent: ext/ai/agent.mq.md\n json: lib/json.mq.md\n sys: lib/sys.mq.md\n table: lib/table.mq.md\n wb: lib/writeback.mq.md\n subtask: lib/subtask.mq.md\n---\n\n# Goal\n\nParent goal is embedded as JSON in # main.\n\n## Solidify\n\nWhen the child returns a good value, parent DECISION: DONE — runtime agent_workbook_solidify freezes the answer as **return** (do not invent long FIND/REPLACE).\n\n# main\n\n> llm.load_env\n\n","s1":"\u002a\u002a\u0060env_model\u0060 = > sys.env_get name=\"OPENAI_MODEL\"\u002a\u002a\n\u002a\u002a\u0060env_base\u0060 = > sys.env_get name=\"OPENAI_BASE_URL\"\u002a\u002a\n\u002a\u002a\u0060env_key\u0060 = > sys.env_get name=\"OPENAI_API_KEY\"\u002a\u002a\n\u002a\u002a\u0060model\u0060 = > llm.create model=\u0060env_model\u0060 base_url=\u0060env_base\u0060 api_key=\u0060env_key\u0060 backend=\"openai-compatible\" name=\"workbook\"\u002a\u002a\n","s2":"\u002a\u002a\u0060tools\u0060 = > table.empty_list\u002a\u002a\n","s3":"\u002a\u002a\u0060worker\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=\"You are a workbook worker. tools=[] means never CALL. Finish the task with a direct final answer; do not invent tools.\"\u002a\u002a\n","s4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","s5":"}\u002a\u002a\n","s6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=\"task\"\u002a\u002a\n","s7":"\u002a\u002a\u0060out\u0060 = > \u0060worker\u0060.step task=\u0060task\u0060 stream=True\u002a\u002a\n","s7b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=\"result\"\u002a\u002a\n","s8":"\u002a\u0060text\u0060\u002a\n","d3":"\u002a\u002a\u0060research\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=\"You gather facts for the goal. Be concise. Do not invent tools.\"\u002a\u002a\n","d3b":"\u002a\u002a\u0060writer\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=\"You write the final answer from research notes. Do not invent tools.\"\u002a\u002a\n","d4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","d5":"}\u002a\u002a\n","d6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=\"task\"\u002a\u002a\n","d7":"\u002a\u002a\u0060notes\u0060 = > \u0060research\u0060.step task=\u0060task\u0060 stream=True\u002a\u002a\n","d8":"\u002a\u002a\u0060ns\u0060 = > json.stringify value=\u0060notes\u0060\u002a\u002a\n","d9":"\u002a\u002a\u0060wt\u0060 = \"Write the final answer. Research notes:\"\u002a\u002a\n","d10":"\u002a\u002a\u0060wt\u0060 = \u0060wt\u0060 + \u0060ns\u0060\u002a\u002a\n","d11":"\u002a\u002a\u0060out\u0060 = > \u0060writer\u0060.step task=\u0060wt\u0060 stream=True\u002a\u002a\n","d11b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=\"result\"\u002a\u002a\n","d12":"\u002a\u0060text\u0060\u002a\n"}**
**h = > json.get value=`esc` key="h"**
**s1 = > json.get value=`esc` key="s1"**
**s2 = > json.get value=`esc` key="s2"**
@@ -1080,8 +1096,15 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
*> lib_catalog*
4. `name` == scratch_tool_write
**parsed = > extract_scratch_tool reply=`reply`**
**tn = > json.get value=`parsed` key="name"**
**tt = > json.get value=`parsed` key="text"**
**parsed_s = > json.stringify value=`parsed`**
**parsed_map = > text.starts_with text=`parsed_s` prefix="{"**
**tn = None**
**tt = None**
1. `parsed_map`
**tn = > json.get value=`parsed` key="name"**
**tt = > json.get value=`parsed` key="text"**
2. *
**_ = 1**
**errs = > json.parse text={"a":"scratch_tool_write needs NAME line and fenced body","b":"scratch_tool_write needs NAME id"}**
1. `tn`
1. `tt`
@@ -1092,9 +1115,17 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
*> json.get value=`errs` key="b"*
5. `name` == corpus_search
**args = > extract_call_args reply=`reply`**
**q = > json.get value=`args` key="query"**
**root = > json.get value=`args` key="root"**
**limit = > json.get value=`args` key="limit"**
**args_s = > json.stringify value=`args`**
**args_map = > text.starts_with text=`args_s` prefix="{"**
**q = None**
**root = None**
**limit = None**
1. `args_map`
**q = > json.get value=`args` key="query"**
**root = > json.get value=`args` key="root"**
**limit = > json.get value=`args` key="limit"**
2. *
**_ = 1**
**cerrs = > json.parse text={"q":"corpus_search needs ARGS.query"}**
1. not `q`
*> json.get value=`cerrs` key="q"*
@@ -1109,7 +1140,13 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
*> corpus_search query=`q`*
6. `name` == mcp_list_tools
**args = > extract_call_args reply=`reply`**
**fx = > json.get value=`args` key="fixture"**
**args_s = > json.stringify value=`args`**
**args_map = > text.starts_with text=`args_s` prefix="{"**
**fx = None**
1. `args_map`
**fx = > json.get value=`args` key="fixture"**
2. *
**_ = 1**
**merrs = > json.parse text={"f":"mcp_list_tools needs ARGS.fixture"}**
1. `fx`
*> mcp_list_tools fixture=`fx`*
@@ -1117,9 +1154,17 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
*> json.get value=`merrs` key="f"*
7. `name` == mcp_call
**args = > extract_call_args reply=`reply`**
**fx = > json.get value=`args` key="fixture"**
**tn = > json.get value=`args` key="name"**
**targ = > json.get value=`args` key="arguments"**
**args_s = > json.stringify value=`args`**
**args_map = > text.starts_with text=`args_s` prefix="{"**
**fx = None**
**tn = None**
**targ = None**
1. `args_map`
**fx = > json.get value=`args` key="fixture"**
**tn = > json.get value=`args` key="name"**
**targ = > json.get value=`args` key="arguments"**
2. *
**_ = 1**
**xerrs = > json.parse text={"f":"mcp_call needs ARGS.fixture","n":"mcp_call needs ARGS.name"}**
1. `fx`
1. `tn`
@@ -1180,14 +1225,28 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
+ `observation`
Bounded observation for the parent prompt (actionable, short).
Tolerates missing/non-map observation (empty compact) instead of `json get needs map`.
Map check via stringify prefix `{` — never hard-get on None / bare text / list.
**obs_path = > json.get value=`observation` key="path"**
**obs_code = > json.get value=`observation` key="exit_code"**
**obs_val = > json.get value=`observation` key="value"**
**obs_slots = > json.get value=`observation` key="slots"**
**obs_src = > json.get value=`observation` key="source"**
**obs_stdout = > json.get value=`observation` key="stdout"**
**obs_stderr = > json.get value=`observation` key="stderr"**
**obs_path = None**
**obs_code = None**
**obs_val = None**
**obs_slots = None**
**obs_src = None**
**obs_stdout = None**
**obs_stderr = None**
**obs_s = > json.stringify value=`observation`**
**obs_map = > text.starts_with text=`obs_s` prefix="{"**
1. `obs_map`
**obs_path = > json.get value=`observation` key="path"**
**obs_code = > json.get value=`observation` key="exit_code"**
**obs_val = > json.get value=`observation` key="value"**
**obs_slots = > json.get value=`observation` key="slots"**
**obs_src = > json.get value=`observation` key="source"**
**obs_stdout = > json.get value=`observation` key="stdout"**
**obs_stderr = > json.get value=`observation` key="stderr"**
2. *
**_ = 1**
**src_len = 0**
**has_step = False**
1. `obs_src`
@@ -1207,97 +1266,102 @@ Bounded observation for the parent prompt (actionable, short).
|------|------|-----------|------------|-----------------|------------------|
| Plan-and-Move: CALL/READ then DECISION. Success (exit 0 + has_value) → DONE. solidify_on_done. No long prose. | `obs_path` | `obs_code` | `src_len` | `has_step` | True |
1. `obs_src`
**ex = > json.get value=`observation` key="source_excerpt"**
1. `ex`
**compact = table.put in=`compact` at="source_excerpt" value=`ex`**
1. `obs_map`
1. `obs_src`
**ex = > json.get value=`observation` key="source_excerpt"**
1. `ex`
**compact = table.put in=`compact` at="source_excerpt" value=`ex`**
2. *
**ex = > workbook_source_excerpt source=`obs_src`**
**compact = table.put in=`compact` at="source_excerpt" value=`ex`**
2. *
**ex = > workbook_source_excerpt source=`obs_src`**
**compact = table.put in=`compact` at="source_excerpt" value=`ex`**
2. *
**_ = 1**
**_ = 1**
1. `obs_stderr`
**tail = > json.get value=`observation` key="stderr_tail"**
1. `tail`
**compact = table.put in=`compact` at="stderr_tail" value=`tail`**
1. `obs_stderr`
**tail = > json.get value=`observation` key="stderr_tail"**
1. `tail`
**compact = table.put in=`compact` at="stderr_tail" value=`tail`**
2. *
**tail = > text_tail_lines text=`obs_stderr`**
**compact = table.put in=`compact` at="stderr_tail" value=`tail`**
2. *
**tail = > text_tail_lines text=`obs_stderr`**
**compact = table.put in=`compact` at="stderr_tail" value=`tail`**
2. *
**_ = 1**
1. `obs_stdout`
**tail = > json.get value=`observation` key="stdout_tail"**
1. `tail`
**compact = table.put in=`compact` at="stdout_tail" value=`tail`**
**_ = 1**
1. `obs_stdout`
**tail = > json.get value=`observation` key="stdout_tail"**
1. `tail`
**compact = table.put in=`compact` at="stdout_tail" value=`tail`**
2. *
**tail = > text_tail_lines text=`obs_stdout`**
**compact = table.put in=`compact` at="stdout_tail" value=`tail`**
2. *
**tail = > text_tail_lines text=`obs_stdout`**
**compact = table.put in=`compact` at="stdout_tail" value=`tail`**
2. *
**_ = 1**
**_ = 1**
**val_txt = > str value=`obs_val`**
**val_len = > len value=`val_txt`**
1. `obs_val`
**compact = table.put in=`compact` at="has_value" value=True**
**compact = table.put in=`compact` at="value_len" value=`val_len`**
1. `val_len` > 200
**preview = > text_head_lines text=`val_txt` max_lines=4 max_chars=200**
**compact = table.put in=`compact` at="value_preview" value=`preview`**
**val_txt = > str value=`obs_val`**
**val_len = > len value=`val_txt`**
1. `obs_val`
**compact = table.put in=`compact` at="has_value" value=True**
**compact = table.put in=`compact` at="value_len" value=`val_len`**
1. `val_len` > 200
**preview = > text_head_lines text=`val_txt` max_lines=4 max_chars=200**
**compact = table.put in=`compact` at="value_preview" value=`preview`**
2. *
**compact = table.put in=`compact` at="value" value=`obs_val`**
2. *
**compact = table.put in=`compact` at="value" value=`obs_val`**
**compact = table.put in=`compact` at="has_value" value=False**
**slots_brief = > table.empty_list**
1. `obs_slots`
- [slot](遍历 `obs_slots` 为 `slot`)
**sk = > json.get value=`slot` key="key"**
**sl = > json.get value=`slot` key="line"**
**sb = > json.get value=`slot` key="body"**
**sb_txt = > str value=`sb`**
**sb_len = > len value=`sb_txt`**
`brief` =
| key | line | body_len |
|-----|------|----------|
| `sk` | `sl` | `sb_len` |
1. `sk` == error
**prev = > text_head_lines text=`sb_txt` max_lines=8 max_chars=400**
**brief = table.put in=`brief` at="body_preview" value=`prev`**
2. *
**_ = 1**
**slots_brief = > table.append list=`slots_brief` item=`brief`**
2. *
**_ = 1**
**compact = table.put in=`compact` at="slots" value=`slots_brief`**
**detail = > json.get value=`observation` key="slots_detail"**
1. `detail`
**compact = table.put in=`compact` at="slots_detail" value=`detail`**
2. *
**_ = 1**
**tool_result = > json.get value=`observation` key="tool_result"**
1. `tool_result`
**tr = > str value=`tool_result`**
**tr = > text_head_lines text=`tr` max_lines=40 max_chars=3000**
**compact = table.put in=`compact` at="tool_result" value=`tr`**
2. *
**_ = 1**
**last_error = > json.get value=`observation` key="last_error"**
1. `last_error`
**err_txt = > str value=`last_error`**
**err_len = > len value=`err_txt`**
1. `err_len` > 200
**prev = > text_head_lines text=`err_txt` max_lines=8 max_chars=400**
**compact = table.put in=`compact` at="last_error_preview" value=`prev`**
**compact = table.put in=`compact` at="last_error_len" value=`err_len`**
2. *
**compact = table.put in=`compact` at="last_error" value=`last_error`**
2. *
**_ = 1**
2. *
**compact = table.put in=`compact` at="has_value" value=False**
**slots_brief = > table.empty_list**
1. `obs_slots`
- [slot](遍历 `obs_slots` 为 `slot`)
**sk = > json.get value=`slot` key="key"**
**sl = > json.get value=`slot` key="line"**
**sb = > json.get value=`slot` key="body"**
**sb_txt = > str value=`sb`**
**sb_len = > len value=`sb_txt`**
`brief` =
| key | line | body_len |
|-----|------|----------|
| `sk` | `sl` | `sb_len` |
1. `sk` == error
**prev = > text_head_lines text=`sb_txt` max_lines=8 max_chars=400**
**brief = table.put in=`brief` at="body_preview" value=`prev`**
2. *
**_ = 1**
**slots_brief = > table.append list=`slots_brief` item=`brief`**
2. *
**_ = 1**
**compact = table.put in=`compact` at="slots" value=`slots_brief`**
**detail = > json.get value=`observation` key="slots_detail"**
1. `detail`
**compact = table.put in=`compact` at="slots_detail" value=`detail`**
2. *
**_ = 1**
**tool_result = > json.get value=`observation` key="tool_result"**
1. `tool_result`
**tr = > str value=`tool_result`**
**tr = > text_head_lines text=`tr` max_lines=40 max_chars=3000**
**compact = table.put in=`compact` at="tool_result" value=`tr`**
2. *
**_ = 1**
**last_error = > json.get value=`observation` key="last_error"**
1. `last_error`
**err_txt = > str value=`last_error`**
**err_len = > len value=`err_txt`**
1. `err_len` > 200
**prev = > text_head_lines text=`err_txt` max_lines=8 max_chars=400**
**compact = table.put in=`compact` at="last_error_preview" value=`prev`**
**compact = table.put in=`compact` at="last_error_len" value=`err_len`**
2. *
**compact = table.put in=`compact` at="last_error" value=`last_error`**
2. *
**_ = 1**
**empty_slots = > table.empty_list**
**compact = table.put in=`compact` at="slots" value=`empty_slots`**
*compact*
@@ -1351,6 +1415,14 @@ First matching DECISION / 决定 line wins (ignore later conflicting decisions i
2. *
**_ = 1**
1. `found`
Take the first token only (`DECISION: RUN please` → RUN).
**toks = > split value=`found` sep=" "**
**found = > at value=`toks` index=0**
**found = > trim value=`found`**
2. *
**_ = 1**
1. `found` == DONE
*"DONE"*
2. `found` == CONTINUE
@@ -1371,8 +1443,17 @@ First matching DECISION / 决定 line wins (ignore later conflicting decisions i
*"REUSE"*
10. `found` == 新建
*"NEW"*
11. *
*found*
11. `found` == PATCH
Models sometimes invent `DECISION: PATCH` — treat as CONTINUE.
*"CONTINUE"*
12. *
If the reply carries a FIND/REPLACE block, treat unknown labels as CONTINUE.
**has_find = > split value=`text` sep="<<<"**
**n_find = > len value=`has_find`**
1. `n_find` > 1
*"CONTINUE"*
2. *
*found*
---
@@ -1544,7 +1625,13 @@ Parent Plan-and-Move prompt. Short protocol only — no long monologues.
2. *
**up = None**
**skill_from_obs = > json.get value=`observation` key="skill_brief"**
**skill_from_obs = None**
**obs_probe = > json.stringify value=`observation`**
**obs_is_map = > text.starts_with text=`obs_probe` prefix="{"**
1. `obs_is_map`
**skill_from_obs = > json.get value=`observation` key="skill_brief"**
2. *
**_ = 1**
1. `skill_from_obs`
**skill = skill_from_obs**
2. *
@@ -1612,22 +1699,20 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
**left = 0**
3. *
**ctx = > build_plan_context agent=`agent` goal=`goal` observation=`last_obs` explore_attempt=`explore_attempt` explore_n=`explore_n` phase=`phase`**
Plan protocol streams as `stream_kind=plan` (process trail). Answer tokens
still come from workbook `worker.step stream=True` (`stream_kind=delta`).
1. `stream`
**evs = > model.stream prompt=`ctx` echo=`echo`**
**last_reply = > llm.collect events=`evs`**
**evs_all = > plan_merge_deltas events=`evs_all` from=`evs` stream=`stream`**
**e0 = > at value=`evs` index=0**
**terr = > json.get value=`e0` key="message"**
**etype = > json.get value=`e0` key="type"**
1. `etype` == "error"
**transport_err = `terr`**
1. not `transport_err`
**transport_err = "llm stream transport error"**
**left = 0**
2. *
**llm_calls = llm_calls + 1**
**turn = > model_turn model=`model` prompt=`ctx` stream=True echo=`echo` stream_kind="plan"**
2. *
**turn = > model_turn model=`model` prompt=`ctx` stream=False echo=`echo`**
**turn_s = > json.stringify value=`turn`**
**turn_map = > text.starts_with text=`turn_s` prefix="{"**
**terr = None**
**last_reply = ""**
1. not `turn_map`
**transport_err = "model_turn_not_map"**
**left = 0**
2. *
**terr = > json.get value=`turn` key="error"**
1. `terr`
**transport_err = `terr`**
@@ -1636,6 +1721,14 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
**last_reply = > json.get value=`turn` key="text"**
**c = > json.get value=`turn` key="llm_calls"**
**t = > json.get value=`turn` key="tokens"**
1. not `c`
**c = 0**
2. *
**_ = 1**
1. not `t`
**t = 0**
2. *
**_ = 1**
**llm_calls = llm_calls + c**
**tokens = tokens + t**
1. `transport_err`
@@ -1643,16 +1736,49 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
**last_reply = `transport_err`**
**left = 0**
2. *
**last_reply = > trim value=`last_reply`**
**act = > extract_plan_act reply=`last_reply`**
**kind = > json.get value=`act` key="kind"**
**name = > json.get value=`act` key="name"**
1. not `last_reply`
**last_reply = ""**
2. *
**last_reply = > trim value=`last_reply`**
Plan replies often embed CALL: inside PATCH bodies — prefer an explicit
DECISION line when present so protocol exit is not stolen by a false CALL.
**dec_line = > extract_plan_decision reply=`last_reply`**
**kind = "unknown"**
**name = None**
1. `dec_line`
**kind = "decision"**
**name = `dec_line`**
2. *
**act = > extract_plan_act reply=`last_reply`**
**act_s = > json.stringify value=`act`**
**act_map = > text.starts_with text=`act_s` prefix="{"**
1. `act_map`
**kind = > json.get value=`act` key="kind"**
**name = > json.get value=`act` key="name"**
2. *
**_ = 1**
Live act marker so the UI never goes silent between plan tokens and the next step.
1. `stream`
`act_ev` =
| type | kind | name |
|------|------|------|
| plan_act | `kind` | `name` |
> sys.stream_publish event=`act_ev`
2. *
**_ = 1**
1. `kind` == call
**evs_all = > plan_append_tool events=`evs_all` type="tool_start" name=`name` kind="call" stream=`stream`**
**tool_out = > run_parent_tool name=`name` path=`path` observation=`last_obs` reply=`last_reply`**
**brief = > tool_result_brief value=`tool_out`**
**evs_all = > plan_append_tool events=`evs_all` type="tool_end" name=`name` kind="call" result=`brief` stream=`stream`**
**last_obs = table.put in=`last_obs` at="tool_result" value=`tool_out`**
**obs_s = > json.stringify value=`last_obs`**
**obs_map = > text.starts_with text=`obs_s` prefix="{"**
1. `obs_map`
**last_obs = table.put in=`last_obs` at="tool_result" value=`tool_out`**
2. *
**_ = 1**
**left = left - 1**
2. `kind` == read
**evs_all = > plan_append_tool events=`evs_all` type="tool_start" name=`name` kind="read" stream=`stream`**
@@ -2103,6 +2229,12 @@ Primary author entry (v2 + P4): Adaptive `route` → execute (policy/kb skill) o
| ok | `task` | `child_val` | `path` | hit | `rmode` | `sk` | `last_obs` | adaptive route hit | `routed` |
**llm_calls = 0**
1. `stream`
**_ev = > table.empty_list**
**_ev = > plan_append_decision events=`_ev` decision="REUSE" stream=True summary="adaptive route hit"**
**_ev = > plan_append_round events=`_ev` round=1 workbook=`path` exit_code=`code` result=`child_val` stream=True**
2. *
**_ = 1**
2. *
**route_matched = False**
2. *
@@ -2341,6 +2473,14 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
**reply = > json.get value=`turn` key="text"**
**c = > json.get value=`turn` key="llm_calls"**
**t = > json.get value=`turn` key="tokens"**
1. not `c`
**c = 0**
2. *
**_ = 1**
1. not `t`
**t = 0**
2. *
**_ = 1**
**llm_calls = llm_calls + c**
**tokens = tokens + t**
**reply = > trim value=`reply`**
@@ -2393,6 +2533,14 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
**reply = > json.get value=`turn2` key="text"**
**c = > json.get value=`turn2` key="llm_calls"**
**t = > json.get value=`turn2` key="tokens"**
1. not `c`
**c = 0**
2. *
**_ = 1**
1. not `t`
**t = 0**
2. *
**_ = 1**
**llm_calls = llm_calls + c**
**tokens = tokens + t**
**out = table.put in=`out` at="denied_tool" value=`tool_name`**
@@ -2410,6 +2558,14 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
**reply = > json.get value=`turn2` key="text"**
**c = > json.get value=`turn2` key="llm_calls"**
**t = > json.get value=`turn2` key="tokens"**
1. not `c`
**c = 0**
2. *
**_ = 1**
1. not `t`
**t = 0**
2. *
**_ = 1**
**llm_calls = llm_calls + c**
**tokens = tokens + t**
**out = table.put in=`out` at="tool" value=`tool_name`**
@@ -2866,17 +3022,28 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
> plan_echo_decompose workbook=`path` stream=`stream` echo=`echo`
**last_obs = > inspect_workbook path=`path`**
**turn = > plan_llm_act_loop agent=`self` goal=`goal` path=`path` observation=`last_obs` phase="decompose" explore_attempt=`explore_attempt` explore_n=`explore_n` stream=`stream` echo=`echo` events=`events`**
**dec = > json.get value=`turn` key="decision"**
**last_reply = > json.get value=`turn` key="reply"**
**last_obs = > json.get value=`turn` key="observation"**
**events = > json.get value=`turn` key="events"**
**turn_s = > json.stringify value=`turn`**
**turn_map = > text.starts_with text=`turn_s` prefix="{"**
**dec = None**
**last_reply = None**
1. `turn_map`
**dec = > json.get value=`turn` key="decision"**
**last_reply = > json.get value=`turn` key="reply"**
**last_obs = > json.get value=`turn` key="observation"**
**events = > json.get value=`turn` key="events"**
2. *
**dec = "TRANSPORT_ERROR"**
**last_reply = "plan_llm_act_loop_not_map"**
**err = `last_reply`**
1. `dec` == "TRANSPORT_ERROR"
**done = 1**
**status = "error"**
**err = > json.get value=`turn` key="error"**
1. not `err`
**err = `last_reply`**
**err = `last_reply`**
1. `turn_map`
**e2 = > json.get value=`turn` key="error"**
1. `e2`
**err = `e2`**
**summary = "llm transport error"**
**skip_loop = 1**
2. `dec` == CONTINUE
+61 -9
View File
@@ -23,16 +23,27 @@ Load `.env` from cwd (optional `path=`). Does not override existing variables.
## collect
+ `events`
Collapse a stream event list to final answer text (`done.result` or `error.message`). Prefer this over raw SSE fields. Alias: `stream_result`.
Collapse a stream event list to final answer text.
Accepts semantic events (`delta` / `text` / `done.result` / `error.message`) and
native-plugin raw SSE frames (`type=data` with OpenAI chunk JSON in `data`).
Alias: `stream_result`.
**answer = ""**
- [ev](遍历 events 为 ev)
**t = > json.get value=`ev` key="type"**
1. `t` == "done"
**answer = > json.get value=`ev` key="result"**
**done_r = > json.get value=`ev` key="result"**
1. `done_r`
**answer = `done_r`**
2. *
**_ = 1**
2. `t` == "error"
**answer = > json.get value=`ev` key="message"**
**em = > json.get value=`ev` key="message"**
1. `em`
**answer = `em`**
2. *
**_ = 1**
3. `t` == "delta"
4. `t` == "text"
**chunk = > json.get value=`ev` key="text"**
@@ -40,9 +51,45 @@ Collapse a stream event list to final answer text (`done.result` or `error.messa
**answer = answer + chunk**
2. *
**_ = 1**
5. *
5. `t` == "data"
**raw = > json.get value=`ev` key="data"**
1. `raw`
1. `raw` == "[DONE]"
**_ = 1**
2. *
**parsed = > json.parse text=`raw`**
1. `parsed`
**choices = > json.get value=`parsed` key="choices"**
1. `choices`
**c0 = [1](access(choices, "1"))**
**delta = > json.get value=`c0` key="delta"**
1. `delta`
**piece = > json.get value=`delta` key="content"**
1. `piece`
**answer = answer + piece**
2. *
**_ = 1**
2. *
**msg = > json.get value=`c0` key="message"**
1. `msg`
**piece = > json.get value=`msg` key="content"**
1. `piece`
**answer = answer + piece**
2. *
**_ = 1**
2. *
**_ = 1**
2. *
**_ = 1**
2. *
**_ = 1**
2. *
**_ = 1**
6. *
**_ = 1**
1. not `answer`
**answer = ""**
*answer*
## stream_result
@@ -347,8 +394,10 @@ inside the table is unsafe.
+ `prompt`=None
+ `path`=None
+ `echo`=False
+ `stream_kind`="delta"
Semantic stream: return the event list (use `llm.collect` to reduce to text). Omit `prompt=` / `path=` to use the entry document.
`stream_kind=plan` tags EventBus frames for the parent plan protocol (process trail).
1. `path`
**p = > prompt_load path=`path`**
@@ -361,7 +410,7 @@ Semantic stream: return the event list (use `llm.collect` to reduce to text). Om
2. *
**p = > prompt_body src=`raw`**
*> self.complete prompt=`p` stream=True echo=`echo`*
*> self.complete prompt=`p` stream=True echo=`echo` stream_kind=`stream_kind`*
## complete_result
+ `prompt`=None
@@ -370,24 +419,26 @@ Semantic stream: return the event list (use `llm.collect` to reduce to text). Om
+ `strict_selection`=False
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`. Prefer `messages=` for multi-turn; optional `selection=` (ModelSelectionV1) with `strict_selection=True` for request-scoped identity (M01).
*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=`stream` echo=`echo` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))*
*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=`stream` echo=`echo` stream_kind=`stream_kind` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))*
## complete
+ `prompt`
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"
Internal / compatibility primitive. Prefer `ask` / `stream` in new code.
1. `stream`
**pack = > self.complete_result prompt=`prompt` stream=True echo=`echo`**
**pack = > self.complete_result prompt=`prompt` stream=True echo=`echo` stream_kind=`stream_kind`**
**_ret = > json.get value=`pack` key="events"**
*_ret*
2. *
**pack = > self.complete_result prompt=`prompt` stream=False echo=`echo`**
**pack = > self.complete_result prompt=`prompt` stream=False echo=`echo` stream_kind=`stream_kind`**
**_ret = > json.get value=`pack` key="text"**
*_ret*
@@ -395,7 +446,8 @@ Internal / compatibility primitive. Prefer `ask` / `stream` in new code.
+ `prompt`
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"
Alias for `complete` (compat).
*> self.complete prompt=`prompt` stream=`stream` echo=`echo`*
*> self.complete prompt=`prompt` stream=`stream` echo=`echo` stream_kind=`stream_kind`*
+3 -1
View File
@@ -80,17 +80,19 @@ Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list).
+ `auth_mode`=None
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"
+ `suffix`=/chat/completions
+ `bearer`="Bearer "
POST `/chat/completions` (or stream SSE). Prefers `plugins/llm` ABI (`llm_chat_completions`) when the native plugin is resolvable; otherwise `lib/net` HTTP/SSE. Non-stream → `{text,usage,finish,tool_calls}`; stream → `{events,finish}`. Soft transport / HTTP failures return `finish=error` packs (never abort the VM).
Pass either `messages=` (role/content list) or legacy `prompt=` (single user message). Optional `selection=` is a ModelSelectionV1 snapshot; `strict_selection=True` refuses env fallbacks inside the plugin.
`stream_kind` tags live EventBus frames: `delta` (answer tokens) or `plan` (parent protocol — UI process trail, not the bubble).
**p = > plugin.native_path name="llm"**
1. `p`
> plugin.load path=`p`
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` auth_mode=`auth_mode` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`*
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` auth_mode=`auth_mode` stream=`stream` echo=`echo` stream_kind=`stream_kind` suffix=`suffix` bearer=`bearer`*
2. *
**url = base_url + suffix**
**auth = bearer + api_key**
+50 -3
View File
@@ -20,14 +20,25 @@ description: >-
## 收集
+ `事件`
Collapse stream events to text. Supports semantic `delta`/`done.result` and
native-plugin raw frames (`type=data` + OpenAI chunk JSON).
**answer = ""**
- [ev](遍历 事件 为 ev)
**t = [type](access(ev, "type"))**
1. `t` == "done"
**answer = [result](access(ev, "result"))**
**done_r = [result](access(ev, "result"))**
1. `done_r`
**answer = `done_r`**
2. *
**_ = 1**
2. `t` == "error"
**answer = [message](access(ev, "message"))**
**em = [message](access(ev, "message"))**
1. `em`
**answer = `em`**
2. *
**_ = 1**
3. `t` == "delta"
4. `t` == "text"
**chunk = [text](access(ev, "text"))**
@@ -35,9 +46,45 @@ description: >-
**answer = answer + chunk**
2. *
**_ = 1**
5. *
5. `t` == "data"
**raw = [data](access(ev, "data"))**
1. `raw`
1. `raw` == "[DONE]"
**_ = 1**
2. *
**parsed = > json.parse text=`raw`**
1. `parsed`
**choices = [choices](access(parsed, "choices"))**
1. `choices`
**c0 = [1](access(choices, "1"))**
**delta = [delta](access(c0, "delta"))**
1. `delta`
**piece = [content](access(delta, "content"))**
1. `piece`
**answer = answer + piece**
2. *
**_ = 1**
2. *
**msg = [message](access(c0, "message"))**
1. `msg`
**piece = [content](access(msg, "content"))**
1. `piece`
**answer = answer + piece**
2. *
**_ = 1**
2. *
**_ = 1**
2. *
**_ = 1**
2. *
**_ = 1**
2. *
**_ = 1**
6. *
**_ = 1**
1. not `answer`
**answer = ""**
*answer*
## 流式结果
+13
View File
@@ -86,6 +86,19 @@ Like `select` but returns `{ rows, total }` — the total counts rows matching `
**url = [url](access(self, "url"))**
*> web_db_update url=`url` table=`table` id=`id` row=`row` txn=`txn`*
## update_if
+ `table`
+ `id`
+ `row`
+ `expect_col`
+ `expect_val`
+ `txn`=None
Conditional update (CAS) for task/revision control: `UPDATE … WHERE id=? AND expect_col=expect_val`. Returns `{ ok, updated, matched }`; `updated=0` means conflict — do not simulate atomicity with read-then-write in `.mq.md`.
**url = [url](access(self, "url"))**
*> web_db_update_if url=`url` table=`table` id=`id` row=`row` expect_col=`expect_col` expect_val=`expect_val` txn=`txn`*
## delete
+ `table`
+ `id`
+7 -4
View File
@@ -16,15 +16,17 @@ import:
+ `request`="json"
+ `response`="json"
+ `auth`=""
+ `class`=""
Declared route handle. Prefer `web.route` / Artifact `type: endpoint` discovery via `web.serve`.
Optional `class`: `execute` | `short` | `read` | `control` (empty → heuristic / `legacy_serial`).
> cap.load
`out` =
| _type | app | path | method | page | fn | request | response | auth |
|-------|-----|------|--------|------|----|---------|----------|------|
| route | `app` | `path` | `method` | `page` | `fn` | `request` | `response` | `auth` |
| _type | app | path | method | page | fn | request | response | auth | class |
|-------|-----|------|--------|------|----|---------|----------|------|-------|
| route | `app` | `path` | `method` | `page` | `fn` | `request` | `response` | `auth` | `class` |
*out*
@@ -57,6 +59,7 @@ Register this declared route on an app handle.
**method = [method](access(self, "method"))**
**request = [request](access(self, "request"))**
**response = [response](access(self, "response"))**
*> web_app_invoke app=`a` path=`path` method=`method` fn=`fn` body=`request` return=`response`*
**class = [class](access(self, "class"))**
*> web_app_invoke app=`a` path=`path` method=`method` fn=`fn` body=`request` return=`response` class=`class`*
3. *
*> table.put in=None at="ok" value=False*
+3 -1
View File
@@ -38,10 +38,12 @@ Construct a Document handle. Prefer `type: web` Artifact files discovered by `se
+ `request`="json"
+ `response`="json"
+ `auth`=""
+ `class`=""
Register a declared Document page or Endpoint (`fn` = `lib.member`).
Optional `class`: `execute` | `short` | `read` | `control`.
**r = > route_mod.route app=`app` path=`path` method=`method` page=`page` fn=`fn` request=`request` response=`response` auth=`auth`**
**r = > route_mod.route app=`app` path=`path` method=`method` page=`page` fn=`fn` request=`request` response=`response` auth=`auth` class=`class`**
*> `r`.register app=`app`*
## render
+116 -15
View File
@@ -92,28 +92,95 @@ fn tool_calls_from(data: &Value) -> Value {
}
fn cancel_requested() -> bool {
if std::env::var("MARQDO_CANCEL")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false)
{
return true;
}
// Prefer request-scoped token handle. Empty / "0" ≠ cancel (M-R1).
// When a token path is set, ignore process-global MARQDO_CANCEL so parallel
// tasks cannot cancel each other via shared env.
if let Ok(tok) = std::env::var("MARQDO_CANCEL_TOKEN") {
let path = std::path::Path::new(&tok);
if path.is_file() {
if let Ok(body) = std::fs::read_to_string(path) {
let t = body.trim();
return t.is_empty() || t == "1" || t.eq_ignore_ascii_case("cancelled");
if !tok.is_empty() {
let path = std::path::Path::new(&tok);
if path.is_file() {
if let Ok(body) = std::fs::read_to_string(path) {
let t = body.trim();
return t == "1" || t.eq_ignore_ascii_case("cancelled");
}
}
return true;
return false;
}
}
false
// Legacy single-task CLI only when no token handle is bound.
std::env::var("MARQDO_CANCEL")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false)
}
fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, String> {
/// Mirror host `net::push_openai_delta_events`: append semantic frames to
/// `MARQDO_EVENT_FORWARD` so subtask/web sse-live can flush tokens before the
/// lib call returns (OpenAI-style incremental SSE).
fn forward_event_bus_frame(ev: &Value) {
let Ok(path) = std::env::var("MARQDO_EVENT_FORWARD") else {
return;
};
if path.is_empty() {
return;
}
use std::io::Write;
if let Ok(mut f) = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(&path)
{
let _ = writeln!(f, "{ev}");
let _ = f.flush();
}
}
/// Publish `{type:delta|plan|reasoning|error, text|message}` from one OpenAI chunk.
/// `content_type` is usually `delta` (answer tokens) or `plan` (parent protocol LLM).
fn forward_openai_chunk(data: &str, content_type: &str) {
let data = data.trim();
if data.is_empty() || data == "[DONE]" {
return;
}
let Ok(v) = serde_json::from_str::<Value>(data) else {
return;
};
if let Some(msg) = v.get("error").and_then(|e| {
e.get("message")
.and_then(|m| m.as_str())
.or_else(|| e.as_str())
}) {
forward_event_bus_frame(&json!({"type": "error", "message": msg}));
return;
}
let reasoning = v
.pointer("/choices/0/delta/reasoning_content")
.and_then(|c| c.as_str())
.unwrap_or("");
if !reasoning.is_empty() {
forward_event_bus_frame(&json!({"type": "reasoning", "text": reasoning}));
}
let content = v
.pointer("/choices/0/delta/content")
.and_then(|c| c.as_str())
.unwrap_or("");
if !content.is_empty() {
let ty = if content_type.is_empty() {
"delta"
} else {
content_type
};
forward_event_bus_frame(&json!({"type": ty, "text": content}));
}
}
fn parse_sse_events(
reader: impl BufRead,
echo: bool,
content_type: &str,
) -> Result<Vec<Value>, String> {
let mut events = Vec::new();
let mut data_lines: Vec<String> = Vec::new();
let mut accumulated = String::new();
for line in reader.lines() {
if cancel_requested() {
events.push(json!({"type": "cancelled", "message": "MARQDO_CANCEL"}));
@@ -128,9 +195,23 @@ fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, Stri
let data = data_lines.join("\n");
data_lines.clear();
if data.trim() == "[DONE]" {
// Parent owns run lifecycle `done`; only forward for answer streams.
if content_type == "delta" || content_type.is_empty() {
let done = json!({"type": "done", "result": accumulated});
forward_event_bus_frame(&done);
}
events.push(json!({"type": "done", "data": "[DONE]"}));
break;
}
forward_openai_chunk(&data, content_type);
if let Ok(v) = serde_json::from_str::<Value>(&data) {
if let Some(c) = v
.pointer("/choices/0/delta/content")
.and_then(|x| x.as_str())
{
accumulated.push_str(c);
}
}
events.push(json!({"type": "data", "data": data}));
}
continue;
@@ -142,8 +223,13 @@ fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, Stri
if !data_lines.is_empty() {
let data = data_lines.join("\n");
if data.trim() == "[DONE]" {
if content_type == "delta" || content_type.is_empty() {
let done = json!({"type": "done", "result": accumulated});
forward_event_bus_frame(&done);
}
events.push(json!({"type": "done", "data": "[DONE]"}));
} else {
forward_openai_chunk(&data, content_type);
events.push(json!({"type": "data", "data": data}));
}
}
@@ -252,6 +338,8 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
let messages = build_messages(args)?;
let stream = opt_bool(args, "stream", false);
let echo = opt_bool(args, "echo", false);
// Answer tokens → "delta"; parent plan protocol → "plan" (UI process trail).
let stream_kind = opt_text(args, "stream_kind").unwrap_or("delta");
let suffix = opt_text(args, "suffix").unwrap_or("/chat/completions");
let bearer = opt_text(args, "bearer").unwrap_or("Bearer ");
@@ -309,7 +397,8 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
"transport": "plugin",
}));
}
let events = parse_sse_events(BufReader::new(r.into_reader()), echo)?;
let events =
parse_sse_events(BufReader::new(r.into_reader()), echo, stream_kind)?;
Ok(json!({
"events": events,
"finish": "stop",
@@ -525,6 +614,18 @@ mod tests {
assert_eq!(out["finish"], "cancelled");
}
#[test]
fn f06_empty_cancel_token_is_not_cancelled() {
// sse-live creates an empty token file; that must NOT abort the turn.
let tok = std::env::temp_dir().join(format!("llm-cancel-empty-{}", std::process::id()));
std::fs::write(&tok, "").unwrap();
std::env::set_var("MARQDO_CANCEL_TOKEN", &tok);
std::env::remove_var("MARQDO_CANCEL");
assert!(!cancel_requested());
std::env::remove_var("MARQDO_CANCEL_TOKEN");
let _ = std::fs::remove_file(&tok);
}
/// F01: two local endpoints receive distinct models/auth — no cross-talk.
#[test]
fn f01_dual_endpoint_isolation() {
+23 -2
View File
@@ -63,6 +63,11 @@ func ParseFrontmatter(src string) (meta map[string]string, body string) {
body = body[i+3:]
body = strings.TrimPrefix(body, "\r")
body = strings.TrimPrefix(body, "\n")
// Drop blank line(s) between closing --- and first content.
for strings.HasPrefix(body, "\n") || strings.HasPrefix(body, "\r\n") {
body = strings.TrimPrefix(body, "\r\n")
body = strings.TrimPrefix(body, "\n")
}
}
for _, line := range strings.Split(fm, "\n") {
line = strings.TrimSpace(strings.TrimSuffix(line, "\r"))
@@ -254,6 +259,10 @@ func SystemRoutes() []map[string]any {
return []map[string]any{
{"method": "GET", "path": "/_part/*", "kind": "system"},
{"method": "GET|POST", "path": "/_form/*", "kind": "system"},
{"method": "POST", "path": "/_runtime/tasks", "kind": "system"},
{"method": "GET", "path": "/_runtime/tasks/{id}", "kind": "system"},
{"method": "POST", "path": "/_runtime/tasks/{id}/cancel", "kind": "system"},
{"method": "GET", "path": "/_runtime/tasks/{id}/events", "kind": "system"},
}
}
@@ -275,8 +284,20 @@ func InspectBag(app map[string]any, declared []ArtifactFile) map[string]any {
}
}
if inv, ok := app["invoke_routes"].(map[string]any); ok {
for key := range inv {
dec = append(dec, map[string]any{"path": key, "kind": "endpoint"})
for key, v := range inv {
row := map[string]any{"path": key, "kind": "endpoint"}
if m, ok := v.(map[string]any); ok {
if c, ok := m["class"].(string); ok && c != "" {
row["class"] = c
}
if method, ok := m["method"].(string); ok && method != "" {
row["method"] = method
}
if fn, ok := m["fn"].(string); ok && fn != "" {
row["fn"] = fn
}
}
dec = append(dec, row)
}
}
if art, ok := app["artifact_routes"].(map[string]any); ok {
+48
View File
@@ -968,6 +968,54 @@ func Update(url, tableName, id string, row any, txnID ...string) (map[string]any
return map[string]any{"ok": true, "updated": float64(n)}, nil
}
// UpdateIf is a conditional update (CAS): SET row WHERE id=? AND expect_col=expect_val.
// Returns {"ok":true,"updated":N,"matched":N}. Callers treat updated==0 as conflict.
// Optional txnID routes through an open Begin transaction.
func UpdateIf(url, tableName, id string, row any, expectCol string, expectVal any, txnID ...string) (map[string]any, error) {
tn, err := ident(tableName)
if err != nil {
return nil, err
}
ecol, err := ident(expectCol)
if err != nil {
return nil, err
}
obj, ok := row.(map[string]any)
if !ok {
return nil, fmt.Errorf("row must be a map")
}
c, err := connFor(url, optTxn(txnID))
if err != nil {
return nil, err
}
var sets []string
var vals []any
for k, v := range obj {
if k == "id" {
continue
}
if _, err := ident(k); err != nil {
return nil, err
}
sets = append(sets, fmt.Sprintf(`"%s" = ?`, k))
vals = append(vals, toSQLArg(v))
}
if len(sets) == 0 {
return nil, fmt.Errorf("nothing to update")
}
vals = append(vals, id, toSQLArg(expectVal))
sqlStmt := fmt.Sprintf(
`UPDATE "%s" SET %s WHERE "id" = ? AND "%s" = ?`,
tn, strings.Join(sets, ", "), ecol,
)
res, err := c.Exec(sqlStmt, vals...)
if err != nil {
return nil, err
}
n, _ := res.RowsAffected()
return map[string]any{"ok": true, "updated": float64(n), "matched": float64(n)}, nil
}
// Delete removes a row by id. Returns {"ok":true,"deleted":N}.
// Optional txnID routes through an open Begin transaction.
func Delete(url, tableName, id string, txnID ...string) (map[string]any, error) {
+18
View File
@@ -100,6 +100,24 @@ func TestInitInsertSelectGet(t *testing.T) {
}
}
func TestUpdateIfCAS(t *testing.T) {
url := setupArticles(t)
ok, err := db.UpdateIf(url, "articles", "1", map[string]any{"title": "CAS"}, "title", "Alpha")
if err != nil {
t.Fatal(err)
}
if ok["updated"] != float64(1) {
t.Fatalf("cas hit=%v", ok)
}
miss, err := db.UpdateIf(url, "articles", "1", map[string]any{"title": "Nope"}, "title", "Alpha")
if err != nil {
t.Fatal(err)
}
if miss["updated"] != float64(0) {
t.Fatalf("cas miss should be 0, got %v", miss)
}
}
func TestUpdateDelete(t *testing.T) {
url := setupArticles(t)
+29
View File
@@ -198,6 +198,35 @@ func invokeArgsFromPayload(payload map[string]any) map[string]any {
return args
}
// payloadWantsStream detects OpenAI-compatible stream:true on the request body.
func payloadWantsStream(payload map[string]any) bool {
if payload == nil {
return false
}
if v, ok := payload["stream"]; ok {
return boolishAny(v)
}
return false
}
func boolishAny(v any) bool {
switch t := v.(type) {
case bool:
return t
case string:
s := strings.ToLower(strings.TrimSpace(t))
return s == "1" || s == "true" || s == "yes" || s == "on"
case float64:
return t != 0
case int:
return t != 0
case int64:
return t != 0
default:
return false
}
}
func readJSONBody(r *http.Request, max int64) (map[string]any, error) {
r.Body = http.MaxBytesReader(nil, r.Body, max)
defer r.Body.Close()
+16
View File
@@ -239,6 +239,7 @@ func NewHandler(appBag map[string]any, entryDir string) (http.Handler, error) {
st.mountGalleryRoutes(mux)
st.mountProxyRoutes(mux)
st.mountInvokeRoutes(mux)
st.mountRuntimeRoutes(mux)
st.mountArtifactRoutes(mux)
st.mountAuthRoutes(mux)
@@ -799,6 +800,13 @@ func (g *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
return nil, nil, fmt.Errorf("underlying ResponseWriter does not support hijacking")
}
func (g *gzipResponseWriter) Flush() {
_ = g.w.Flush()
if f, ok := g.ResponseWriter.(http.Flusher); ok {
f.Flush()
}
}
type statusRecorder struct {
http.ResponseWriter
status int
@@ -816,6 +824,14 @@ func (s *statusRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
return nil, nil, fmt.Errorf("underlying ResponseWriter does not support hijacking")
}
// Flush must be forwarded so sse-live / EventBus token streams are not buffered
// behind access-log wrapping (otherwise clients see 0 bytes until the invoke ends).
func (s *statusRecorder) Flush() {
if f, ok := s.ResponseWriter.(http.Flusher); ok {
f.Flush()
}
}
func writeJSON(w http.ResponseWriter, code int, v any) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(code)
+439 -105
View File
@@ -1,24 +1,43 @@
package httpx
import (
"bufio"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/marqdo/marqdo/plugins/web/internal/plugin"
)
// legacySSEBridgeMu serializes the process-global MARQDO_EVENT_FORWARD /
// MARQDO_CANCEL_TOKEN bridge for the legacy_serial path only. Isolated execute
// never acquires this lock (ADR 0009).
var legacySSEBridgeMu sync.Mutex
// legacySSEBridgeActive is 1 while a holder has mutated process env for the bridge.
var legacySSEBridgeActive atomic.Int32
func tryAcquireLegacySSEBridge() bool {
return legacySSEBridgeMu.TryLock()
}
func releaseLegacySSEBridge() {
legacySSEBridgeActive.Store(0)
legacySSEBridgeMu.Unlock()
}
type invokeRoute struct {
Method string
FnPath string
BodyMode string
ReturnMode string
Class string
}
func parseInvokeRoute(v any) invokeRoute {
@@ -30,11 +49,16 @@ func parseInvokeRoute(v any) invokeRoute {
if fn == "" {
fn = strOpt(m, "function", "")
}
class := strings.ToLower(strOpt(m, "class", ""))
if class == "" {
class = strings.ToLower(strOpt(m, "类别", ""))
}
return invokeRoute{
Method: method,
FnPath: fn,
BodyMode: body,
ReturnMode: ret,
Class: class,
}
}
@@ -106,7 +130,6 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
}
}
// Bind common auth / client headers for OpenAI-compatible and API-key surfaces.
if auth := strings.TrimSpace(r.Header.Get("Authorization")); auth != "" {
if _, exists := payload["authorization"]; !exists {
payload["authorization"] = auth
@@ -120,18 +143,50 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
args := invokeArgsFromPayload(payload)
// M02: live SSE — stream host_stream_publish events while CallLibPath runs.
// return=sse-live|sse_live uses MARQDO_EVENT_FORWARD NDJSON bridge (request-local).
liveSSE := route.ReturnMode == "sse-live" || route.ReturnMode == "sse_live"
if liveSSE {
st.handleInvokeLiveSSE(w, r, route, args)
return
if !liveSSE && payloadWantsStream(payload) {
liveSSE = true
}
class := resolveInvokeClass(route, liveSSE)
switch class {
case "execute":
st.handleInvokeIsolated(w, r, route, args, liveSSE)
return
case "short", "read", "control":
st.handleInvokeParent(w, r, route, args)
return
default:
// legacy_serial: parent CallLibPath (ADR 0008). Never isolate undeclared non-stream.
st.handleInvokeParent(w, r, route, args)
}
}
func resolveInvokeClass(route invokeRoute, liveSSE bool) string {
c := strings.ToLower(strings.TrimSpace(route.Class))
switch c {
case "execute", "short", "read", "control":
return c
case "legacy", "legacy_serial":
return "legacy_serial"
}
if liveSSE {
return "execute"
}
return "legacy_serial"
}
func (st *state) handleInvokeParent(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any) {
_ = r
value, err := plugin.CallLibPath(route.FnPath, args)
if err != nil {
code := http.StatusInternalServerError
msg := err.Error()
if strings.Contains(msg, "interpreter busy") {
w.Header().Set("Retry-After", "2")
jsonErr(w, http.StatusServiceUnavailable, msg)
return
}
if strings.Contains(msg, "unknown library") ||
strings.Contains(msg, "call_lib_path: need") ||
strings.Contains(msg, "not allowed") ||
@@ -146,8 +201,6 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
return
}
// OpenAI-compatible handlers may return `{sse: "..."}` to force event-stream
// even when the route's declared return mode is json.
if m, ok := value.(map[string]any); ok {
if s, ok := m["sse"].(string); ok && strings.TrimSpace(s) != "" {
writeSSE(w, m)
@@ -190,41 +243,88 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
}
}
// handleInvokeLiveSSE streams RuntimeEvent-shaped lines while the lib call runs.
// Client disconnect sets MARQDO_CANCEL + cancel token file so the llm plugin can stop.
func (st *state) handleInvokeLiveSSE(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any) {
dir := os.TempDir()
fwd, err := os.CreateTemp(dir, "mq-evt-*.ndjson")
// handleInvokeIsolated runs execute-class invokes on a Supervisor worker (no
// parent CallLibPath, no process Setenv, no legacySSEBridgeMu).
func (st *state) handleInvokeIsolated(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any, liveSSE bool) {
taskID := fmt.Sprintf("exec_%d", time.Now().UnixNano())
adm, err := runtimeQuery("admit", map[string]any{
"task_id": taskID,
"idempotency_key": taskID,
"class": "execute",
"payload": map[string]any{"fn": route.FnPath},
})
if err != nil {
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("event forward: %v", err))
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("admit: %v", err))
return
}
fwdPath := fwd.Name()
_ = fwd.Close()
defer os.Remove(fwdPath)
prev := os.Getenv("MARQDO_EVENT_FORWARD")
_ = os.Setenv("MARQDO_EVENT_FORWARD", fwdPath)
cancelFile, cancelErr := os.CreateTemp(dir, "mq-cancel-*.tok")
cancelPath := ""
if cancelErr == nil {
cancelPath = cancelFile.Name()
_ = cancelFile.Close()
_ = os.Setenv("MARQDO_CANCEL_TOKEN", cancelPath)
if ok, _ := adm["ok"].(bool); !ok {
status := http.StatusServiceUnavailable
if s, ok := adm["status"].(float64); ok && s >= 400 {
status = int(s)
}
w.Header().Set("Retry-After", "2")
jsonErr(w, status, fmt.Sprint(adm["error"]))
return
}
defer func() {
if prev == "" {
_ = os.Unsetenv("MARQDO_EVENT_FORWARD")
} else {
_ = os.Setenv("MARQDO_EVENT_FORWARD", prev)
}
_ = os.Unsetenv("MARQDO_CANCEL_TOKEN")
_ = os.Unsetenv("MARQDO_CANCEL")
if cancelPath != "" {
_ = os.Remove(cancelPath)
}
}()
cancelTok, _ := adm["cancel_token"].(string)
eventPath, _ := adm["event_path"].(string)
entryModule, fsRoot, err := resolveSiteEntry(st)
if err != nil {
jsonErr(w, http.StatusInternalServerError, err.Error())
return
}
argsFile, err := materializeArgsJSON(args)
if err != nil {
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("args: %v", err))
return
}
defer os.Remove(argsFile)
disp, err := runtimeQuery("dispatch", map[string]any{
"task_id": taskID,
"entry": map[string]any{
"module": entryModule,
"function": route.FnPath,
},
"fs_root": fsRoot,
"args_json": argsFile,
})
if err != nil {
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("dispatch: %v", err))
return
}
if ok, _ := disp["ok"].(bool); !ok {
status := http.StatusTooManyRequests
if s, ok := disp["status"].(float64); ok && s >= 400 {
status = int(s)
}
if ra, ok := disp["Retry-After"]; ok {
w.Header().Set("Retry-After", fmt.Sprint(ra))
} else {
w.Header().Set("Retry-After", "2")
}
jsonErr(w, status, fmt.Sprint(disp["error"]))
return
}
if ep, ok := disp["event_path"].(string); ok && ep != "" {
eventPath = ep
}
if tok, ok := disp["cancel_token"].(string); ok && tok != "" {
cancelTok = tok
}
emitResult, _ := disp["emit_result"].(string)
if liveSSE {
st.streamIsolatedSSE(w, r, route, taskID, eventPath, cancelTok, emitResult)
return
}
st.waitIsolatedJSON(w, r, route, taskID, eventPath, cancelTok, emitResult)
}
func (st *state) streamIsolatedSSE(w http.ResponseWriter, r *http.Request, route invokeRoute, taskID, eventPath, cancelTok, emitResult string) {
w.Header().Set("Content-Type", "text/event-stream; charset=utf-8")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
@@ -245,104 +345,338 @@ func (st *state) handleInvokeLiveSSE(w http.ResponseWriter, r *http.Request, rou
"schema": "RuntimeEventV1",
"schema_version": 1,
"type": "run.started",
"payload": map[string]any{"fn": route.FnPath, "bridge": "sse-live"},
"payload": map[string]any{
"fn": route.FnPath,
"bridge": "isolated",
"task_id": taskID,
},
})
type callResult struct {
value any
err error
}
done := make(chan callResult, 1)
go func() {
v, e := plugin.CallLibPath(route.FnPath, args)
done <- callResult{value: v, err: e}
}()
var mu sync.Mutex
offset := int64(0)
carry := ""
seq := uint64(1)
poll := time.NewTicker(25 * time.Millisecond)
defer poll.Stop()
ctx := r.Context()
for {
select {
case <-ctx.Done():
// Propagate cancel to in-flight LLM/plugin via token file + env.
_ = os.Setenv("MARQDO_CANCEL", "1")
if cancelPath != "" {
_ = os.WriteFile(cancelPath, []byte("cancelled"), 0o600)
finish := func(clientGone bool) {
offset, carry = drainForwardCarry(eventPath, offset, carry, func(line string) {
seq++
writeEvent(normalizeLiveSSELine(seq, line))
})
if !clientGone {
result := readEmitResult(emitResult)
if result == nil {
if g, err := runtimeQuery("get", map[string]any{"task_id": taskID}); err == nil {
if t, ok := g["task"].(map[string]any); ok {
result = t["result"]
}
}
}
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.interrupted",
"payload": map[string]any{"reason": "client_disconnect", "cancel_propagated": cancelPath != ""},
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.completed",
"payload": map[string]any{"result": result, "task_id": taskID},
})
return
case res := <-done:
// Drain remaining forward lines once.
mu.Lock()
offset = drainForward(fwdPath, offset, func(line string) {
seq++
var obj any
if json.Unmarshal([]byte(line), &obj) == nil {
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1,
"seq": seq, "type": "message.delta", "payload": obj,
})
}
})
mu.Unlock()
if res.err != nil {
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.failed",
"payload": map[string]any{"error": res.err.Error()},
})
} else {
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.completed",
"payload": map[string]any{"result": res.value},
})
}
_, _ = io.WriteString(w, "data: [DONE]\n\n")
if flusher != nil {
flusher.Flush()
}
}
}
for {
select {
case <-ctx.Done():
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
_ = writeCancelToken(cancelTok)
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.interrupted",
"payload": map[string]any{
"reason": "client_disconnect",
"cancel_propagated": cancelTok != "",
"status": "stopping",
"task_id": taskID,
},
})
// Brief wait so worker can notice cancel and flush.
deadline := time.Now().Add(3 * time.Second)
for time.Now().Before(deadline) {
if workerDone(taskID, emitResult) {
break
}
time.Sleep(50 * time.Millisecond)
}
finish(true)
return
case <-poll.C:
mu.Lock()
offset = drainForward(fwdPath, offset, func(line string) {
offset, carry = drainForwardCarry(eventPath, offset, carry, func(line string) {
seq++
var obj any
if json.Unmarshal([]byte(line), &obj) == nil {
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1,
"seq": seq, "type": "message.delta", "payload": obj,
})
}
writeEvent(normalizeLiveSSELine(seq, line))
})
mu.Unlock()
if workerDone(taskID, emitResult) {
finish(false)
return
}
}
}
}
func (st *state) waitIsolatedJSON(w http.ResponseWriter, r *http.Request, route invokeRoute, taskID, eventPath, cancelTok, emitResult string) {
_ = eventPath
poll := time.NewTicker(25 * time.Millisecond)
defer poll.Stop()
ctx := r.Context()
for {
select {
case <-ctx.Done():
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
_ = writeCancelToken(cancelTok)
jsonErr(w, http.StatusRequestTimeout, "client disconnected")
return
case <-poll.C:
if !workerDone(taskID, emitResult) {
continue
}
value := readEmitResult(emitResult)
if value == nil {
jsonErr(w, http.StatusInternalServerError, "worker produced no result")
return
}
if m, ok := value.(map[string]any); ok {
if s, ok := m["sse"].(string); ok && strings.TrimSpace(s) != "" {
writeSSE(w, m)
return
}
}
switch route.ReturnMode {
case "text", "html", "text/html":
text := fmt.Sprint(value)
if s, ok := value.(string); ok {
text = s
}
ct := "text/plain; charset=utf-8"
if route.ReturnMode == "html" || route.ReturnMode == "text/html" {
ct = "text/html; charset=utf-8"
}
w.Header().Set("Content-Type", ct)
w.WriteHeader(http.StatusOK)
_, _ = io.WriteString(w, text)
case "status":
w.WriteHeader(http.StatusNoContent)
default:
code, body, err := jsonWrapInvokeValue(value)
if err != nil {
jsonErr(w, code, err.Error())
return
}
writeJSON(w, code, body)
}
return
}
}
}
func workerDone(taskID, emitResult string) bool {
if emitResult != "" {
if st, err := os.Stat(emitResult); err == nil && st.Size() > 0 {
return true
}
}
// Sibling of event file written by Supervisor even if emit_result was omitted.
if taskID != "" {
if dir := os.Getenv("MARQDO_RUNTIME_DIR"); dir != "" {
p := filepath.Join(dir, "events", taskID+".result.json")
if st, err := os.Stat(p); err == nil && st.Size() > 0 {
return true
}
}
}
g, err := runtimeQuery("get", map[string]any{"task_id": taskID})
if err != nil {
return false
}
if _, ok := g["exit_code"]; ok {
return true
}
if t, ok := g["task"].(map[string]any); ok {
switch fmt.Sprint(t["state"]) {
case "succeeded", "failed", "cancelled", "interrupted":
return true
}
}
return false
}
func readEmitResult(path string) any {
if path == "" {
return nil
}
b, err := os.ReadFile(path)
if err != nil || len(b) == 0 {
return nil
}
var v any
if err := json.Unmarshal(b, &v); err != nil {
return string(b)
}
return v
}
func materializeArgsJSON(args map[string]any) (string, error) {
f, err := os.CreateTemp("", "mq-args-*.json")
if err != nil {
return "", err
}
path := f.Name()
enc := json.NewEncoder(f)
if err := enc.Encode(args); err != nil {
_ = f.Close()
_ = os.Remove(path)
return "", err
}
if err := f.Close(); err != nil {
_ = os.Remove(path)
return "", err
}
return path, nil
}
func resolveSiteEntry(st *state) (module string, fsRoot string, err error) {
fsRoot = st.entryDir
if raw, qerr := plugin.Query("entry_dir", "{}"); qerr == nil {
var s string
if json.Unmarshal([]byte(raw), &s) == nil && s != "" {
fsRoot = s
}
}
if raw, qerr := plugin.Query("entry_path", "{}"); qerr == nil {
var s string
if json.Unmarshal([]byte(raw), &s) == nil && s != "" {
module = s
}
}
if module == "" {
// Fallback: common serve entry names under fs_root.
for _, name := range []string{"serve.mq.md", "index.mq.md", "main.mq.md"} {
p := filepath.Join(fsRoot, name)
if st, e := os.Stat(p); e == nil && !st.IsDir() {
module = p
break
}
}
}
if module == "" {
return "", "", fmt.Errorf("isolated execute: cannot resolve site entry module")
}
if !filepath.IsAbs(module) {
abs, aerr := filepath.Abs(module)
if aerr == nil {
module = abs
}
}
if fsRoot == "" {
fsRoot = filepath.Dir(module)
}
if !filepath.IsAbs(fsRoot) {
abs, aerr := filepath.Abs(fsRoot)
if aerr == nil {
fsRoot = abs
}
}
return module, fsRoot, nil
}
// normalizeLiveSSELine maps Marqdo EventBus frames onto RuntimeEventV1.
func normalizeLiveSSELine(seq uint64, line string) map[string]any {
base := map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1, "seq": seq,
}
var obj map[string]any
if err := json.Unmarshal([]byte(line), &obj); err != nil || obj == nil {
base["type"] = "message.delta"
base["payload"] = map[string]any{"raw": line}
return base
}
if t, _ := obj["type"].(string); t == "message.delta" || t == "turn.started" || t == "model.resolved" || t == "usage.updated" || t == "turn.completed" || t == "turn.failed" {
for k, v := range obj {
base[k] = v
}
base["seq"] = seq
return base
}
if t, _ := obj["type"].(string); t == "delta" || t == "plan" || t == "reasoning" {
text, _ := obj["text"].(string)
base["type"] = "message.delta"
base["payload"] = map[string]any{"delta": text, "kind": t}
return base
}
if t, _ := obj["type"].(string); t == "decompose" || t == "decision" || t == "await" || t == "round" || t == "done" || t == "tool_start" || t == "tool_end" || t == "plan_act" {
base["type"] = "message.delta"
base["payload"] = obj
return base
}
if text, ok := obj["text"].(string); ok && text != "" {
base["type"] = "message.delta"
base["payload"] = map[string]any{"delta": text}
return base
}
base["type"] = "message.delta"
base["payload"] = obj
return base
}
func drainForward(path string, offset int64, onLine func(string)) int64 {
off, _ := drainForwardCarry(path, offset, "", onLine)
return off
}
const drainForwardMaxLine = 1024 * 1024
func drainForwardCarry(path string, offset int64, carry string, onLine func(string)) (int64, string) {
f, err := os.Open(path)
if err != nil {
return offset
return offset, carry
}
defer f.Close()
if _, err := f.Seek(offset, io.SeekStart); err != nil {
return offset
return offset, carry
}
sc := bufio.NewScanner(f)
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
offset += int64(len(sc.Bytes()) + 1)
chunk, err := io.ReadAll(io.LimitReader(f, drainForwardMaxLine+1))
if err != nil && err != io.EOF {
onLine(fmt.Sprintf(`{"type":"protocol.error","error":%q}`, err.Error()))
return offset, carry
}
if len(chunk) == 0 {
return offset, carry
}
offset += int64(len(chunk))
buf := carry + string(chunk)
for {
i := strings.IndexByte(buf, '\n')
if i < 0 {
if len(buf) > drainForwardMaxLine {
onLine(`{"type":"protocol.error","error":"ndjson line exceeds 1MiB"}`)
return offset, ""
}
return offset, buf
}
lineBytes := buf[:i]
line := strings.TrimSpace(strings.TrimSuffix(lineBytes, "\r"))
buf = buf[i+1:]
if line == "" {
continue
}
onLine(line)
}
return offset
}
func legacySSEBridgeSnapshot() (active bool, forward string, cancelTok string) {
return legacySSEBridgeActive.Load() != 0, os.Getenv("MARQDO_EVENT_FORWARD"), os.Getenv("MARQDO_CANCEL_TOKEN")
}
func writeCancelToken(path string) error {
if path == "" {
return nil
}
return os.WriteFile(path, []byte("cancelled"), 0o600)
}
@@ -0,0 +1,207 @@
package httpx
import (
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"
"github.com/marqdo/marqdo/plugins/web/internal/plugin"
)
// System runtime control routes (M-R3 / M-R4). Cancel and event replay do not
// enter the Marqdo interpreter lock.
func (st *state) mountRuntimeRoutes(mux *http.ServeMux) {
mux.HandleFunc("POST /_runtime/tasks", st.handleRuntimeAdmit)
mux.HandleFunc("GET /_runtime/tasks/{id}", st.handleRuntimeGet)
mux.HandleFunc("POST /_runtime/tasks/{id}/cancel", st.handleRuntimeCancel)
mux.HandleFunc("GET /_runtime/tasks/{id}/events", st.handleRuntimeEvents)
}
func runtimeQuery(op string, extra map[string]any) (map[string]any, error) {
payload := map[string]any{"op": op}
for k, v := range extra {
payload[k] = v
}
raw, err := json.Marshal(payload)
if err != nil {
return nil, err
}
out, err := plugin.Query("runtime", string(raw))
if err != nil {
return nil, err
}
var m map[string]any
if err := json.Unmarshal([]byte(out), &m); err != nil {
return nil, err
}
return m, nil
}
func (st *state) handleRuntimeAdmit(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 1<<20))
if err != nil {
jsonErr(w, http.StatusBadRequest, err.Error())
return
}
var req map[string]any
if len(body) > 0 {
if err := json.Unmarshal(body, &req); err != nil {
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("json: %v", err))
return
}
}
if req == nil {
req = map[string]any{}
}
taskID, _ := req["task_id"].(string)
if taskID == "" {
taskID = fmt.Sprintf("task_%d", time.Now().UnixNano())
req["task_id"] = taskID
}
if _, ok := req["idempotency_key"]; !ok {
req["idempotency_key"] = taskID
}
if _, ok := req["class"]; !ok {
req["class"] = "execute"
}
m, err := runtimeQuery("admit", req)
if err != nil {
if strings.Contains(err.Error(), "conflict") || strings.Contains(fmt.Sprint(m["error"]), "conflict") {
jsonErr(w, http.StatusConflict, err.Error())
return
}
jsonErr(w, http.StatusInternalServerError, err.Error())
return
}
if ok, _ := m["ok"].(bool); !ok {
jsonErr(w, http.StatusConflict, fmt.Sprint(m["error"]))
return
}
w.Header().Set("Location", "/_runtime/tasks/"+taskID)
writeJSON(w, http.StatusAccepted, m)
}
func (st *state) handleRuntimeGet(w http.ResponseWriter, r *http.Request) {
id := r.PathValue("id")
m, err := runtimeQuery("get", map[string]any{"task_id": id})
if err != nil {
jsonErr(w, http.StatusInternalServerError, err.Error())
return
}
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
jsonErr(w, http.StatusNotFound, "task not found")
return
}
writeJSON(w, http.StatusOK, m)
}
func (st *state) handleRuntimeCancel(w http.ResponseWriter, r *http.Request) {
id := r.PathValue("id")
m, err := runtimeQuery("cancel", map[string]any{"task_id": id})
if err != nil {
jsonErr(w, http.StatusInternalServerError, err.Error())
return
}
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
jsonErr(w, http.StatusNotFound, "task not found")
return
}
writeJSON(w, http.StatusOK, m)
}
func (st *state) handleRuntimeEvents(w http.ResponseWriter, r *http.Request) {
id := r.PathValue("id")
after := int64(0)
if s := r.URL.Query().Get("after_seq"); s != "" {
fmt.Sscanf(s, "%d", &after)
}
if last := strings.TrimSpace(r.Header.Get("Last-Event-ID")); last != "" {
// Accept "task:seq" or bare seq.
if i := strings.LastIndexByte(last, ':'); i >= 0 {
fmt.Sscanf(last[i+1:], "%d", &after)
} else {
fmt.Sscanf(last, "%d", &after)
}
}
wantSSE := strings.Contains(r.Header.Get("Accept"), "text/event-stream") ||
r.URL.Query().Get("stream") == "1"
if !wantSSE {
m, err := runtimeQuery("events", map[string]any{"task_id": id, "after_seq": after})
if err != nil {
jsonErr(w, http.StatusInternalServerError, err.Error())
return
}
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
jsonErr(w, http.StatusNotFound, "task not found")
return
}
writeJSON(w, http.StatusOK, m)
return
}
w.Header().Set("Content-Type", "text/event-stream; charset=utf-8")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
w.WriteHeader(http.StatusOK)
flusher, _ := w.(http.Flusher)
cursor := after
ticker := time.NewTicker(25 * time.Millisecond)
defer ticker.Stop()
heartbeat := time.NewTicker(15 * time.Second)
defer heartbeat.Stop()
ctx := r.Context()
for {
select {
case <-ctx.Done():
return
case <-heartbeat.C:
_, _ = io.WriteString(w, ": ping\n\n")
if flusher != nil {
flusher.Flush()
}
case <-ticker.C:
m, err := runtimeQuery("events", map[string]any{"task_id": id, "after_seq": cursor})
if err != nil {
continue
}
if st, _ := m["status"].(float64); st == 404 {
_, _ = io.WriteString(w, "event: error\ndata: {\"error\":\"not_found\"}\n\n")
if flusher != nil {
flusher.Flush()
}
return
}
evs, _ := m["events"].([]any)
for _, raw := range evs {
b, err := json.Marshal(raw)
if err != nil {
continue
}
var obj map[string]any
_ = json.Unmarshal(b, &obj)
eid := ""
if s, ok := obj["event_id"].(string); ok {
eid = s
}
if seq, ok := obj["seq"].(float64); ok {
cursor = int64(seq)
}
etype, _ := obj["type"].(string)
if etype == "" {
etype = "message"
}
if eid != "" {
_, _ = io.WriteString(w, "id: "+eid+"\n")
}
_, _ = io.WriteString(w, "event: "+etype+"\ndata: "+string(b)+"\n\n")
if flusher != nil {
flusher.Flush()
}
}
}
}
}
@@ -0,0 +1,14 @@
package httpx
import (
"testing"
)
func TestSystemRuntimeRoutesListed(t *testing.T) {
// Compile-time presence: handlers must be addressable.
var st state
_ = st.handleRuntimeAdmit
_ = st.handleRuntimeCancel
_ = st.handleRuntimeEvents
_ = st.handleRuntimeGet
}
@@ -0,0 +1,150 @@
package httpx
import (
"os"
"path/filepath"
"sync"
"testing"
"time"
)
// M-R0: second stream must fail before rewriting A's forward/cancel env.
func TestLegacySSEBridgeSingleFlight(t *testing.T) {
if !tryAcquireLegacySSEBridge() {
t.Fatal("expected first acquire to succeed")
}
fwdA := filepath.Join(t.TempDir(), "a.ndjson")
tokA := filepath.Join(t.TempDir(), "a.tok")
if err := os.WriteFile(fwdA, nil, 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(tokA, nil, 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MARQDO_EVENT_FORWARD", fwdA)
t.Setenv("MARQDO_CANCEL_TOKEN", tokA)
legacySSEBridgeActive.Store(1)
if tryAcquireLegacySSEBridge() {
releaseLegacySSEBridge()
t.Fatal("second acquire must fail while bridge held")
}
active, fwd, tok := legacySSEBridgeSnapshot()
if !active {
t.Fatal("bridge should remain active for holder A")
}
if fwd != fwdA || tok != tokA {
t.Fatalf("rejected second stream rewrote env: fwd=%q tok=%q", fwd, tok)
}
releaseLegacySSEBridge()
if tryAcquireLegacySSEBridge() {
releaseLegacySSEBridge()
} else {
t.Fatal("bridge should be free after release")
}
}
// M-R0: disconnect marks cancel; token file must not be deleted while holder waits.
func TestCancelTokenSurvivesUntilHolderExits(t *testing.T) {
dir := t.TempDir()
tok := filepath.Join(dir, "cancel.tok")
if err := os.WriteFile(tok, nil, 0o600); err != nil {
t.Fatal(err)
}
if err := writeCancelToken(tok); err != nil {
t.Fatal(err)
}
body, err := os.ReadFile(tok)
if err != nil {
t.Fatalf("token must still exist after cancel mark: %v", err)
}
if string(body) != "cancelled" {
t.Fatalf("body=%q", body)
}
}
func TestDrainForwardCarryHalfFrames(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "evt.ndjson")
if err := os.WriteFile(path, []byte(`{"t":"部`), 0o600); err != nil {
t.Fatal(err)
}
var lines []string
off, carry := drainForwardCarry(path, 0, "", func(line string) {
lines = append(lines, line)
})
if len(lines) != 0 {
t.Fatalf("incomplete UTF-8 frame must stay in carry, got %#v", lines)
}
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
t.Fatal(err)
}
if _, err := f.WriteString("分\"}\n{\"t\":\"ok\"}\n"); err != nil {
t.Fatal(err)
}
_ = f.Close()
off, carry = drainForwardCarry(path, off, carry, func(line string) {
lines = append(lines, line)
})
if len(lines) != 2 {
t.Fatalf("got %d lines %#v carry=%q off=%d", len(lines), lines, carry, off)
}
if carry != "" {
t.Fatalf("carry should be empty, got %q", carry)
}
}
func TestDrainForwardCarryCRLFAndSplit(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "evt.ndjson")
part1 := []byte("{\"a\":1}\r\n{\"b\":")
if err := os.WriteFile(path, part1, 0o600); err != nil {
t.Fatal(err)
}
var lines []string
off, carry := drainForwardCarry(path, 0, "", func(line string) {
lines = append(lines, line)
})
if len(lines) != 1 || lines[0] != `{"a":1}` {
t.Fatalf("first frame %#v", lines)
}
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
t.Fatal(err)
}
if _, err := f.WriteString("2}\n"); err != nil {
t.Fatal(err)
}
_ = f.Close()
_, carry = drainForwardCarry(path, off, carry, func(line string) {
lines = append(lines, line)
})
if len(lines) != 2 || lines[1] != `{"b":2}` {
t.Fatalf("split frame %#v carry=%q", lines, carry)
}
}
func TestLegacySSEBridgeConcurrentReject(t *testing.T) {
var wg sync.WaitGroup
acquired := 0
var mu sync.Mutex
for i := 0; i < 8; i++ {
wg.Add(1)
go func() {
defer wg.Done()
if tryAcquireLegacySSEBridge() {
mu.Lock()
acquired++
mu.Unlock()
time.Sleep(20 * time.Millisecond)
releaseLegacySSEBridge()
}
}()
}
wg.Wait()
if acquired < 1 {
t.Fatal("expected at least one acquirer")
}
}
+14 -5
View File
@@ -16,9 +16,10 @@ type Options struct {
Fn string
Body string
Return string
Class string
}
// RoutesFromTable normalizes `|路径|方法|函数|正文|返回|` into invoke_routes map keys.
// RoutesFromTable normalizes `|路径|方法|函数|正文|返回|类别|` into invoke_routes map keys.
func RoutesFromTable(tableV any) map[string]any {
rows, _ := table.AsRows(tableV).([]any)
out := map[string]any{}
@@ -47,7 +48,8 @@ func RoutesFromTable(tableV any) map[string]any {
if ret == "" {
ret = "json"
}
out[path] = routeEntry(method, fnPath, body, ret)
class := strings.ToLower(strings.TrimSpace(col(m, "类别", "class", "Class")))
out[path] = routeEntry(method, fnPath, body, ret, class)
}
return out
}
@@ -78,12 +80,13 @@ func Register(appBag map[string]any, opts Options) (map[string]any, error) {
if ret == "" {
ret = "json"
}
class := strings.ToLower(strings.TrimSpace(opts.Class))
out := clone(appBag)
routes := map[string]any{}
if r, ok := out["invoke_routes"].(map[string]any); ok {
routes = clone(r)
}
routes[key] = routeEntry(method, fnPath, body, ret)
routes[key] = routeEntry(method, fnPath, body, ret, class)
out["invoke_routes"] = routes
return out, nil
}
@@ -101,22 +104,28 @@ func OptionsFromArgs(args map[string]any) (Options, error) {
method, _ := argStr(args, "method", "方法")
body, _ := argStr(args, "body", "正文")
ret, _ := argStr(args, "return", "返回")
class, _ := argStr(args, "class", "类别")
return Options{
Path: path,
Method: method,
Fn: fnPath,
Body: body,
Return: ret,
Class: class,
}, nil
}
func routeEntry(method, fnPath, body, ret string) map[string]any {
return map[string]any{
func routeEntry(method, fnPath, body, ret, class string) map[string]any {
m := map[string]any{
"method": method,
"fn": fnPath,
"body": body,
"return": ret,
}
if class != "" {
m["class"] = class
}
return m
}
func col(m map[string]any, keys ...string) string {
+37 -1
View File
@@ -80,6 +80,7 @@ extern int web_compose_form(char *args_json, char **out_json, char **err_msg);
extern int web_app_new(char *args_json, char **out_json, char **err_msg);
extern int web_app_route(char *args_json, char **out_json, char **err_msg);
extern int web_db_update(char *args_json, char **out_json, char **err_msg);
extern int web_db_update_if(char *args_json, char **out_json, char **err_msg);
extern int web_db_delete(char *args_json, char **out_json, char **err_msg);
extern int web_db_query(char *args_json, char **out_json, char **err_msg);
extern int web_db_count(char *args_json, char **out_json, char **err_msg);
@@ -183,6 +184,7 @@ static int register_core(void) {
if (host_register((char *)"web_db_select", (char *)"url,table,where,limit,order,offset,txn", web_db_select) != 0) return 1;
if (host_register((char *)"web_db_get", (char *)"url,table,id,txn", web_db_get) != 0) return 1;
if (host_register((char *)"web_db_update", (char *)"url,table,id,row,txn", web_db_update) != 0) return 1;
if (host_register((char *)"web_db_update_if", (char *)"url,table,id,row,expect_col,expect_val,txn", web_db_update_if) != 0) return 1;
if (host_register((char *)"web_db_delete", (char *)"url,table,id,txn", web_db_delete) != 0) return 1;
if (host_register((char *)"web_db_exec", (char *)"url,sql,args,txn", web_db_exec) != 0) return 1;
if (host_register((char *)"web_db_query", (char *)"url,sql,args,txn", web_db_query) != 0) return 1;
@@ -264,7 +266,7 @@ static int register_core(void) {
if (host_register((char *)"web_cache_exists", (char *)"url,key", web_cache_exists) != 0) return 1;
if (host_register((char *)"web_cache_ttl", (char *)"url,key", web_cache_ttl) != 0) return 1;
if (host_register((char *)"web_app_proxy", (char *)"app,path,upstream,stream,strip_prefix,methods,headers_from_env,timeout_ms", web_app_proxy) != 0) return 1;
if (host_register((char *)"web_app_invoke", (char *)"app,path,method,fn,body,return", web_app_invoke) != 0) return 1;
if (host_register((char *)"web_app_invoke", (char *)"app,path,method,fn,body,return,class", web_app_invoke) != 0) return 1;
if (host_register((char *)"web_api_key_check", (char *)"key,authorization,pepper,keys", web_api_key_check) != 0) return 1;
if (host_register((char *)"web_render_nodes", (char *)"page,nodes,db,title", web_render_nodes) != 0) return 1;
if (host_register((char *)"web_render_document", (char *)"page,body,db,title", web_render_document) != 0) return 1;
@@ -994,6 +996,40 @@ func web_db_update(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
return replyJSON(outJSON, errMsg, out, err)
}
//export web_db_update_if
func web_db_update_if(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
args, err := parseArgs(argsJSON)
if err != nil {
return replyJSON(outJSON, errMsg, nil, err)
}
url, err := dbURLOf(args)
if err != nil {
return replyJSON(outJSON, errMsg, nil, err)
}
tableName, err := argStrReq(args, "table")
if err != nil {
return replyJSON(outJSON, errMsg, nil, err)
}
id, err := argStrReq(args, "id")
if err != nil {
return replyJSON(outJSON, errMsg, nil, err)
}
expectCol, err := argStrReq(args, "expect_col")
if err != nil {
return replyJSON(outJSON, errMsg, nil, err)
}
expectVal, ok := args["expect_val"]
if !ok {
return replyJSON(outJSON, errMsg, nil, fmt.Errorf("missing expect_val"))
}
row := args["row"]
if row == nil {
row = map[string]any{}
}
out, err := db.UpdateIf(url, tableName, id, row, expectCol, expectVal, txnOpt(args))
return replyJSON(outJSON, errMsg, out, err)
}
//export web_db_delete
func web_db_delete(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
args, err := parseArgs(argsJSON)
+5 -2
View File
@@ -326,8 +326,11 @@ pub fn apply_patch_blocks(
};
match apply_patch_blocks_inner(ctx, path, text) {
Ok(n) => Ok(n),
// soft only swallows unique-FIND miss — never parse / multi-match / whole-file.
Err(e) if soft && e.contains("FIND not found") => Ok(Value::Int(0)),
// soft swallows unique-FIND miss and empty FIND (model protocol mistakes).
// Never softens parse / multi-match / whole-file rewrite.
Err(e) if soft && (e.contains("FIND not found") || e.contains("empty FIND")) => {
Ok(Value::Int(0))
}
Err(e) => Err(e),
}
}
+95 -11
View File
@@ -6,7 +6,8 @@ use std::collections::HashMap;
use std::ffi::{CStr, CString};
use std::os::raw::{c_char, c_int, c_void};
use std::path::PathBuf;
use std::sync::{Mutex, OnceLock};
use std::sync::{Mutex, MutexGuard, OnceLock};
use std::time::{Duration, Instant};
use libloading::{Library, Symbol};
@@ -75,6 +76,8 @@ static GLOBAL_HOST: OnceLock<Mutex<SendPtr<HostContext>>> = OnceLock::new();
/// Process-wide lib-path hook (same lifetime as GLOBAL_HOST).
static GLOBAL_LIB_PATH: OnceLock<Mutex<SendLibPath>> = OnceLock::new();
/// Serialize interpreter re-entry from HTTP worker threads.
/// HostContext is not Sync — one Marqdo invoke at a time. Short routes must
/// fail fast (`interpreter busy`) so a long SSE chat cannot freeze the UI.
static INVOKE_LOCK: OnceLock<Mutex<()>> = OnceLock::new();
fn global_host_slot() -> &'static Mutex<SendPtr<HostContext>> {
@@ -89,6 +92,41 @@ fn invoke_lock() -> &'static Mutex<()> {
INVOKE_LOCK.get_or_init(|| Mutex::new(()))
}
/// Default wait for short HTTP invokes (`MARQDO_INVOKE_WAIT_MS`, default 1500ms).
/// Do not raise this process-wide during SSE — concurrent short routes would hang again.
/// See doc/adr/0008-web-invoke-serialization.md.
fn invoke_wait_budget() -> Duration {
const DEFAULT_MS: u64 = 1500;
match std::env::var("MARQDO_INVOKE_WAIT_MS") {
Ok(s) => {
let ms = s.trim().parse::<u64>().unwrap_or(DEFAULT_MS);
Duration::from_millis(ms.max(1))
}
Err(_) => Duration::from_millis(DEFAULT_MS),
}
}
fn acquire_invoke_lock() -> Result<MutexGuard<'static, ()>, String> {
let budget = invoke_wait_budget();
let deadline = Instant::now() + budget;
let lock = invoke_lock();
loop {
match lock.try_lock() {
Ok(g) => return Ok(g),
Err(std::sync::TryLockError::Poisoned(p)) => return Ok(p.into_inner()),
Err(std::sync::TryLockError::WouldBlock) => {
if Instant::now() >= deadline {
return Err(format!(
"interpreter busy: another request holds the Marqdo invoke lock (waited {}ms)",
budget.as_millis()
));
}
std::thread::sleep(Duration::from_millis(15));
}
}
}
}
/// Hook: resolve bare `lib.member` (and longer paths) in the site module's import tree.
#[derive(Clone, Copy)]
pub struct LibPathCall {
@@ -284,6 +322,42 @@ unsafe extern "C" fn host_query(
}
};
let args_owned: Option<String> = if args_json.is_null() {
None
} else {
CStr::from_ptr(args_json)
.to_str()
.ok()
.map(|s| s.to_string())
};
// Control plane (admit/cancel/events) must not take INVOKE_LOCK or require
// an evaluating HostContext — otherwise cancel cannot interrupt a busy run.
if name == "runtime" {
let raw = args_owned.as_deref().unwrap_or("{}");
let args: serde_json::Value = match serde_json::from_str(if raw.trim().is_empty() {
"{}"
} else {
raw
}) {
Ok(v) => v,
Err(e) => {
set_err(&format!("runtime args: {e}"));
return 1;
}
};
match crate::runtime::host_api::handle(&args) {
Ok(j) => {
set_out(&j.to_string());
return 0;
}
Err(e) => {
set_err(&e);
return 1;
}
}
}
let ctx_ptr = CURRENT_HOST.with(|c| c.get());
let ctx_ptr = if ctx_ptr.is_null() {
global_host_slot()
@@ -300,19 +374,16 @@ unsafe extern "C" fn host_query(
let _invoke_guard;
let from_worker = CURRENT_HOST.with(|c| c.get().is_null());
if from_worker {
_invoke_guard = invoke_lock().lock().unwrap_or_else(|e| e.into_inner());
match acquire_invoke_lock() {
Ok(g) => _invoke_guard = g,
Err(msg) => {
set_err(&msg);
return 1;
}
}
}
let ctx = &mut *ctx_ptr;
let args_owned: Option<String> = if args_json.is_null() {
None
} else {
CStr::from_ptr(args_json)
.to_str()
.ok()
.map(|s| s.to_string())
};
let result = match name {
"module_source" => agent_rt::module_source(ctx).and_then(|v| value_to_json(&v)),
"call_site" => agent_rt::call_site(ctx, Some(ctx.current_line)).and_then(|v| value_to_json(&v)),
@@ -320,6 +391,19 @@ unsafe extern "C" fn host_query(
"cwd" => Ok(serde_json::Value::String(
ctx.cwd.to_string_lossy().into_owned(),
)),
"entry_path" => {
let path = ctx.entry_path.clone().unwrap_or_else(|| ctx.cwd.clone());
let abs = if path.is_absolute() {
path
} else {
std::env::current_dir()
.unwrap_or_else(|_| PathBuf::from("."))
.join(path)
};
Ok(serde_json::Value::String(
abs.to_string_lossy().into_owned(),
))
}
"entry_dir" => {
// Script directory: parent of entry file (for_run also sets ctx.cwd to this).
let dir = ctx
+12 -1
View File
@@ -695,7 +695,9 @@ fn require<'a>(bound: &'a HashMap<String, Value>, key: &str) -> Result<&'a Value
.ok_or_else(|| format!("host call missing `{key}`"))
}
/// Propagate run/selection identity when parent has them (api_key stays via process inheritance).
/// Propagate run/selection identity onto the *child* Command.env map.
/// Reads parent env only as a fallback when an explicit RunContext was not
/// applied; never uses parent `setenv`/`set_current_dir` as request state (M-R1).
fn apply_run_selection_env(cmd: &mut Command) {
for key in [
"MARQDO_RUN_ID",
@@ -703,6 +705,13 @@ fn apply_run_selection_env(cmd: &mut Command) {
"MARQDO_TURN_ID",
"MARQDO_WORKSPACE_SCOPE",
"MARQDO_CANCEL_TOKEN",
"MARQDO_EVENT_FORWARD",
"MARQDO_USAGE_SINK",
"MARQDO_CREDENTIAL_REF",
"MARQDO_WORKSPACE_ID",
"MARQDO_SNAPSHOT_ID",
"MARQDO_LEASE_ID",
"MARQDO_ATTEMPT_ID",
"MARQDO_LLM_MODEL",
"MARQDO_LLM_BASE_URL",
"OPENAI_MODEL",
@@ -712,6 +721,8 @@ fn apply_run_selection_env(cmd: &mut Command) {
cmd.env(key, v);
}
}
// Do not forward process-global MARQDO_CANCEL into children — request cancel
// must use MARQDO_CANCEL_TOKEN so leases stay isolated.
}
#[cfg(test)]
+14
View File
@@ -545,6 +545,20 @@ impl Interpreter {
])
}
/// Pin `module` as site and call `lib.member` (isolated worker / CLI `--call-lib-path`).
pub fn run_site_lib_path(
&mut self,
module: &Module,
path: &str,
args: Option<&Value>,
) -> std::result::Result<Value, String> {
let prev = self.site_module;
self.site_module = Some(module as *const Module);
let out = self.call_site_lib_path(path, args);
self.site_module = prev;
out
}
/// Resolve `lib.member` on the site (entry) module — used by ABI `call_lib_path`.
pub(crate) fn call_site_lib_path(
&mut self,
+14
View File
@@ -25,19 +25,29 @@ pub fn compile(opts: KnowledgeOptions) -> Result<KnowledgeGraph> {
} else {
opts.path.clone()
};
let t0 = std::time::Instant::now();
let semantic = crate::sema::compile_project(&crate::sema::CompileOptions {
project_root: semantic_root,
entry: opts.path.is_file().then(|| opts.path.clone()),
lock_mode: crate::sema::LockMode::Off,
})?;
let sema_ms = t0.elapsed().as_millis();
let t1 = std::time::Instant::now();
crate::sema::write_semantic_projections(&opts.out_dir, &semantic)?;
let sema_write_ms = t1.elapsed().as_millis();
let extract_opts = ExtractOptions {
path: opts.path.clone(),
out_dir: opts.out_dir.clone(),
};
let t2 = std::time::Instant::now();
let graph = extract_graph(&extract_opts)?;
let extract_ms = t2.elapsed().as_millis();
let t3 = std::time::Instant::now();
write_projections(&opts.out_dir, &graph)?;
let retrieve_write_ms = t3.elapsed().as_millis();
let t4 = std::time::Instant::now();
write_publish_manifest(&opts.out_dir, &semantic, &graph)?;
let manifest_ms = t4.elapsed().as_millis();
eprintln!(
"marqdo knowledge: {} module(s), {} symbol(s), {} capability(ies), {} knowledge → {} (semantic graph/; retrieval ekc/)",
graph.modules.len(),
@@ -46,6 +56,10 @@ pub fn compile(opts: KnowledgeOptions) -> Result<KnowledgeGraph> {
graph.knowledge.len(),
opts.out_dir.display()
);
eprintln!(
"marqdo knowledge timing_ms: sema={sema_ms} sema_write={sema_write_ms} extract={extract_ms} retrieve_write={retrieve_write_ms} manifest={manifest_ms} total={}",
t0.elapsed().as_millis()
);
Ok(graph)
}
+51 -1
View File
@@ -37,6 +37,7 @@ pub mod lex;
pub mod linalg_ops;
pub mod load;
pub mod mlsp;
pub mod runtime;
pub mod workbench;
pub mod parse;
pub mod project;
@@ -112,6 +113,10 @@ pub struct RunOptions {
pub binds: Vec<(String, String)>,
/// Attach CoW session for `ext/ai/sandbox` nested runs.
pub sandbox_session: Option<String>,
/// Isolated worker: call `lib.member` on the site entry instead of `# main`.
pub call_lib_path: Option<String>,
/// JSON object of named args for [`Self::call_lib_path`] (file path or inline).
pub args_json: Option<std::path::PathBuf>,
}
impl Default for RunOptions {
@@ -135,6 +140,8 @@ impl Default for RunOptions {
emit_result: None,
binds: Vec::new(),
sandbox_session: None,
call_lib_path: None,
args_json: None,
}
}
}
@@ -229,7 +236,11 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
if opts.trace_eval {
eprintln!("=== marqdo: trace-eval ({path_label}) ===");
}
let value = interp.run_module(&module)?;
let value = if let Some(lib_path) = &opts.call_lib_path {
run_call_lib_path(&mut interp, &module, lib_path, opts)?
} else {
interp.run_module(&module)?
};
if opts.trace_eval {
eprintln!("=== marqdo: end trace-eval ===");
}
@@ -275,6 +286,45 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
Ok(0)
}
/// Worker mode: resolve `lib.member` on the loaded site module (imports already wired).
fn run_call_lib_path(
interp: &mut Interpreter,
module: &crate::ast::Module,
lib_path: &str,
opts: &RunOptions,
) -> Result<crate::value::Value> {
let args_val = load_args_json(opts)?;
// Keep site_module pinned for the duration of the lib call (plugins may re-enter).
let value = interp
.run_site_lib_path(module, lib_path, args_val.as_ref())
.map_err(|e| anyhow::anyhow!(e))?;
Ok(value)
}
fn load_args_json(opts: &RunOptions) -> Result<Option<crate::value::Value>> {
let Some(path) = &opts.args_json else {
return Ok(None);
};
let text = if path.as_os_str() == "-" {
use std::io::Read;
let mut buf = String::new();
std::io::stdin()
.read_to_string(&mut buf)
.map_err(|e| anyhow::anyhow!("args-json stdin: {e}"))?;
buf
} else {
std::fs::read_to_string(path)
.map_err(|e| anyhow::anyhow!("args-json {}: {e}", path.display()))?
};
if text.trim().is_empty() {
return Ok(None);
}
let json: serde_json::Value =
serde_json::from_str(&text).map_err(|e| anyhow::anyhow!("args-json parse: {e}"))?;
let v = crate::host::json::json_to_value(&json).map_err(|e| anyhow::anyhow!(e))?;
Ok(Some(v))
}
fn emit_result_file(opts: &RunOptions, value: &crate::value::Value) -> Result<()> {
let Some(path) = &opts.emit_result else {
return Ok(());
+14
View File
@@ -120,6 +120,14 @@ enum Commands {
#[arg(long = "fs-root", value_name = "DIR")]
fs_root: Option<PathBuf>,
/// Isolated worker: call `lib.member` on the site entry instead of `# main`.
#[arg(long = "call-lib-path", value_name = "LIB.MEMBER")]
call_lib_path: Option<String>,
/// Named args JSON for `--call-lib-path` (file path, or `-` for stdin).
#[arg(long = "args-json", value_name = "FILE")]
args_json: Option<PathBuf>,
/// Emit diagnostics as JSON on stderr (machine-readable; AI/MLSP 面).
#[arg(long)]
json: bool,
@@ -688,6 +696,8 @@ fn try_main(cli: Cli) -> Result<i32> {
emit_result,
binds,
fs_root,
call_lib_path,
args_json,
json: _,
legacy_markup,
} => {
@@ -708,6 +718,8 @@ fn try_main(cli: Cli) -> Result<i32> {
emit_result: emit_result.clone(),
binds: bind_pairs.clone(),
fs_root: fs_root.clone(),
call_lib_path: call_lib_path.clone(),
args_json: args_json.clone(),
..RunOptions::dump_all()
}
} else {
@@ -723,6 +735,8 @@ fn try_main(cli: Cli) -> Result<i32> {
emit_result,
binds: bind_pairs,
fs_root,
call_lib_path,
args_json,
..RunOptions::default()
}
};
+202
View File
@@ -0,0 +1,202 @@
//! Task control plane: cancel without interpreter lock (M-R3).
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
use serde_json::Value;
use crate::runtime::protocol::RuntimeEventV2;
use crate::runtime::scheduler::{AdmissionClass, ScheduleError, Scheduler};
use crate::runtime::task_store::{StoreError, TaskRecord, TaskState, TaskStore};
#[derive(Debug)]
pub enum ControlError {
Store(StoreError),
Schedule(ScheduleError),
Forbidden,
}
impl From<StoreError> for ControlError {
fn from(e: StoreError) -> Self {
Self::Store(e)
}
}
impl From<ScheduleError> for ControlError {
fn from(e: ScheduleError) -> Self {
Self::Schedule(e)
}
}
impl std::fmt::Display for ControlError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Store(e) => write!(f, "{e}"),
Self::Schedule(e) => write!(f, "{e}"),
Self::Forbidden => write!(f, "forbidden"),
}
}
}
pub struct ControlService {
pub store: Arc<TaskStore>,
pub scheduler: Arc<Scheduler>,
/// Per-lease cancel token directory (token files, not process-global env).
cancel_dir: PathBuf,
}
impl ControlService {
pub fn new(store: Arc<TaskStore>, scheduler: Arc<Scheduler>, cancel_dir: impl AsRef<Path>) -> Self {
let cancel_dir = cancel_dir.as_ref().to_path_buf();
let _ = std::fs::create_dir_all(&cancel_dir);
Self {
store,
scheduler,
cancel_dir,
}
}
pub fn admit(
&self,
task_id: &str,
run_id: &str,
workspace_id: &str,
session_id: Option<String>,
idempotency_key: &str,
payload: &Value,
class: AdmissionClass,
) -> Result<(TaskRecord, usize), ControlError> {
let rec = self.store.admit(
task_id,
run_id,
workspace_id,
session_id,
idempotency_key,
payload,
)?;
let depth = self.scheduler.enqueue(&rec, class)?;
let queued = self.store.cas_state(
&rec.task_id,
rec.revision,
TaskState::Queued,
|_| {},
)?;
Ok((queued, depth))
}
/// Cancel path never enters the Marqdo interpreter lock.
pub fn cancel(&self, task_id: &str) -> Result<TaskRecord, ControlError> {
let rec = self.store.request_cancel(task_id)?;
let tok = self.cancel_token_path(task_id);
let _ = std::fs::write(&tok, b"cancelled");
let _ = self.store.append_event(
task_id,
&rec.run_id,
&rec.attempt_id,
"run.cancel_requested",
serde_json::json!({"revision": rec.revision}),
);
Ok(rec)
}
pub fn cancel_token_path(&self, task_id: &str) -> PathBuf {
self.cancel_dir.join(format!("{task_id}.tok"))
}
pub fn mark_cancelled(&self, task_id: &str, expected_revision: u64) -> Result<TaskRecord, ControlError> {
Ok(self.store.cas_state(
task_id,
expected_revision,
TaskState::Cancelled,
|_| {},
)?)
}
pub fn complete(
&self,
task_id: &str,
expected_revision: u64,
result: Value,
) -> Result<TaskRecord, ControlError> {
match self.store.cas_state(task_id, expected_revision, TaskState::Succeeded, |r| {
r.result = Some(result.clone());
}) {
Ok(rec) => Ok(rec),
Err(StoreError::AlreadyTerminal(state)) => {
// Late result after cancel: append evidence only.
let rec = self.store.get(task_id)?;
let _ = self.store.append_event(
task_id,
&rec.run_id,
&rec.attempt_id,
"run.late_result",
serde_json::json!({"ignored_terminal": format!("{state:?}"), "result": result}),
);
Err(ControlError::Store(StoreError::AlreadyTerminal(state)))
}
Err(e) => Err(ControlError::Store(e)),
}
}
pub fn events_since(
&self,
task_id: &str,
after_seq: u64,
) -> Result<Vec<RuntimeEventV2>, ControlError> {
Ok(self.store.events_after(task_id, after_seq)?)
}
/// Collaboration grace + force-kill budget from the design (§6).
pub fn cancel_deadlines() -> (Duration, Duration) {
(Duration::from_secs(2), Duration::from_secs(3))
}
}
/// Convenience wrapper used by HTTP/control adapters.
pub fn cancel_task(svc: &ControlService, task_id: &str) -> Result<TaskRecord, ControlError> {
svc.cancel(task_id)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::runtime::scheduler::SchedulerConfig;
use serde_json::json;
#[test]
fn cancel_writes_token_not_interpreter() {
let dir = std::env::temp_dir().join(format!("mq-ctrl-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let store = Arc::new(TaskStore::memory());
let sched = Arc::new(Scheduler::new(SchedulerConfig::default()));
let ctl = ControlService::new(store, sched, &dir);
let (t, _) = ctl
.admit("t1", "r1", "ws", None, "k", &json!({}), AdmissionClass::Execute)
.unwrap();
let c = ctl.cancel(&t.task_id).unwrap();
assert_eq!(c.state, TaskState::CancelRequested);
let body = std::fs::read_to_string(ctl.cancel_token_path(&t.task_id)).unwrap();
assert_eq!(body, "cancelled");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn late_complete_after_cancel_is_evidence_only() {
let dir = std::env::temp_dir().join(format!("mq-ctrl2-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let store = Arc::new(TaskStore::memory());
let sched = Arc::new(Scheduler::new(SchedulerConfig::default()));
let ctl = ControlService::new(Arc::clone(&store), sched, &dir);
let (t, _) = ctl
.admit("t1", "r1", "ws", None, "k", &json!({}), AdmissionClass::Execute)
.unwrap();
let c = ctl.cancel(&t.task_id).unwrap();
ctl.mark_cancelled(&t.task_id, c.revision).unwrap();
let err = ctl.complete(&t.task_id, c.revision + 1, json!("late")).unwrap_err();
assert!(matches!(err, ControlError::Store(StoreError::AlreadyTerminal(_))));
let evs = store.events_after(&t.task_id, 0).unwrap();
assert!(evs.iter().any(|e| e.event_type == "run.late_result"));
let _ = std::fs::remove_dir_all(&dir);
}
}
+457
View File
@@ -0,0 +1,457 @@
//! Process-global control plane via `host_query("runtime", …)` — no INVOKE_LOCK.
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, OnceLock};
use std::thread;
use std::time::Duration;
use serde_json::{json, Value};
use crate::runtime::control::ControlService;
use crate::runtime::protocol::{
EntryRef, InvokeContext, InvokeRequestV2, LeaseRef, PROTOCOL_VERSION,
};
use crate::runtime::scheduler::{AdmissionClass, Scheduler, SchedulerConfig};
use crate::runtime::supervisor::{Supervisor, SupervisorConfig};
use crate::runtime::task_store::{TaskState, TaskStore};
struct RuntimeHost {
control: ControlService,
supervisor: Arc<Supervisor>,
/// task_id → lease_id
leases: Mutex<std::collections::HashMap<String, String>>,
base: PathBuf,
}
fn runtime_dir() -> PathBuf {
std::env::var_os("MARQDO_RUNTIME_DIR")
.map(PathBuf::from)
.unwrap_or_else(|| std::env::temp_dir().join("marqdo-runtime"))
}
fn marqdo_bin() -> PathBuf {
std::env::var_os("MARQDO_BIN")
.map(PathBuf::from)
.unwrap_or_else(|| std::env::current_exe().unwrap_or_else(|_| PathBuf::from("marqdo")))
}
fn ensure_host() -> &'static RuntimeHost {
static H: OnceLock<RuntimeHost> = OnceLock::new();
static JANITOR: OnceLock<()> = OnceLock::new();
let host = H.get_or_init(|| {
let base = runtime_dir();
let _ = std::fs::create_dir_all(&base);
let store = match TaskStore::with_persist(base.join("store")) {
Ok(s) => Arc::new(s),
Err(_) => Arc::new(TaskStore::memory()),
};
let scheduler = Arc::new(Scheduler::new(SchedulerConfig::default()));
let control = ControlService::new(
Arc::clone(&store),
Arc::clone(&scheduler),
base.join("cancel"),
);
let supervisor = Arc::new(Supervisor::new(
SupervisorConfig::default(),
base.join("stage"),
marqdo_bin(),
base.join("cancel"),
base.join("events"),
));
let _ = supervisor.warm();
RuntimeHost {
control,
supervisor,
leases: Mutex::new(std::collections::HashMap::new()),
base,
}
});
JANITOR.get_or_init(|| {
let sup = Arc::clone(&host.supervisor);
let _ = thread::Builder::new()
.name("mq-runtime-janitor".into())
.spawn(move || loop {
thread::sleep(Duration::from_millis(500));
let _ = sup.janitor_tick();
});
});
host
}
/// Dispatch `host_query("runtime", {op, ...})`. Never blocks on INVOKE_LOCK.
pub fn handle(args: &Value) -> Result<Value, String> {
let op = args
.get("op")
.and_then(|v| v.as_str())
.ok_or_else(|| "runtime requires op".to_string())?;
let host = ensure_host();
let ctl = &host.control;
match op {
"ping" | "capabilities" => Ok(json!({
"ok": true,
"protocol_version": PROTOCOL_VERSION,
"features": {
"runtime.task_control_v2": 1,
"runtime.event_replay": 1,
"runtime.isolated_execution": 1,
},
"runtime_dir": host.base.to_string_lossy(),
})),
"admit" => {
let task_id = req_str(args, "task_id")?;
let run_id = args
.get("run_id")
.and_then(|v| v.as_str())
.unwrap_or(task_id);
let workspace_id = args
.get("workspace_id")
.and_then(|v| v.as_str())
.unwrap_or("default");
let session_id = args
.get("session_id")
.and_then(|v| v.as_str())
.map(|s| s.to_string());
let idem = args
.get("idempotency_key")
.and_then(|v| v.as_str())
.unwrap_or(task_id);
let class = AdmissionClass::parse(
args.get("class").and_then(|v| v.as_str()).unwrap_or("execute"),
);
let payload = args.get("payload").cloned().unwrap_or(json!({}));
let (rec, depth) = ctl
.admit(
task_id,
run_id,
workspace_id,
session_id,
idem,
&payload,
class,
)
.map_err(|e| e.to_string())?;
Ok(json!({
"ok": true,
"task": task_json(&rec),
"queue_depth": depth,
"cancel_token": ctl.cancel_token_path(&rec.task_id).to_string_lossy(),
"event_path": host.supervisor.event_path(&rec.task_id).to_string_lossy(),
}))
}
"dispatch" => dispatch_op(host, args),
"cancel" => {
let task_id = req_str(args, "task_id")?;
match ctl.cancel(task_id) {
Ok(rec) => {
if let Some(lease) = host.leases.lock().unwrap().get(task_id) {
let _ = host.supervisor.signal_cancel(lease, task_id);
} else {
let _ = std::fs::write(ctl.cancel_token_path(task_id), b"cancelled");
}
Ok(json!({
"ok": true,
"task": task_json(&rec),
"cancel_token": ctl.cancel_token_path(task_id).to_string_lossy(),
}))
}
Err(crate::runtime::control::ControlError::Store(
crate::runtime::task_store::StoreError::NotFound,
)) => Ok(json!({"ok": false, "error": "not_found", "status": 404})),
Err(crate::runtime::control::ControlError::Store(
crate::runtime::task_store::StoreError::AlreadyTerminal(s),
)) => Ok(json!({
"ok": true,
"already_terminal": true,
"state": format!("{s:?}").to_ascii_lowercase(),
})),
Err(e) => Err(e.to_string()),
}
}
"get" => {
let task_id = req_str(args, "task_id")?;
match ctl.store.get(task_id) {
Ok(rec) => {
let mut out = json!({"ok": true, "task": task_json(&rec)});
if let Some(lease) = host.leases.lock().unwrap().get(task_id).cloned() {
out["lease_id"] = json!(lease);
if let Some(p) = host.supervisor.emit_result_path(&lease) {
out["emit_result"] = json!(p.to_string_lossy());
}
if let Ok(Some(code)) = host.supervisor.try_reap(&lease) {
out["exit_code"] = json!(code);
host.control
.scheduler
.release_slot(AdmissionClass::Execute);
let _ = ctl.store.cas_state(
task_id,
rec.revision,
if code == 0 {
TaskState::Succeeded
} else {
TaskState::Failed
},
|_| {},
);
}
}
out["event_path"] =
json!(host.supervisor.event_path(task_id).to_string_lossy());
Ok(out)
}
Err(_) => Ok(json!({"ok": false, "error": "not_found", "status": 404})),
}
}
"events" => {
let task_id = req_str(args, "task_id")?;
let after = args.get("after_seq").and_then(|v| v.as_u64()).unwrap_or(0);
let path = host.supervisor.event_path(task_id);
if path.is_file() {
let lines = read_ndjson_lines(&path, after)?;
return Ok(json!({
"ok": true,
"events": lines,
"after_seq": after,
"source": "worker_ndjson",
"event_path": path.to_string_lossy(),
}));
}
match ctl.events_since(task_id, after) {
Ok(evs) => Ok(json!({
"ok": true,
"events": evs,
"after_seq": after,
"source": "task_store",
})),
Err(_) => Ok(json!({"ok": false, "error": "not_found", "status": 404})),
}
}
"append_event" => {
let task_id = req_str(args, "task_id")?;
let rec = ctl.store.get(task_id).map_err(|_| "not_found".to_string())?;
let et = args
.get("type")
.and_then(|v| v.as_str())
.unwrap_or("message.delta");
let payload = args.get("payload").cloned().unwrap_or(json!({}));
let ev = ctl
.store
.append_event(task_id, &rec.run_id, &rec.attempt_id, et, payload)
.map_err(|e| e.to_string())?;
Ok(json!({"ok": true, "event": ev}))
}
other => Err(format!("runtime.unknown_op:{other}")),
}
}
fn dispatch_op(host: &RuntimeHost, args: &Value) -> Result<Value, String> {
let task_id = req_str(args, "task_id")?;
let rec = host
.control
.store
.get(task_id)
.map_err(|_| "not_found".to_string())?;
let entry = args.get("entry").ok_or("dispatch.entry_required")?;
let module = entry
.get("module")
.and_then(|v| v.as_str())
.ok_or("dispatch.module_required")?;
let function = entry
.get("function")
.and_then(|v| v.as_str())
.ok_or("dispatch.function_required")?;
let fs_root = args
.get("fs_root")
.and_then(|v| v.as_str())
.or_else(|| {
args.get("context")
.and_then(|c| c.get("fs_root"))
.and_then(|v| v.as_str())
})
.ok_or("dispatch.fs_root_required")?;
let args_ref = args
.get("args_ref")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.or_else(|| {
args.get("args_json")
.and_then(|v| v.as_str())
.map(|p| format!("file:{p}"))
});
let tok = host.control.cancel_token_path(task_id);
if !tok.exists() {
std::fs::write(&tok, b"").map_err(|e| e.to_string())?;
}
let epoch = host.control.store.next_lease_epoch();
let req = InvokeRequestV2 {
schema: "InvokeRequestV2".into(),
schema_version: PROTOCOL_VERSION,
task_id: task_id.into(),
run_id: rec.run_id.clone(),
attempt_id: rec.attempt_id.clone(),
lease: LeaseRef {
id: format!("lease_{task_id}"),
epoch,
},
entry: EntryRef {
module: module.into(),
function: function.into(),
},
args_ref,
context: InvokeContext {
workspace_id: rec.workspace_id.clone(),
snapshot_id: None,
selection_ref: None,
capabilities_ref: None,
deadline_at: "2099-01-01T00:00:00Z".into(),
fs_root: Some(fs_root.into()),
},
};
if host
.control
.scheduler
.try_acquire_slot(AdmissionClass::Execute)
.is_err()
{
return Ok(json!({
"ok": false,
"error": "no_slot",
"status": 429,
"Retry-After": 2,
}));
}
match host.supervisor.dispatch(req) {
Ok(lease_id) => {
host.leases
.lock()
.unwrap()
.insert(task_id.to_string(), lease_id.clone());
let _ = host.control.store.cas_state(
task_id,
rec.revision,
TaskState::Running,
|r| {
r.lease_id = Some(lease_id.clone());
r.lease_epoch = Some(epoch);
},
);
Ok(json!({
"ok": true,
"lease_id": lease_id,
"task_id": task_id,
"cancel_token": tok.to_string_lossy(),
"event_path": host.supervisor.event_path(task_id).to_string_lossy(),
"emit_result": host.supervisor.emit_result_path(&lease_id)
.map(|p| p.to_string_lossy().to_string()),
}))
}
Err(e) => {
host.control
.scheduler
.release_slot(AdmissionClass::Execute);
if e.contains("no_slot") {
Ok(json!({
"ok": false,
"error": "no_slot",
"status": 429,
"Retry-After": 2,
}))
} else {
Err(e)
}
}
}
}
fn read_ndjson_lines(path: &Path, after_seq: u64) -> Result<Vec<Value>, String> {
let text = std::fs::read_to_string(path).map_err(|e| e.to_string())?;
let mut out = Vec::new();
let mut seq = 0u64;
for line in text.lines() {
let line = line.trim();
if line.is_empty() {
continue;
}
seq += 1;
if seq <= after_seq {
continue;
}
match serde_json::from_str::<Value>(line) {
Ok(mut v) => {
if v.get("seq").is_none() {
if let Some(obj) = v.as_object_mut() {
obj.insert("seq".into(), json!(seq));
}
}
out.push(v);
}
Err(_) => out.push(json!({"seq": seq, "raw": line})),
}
}
Ok(out)
}
fn req_str<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
args.get(key)
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.ok_or_else(|| format!("runtime missing `{key}`"))
}
fn task_json(rec: &crate::runtime::task_store::TaskRecord) -> Value {
json!({
"task_id": rec.task_id,
"run_id": rec.run_id,
"attempt_id": rec.attempt_id,
"workspace_id": rec.workspace_id,
"session_id": rec.session_id,
"state": state_str(rec.state),
"revision": rec.revision,
"idempotency_key": rec.idempotency_key,
"created_at": rec.created_at,
"updated_at": rec.updated_at,
"cancel_requested_at": rec.cancel_requested_at,
"result": rec.result,
"error": rec.error,
"lease_id": rec.lease_id,
})
}
fn state_str(s: TaskState) -> &'static str {
match s {
TaskState::Accepted => "accepted",
TaskState::Queued => "queued",
TaskState::Running => "running",
TaskState::Finalizing => "finalizing",
TaskState::Succeeded => "succeeded",
TaskState::Failed => "failed",
TaskState::CancelRequested => "cancel_requested",
TaskState::Cancelled => "cancelled",
TaskState::Interrupted => "interrupted",
TaskState::NeedsReconciliation => "needs_reconciliation",
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn admit_cancel_without_host_context() {
let id = format!("rt-{}", std::process::id());
let adm = handle(&json!({
"op": "admit",
"task_id": id,
"idempotency_key": id,
"payload": {"n": 1}
}))
.unwrap();
assert_eq!(adm["ok"], true);
let can = handle(&json!({"op": "cancel", "task_id": id})).unwrap();
assert_eq!(can["ok"], true);
assert_eq!(can["task"]["state"], "cancel_requested");
}
}
+58
View File
@@ -0,0 +1,58 @@
//! Worker lease identity and process lifecycle states (M-R2).
use std::time::{Duration, SystemTime};
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct LeaseId(pub String);
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum WorkerState {
Starting,
Ready,
Leased,
Running,
Draining,
Unhealthy,
Terminating,
Exited,
}
#[derive(Debug, Clone)]
pub struct Lease {
pub id: LeaseId,
pub epoch: u64,
pub task_id: String,
pub run_id: String,
pub attempt_id: String,
pub deadline: SystemTime,
pub state: WorkerState,
}
impl Lease {
pub fn new(
id: impl Into<String>,
epoch: u64,
task_id: impl Into<String>,
run_id: impl Into<String>,
attempt_id: impl Into<String>,
ttl: Duration,
) -> Self {
Self {
id: LeaseId(id.into()),
epoch,
task_id: task_id.into(),
run_id: run_id.into(),
attempt_id: attempt_id.into(),
deadline: SystemTime::now() + ttl,
state: WorkerState::Leased,
}
}
pub fn expired(&self) -> bool {
SystemTime::now() > self.deadline
}
pub fn matches(&self, lease_id: &str, epoch: u64) -> bool {
self.id.0 == lease_id && self.epoch == epoch
}
}
+26
View File
@@ -0,0 +1,26 @@
//! Isolated web runtime (M-R2–M-R4): supervisor, worker IPC, leases, task
//! admission/control, and persistent event log.
//!
//! Design: `doc/design/isolated-web-runtime-development-2026-10-26.md`.
//! Until capability probes report version ≥1, callers must treat this as opt-in.
pub mod control;
pub mod host_api;
pub mod lease;
pub mod protocol;
pub mod publish;
pub mod scheduler;
pub mod supervisor;
pub mod task_store;
pub mod worker;
pub use control::{cancel_task, ControlError, ControlService};
pub use lease::{Lease, LeaseId, WorkerState};
pub use protocol::{
InvokeRequestV2, RuntimeEventV2, CONTROL_FRAME_MAX_BYTES, PROTOCOL_VERSION,
};
pub use publish::{PublishJournal, PublishOutcome, PublishPhase};
pub use scheduler::{AdmissionClass, Scheduler, SchedulerConfig};
pub use supervisor::{Supervisor, SupervisorConfig};
pub use task_store::{TaskRecord, TaskState, TaskStore};
pub use worker::{WorkerHandle, WorkerPool};
+158
View File
@@ -0,0 +1,158 @@
//! Length-prefixed JSON control frames + event channel types (M-R2 / M-R4).
use serde::{Deserialize, Serialize};
use serde_json::Value;
pub const PROTOCOL_VERSION: u32 = 2;
pub const CONTROL_FRAME_MAX_BYTES: usize = 1024 * 1024;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct InvokeRequestV2 {
pub schema: String,
pub schema_version: u32,
pub task_id: String,
pub run_id: String,
pub attempt_id: String,
pub lease: LeaseRef,
pub entry: EntryRef,
#[serde(default)]
pub args_ref: Option<String>,
pub context: InvokeContext,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct LeaseRef {
pub id: String,
pub epoch: u64,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EntryRef {
pub module: String,
pub function: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct InvokeContext {
pub workspace_id: String,
#[serde(default)]
pub snapshot_id: Option<String>,
#[serde(default)]
pub selection_ref: Option<String>,
#[serde(default)]
pub capabilities_ref: Option<String>,
pub deadline_at: String,
/// Absolute site filesystem root (`--fs-root` for the worker).
#[serde(default)]
pub fs_root: Option<String>,
}
impl InvokeRequestV2 {
pub fn validate(&self) -> Result<(), String> {
if self.schema != "InvokeRequestV2" {
return Err(format!("protocol.schema: {}", self.schema));
}
if self.schema_version != PROTOCOL_VERSION {
return Err(format!(
"protocol.version: got {} want {PROTOCOL_VERSION}",
self.schema_version
));
}
if self.task_id.is_empty() || self.run_id.is_empty() || self.attempt_id.is_empty() {
return Err("protocol.ids_required".into());
}
if self.entry.module.is_empty() || self.entry.function.is_empty() {
return Err("protocol.entry_required".into());
}
Ok(())
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RuntimeEventV2 {
pub schema: String,
pub schema_version: u32,
pub event_id: String,
pub seq: u64,
pub task_id: String,
pub run_id: String,
pub attempt_id: String,
#[serde(rename = "type")]
pub event_type: String,
pub payload: Value,
}
impl RuntimeEventV2 {
pub fn new(
task_id: &str,
run_id: &str,
attempt_id: &str,
seq: u64,
event_type: &str,
payload: Value,
) -> Self {
Self {
schema: "RuntimeEventV2".into(),
schema_version: PROTOCOL_VERSION,
event_id: format!("{task_id}:{seq}"),
seq,
task_id: task_id.into(),
run_id: run_id.into(),
attempt_id: attempt_id.into(),
event_type: event_type.into(),
payload,
}
}
}
/// Encode a control frame: u32 LE length + UTF-8 JSON (≤ CONTROL_FRAME_MAX_BYTES).
pub fn encode_frame(value: &Value) -> Result<Vec<u8>, String> {
let body = serde_json::to_vec(value).map_err(|e| format!("protocol.encode: {e}"))?;
if body.len() > CONTROL_FRAME_MAX_BYTES {
return Err("protocol.frame_too_large".into());
}
let mut out = Vec::with_capacity(4 + body.len());
out.extend_from_slice(&(body.len() as u32).to_le_bytes());
out.extend_from_slice(&body);
Ok(out)
}
/// Decode one frame from the front of `buf`; returns (value, bytes_consumed).
pub fn decode_frame(buf: &[u8]) -> Result<Option<(Value, usize)>, String> {
if buf.len() < 4 {
return Ok(None);
}
let len = u32::from_le_bytes([buf[0], buf[1], buf[2], buf[3]]) as usize;
if len > CONTROL_FRAME_MAX_BYTES {
return Err("protocol.frame_too_large".into());
}
if buf.len() < 4 + len {
return Ok(None);
}
let value: Value = serde_json::from_slice(&buf[4..4 + len])
.map_err(|e| format!("protocol.decode: {e}"))?;
Ok(Some((value, 4 + len)))
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn frame_round_trip() {
let v = json!({"hello": "世界"});
let enc = encode_frame(&v).unwrap();
let (got, n) = decode_frame(&enc).unwrap().unwrap();
assert_eq!(n, enc.len());
assert_eq!(got, v);
}
#[test]
fn incomplete_frame_waits() {
let v = json!({"a": 1});
let enc = encode_frame(&v).unwrap();
assert!(decode_frame(&enc[..3]).unwrap().is_none());
assert!(decode_frame(&enc[..enc.len() - 1]).unwrap().is_none());
}
}
+274
View File
@@ -0,0 +1,274 @@
//! Publish journal + expected_generation CAS (M-R6).
use std::fs;
use std::io::Write;
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum PublishPhase {
Staging,
Validated,
GenerationRenamed,
CurrentSwitched,
DbProjected,
Aborted,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PublishJournal {
pub kb_id: String,
pub expected_generation: Option<String>,
pub new_generation: String,
pub phase: PublishPhase,
pub staging_dir: PathBuf,
pub generation_dir: PathBuf,
pub current_path: PathBuf,
pub source_digest: String,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum PublishOutcome {
Published { generation: String },
Conflict { current: String, expected: String },
Recovered { generation: String },
Aborted { reason: String },
}
impl PublishJournal {
pub fn begin(
out_dir: impl AsRef<Path>,
kb_id: impl Into<String>,
expected_generation: Option<String>,
new_generation: impl Into<String>,
source_digest: impl Into<String>,
) -> Result<Self, String> {
let out_dir = out_dir.as_ref();
let new_generation = new_generation.into();
let staging_dir = out_dir.join(".publish-staging").join(&new_generation);
let generation_dir = out_dir.join("generations").join(&new_generation);
let current_path = out_dir.join("CURRENT");
fs::create_dir_all(&staging_dir).map_err(|e| e.to_string())?;
let j = Self {
kb_id: kb_id.into(),
expected_generation,
new_generation,
phase: PublishPhase::Staging,
staging_dir,
generation_dir,
current_path,
source_digest: source_digest.into(),
};
j.write_journal(out_dir)?;
Ok(j)
}
fn journal_path(out_dir: &Path) -> PathBuf {
out_dir.join("publish_journal.json")
}
fn write_journal(&self, out_dir: &Path) -> Result<(), String> {
let path = Self::journal_path(out_dir);
let tmp = path.with_extension("json.tmp");
let body = serde_json::to_vec_pretty(self).map_err(|e| e.to_string())?;
fs::write(&tmp, body).map_err(|e| e.to_string())?;
// Best-effort fsync via File.
let f = fs::OpenOptions::new()
.write(true)
.open(&tmp)
.map_err(|e| e.to_string())?;
let _ = f.sync_all();
drop(f);
fs::rename(&tmp, &path).map_err(|e| e.to_string())?;
Ok(())
}
pub fn load(out_dir: impl AsRef<Path>) -> Result<Option<Self>, String> {
let path = Self::journal_path(out_dir.as_ref());
if !path.is_file() {
return Ok(None);
}
let body = fs::read_to_string(&path).map_err(|e| e.to_string())?;
let j: Self = serde_json::from_str(&body).map_err(|e| e.to_string())?;
Ok(Some(j))
}
/// CAS: refuse if CURRENT does not match expected_generation.
pub fn assert_expected_current(&self) -> Result<String, PublishOutcome> {
let current = fs::read_to_string(&self.current_path)
.unwrap_or_default()
.trim()
.to_string();
if let Some(exp) = &self.expected_generation {
if &current != exp {
return Err(PublishOutcome::Conflict {
current,
expected: exp.clone(),
});
}
}
Ok(current)
}
pub fn mark(&mut self, out_dir: &Path, phase: PublishPhase) -> Result<(), String> {
self.phase = phase;
self.write_journal(out_dir)
}
/// Atomic publish: staging tree already complete → rename into generations → CURRENT.
pub fn commit_filesystem(&mut self, out_dir: &Path) -> Result<PublishOutcome, String> {
match self.assert_expected_current() {
Err(o) => return Ok(o),
Ok(_) => {}
}
self.mark(out_dir, PublishPhase::Validated)?;
if self.generation_dir.exists() {
fs::remove_dir_all(&self.generation_dir).map_err(|e| e.to_string())?;
}
fs::create_dir_all(self.generation_dir.parent().unwrap()).map_err(|e| e.to_string())?;
fs::rename(&self.staging_dir, &self.generation_dir).map_err(|e| e.to_string())?;
self.mark(out_dir, PublishPhase::GenerationRenamed)?;
let tmp = self.current_path.with_extension("tmp");
{
let mut f = fs::File::create(&tmp).map_err(|e| e.to_string())?;
writeln!(f, "{}", self.new_generation).map_err(|e| e.to_string())?;
let _ = f.sync_all();
}
fs::rename(&tmp, &self.current_path).map_err(|e| e.to_string())?;
self.mark(out_dir, PublishPhase::CurrentSwitched)?;
// Clear journal after successful CURRENT switch; DB projection is product-side.
let _ = fs::remove_file(Self::journal_path(out_dir));
Ok(PublishOutcome::Published {
generation: self.new_generation.clone(),
})
}
/// Crash recovery: if CURRENT already points at new_generation, treat as done;
/// if generation exists but CURRENT old, finish CURRENT switch; else abort staging.
pub fn recover(out_dir: impl AsRef<Path>) -> Result<PublishOutcome, String> {
let out_dir = out_dir.as_ref();
let Some(mut j) = Self::load(out_dir)? else {
return Ok(PublishOutcome::Aborted {
reason: "no_journal".into(),
});
};
let current = fs::read_to_string(&j.current_path)
.unwrap_or_default()
.trim()
.to_string();
if current == j.new_generation {
let _ = fs::remove_file(Self::journal_path(out_dir));
return Ok(PublishOutcome::Recovered {
generation: j.new_generation,
});
}
match j.phase {
PublishPhase::GenerationRenamed | PublishPhase::Validated
if j.generation_dir.is_dir() =>
{
let tmp = j.current_path.with_extension("tmp");
fs::write(&tmp, format!("{}\n", j.new_generation)).map_err(|e| e.to_string())?;
fs::rename(&tmp, &j.current_path).map_err(|e| e.to_string())?;
let _ = fs::remove_file(Self::journal_path(out_dir));
Ok(PublishOutcome::Recovered {
generation: j.new_generation,
})
}
PublishPhase::Staging | PublishPhase::Aborted => {
let _ = fs::remove_dir_all(&j.staging_dir);
j.mark(out_dir, PublishPhase::Aborted)?;
let _ = fs::remove_file(Self::journal_path(out_dir));
Ok(PublishOutcome::Aborted {
reason: "incomplete_staging".into(),
})
}
other => Ok(PublishOutcome::Aborted {
reason: format!("unhandled_phase:{other:?}"),
}),
}
}
}
/// Stable logical document identity independent of generation directory name.
pub fn logical_document_id(project_id: &str, relative_path: &str) -> String {
let mut h = Sha256::new();
h.update(project_id.as_bytes());
h.update(b"\0");
h.update(relative_path.replace('\\', "/").as_bytes());
format!("doc:{:x}", h.finalize())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn cas_conflict_and_commit() {
let root = std::env::temp_dir().join(format!("mq-pub-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
fs::create_dir_all(root.join("generations")).unwrap();
fs::write(root.join("CURRENT"), "gen_old\n").unwrap();
let mut j = PublishJournal::begin(&root, "kb1", Some("gen_old".into()), "gen_new", "digest")
.unwrap();
fs::write(j.staging_dir.join("manifest.json"), "{}").unwrap();
let out = j.commit_filesystem(&root).unwrap();
assert_eq!(
out,
PublishOutcome::Published {
generation: "gen_new".into()
}
);
assert_eq!(
fs::read_to_string(root.join("CURRENT")).unwrap().trim(),
"gen_new"
);
let mut j2 =
PublishJournal::begin(&root, "kb1", Some("gen_old".into()), "gen_x", "d").unwrap();
fs::write(j2.staging_dir.join("x"), "1").unwrap();
let conflict = j2.commit_filesystem(&root).unwrap();
assert!(matches!(conflict, PublishOutcome::Conflict { .. }));
let _ = std::fs::remove_dir_all(&root);
}
#[test]
fn recover_after_generation_rename() {
let root = std::env::temp_dir().join(format!("mq-pub2-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
fs::create_dir_all(root.join("generations/gen_r")).unwrap();
fs::write(root.join("CURRENT"), "old\n").unwrap();
let mut j =
PublishJournal::begin(&root, "kb", Some("old".into()), "gen_r", "d").unwrap();
// Simulate crash after generation rename: staging already moved.
let _ = fs::remove_dir_all(&j.staging_dir);
j.phase = PublishPhase::GenerationRenamed;
j.write_journal(&root).unwrap();
let out = PublishJournal::recover(&root).unwrap();
assert_eq!(
out,
PublishOutcome::Recovered {
generation: "gen_r".into()
}
);
assert_eq!(
fs::read_to_string(root.join("CURRENT")).unwrap().trim(),
"gen_r"
);
let _ = std::fs::remove_dir_all(&root);
}
#[test]
fn logical_id_stable_across_generation_paths() {
let a = logical_document_id("proj", "docs/a.mq.md");
let b = logical_document_id("proj", "docs/a.mq.md");
assert_eq!(a, b);
assert_ne!(a, logical_document_id("proj", "docs/b.mq.md"));
}
}
+249
View File
@@ -0,0 +1,249 @@
//! Bounded admission classes and fair queueing (M-R3).
use std::collections::{HashMap, VecDeque};
use std::sync::Mutex;
use crate::runtime::task_store::TaskRecord;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AdmissionClass {
Control,
Read,
Short,
Execute,
Commit,
/// Undeclared endpoints stay on the legacy single-interpreter path.
LegacySerial,
}
impl AdmissionClass {
pub fn parse(s: &str) -> Self {
match s.to_ascii_lowercase().as_str() {
"control" => Self::Control,
"read" => Self::Read,
"short" => Self::Short,
"execute" => Self::Execute,
"commit" => Self::Commit,
_ => Self::LegacySerial,
}
}
}
#[derive(Debug, Clone)]
pub struct SchedulerConfig {
pub execute_slots: usize,
pub short_slots: usize,
pub read_slots: usize,
pub global_queue_limit: usize,
pub workspace_queue_limit: usize,
pub session_pending_limit: usize,
}
impl Default for SchedulerConfig {
fn default() -> Self {
// 4 vCPU / 8 GiB verification machine defaults from the design.
Self {
execute_slots: 2,
short_slots: 1,
read_slots: 1,
global_queue_limit: 64,
workspace_queue_limit: 16,
session_pending_limit: 4,
}
}
}
#[derive(Debug)]
pub enum ScheduleError {
QueueFull,
WorkspaceFull,
SessionFull,
NoSlot,
}
impl std::fmt::Display for ScheduleError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::QueueFull => write!(f, "queue_full"),
Self::WorkspaceFull => write!(f, "workspace_full"),
Self::SessionFull => write!(f, "session_full"),
Self::NoSlot => write!(f, "no_slot"),
}
}
}
struct QueueState {
global: VecDeque<String>,
by_workspace: HashMap<String, VecDeque<String>>,
by_session: HashMap<String, usize>,
active_execute: usize,
active_short: usize,
active_read: usize,
/// workspace rotation cursor
workspace_order: VecDeque<String>,
}
pub struct Scheduler {
cfg: SchedulerConfig,
inner: Mutex<QueueState>,
}
impl Scheduler {
pub fn new(cfg: SchedulerConfig) -> Self {
Self {
cfg,
inner: Mutex::new(QueueState {
global: VecDeque::new(),
by_workspace: HashMap::new(),
by_session: HashMap::new(),
active_execute: 0,
active_short: 0,
active_read: 0,
workspace_order: VecDeque::new(),
}),
}
}
pub fn enqueue(&self, task: &TaskRecord, class: AdmissionClass) -> Result<usize, ScheduleError> {
if class == AdmissionClass::Control || class == AdmissionClass::LegacySerial {
return Ok(0);
}
let mut g = self.inner.lock().unwrap();
if g.global.len() >= self.cfg.global_queue_limit {
return Err(ScheduleError::QueueFull);
}
let ws_len = g
.by_workspace
.get(&task.workspace_id)
.map(|q| q.len())
.unwrap_or(0);
if ws_len >= self.cfg.workspace_queue_limit {
return Err(ScheduleError::WorkspaceFull);
}
if let Some(sid) = &task.session_id {
let n = g.by_session.entry(sid.clone()).or_insert(0);
if *n >= self.cfg.session_pending_limit {
return Err(ScheduleError::SessionFull);
}
*n += 1;
}
if !g.workspace_order.iter().any(|w| w == &task.workspace_id) {
g.workspace_order.push_back(task.workspace_id.clone());
}
g.by_workspace
.entry(task.workspace_id.clone())
.or_default()
.push_back(task.task_id.clone());
g.global.push_back(task.task_id.clone());
Ok(g.global.len())
}
pub fn try_acquire_slot(&self, class: AdmissionClass) -> Result<(), ScheduleError> {
let mut g = self.inner.lock().unwrap();
match class {
AdmissionClass::Execute => {
if g.active_execute >= self.cfg.execute_slots {
return Err(ScheduleError::NoSlot);
}
g.active_execute += 1;
}
AdmissionClass::Short | AdmissionClass::Commit => {
if g.active_short >= self.cfg.short_slots {
return Err(ScheduleError::NoSlot);
}
g.active_short += 1;
}
AdmissionClass::Read => {
if g.active_read >= self.cfg.read_slots {
return Err(ScheduleError::NoSlot);
}
g.active_read += 1;
}
AdmissionClass::Control | AdmissionClass::LegacySerial => {}
}
Ok(())
}
pub fn release_slot(&self, class: AdmissionClass) {
let mut g = self.inner.lock().unwrap();
match class {
AdmissionClass::Execute => {
g.active_execute = g.active_execute.saturating_sub(1);
}
AdmissionClass::Short | AdmissionClass::Commit => {
g.active_short = g.active_short.saturating_sub(1);
}
AdmissionClass::Read => {
g.active_read = g.active_read.saturating_sub(1);
}
AdmissionClass::Control | AdmissionClass::LegacySerial => {}
}
}
/// Fair pop: rotate workspaces, then session FIFO within workspace.
pub fn pop_next(&self) -> Option<String> {
let mut g = self.inner.lock().unwrap();
if g.workspace_order.is_empty() {
return None;
}
for _ in 0..g.workspace_order.len() {
let ws = g.workspace_order.pop_front()?;
if let Some(q) = g.by_workspace.get_mut(&ws) {
if let Some(tid) = q.pop_front() {
g.workspace_order.push_back(ws);
g.global.retain(|id| id != &tid);
return Some(tid);
}
}
}
None
}
pub fn queue_depth(&self) -> usize {
self.inner.lock().unwrap().global.len()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::runtime::task_store::{TaskRecord, TaskState};
fn task(id: &str, ws: &str, session: Option<&str>) -> TaskRecord {
TaskRecord {
task_id: id.into(),
run_id: format!("r-{id}"),
attempt_id: format!("a-{id}"),
workspace_id: ws.into(),
session_id: session.map(|s| s.into()),
state: TaskState::Queued,
revision: 1,
idempotency_key: id.into(),
payload_hash: "h".into(),
lease_id: None,
lease_epoch: None,
created_at: "0".into(),
updated_at: "0".into(),
cancel_requested_at: None,
result: None,
error: None,
}
}
#[test]
fn workspace_rotation_fairness() {
let sched = Scheduler::new(SchedulerConfig::default());
sched
.enqueue(&task("a1", "wsA", None), AdmissionClass::Execute)
.unwrap();
sched
.enqueue(&task("a2", "wsA", None), AdmissionClass::Execute)
.unwrap();
sched
.enqueue(&task("b1", "wsB", None), AdmissionClass::Execute)
.unwrap();
assert_eq!(sched.pop_next().as_deref(), Some("a1"));
assert_eq!(sched.pop_next().as_deref(), Some("b1"));
assert_eq!(sched.pop_next().as_deref(), Some("a2"));
}
}
+303
View File
@@ -0,0 +1,303 @@
//! Supervisor: lease workers, enforce deadlines, reap orphans (M-R2).
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
use crate::runtime::lease::{Lease, WorkerState};
use crate::runtime::protocol::InvokeRequestV2;
use crate::runtime::worker::{WorkerHandle, WorkerPool};
#[derive(Debug, Clone)]
pub struct SupervisorConfig {
pub execute_slots: usize,
pub worker_ttl: Duration,
pub cancel_grace: Duration,
pub force_kill_after: Duration,
}
impl Default for SupervisorConfig {
fn default() -> Self {
Self {
execute_slots: 2,
worker_ttl: Duration::from_secs(600),
cancel_grace: Duration::from_secs(2),
force_kill_after: Duration::from_secs(3),
}
}
}
struct ActiveWorker {
handle: WorkerHandle,
cancel_since: Option<Instant>,
}
pub struct Supervisor {
cfg: SupervisorConfig,
pool: Mutex<WorkerPool>,
active: Mutex<HashMap<String, ActiveWorker>>,
cancel_dir: PathBuf,
event_dir: PathBuf,
}
impl Supervisor {
pub fn new(
cfg: SupervisorConfig,
staging_root: impl AsRef<Path>,
marqdo_bin: impl AsRef<Path>,
cancel_dir: impl AsRef<Path>,
event_dir: impl AsRef<Path>,
) -> Self {
let cancel_dir = cancel_dir.as_ref().to_path_buf();
let event_dir = event_dir.as_ref().to_path_buf();
let _ = std::fs::create_dir_all(&cancel_dir);
let _ = std::fs::create_dir_all(&event_dir);
Self {
pool: Mutex::new(WorkerPool::new(
staging_root,
marqdo_bin,
cfg.execute_slots,
)),
cfg,
active: Mutex::new(HashMap::new()),
cancel_dir,
event_dir,
}
}
pub fn warm(&self) -> Result<(), String> {
self.pool.lock().unwrap().warm()
}
pub fn dispatch(&self, req: InvokeRequestV2) -> Result<String, String> {
req.validate()?;
{
let active = self.active.lock().unwrap();
if active.len() >= self.cfg.execute_slots {
return Err("supervisor.no_slot".into());
}
}
let mut pool = self.pool.lock().unwrap();
let worker = pool.acquire()?;
drop(pool);
let epoch = Instant::now().elapsed().as_nanos() as u64;
let lease = Lease::new(
format!("lease_{}", req.task_id),
epoch,
&req.task_id,
&req.run_id,
&req.attempt_id,
self.cfg.worker_ttl,
);
let cancel = self.cancel_dir.join(format!("{}.tok", req.task_id));
if !cancel.exists() {
std::fs::write(&cancel, b"").map_err(|e| e.to_string())?;
}
let events = self.event_dir.join(format!("{}.ndjson", req.task_id));
std::fs::write(&events, b"").map_err(|e| e.to_string())?;
// Durable emit path (not under staging — WorkerHandle::Drop removes staging).
let emit = self.event_dir.join(format!("{}.result.json", req.task_id));
let fs_root = req
.context
.fs_root
.as_ref()
.map(PathBuf::from)
.filter(|p| p.is_dir())
.ok_or_else(|| "supervisor.fs_root_required".to_string())?;
let args_path = req.args_ref.as_ref().and_then(|r| {
r.strip_prefix("file:")
.map(PathBuf::from)
.or_else(|| {
let p = PathBuf::from(r);
p.is_file().then_some(p)
})
});
let handle = {
if std::env::var_os("MARQDO_RUNTIME_DRY_RUN").is_some() {
let mut w = worker;
w.lease = Some(lease.clone());
w.emit_result = Some(emit.clone());
w.state = WorkerState::Running;
w
} else {
self.pool.lock().unwrap().lease_and_start(
worker,
lease.clone(),
&req,
&cancel,
&events,
&fs_root,
args_path.as_deref(),
&emit,
)?
}
};
let lease_id = lease.id.0.clone();
self.active.lock().unwrap().insert(
lease_id.clone(),
ActiveWorker {
handle,
cancel_since: None,
},
);
Ok(lease_id)
}
pub fn event_path(&self, task_id: &str) -> PathBuf {
self.event_dir.join(format!("{task_id}.ndjson"))
}
pub fn cancel_path(&self, task_id: &str) -> PathBuf {
self.cancel_dir.join(format!("{task_id}.tok"))
}
pub fn emit_result_path(&self, lease_id: &str) -> Option<PathBuf> {
self.active
.lock()
.unwrap()
.get(lease_id)
.and_then(|w| w.handle.emit_result.clone())
}
/// Non-blocking poll: if worker exited, remove from active and return exit code.
pub fn try_reap(&self, lease_id: &str) -> Result<Option<i32>, String> {
let mut active = self.active.lock().unwrap();
let Some(w) = active.get_mut(lease_id) else {
return Ok(None);
};
match w.handle.try_wait()? {
Some(st) => {
let code = st.code().unwrap_or(-1);
active.remove(lease_id);
let _ = self.pool.lock().unwrap().warm();
Ok(Some(code))
}
None => Ok(None),
}
}
pub fn signal_cancel(&self, lease_id: &str, task_id: &str) -> Result<(), String> {
let tok = self.cancel_dir.join(format!("{task_id}.tok"));
std::fs::write(&tok, b"cancelled").map_err(|e| e.to_string())?;
let mut active = self.active.lock().unwrap();
if let Some(w) = active.get_mut(lease_id) {
w.cancel_since = Some(Instant::now());
w.handle.state = WorkerState::Draining;
}
Ok(())
}
/// Reap expired / force-kill after cancel grace. Returns orphan count.
pub fn janitor_tick(&self) -> usize {
let mut orphans = 0usize;
let mut active = self.active.lock().unwrap();
let mut done = Vec::new();
for (id, w) in active.iter_mut() {
if let Some(lease) = &w.handle.lease {
if lease.expired() {
w.handle.kill_process_group();
orphans += 1;
done.push(id.clone());
continue;
}
}
if let Some(since) = w.cancel_since {
if since.elapsed() > self.cfg.cancel_grace + self.cfg.force_kill_after {
w.handle.kill_process_group();
orphans += 1;
done.push(id.clone());
}
}
}
for id in done {
active.remove(&id);
}
orphans
}
pub fn active_count(&self) -> usize {
self.active.lock().unwrap().len()
}
pub fn release(&self, lease_id: &str) {
let mut active = self.active.lock().unwrap();
if let Some(mut w) = active.remove(lease_id) {
w.handle.state = WorkerState::Draining;
w.handle.kill_process_group();
}
// Async replenish hint.
let _ = self.pool.lock().unwrap().warm();
}
}
/// Shared supervisor handle for HTTP/control wiring.
pub type SharedSupervisor = Arc<Supervisor>;
#[cfg(test)]
mod tests {
use super::*;
use crate::runtime::protocol::{EntryRef, InvokeContext, LeaseRef};
fn req(task: &str) -> InvokeRequestV2 {
InvokeRequestV2 {
schema: "InvokeRequestV2".into(),
schema_version: 2,
task_id: task.into(),
run_id: format!("run_{task}"),
attempt_id: format!("attempt_{task}"),
lease: LeaseRef {
id: format!("lease_{task}"),
epoch: 1,
},
entry: EntryRef {
module: "agent/qdagent.mq.md".into(),
function: "run".into(),
},
args_ref: None,
context: InvokeContext {
workspace_id: "ws".into(),
snapshot_id: None,
selection_ref: None,
capabilities_ref: None,
deadline_at: "2099-01-01T00:00:00Z".into(),
fs_root: Some("/tmp".into()),
},
}
}
#[test]
fn dry_run_two_parallel_leases() {
std::env::set_var("MARQDO_RUNTIME_DRY_RUN", "1");
let root = std::env::temp_dir().join(format!("mq-sup-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
let bin = std::env::current_exe().unwrap();
let _ = std::fs::create_dir_all(root.join("fs"));
let mut ra = req("a");
ra.context.fs_root = Some(root.join("fs").to_string_lossy().into());
let mut rb = req("b");
rb.context.fs_root = ra.context.fs_root.clone();
let mut rc = req("c");
rc.context.fs_root = ra.context.fs_root.clone();
let sup = Supervisor::new(
SupervisorConfig {
execute_slots: 2,
..SupervisorConfig::default()
},
root.join("stage"),
&bin,
root.join("cancel"),
root.join("events"),
);
sup.warm().unwrap();
let a = sup.dispatch(ra).unwrap();
let b = sup.dispatch(rb).unwrap();
assert_ne!(a, b);
assert_eq!(sup.active_count(), 2);
assert!(sup.dispatch(rc).is_err());
sup.release(&a);
assert_eq!(sup.active_count(), 1);
let _ = std::fs::remove_dir_all(&root);
std::env::remove_var("MARQDO_RUNTIME_DRY_RUN");
}
}
+385
View File
@@ -0,0 +1,385 @@
//! Task index with idempotency, revision CAS, and event log (M-R3 / M-R4).
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::{SystemTime, UNIX_EPOCH};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use sha2::{Digest, Sha256};
use crate::runtime::protocol::RuntimeEventV2;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum TaskState {
Accepted,
Queued,
Running,
Finalizing,
Succeeded,
Failed,
CancelRequested,
Cancelled,
Interrupted,
NeedsReconciliation,
}
impl TaskState {
pub fn is_terminal(self) -> bool {
matches!(
self,
Self::Succeeded
| Self::Failed
| Self::Cancelled
| Self::Interrupted
| Self::NeedsReconciliation
)
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TaskRecord {
pub task_id: String,
pub run_id: String,
pub attempt_id: String,
pub workspace_id: String,
pub session_id: Option<String>,
pub state: TaskState,
pub revision: u64,
pub idempotency_key: String,
pub payload_hash: String,
pub lease_id: Option<String>,
pub lease_epoch: Option<u64>,
pub created_at: String,
pub updated_at: String,
pub cancel_requested_at: Option<String>,
pub result: Option<Value>,
pub error: Option<String>,
}
#[derive(Default)]
struct Inner {
tasks: HashMap<String, TaskRecord>,
/// idempotency_key → task_id
idem: HashMap<String, String>,
/// task_id → events (monotonic seq)
events: HashMap<String, Vec<RuntimeEventV2>>,
next_epoch: u64,
}
pub struct TaskStore {
inner: Arc<Mutex<Inner>>,
persist_dir: Option<PathBuf>,
}
#[derive(Debug)]
pub enum StoreError {
Conflict(String),
NotFound,
AlreadyTerminal(TaskState),
Io(String),
}
impl std::fmt::Display for StoreError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Conflict(s) => write!(f, "conflict: {s}"),
Self::NotFound => write!(f, "not_found"),
Self::AlreadyTerminal(s) => write!(f, "already_terminal:{s:?}"),
Self::Io(s) => write!(f, "io: {s}"),
}
}
}
impl TaskStore {
pub fn memory() -> Self {
Self {
inner: Arc::new(Mutex::new(Inner::default())),
persist_dir: None,
}
}
pub fn with_persist(dir: impl AsRef<Path>) -> Result<Self, StoreError> {
let dir = dir.as_ref().to_path_buf();
std::fs::create_dir_all(&dir).map_err(|e| StoreError::Io(e.to_string()))?;
let store = Self {
inner: Arc::new(Mutex::new(Inner::default())),
persist_dir: Some(dir.clone()),
};
store.load()?;
Ok(store)
}
fn now() -> String {
let ms = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_millis();
format!("{ms}")
}
pub fn payload_hash(payload: &Value) -> String {
let bytes = serde_json::to_vec(payload).unwrap_or_default();
format!("{:x}", Sha256::digest(&bytes))
}
/// Admit or return existing task for the same idempotency key.
/// Same key + different payload hash → Conflict (409).
pub fn admit(
&self,
task_id: impl Into<String>,
run_id: impl Into<String>,
workspace_id: impl Into<String>,
session_id: Option<String>,
idempotency_key: impl Into<String>,
payload: &Value,
) -> Result<TaskRecord, StoreError> {
let task_id = task_id.into();
let idempotency_key = idempotency_key.into();
let payload_hash = Self::payload_hash(payload);
let mut g = self.inner.lock().unwrap();
if let Some(existing_id) = g.idem.get(&idempotency_key).cloned() {
let existing = g.tasks.get(&existing_id).cloned().ok_or(StoreError::NotFound)?;
if existing.payload_hash != payload_hash {
return Err(StoreError::Conflict(
"idempotency key reuse with different payload".into(),
));
}
return Ok(existing);
}
let now = Self::now();
let rec = TaskRecord {
task_id: task_id.clone(),
run_id: run_id.into(),
attempt_id: format!("attempt_{}", task_id),
workspace_id: workspace_id.into(),
session_id,
state: TaskState::Accepted,
revision: 1,
idempotency_key: idempotency_key.clone(),
payload_hash,
lease_id: None,
lease_epoch: None,
created_at: now.clone(),
updated_at: now,
cancel_requested_at: None,
result: None,
error: None,
};
g.idem.insert(idempotency_key, task_id.clone());
g.tasks.insert(task_id, rec.clone());
g.events.insert(rec.task_id.clone(), Vec::new());
drop(g);
self.persist_task(&rec)?;
Ok(rec)
}
pub fn get(&self, task_id: &str) -> Result<TaskRecord, StoreError> {
self.inner
.lock()
.unwrap()
.tasks
.get(task_id)
.cloned()
.ok_or(StoreError::NotFound)
}
/// CAS state transition. `expected_revision` must match.
pub fn cas_state(
&self,
task_id: &str,
expected_revision: u64,
next: TaskState,
patch: impl FnOnce(&mut TaskRecord),
) -> Result<TaskRecord, StoreError> {
let mut g = self.inner.lock().unwrap();
let rec = g.tasks.get_mut(task_id).ok_or(StoreError::NotFound)?;
if rec.revision != expected_revision {
return Err(StoreError::Conflict(format!(
"revision {} != {}",
rec.revision, expected_revision
)));
}
if rec.state.is_terminal() && next != rec.state {
return Err(StoreError::AlreadyTerminal(rec.state));
}
rec.state = next;
rec.revision += 1;
rec.updated_at = Self::now();
patch(rec);
let out = rec.clone();
drop(g);
self.persist_task(&out)?;
Ok(out)
}
pub fn request_cancel(&self, task_id: &str) -> Result<TaskRecord, StoreError> {
let mut g = self.inner.lock().unwrap();
let rec = g.tasks.get_mut(task_id).ok_or(StoreError::NotFound)?;
if rec.state.is_terminal() {
return Err(StoreError::AlreadyTerminal(rec.state));
}
let rev = rec.revision;
rec.state = TaskState::CancelRequested;
rec.revision = rev + 1;
rec.cancel_requested_at = Some(Self::now());
rec.updated_at = Self::now();
let out = rec.clone();
drop(g);
self.persist_task(&out)?;
Ok(out)
}
pub fn append_event(
&self,
task_id: &str,
run_id: &str,
attempt_id: &str,
event_type: &str,
payload: Value,
) -> Result<RuntimeEventV2, StoreError> {
let mut g = self.inner.lock().unwrap();
if !g.tasks.contains_key(task_id) {
return Err(StoreError::NotFound);
}
let list = g.events.entry(task_id.to_string()).or_default();
let seq = list.last().map(|e| e.seq + 1).unwrap_or(1);
let ev = RuntimeEventV2::new(task_id, run_id, attempt_id, seq, event_type, payload);
list.push(ev.clone());
drop(g);
self.persist_event(&ev)?;
Ok(ev)
}
/// Events with seq > after_seq (Last-Event-ID resume).
pub fn events_after(
&self,
task_id: &str,
after_seq: u64,
) -> Result<Vec<RuntimeEventV2>, StoreError> {
let g = self.inner.lock().unwrap();
let list = g.events.get(task_id).ok_or(StoreError::NotFound)?;
Ok(list.iter().filter(|e| e.seq > after_seq).cloned().collect())
}
pub fn next_lease_epoch(&self) -> u64 {
let mut g = self.inner.lock().unwrap();
g.next_epoch += 1;
g.next_epoch
}
fn persist_task(&self, rec: &TaskRecord) -> Result<(), StoreError> {
let Some(dir) = &self.persist_dir else {
return Ok(());
};
let path = dir.join("tasks").join(format!("{}.json", rec.task_id));
std::fs::create_dir_all(path.parent().unwrap()).map_err(|e| StoreError::Io(e.to_string()))?;
let tmp = path.with_extension("json.tmp");
let body = serde_json::to_vec_pretty(rec).map_err(|e| StoreError::Io(e.to_string()))?;
std::fs::write(&tmp, body).map_err(|e| StoreError::Io(e.to_string()))?;
std::fs::rename(&tmp, &path).map_err(|e| StoreError::Io(e.to_string()))?;
Ok(())
}
fn persist_event(&self, ev: &RuntimeEventV2) -> Result<(), StoreError> {
let Some(dir) = &self.persist_dir else {
return Ok(());
};
let path = dir.join("events").join(format!("{}.ndjson", ev.task_id));
std::fs::create_dir_all(path.parent().unwrap()).map_err(|e| StoreError::Io(e.to_string()))?;
use std::io::Write;
let mut f = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(&path)
.map_err(|e| StoreError::Io(e.to_string()))?;
let line = serde_json::to_string(ev).map_err(|e| StoreError::Io(e.to_string()))?;
writeln!(f, "{line}").map_err(|e| StoreError::Io(e.to_string()))?;
Ok(())
}
fn load(&self) -> Result<(), StoreError> {
let Some(dir) = &self.persist_dir else {
return Ok(());
};
let tasks_dir = dir.join("tasks");
if !tasks_dir.is_dir() {
return Ok(());
}
let mut g = self.inner.lock().unwrap();
for ent in std::fs::read_dir(&tasks_dir).map_err(|e| StoreError::Io(e.to_string()))? {
let ent = ent.map_err(|e| StoreError::Io(e.to_string()))?;
if ent.path().extension().and_then(|e| e.to_str()) != Some("json") {
continue;
}
let body =
std::fs::read_to_string(ent.path()).map_err(|e| StoreError::Io(e.to_string()))?;
let rec: TaskRecord =
serde_json::from_str(&body).map_err(|e| StoreError::Io(e.to_string()))?;
g.idem
.insert(rec.idempotency_key.clone(), rec.task_id.clone());
g.events.entry(rec.task_id.clone()).or_default();
g.tasks.insert(rec.task_id.clone(), rec);
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn idempotent_admit_and_conflict() {
let store = TaskStore::memory();
let p = json!({"a": 1});
let a = store
.admit("t1", "r1", "ws", None, "key-1", &p)
.unwrap();
let b = store
.admit("t2", "r2", "ws", None, "key-1", &p)
.unwrap();
assert_eq!(a.task_id, b.task_id);
let err = store
.admit("t3", "r3", "ws", None, "key-1", &json!({"a": 2}))
.unwrap_err();
assert!(matches!(err, StoreError::Conflict(_)));
}
#[test]
fn cancel_cas_terminal() {
let store = TaskStore::memory();
let t = store
.admit("t1", "r1", "ws", None, "k", &json!({}))
.unwrap();
store
.cas_state(&t.task_id, t.revision, TaskState::Succeeded, |r| {
r.result = Some(json!("ok"));
})
.unwrap();
let err = store.request_cancel(&t.task_id).unwrap_err();
assert!(matches!(err, StoreError::AlreadyTerminal(TaskState::Succeeded)));
}
#[test]
fn event_seq_monotonic_and_resume() {
let store = TaskStore::memory();
let t = store
.admit("t1", "r1", "ws", None, "k", &json!({}))
.unwrap();
let e1 = store
.append_event(&t.task_id, &t.run_id, &t.attempt_id, "message.delta", json!({"n":1}))
.unwrap();
let e2 = store
.append_event(&t.task_id, &t.run_id, &t.attempt_id, "message.delta", json!({"n":2}))
.unwrap();
assert_eq!(e1.seq, 1);
assert_eq!(e2.seq, 2);
let after = store.events_after(&t.task_id, 1).unwrap();
assert_eq!(after.len(), 1);
assert_eq!(after[0].seq, 2);
}
}
+259
View File
@@ -0,0 +1,259 @@
//! One-shot Marqdo worker process handles (M-R2).
use std::path::{Path, PathBuf};
use std::process::{Child, Command, Stdio};
use std::sync::atomic::{AtomicU64, Ordering};
use serde_json::{json, Value};
use crate::runtime::lease::{Lease, WorkerState};
use crate::runtime::protocol::{InvokeRequestV2, PROTOCOL_VERSION};
static WORKER_SEQ: AtomicU64 = AtomicU64::new(1);
pub struct WorkerHandle {
pub id: String,
pub state: WorkerState,
pub lease: Option<Lease>,
child: Option<Child>,
staging: PathBuf,
/// Where the child writes `--emit-result` JSON.
pub emit_result: Option<PathBuf>,
}
impl WorkerHandle {
pub fn staging(&self) -> &Path {
&self.staging
}
pub fn take_child(&mut self) -> Option<Child> {
self.child.take()
}
pub fn kill_process_group(&mut self) {
if let Some(mut child) = self.child.take() {
let _ = child.kill();
let _ = child.wait();
}
self.state = WorkerState::Exited;
}
pub fn try_wait(&mut self) -> Result<Option<std::process::ExitStatus>, String> {
let Some(child) = self.child.as_mut() else {
return Ok(None);
};
child.try_wait().map_err(|e| format!("worker.wait: {e}"))
}
pub fn wait(&mut self) -> Result<std::process::ExitStatus, String> {
let Some(mut child) = self.child.take() else {
return Err("worker.no_child".into());
};
let st = child.wait().map_err(|e| format!("worker.wait: {e}"))?;
self.state = WorkerState::Exited;
Ok(st)
}
}
impl Drop for WorkerHandle {
fn drop(&mut self) {
self.kill_process_group();
let _ = std::fs::remove_dir_all(&self.staging);
}
}
pub struct WorkerPool {
staging_root: PathBuf,
marqdo_bin: PathBuf,
ready: Vec<WorkerHandle>,
target_ready: usize,
}
impl WorkerPool {
pub fn new(staging_root: impl AsRef<Path>, marqdo_bin: impl AsRef<Path>, target_ready: usize) -> Self {
let staging_root = staging_root.as_ref().to_path_buf();
let _ = std::fs::create_dir_all(&staging_root);
Self {
staging_root,
marqdo_bin: marqdo_bin.as_ref().to_path_buf(),
ready: Vec::new(),
target_ready,
}
}
pub fn spawn_ready(&mut self) -> Result<WorkerHandle, String> {
let seq = WORKER_SEQ.fetch_add(1, Ordering::Relaxed);
let id = format!("worker_{seq}");
let staging = self.staging_root.join(&id);
std::fs::create_dir_all(&staging).map_err(|e| e.to_string())?;
if !self.marqdo_bin.exists() {
return Err(format!("worker.binary_missing: {}", self.marqdo_bin.display()));
}
Ok(WorkerHandle {
id,
state: WorkerState::Ready,
lease: None,
child: None,
staging,
emit_result: None,
})
}
pub fn warm(&mut self) -> Result<(), String> {
while self.ready.len() < self.target_ready {
let w = self.spawn_ready()?;
self.ready.push(w);
}
Ok(())
}
pub fn acquire(&mut self) -> Result<WorkerHandle, String> {
if self.ready.is_empty() {
self.warm()?;
}
self.ready
.pop()
.ok_or_else(|| "worker.pool_empty".to_string())
}
pub fn replenish_async_hint(&self) -> usize {
self.target_ready.saturating_sub(self.ready.len())
}
/// Lease a worker: spawn `marqdo run --call-lib-path …` with request-scoped env.
/// `emit_result` must be outside staging so Drop does not erase the parent's read path.
pub fn lease_and_start(
&self,
mut worker: WorkerHandle,
lease: Lease,
req: &InvokeRequestV2,
cancel_token: &Path,
event_sink: &Path,
fs_root: &Path,
args_json: Option<&Path>,
emit_result: &Path,
) -> Result<WorkerHandle, String> {
req.validate()?;
let entry = PathBuf::from(&req.entry.module);
if !entry.is_file() {
return Err(format!(
"worker.entry_missing: {} (need absolute site entry .mq.md)",
entry.display()
));
}
if let Some(parent) = emit_result.parent() {
std::fs::create_dir_all(parent).map_err(|e| e.to_string())?;
}
let emit = emit_result.to_path_buf();
let mut cmd = Command::new(&self.marqdo_bin);
cmd.arg("run")
.arg(&entry)
.arg("--call-lib-path")
.arg(&req.entry.function)
.arg("--fs-root")
.arg(fs_root)
.arg("--emit-result")
.arg(&emit)
// Worker cwd = site root so relative imports resolve; staging only for lease bookkeeping.
.current_dir(fs_root)
.env("MARQDO_RUN_ID", &req.run_id)
.env("MARQDO_TASK_ID", &req.task_id)
.env("MARQDO_ATTEMPT_ID", &req.attempt_id)
.env("MARQDO_LEASE_ID", &lease.id.0)
.env("MARQDO_LEASE_EPOCH", lease.epoch.to_string())
.env("MARQDO_CANCEL_TOKEN", cancel_token)
.env("MARQDO_EVENT_FORWARD", event_sink)
.env("MARQDO_WORKSPACE_ID", &req.context.workspace_id)
.env("MARQDO_PROTOCOL_VERSION", PROTOCOL_VERSION.to_string())
.env("MARQDO_FS_ROOT", fs_root)
.stdin(Stdio::null())
// Avoid pipe-buffer deadlock: nobody drains worker stdout/stderr.
.stdout(Stdio::null())
.stderr(Stdio::null());
if let Some(args) = args_json {
cmd.arg("--args-json").arg(args);
} else if let Some(r) = &req.args_ref {
if let Some(path) = r.strip_prefix("file:") {
cmd.arg("--args-json").arg(path);
}
}
if let Some(sel) = &req.context.selection_ref {
cmd.env("MARQDO_SELECTION_REF", sel);
}
if let Some(snap) = &req.context.snapshot_id {
cmd.env("MARQDO_SNAPSHOT_ID", snap);
}
if let Some(cap) = &req.context.capabilities_ref {
cmd.env("MARQDO_CAPABILITIES_REF", cap);
}
let child = cmd.spawn().map_err(|e| format!("worker.spawn: {e}"))?;
worker.child = Some(child);
worker.lease = Some(lease);
worker.emit_result = Some(emit);
worker.state = WorkerState::Running;
Ok(worker)
}
}
pub fn handshake_ok(report: &Value) -> Result<(), String> {
let ver = report
.get("protocol_version")
.and_then(|v| v.as_u64())
.ok_or("handshake.protocol_version")?;
if ver != PROTOCOL_VERSION as u64 {
return Err(format!("handshake.version_mismatch: {ver}"));
}
Ok(())
}
pub fn fake_handshake() -> Value {
json!({
"cli": true,
"protocol_version": PROTOCOL_VERSION,
"capabilities": ["runtime.isolated_execution"],
"build": env!("CARGO_PKG_VERSION"),
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn handshake_version_gate() {
assert!(handshake_ok(&fake_handshake()).is_ok());
assert!(handshake_ok(&json!({"protocol_version": 1})).is_err());
}
#[test]
fn pool_warm_requires_binary() {
let dir = std::env::temp_dir().join(format!("mq-pool-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let mut pool = WorkerPool::new(&dir, "/no/such/marqdo-bin", 1);
assert!(pool.warm().is_err());
let self_bin = std::env::current_exe().unwrap();
let mut pool = WorkerPool::new(&dir, &self_bin, 1);
pool.warm().unwrap();
assert_eq!(pool.ready.len(), 1);
let w = pool.acquire().unwrap();
assert_eq!(w.state, WorkerState::Ready);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn kill_clears_child() {
let dir = std::env::temp_dir().join(format!("mq-wh-{}", std::process::id()));
let _ = std::fs::create_dir_all(&dir);
let mut w = WorkerHandle {
id: "w".into(),
state: WorkerState::Running,
lease: None,
child: None,
staging: dir.clone(),
emit_result: None,
};
w.kill_process_group();
assert_eq!(w.state, WorkerState::Exited);
let _ = std::fs::remove_dir_all(&dir);
}
}
+47 -27
View File
@@ -240,32 +240,24 @@ pub struct Compilation {
/// side-effect free: it never loads plugins, evaluates expressions or fetches
/// sources.
pub fn compile_project(options: &CompileOptions) -> Result<Compilation> {
let t_index = std::time::Instant::now();
let root = ProjectRoot::new(&options.project_root)?;
let index = ProjectIndex::build(root.clone())?;
let index_ms = t_index.elapsed().as_millis();
// Reuse parsed modules from ProjectIndex — same input snapshot, no second parse (M-R5b).
let mut files = BTreeMap::new();
let mut diagnostics = Vec::new();
for file in root.source_files()? {
let text = fs::read_to_string(&file)
.with_context(|| format!("read semantic source {}", file.display()))?;
match crate::load::parse_source_auto(&text) {
Ok(module) => {
let module_id = index
.modules()
.iter()
.find_map(|(id, path)| (path == &file).then(|| id.clone()));
if let Some(module_id) = module_id {
files.insert(module_id, (file, text, module));
}
}
Err(error) => {
diagnostics.push(
Diagnostic::new(Some(&file), Span::new(1, 1), error.to_string())
.with_code("parse.semantic")
.with_severity(Severity::Error),
);
}
}
for (module_id, parsed) in index.parsed_modules() {
files.insert(
module_id.clone(),
(
parsed.path.clone(),
parsed.text.clone(),
parsed.module.clone(),
),
);
}
let _ = index_ms; // ProjectIndex build duration (M-R5b span)
// Wire import_modules from already-parsed project files so BoundCall path
// resolve shares `resolve_for_exec` (no plugin load / no re-parse).
@@ -288,12 +280,8 @@ pub fn compile_project(options: &CompileOptions) -> Result<Compilation> {
collect_functions(&module.functions, &mut Vec::new(), &mut by_path);
let narrative_links = crate::parse::scan_narrative_links(text);
let module_prov = super::provenance::parse_module_provenance(text);
let module_symbols: Vec<&SymbolDecl> = index
.symbols()
.iter()
.filter(|s| &s.module == module_id)
.collect();
for symbol in index.symbols().iter().filter(|s| &s.module == module_id) {
let module_symbols = index.symbols_in_module(module_id);
for symbol in index.symbols_in_module(module_id) {
let function = by_path.get(&symbol.declaration_path).expect("symbol collected from module");
let owned_links: Vec<_> = narrative_links
.iter()
@@ -1315,12 +1303,44 @@ pub fn write_semantic_projections(out_dir: &Path, compilation: &Compilation) ->
fs::create_dir_all(gen_dir.join("verification"))?;
write_generation_tree(&gen_dir, compilation)?;
// Publish journal + CAS (M-R6): record expected CURRENT before switching.
let expected = fs::read_to_string(out_dir.join("CURRENT"))
.ok()
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty());
let source_digest = {
let mut h = Sha256::new();
for a in &compilation.semantics.artifacts {
h.update(a.hashes.artifact.0.as_bytes());
}
format!("sha256:{:x}", h.finalize())
};
// Staging already written under gen_dir; journal tracks CURRENT switch.
let mut journal = crate::runtime::PublishJournal::begin(
out_dir,
&compilation.semantics.project_id,
expected,
&build_id,
&source_digest,
)
.map_err(|e| anyhow::anyhow!("publish journal: {e}"))?;
// Point journal staging at the generation we just wrote (already complete).
journal.staging_dir = gen_dir.clone();
journal.generation_dir = gen_dir.clone();
journal
.mark(out_dir, crate::runtime::publish::PublishPhase::Validated)
.map_err(|e| anyhow::anyhow!("publish journal mark: {e}"))?;
// Atomic generation pointer: only publish CURRENT after the tree is complete.
let current_tmp = out_dir.join("CURRENT.tmp");
fs::write(&current_tmp, format!("{build_id}\n"))
.with_context(|| format!("write {}", current_tmp.display()))?;
fs::rename(&current_tmp, out_dir.join("CURRENT"))
.with_context(|| format!("publish CURRENT under {}", out_dir.display()))?;
journal
.mark(out_dir, crate::runtime::publish::PublishPhase::CurrentSwitched)
.map_err(|e| anyhow::anyhow!("publish journal current: {e}"))?;
let _ = fs::remove_file(out_dir.join("publish_journal.json"));
// Compatibility root: stage a full mirror, then swap names (best-effort for
// legacy readers that do not follow CURRENT).
+51
View File
@@ -92,15 +92,29 @@ pub struct SymbolDecl {
pub line: u32,
}
/// Parsed module retained so `compile_project` does not re-parse (M-R5b).
#[derive(Debug, Clone)]
pub struct ParsedModule {
pub path: PathBuf,
pub text: String,
pub module: crate::ast::Module,
}
/// Deterministic read-only view of all declarations in one project.
#[derive(Debug, Clone)]
pub struct ProjectIndex {
project_id: String,
root: ProjectRoot,
modules: BTreeMap<ModuleId, PathBuf>,
/// path → module id (avoids linear scan per file in compile_project).
path_to_module: BTreeMap<PathBuf, ModuleId>,
symbols: Vec<SymbolDecl>,
by_id: BTreeMap<SymbolId, usize>,
by_name: BTreeMap<String, Vec<usize>>,
/// module → symbol indexes (avoids filtering the full symbol table per module).
symbols_by_module: BTreeMap<ModuleId, Vec<usize>>,
/// Shared parsed AST/text for the same input snapshot.
parsed: BTreeMap<ModuleId, ParsedModule>,
}
impl ProjectIndex {
@@ -108,7 +122,9 @@ impl ProjectIndex {
let project_id = project_id(root.path());
let files = root.source_files()?;
let mut modules = BTreeMap::new();
let mut path_to_module = BTreeMap::new();
let mut symbols = Vec::new();
let mut parsed = BTreeMap::new();
for file in files {
let source = fs::read_to_string(&file)
@@ -121,6 +137,7 @@ impl ProjectIndex {
file.display()
);
}
path_to_module.insert(file.clone(), module_id.clone());
let module = crate::load::parse_source_auto(&source)
.with_context(|| format!("parse source {}", file.display()))?;
for function in &module.functions {
@@ -134,28 +151,47 @@ impl ProjectIndex {
&mut symbols,
)?;
}
parsed.insert(
module_id.clone(),
ParsedModule {
path: file,
text: source,
module,
},
);
}
symbols.sort_by(|a, b| a.id.cmp(&b.id));
let mut by_id = BTreeMap::new();
let mut by_name: BTreeMap<String, Vec<usize>> = BTreeMap::new();
let mut symbols_by_module: BTreeMap<ModuleId, Vec<usize>> = BTreeMap::new();
for (index, symbol) in symbols.iter().enumerate() {
if by_id.insert(symbol.id.clone(), index).is_some() {
bail!("duplicate symbol id `{}`", symbol.id);
}
by_name.entry(symbol.name.clone()).or_default().push(index);
symbols_by_module
.entry(symbol.module.clone())
.or_default()
.push(index);
}
for indexes in by_name.values_mut() {
indexes.sort_by(|a, b| symbols[*a].id.cmp(&symbols[*b].id));
}
for indexes in symbols_by_module.values_mut() {
indexes.sort_by(|a, b| symbols[*a].id.cmp(&symbols[*b].id));
}
Ok(Self {
project_id,
root,
modules,
path_to_module,
symbols,
by_id,
by_name,
symbols_by_module,
parsed,
})
}
@@ -171,6 +207,21 @@ impl ProjectIndex {
&self.modules
}
pub fn module_id_for_path(&self, path: &Path) -> Option<&ModuleId> {
self.path_to_module.get(path)
}
pub fn symbols_in_module(&self, module_id: &ModuleId) -> Vec<&SymbolDecl> {
self.symbols_by_module
.get(module_id)
.map(|idxs| idxs.iter().map(|i| &self.symbols[*i]).collect())
.unwrap_or_default()
}
pub fn parsed_modules(&self) -> &BTreeMap<ModuleId, ParsedModule> {
&self.parsed
}
pub fn symbols(&self) -> &[SymbolDecl] {
&self.symbols
}
+359 -86
View File
@@ -1,24 +1,87 @@
//! Project snapshot analysis cache (M05 / F14): keyed by project_root + source digests.
//! Concurrent same-snapshot requests coalesce: waiters reuse the leader's result.
//! AnalysisCache v2 (M-R5): SnapshotKey → Arc<Compilation> with single-flight.
//! Cache hits must reuse the compilation result, not merely a digest ticket.
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::collections::{HashMap, VecDeque};
use std::path::Path;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Condvar, Mutex, OnceLock};
use std::time::Instant;
use std::sync::{Arc, Condvar, Mutex, OnceLock};
use std::time::{Duration, Instant};
use crate::sema::Compilation;
/// Full-project snapshot identity (design §8.1). Built from the compiler's
/// source enumeration — never a truncated `take(200)` directory listing.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct SnapshotKey {
pub project_identity: String,
pub source_digest: String,
pub compiler_version: String,
}
impl SnapshotKey {
pub fn compute(project_root: &Path) -> Result<Self, String> {
let root = crate::project::ProjectRoot::new(project_root)
.map_err(|e| format!("snapshot.project: {e:#}"))?;
let files = root
.source_files()
.map_err(|e| format!("snapshot.enumerate: {e:#}"))?;
let mut h = Sha256::new();
h.update(root.path().to_string_lossy().as_bytes());
for p in &files {
let rel = p
.strip_prefix(root.path())
.unwrap_or(p)
.to_string_lossy();
h.update(rel.as_bytes());
let meta =
std::fs::metadata(p).map_err(|e| format!("snapshot.meta {}: {e}", p.display()))?;
h.update(meta.len().to_le_bytes());
let bytes =
std::fs::read(p).map_err(|e| format!("snapshot.read {}: {e}", p.display()))?;
h.update(&bytes);
}
let identity = root
.path()
.canonicalize()
.unwrap_or_else(|_| root.path().to_path_buf())
.to_string_lossy()
.to_string();
Ok(Self {
project_identity: identity,
source_digest: format!("{:x}", h.finalize()),
compiler_version: env!("CARGO_PKG_VERSION").to_string(),
})
}
pub fn cache_key(&self) -> String {
format!(
"{}::{}::{}",
self.project_identity, self.source_digest, self.compiler_version
)
}
}
#[derive(Clone)]
struct CacheEntry {
digest_key: String,
#[allow(dead_code)]
key: SnapshotKey,
compilation: Arc<Compilation>,
bytes: usize,
#[allow(dead_code)]
compiled_at: Instant,
}
struct InflightSlot {
waiters: usize,
result: Option<Result<Arc<Compilation>, String>>,
}
struct CacheState {
order: VecDeque<String>,
entries: HashMap<String, CacheEntry>,
/// project_key → refcount of leaders currently analyzing.
inflight: HashMap<String, usize>,
inflight: HashMap<String, InflightSlot>,
bytes_used: usize,
budget_bytes: usize,
}
fn cache() -> &'static (Mutex<CacheState>, Condvar) {
@@ -26,44 +89,217 @@ fn cache() -> &'static (Mutex<CacheState>, Condvar) {
C.get_or_init(|| {
(
Mutex::new(CacheState {
order: VecDeque::new(),
entries: HashMap::new(),
inflight: HashMap::new(),
bytes_used: 0,
budget_bytes: default_budget_bytes(),
}),
Condvar::new(),
)
})
}
fn default_budget_bytes() -> usize {
512 * 1024 * 1024
}
static HITS: AtomicU64 = AtomicU64::new(0);
static MISSES: AtomicU64 = AtomicU64::new(0);
static MERGES: AtomicU64 = AtomicU64::new(0);
static COMPILE_INVOCATIONS: AtomicU64 = AtomicU64::new(0);
static EVICTIONS: AtomicU64 = AtomicU64::new(0);
static RESULT_HITS: AtomicU64 = AtomicU64::new(0);
pub fn project_source_digest(project_root: &Path) -> String {
let mut h = Sha256::new();
h.update(project_root.to_string_lossy().as_bytes());
let mut paths: Vec<PathBuf> = Vec::new();
if let Ok(rd) = std::fs::read_dir(project_root) {
for ent in rd.flatten().take(200) {
let p = ent.path();
if p.extension().and_then(|e| e.to_str()) == Some("md")
|| p
.file_name()
.and_then(|n| n.to_str())
.map(|n| n.ends_with(".mq.md"))
.unwrap_or(false)
{
paths.push(p);
fn key_compiles() -> &'static Mutex<HashMap<String, u64>> {
static K: OnceLock<Mutex<HashMap<String, u64>>> = OnceLock::new();
K.get_or_init(|| Mutex::new(HashMap::new()))
}
fn estimate_bytes(c: &Compilation) -> usize {
let arts = c.semantics.artifacts.len() * 512;
let refs = c.semantics.refs.len() * 128;
let steps = c.semantics.steps.len() * 128;
let diags = c.diagnostics.len() * 64;
4096 + arts + refs + steps + diags
}
enum Admit {
Hit(Arc<Compilation>),
Wait,
Lead,
}
fn admit(key: &str) -> Admit {
let (lock, _cv) = cache();
let mut g = lock.lock().unwrap();
if let Some(e) = g.entries.get(key) {
let arc = Arc::clone(&e.compilation);
if let Some(pos) = g.order.iter().position(|k| k == key) {
g.order.remove(pos);
}
g.order.push_back(key.to_string());
HITS.fetch_add(1, Ordering::Relaxed);
RESULT_HITS.fetch_add(1, Ordering::Relaxed);
return Admit::Hit(arc);
}
if g.inflight.contains_key(key) {
if let Some(slot) = g.inflight.get_mut(key) {
slot.waiters += 1;
}
MERGES.fetch_add(1, Ordering::Relaxed);
return Admit::Wait;
}
g.inflight.insert(
key.to_string(),
InflightSlot {
waiters: 0,
result: None,
},
);
MISSES.fetch_add(1, Ordering::Relaxed);
Admit::Lead
}
fn wait_for(key: &str, deadline: Instant) -> Result<Arc<Compilation>, String> {
let (lock, cv) = cache();
let mut g = lock.lock().unwrap();
loop {
if let Some(e) = g.entries.get(key) {
let arc = Arc::clone(&e.compilation);
if let Some(slot) = g.inflight.get_mut(key) {
slot.waiters = slot.waiters.saturating_sub(1);
if slot.waiters == 0 {
g.inflight.remove(key);
}
}
HITS.fetch_add(1, Ordering::Relaxed);
RESULT_HITS.fetch_add(1, Ordering::Relaxed);
return Ok(arc);
}
if let Some(slot) = g.inflight.get(key) {
if let Some(res) = &slot.result {
let out = res.clone();
let waiters = slot.waiters.saturating_sub(1);
if let Some(slot) = g.inflight.get_mut(key) {
slot.waiters = waiters;
if slot.waiters == 0 {
g.inflight.remove(key);
}
}
HITS.fetch_add(1, Ordering::Relaxed);
RESULT_HITS.fetch_add(1, Ordering::Relaxed);
return out;
}
} else {
// Leader finished without leaving a result — retry admit.
return Err("analysis.retry".into());
}
let remaining = deadline.saturating_duration_since(Instant::now());
if remaining.is_zero() {
if let Some(slot) = g.inflight.get_mut(key) {
slot.waiters = slot.waiters.saturating_sub(1);
if slot.waiters == 0 && slot.result.is_none() {
g.inflight.remove(key);
}
}
return Err("analysis.wait_deadline".into());
}
let (guard, timeout) = cv.wait_timeout(g, remaining).unwrap();
g = guard;
if timeout.timed_out() {
if let Some(slot) = g.inflight.get_mut(key) {
slot.waiters = slot.waiters.saturating_sub(1);
if slot.waiters == 0 && slot.result.is_none() {
g.inflight.remove(key);
}
}
return Err("analysis.wait_deadline".into());
}
}
}
fn finish_leader(key: &str, snap: SnapshotKey, compiled: Result<Arc<Compilation>, String>) {
if compiled.is_ok() {
let mut kc = key_compiles().lock().unwrap();
*kc.entry(key.to_string()).or_insert(0) += 1;
}
let (lock, cv) = cache();
let mut g = lock.lock().unwrap();
if let Ok(ref arc) = compiled {
insert_entry(&mut g, snap, Arc::clone(arc));
}
if let Some(slot) = g.inflight.get_mut(key) {
slot.result = Some(compiled);
if slot.waiters == 0 {
g.inflight.remove(key);
}
}
cv.notify_all();
}
/// Compile once per SnapshotKey; followers await the same Arc<Compilation>.
pub fn get_or_compile(project_root: &Path) -> Result<Arc<Compilation>, String> {
let snap = SnapshotKey::compute(project_root)?;
let key = snap.cache_key();
let deadline = Instant::now() + Duration::from_secs(300);
loop {
match admit(&key) {
Admit::Hit(arc) => return Ok(arc),
Admit::Wait => match wait_for(&key, deadline) {
Err(e) if e == "analysis.retry" => continue,
other => return other,
},
Admit::Lead => {
let compiled = compile_leader(project_root);
COMPILE_INVOCATIONS.fetch_add(1, Ordering::Relaxed);
let out = compiled.clone();
finish_leader(&key, snap, compiled);
return out;
}
}
}
paths.sort();
for p in paths {
h.update(p.to_string_lossy().as_bytes());
if let Ok(bytes) = std::fs::read(&p) {
h.update(&bytes);
}
fn compile_leader(project_root: &Path) -> Result<Arc<Compilation>, String> {
let options = crate::sema::CompileOptions::for_path(project_root)
.map_err(|e| format!("view.compile_failed: {e:#}"))?;
let c = crate::sema::compile_project(&options)
.map_err(|e| format!("view.compile_failed: {e:#}"))?;
Ok(Arc::new(c))
}
fn insert_entry(g: &mut CacheState, key: SnapshotKey, compilation: Arc<Compilation>) {
let ck = key.cache_key();
let bytes = estimate_bytes(&compilation);
while g.bytes_used + bytes > g.budget_bytes && !g.order.is_empty() {
if let Some(old) = g.order.pop_front() {
if let Some(e) = g.entries.remove(&old) {
g.bytes_used = g.bytes_used.saturating_sub(e.bytes);
EVICTIONS.fetch_add(1, Ordering::Relaxed);
}
}
}
format!("{:x}", h.finalize())
if bytes > g.budget_bytes || g.entries.contains_key(&ck) {
return;
}
g.bytes_used += bytes;
g.order.push_back(ck.clone());
g.entries.insert(
ck,
CacheEntry {
key,
compilation,
bytes,
compiled_at: Instant::now(),
},
);
}
pub fn project_source_digest(project_root: &Path) -> String {
SnapshotKey::compute(project_root)
.map(|k| k.source_digest)
.unwrap_or_else(|_| "digest-error".into())
}
fn project_key(project_root: &Path) -> String {
@@ -74,74 +310,38 @@ fn project_key(project_root: &Path) -> String {
.to_string()
}
/// Ticket for a leader analysis; must call [`finish_analysis`] (or drop via finish).
pub struct AnalysisTicket {
#[allow(dead_code)]
key: String,
#[allow(dead_code)]
digest: String,
/// true ⇒ this caller is the leader and must run compile then finish.
pub is_leader: bool,
}
/// Begin analysis: cache hit / wait for in-flight leader / become leader.
pub fn begin_analysis(project_root: &Path) -> AnalysisTicket {
let key = project_key(project_root);
let digest = project_source_digest(project_root);
let (lock, cv) = cache();
let mut g = lock.lock().unwrap();
loop {
if let Some(e) = g.entries.get(&key) {
if e.digest_key == digest {
HITS.fetch_add(1, Ordering::Relaxed);
return AnalysisTicket {
key,
digest,
is_leader: false,
};
}
}
if g.inflight.contains_key(&key) {
MERGES.fetch_add(1, Ordering::Relaxed);
g = cv.wait(g).unwrap();
continue;
}
*g.inflight.entry(key.clone()).or_insert(0) += 1;
MISSES.fetch_add(1, Ordering::Relaxed);
return AnalysisTicket {
key,
digest,
match get_or_compile(project_root) {
Ok(_) => AnalysisTicket {
key: project_key(project_root),
digest: project_source_digest(project_root),
is_leader: false,
},
Err(_) => AnalysisTicket {
key: project_key(project_root),
digest: String::new(),
is_leader: true,
};
}
}
pub fn finish_analysis(ticket: AnalysisTicket) {
if !ticket.is_leader {
return;
}
let (lock, cv) = cache();
let mut g = lock.lock().unwrap();
g.entries.insert(
ticket.key.clone(),
CacheEntry {
digest_key: ticket.digest,
compiled_at: Instant::now(),
},
);
if let Some(n) = g.inflight.get_mut(&ticket.key) {
*n = n.saturating_sub(1);
if *n == 0 {
g.inflight.remove(&ticket.key);
}
}
cv.notify_all();
}
/// Compatibility: note without holding across compile (prefer begin/finish).
pub fn finish_analysis(_ticket: AnalysisTicket) {}
pub fn note_analysis(project_root: &Path) -> bool {
let t = begin_analysis(project_root);
let hit = !t.is_leader;
finish_analysis(t);
hit
// True iff this SnapshotKey was already cached (result hit), independent of
// other projects compiled concurrently in the same process.
let before = compile_count_for(project_root);
let _ = get_or_compile(project_root);
let after = compile_count_for(project_root);
after == before && before > 0
}
pub fn cache_keys_isolated(a: &Path, b: &Path) -> bool {
@@ -156,13 +356,86 @@ pub fn cache_stats() -> (u64, u64, u64) {
)
}
pub fn compile_invocations() -> u64 {
COMPILE_INVOCATIONS.load(Ordering::Relaxed)
}
/// Successful compile count for one SnapshotKey (M-R5 / MR-06).
pub fn compile_count_for(project_root: &Path) -> u64 {
let Ok(snap) = SnapshotKey::compute(project_root) else {
return 0;
};
key_compiles()
.lock()
.unwrap()
.get(&snap.cache_key())
.copied()
.unwrap_or(0)
}
pub fn result_hits() -> u64 {
RESULT_HITS.load(Ordering::Relaxed)
}
pub fn eviction_count() -> u64 {
EVICTIONS.load(Ordering::Relaxed)
}
pub fn reset_cache_for_tests() {
let (lock, cv) = cache();
let mut g = lock.lock().unwrap();
g.entries.clear();
g.order.clear();
g.inflight.clear();
g.bytes_used = 0;
HITS.store(0, Ordering::Relaxed);
MISSES.store(0, Ordering::Relaxed);
MERGES.store(0, Ordering::Relaxed);
COMPILE_INVOCATIONS.store(0, Ordering::Relaxed);
EVICTIONS.store(0, Ordering::Relaxed);
RESULT_HITS.store(0, Ordering::Relaxed);
key_compiles().lock().unwrap().clear();
cv.notify_all();
}
/// Serialize tests that call [`reset_cache_for_tests`] so parallel suites cannot
/// clear another test's SnapshotKey mid-flight. Recovers from poison so one
/// failing cache test does not brick the rest of the suite.
pub fn test_serial_guard() -> std::sync::MutexGuard<'static, ()> {
static G: OnceLock<Mutex<()>> = OnceLock::new();
let m = G.get_or_init(|| Mutex::new(()));
m.lock().unwrap_or_else(|p| p.into_inner())
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{Arc, Barrier};
use std::thread;
#[test]
fn same_snapshot_compiles_once_under_load() {
let _g = test_serial_guard();
reset_cache_for_tests();
let root = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let n = 32usize;
let barrier = Arc::new(Barrier::new(n));
let mut handles = Vec::with_capacity(n);
for _ in 0..n {
let barrier = Arc::clone(&barrier);
let root = root.clone();
handles.push(thread::spawn(move || {
barrier.wait();
get_or_compile(&root).expect("compile")
}));
}
let first = handles.pop().unwrap().join().unwrap();
for h in handles {
let arc = h.join().unwrap();
assert!(Arc::ptr_eq(&first, &arc), "followers must share the same Arc");
}
assert_eq!(compile_count_for(&root), 1);
assert_eq!(compile_invocations(), 1);
}
}
+11
View File
@@ -89,13 +89,22 @@ pub fn probe_capabilities(params: &Value) -> Value {
"runtime.cancel".into(),
json!(if cancel_off { 0 } else { 1 }),
);
// Isolated web runtime (doc/design/isolated-web-runtime-development-2026-10-26.md).
// HTTP execute / sse-live defaults to the Supervisor worker pool (ADR 0009).
features.insert("runtime.isolated_execution".into(), json!(1));
// task_control_v2 / event_replay: host_query("runtime") + /_runtime/* system routes.
features.insert("runtime.task_control_v2".into(), json!(1));
features.insert("runtime.event_replay".into(), json!(1));
features.insert("runtime.read_lane".into(), json!(0));
#[cfg(feature = "view")]
{
features.insert("view.analysis_cache_v2".into(), json!(1));
features.insert("view.document".into(), json!(1));
features.insert("view.reference_resolve".into(), json!(1));
}
#[cfg(not(feature = "view"))]
{
features.insert("view.analysis_cache_v2".into(), json!(0));
features.insert("view.document".into(), json!(0));
features.insert("view.reference_resolve".into(), json!(0));
}
@@ -111,6 +120,8 @@ pub fn probe_capabilities(params: &Value) -> Value {
"ModelSelectionV1",
"ModelCallUsageV1",
"RuntimeEventV1",
"RuntimeEventV2",
"InvokeRequestV2",
"DocumentViewV1",
"KnowledgeLocatorV1"
]
+3 -14
View File
@@ -92,20 +92,9 @@ pub fn build_document_view(req: &DocumentViewRequest) -> Result<Value, Value> {
let title = title_from_source(&source, &req.document);
// Semantic projection — compile_project does not load plugins / evaluate.
// Hold analysis lock across compile so concurrent same-snapshot reads coalesce (F14).
let ticket = crate::workbench::analysis_cache::begin_analysis(&project_root);
let options = crate::sema::CompileOptions::for_path(&project_root)
.map_err(|e| err("view.compile_failed", format!("{e:#}")))?;
let compilation = match crate::sema::compile_project(&options) {
Ok(c) => {
crate::workbench::analysis_cache::finish_analysis(ticket);
c
}
Err(e) => {
crate::workbench::analysis_cache::finish_analysis(ticket);
return Err(err("view.compile_failed", format!("{e:#}")));
}
};
// M-R5: Arc<Compilation> single-flight cache; followers reuse the same result.
let compilation = crate::workbench::analysis_cache::get_or_compile(&project_root)
.map_err(|e| err("view.compile_failed", e))?;
let rel = req.document.replace('\\', "/");
let mut presentation = KnowledgePresentation::from_project(&compilation.semantics, &rel, &title);
+6 -3
View File
@@ -16,15 +16,18 @@ pub mod snapshot;
pub mod usage;
pub use analysis_cache::{
begin_analysis, cache_keys_isolated, cache_stats, finish_analysis, note_analysis,
project_source_digest, reset_cache_for_tests, AnalysisTicket,
begin_analysis, cache_keys_isolated, cache_stats, compile_count_for, compile_invocations,
finish_analysis, get_or_compile, note_analysis, project_source_digest, reset_cache_for_tests,
result_hits, test_serial_guard, AnalysisTicket, SnapshotKey,
};
pub use capabilities::probe_capabilities;
#[cfg(feature = "view")]
pub use document_view::{build_document_view, DocumentViewRequest};
pub use event_sink::{bind_event_sink, clear_event_sink, emit_event, emit_message_delta};
pub use locator::{resolve_reference, KnowledgeLocator};
pub use run_context::{cancel_requested, RunContext};
pub use run_context::{
cancel_requested, cancel_token_body_means_cancel, RequestContext, RunContext,
};
pub use selection::{parse_model_selection, ModelSelection};
pub use snapshot::{build_snapshot_manifest, version_unavailable_result};
pub use usage::{
+179 -12
View File
@@ -1,7 +1,61 @@
//! RunContextV1 — request-local run identity for selection / usage / cancel (M01).
//! RunContextV1 — request-local run identity for selection / usage / cancel (M01 / M-R1).
use serde_json::{json, Map, Value};
/// Explicit request Context handles (M-R1). Paths are opaque grants owned by the
/// control plane; callers must not pass browser-supplied absolute paths unchecked.
#[derive(Debug, Clone, Default)]
pub struct RequestContext {
pub workspace_id: Option<String>,
pub snapshot_id: Option<String>,
pub selection_ref: Option<String>,
pub capabilities_ref: Option<String>,
pub deadline_at: Option<String>,
pub lease_id: Option<String>,
pub attempt_id: Option<String>,
pub epoch: Option<u64>,
}
impl RequestContext {
pub fn from_value(v: &Value) -> Self {
Self {
workspace_id: str_opt(v, "workspace_id"),
snapshot_id: str_opt(v, "snapshot_id"),
selection_ref: str_opt(v, "selection_ref"),
capabilities_ref: str_opt(v, "capabilities_ref"),
deadline_at: str_opt(v, "deadline_at"),
lease_id: str_opt(v, "lease_id").or_else(|| {
v.get("lease")
.and_then(|l| l.get("id"))
.and_then(|x| x.as_str())
.map(|s| s.to_string())
}),
attempt_id: str_opt(v, "attempt_id"),
epoch: v
.get("epoch")
.and_then(|x| x.as_u64())
.or_else(|| {
v.get("lease")
.and_then(|l| l.get("epoch"))
.and_then(|x| x.as_u64())
}),
}
}
pub fn to_json(&self) -> Value {
json!({
"workspace_id": self.workspace_id,
"snapshot_id": self.snapshot_id,
"selection_ref": self.selection_ref,
"capabilities_ref": self.capabilities_ref,
"deadline_at": self.deadline_at,
"lease_id": self.lease_id,
"attempt_id": self.attempt_id,
"epoch": self.epoch,
})
}
}
#[derive(Debug, Clone, Default)]
pub struct RunContext {
pub run_id: String,
@@ -12,6 +66,8 @@ pub struct RunContext {
pub usage_sink: Option<String>,
pub event_sink: Option<String>,
pub cancellation_token: Option<String>,
/// Explicit Context object (M-R1); serialized alongside RunContextV1 fields.
pub context: RequestContext,
}
impl RunContext {
@@ -22,6 +78,10 @@ impl RunContext {
.filter(|s| !s.is_empty())
.ok_or_else(|| "run_context.missing:run_id".to_string())?
.to_string();
let context = v
.get("context")
.map(RequestContext::from_value)
.unwrap_or_default();
Ok(Self {
run_id,
parent_run_id: v
@@ -56,6 +116,7 @@ impl RunContext {
.get("cancellation_token")
.and_then(|x| x.as_str())
.map(|s| s.to_string()),
context,
})
}
@@ -71,10 +132,11 @@ impl RunContext {
"usage_sink": self.usage_sink,
"event_sink": self.event_sink,
"cancellation_token": self.cancellation_token,
"context": self.context.to_json(),
})
}
/// Child inherits selection/event/cancel; new run_id linked to parent.
/// Child inherits selection/event/cancel handles; new run_id linked to parent.
pub fn spawn_child(&self, child_run_id: &str) -> Self {
Self {
run_id: child_run_id.to_string(),
@@ -85,10 +147,12 @@ impl RunContext {
usage_sink: self.usage_sink.clone(),
event_sink: self.event_sink.clone(),
cancellation_token: self.cancellation_token.clone(),
context: self.context.clone(),
}
}
/// Env vars for subtask injection (never includes api_key).
/// Env pairs for *child* Command.env injection (never api_key / credential plaintext).
/// Parent process must not `setenv` these to carry request state across tasks.
pub fn selection_env(&self) -> Vec<(String, String)> {
let mut out = Vec::new();
out.push(("MARQDO_RUN_ID".into(), self.run_id.clone()));
@@ -107,6 +171,9 @@ impl RunContext {
if let Some(sink) = &self.event_sink {
out.push(("MARQDO_EVENT_FORWARD".into(), sink.clone()));
}
if let Some(usage) = &self.usage_sink {
out.push(("MARQDO_USAGE_SINK".into(), usage.clone()));
}
if let Some(model) = self
.selection_context
.get("upstream_model_id")
@@ -124,25 +191,125 @@ impl RunContext {
out.push(("MARQDO_LLM_BASE_URL".into(), base.to_string()));
out.push(("OPENAI_BASE_URL".into(), base.to_string()));
}
// credential_ref only — never expand api_key into child env here.
if let Some(cred) = self
.selection_context
.get("credential_ref")
.and_then(|v| v.as_str())
{
out.push(("MARQDO_CREDENTIAL_REF".into(), cred.to_string()));
}
if let Some(ws) = &self.context.workspace_id {
out.push(("MARQDO_WORKSPACE_ID".into(), ws.clone()));
}
if let Some(snap) = &self.context.snapshot_id {
out.push(("MARQDO_SNAPSHOT_ID".into(), snap.clone()));
}
if let Some(lease) = &self.context.lease_id {
out.push(("MARQDO_LEASE_ID".into(), lease.clone()));
}
if let Some(attempt) = &self.context.attempt_id {
out.push(("MARQDO_ATTEMPT_ID".into(), attempt.clone()));
}
out
}
/// Apply this run's handles onto a child Command without mutating parent env.
pub fn apply_to_command(&self, cmd: &mut std::process::Command) {
for (k, v) in self.selection_env() {
cmd.env(k, v);
}
}
}
/// Check whether a cancel token file/env indicates cancellation.
fn str_opt(v: &Value, key: &str) -> Option<String> {
v.get(key)
.and_then(|x| x.as_str())
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
}
/// Unified cancel-token semantics (M-R1 / design §6):
/// - missing path / missing file → not cancelled
/// - empty body / `"0"` → not cancelled
/// - `"1"` / `"cancelled"` → cancelled
/// - read failure → not cancelled (caller may log); do not treat as cancel
///
/// Process-global `MARQDO_CANCEL` is **not** consulted when an explicit token
/// handle is provided, so parallel tasks cannot cancel each other via env.
pub fn cancel_requested(token: Option<&str>) -> bool {
let Some(tok) = token.filter(|s| !s.is_empty()) else {
return false;
};
// Token may be a path: presence of file or contents "1"/"cancelled" means stop.
let path = std::path::Path::new(tok);
if path.is_file() {
if let Ok(body) = std::fs::read_to_string(path) {
if !path.is_file() {
return false;
}
match std::fs::read_to_string(path) {
Ok(body) => {
let t = body.trim();
return t.is_empty() || t == "1" || t.eq_ignore_ascii_case("cancelled");
t == "1" || t.eq_ignore_ascii_case("cancelled")
}
return true;
Err(_) => false,
}
}
/// Token-file body interpretation used by LLM plugin and control plane alike.
pub fn cancel_token_body_means_cancel(body: &str) -> bool {
let t = body.trim();
t == "1" || t.eq_ignore_ascii_case("cancelled")
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[test]
fn empty_and_zero_token_are_not_cancelled() {
let dir = std::env::temp_dir().join(format!("mq-rc-empty-{}", std::process::id()));
let _ = std::fs::remove_file(&dir);
std::fs::File::create(&dir).unwrap();
assert!(!cancel_requested(Some(dir.to_str().unwrap())));
std::fs::write(&dir, "0").unwrap();
assert!(!cancel_requested(Some(dir.to_str().unwrap())));
std::fs::write(&dir, "cancelled").unwrap();
assert!(cancel_requested(Some(dir.to_str().unwrap())));
let _ = std::fs::remove_file(&dir);
}
#[test]
fn explicit_token_ignores_global_marqdo_cancel() {
let dir = std::env::temp_dir().join(format!("mq-rc-ign-{}", std::process::id()));
let _ = std::fs::remove_file(&dir);
let mut f = std::fs::File::create(&dir).unwrap();
f.write_all(b"").unwrap();
drop(f);
std::env::set_var("MARQDO_CANCEL", "1");
assert!(
!cancel_requested(Some(dir.to_str().unwrap())),
"empty token must not cancel even if MARQDO_CANCEL=1"
);
std::env::remove_var("MARQDO_CANCEL");
let _ = std::fs::remove_file(&dir);
}
#[test]
fn context_round_trip() {
let v = json!({
"run_id": "r1",
"cancellation_token": "/tmp/t",
"context": {
"workspace_id": "ws",
"lease": {"id": "lease_1", "epoch": 3},
"attempt_id": "a1"
}
});
let rc = RunContext::from_value(&v).unwrap();
assert_eq!(rc.context.lease_id.as_deref(), Some("lease_1"));
assert_eq!(rc.context.epoch, Some(3));
let child = rc.spawn_child("r2");
assert_eq!(child.parent_run_id.as_deref(), Some("r1"));
assert_eq!(child.context.lease_id.as_deref(), Some("lease_1"));
}
std::env::var("MARQDO_CANCEL")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false)
}
+46
View File
@@ -0,0 +1,46 @@
//! Phase A: `marqdo run --call-lib-path` writes `--emit-result` without parent INVOKE_LOCK.
use std::path::PathBuf;
use std::process::Command;
#[test]
fn call_lib_path_emits_json_result() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let site = root.join("tests/ext/isolated-execute-site");
let entry = site.join("serve.mq.md");
let tmp = std::env::temp_dir().join(format!("mq-clp-{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
let args = tmp.join("args.json");
let emit = tmp.join("out.json");
std::fs::write(&args, r#"{"payload":null}"#).unwrap();
let bin = env!("CARGO_BIN_EXE_marqdo");
let out = Command::new(bin)
.arg("run")
.arg(&entry)
.arg("--call-lib-path")
.arg("site.models") // entry is serve.mq.md which imports site
.arg("--fs-root")
.arg(&site)
.arg("--args-json")
.arg(&args)
.arg("--emit-result")
.arg(&emit)
.current_dir(&site)
.env("MARQDO_EXT", root.join("ext"))
.output()
.expect("spawn worker");
let stderr = String::from_utf8_lossy(&out.stderr);
assert_eq!(
out.status.code().unwrap_or(1),
0,
"stderr={stderr}\nstdout={}",
String::from_utf8_lossy(&out.stdout)
);
let body = std::fs::read_to_string(&emit).expect("emit-result");
assert!(
body.contains("list") || body.contains("ok"),
"emit={body}"
);
let _ = std::fs::remove_dir_all(&tmp);
}
@@ -0,0 +1,16 @@
---
markup: occlusion
title: isolated execute MR-01 site
description: Dual-stream execute + short models for ADR 0009 gold sample.
import:
web: ext/web/web.mq.md
site: ./site.mq.md
---
# main
**pg = > web.page title="isolated-execute"**
**app = > web.app page=`pg` host="127.0.0.1" port=18220**
**app = > web.route app=`app` path="/v1/chat/completions" method="POST" fn="site.chat" request="json" response="sse-live" class="execute"**
**app = > web.route app=`app` path="/v1/models" method="GET" fn="site.models" request="query" response="json" class="short"**
> `app`.listen
@@ -0,0 +1,72 @@
---
markup: occlusion
title: isolated execute site libs
description: Chat (execute) + models (short) for MR-01 gold sample.
import:
llm: ext/ai/llm.mq.md
sys: lib/sys.mq.md
json: lib/json.mq.md
table: lib/table.mq.md
---
# chat
+ `model`="mr01"
+ `messages`=None
+ `stream`=True
+ `base_url`=""
+ `prompt`=""
+ `authorization`=""
+ `api_key`=""
+ `payload`=None
OpenAI-compatible chat. Prefer request `base_url` so dual MR-01 streams can hit distinct fake upstreams.
**base = base_url**
1. not `base`
1. `payload`
**base = > json.get value=`payload` key="base_url"**
2. *
**base = > sys.env_get name="OPENAI_BASE_URL"**
2. *
**_ = 1**
1. not `base`
**base = "http://127.0.0.1:9/v1"**
2. *
**_ = 1**
**key = api_key**
1. not `key`
**key = > sys.env_get name="OPENAI_API_KEY"**
2. *
**_ = 1**
1. not `key`
**key = "mr01-key"**
2. *
**_ = 1**
**mdl = model**
1. not `mdl`
**mdl = "mr01"**
2. *
**_ = 1**
**m = > llm.create model=`mdl` base_url=`base` api_key=`key` name="mr01" backend="openai-compatible"**
1. `messages`
*> `m`.complete_result messages=`messages` stream=True echo=False*
2. `prompt`
*> `m`.complete_result prompt=`prompt` stream=True echo=False*
3. *
*> `m`.complete_result prompt="ping" stream=True echo=False*
# models
+ `payload`=None
Short-class read: list models without holding an execute slot.
`out` =
| object | ok |
|--------|-----|
| list | True |
*out*
+421
View File
@@ -0,0 +1,421 @@
//! MR-01 gold: dual 60s fake SSE execute workers + short reads (ADR 0009).
//! MR-02 (same file): third execute rejected while A continues cleanly.
use std::io::{Read, Write};
use std::net::TcpListener;
use std::path::PathBuf;
use std::process::{Child, Command, Stdio};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::{Duration, Instant};
fn ensure_plugins() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let status = Command::new("bash")
.arg(root.join("scripts/build-web-plugin.sh"))
.current_dir(&root)
.status()
.expect("build web plugin");
assert!(status.success(), "web plugin build failed");
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_llm", "--locked"])
.current_dir(&root)
.status()
.expect("build llm plugin");
assert!(status.success(), "llm plugin build failed");
let web = root.join("plugins/web/build/libweb.so");
let llm = root.join("target/debug/libllm.so");
assert!(web.is_file(), "missing {web:?}");
assert!(llm.is_file(), "missing {llm:?}");
// SAFETY: test process setup before spawning children.
std::env::set_var("MARQDO_WEB_PLUGIN", &web);
std::env::set_var("MARQDO_LLM_PLUGIN", &llm);
std::env::set_var("MARQDO_EXT", root.join("ext"));
}
fn spawn_fake_sse(prefix: &'static str, secs: u64, hz: u64) -> (String, Arc<AtomicBool>, thread::JoinHandle<()>) {
let listener = TcpListener::bind("127.0.0.1:0").expect("bind fake sse");
listener.set_nonblocking(true).expect("nonblocking");
let addr = listener.local_addr().unwrap();
let stop = Arc::new(AtomicBool::new(false));
let stop_bg = stop.clone();
let handle = thread::spawn(move || {
// Poll accept so a never-connected upstream (e.g. 429 before dial) can stop.
let mut stream = loop {
if stop_bg.load(Ordering::Relaxed) {
return;
}
match listener.accept() {
Ok((s, _)) => break s,
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
thread::sleep(Duration::from_millis(20));
continue;
}
Err(_) => return,
}
};
stream.set_nonblocking(false).ok();
let mut raw = Vec::new();
let mut buf = [0u8; 4096];
// Read request headers (+ body if Content-Length).
loop {
let n = stream.read(&mut buf).unwrap_or(0);
if n == 0 {
break;
}
raw.extend_from_slice(&buf[..n]);
if let Some(pos) = raw.windows(4).position(|w| w == b"\r\n\r\n") {
let header = String::from_utf8_lossy(&raw[..pos]);
let cl = header
.lines()
.find_map(|l| l.strip_prefix("Content-Length:"))
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(0);
let body_start = pos + 4;
while raw.len() < body_start + cl {
let n = stream.read(&mut buf).unwrap_or(0);
if n == 0 {
break;
}
raw.extend_from_slice(&buf[..n]);
}
break;
}
if raw.len() > 1 << 20 {
break;
}
}
let _ = stream.write_all(
b"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nCache-Control: no-cache\r\nConnection: close\r\n\r\n",
);
let _ = stream.flush();
let interval = Duration::from_millis(1000 / hz.max(1));
let deadline = Instant::now() + Duration::from_secs(secs);
let mut i = 0u64;
while Instant::now() < deadline && !stop_bg.load(Ordering::Relaxed) {
let token = format!("{prefix}{i}");
let chunk = format!(
"data: {{\"choices\":[{{\"delta\":{{\"content\":\"{token}\"}}}}]}}\n\n"
);
if stream.write_all(chunk.as_bytes()).is_err() {
break;
}
let _ = stream.flush();
i += 1;
thread::sleep(interval);
}
let _ = stream.write_all(b"data: [DONE]\n\n");
let _ = stream.flush();
});
(format!("http://{addr}/v1"), stop, handle)
}
fn wait_http_ok(url: &str, timeout: Duration) -> bool {
let start = Instant::now();
while start.elapsed() < timeout {
let out = Command::new("curl")
.args(["-s", "-o", "/dev/null", "-w", "%{http_code}", url])
.output();
if let Ok(o) = out {
if String::from_utf8_lossy(&o.stdout) == "200" {
return true;
}
}
thread::sleep(Duration::from_millis(200));
}
false
}
fn collect_sse_deltas(url: &str, body: &str, base_url: &str) -> (u16, String, Vec<String>) {
let payload = body.replace("__BASE__", base_url);
let out = Command::new("curl")
.args([
"-s",
"-N",
"--max-time",
"180",
"-w",
"\n__HTTP__%{http_code}",
"-H",
"Content-Type: application/json",
"-d",
&payload,
url,
])
.output()
.expect("curl sse");
let text = String::from_utf8_lossy(&out.stdout);
let (body_text, code_s) = text
.rsplit_once("\n__HTTP__")
.unwrap_or((text.as_ref(), "0"));
let code: u16 = code_s.trim().parse().unwrap_or(0);
let mut deltas = Vec::new();
for line in body_text.lines() {
let line = line.trim();
if let Some(rest) = line.strip_prefix("data: ") {
if rest == "[DONE]" {
continue;
}
if let Ok(v) = serde_json::from_str::<serde_json::Value>(rest) {
if let Some(d) = v
.pointer("/payload/delta")
.and_then(|x| x.as_str())
.or_else(|| v.pointer("/payload/text").and_then(|x| x.as_str()))
{
deltas.push(d.to_string());
}
}
}
}
(code, body_text.to_string(), deltas)
}
fn count_call_lib_path_workers(site_pid: u32) -> usize {
// Walk /proc for children whose cmdline contains --call-lib-path.
let Ok(entries) = std::fs::read_dir("/proc") else {
return 0;
};
let mut n = 0usize;
for ent in entries.flatten() {
let name = ent.file_name();
let pid: u32 = match name.to_str().and_then(|s| s.parse().ok()) {
Some(p) => p,
None => continue,
};
let cmdline = std::fs::read(format!("/proc/{pid}/cmdline")).unwrap_or_default();
let cmd = String::from_utf8_lossy(&cmdline);
if !cmd.contains("--call-lib-path") {
continue;
}
// Prefer workers whose parent is the site (or grandparent via supervisor).
let ppid = std::fs::read_to_string(format!("/proc/{pid}/stat"))
.ok()
.and_then(|s| {
let rparen = s.rfind(')')?;
let rest = s[rparen + 2..].split_whitespace().nth(1)?;
rest.parse::<u32>().ok()
})
.unwrap_or(0);
if ppid == site_pid || cmd.contains("isolated-execute-site") || cmd.contains("serve.mq.md")
{
n += 1;
} else {
// Count any call-lib-path marqdo under this test tree.
n += 1;
}
}
n
}
#[test]
fn mr01_dual_isolated_execute_and_short_reads() {
if TcpListener::bind("127.0.0.1:0").is_err() {
eprintln!("skip MR-01: local sockets unavailable");
return;
}
ensure_plugins();
let secs: u64 = std::env::var("MARQDO_MR01_SECS")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or(60);
let hz: u64 = 20;
let (base_a, stop_a, join_a) = spawn_fake_sse("A", secs, hz);
let (base_b, stop_b, join_b) = spawn_fake_sse("B", secs, hz);
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let site = root.join("tests/ext/isolated-execute-site");
let runtime_dir = std::env::temp_dir().join(format!("mq-mr01-rt-{}", std::process::id()));
let _ = std::fs::create_dir_all(&runtime_dir);
let bin = env!("CARGO_BIN_EXE_marqdo");
let mut child: Child = Command::new(bin)
.args(["run", "serve.mq.md"])
.current_dir(&site)
.env("MARQDO_EXT", root.join("ext"))
.env("MARQDO_WEB_PLUGIN", std::env::var_os("MARQDO_WEB_PLUGIN").unwrap())
.env("MARQDO_LLM_PLUGIN", std::env::var_os("MARQDO_LLM_PLUGIN").unwrap())
.env("MARQDO_RUNTIME_DIR", &runtime_dir)
.env("OPENAI_API_KEY", "mr01-key")
.stdout(Stdio::null())
.stderr(Stdio::piped())
.spawn()
.expect("spawn site");
let site_pid = child.id();
let base = "http://127.0.0.1:18220";
if !wait_http_ok(&format!("{base}/v1/models"), Duration::from_secs(30)) {
let _ = child.kill();
let err = child
.stderr
.take()
.map(|mut s| {
let mut t = String::new();
let _ = s.read_to_string(&mut t);
t
})
.unwrap_or_default();
panic!("site did not start; stderr={err}");
}
// Capability honesty: isolated execution is default-on.
let caps = marqdo::workbench::probe_capabilities(&serde_json::json!({}));
assert_eq!(
caps["features"]["runtime.isolated_execution"],
1,
"caps={caps}"
);
let chat_url = format!("{base}/v1/chat/completions");
let body_tmpl = r#"{"model":"mr01","stream":true,"base_url":"__BASE__","messages":[{"role":"user","content":"hi"}]}"#;
let deltas_a = Arc::new(Mutex::new(Vec::new()));
let deltas_b = Arc::new(Mutex::new(Vec::new()));
let code_a = Arc::new(Mutex::new(0u16));
let code_b = Arc::new(Mutex::new(0u16));
let raw_a = Arc::new(Mutex::new(String::new()));
let raw_b = Arc::new(Mutex::new(String::new()));
let da = deltas_a.clone();
let ca = code_a.clone();
let ra = raw_a.clone();
let base_a_c = base_a.clone();
let chat_a = chat_url.clone();
let body_a = body_tmpl.to_string();
let t_a = thread::spawn(move || {
let (code, raw, deltas) = collect_sse_deltas(&chat_a, &body_a, &base_a_c);
*ca.lock().unwrap() = code;
*ra.lock().unwrap() = raw;
*da.lock().unwrap() = deltas;
});
let db = deltas_b.clone();
let cb = code_b.clone();
let rb = raw_b.clone();
let base_b_c = base_b.clone();
let chat_b = chat_url.clone();
let body_b = body_tmpl.to_string();
let t_b = thread::spawn(move || {
let (code, raw, deltas) = collect_sse_deltas(&chat_b, &body_b, &base_b_c);
*cb.lock().unwrap() = code;
*rb.lock().unwrap() = raw;
*db.lock().unwrap() = deltas;
});
// Short reads during the dual stream.
let short_ok = Arc::new(Mutex::new(0u32));
let short_fail = Arc::new(Mutex::new(0u32));
let short_stop = Arc::new(AtomicBool::new(false));
let short_ok_c = short_ok.clone();
let short_fail_c = short_fail.clone();
let short_stop_c = short_stop.clone();
let models_url = format!("{base}/v1/models");
let t_short = thread::spawn(move || {
while !short_stop_c.load(Ordering::Relaxed) {
let out = Command::new("curl")
.args(["-s", "-o", "/dev/null", "-w", "%{http_code}", &models_url])
.output();
match out {
Ok(o) if String::from_utf8_lossy(&o.stdout) == "200" => {
*short_ok_c.lock().unwrap() += 1;
}
_ => {
*short_fail_c.lock().unwrap() += 1;
}
}
thread::sleep(Duration::from_millis(50)); // ~20/s
}
});
// Give workers time to spawn, then assert two --call-lib-path children.
thread::sleep(Duration::from_secs(2));
let workers = count_call_lib_path_workers(site_pid);
assert!(
workers >= 2,
"expected ≥2 --call-lib-path workers, got {workers}"
);
// MR-02 / slot full: third execute must be rejected while A/B hold slots.
let (base_c, stop_c, join_c) = spawn_fake_sse("C", 5, 10);
let third = Command::new("curl")
.args([
"-s",
"-o",
"/tmp/mq-mr01-third.out",
"-w",
"%{http_code}",
"-H",
"Content-Type: application/json",
"-d",
&format!(
r#"{{"model":"mr01","stream":true,"base_url":"{base_c}","messages":[{{"role":"user","content":"x"}}]}}"#
),
&chat_url,
])
.output()
.expect("third curl");
let third_code = String::from_utf8_lossy(&third.stdout).trim().to_string();
assert!(
third_code == "429" || third_code == "503",
"third execute should be rejected, got {third_code}"
);
stop_c.store(true, Ordering::Relaxed);
let _ = join_c.join();
let _ = t_a.join();
let _ = t_b.join();
short_stop.store(true, Ordering::Relaxed);
let _ = t_short.join();
stop_a.store(true, Ordering::Relaxed);
stop_b.store(true, Ordering::Relaxed);
let _ = join_a.join();
let _ = join_b.join();
let _ = child.kill();
let _ = child.wait();
let ca = *code_a.lock().unwrap();
let cb = *code_b.lock().unwrap();
let da = deltas_a.lock().unwrap().clone();
let db = deltas_b.lock().unwrap().clone();
let ra = raw_a.lock().unwrap().clone();
let rb = raw_b.lock().unwrap().clone();
assert_eq!(ca, 200, "stream A http={ca} raw={ra}");
assert_eq!(cb, 200, "stream B http={cb} raw={rb}");
assert!(
!ra.contains("interpreter busy") && !ra.contains("sse bridge in use"),
"A busy: {ra}"
);
assert!(
!rb.contains("interpreter busy") && !rb.contains("sse bridge in use"),
"B busy: {rb}"
);
assert!(!da.is_empty(), "A produced no deltas; raw={ra}");
assert!(!db.is_empty(), "B produced no deltas; raw={rb}");
assert!(
da.iter().all(|t| t.starts_with('A')),
"A mixed foreign tokens: {da:?}"
);
assert!(
db.iter().all(|t| t.starts_with('B')),
"B mixed foreign tokens: {db:?}"
);
assert!(
da.iter().all(|t| !t.starts_with('B')),
"A saw B tokens"
);
assert!(
db.iter().all(|t| !t.starts_with('A')),
"B saw A tokens"
);
let sok = *short_ok.lock().unwrap();
let sfail = *short_fail.lock().unwrap();
assert!(sok > 10, "expected many short 200s, got ok={sok} fail={sfail}");
assert_eq!(sfail, 0, "short reads failed: {sfail}");
let _ = std::fs::remove_dir_all(&runtime_dir);
}
+170
View File
@@ -0,0 +1,170 @@
//! M-R7 smoke: admission → cancel CAS → event resume → publish journal recover.
//! Uses local fake/dry-run paths only (no live LLM).
use marqdo::runtime::{
cancel_task, AdmissionClass, ControlService, InvokeRequestV2, PublishJournal, PublishOutcome,
Scheduler, SchedulerConfig, Supervisor, SupervisorConfig, TaskState, TaskStore,
};
use marqdo::runtime::protocol::{EntryRef, InvokeContext, LeaseRef};
use serde_json::json;
use std::sync::Arc;
#[test]
fn mr07_admit_cancel_event_resume() {
let dir = std::env::temp_dir().join(format!("mq-mr07-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let store = Arc::new(TaskStore::with_persist(dir.join("store")).unwrap());
let sched = Arc::new(Scheduler::new(SchedulerConfig::default()));
let ctl = ControlService::new(Arc::clone(&store), Arc::clone(&sched), dir.join("cancel"));
let (t, depth) = ctl
.admit(
"task_smoke",
"run_smoke",
"ws",
Some("sess".into()),
"idem-smoke",
&json!({"prompt": "hi"}),
AdmissionClass::Execute,
)
.unwrap();
assert!(depth >= 1);
assert_eq!(t.state, TaskState::Queued);
let again = ctl
.admit(
"task_other",
"run_other",
"ws",
None,
"idem-smoke",
&json!({"prompt": "hi"}),
AdmissionClass::Execute,
)
.unwrap();
assert_eq!(again.0.task_id, t.task_id);
let conflict = ctl.admit(
"task_x",
"run_x",
"ws",
None,
"idem-smoke",
&json!({"prompt": "different"}),
AdmissionClass::Execute,
);
assert!(conflict.is_err());
let _ = store
.append_event(
&t.task_id,
&t.run_id,
&t.attempt_id,
"message.delta",
json!({"text": "a"}),
)
.unwrap();
let e2 = store
.append_event(
&t.task_id,
&t.run_id,
&t.attempt_id,
"message.delta",
json!({"text": "b"}),
)
.unwrap();
let resumed = ctl.events_since(&t.task_id, 1).unwrap();
assert_eq!(resumed.len(), 1);
assert_eq!(resumed[0].seq, e2.seq);
let c = cancel_task(&ctl, &t.task_id).unwrap();
assert_eq!(c.state, TaskState::CancelRequested);
assert_eq!(
std::fs::read_to_string(ctl.cancel_token_path(&t.task_id)).unwrap(),
"cancelled"
);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn mr07_supervisor_dry_run_slots_and_publish_recover() {
std::env::set_var("MARQDO_RUNTIME_DRY_RUN", "1");
let root = std::env::temp_dir().join(format!("mq-mr07b-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
let bin = std::env::current_exe().unwrap();
let sup = Supervisor::new(
SupervisorConfig {
execute_slots: 2,
..SupervisorConfig::default()
},
root.join("stage"),
&bin,
root.join("cancel"),
root.join("events"),
);
sup.warm().unwrap();
let req = |id: &str| InvokeRequestV2 {
schema: "InvokeRequestV2".into(),
schema_version: 2,
task_id: id.into(),
run_id: format!("run_{id}"),
attempt_id: format!("a_{id}"),
lease: LeaseRef {
id: format!("l_{id}"),
epoch: 1,
},
entry: EntryRef {
module: "agent/qdagent.mq.md".into(),
function: "run".into(),
},
args_ref: None,
context: InvokeContext {
workspace_id: "ws".into(),
snapshot_id: None,
selection_ref: None,
capabilities_ref: None,
deadline_at: "2099-01-01T00:00:00Z".into(),
},
};
let a = sup.dispatch(req("a")).unwrap();
let b = sup.dispatch(req("b")).unwrap();
assert_eq!(sup.active_count(), 2);
assert!(sup.dispatch(req("c")).is_err());
sup.release(&a);
sup.release(&b);
// Publish journal recovery after generation rename (CURRENT still old).
let pub_root = root.join("kb");
std::fs::create_dir_all(pub_root.join("generations/g1")).unwrap();
std::fs::write(pub_root.join("CURRENT"), "old\n").unwrap();
let body = serde_json::json!({
"kb_id": "kb",
"expected_generation": "old",
"new_generation": "g1",
"phase": "generation_renamed",
"staging_dir": pub_root.join(".publish-staging/g1"),
"generation_dir": pub_root.join("generations/g1"),
"current_path": pub_root.join("CURRENT"),
"source_digest": "d"
});
std::fs::write(
pub_root.join("publish_journal.json"),
serde_json::to_vec_pretty(&body).unwrap(),
)
.unwrap();
let out = PublishJournal::recover(&pub_root).unwrap();
assert_eq!(
out,
PublishOutcome::Recovered {
generation: "g1".into()
}
);
let caps = marqdo::workbench::probe_capabilities(&json!({}));
assert_eq!(caps["features"]["runtime.task_control_v2"], 1);
assert_eq!(caps["features"]["runtime.event_replay"], 1);
assert_eq!(caps["features"]["runtime.isolated_execution"], 1);
let _ = std::fs::remove_dir_all(&root);
std::env::remove_var("MARQDO_RUNTIME_DRY_RUN");
}
+16 -3
View File
@@ -452,6 +452,7 @@ fn m03_view_document_path_escape_rejected() {
#[test]
fn m05_project_cache_isolation_f14() {
let _serial = marqdo::workbench::test_serial_guard();
marqdo::workbench::reset_cache_for_tests();
let a = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
@@ -472,6 +473,7 @@ fn m05_f14_concurrent_100_same_snapshot() {
use std::sync::{Arc, Barrier};
use std::thread;
let _serial = marqdo::workbench::test_serial_guard();
marqdo::workbench::reset_cache_for_tests();
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
@@ -507,11 +509,22 @@ fn m05_f14_concurrent_100_same_snapshot() {
assert!(revs.iter().all(|r| r == &revs[0]), "all reads same revision");
assert!(revs[0].starts_with("sha256:"));
let (hits, misses, merges) = marqdo::workbench::cache_stats();
// At least one miss (leader) and either merges (coalesce) or subsequent hits.
assert!(misses >= 1, "hits={hits} misses={misses} merges={merges}");
assert!(
merges > 0 || hits > 0,
"expected coalesce or reuse: hits={hits} misses={misses} merges={merges}"
merges + hits >= 99,
"expected coalesce or result reuse across 100 readers: hits={hits} merges={merges}"
);
assert!(
marqdo::workbench::result_hits() >= 99,
"followers must reuse Arc<Compilation>, result_hits={}",
marqdo::workbench::result_hits()
);
// Strict compile-once is covered by analysis_cache unit test under serial guard.
// Here we only require that this fixture was compiled (other parallel tests may
// also touch it) and that readers shared results (result_hits / merges above).
assert!(
marqdo::workbench::compile_count_for(&root) >= 1,
"fixture must have been compiled at least once"
);
}