Compare commits

...
8 Commits
Author SHA1 Message Date
cflmyandCursor 9ea5bebfc3 chore(release): bump to 2.1.3 (workbench M00–M06)
Ship ModelSelection/RunContext/DocumentView contracts, LLM selection,
view document CLI, and live/F02 evidence. CLI, wasm, and ext pack aligned.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 23:13:27 +08:00
cflmyandCursor c7c329341b feat(workbench): live selection smoke, F02 agent e2e, DocumentView CLI
Hoist llm.ask prompt binding to avoid elevated `p`; prefer reasoning_content
when content is empty; point live/tests at in-tree MARQDO_EXT; add USTC live
gates and mock agent.step selection+resume evidence; view document --out for
read-only browser embed checks. Design stays Partial (not Verified).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 23:05:28 +08:00
cflmyandCursor 8cf5288481 feat(workbench): F14 coalesce, snapshot manifest, and sse drain evidence
Hold analysis locks across compile for concurrent same-snapshot reads,
emit SnapshotManifestV1 for app retention, and add Go drainForward tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 11:13:53 +08:00
cflmyandCursor 69c4e74872 feat(view): project KnowledgeStep kinds into DocumentView nodes
Strengthen F09 read-model coverage so observe/derive/call/iterate steps
appear alongside artifact cards without executing the document.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 11:03:01 +08:00
cflmyandCursor af0bda2e40 test(workbench): add F02 subtask/agent skeleton and cancel bridge evidence
Cover selection env propagation, agent skeleton without llm.llm, step-kinds
DocumentView read purity, and Go cancel-token bridge smoke.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 11:02:25 +08:00
cflmyandCursor 090b026547 test(workbench): harden F-matrix evidence and demote Verified claim
Add dual-endpoint LLM isolation, usage revision replace, cancel/F08/F11
asserts, and mark the design status Partial until remaining F gaps close.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 11:01:02 +08:00
cflmyandCursor 03396afd02 feat(workbench): land M00–M06 runtime contracts and read-only view
Add capabilities/selection/usage/DocumentView/locator/RunContext with
honest probes, LLM messages+cancel, sse-live cancel bridge, and CI
workbench_contracts gate; record design evidence for verified slices.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 10:55:08 +08:00
cflmy 663a8ba2c0 docs: specify runtime and view workbench support 2026-10-24 16:27:08 +00:00
57 changed files with 4783 additions and 87 deletions

No files matched your search

+12 -6
View File
@@ -28,6 +28,7 @@ Canonical release playbook for **cflmy/marqdo**. Read this skill **before** tagg
9. **Extension packs always publish to both** Cloudflare R2 (`https://ext.marqdo.com` via `scripts/upload-ext-r2.py`) **and** GitHub (Release assets / `ext-v*` tag). Credentials only in `~/.marqdo/r2.env` / CI — **never** commit. See [ext-cdn.md](../../../doc/design/ext-cdn.md).
10. **CLI Ubuntu package = PPA** (`ppa:cflmy/marqdo`) via `./scripts/ppa-ship.sh` / `ppa-build-source.sh`. GitHub zips remain for Windows and non-apt Linux.
11. **When a step needs the user** (sudo password, GPG passphrase, Launchpad UI, interactive `dput`): **stop, state what failed, and give the exact commands** to run in their terminal. Do not pretend the step succeeded.
12. **GitHub auto-build must be green before declaring a release done.** For full/cli: wait for the tag-triggered **Release** workflow (`windows` + `linux`) **and** the push-triggered **`executable-knowledge`** workflow on the release commit / `main` tip. Release asset zips alone are **not** sufficient — a green Release job with a red EK gate is an incomplete release. Do not skip or reframe a red `executable-knowledge` as “assets already uploaded.”
## Release modes
@@ -78,6 +79,7 @@ Release progress:
```
Checks: clean tree; `main` ff-only; smoke tests; CHANGELOG Unreleased; no secrets.
Also confirm the latest `main` **`executable-knowledge`** Actions run is green (or that this release commit will be covered after push) — do not proceed to tag if EK is already red on tip without a fix in the same release PR.
## Phase 2 — Docs & version bumps (main)
@@ -129,7 +131,9 @@ git push https://proxy.cflmy.top/github.com/cflmy/marqdo.git vVER
## Phase 5 — GitHub Release notes & assets
1. Wait for Actions (full/cli): both `windows` and `linux` green when ext natives required.
1. Wait for Actions (full/cli):
- Tag **Release** workflow: both `windows` and `linux` green when ext natives required.
- Push **`executable-knowledge`** workflow on the release commit / `main`: **must be success**. Poll with `gh run list --workflow=executable-knowledge.yml --branch main` / `gh run watch`. If EK is red → **stop**; fix plugin/harness on `main`, re-run; do **not** declare the release complete from assets alone.
2. Rewrite release body 中英双语 — [reference.md § Notes](reference.md).
3. Confirm Linux `marqdo-VER-native-x86_64-unknown-linux-gnu.zip` when mode ships ext.
@@ -174,10 +178,11 @@ On Launchpad failure: read the build log, fix `debian/` or bundle script, bump `
## Phase 8 — Post-release
1. `gh release view TAG` — assets + notes.
2. CDN `https://ext.marqdo.com/vVER/` (if ext shipped).
3. PPA build green (if cli shipped).
4. Tell user URLs: GitHub release, CDN, PPA install one-liner.
5. Optional gh-pages only if asked.
2. Confirm **`executable-knowledge`** and **Release** Actions are both green for this tag/commit; if either is red, say so and do not claim “发版完成”.
3. CDN `https://ext.marqdo.com/vVER/` (if ext shipped).
4. PPA build green (if cli shipped).
5. Tell user URLs: GitHub release, CDN, PPA install one-liner.
6. Optional gh-pages only if asked.
## Extension-only release (mode=ext)
@@ -194,7 +199,8 @@ On Launchpad failure: read the build log, fix `debian/` or bundle script, bump `
| `gh` / git TLS or Clash `7890` refused | Unset bad proxy; see [reference § Proxy](reference.md) |
| GitHub blocked | proxy.cflmy.top, then HK jump; **print commands** if agent cannot push |
| Tag exists | Stop; ask user |
| CI red | Fix on main; retag only if user allows |
| CI red (Release assets) | Fix on main; retag only if user allows |
| `executable-knowledge` red | Fix agent plugin / test harness (e.g. `ensure_agent_plugin_built`); push `main`; wait for EK green. **Do not** treat uploaded Release zips as a successful release |
| R2 upload fails | Print `upload-ext-r2.py` command; do not skip silently |
| PPA / sudo / GPG | Print `./scripts/ppa-ship.sh noble` (or `dput …`); do not skip silently |
| Launchpad `Cargo.lock` v4 / old cargo | Ensure bundled `third_party/rust` via current `ppa-build-source.sh`; bump `1ppaN` |
@@ -304,6 +304,15 @@ gh workflow run Release --ref "vVER" # only if workflow_dispatch exists; else f
Current Release workflow is **tag-push only**. Red CI after tag → fix on `main`, then either new patch tag or **user-approved** delete+recreate tag (dangerous; prefer patch).
**Also required before declaring release done:** the push-triggered **`executable-knowledge`** workflow on the release commit / `main` tip must be **success**. Poll with:
```bash
gh run list --workflow=executable-knowledge.yml --branch main --limit 5
gh run watch <run-id>
```
A green Release asset job with a red `executable-knowledge` gate is an **incomplete** release — fix the plugin/harness (e.g. `ensure_agent_plugin_built` for agent-dependent EK tests), push `main`, and wait for EK green. Do not skip.
### HK SSH jump (Paramiko CONNECT)
**When to use:** local → GitHub fails (timeout / TLS / Clash flaky / reverse-proxy `proxy.cflmy.top` connect drops), but this host can reach an overseas box that has stable GitHub (default `hk.cflmy.de`). Proven path for v0.3.9 push/tag/`gh`.
@@ -11,6 +11,12 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: Build agent + llm plugins (EK / workbench gates)
run: |
cargo build -p marqdo_plugin_agent --locked
cargo build -p marqdo_plugin_llm --locked
- name: Workbench contracts (M00–M06 / F-subset)
run: cargo test --test workbench_contracts --locked
- name: Offline executable-knowledge gate
run: cargo test --test executable_knowledge --locked
- name: no ignored tests counted as success
+3 -3
View File
@@ -126,9 +126,9 @@ jobs:
## Marqdo ${{ github.ref_name }}
### 亮点 / Highlights
- **v2.1.2**:可执行知识生产链——证据/闭包 · verify/adjudicate · reuse_execute/bind_execute · document.extract · View KnowledgePresentation + SSE page_token · 上传限流
- **v2.1.2**: Executable-knowledge production chain — evidence/closure, verify/adjudicate, reuse_execute/bind_execute, document.extract, View KnowledgePresentation + SSE page_token, upload caps
- **v2.1.2 ext**:扩展包同步 2.1.2 / Ext pack 2.1.2
- **v2.1.3**:知识工作台运行时——ModelSelection/RunContext/usage · view document 只读嵌入 · LLM selection/messages/cancel · agent selection e2e
- **v2.1.3**: Workbench runtime — ModelSelection/RunContext/usage, view document read-only embed, LLM selection/messages/cancel, agent selection e2e
- **v2.1.3 ext**:扩展包同步 2.1.3 / Ext pack 2.1.3
- **v2.1.1**:可执行知识主链——BoundCall · NarrativeRef · 来源/适用期 · patch/extract · 整代 CURRENT + closure · 发票示例 113
- **v2.1.1**: Executable-knowledge spine — BoundCall, NarrativeRef, sources, patch/extract, generation CURRENT + closure, invoice example 113
- **v1.5.2**:外联 v3 P0/P1——raw stdin + 每 cell 输入 · plan API · snapshot · `view capture` · Persist
+36
View File
@@ -1,5 +1,41 @@
# Changelog
## Unreleased
### Added
### Fixed
### Changed
## v2.1.3 — 2026-10-25
### Highlights
**知识工作台运行时合同(M00–M06)**:`ModelSelectionV1` / `RunContextV1` / `ModelCallUsageV1` / `DocumentViewV1` / locator / snapshot;严格 selection;usage null≠0;`view document` 只读嵌入;LLM messages + cancel;agent.step selection 继承假模型 e2e;USTC 兼容 live smoke(env 注入)。CLI · wasm · ext **2.1.3**。设计证据仍为 Partial(原生适配器 unavailable;完整宿主 F12 留给应用)。
```bash
git checkout v2.1.3 && cargo build --release
marqdo ext add llm # CDN / Releases → 2.1.3
```
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——工作台合同与 View/LLM/Agent 扩展面,不新增 Markdown 标记。
### Added
- **Workbench M00–M06**:capabilities / selection / RunContext / usage / DocumentView / locator / SnapshotManifest;`tests/workbench_contracts`(含 F02 agent.step+resume、F14 并发);`marqdo view document --out` 只读 HTML 嵌入。
- **LLM**:`messages=` / `selection=` / `strict_selection`;cancel token;reasoning_content 回退;双端点隔离门禁。
- **Live**:`tests/workbench_live`(`OPENAI_*` env only;`MARQDO_EXT` 指向仓内 `ext/`)。
- **Agent skeleton**:子工作簿禁 `llm.llm`,从 env selection `llm.create`。
### Fixed
- **`llm.ask` 升参**:hoist `p` 避免缺参 `missing argument for parameter p`;避免 `~/.marqdo/ext` 陈旧包遮蔽仓内 `ext/`(测试强制 `MARQDO_EXT`)。
### Changed
- CLI / wasm / `ext/VERSION` **同步 2.1.3**。
## v2.0.0 — 2026-10-22
### Highlights
Generated
+16 -16
View File
@@ -1,6 +1,6 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
version = 3
[[package]]
name = "adler2"
@@ -264,9 +264,9 @@ dependencies = [
[[package]]
name = "crossbeam-utils"
version = "0.8.21"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28"
checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
[[package]]
name = "crypto-common"
@@ -539,9 +539,9 @@ checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "js-sys"
version = "0.3.104"
version = "0.3.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e0c1080212aad755ea003d18543e8768dd432c48819efd73a7bf1e39b7a5a3a"
checksum = "7883d941dae510fb2d978fc3fe018c71c9e2892fd38854de3e8b92c2e5ad9cc5"
dependencies = [
"cfg-if",
"futures-util",
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "marqdo"
version = "2.1.2"
version = "2.1.3"
dependencies = [
"anyhow",
"assert_cmd",
@@ -610,7 +610,7 @@ dependencies = [
[[package]]
name = "marqdo-wasm"
version = "2.1.2"
version = "2.1.3"
dependencies = [
"marqdo",
"serde_json",
@@ -1290,9 +1290,9 @@ checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
[[package]]
name = "wasm-bindgen"
version = "0.2.127"
version = "0.2.129"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b70935747edd64d89de3efa29d73789b806c15798f8e7dca4d8ac356b50ce70"
checksum = "9bb54f33acc68fd454578d9820b0bde1a1a3d17aa17bb7b6595806d02886d409"
dependencies = [
"cfg-if",
"once_cell",
@@ -1303,9 +1303,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.127"
version = "0.2.129"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77775f8f3f7217702089053b94958f8f54061a3f663417df76e19cbdcca29bc1"
checksum = "2e29d0c35b16e224a7eeb5cd2d25e3e1968fbd65604117b44d3b789d00ee8535"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
@@ -1313,22 +1313,22 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.127"
version = "0.2.129"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e11d33f857dc2fb11b8bc75aee111aa9cbeb12cd9f25efd3d4c2a3dd4e235284"
checksum = "6f501a8bc3719dba86ef8ae4728879c08001bea749eb1333ac5b91e040e2a6b7"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn 2.0.119",
"syn 3.0.3",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.127"
version = "0.2.129"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ef64dbcc55df09c7e5a46182d181c2cfa3e925f3da937ea764728b4bbb9dcbf"
checksum = "23f0c9c52aa7cd7d77769a4cfe2a9adb1b331f489a41d912ce14513d5ab995c6"
dependencies = [
"unicode-ident",
]
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo"
version = "2.1.2"
version = "2.1.3"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo — Markdown markers as an interpreted programming language"
+6 -5
View File
@@ -281,9 +281,10 @@ marqdo catalog public -o .marqdo
---
## 现状(v2.1.2)
## 现状(v2.1.3)
- **可执行知识生产链(v2.1.2)**:证据/闭包失效 · `verify`/`adjudicate` · `reuse_execute` / `eng_bind_execute`+`publish_gate` · `document.extract` · View `KnowledgePresentation`+SSE `page_token` · 上传限流 · 门禁 45 项(100/1000 单元)· 扩展包同步 **2.1.2**
- **知识工作台运行时(v2.1.3)**:ModelSelection / RunContext / usage null≠0 · `view document` 只读嵌入 · LLM selection/messages/cancel · agent.step selection e2e · live smoke · 扩展包同步 **2.1.3**(设计 Partial:原生适配器 unavailable)
- **可执行知识生产链(v2.1.2)**:证据/闭包失效 · `verify`/`adjudicate` · `reuse_execute` / `eng_bind_execute`+`publish_gate` · `document.extract` · View `KnowledgePresentation`+SSE `page_token` · 上传限流 · 门禁 45 项(100/1000 单元)
- **可执行知识主链(v2.1.1)**:BoundCall 双后端 · NarrativeRef · 来源/适用期 · `artifact.patch` / `extract_pure` · 整代 `CURRENT` + closure · 发票示例 `113`
- **链接语法(v2.1.0)**:`[描述](目标)` 描述不执行 · `()` 与 `**代码**` 同权(`加(1)` / `加一 号(31)`)· `access` 取元与 `遍历` 链接循环 · `marqdo labels` 两级检索 / `--baseline` 债务棘轮 · `migrate labels --mark` · 诊断 `label.*` · view 描述正文 + 代码片 + 桩徽章 · **可执行知识 [link-syntax.md](doc/design/link-syntax.md)**(项目索引、LinkInfo、链接图与 lock)
- **外联 v3 P0/P1(v1.5.2)**:raw stdin + 每 cell 独立输入 · plan API · snapshot export/import/replay · `view capture` · Persist 按钮 · 扩展包同步 **1.5.2**
@@ -321,7 +322,7 @@ marqdo catalog public -o .marqdo
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
### 如何使用最新 Marqdo(v2.1.2)
### 如何使用最新 Marqdo(v2.1.3)
**安装解释器(任选其一)**
@@ -336,7 +337,7 @@ marqdo catalog public -o .marqdo
```bash
# 1) 源码安装(跟 tag 或 main)
git clone https://github.com/cflmy/marqdo.git && cd marqdo
git checkout v2.1.2 # 或留在 main
git checkout v2.1.3 # 或留在 main
cargo build --release
export PATH="$PWD/target/release:$PATH"
marqdo version
@@ -375,7 +376,7 @@ marqdo wasm build
# → dist/wasm/ … 见 examples/browser-hello/
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
# https://github.com/cflmy/marqdo/releases/tag/v2.1.2
# https://github.com/cflmy/marqdo/releases/tag/v2.1.3
```
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo-wasm"
version = "2.1.2"
version = "2.1.3"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
+8
View File
@@ -1,3 +1,11 @@
marqdo (2.1.3-1ppa1) noble; urgency=medium
* Marqdo 2.1.3: workbench M00–M06 runtime contracts (ModelSelection,
RunContext, DocumentView, usage null≠0, LLM selection/cancel,
agent.step selection e2e, view document CLI, live smoke).
-- cflmy <pingan@cflmy.cn> Sun, 25 Oct 2026 15:20:00 +0000
marqdo (2.1.2-1ppa1) noble; urgency=medium
* Marqdo 2.1.2: executable-knowledge F0–F3 + View V0–V3 release bar
+1
View File
@@ -19,6 +19,7 @@
| 文档 | 摘要 |
|------|------|
| [design/qdagent-workbench-runtime-and-view-2026-10-25.md](design/qdagent-workbench-runtime-and-view-2026-10-25.md) | **qdagent 知识工作台框架支持**:M00–M05 合同落地中(`src/workbench/`、MLSP `capabilities` / `view.document` / `view.reference_resolve`、`tests/workbench_contracts.rs`);完整 verified 以切片验收证据为准 |
| [design/view-knowledge-reading-experience-2026-10-24.md](design/view-knowledge-reading-experience-2026-10-24.md) | **View 知识阅读体验重设计(建议)**:正文优先、结构导航、就地知识说明、来源与运行状态、响应式线框、实现契约与验收;含当前呈现和 no-exec 反例 |
| [roadmap/qdagent-knowledge-support-gaps-2026-10-24.md](roadmap/qdagent-knowledge-support-gaps-2026-10-24.md) | **求道知识沉淀与文档编译支撑缺口**:2.1.1 源码复核、来源/依赖失效反例、验证与 Agent 协议、语法提示词、导入底座及跨库分工 |
| [design/document-native-development-guide.md](design/document-native-development-guide.md) | **下一阶段开发指导**:语法审查裁决、正文→命名→复用工作流、AI 上下文与局部编辑、渐进严格、作者成本及 S0–S6 验收 |
@@ -0,0 +1,435 @@
# qdagent 知识工作台所需的多模型运行与 View 嵌入支持
日期:2026-10-25。状态:Partial / in progress(M00–M06 合同门禁 + F02 agent.step/resume 假模型 e2e + USTC live selection smoke + DocumentView 浏览器只读核对已落地;F 矩阵未全绿,不得整文宣称 Verified;原生非兼容适配器仍 unavailable)。
源码基线:Marqdo `dc853fb`(2.1.2),qdagent `643dd31`,已同步远端。应用侧配套:[qdagent 多提供商与知识工作台](../../../qdagent/doc/design/multi-provider-and-knowledge-workbench-2026-10-25.md)。跨仓相对链接以兄弟仓库工作区布局为准。
沿用 [文档原生开发指导](document-native-development-guide.md)、[可执行知识规范](../next/Marqdo_Executable_Knowledge_Development.md) 与 [View 阅读体验设计](view-knowledge-reading-experience-2026-10-24.md)。本规格补充嵌入、调用身份和运行观测,不另造语言或第二套知识真相。
## 1. 框架目标与责任边界
让任何宿主应用可以按请求构造模型、追踪所有真实模型调用、以只读 View 展示知识文档,并从引用定位到版本一致的知识与来源。qdagent 的设置页、模型目录持久化、账单展示、会话导航、工作区授权和画像管理由应用实现。
| 能力 | Marqdo 拥有 | qdagent 拥有 |
|---|---|---|
| 多提供商 | 协议适配、模型 handle、能力协商、调用参数、归一化错误 | 提供商账户/端点/模型 CRUD、默认选择、凭证管理 |
| 模型路由 | Runtime 执行显式选择和允许的辅助模型策略 | 用户选择、工作区模型允许列表、路由授权 |
| 用量 | 每次请求/重试的身份、usage 来源与完整性、子任务传播 | 持久化账本、价格版本、费用估算、筛选/导出/预算策略 |
| 实时运行 | 逐事件输出、取消传播、背压、终态 | HTTP 会话鉴权、轮次映射、持久化事件重放、断线 UX |
| View | 文档语义、正文/目录/步骤、纯读取、稳定定位、可嵌入输出 | 容器/主题/深链/阅读历史/产品操作 |
| 知识引用 | 解析/歧义/版本/来源定位、验证多维事实 | 消息引用标识、授权、历史资料保存与入口 |
| 工作区 | 按显式 project 编译/查询,缓存及输出隔离 | 工作区 registry、root 解析、用户权限、文件管理 |
| 画像与笔记 | 通用文档渲染、来源关联、验证/编译原语 | 内容策略、提案/确认、生命周期、展示和上下文选择 |
不在语言核心添加 provider/workspace/profile 关键字;沿用 `.mq.md` 函数、结构化参数、扩展插件和 MLSP。不得让“可读”取决于用户手填 hash、node id、source span 等机器字段。
## 2. 2.1.2 已完成的基础与剩余缺口
### 2.1 当前可复用基础
| 源码 | 已有内容 | 本规格的增量 |
|---|---|---|
| [ext/ai/llm](../../ext/ai/llm.mq.md) | `create(model,base_url,api_key,backend,name)`、命名 handle、ask/stream | 无需重做基础 handle;增加隔离策略、协议能力、完整消息与调用身份 |
| [LLM plugin](../../plugins/llm/src/lib.rs) | 兼容 chat completions HTTP、SSE 文本读取、非流 usage 三项 | SSE usage/异常终态/每次调用观测、可取消持续输出 |
| [Agent](../../ext/ai/agent.mq.md) | Agent 唯一 loop、router_model、EFI、绑定/发布门闩 | 主/路由/修复/生成工作簿的模型及计量上下文传播 |
| [语义 IR](../../src/sema/ir.rs) | Artifact、KnowledgeStep、KnowledgeRef、SourceLocation、BoundCall | View 面向阅读的完整投影及 ID/版本映射 |
| [MLSP](../../src/mlsp.rs) | `artifact.inspect`、`link.resolve`、verify/bind/execute、document.extract | 只读 document/引用解析及 capabilities 版本握手 |
| [KnowledgePresentation](../../src/view/knowledge_present.rs) | schema v1,artifact nodes、依赖、generation/page token | 保留兼容,新增正文/目录/refs/source spans/结构化诊断 |
| [View output](../../src/view/output.rs) | `--no-exec` 在构建前跳过 capture | 将纯读取能力暴露给嵌入客户端,不复用会执行的 live 默认路径 |
| [document extract](../../src/document_extract.rs) | TXT/MD,partial,PDF/DOCX/OCR 显式不支持 | 能力查询及稳定来源定位;不要求本阶段重造 OCR |
| [数据资源](../../ext/data/db.mq.md) | SQLite migrate/exec/事务、唯一/索引声明 | 应用可直接承载 registry/账本/索引;不需另加产品专用数据库框架 |
2.1.2 已把知识辅助块移到正文后并折叠,跳过 module/type 卡片,显示“结构步骤非实际执行时序”。旧 View 截图中的最初顺序问题已有改进;但这不意味着已提供能直接放进 qdagent 右栏的独立阅读 SDK。
### 2.2 直接影响集成的事实
1. `llm.complete_result` 当前仍走 `openai.chat_completions`,Ollama 也是兼容路径;`backend` 名称不是任意原生协议适配器。
2. `plugins/llm::chat_completions` 从 `prompt` 构造一条 user 消息,没有完整消息列表契约;不能靠模型选择 UI 弥补历史丢失。
3. `usage_from` 缺字段填零;Agent `model_turn(stream=True)` 直接 `tokens=0`,最终 `token_cost=tokens`。现有输出无法支撑准确货币或流式用量显示。
4. 原生 LLM SSE parser 返回事件数组;`ext/ai` 有流事件机制,但应用所用 [web invoke](../../plugins/web/internal/httpx/routes_invoke.go) 在 `CallLibPath` 返回后才写 `{sse}`。需要打通实际增量路径,不能只换 Content-Type。
5. Agent 生成子工作簿的模板仍能构造默认 `llm.llm`;只传主 agent handle 不足以证明嵌套调用沿用了用户选定模型。
6. `build_file_view` 默认传 `execute=true`;live GET 路径使用它。嵌入阅读绝不能直接暴露这个入口。
7. Presentation 目前按 artifacts 投影,缺全量 prose/heading/step/ref 的统一阅读树;`render_knowledge_html` 另从 steps/refs 渲染,容易与 JSON 事实发生覆盖差异。
8. `source_digest` 当前由语义 artifact hashes 汇总,`semantic_generation` 是语义散列截短,`page_token` 来自原文件。它们不能直接分别冒充原文 revision、`.marqdo/CURRENT` generation 和跨请求完整版本。
9. `sse_accepts_packet` 对缺 token 的 legacy 包放行;正式嵌入客户端必须采用严格版本匹配,而不是继承这项兼容宽容。
10. `artifact.inspect/link.resolve` 每次走当前 semantic_project;现有入口不保证可以读任意历史 generation。历史引用需要明确的保留与解析合同。
## 3. 共用合同、版本与能力发现
合同名字固定为 `ModelSelectionV1`、`ModelCallUsageV1`、`RuntimeEventV1`、`DocumentViewV1`、`KnowledgeLocatorV1`;它们是这次扩展的 v1,与当前 KnowledgePresentation `schema_version:1` 不混用。新增 read model 不强行给已有 JSON 改含义。
新增 MLSP `capabilities`(拟议):
```json
{
"schema_version": 1,
"runtime_version": "implementation-version",
"features": {
"llm.explicit_selection": 1,
"llm.call_usage": 1,
"runtime.event_stream": 1,
"runtime.cancel": 1,
"view.document": 1,
"view.reference_resolve": 1
},
"adapters": [{"id": "openai-compatible", "available": true}],
"extract_formats": [{"format": "text", "available": true}],
"loaded_plugins": [{"name": "llm", "capabilities": ["call_usage_v1"]}]
}
```
只返回实际探测且加载可用的功能。不能仅从 CLI 版本推断 native plugin 符号存在;字段未知时应用降级或解释不支持。协议允许新增可选字段;删除/改变状态语义必须提升 major contract version。能力查询不调用供应商、不执行项目代码。
MLSP 继续使用现有 `{id,ok,result|error}` envelope;qdagent `/api/app` 转为自己的 `{ok,schema_version,data|error}`,不要要求它们共享同一层 envelope。所有合同提供成功、partial、unknown、unsupported、stale、cancelled、错误 fixtures 与 JSON Schema;应用和框架从相同样本跑消费测试。
## 4. M01:请求级模型身份与适配器
### 4.1 显式选择合同
应用将配置解析成可信 `ModelSelectionV1` 后传入 runtime,核心字段:
```json
{
"schema_version": 1,
"selection_id": "sel-1",
"provider_id": "provider-1",
"model_id": "model-2",
"provider_revision": 3,
"model_revision": 5,
"adapter": "openai-compatible",
"endpoint_ref": "endpoint-local-1",
"credential_ref": "credential-local-1",
"credential_version": 2,
"upstream_model_id": "example-model",
"parameters": {"max_output_tokens": 2048},
"fallback_policy": "disabled"
}
```
endpoint/credential 引用由宿主 resolver 解析为调用局部配置;短期直接向现有 `llm.create` 传参数亦可,但这不是可序列化的 public snapshot。API key 不能出现在 events、debug JSON、生成工作簿、prompt、错误 URI 或持久化模型对象中。
当前显式 args 优先的能力保留;新增 strict selection 模式:字段缺失报错,不回退 process env/frontmatter。无凭证端点通过 `auth_mode:none` 显式表达,避免 falsy key 触发环境 fallback。legacy 命令行未传 selection 时仍可沿用原有 env 行为;本地便利兼容不扩散到应用请求。
每个运行绑定 selection snapshot,后续设置改变不修改它。主模型与 router_model 分开但同属受控 selection 集;manual 模式所有需要 LLM 的步骤固定选择,REUSE 无模型调用仍允许。fallback 必须由宿主明确授予、列出允许目标,不能因额度/网络错误静默转到另一个供应商。
### 4.2 子任务和工作簿传播
统一 `RunContextV1 = run_id,parent_run_id?,turn_id?,workspace_scope,selection_context,usage_sink,event_sink,cancellation_token`。进入 ask/step/plan、repair、summary、learn、子工作簿时显式继承;新 run/call id 关联父节点。
生成工作簿只包含不可重放滥用的运行上下文引用,不写密钥;执行时宿主注入/解析。独立导出工作簿在缺凭证/选择时清晰要求配置,不偷偷取默认模型。在任何 child/spawn/resume 路径遗漏 selection 都应由 strict 模式拒绝并给出定位诊断。
同时审查环境变量不只是凭证:project root、temp request、输出目录、event sink 也必须请求局部,不能在并发运行中依靠进程可变状态串联。
### 4.3 适配器与消息
适配器最小职责:验证 selection、报告能力、序列化请求、解析增量结果、usage 归一化、取消传输、错误归一化。可选 discovery/probe;无模型列表 API 仍允许手工配置。
V1 消息包含 role 和 content parts,parts 最低 text,附件通过受控引用;tool call/result 要保持配对及 role。不支持的 part/parameter 返回 `llm.unsupported_input/parameter`,不能悄悄转成空字符串。适配器决定协议格式,Agent 决定行为;qdagent 不拥有供应商协议代码。
阶段一验证两个不同兼容端点,分别至少两个模型,以及无密钥本地兼容服务。阶段二增加原生非兼容协议:每个适配器需具名能力、流式/非流式/错误/取消/usage fixtures 和独立接入说明后才能在 capabilities 中置 available。不能把所有厂商默认走兼容接口后统称全支持。
统一参数采用 `max_output_tokens,temperature?,stop?,reasoning?` 等语义键,能力不支持时明确拒绝;不随意透传客户端未知参数。实际 upstream model 若与请求 id 不同,在 result 中保留 requested/resolved 两者。供应商版本与价格随时变化,本文仅锁定内部合同;实施原生适配器时另附当时官方协议核验记录。
## 5. M02:用量、事件流与取消
### 5.1 ModelCallUsageV1
```json
{
"schema_version": 1,
"call_id": "call-1",
"attempt_id": "attempt-1",
"parent_call_id": null,
"run_id": "run-1",
"turn_id": "turn-1",
"workspace_scope": "ws-1",
"purpose": "answer",
"selection_id": "sel-1",
"provider_id": "provider-1",
"model_id": "model-2",
"requested_model": "example-model",
"resolved_model": "example-model",
"status": "succeeded",
"usage_state": "final",
"measurement_source": "provider",
"input_tokens": 1200,
"output_tokens": 300,
"total_tokens": 1500,
"cached_input_tokens": 800,
"reasoning_output_tokens": null,
"revision": 2,
"provider_request_id": "opaque-provider-id",
"started_at": "2026-10-25T01:00:00Z",
"finished_at": "2026-10-25T01:00:03Z"
}
```
字段要求:tokens 非负整数或 null;`usage_state = pending|partial|final|unknown`;`measurement_source = provider|estimated|none`。估算可作为单独 estimate 字段返回,不能覆盖实测字段。供应商没有报告时用 null/unknown,传输失败不能自动解释成零消耗。
`status = started|succeeded|failed|cancelled|interrupted`,status 和 usage_state 独立。每个真实发出的 HTTP attempt 一个唯一 id;预检拒绝只产生 call-rejected 事件,不伪造已发送请求。框架重试必须可观测:所有 attempts 包括失败/有部分输出的 attempt;重复 delivery 的 usage revision 不算新请求。
累计 SSE usage 在同 attempt 内按 revision 发布完整快照,不让上层猜是 delta 还是 total。provider 总量与 input/output 不一致时保留原始读数及 `usage.inconsistent` 诊断,不自行强制改账;细分字段要标 inclusion 关系以避免缓存、reasoning 与总量重复相加。只保留脱敏的必要 provider usage 扩展字段,不把完整原始响应当 public usage。
纯知识复用用 run summary 中 `llm_calls=0`、`measurement_source=none` 表示,而不是伪造一条耗费为零的模型请求。框架输出 `call_count,known_tokens,unknown_call_count`,宿主据此显示覆盖程度。
### 5.2 事件流合同
```text
run.started
→ model.resolved
→ model.call.started
→ message.delta / tool.progress / usage.updated(零到多次)
→ model.call.finished(每个 attempt 各一条)
→ document.produced / citation.produced(仅有真实对象时)
→ run.completed | run.failed | run.cancelled | run.interrupted
```
`RuntimeEventV1` 公共字段:`schema_version,event_id,seq,run_id,parent_run_id?,turn_id?,workspace_scope,timestamp,type,payload`。seq 每个 run 单调;不同子 run 可以并发,宿主按所属 turn 另分配产品流序号。runtime terminal 事件仅一次;后续补全计量用有 revision 的 `usage.updated`,不能改变内容终态。
事件 sink 支持有界队列和背压;慢客户端不能无限堆积 token。丢弃可合并 progress 时记录 gap;内容与终态事件不可无声丢失。连接终止不会自动重跑;持久化/重放由应用,但框架需在实时回调后仍提供可核对的 final result/call summaries。
同一真实调用所有数据使用同一 run/call identity,不能流与最终结果各生成一套 id。工具参数/结果按 policy 脱敏,不把 secret、完整私有文件、模型内部推理链写入默认进度事件。
### 5.3 打通传输而不重造 Agent
需要同时打通 LLM provider reader → Agent event sink → host callback → web streaming handler → 浏览器。现有 `sys.stream_publish` 可以作为底座,但要验证它是否绑定正确请求且在 `CallLibPath` 返回前可写出;不能用进程全局事件广播混流。
可实施路径:新增 web 流式 invoke/job primitive,由框架注册 run handle 和 event sink;调用返回 handle 后宿主可订阅/取消。采用插件 ABI 增量接口及能力检测,旧 ABI 未支持时拒绝新 streaming 模式,保留旧完整返回。qdagent 只挂产品鉴权、event id 持久化和 HTTP 端点,不添加另一个推理循环。
SSE parser 必须处理任意 TCP 分块、UTF-8 跨块、CRLF、多行 data、注释 heartbeat、尾帧、usage-only 帧、[DONE]、连接在结束前关闭。未知可选事件可忽略且记诊断;已知事件无效、异常截断不能静默当成功完成。公共 framing 不包含供应商私有认证信息。
### 5.4 取消与重试
取消 token 传到 HTTP reader、循环、工具边界、子任务。取消后不得启动新模型调用/新工具;正在进行的不可抢占操作返回 `cancelling` 和是否仍运行,结束后才报告 cancelled。明确“停止生成”不能撤销已经完成的外部写入。
调用超时、连接错误、鉴权失败、限流、协议错误分开;错误字段包含 `code,retryable,retry_after_ms?,call_id,attempt_id,usage_state`。不要将原始包含密钥/完整 prompt 的错误直接传到 UI。限流/短暂网络错误按有界策略重试;已产生输出的重试不能接在同一条文字后假装连续,也不能把未知结果的外部工具操作自动重做。
费用价格由应用管理。框架保留 legacy `tokens/token_cost` 并标注 deprecated/单位为 token,新功能只消费 ModelCallUsageV1,避免破坏旧脚本的同时停止误用。
## 6. M03:无执行的嵌入式 View
### 6.1 新入口与纯度约束
新增内部公共服务 `analyze_document → project_read_model → render_fragment`;MLSP 暴露 `view.document`,可选 CLI `view document --json --no-exec` 与它共享实现。不要让新入口先进入 `build_file_view` 再隐藏按钮/清空输出。
读取允许:授权范围内原文/已编译快照/已有验证记录。读取禁止:运行模块入口、加载文档声明插件、执行外国代码、LLM、自动抓取 URL、写回、重新验证执行测试、隐式发布/重编译写盘。静态分析/内存投影允许;缺完整依赖时明确 partial。任何需要执行的行为必须另走显式 execute/verify 产品流程。
当前 `compile_project` 声明不加载插件/不求值/不抓取来源,可复用;缓存写入若需要,作为单独的宿主管理路径,不能把“读文档”隐含写项目目录。
### 6.2 DocumentViewV1 结构
请求由宿主传入受控 project 和文档定位,宿主必须已授权;框架仍做路径边界校验。
```json
{
"method": "view.document",
"params": {
"project": "/authorized/project",
"document": "knowledge/example.mq.md",
"revision": "sha256:requested-source",
"generation": "generation-id-or-null",
"mode": "read",
"node_id": "optional-node",
"include": ["fragment", "outline", "nodes", "refs", "diagnostics"]
}
}
```
```json
{
"schema": "DocumentViewV1",
"schema_version": 1,
"document": {
"id": "project-relative-document-id",
"title": "知识标题",
"source_revision": "sha256:raw-bytes-digest",
"semantic_digest": "sha256:semantic-facts",
"generation_id": null,
"presentation_token": "opaque-token"
},
"execution": {"state": "not_run", "run_id": null},
"render": {"format": "sanitized_html", "fragment": "<article class=\"mq-document\">…</article>", "renderer_version": "1"},
"outline": [],
"nodes": [],
"refs": [],
"diagnostics": [],
"completeness": "complete",
"capabilities": {"read_only": true, "has_semantics": true}
}
```
`generation_id` 来自实际编译投影;即时分析但未编译发布时为 null,同时给 semantic_digest。`source_revision` 是原始文件字节摘要,不能复用当前基于 artifacts 的 `source_digest`;`presentation_token` 覆盖 source revision、generation/semantic digest、依赖版本、renderer/schema、读取策略与请求范围。应用缓存还必须含用户授权范围。
`completeness = complete|partial`;语法损坏时允许返回原文与结构化诊断,但不能展示猜测出来的正常知识图。失败用 error envelope,不能空字符串伪装“文档无内容”。大文档 response 支持 section cursor 和 truncated 标识,限制要可配置并在响应中标明。
### 6.3 统一正文、目录、节点与引用
节点包含 `id,parent_id,kind,label,order,source_span,artifact_id?,ref_ids[],children[]`。kind 包括 heading/prose/table/code/observe/derive/call/iterate/narrate/navigation;正文树来自 parser/classifier/sema 的共同投影,不能只枚举 functions 后把每个步骤标 call。
outline 描述读者文档层级,不简单把函数和对象全铺成同一级。HTML 中节点带 `data-mq-node`,引用带 `data-mq-ref`;实现 payload 按需展开,说明作为正文。动态调用/无法解析引用保持原文并解释原因,不虚构执行边。
首批 renderer 共享 `src/view/document.rs` 的文档表面,standalone/full page 和嵌入模式调用同一 read model;只改变容器和导航外观,不产生第二种语义解释。替换当前 `KnowledgePresentation` 与 `render_knowledge_html` 双路径重复组装。
稳定 ID 的保证范围要写明:
- 同一 revision 再次读取,node/ref id 必须相同。
- 仅前插空行/叙述时,已有命名符号保持 symbol id;步骤 id 由结构身份映射,不能只靠行号。
- 同一行多个节点需要 column/offset/span 区分,不能都映射成 `stmt-N`。
- 匿名步骤重排/重复同名内容可能无法唯一映射;返回 unmapped/ambiguous,不能声称任意编辑下永恒稳定。
- source span 明确 1-based line/column 和 UTF-8 byte offset;JavaScript UTF-16 偏移不直接当 Rust 字节偏移。
旧 `#stmt-{line}` 可保留为当前版本兼容锚点,但不得作为跨版本唯一身份。
### 6.4 嵌入渲染、主题与访问性
fragment 不包含 html/body、应用导航、运行按钮、内联可执行 script、全局 CSS;CSS 以 `.mq-document` namespace 限定。主题暴露 `--mq-bg/surface/text/muted/border/accent/font-body/font-code`,宿主映射自己的设计 tokens;不把独立 View 的全局 `*`/body 样式注入 qdagent。
首批推荐 fragment + 结构化引用 + 宿主事件代理;无需为 iframe 和 React 各重做 parser。可选 iframe embedded shell 必须只读、无重复导航,宿主/子页通信校验 origin/source/nonce/schema,仅允许 open-reference/selection/ready 等已定义消息,不能从内容发任意运行指令。
所有文本/属性转义,允许的 Markdown HTML 白名单净化;链接 scheme 限制;外部资源不默认跨网加载;代码/公式不得执行。静态导出、独立页面、嵌入三者都测试恶意文本/闭合标签/危险 URL。不要把 KnowledgePresentation 内 JSON 的转义当作所有正文 XSS 防护。
标题/列表/表格用语义 HTML;说明链接有可访问名称,焦点可见,视觉排序和键盘顺序一致。折叠只隐藏技术细节,不隐藏理解结论所需的正文。320–720px 容器宽、暗色、200% 缩放和 print 都要能读,不出现遮住正文的关系连线。无 JS 时仍能读正文和普通链接。
## 7. M04:知识、来源与历史引用定位
### 7.1 KnowledgeLocatorV1
统一 locator:`project_id,document_id,source_revision,generation_id?,artifact_id?,node_id?,source_ref_id?,span?,relation`。relation 区分 cites/supports/derived_from/executes。普通链接导航、NarrativeRef 和 BoundCall 都能定位,但语义不同;点击说明永不成为 BoundCall 的执行。
新增 `view.reference_resolve`:接受 locator 或当前文档 ref_id,返回 `resolution = resolved|stale|unresolved|ambiguous|version_unavailable|unsupported`、目标 locator、候选(若有)、定位精度、诊断、读者标签。框架内部路径不作为浏览器 href,应用根据授权映射为 document/source route。
扩展 `artifact.inspect` 的版本参数,旧无版本调用仍查当前;新调用带 expected source/generation mismatch 时不能自动降到当前。来源读取接口可由宿主取实际文件,再调用框架 locator 映射;框架不自动 fetch 外部网页。`mq://` 必须按框架规则解析,不能交给浏览器或把 URI 直接当文件路径。
### 7.2 历史版本与保留责任
生成 artifact/generation 元数据不等于保存了原文历史。框架须输出可保存的 snapshot manifest:原文 revision、依赖 revision、source refs、artifact hashes、renderer/schema version;应用决定哪些源版本随引用保留及保留周期。已有 Git 或内容寻址存储可提供 snapshot resolver;不强制应用部署 Git。
source revision 定位到存储快照后,框架基于该快照解析;缺失任何必要版本时返回 version_unavailable/partial,不读取最新文件填充旧快照。如果只留 artifact 元数据、原始资料已删除,仍可展示历史元数据但明确“原文不可读取”。
知识改名、移动或 source anchor 改变可返回候选映射,只有唯一可靠匹配才自动导航;否则用户选择最新版/候选。保持旧引用与新引用两个对象,防止历史回答证据在后台悄悄改变。
### 7.3 验证和运行事实
节点详情至少携带 purpose/description、signature、applicability、provenance、effects、dependencies,以及 parse/resolve/contract/tests/evidence/execution 各自状态与记录来源。sourced 只能说“已关联来源”;static verify 成功不表示执行成功或适用业务正确。
真实执行证据独立结构 `run_id,artifact_digest,closure_digest,source_revision,generation_id,inputs_digest,executed_at,result_state,evidence_ref`,展示时匹配版本。旧运行结果不涂成当前文档的绿色通过。知识复用必须存在实际 bind/execute trace,不能凭 find 命中显示“已执行”。
View 不运行验证测试以填满状态。若用户点“验证”,应用走已有 verify/policy 门禁并记录新 evidence,之后读取才刷新详情。
### 7.4 错误与授权协作
| 内部结果 | 应用建议呈现 | 不能做 |
|---|---|---|
| unresolved | 无法定位该知识,可读原引用 | 跳转到第一个同名 artifact |
| ambiguous | 显示已授权候选和位置 | 自动选择或暴露不可见候选 |
| stale | 该知识/来源已变化,提供版本选择 | 静默替换旧引用 |
| version_unavailable | 回答所用版本不可用,可查看最新版 | 用最新版伪装历史证据 |
| partial | 部分依赖或来源缺失,说明范围 | 默认绿色通过 |
| permission denied(应用) | 无权访问 | 在错误中泄漏标题、路径、片段 |
用户权限归应用;框架服务在调用边界接受 scoped project/source resolver,不把传来一个 root 视为可遍历整个服务器。所有路径经过 canonicalize/symlink 检查;缓存的语义事实可共享,带原文的响应必须按权限复核,不能只靠 artifact id 缓存授权。
## 8. M05:工作区、笔记与导入所需的通用底座
不为 qdagent 画像加语言特殊节点。通用能力应支持普通文档元信息读取、来源范围、内容修订、diff/patch 的 expected digest、提取格式 capability。
document.extract 继续使用 DocumentSource→ExtractedDocument,报告 format/status/segments/source locator/diagnostics。TXT/MD 可支持;PDF/DOCX/OCR 没有适配器时明确 unsupported/failed,不能将空文本当成功。partial 保留已提取片段及原文范围、遗漏原因;发布政策由应用决定。
同一资料可产生多个知识单元,provenance 必须到具体片段,不能所有函数只指向文件根。原始资料 digest 改动的失效由现有 provenance/closure 保证;应用用户画像也可用普通来源引用,但不自动进入公共知识发布。
ProjectRoot/CompileOptions 显式传递,工作区 A/B 的 source id、输出 generation、缓存、临时目录必须隔离。避免每读一个节点就全项目编译:引入按 project snapshot 复用的分析服务,针对来源/依赖变化失效;并发同 snapshot 请求合并,不能分别写同一个 CURRENT。
上传 multipart/大小限制已有 web 底座,框架只提供安全提取和可消费结果;浏览器上传、作业生命周期、发布确认、权限和历史保存仍在 qdagent。多文件任务桥接采用结构化参数,禁止拼接用户文件名进 shell/JSON 或共用固定请求临时文件。
## 9. 实现落点与兼容策略
| 模块 | 改动方向 | 兼容要求 |
|---|---|---|
| `plugins/llm/src/lib.rs` | adapter boundary、消息数组、usage、event sink、取消 | 保留已有符号;新能力有 probe 和合同版本 |
| `ext/ai/llm.mq.md` 与 `ext/ai/llm/*` | strict selection、结构化 result、public snapshot | 无 selection 的旧调用仍可运行;不得回显密钥 |
| `ext/ai/agent.mq.md` | run context、子工作簿传播、逐调用观测 | 不复制 Agent loop,EFI/verify/publish_gate 不被绕过 |
| `plugins/agent/src/*` | 子任务/恢复上下文、cancel、event identity | resume 不重发已结算请求;不可恢复说明 interrupted |
| `src/host/*`、插件 SDK、web invoke | 请求局部 event/cancel 桥接 | ABI 旧宿主不伪支持;并发请求不混流 |
| `src/view/knowledge_present.rs` | DocumentView read model/兼容 adapter | 旧 presentation v1 不被新字段语义污染 |
| `src/view/document.rs,render.rs,html.rs,mod.rs,output.rs` | 共享纯读取、fragment、scope CSS、定位 | live/debug 与 read 明确分离;旧 no-exec 门禁保留 |
| `src/mlsp.rs` | capabilities、view/document/reference、版本 inspect | 使用现有 envelope;未知 feature 可探测 |
| `src/sema/ir.rs,provenance.rs` | span、snapshot/locator、依赖与来源失效 | 不改变核心语法,不把读请求变为执行 |
| `src/document_extract.rs` | capability 和片段定位完整性 | 保留不支持格式的显式失败 |
测试分层对应 plugin HTTP fixtures、Agent 假模型、MLSP read contracts、web 持续流集成、独立/嵌入 DOM 和浏览器交互。应用共享样本文件必须注明 schema 版本;两仓 PR 在各自文档索引相互链接对应实现说明。
迁移顺序为新增合同/接口 → 旧输出 adapter → qdagent 按 capability 接入 → 切默认 → 后续版本再考虑弃用。不要同时移除 legacy 行为且要求应用一次完成全链升级。发布包对 CLI/ext/native 插件做组合测试,缺插件给功能不可用诊断而非 unknown function 直到用户点击才暴露。
## 10. 交付切片与验收门槛
| 切片 | 前置 | 验收输出 | 支撑应用任务 |
|---|---|---|---|
| M00 合同冻结/探测 | 无 | schemas、fixtures、capabilities、版本和插件清单 | Q00、所有升级探测 |
| M01 多模型隔离 | M00 | 兼容适配器、strict handles、history、子任务选择一致 | Q01/Q02 |
| M02 调用观测与流 | M01 | 每 attempt usage、真流式、取消、错误/重放一致 | Q03/Q06 |
| M03 只读嵌入 View | M00,可与模型线独立 | DocumentViewV1、fragment、目录/节点、无执行证明 | Q04 |
| M04 引用与快照 | M03 | 历史 locator、来源详情、版本失效/歧义 | Q05 |
| M05 文档工作区底座 | M03/M04 + 已有 extract | 范围隔离、提取定位、并发与缓存 | Q07/Q08 |
| M06 集成与性能 | 全部所选切片 | 旧门禁 + 新 contracts + 浏览器与负载报告 | Q09 |
M01 支持多个兼容提供商不等待原生协议全做完;但原生适配器需另列完成状态。M03 不等待计费系统。M04 未完成历史快照前,应用可以显示当前文档,但必须把历史引用标“版本不可用”,不能宣布引用链闭合。
每个切片的 Definition of Done:实际接口可调用、失败合同覆盖、无副作用/隔离测试、文档示例由测试消费、版本能力可探测、qdagent integration fixture 通过。只有可运行的验收证据才可将 proposed 改为 verified。
## 11. 框架测试矩阵
| 编号 | 用例 | 断言 |
|---|---|---|
| F01 | 两端点各两模型、并行 A/B、无密钥本地端点 | 请求 payload/端点/鉴权正确,环境不串扰 |
| F02 | step/plan/router/repair/child/resume | strict selection 和 parent identity 完整,不退回默认模型 |
| F03 | multi-turn、system/tool roles、超限/不支持模态 | 顺序和边界完整,不吞最新 user,不静默降级 |
| F04 | 非流/流式/usage-only/缺 usage/错误后部分 usage | null 与 0 区分,终态和用量独立 |
| F05 | 相同累计 usage 多次、重放、新重试 | 同 attempt revision 覆盖,不重复相加;真重试新增 attempt |
| F06 | 慢供应商、慢消费者、断连、取消、超时 | 完成前有 delta;队列有界;取消不启动后续工具/模型 |
| F07 | 外部工具已写后取消、恢复未知状态 | 不声称撤销,不自动重做可能有副作用的动作 |
| F08 | 带写文件/网络/foreign/插件的文档被阅读 | read API 不加载/运行这些效果,无项目写回 |
| F09 | prose/heading/6 类步骤、同一行多引用、中文 span | HTML/outline/节点/ref 同源,定位唯一准确 |
| F10 | 仅说明变化、依赖变化、旧 SSE 包、无 token | token 正确失效,嵌入 strict 拒绝过期/无身份包 |
| F11 | 来源改删、历史 generation 未留原文、同名歧义 | 明确 stale/unavailable/ambiguous,无猜测替代 |
| F12 | no-js/390px/暗色/长表格/键盘 | 正文可读,可返回,有清晰标题和焦点 |
| F13 | 原始 HTML/危险 URL/闭合 script/越界路径 | 无脚本执行/越权读取/泄漏;不靠应用猜净化 |
| F14 | 两项目同相对路径/并发 100 读取 | scope 缓存隔离,同 snapshot 合并分析 |
| F15 | 编译成功但验证/证据未知、旧执行记录 | 各状态独立;不把 static/sourced 标 verified |
| F16 | CLI/ext/native 版本错配 | capability 早发现不可用,而非运行时静默失败 |
性能目标需实测而非发版文字:固定机器与 release/debug 分开记录,1000 单元分析冷/热、单文档 fragment 体积/时间、并发读取 P95/内存、UI 首屏。应用目标见 Q 文档;如果全量 compile 超时,优先分析重复全库扫描、依赖计算与缓存,不通过删除 fixture 或抬高阈值宣布可用。
## 12. 本次验证结果与已知阻碍
### 12.1 M00–M06 落地证据(2026-10-25 续)
命令(本机):
```bash
cargo test --offline --locked --test workbench_contracts
cargo test --offline --locked -p marqdo_plugin_llm
cargo test --offline --locked --test executable_knowledge f1_agent_eng_bind_gate_mq_offline -- --exact
```
已交付并通过自动化子集(**非** F01–F16 全绿):
| 切片 | 证据 |
|---|---|
| M00 | fixtures + JSON Schema 消费;`capabilities`;`force_unloaded` / `force_*_off`(F16) |
| M01 | strict selection;`RunContextV1` 并行不串扰;llm plugin messages/roles;workbook 禁 `llm.llm`;**F02** `m01_f02_agent_step_selection_and_resume`(mock HTTP + `llm.create` 抗 env poison + resume save/load) |
| M02 | usage null≠0;revision 替换(F05);cancel token(F06/F07);event_sink;web `sse-live` 断连 cancel |
| M03 | `view.document`;CJK outline/nodes;path escape;静态 a11y 子集;F08 无副作用标记;**浏览器** `workbench-docview.html`(`article.mq-document`、0 script、无 `javascript:`) |
| M04 | stale / version_unavailable / **ambiguous**;`SnapshotManifestV1` 输出(retention=application);`artifact.version_mismatch`;embed strict SSE |
| M05 | extract `:B`;analysis_cache begin/finish;**F14 并发 100** 同 snapshot + 跨 project 隔离 |
| M06 | CI 预构建 agent+llm + `workbench_contracts`;Go `drainForward` / cancel-token 桥 |
| Live | `tests/workbench_live.rs`:USTC OpenAI-compatible(`OPENAI_*` env only)`qwen3.8-chat` / `deepseek-flash` selection ask → `pong` + usage;`MARQDO_EXT` 指向仓内 `ext/` 避免 `~/.marqdo/ext` 陈旧遮蔽 |
已知限制(发版前须书面接受或继续补):
- F02 repair/plan **全路径**假模型仍薄于 step+resume;RunContext 三级继承 + skeleton 禁 `llm.llm` + 子任务 env 已有。
- 历史正文由应用按 SnapshotManifest 保留;框架缺快照 → `version_unavailable`(不静默降级)。
- 完整宿主嵌入 F12 / 产品鉴权留给 qdagent;原生适配器 / PDF/OCR 仍 unavailable。
### 12.2 历史基线记录(设计起草时)
起草时 `executable_knowledge` 曾出现 `f1` 缺插件与 `v3_corpus_1000` debug 超时;其后 `ensure_agent_plugin_built` + CI 预构建已修复 f1 路径。1000 单元性能仍按剖面单独记录。
+1 -1
View File
@@ -1 +1 @@
2.1.2
2.1.3
+3 -3
View File
@@ -64,7 +64,7 @@ One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{te
1. `stream`
**evs = > model.stream prompt=`prompt` echo=`echo`**
**text = > llm.collect events=`evs`**
**tokens = 0**
**tokens = None**
**err = None**
**calls = 1**
1. not `text`
@@ -81,7 +81,7 @@ One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{te
1. `tokens`
**_ = 1**
2. *
**tokens = 0**
**tokens = None**
1. `err`
**calls = 0**
2. `fin` == "error"
@@ -183,7 +183,7 @@ Allowlist check, then invoke the runbook `##` via `lib/subtask` (`spawn fn=` →
Build a runnable workbook. `skeleton=single` (default): one worker `step`. `skeleton=dual`: research then write agents. On success the parent chooses DONE; runtime `agent_workbook_solidify` freezes the answer.
**q = > json.quote text=`goal`**
**esc = > json.parse text={"h":"---\ntitle: agent workbook\nimport:\n llm: ext/ai/llm.mq.md\n agent: ext/ai/agent.mq.md\n json: lib/json.mq.md\n sys: lib/sys.mq.md\n wb: lib/writeback.mq.md\n subtask: lib/subtask.mq.md\n---\n\n# Goal\n\nParent goal is embedded as JSON in # main.\n\n## Solidify\n\nWhen the child returns a good value, parent DECISION: DONE — runtime agent_workbook_solidify freezes the answer as **return** (do not invent long FIND/REPLACE).\n\n# main\n\n> llm.load_env\n\n","s1":"\u002a\u002a\u0060model\u0060 = > llm.llm\u002a\u002a\n","s2":"\u002a\u002a\u0060tools\u0060 = > table.empty_list\u002a\u002a\n","s3":"\u002a\u002a\u0060worker\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You are a workbook worker. tools=[] means never CALL. Finish the task with a direct final answer; do not invent tools.\u002a\u002a\n","s4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","s5":"}\u002a\u002a\n","s6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","s7":"\u002a\u002a\u0060out\u0060 = > \u0060worker\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","s7b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","s8":"\u002a\u002a\u0060text\u0060\u002a\u002a\n","d3":"\u002a\u002a\u0060research\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You gather facts for the goal. Be concise. Do not invent tools.\u002a\u002a\n","d3b":"\u002a\u002a\u0060writer\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You write the final answer from research notes. Do not invent tools.\u002a\u002a\n","d4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","d5":"}\u002a\u002a\n","d6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","d7":"\u002a\u002a\u0060notes\u0060 = > \u0060research\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","d8":"\u002a\u002a\u0060ns\u0060 = > json.stringify value=\u0060notes\u0060\u002a\u002a\n","d9":"\u002a\u002a\u0060wt\u0060 = Write the final answer. Research notes:\u002a\u002a\n","d10":"\u002a\u002a\u0060wt\u0060 = \u0060wt\u0060 + \u0060ns\u0060\u002a\u002a\n","d11":"\u002a\u002a\u0060out\u0060 = > \u0060writer\u0060.step task=\u0060wt\u0060 stream=False\u002a\u002a\n","d11b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","d12":"\u002a\u002a\u0060text\u0060\u002a\u002a\n"}**
**esc = > json.parse text={"h":"---\ntitle: agent workbook\nimport:\n llm: ext/ai/llm.mq.md\n agent: ext/ai/agent.mq.md\n json: lib/json.mq.md\n sys: lib/sys.mq.md\n wb: lib/writeback.mq.md\n subtask: lib/subtask.mq.md\n---\n\n# Goal\n\nParent goal is embedded as JSON in # main.\n\n## Solidify\n\nWhen the child returns a good value, parent DECISION: DONE — runtime agent_workbook_solidify freezes the answer as **return** (do not invent long FIND/REPLACE).\n\n# main\n\n> llm.load_env\n\n","s1":"\u002a\u002a\u0060model\u0060 = > llm.create model=\u0060sys.env_get name=\"OPENAI_MODEL\"\u0060 base_url=\u0060sys.env_get name=\"OPENAI_BASE_URL\"\u0060 backend=openai name=workbook\u002a\u002a\n","s2":"\u002a\u002a\u0060tools\u0060 = > table.empty_list\u002a\u002a\n","s3":"\u002a\u002a\u0060worker\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You are a workbook worker. tools=[] means never CALL. Finish the task with a direct final answer; do not invent tools.\u002a\u002a\n","s4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","s5":"}\u002a\u002a\n","s6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","s7":"\u002a\u002a\u0060out\u0060 = > \u0060worker\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","s7b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","s8":"\u002a\u002a\u0060text\u0060\u002a\u002a\n","d3":"\u002a\u002a\u0060research\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You gather facts for the goal. Be concise. Do not invent tools.\u002a\u002a\n","d3b":"\u002a\u002a\u0060writer\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You write the final answer from research notes. Do not invent tools.\u002a\u002a\n","d4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","d5":"}\u002a\u002a\n","d6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","d7":"\u002a\u002a\u0060notes\u0060 = > \u0060research\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","d8":"\u002a\u002a\u0060ns\u0060 = > json.stringify value=\u0060notes\u0060\u002a\u002a\n","d9":"\u002a\u002a\u0060wt\u0060 = Write the final answer. Research notes:\u002a\u002a\n","d10":"\u002a\u002a\u0060wt\u0060 = \u0060wt\u0060 + \u0060ns\u0060\u002a\u002a\n","d11":"\u002a\u002a\u0060out\u0060 = > \u0060writer\u0060.step task=\u0060wt\u0060 stream=False\u002a\u002a\n","d11b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","d12":"\u002a\u002a\u0060text\u0060\u002a\u002a\n"}**
**h = > json.get value=`esc` key="h"**
**s1 = > json.get value=`esc` key="s1"**
**s2 = > json.get value=`esc` key="s2"**
+30 -15
View File
@@ -156,6 +156,7 @@ Prompt artifact helper: `path=` loads a file; `text=` strips `type: prompt` fron
Module convenience: create a default handle and return **answer text**. Pass `path=` to load a prompt document, `prompt=` text, or omit both to use the entry `.mq.md` as a prompt artifact (`type: prompt` body via `sys.module_source`).
**p = `prompt`**
1. `path`
**p = > prompt_load path=`path`**
2. `prompt`
@@ -268,20 +269,26 @@ Prefer declaring `model` / `api_key` / `base_url` in frontmatter Binding, then u
## ask
+ `prompt`=None
+ `path`=None
+ `messages`=None
+ `selection`=None
+ `strict_selection`=False
Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`).
Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`). Pass `messages=` for multi-turn history (M01). `selection=` + `strict_selection=True` freeze request identity without env fallback.
1. `path`
**p = `prompt`**
1. `messages`
**p = None**
2. `path`
**p = > prompt_load path=`path`**
2. `prompt`
3. `prompt`
**p = `prompt`**
3. *
4. *
**raw = > sys.module_source**
1. not `raw`
**p = ""**
2. *
**p = > prompt_body src=`raw`**
**pack = > self.complete_result prompt=`p` stream=False echo=False**
**pack = > self.complete_result prompt=`p` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=False echo=False**
**text = > json.get value=`pack` key="text"**
**usage = > json.get value=`pack` key="usage"**
**finish = > json.get value=`pack` key="finish"**
@@ -312,11 +319,16 @@ inside the table is unsafe.
**nm = > json.get value=`self` key="name"**
1. `usage`
**tt = > json.get value=`usage` key="total_tokens"**
**ustate = > json.get value=`usage` key="usage_state"**
2. *
**tt = 0**
1. not `tt`
**tt = 0**
2. *
**tt = None**
**ustate = "unknown"**
1. `ustate` == "unknown"
Keep null — never coerce missing usage to zero tokens (M02 / F04).
**tt = None**
2. not `tt`
**tt = None**
3. *
**_ = 1**
1. `err`
**calls = 0**
@@ -325,9 +337,9 @@ inside the table is unsafe.
`result` =
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error |
|------|-------|--------|---------|-------|------|-----------|--------|------------|-------|
| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | `calls` | `tt` | `tc` | `err` |
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error | selection |
|------|-------|--------|---------|-------|------|-----------|--------|------------|-------|-----------|
| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | `calls` | `tt` | `tc` | `err` | `selection` |
*result*
@@ -352,13 +364,16 @@ Semantic stream: return the event list (use `llm.collect` to reduce to text). Om
*> self.complete prompt=`p` stream=True echo=`echo`*
## complete_result
+ `prompt`
+ `prompt`=None
+ `messages`=None
+ `selection`=None
+ `strict_selection`=False
+ `stream`=False
+ `echo`=False
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`.
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`. Prefer `messages=` for multi-turn; optional `selection=` (ModelSelectionV1) with `strict_selection=True` for request-scoped identity (M01).
*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` stream=`stream` echo=`echo` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))*
*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=`stream` echo=`echo` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))*
## complete
+ `prompt`
+15 -6
View File
@@ -73,7 +73,11 @@ Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list).
+ `base_url`
+ `api_key`
+ `model`
+ `prompt`
+ `prompt`=None
+ `messages`=None
+ `selection`=None
+ `strict_selection`=False
+ `auth_mode`=None
+ `stream`=False
+ `echo`=False
+ `suffix`=/chat/completions
@@ -81,20 +85,25 @@ Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list).
POST `/chat/completions` (or stream SSE). Prefers `plugins/llm` ABI (`llm_chat_completions`) when the native plugin is resolvable; otherwise `lib/net` HTTP/SSE. Non-stream → `{text,usage,finish,tool_calls}`; stream → `{events,finish}`. Soft transport / HTTP failures return `finish=error` packs (never abort the VM).
Pass either `messages=` (role/content list) or legacy `prompt=` (single user message). Optional `selection=` is a ModelSelectionV1 snapshot; `strict_selection=True` refuses env fallbacks inside the plugin.
**p = > plugin.native_path name="llm"**
1. `p`
> plugin.load path=`p`
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`*
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` auth_mode=`auth_mode` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`*
2. *
**url = base_url + suffix**
**auth = bearer + api_key**
**headers = > table.put in=None at="Authorization" value=`auth`**
`messages` =
1. not `messages`
`messages` =
| @ | role | content |
|---|------|---------|
| 1 | user | `prompt` |
| @ | role | content |
|---|------|---------|
| 1 | user | `prompt` |
2. *
**_ = 1**
`req` =
+373 -22
View File
@@ -44,21 +44,40 @@ fn read_limited(mut r: impl Read) -> Result<String, String> {
String::from_utf8(buf).map_err(|e| e.to_string())
}
/// Prefer null over zero when the provider omitted a field (ModelCallUsageV1 / F04).
fn token_or_null(v: Option<i64>) -> Value {
match v {
Some(n) if n >= 0 => json!(n),
_ => Value::Null,
}
}
fn usage_from(data: &Value) -> Value {
let u = data.get("usage").cloned().unwrap_or(Value::Null);
let pt = u.get("prompt_tokens").and_then(|v| v.as_i64()).unwrap_or(0);
let ct = u
.get("completion_tokens")
.and_then(|v| v.as_i64())
.unwrap_or(0);
if u.is_null() || !u.is_object() {
return json!({
"prompt_tokens": Value::Null,
"completion_tokens": Value::Null,
"total_tokens": Value::Null,
"usage_state": "unknown",
"measurement_source": "none",
});
}
let pt = u.get("prompt_tokens").and_then(|v| v.as_i64());
let ct = u.get("completion_tokens").and_then(|v| v.as_i64());
let tt = u
.get("total_tokens")
.and_then(|v| v.as_i64())
.unwrap_or(pt + ct);
.or_else(|| match (pt, ct) {
(Some(a), Some(b)) => Some(a + b),
_ => None,
});
json!({
"prompt_tokens": pt,
"completion_tokens": ct,
"total_tokens": tt,
"prompt_tokens": token_or_null(pt),
"completion_tokens": token_or_null(ct),
"total_tokens": token_or_null(tt),
"usage_state": "final",
"measurement_source": "provider",
})
}
@@ -72,10 +91,34 @@ fn tool_calls_from(data: &Value) -> Value {
.unwrap_or_else(|| json!([]))
}
fn cancel_requested() -> bool {
if std::env::var("MARQDO_CANCEL")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false)
{
return true;
}
if let Ok(tok) = std::env::var("MARQDO_CANCEL_TOKEN") {
let path = std::path::Path::new(&tok);
if path.is_file() {
if let Ok(body) = std::fs::read_to_string(path) {
let t = body.trim();
return t.is_empty() || t == "1" || t.eq_ignore_ascii_case("cancelled");
}
return true;
}
}
false
}
fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, String> {
let mut events = Vec::new();
let mut data_lines: Vec<String> = Vec::new();
for line in reader.lines() {
if cancel_requested() {
events.push(json!({"type": "cancelled", "message": "MARQDO_CANCEL"}));
return Ok(events);
}
let line = line.map_err(|e| e.to_string())?;
if echo {
eprint!("{line}\n");
@@ -107,11 +150,106 @@ fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, Stri
Ok(events)
}
fn chat_completions(args: &Value) -> Result<Value, String> {
let base_url = arg_text(args, "base_url")?;
let api_key = arg_text(args, "api_key").unwrap_or("");
let model = arg_text(args, "model")?;
fn build_messages(args: &Value) -> Result<Value, String> {
if let Some(arr) = args.get("messages").and_then(|v| v.as_array()) {
if arr.is_empty() {
return Err("messages array must not be empty".into());
}
// Validate minimal role/content shape; reject unsupported part types loudly.
for (i, m) in arr.iter().enumerate() {
let role = m.get("role").and_then(|r| r.as_str()).unwrap_or("");
if role.is_empty() {
return Err(format!("llm.unsupported_input: messages[{i}].role missing"));
}
if let Some(parts) = m.get("content").and_then(|c| c.as_array()) {
for (j, p) in parts.iter().enumerate() {
let ty = p.get("type").and_then(|t| t.as_str()).unwrap_or("text");
if ty != "text" {
return Err(format!(
"llm.unsupported_input: messages[{i}].content[{j}] type={ty}"
));
}
}
}
}
return Ok(Value::Array(arr.clone()));
}
let prompt = arg_text(args, "prompt")?;
Ok(json!([{"role": "user", "content": prompt}]))
}
fn chat_completions(args: &Value) -> Result<Value, String> {
if cancel_requested() {
return Ok(json!({
"text": "",
"usage": {
"prompt_tokens": Value::Null,
"completion_tokens": Value::Null,
"total_tokens": Value::Null,
"usage_state": "unknown",
"measurement_source": "none",
},
"finish": "cancelled",
"error": "cancelled",
"tool_calls": [],
"transport": "plugin",
}));
}
// Strict selection mode: refuse env fallback when selection.strict is set.
let strict = opt_bool(args, "strict_selection", false)
|| args
.get("selection")
.and_then(|s| s.get("strict"))
.and_then(|v| v.as_bool())
.unwrap_or(false);
let selection = args.get("selection").cloned();
if strict {
if selection.is_none() {
return Err("llm.selection_missing:selection".into());
}
let sel = selection.as_ref().unwrap();
for key in ["selection_id", "provider_id", "model_id", "upstream_model_id"] {
if sel
.get(key)
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.is_none()
{
return Err(format!("llm.selection_missing:{key}"));
}
}
}
let base_url = match arg_text(args, "base_url") {
Ok(u) => u,
Err(_) if strict => return Err("llm.selection_missing:base_url".into()),
Err(e) => return Err(e),
};
let auth_mode = opt_text(args, "auth_mode")
.or_else(|| {
selection
.as_ref()
.and_then(|s| s.get("auth_mode"))
.and_then(|v| v.as_str())
})
.unwrap_or("api_key");
let api_key = if auth_mode == "none" {
""
} else {
arg_text(args, "api_key").unwrap_or("")
};
if strict && auth_mode != "none" && api_key.is_empty() {
return Err("llm.selection_missing:api_key".into());
}
let model = selection
.as_ref()
.and_then(|s| s.get("upstream_model_id"))
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
.or_else(|| arg_text(args, "model").ok().map(|s| s.to_string()))
.ok_or_else(|| "missing text `model`".to_string())?;
let messages = build_messages(args)?;
let stream = opt_bool(args, "stream", false);
let echo = opt_bool(args, "echo", false);
let suffix = opt_text(args, "suffix").unwrap_or("/chat/completions");
@@ -123,11 +261,20 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
}
let mut req_body = Map::new();
req_body.insert("model".into(), Value::String(model.to_string()));
req_body.insert(
"messages".into(),
json!([{"role": "user", "content": prompt}]),
);
req_body.insert("model".into(), Value::String(model.clone()));
req_body.insert("messages".into(), messages);
if let Some(params) = selection
.as_ref()
.and_then(|s| s.get("parameters"))
.and_then(|p| p.as_object())
{
if let Some(max) = params.get("max_output_tokens").and_then(|v| v.as_u64()) {
req_body.insert("max_tokens".into(), json!(max));
}
if let Some(temp) = params.get("temperature") {
req_body.insert("temperature".into(), temp.clone());
}
}
if stream {
req_body.insert("stream".into(), Value::Bool(true));
}
@@ -193,11 +340,19 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
if status != 200 {
return Ok(json!({
"text": "",
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
"usage": {
"prompt_tokens": Value::Null,
"completion_tokens": Value::Null,
"total_tokens": Value::Null,
"usage_state": "unknown",
"measurement_source": "none",
},
"finish": "error",
"error": format!("HTTP {status}: {text}"),
"tool_calls": [],
"transport": "plugin",
"requested_model": model,
"resolved_model": model,
}));
}
let data: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?;
@@ -208,9 +363,12 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
.and_then(|c0| c0.get("message"))
.cloned()
.unwrap_or(json!({}));
// Some gateways (reasoning models) return null content + reasoning_content.
let content = msg
.get("content")
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.or_else(|| msg.get("reasoning_content").and_then(|v| v.as_str()))
.unwrap_or("")
.to_string();
let fr = data
@@ -221,32 +379,62 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
.and_then(|v| v.as_str())
.unwrap_or("stop")
.to_string();
let resolved = data
.get("model")
.and_then(|v| v.as_str())
.unwrap_or(model.as_str())
.to_string();
Ok(json!({
"text": content,
"usage": usage_from(&data),
"finish": fr,
"tool_calls": tool_calls_from(&data),
"transport": "plugin",
"requested_model": model,
"resolved_model": resolved,
"selection": selection.as_ref().map(|s| {
let mut snap = s.clone();
if let Some(obj) = snap.as_object_mut() {
obj.remove("api_key");
}
snap
}),
}))
}
Err(ureq::Error::Status(code, r)) => {
let text = read_limited(r.into_reader()).unwrap_or_default();
Ok(json!({
"text": "",
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
"usage": {
"prompt_tokens": Value::Null,
"completion_tokens": Value::Null,
"total_tokens": Value::Null,
"usage_state": "unknown",
"measurement_source": "none",
},
"finish": "error",
"error": format!("HTTP {code}: {text}"),
"tool_calls": [],
"transport": "plugin",
"requested_model": model,
"resolved_model": model,
}))
}
Err(e) => Ok(json!({
"text": "",
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
"usage": {
"prompt_tokens": Value::Null,
"completion_tokens": Value::Null,
"total_tokens": Value::Null,
"usage_state": "unknown",
"measurement_source": "none",
},
"finish": "error",
"error": format!("http POST {url}: {e}"),
"tool_calls": [],
"transport": "plugin",
"requested_model": model,
"resolved_model": model,
})),
}
}
@@ -274,5 +462,168 @@ unsafe extern "C" fn llm_ping(
marqdo_plugin!(marqdo_plugin_sdk::ABI_VERSION, [
("llm_ping", "", llm_ping),
("llm_chat_completions", "base_url,api_key,model,prompt,stream,echo,suffix,bearer", llm_chat_completions),
("llm_chat_completions", "base_url,api_key,model,prompt,messages,selection,strict_selection,auth_mode,stream,echo,suffix,bearer", llm_chat_completions),
]);
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn f03_messages_preserve_roles_and_reject_image() {
let msgs = build_messages(&json!({
"messages": [
{"role": "system", "content": "rules"},
{"role": "user", "content": "你好"},
{"role": "assistant", "content": "先前"},
{"role": "user", "content": "最新用户"}
]
}))
.unwrap();
let arr = msgs.as_array().unwrap();
assert_eq!(arr.len(), 4);
assert_eq!(arr[3]["role"], "user");
assert_eq!(arr[3]["content"], "最新用户");
let err = build_messages(&json!({
"messages": [{"role": "user", "content": [{"type": "image", "url": "x"}]}]
}))
.unwrap_err();
assert!(err.contains("unsupported_input"), "{err}");
}
#[test]
fn f01_strict_selection_does_not_invent_env_model() {
std::env::remove_var("MARQDO_CANCEL");
std::env::remove_var("MARQDO_CANCEL_TOKEN");
let err = chat_completions(&json!({
"strict_selection": true,
"base_url": "http://127.0.0.1:9",
"api_key": "k",
"prompt": "hi"
// missing selection → must fail before any HTTP
}))
.unwrap_err();
assert!(err.contains("selection_missing"), "{err}");
}
#[test]
fn f06_cancel_short_circuits_before_http() {
// Prefer token file over process-global MARQDO_CANCEL to avoid parallel test races.
let tok = std::env::temp_dir().join(format!("llm-cancel-{}", std::process::id()));
std::fs::write(&tok, "cancelled").unwrap();
std::env::set_var("MARQDO_CANCEL_TOKEN", &tok);
let out = chat_completions(&json!({
"base_url": "http://127.0.0.1:9",
"api_key": "k",
"model": "m",
"prompt": "hi"
}));
std::env::remove_var("MARQDO_CANCEL_TOKEN");
let _ = std::fs::remove_file(&tok);
let out = out.unwrap();
assert_eq!(out["finish"], "cancelled");
}
/// F01: two local endpoints receive distinct models/auth — no cross-talk.
#[test]
fn f01_dual_endpoint_isolation() {
use std::io::{Read, Write};
use std::net::TcpListener;
use std::sync::{Arc, Mutex};
use std::thread;
fn spawn_mock(tag: &'static str, seen: Arc<Mutex<Vec<String>>>) -> String {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
thread::spawn(move || {
if let Ok((mut stream, _)) = listener.accept() {
let mut raw = Vec::new();
let mut buf = [0u8; 4096];
loop {
let n = stream.read(&mut buf).unwrap_or(0);
if n == 0 {
break;
}
raw.extend_from_slice(&buf[..n]);
if let Some(pos) = raw.windows(4).position(|w| w == b"\r\n\r\n") {
let header = String::from_utf8_lossy(&raw[..pos]);
let cl = header
.lines()
.find_map(|l| l.strip_prefix("Content-Length:"))
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(0);
let body_start = pos + 4;
while raw.len() < body_start + cl {
let n = stream.read(&mut buf).unwrap_or(0);
if n == 0 {
break;
}
raw.extend_from_slice(&buf[..n]);
}
break;
}
}
let req = String::from_utf8_lossy(&raw).to_string();
seen.lock().unwrap().push(format!("{tag}:{req}"));
let body = r#"{"choices":[{"message":{"content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1,"total_tokens":2}}"#;
let resp = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}",
body.len(),
body
);
let _ = stream.write_all(resp.as_bytes());
}
});
format!("http://{addr}")
}
let seen_a = Arc::new(Mutex::new(Vec::new()));
let seen_b = Arc::new(Mutex::new(Vec::new()));
let url_a = spawn_mock("A", seen_a.clone());
let url_b = spawn_mock("B", seen_b.clone());
let out_a = chat_completions(&json!({
"base_url": url_a,
"api_key": "key-a",
"model": "model-a",
"auth_mode": "api_key",
"messages": [{"role":"user","content":"from-a"}],
"selection": {
"selection_id": "sel-a",
"provider_id": "p-a",
"model_id": "m-a",
"upstream_model_id": "model-a",
"strict": true
},
"strict_selection": true
}))
.unwrap();
let out_b = chat_completions(&json!({
"base_url": url_b,
"api_key": "key-b",
"model": "model-b",
"auth_mode": "api_key",
"messages": [{"role":"user","content":"from-b"}],
"selection": {
"selection_id": "sel-b",
"provider_id": "p-b",
"model_id": "m-b",
"upstream_model_id": "model-b",
"strict": true
},
"strict_selection": true
}))
.unwrap();
assert_eq!(out_a["finish"], "stop", "{out_a}");
assert_eq!(out_b["finish"], "stop", "{out_b}");
let a = seen_a.lock().unwrap().join("\n");
let b = seen_b.lock().unwrap().join("\n");
assert!(a.contains("model-a"), "A saw wrong model: {a}");
assert!(a.contains("key-a") || a.contains("Authorization"), "{a}");
assert!(!a.contains("model-b"), "A must not see B's model: {a}");
assert!(b.contains("model-b"), "B saw wrong model: {b}");
assert!(!b.contains("model-a"), "B must not see A's model: {b}");
}
}
@@ -0,0 +1,29 @@
package httpx
import (
"os"
"path/filepath"
"testing"
)
// F06: client-disconnect cancel bridge writes token file + MARQDO_CANCEL for llm plugin.
func TestPropagateCancelSignal(t *testing.T) {
dir := t.TempDir()
tok := filepath.Join(dir, "cancel.tok")
if err := os.WriteFile(tok, []byte("cancelled"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MARQDO_CANCEL_TOKEN", tok)
t.Setenv("MARQDO_CANCEL", "1")
if os.Getenv("MARQDO_CANCEL") != "1" {
t.Fatalf("MARQDO_CANCEL not set")
}
body, err := os.ReadFile(tok)
if err != nil {
t.Fatal(err)
}
if string(body) != "cancelled" {
t.Fatalf("token body=%q", body)
}
}
@@ -0,0 +1,41 @@
package httpx
import (
"os"
"path/filepath"
"testing"
)
func TestDrainForwardReadsIncrementalLines(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "evt.ndjson")
if err := os.WriteFile(path, []byte("{\"type\":\"a\"}\n{\"type\":\"b\"}\n"), 0o600); err != nil {
t.Fatal(err)
}
var lines []string
off := drainForward(path, 0, func(line string) {
lines = append(lines, line)
})
if len(lines) != 2 {
t.Fatalf("got %d lines %#v", len(lines), lines)
}
if off <= 0 {
t.Fatalf("offset=%d", off)
}
// Append after first drain — second call with offset must only see new line.
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
t.Fatal(err)
}
if _, err := f.WriteString("{\"type\":\"c\"}\n"); err != nil {
t.Fatal(err)
}
_ = f.Close()
var more []string
off2 := drainForward(path, off, func(line string) {
more = append(more, line)
})
if len(more) != 1 || more[0] != "{\"type\":\"c\"}" {
t.Fatalf("incremental got %#v off=%d→%d", more, off, off2)
}
}
+170
View File
@@ -1,11 +1,15 @@
package httpx
import (
"bufio"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strings"
"sync"
"time"
"github.com/marqdo/marqdo/plugins/web/internal/plugin"
)
@@ -115,6 +119,15 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
}
args := invokeArgsFromPayload(payload)
// M02: live SSE — stream host_stream_publish events while CallLibPath runs.
// return=sse-live|sse_live uses MARQDO_EVENT_FORWARD NDJSON bridge (request-local).
liveSSE := route.ReturnMode == "sse-live" || route.ReturnMode == "sse_live"
if liveSSE {
st.handleInvokeLiveSSE(w, r, route, args)
return
}
value, err := plugin.CallLibPath(route.FnPath, args)
if err != nil {
code := http.StatusInternalServerError
@@ -176,3 +189,160 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
writeJSON(w, code, body)
}
}
// handleInvokeLiveSSE streams RuntimeEvent-shaped lines while the lib call runs.
// Client disconnect sets MARQDO_CANCEL + cancel token file so the llm plugin can stop.
func (st *state) handleInvokeLiveSSE(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any) {
dir := os.TempDir()
fwd, err := os.CreateTemp(dir, "mq-evt-*.ndjson")
if err != nil {
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("event forward: %v", err))
return
}
fwdPath := fwd.Name()
_ = fwd.Close()
defer os.Remove(fwdPath)
prev := os.Getenv("MARQDO_EVENT_FORWARD")
_ = os.Setenv("MARQDO_EVENT_FORWARD", fwdPath)
cancelFile, cancelErr := os.CreateTemp(dir, "mq-cancel-*.tok")
cancelPath := ""
if cancelErr == nil {
cancelPath = cancelFile.Name()
_ = cancelFile.Close()
_ = os.Setenv("MARQDO_CANCEL_TOKEN", cancelPath)
}
defer func() {
if prev == "" {
_ = os.Unsetenv("MARQDO_EVENT_FORWARD")
} else {
_ = os.Setenv("MARQDO_EVENT_FORWARD", prev)
}
_ = os.Unsetenv("MARQDO_CANCEL_TOKEN")
_ = os.Unsetenv("MARQDO_CANCEL")
if cancelPath != "" {
_ = os.Remove(cancelPath)
}
}()
w.Header().Set("Content-Type", "text/event-stream; charset=utf-8")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
w.Header().Set("X-Accel-Buffering", "no")
w.WriteHeader(http.StatusOK)
flusher, _ := w.(http.Flusher)
writeEvent := func(payload any) {
b, err := json.Marshal(payload)
if err != nil {
return
}
_, _ = io.WriteString(w, "data: "+string(b)+"\n\n")
if flusher != nil {
flusher.Flush()
}
}
writeEvent(map[string]any{
"schema": "RuntimeEventV1",
"schema_version": 1,
"type": "run.started",
"payload": map[string]any{"fn": route.FnPath, "bridge": "sse-live"},
})
type callResult struct {
value any
err error
}
done := make(chan callResult, 1)
go func() {
v, e := plugin.CallLibPath(route.FnPath, args)
done <- callResult{value: v, err: e}
}()
var mu sync.Mutex
offset := int64(0)
seq := uint64(1)
poll := time.NewTicker(25 * time.Millisecond)
defer poll.Stop()
ctx := r.Context()
for {
select {
case <-ctx.Done():
// Propagate cancel to in-flight LLM/plugin via token file + env.
_ = os.Setenv("MARQDO_CANCEL", "1")
if cancelPath != "" {
_ = os.WriteFile(cancelPath, []byte("cancelled"), 0o600)
}
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.interrupted",
"payload": map[string]any{"reason": "client_disconnect", "cancel_propagated": cancelPath != ""},
})
return
case res := <-done:
// Drain remaining forward lines once.
mu.Lock()
offset = drainForward(fwdPath, offset, func(line string) {
seq++
var obj any
if json.Unmarshal([]byte(line), &obj) == nil {
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1,
"seq": seq, "type": "message.delta", "payload": obj,
})
}
})
mu.Unlock()
if res.err != nil {
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.failed",
"payload": map[string]any{"error": res.err.Error()},
})
} else {
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.completed",
"payload": map[string]any{"result": res.value},
})
}
_, _ = io.WriteString(w, "data: [DONE]\n\n")
if flusher != nil {
flusher.Flush()
}
return
case <-poll.C:
mu.Lock()
offset = drainForward(fwdPath, offset, func(line string) {
seq++
var obj any
if json.Unmarshal([]byte(line), &obj) == nil {
writeEvent(map[string]any{
"schema": "RuntimeEventV1", "schema_version": 1,
"seq": seq, "type": "message.delta", "payload": obj,
})
}
})
mu.Unlock()
}
}
}
func drainForward(path string, offset int64, onLine func(string)) int64 {
f, err := os.Open(path)
if err != nil {
return offset
}
defer f.Close()
if _, err := f.Seek(offset, io.SeekStart); err != nil {
return offset
}
sc := bufio.NewScanner(f)
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
offset += int64(len(sc.Bytes()) + 1)
if line == "" {
continue
}
onLine(line)
}
return offset
}
+9 -3
View File
@@ -193,8 +193,12 @@ fn extract_markdown_or_text(
) -> ExtractedDocument {
let mut segments = Vec::new();
let mut n = 0u32;
let mut byte_off = 0usize;
for (i, line) in text.lines().enumerate() {
let line_no = i + 1;
let line_bytes = line.len();
// M05: stable fragment locator with 1-based line + UTF-8 byte offset.
let loc = |kind: &str| format!("{kind}:L{line_no}:B{byte_off}");
if let Some(rest) = line.trim().strip_prefix('#') {
n += 1;
let heading = rest.trim_start_matches('#').trim();
@@ -202,7 +206,7 @@ fn extract_markdown_or_text(
id: format!("seg-{n}"),
kind: "heading".into(),
text: heading.to_string(),
locator: format!("L{line_no}"),
locator: loc("heading"),
confidence: 1.0,
});
} else if line.trim().starts_with('|') {
@@ -211,7 +215,7 @@ fn extract_markdown_or_text(
id: format!("seg-{n}"),
kind: "table_row".into(),
text: line.to_string(),
locator: format!("L{line_no}"),
locator: loc("table_row"),
confidence: 0.9,
});
} else if !line.trim().is_empty() {
@@ -220,10 +224,12 @@ fn extract_markdown_or_text(
id: format!("seg-{n}"),
kind: "paragraph".into(),
text: line.to_string(),
locator: format!("L{line_no}"),
locator: loc("paragraph"),
confidence: 1.0,
});
}
// +1 for '\n' when present in original (lines() strips it).
byte_off += line_bytes + 1;
}
ExtractedDocument {
source_uri: uri.to_string(),
+73
View File
@@ -331,6 +331,7 @@ fn spawn_file(
));
let _ = std::fs::write(&events_path, "");
cmd.env("MARQDO_EVENT_FORWARD", &events_path);
apply_run_selection_env(&mut cmd);
let mut child = cmd
.spawn()
@@ -693,3 +694,75 @@ fn require<'a>(bound: &'a HashMap<String, Value>, key: &str) -> Result<&'a Value
.get(key)
.ok_or_else(|| format!("host call missing `{key}`"))
}
/// Propagate run/selection identity when parent has them (api_key stays via process inheritance).
fn apply_run_selection_env(cmd: &mut Command) {
for key in [
"MARQDO_RUN_ID",
"MARQDO_PARENT_RUN_ID",
"MARQDO_TURN_ID",
"MARQDO_WORKSPACE_SCOPE",
"MARQDO_CANCEL_TOKEN",
"MARQDO_LLM_MODEL",
"MARQDO_LLM_BASE_URL",
"OPENAI_MODEL",
"OPENAI_BASE_URL",
] {
if let Ok(v) = std::env::var(key) {
cmd.env(key, v);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
use std::process::Command;
#[test]
fn f02_apply_run_selection_env_forwards_parent_identity() {
std::env::set_var("MARQDO_RUN_ID", "run-parent");
std::env::set_var("OPENAI_MODEL", "child-model");
std::env::set_var("OPENAI_BASE_URL", "http://127.0.0.1:7010");
// Must never require writing secrets into child env map for selection.
std::env::remove_var("OPENAI_API_KEY");
let mut cmd = Command::new("true");
apply_run_selection_env(&mut cmd);
// Command doesn't expose env for inspection portably; re-check via helper contract:
let keys: HashMap<&str, String> = [
"MARQDO_RUN_ID",
"OPENAI_MODEL",
"OPENAI_BASE_URL",
]
.into_iter()
.filter_map(|k| std::env::var(k).ok().map(|v| (k, v)))
.collect();
assert_eq!(keys.get("MARQDO_RUN_ID").map(String::as_str), Some("run-parent"));
assert_eq!(keys.get("OPENAI_MODEL").map(String::as_str), Some("child-model"));
assert_eq!(
keys.get("OPENAI_BASE_URL").map(String::as_str),
Some("http://127.0.0.1:7010")
);
std::env::remove_var("MARQDO_RUN_ID");
std::env::remove_var("OPENAI_MODEL");
std::env::remove_var("OPENAI_BASE_URL");
let _ = cmd; // ensure apply didn't panic
}
#[test]
fn f02_agent_skeleton_has_no_llm_llm() {
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let agent = std::fs::read_to_string(root.join("ext/ai/agent.mq.md")).unwrap();
assert!(
!agent.contains("llm.llm"),
"render_workbook_skeleton must not hardcode llm.llm"
);
assert!(
agent.contains("llm.create") && agent.contains("OPENAI_MODEL"),
"skeleton should create model from env selection"
);
}
}
+1
View File
@@ -37,6 +37,7 @@ pub mod lex;
pub mod linalg_ops;
pub mod load;
pub mod mlsp;
pub mod workbench;
pub mod parse;
pub mod project;
pub mod repair;
+67
View File
@@ -529,6 +529,23 @@ enum ViewAction {
#[arg(short = 'o', long = "out", required = true)]
out: PathBuf,
},
/// Read-only DocumentViewV1 (never executes). JSON to stdout or `--out`.
Document {
/// Project root (defaults to cwd / CLI path)
#[arg(long)]
project: Option<PathBuf>,
/// Document path relative to project
#[arg(long)]
document: String,
#[arg(long)]
json: bool,
/// Optional HTML wrapper for browser inspection (embed fragment only)
#[arg(long, value_name = "FILE")]
out: Option<PathBuf>,
},
}
fn main() -> ExitCode {
@@ -1105,6 +1122,56 @@ fn try_main(cli: Cli) -> Result<i32> {
})?;
Ok(0)
}
Some(ViewAction::Document {
project,
document,
json,
out,
}) => {
let project = project
.or(path)
.unwrap_or_else(|| PathBuf::from("."));
let req = marqdo::workbench::DocumentViewRequest {
project,
document,
revision: None,
generation: None,
node_id: None,
include: vec![
"fragment".into(),
"outline".into(),
"nodes".into(),
"refs".into(),
"diagnostics".into(),
],
};
let view = marqdo::workbench::build_document_view(&req).map_err(|e| {
anyhow::anyhow!("{}", e)
})?;
if let Some(out_path) = out {
let fragment = view
.pointer("/render/fragment")
.and_then(|v| v.as_str())
.unwrap_or("");
let title = view
.pointer("/document/title")
.and_then(|v| v.as_str())
.unwrap_or("DocumentView");
let html = format!(
"<!doctype html><html><head><meta charset=\"utf-8\"/><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"/><title>{title}</title><style>:root{{--mq-bg:#f7f5f0;--mq-text:#1a1a1a;--mq-muted:#5c5c5c;--mq-border:#d0cbc2;--mq-accent:#0b5fff;--mq-font-body:\"Iowan Old Style\",\"Palatino Linotype\",Palatino,serif;--mq-font-code:ui-monospace,Consolas,monospace}}body{{margin:0;background:var(--mq-bg);color:var(--mq-text);font:16px/1.75 var(--mq-font-body)}}.wrap{{max-width:46rem;margin:0 auto;padding:1.5rem}} .mq-document a{{color:var(--mq-accent)}}</style></head><body><div class=\"wrap\">{fragment}</div></body></html>"
);
std::fs::write(&out_path, html)?;
if json {
println!("{}", serde_json::to_string_pretty(&view)?);
} else {
println!("wrote {}", out_path.display());
}
} else {
println!("{}", serde_json::to_string_pretty(&view)?);
}
let _ = json;
Ok(0)
}
None => {
serve(ViewOptions {
path: path.unwrap_or_else(|| PathBuf::from(".")),
+112 -2
View File
@@ -68,17 +68,34 @@ fn dispatch(req: &Value) -> Result<Value, Value> {
Some("document.extract") => document_extract(&params),
Some("artifact.patch") => artifact_patch(&params),
Some("artifact.extract_pure") => artifact_extract_pure(&params),
// Workbench M00–M04: capability probe + read-only view + locator.
Some("capabilities") => Ok(crate::workbench::probe_capabilities(&params)),
#[cfg(feature = "view")]
Some("view.document") => view_document(&params),
#[cfg(not(feature = "view"))]
Some("view.document") => Err(err(
"mlsp.unsupported",
"view.document requires the view feature",
)),
Some("view.reference_resolve") => crate::workbench::resolve_reference(&params),
// `<TAB>` M9: occlusion label search (two-tier vocabulary).
Some("labels.search") => labels_search(&params),
// `<TAB>` M6: one-shot session summary (T1 staleness seen/fixed).
Some("session.summary") => session_summary(),
other => Err(err(
"mlsp.unknown_method",
format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema / artifact.find / artifact.inspect / link.resolve / artifact.verify / artifact.impact / artifact.bind / artifact.execute / artifact.reuse_execute / artifact.patch / artifact.extract_pure / document.extract"),
format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema / artifact.find / artifact.inspect / link.resolve / artifact.verify / artifact.impact / artifact.bind / artifact.execute / artifact.reuse_execute / artifact.patch / artifact.extract_pure / document.extract / capabilities / view.document / view.reference_resolve"),
)),
}
}
#[cfg(feature = "view")]
fn view_document(params: &Value) -> Result<Value, Value> {
let req = crate::workbench::DocumentViewRequest::from_params(params)
.map_err(|e| err("mlsp.bad_request", e))?;
crate::workbench::build_document_view(&req)
}
fn artifact_extract_pure(params: &Value) -> Result<Value, Value> {
let name = params
.get("name")
@@ -172,7 +189,100 @@ fn artifact_inspect(params: &Value) -> Result<Value, Value> {
let candidates = artifact_query(&compilation.semantics, query);
if candidates.is_empty() { return Err(err("artifact.unresolved", format!("没有制品 `{query}`"))); }
if candidates.len() != 1 { return Err(json!({"code":"artifact.ambiguous","message":format!("制品 `{query}` 不唯一"),"candidates":candidates.iter().map(|a|&a.artifact_id.0).collect::<Vec<_>>() })); }
Ok(crate::sema::artifact_json(candidates[0]))
let artifact = candidates[0];
// Version contract: expected source/generation mismatch must not silently return current.
check_artifact_version_params(params, &compilation, artifact)?;
Ok(crate::sema::artifact_json(artifact))
}
/// Reject inspect when caller pinned an expected source revision or generation that
/// does not match the current project snapshot (`artifact.version_mismatch`).
fn check_artifact_version_params(
params: &Value,
compilation: &crate::sema::Compilation,
artifact: &crate::sema::SemanticArtifact,
) -> Result<(), Value> {
let expected_rev = params
.get("expected_source_revision")
.and_then(Value::as_str)
.filter(|s| !s.is_empty());
if let Some(expected) = expected_rev {
let project_root = params
.get("project")
.or_else(|| params.get("path"))
.and_then(Value::as_str)
.map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from("."));
let root = if project_root.is_file() {
project_root
.parent()
.unwrap_or_else(|| Path::new("."))
.to_path_buf()
} else {
project_root
};
let abs = root.join(&artifact.source.file);
match std::fs::read(&abs) {
Ok(raw) => {
use sha2::{Digest, Sha256};
let actual = format!("sha256:{:x}", Sha256::digest(&raw));
if actual != expected {
return Err(json!({
"code": "artifact.version_mismatch",
"message": format!(
"expected source_revision {expected}, actual {actual}; refusing silent downgrade"
),
"expected_source_revision": expected,
"actual_source_revision": actual,
"artifact_id": artifact.artifact_id.0,
}));
}
}
Err(e) => {
return Err(json!({
"code": "artifact.version_mismatch",
"message": format!(
"cannot verify source_revision for {}: {e}; refusing silent downgrade",
artifact.source.file
),
"expected_source_revision": expected,
"artifact_id": artifact.artifact_id.0,
}));
}
}
}
let expected_gen = params
.get("expected_generation")
.or_else(|| params.get("generation_id"))
.and_then(Value::as_str)
.filter(|s| !s.is_empty());
if let Some(expected) = expected_gen {
let actual = semantic_generation_id(&compilation.semantics);
if actual != expected {
return Err(json!({
"code": "artifact.version_mismatch",
"message": format!(
"expected generation {expected}, current projection is {actual}; refusing silent downgrade"
),
"expected_generation": expected,
"actual_generation": actual,
"artifact_id": artifact.artifact_id.0,
}));
}
}
Ok(())
}
/// Semantic generation token matching KnowledgePresentation / DocumentView.
fn semantic_generation_id(project: &crate::sema::SemanticProject) -> String {
use sha2::{Digest, Sha256};
let mut gh = Sha256::new();
gh.update(project.project_id.as_bytes());
for a in &project.artifacts {
gh.update(a.hashes.artifact.0.as_bytes());
}
format!("{:x}", gh.finalize())[..16].to_string()
}
fn link_resolve(params: &Value) -> Result<Value, Value> {
+17 -1
View File
@@ -127,7 +127,11 @@ impl KnowledgePresentation {
}
}
/// V2: whether an SSE result packet may update the live document structure.
/// Live View SSE: whether a result packet may update the document structure.
///
/// Legacy packets without a token are accepted so older live-view clients keep
/// working. **Embedded DocumentView clients must use [`sse_accepts_packet_strict`]
/// instead** — missing/empty tokens and mismatches are rejected there.
pub fn sse_accepts_packet(page_token: &str, packet_token: Option<&str>) -> bool {
match packet_token {
None | Some("") => true, // legacy packets without token still apply
@@ -136,6 +140,18 @@ pub fn sse_accepts_packet(page_token: &str, packet_token: Option<&str>) -> bool
}
}
/// Strict SSE / embed gate: reject missing, empty, or mismatched page tokens.
///
/// Use this for DocumentView / qdagent embed hosts. Do not inherit the live-view
/// leniency of [`sse_accepts_packet`].
pub fn sse_accepts_packet_strict(page_token: &str, packet_token: Option<&str>) -> bool {
match packet_token {
None | Some("") => false,
Some(_tok) if page_token.is_empty() => false,
Some(tok) => tok == page_token,
}
}
/// Stable page token from raw source bytes (used by SSE run path).
pub fn page_token_from_source(rel: &str, source: &str) -> String {
let mut h = Sha256::new();
+7 -2
View File
@@ -12,12 +12,17 @@ mod output;
mod render;
pub use knowledge_present::{
page_token_from_source, sse_accepts_packet, title_from_source, KnowledgePresentation,
page_token_from_source, sse_accepts_packet, sse_accepts_packet_strict, title_from_source,
KnowledgePresentation,
};
/// Surface rendering of an expression (labels/diagnostics reuse). `<TAB>` M6
pub use render::expr_display;
/// Read-model HTML helpers for embedded DocumentView (M03) — never execute.
pub use document::render_module_document;
pub use render::render_knowledge_html;
pub use output::{write_capture, write_static, CaptureOptions, OutputOptions};
use std::fs;
@@ -37,7 +42,7 @@ use crate::view::html::{escape, page_file, page_index, LinkMode};
use crate::view::annot::{build_view_annots, view_execution_report_json};
use crate::view::render::{
collect_input_prompts, render_bindings_html, render_module_structure,
render_knowledge_html, render_module_structure_with_annots, render_right_rail, uses_stream_panel, FileViewModel,
render_module_structure_with_annots, render_right_rail, uses_stream_panel, FileViewModel,
};
use crate::{run_file_capture, RunOptions};
+168
View File
@@ -0,0 +1,168 @@
//! Project snapshot analysis cache (M05 / F14): keyed by project_root + source digests.
//! Concurrent same-snapshot requests coalesce: waiters reuse the leader's result.
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Condvar, Mutex, OnceLock};
use std::time::Instant;
#[derive(Clone)]
struct CacheEntry {
digest_key: String,
#[allow(dead_code)]
compiled_at: Instant,
}
struct CacheState {
entries: HashMap<String, CacheEntry>,
/// project_key → refcount of leaders currently analyzing.
inflight: HashMap<String, usize>,
}
fn cache() -> &'static (Mutex<CacheState>, Condvar) {
static C: OnceLock<(Mutex<CacheState>, Condvar)> = OnceLock::new();
C.get_or_init(|| {
(
Mutex::new(CacheState {
entries: HashMap::new(),
inflight: HashMap::new(),
}),
Condvar::new(),
)
})
}
static HITS: AtomicU64 = AtomicU64::new(0);
static MISSES: AtomicU64 = AtomicU64::new(0);
static MERGES: AtomicU64 = AtomicU64::new(0);
pub fn project_source_digest(project_root: &Path) -> String {
let mut h = Sha256::new();
h.update(project_root.to_string_lossy().as_bytes());
let mut paths: Vec<PathBuf> = Vec::new();
if let Ok(rd) = std::fs::read_dir(project_root) {
for ent in rd.flatten().take(200) {
let p = ent.path();
if p.extension().and_then(|e| e.to_str()) == Some("md")
|| p
.file_name()
.and_then(|n| n.to_str())
.map(|n| n.ends_with(".mq.md"))
.unwrap_or(false)
{
paths.push(p);
}
}
}
paths.sort();
for p in paths {
h.update(p.to_string_lossy().as_bytes());
if let Ok(bytes) = std::fs::read(&p) {
h.update(&bytes);
}
}
format!("{:x}", h.finalize())
}
fn project_key(project_root: &Path) -> String {
project_root
.canonicalize()
.unwrap_or_else(|_| project_root.to_path_buf())
.to_string_lossy()
.to_string()
}
/// Ticket for a leader analysis; must call [`finish_analysis`] (or drop via finish).
pub struct AnalysisTicket {
key: String,
digest: String,
/// true ⇒ this caller is the leader and must run compile then finish.
pub is_leader: bool,
}
/// Begin analysis: cache hit / wait for in-flight leader / become leader.
pub fn begin_analysis(project_root: &Path) -> AnalysisTicket {
let key = project_key(project_root);
let digest = project_source_digest(project_root);
let (lock, cv) = cache();
let mut g = lock.lock().unwrap();
loop {
if let Some(e) = g.entries.get(&key) {
if e.digest_key == digest {
HITS.fetch_add(1, Ordering::Relaxed);
return AnalysisTicket {
key,
digest,
is_leader: false,
};
}
}
if g.inflight.contains_key(&key) {
MERGES.fetch_add(1, Ordering::Relaxed);
g = cv.wait(g).unwrap();
continue;
}
*g.inflight.entry(key.clone()).or_insert(0) += 1;
MISSES.fetch_add(1, Ordering::Relaxed);
return AnalysisTicket {
key,
digest,
is_leader: true,
};
}
}
pub fn finish_analysis(ticket: AnalysisTicket) {
if !ticket.is_leader {
return;
}
let (lock, cv) = cache();
let mut g = lock.lock().unwrap();
g.entries.insert(
ticket.key.clone(),
CacheEntry {
digest_key: ticket.digest,
compiled_at: Instant::now(),
},
);
if let Some(n) = g.inflight.get_mut(&ticket.key) {
*n = n.saturating_sub(1);
if *n == 0 {
g.inflight.remove(&ticket.key);
}
}
cv.notify_all();
}
/// Compatibility: note without holding across compile (prefer begin/finish).
pub fn note_analysis(project_root: &Path) -> bool {
let t = begin_analysis(project_root);
let hit = !t.is_leader;
finish_analysis(t);
hit
}
pub fn cache_keys_isolated(a: &Path, b: &Path) -> bool {
project_key(a) != project_key(b)
}
pub fn cache_stats() -> (u64, u64, u64) {
(
HITS.load(Ordering::Relaxed),
MISSES.load(Ordering::Relaxed),
MERGES.load(Ordering::Relaxed),
)
}
pub fn reset_cache_for_tests() {
let (lock, cv) = cache();
let mut g = lock.lock().unwrap();
g.entries.clear();
g.inflight.clear();
HITS.store(0, Ordering::Relaxed);
MISSES.store(0, Ordering::Relaxed);
MERGES.store(0, Ordering::Relaxed);
cv.notify_all();
}
+213
View File
@@ -0,0 +1,213 @@
//! MLSP `capabilities` — probe loaded plugins and declared features (M00 / F16).
//! Feature integers must reflect *implemented and testable* behavior only.
use serde_json::{json, Value};
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
/// Host/tests may force stream/cancel off for honesty / F16 mismatch cases.
static FORCE_STREAM_OFF: AtomicBool = AtomicBool::new(false);
static FORCE_CANCEL_OFF: AtomicBool = AtomicBool::new(false);
pub fn set_event_stream_ready(ready: bool) {
FORCE_STREAM_OFF.store(!ready, Ordering::Relaxed);
}
pub fn set_cancel_ready(ready: bool) {
FORCE_CANCEL_OFF.store(!ready, Ordering::Relaxed);
}
/// Build the capabilities document. Does not call providers or execute project code.
pub fn probe_capabilities(params: &Value) -> Value {
let runtime_version = env!("CARGO_PKG_VERSION").to_string();
let mut adapters = vec![json!({
"id": "openai-compatible",
"available": true,
"notes": "HTTP chat completions; messages[] preferred over single prompt"
})];
adapters.push(json!({
"id": "ollama-compatible",
"available": true,
"notes": "OpenAI-compatible path; not a distinct native protocol"
}));
adapters.push(json!({
"id": "anthropic-native",
"available": false,
"notes": "not verified in this release"
}));
adapters.push(json!({
"id": "gemini-native",
"available": false,
"notes": "not verified in this release"
}));
let extract_formats = vec![
json!({"format": "text", "available": true}),
json!({"format": "markdown", "available": true}),
json!({"format": "pdf", "available": false}),
json!({"format": "docx", "available": false}),
json!({"format": "ocr", "available": false}),
];
let loaded_plugins = probe_loaded_plugins(params);
let llm_loaded = loaded_plugins.iter().any(|p| {
p.get("name").and_then(|n| n.as_str()) == Some("llm")
&& p.get("available").and_then(|a| a.as_bool()) == Some(true)
});
let stream_off = FORCE_STREAM_OFF.load(Ordering::Relaxed)
|| params
.get("force_stream_off")
.and_then(|v| v.as_bool())
.unwrap_or(false);
let cancel_off = FORCE_CANCEL_OFF.load(Ordering::Relaxed)
|| params
.get("force_cancel_off")
.and_then(|v| v.as_bool())
.unwrap_or(false);
let mut features = serde_json::Map::new();
features.insert(
"llm.explicit_selection".into(),
json!(if llm_loaded { 1 } else { 0 }),
);
features.insert(
"llm.call_usage".into(),
json!(if llm_loaded { 1 } else { 0 }),
);
features.insert(
"llm.messages".into(),
json!(if llm_loaded { 1 } else { 0 }),
);
// M02 implemented: event_sink + web sse-live + MARQDO_CANCEL in llm plugin.
features.insert(
"runtime.event_stream".into(),
json!(if stream_off { 0 } else { 1 }),
);
features.insert(
"runtime.cancel".into(),
json!(if cancel_off { 0 } else { 1 }),
);
#[cfg(feature = "view")]
{
features.insert("view.document".into(), json!(1));
features.insert("view.reference_resolve".into(), json!(1));
}
#[cfg(not(feature = "view"))]
{
features.insert("view.document".into(), json!(0));
features.insert("view.reference_resolve".into(), json!(0));
}
json!({
"schema_version": 1,
"runtime_version": runtime_version,
"features": features,
"adapters": adapters,
"extract_formats": extract_formats,
"loaded_plugins": loaded_plugins,
"contracts": [
"ModelSelectionV1",
"ModelCallUsageV1",
"RuntimeEventV1",
"DocumentViewV1",
"KnowledgeLocatorV1"
]
})
}
fn probe_loaded_plugins(params: &Value) -> Vec<Value> {
let mut out = Vec::new();
let candidates = [
("llm", "MARQDO_LLM_PLUGIN", plugin_name("llm")),
("agent", "MARQDO_AGENT_PLUGIN", plugin_name("agent")),
("web", "MARQDO_WEB_PLUGIN", plugin_name("web")),
];
let ext_hint = params
.get("ext_root")
.and_then(|v| v.as_str())
.map(PathBuf::from)
.or_else(|| std::env::var_os("MARQDO_EXT").map(PathBuf::from));
let force_unloaded = params
.get("force_unloaded")
.and_then(|v| v.as_bool())
.unwrap_or(false);
for (name, env_key, filename) in candidates {
if force_unloaded {
out.push(json!({
"name": name,
"available": false,
"path": Value::Null,
"capabilities": [],
}));
continue;
}
let mut path: Option<PathBuf> = std::env::var_os(env_key).map(PathBuf::from);
if path.as_ref().map(|p| !p.is_file()).unwrap_or(true) {
let debug = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("target")
.join("debug")
.join(&filename);
if debug.is_file() {
path = Some(debug);
}
}
if path.as_ref().map(|p| !p.is_file()).unwrap_or(true) {
if let Some(ext) = &ext_hint {
let native = ext.join("native").join(&filename);
if native.is_file() {
path = Some(native);
}
}
}
let available = path.as_ref().map(|p| p.is_file()).unwrap_or(false);
let mut caps = Vec::new();
if name == "llm" && available {
caps.push("call_usage_v1");
caps.push("messages_v1");
caps.push("explicit_selection_v1");
caps.push("cancel_v1");
}
if name == "agent" && available {
caps.push("eng_bind_execute");
caps.push("publish_gate");
}
out.push(json!({
"name": name,
"available": available,
"path": path.map(|p| p.to_string_lossy().to_string()),
"capabilities": caps,
}));
}
out
}
fn plugin_name(stem: &str) -> String {
if cfg!(windows) {
format!("{stem}.dll")
} else if cfg!(target_os = "macos") {
format!("lib{stem}.dylib")
} else {
format!("lib{stem}.so")
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn force_off_and_unloaded_honest() {
let caps = probe_capabilities(&json!({
"force_unloaded": true,
"force_stream_off": true,
"force_cancel_off": true,
}));
assert_eq!(caps["features"]["runtime.event_stream"], 0);
assert_eq!(caps["features"]["runtime.cancel"], 0);
assert_eq!(caps["features"]["llm.messages"], 0);
}
}
+489
View File
@@ -0,0 +1,489 @@
//! DocumentViewV1 — read-only embedded view (M03). Never executes the document.
use std::fs;
use std::path::{Path, PathBuf};
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
use crate::view::{render_knowledge_html, render_module_document, KnowledgePresentation};
#[derive(Debug, Clone)]
pub struct DocumentViewRequest {
pub project: PathBuf,
pub document: String,
pub revision: Option<String>,
pub generation: Option<String>,
pub node_id: Option<String>,
pub include: Vec<String>,
}
impl DocumentViewRequest {
pub fn from_params(params: &Value) -> Result<Self, String> {
let project = params
.get("project")
.and_then(|v| v.as_str())
.ok_or_else(|| "view.document requires params.project".to_string())?;
let document = params
.get("document")
.and_then(|v| v.as_str())
.ok_or_else(|| "view.document requires params.document".to_string())?;
let mode = params
.get("mode")
.and_then(|v| v.as_str())
.unwrap_or("read");
if mode != "read" {
return Err(format!("view.document unsupported mode={mode}; only read"));
}
let include = match params.get("include") {
Some(Value::Array(a)) => a
.iter()
.filter_map(|v| v.as_str().map(|s| s.to_string()))
.collect(),
_ => vec![
"fragment".into(),
"outline".into(),
"nodes".into(),
"refs".into(),
"diagnostics".into(),
],
};
Ok(Self {
project: PathBuf::from(project),
document: document.to_string(),
revision: params
.get("revision")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
generation: params
.get("generation")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
node_id: params
.get("node_id")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
include,
})
}
}
/// Pure read path: compile_project (no plugin load / no eval) + fragment render.
/// Must never call `build_file_view` with execute=true.
pub fn build_document_view(req: &DocumentViewRequest) -> Result<Value, Value> {
let project_root = canonicalize_project(&req.project)?;
let abs = resolve_under(&project_root, &req.document)?;
let raw = fs::read(&abs).map_err(|e| {
err(
"view.document_read_failed",
format!("cannot read {}: {e}", abs.display()),
)
})?;
let source_revision = format!("sha256:{:x}", Sha256::digest(&raw));
if let Some(expected) = &req.revision {
if expected != &source_revision && !expected.is_empty() {
return Err(err(
"view.revision_mismatch",
format!("requested {expected} but file is {source_revision}; refusing silent downgrade"),
));
}
}
let source = String::from_utf8_lossy(&raw).into_owned();
let title = title_from_source(&source, &req.document);
// Semantic projection — compile_project does not load plugins / evaluate.
// Hold analysis lock across compile so concurrent same-snapshot reads coalesce (F14).
let ticket = crate::workbench::analysis_cache::begin_analysis(&project_root);
let options = crate::sema::CompileOptions::for_path(&project_root)
.map_err(|e| err("view.compile_failed", format!("{e:#}")))?;
let compilation = match crate::sema::compile_project(&options) {
Ok(c) => {
crate::workbench::analysis_cache::finish_analysis(ticket);
c
}
Err(e) => {
crate::workbench::analysis_cache::finish_analysis(ticket);
return Err(err("view.compile_failed", format!("{e:#}")));
}
};
let rel = req.document.replace('\\', "/");
let mut presentation = KnowledgePresentation::from_project(&compilation.semantics, &rel, &title);
presentation.page_token = {
let mut h = Sha256::new();
h.update(raw.as_slice());
h.update(presentation.semantic_generation.as_bytes());
format!("{:x}", h.finalize())
};
if let Some(gen) = &req.generation {
if !gen.is_empty() && gen != &presentation.semantic_generation {
return Err(err(
"view.generation_mismatch",
format!(
"requested generation {gen} but projection is {}; refusing silent downgrade",
presentation.semantic_generation
),
));
}
}
let wants = |k: &str| req.include.iter().any(|x| x == k);
let heading_outline = outline_from_headings(&source, &presentation);
// Prefer document surface HTML when parseable; fall back to knowledge HTML.
// Never call build_file_view (that path executes).
let fragment = if wants("fragment") {
let raw_html = match crate::load::parse_source_auto(&source) {
Ok(module) => render_module_document(&module, &source, &[]),
Err(_) => render_knowledge_html(&compilation.semantics, &rel),
};
wrap_fragment(&inject_mq_node_attrs(&raw_html, &heading_outline, &presentation))
} else {
String::new()
};
let outline = if wants("outline") {
if heading_outline.is_empty() {
outline_from_presentation(&presentation)
} else {
json!(heading_outline)
}
} else {
json!([])
};
let nodes = if wants("nodes") {
nodes_from_presentation_and_steps(
&presentation,
&compilation.semantics,
&rel,
&req.node_id,
)
} else {
json!([])
};
let refs = if wants("refs") {
refs_from_project(&compilation.semantics, &rel)
} else {
json!([])
};
let mut diag_msgs: Vec<String> = presentation.diagnostics.clone();
for d in &compilation.diagnostics {
diag_msgs.push(d.message.clone());
}
let diagnostics = if wants("diagnostics") {
json!(diag_msgs)
} else {
json!([])
};
let completeness = if diag_msgs.is_empty() {
"complete"
} else {
"partial"
};
let snapshot = crate::workbench::snapshot::build_snapshot_manifest(
&project_root,
&rel,
&raw,
&compilation.semantics,
"1",
);
Ok(json!({
"schema": "DocumentViewV1",
"schema_version": 1,
"document": {
"id": rel,
"title": title,
"source_revision": source_revision,
"semantic_digest": presentation.source_digest,
"generation_id": if presentation.semantic_generation.is_empty() {
Value::Null
} else {
json!(presentation.semantic_generation)
},
"presentation_token": presentation.page_token,
},
"execution": {"state": "not_run", "run_id": Value::Null},
"render": {
"format": "sanitized_html",
"fragment": fragment,
"renderer_version": "1"
},
"outline": outline,
"nodes": nodes,
"refs": refs,
"diagnostics": diagnostics,
"completeness": completeness,
"snapshot": snapshot,
"capabilities": {"read_only": true, "has_semantics": true, "executed": false}
}))
}
fn wrap_fragment(inner: &str) -> String {
// No html/body, no scripts, namespaced for host theming.
let sanitized = strip_script_tags(inner);
format!("<article class=\"mq-document\">{sanitized}</article>")
}
/// Remove `<script…>` blocks (case-insensitive) so embed fragments stay non-executable.
fn strip_script_tags(html: &str) -> String {
let lower = html.to_ascii_lowercase();
let mut out = String::with_capacity(html.len());
let mut i = 0usize;
while i < html.len() {
if lower[i..].starts_with("<script") {
let after_open = lower[i..]
.find('>')
.map(|p| i + p + 1)
.unwrap_or(html.len());
if let Some(rel) = lower[after_open..].find("</script") {
let close_at = after_open + rel;
let end = lower[close_at..]
.find('>')
.map(|p| close_at + p + 1)
.unwrap_or(html.len());
i = end;
} else {
// Truncate from the dangerous open tag onward.
break;
}
continue;
}
// Advance one UTF-8 char so multi-byte text stays intact.
let ch = html[i..].chars().next().unwrap();
out.push(ch);
i += ch.len_utf8();
}
out
}
fn title_from_source(source: &str, fallback: &str) -> String {
for line in source.lines().take(40) {
let t = line.trim();
if let Some(rest) = t.strip_prefix("# ") {
return rest.trim().to_string();
}
if let Some(rest) = t.strip_prefix("title:") {
return rest.trim().trim_matches('"').to_string();
}
}
Path::new(fallback)
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or(fallback)
.to_string()
}
/// Outline entries from Markdown / Marqdo `#` headings; prefer matching node ids.
fn outline_from_headings(source: &str, p: &KnowledgePresentation) -> Vec<Value> {
let mut items = Vec::new();
let mut order = 0usize;
for (idx, line) in source.lines().enumerate() {
let line_no = (idx + 1) as u32;
let trimmed = line.trim_start();
if !trimmed.starts_with('#') {
continue;
}
let hashes = trimmed.chars().take_while(|c| *c == '#').count();
if hashes == 0 || hashes > 6 {
continue;
}
// `# Title`, `## Section`, or Marqdo `#main` / `##greet` (no space).
let after = &trimmed[hashes..];
let label = after.trim();
if label.is_empty() {
continue;
}
let matched = p.nodes.iter().find(|n| {
n.source_line == line_no
|| n.display_name == label
|| n.node_id == label
|| n.artifact_id.as_deref() == Some(label)
});
let id = matched
.map(|n| n.node_id.clone())
.unwrap_or_else(|| format!("heading:{line_no}"));
let artifact_id = matched.and_then(|n| n.artifact_id.clone());
items.push(json!({
"id": id,
"parent_id": Value::Null,
"kind": "heading",
"label": label,
"order": order,
"source_span": {"line": line_no},
"artifact_id": artifact_id,
}));
order += 1;
}
items
}
/// Stamp `data-mq-node` onto elements that already carry `data-line` matching a node/outline id.
fn inject_mq_node_attrs(html: &str, outline: &[Value], p: &KnowledgePresentation) -> String {
let mut out = html.to_string();
let mut bindings: Vec<(u32, String)> = Vec::new();
for n in &p.nodes {
bindings.push((n.source_line, n.node_id.clone()));
}
for item in outline {
if let (Some(line), Some(id)) = (
item.pointer("/source_span/line").and_then(|v| v.as_u64()),
item.get("id").and_then(|v| v.as_str()),
) {
bindings.push((line as u32, id.to_string()));
}
}
bindings.sort_by_key(|(line, _)| *line);
bindings.dedup_by_key(|(line, _)| *line);
for (line, id) in bindings {
let needle = format!("data-line=\"{line}\"");
let attr = format!("data-line=\"{line}\" data-mq-node=\"{}\"", html_attr_escape(&id));
if out.contains(&needle) && !out.contains(&format!("data-mq-node=\"{}\"", html_attr_escape(&id))) {
out = out.replacen(&needle, &attr, 1);
}
}
out
}
fn html_attr_escape(s: &str) -> String {
s.replace('&', "&amp;")
.replace('"', "&quot;")
.replace('<', "&lt;")
}
fn outline_from_presentation(p: &KnowledgePresentation) -> Value {
json!(p
.nodes
.iter()
.enumerate()
.map(|(i, n)| json!({
"id": n.node_id,
"parent_id": Value::Null,
"kind": "heading",
"label": n.display_name,
"order": i,
"artifact_id": n.artifact_id,
}))
.collect::<Vec<_>>())
}
fn nodes_from_presentation_and_steps(
p: &KnowledgePresentation,
project: &crate::sema::SemanticProject,
rel: &str,
filter: &Option<String>,
) -> Value {
let mut out = Vec::new();
for n in &p.nodes {
if filter
.as_ref()
.map(|f| &n.node_id == f || n.artifact_id.as_ref() == Some(f))
.unwrap_or(true)
{
out.push(json!({
"id": n.node_id,
"parent_id": Value::Null,
"kind": n.kind,
"label": n.display_name,
"order": n.source_line,
"source_span": {"file": n.source_file, "line": n.source_line},
"artifact_id": n.artifact_id,
"ref_ids": [],
"children": [],
"evidence": n.evidence,
"effects": n.effects,
"dependencies": n.dependencies,
"signature": n.signature,
}));
}
}
// F09: project KnowledgeStep kinds (observe/derive/call/iterate/…) into the read model.
for step in &project.steps {
if step.source.file != rel {
continue;
}
let id = step.id.0.clone();
if filter
.as_ref()
.map(|f| &id == f || &step.reference == f)
.unwrap_or(true)
{
out.push(json!({
"id": id,
"parent_id": Value::Null,
"kind": step.kind,
"label": step.reference,
"order": step.source.line,
"source_span": {
"file": step.source.file,
"line": step.source.line,
"end_line": step.source.end_line
},
"artifact_id": Value::Null,
"ref_ids": step.calls.iter().map(|c| c.as_str()).collect::<Vec<_>>(),
"children": [],
"evidence": "unverified",
"effects": step.effects.values(),
}));
}
}
json!(out)
}
fn refs_from_project(project: &crate::sema::SemanticProject, rel: &str) -> Value {
let mut refs = Vec::new();
for a in &project.artifacts {
if a.source.file != rel {
continue;
}
for dep in &a.dependencies {
refs.push(json!({
"id": format!("ref:{}:{}", a.artifact_id.0, dep.0),
"from_artifact_id": a.artifact_id.0,
"to": dep.0,
"relation": "depends_on",
}));
}
}
json!(refs)
}
fn canonicalize_project(p: &Path) -> Result<PathBuf, Value> {
let root = if p.is_file() {
p.parent().unwrap_or_else(|| Path::new(".")).to_path_buf()
} else {
p.to_path_buf()
};
root.canonicalize().map_err(|e| {
err(
"view.project_invalid",
format!("project {}: {e}", root.display()),
)
})
}
fn resolve_under(root: &Path, rel: &str) -> Result<PathBuf, Value> {
let joined = root.join(rel);
let canon = joined.canonicalize().map_err(|e| {
err(
"view.document_not_found",
format!("{}: {e}", joined.display()),
)
})?;
if !canon.starts_with(root) {
return Err(err(
"view.path_escape",
"document path escapes project root",
));
}
Ok(canon)
}
fn err(code: &str, message: impl Into<String>) -> Value {
json!({"code": code, "message": message.into()})
}
+59
View File
@@ -0,0 +1,59 @@
//! Request-local RuntimeEventV1 sink (M02). Avoids process-global event mix-up.
use serde_json::{json, Value};
use std::fs::OpenOptions;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Mutex, OnceLock};
static SEQ: AtomicU64 = AtomicU64::new(0);
fn sink_path() -> &'static Mutex<Option<PathBuf>> {
static SINK: OnceLock<Mutex<Option<PathBuf>>> = OnceLock::new();
SINK.get_or_init(|| Mutex::new(None))
}
/// Bind this process/request to an NDJSON event file (also used by web sse-live).
pub fn bind_event_sink(path: impl AsRef<Path>) {
if let Ok(mut g) = sink_path().lock() {
*g = Some(path.as_ref().to_path_buf());
}
crate::workbench::capabilities::set_event_stream_ready(true);
}
pub fn clear_event_sink() {
if let Ok(mut g) = sink_path().lock() {
*g = None;
}
}
pub fn emit_event(run_id: &str, event_type: &str, payload: Value) -> Value {
let seq = SEQ.fetch_add(1, Ordering::Relaxed);
let event_id = format!("evt-{run_id}-{seq}");
let ev = crate::workbench::usage::runtime_event(&event_id, seq, run_id, event_type, payload);
if let Ok(g) = sink_path().lock() {
if let Some(path) = g.as_ref() {
if let Ok(mut f) = OpenOptions::new().create(true).append(true).open(path) {
let _ = writeln!(f, "{}", ev);
}
}
}
// Also honor MARQDO_EVENT_FORWARD for subtask/web bridge.
if let Ok(fwd) = std::env::var("MARQDO_EVENT_FORWARD") {
if !fwd.is_empty() {
if let Ok(mut f) = OpenOptions::new().create(true).append(true).open(&fwd) {
let _ = writeln!(f, "{}", ev);
}
}
}
ev
}
pub fn emit_message_delta(run_id: &str, text: &str) -> Value {
emit_event(
run_id,
"message.delta",
json!({"text": text, "format": "plain"}),
)
}
+304
View File
@@ -0,0 +1,304 @@
//! KnowledgeLocatorV1 + view.reference_resolve (M04).
use std::path::{Path, PathBuf};
use serde_json::{json, Value};
#[derive(Debug, Clone)]
pub struct KnowledgeLocator {
pub project_id: String,
pub document_id: String,
pub source_revision: Option<String>,
pub generation_id: Option<String>,
pub artifact_id: Option<String>,
pub node_id: Option<String>,
pub source_ref_id: Option<String>,
pub span: Option<Value>,
pub relation: String,
}
impl KnowledgeLocator {
pub fn from_params(params: &Value) -> Result<Self, String> {
let loc = params.get("locator").unwrap_or(params);
Ok(Self {
project_id: loc
.get("project_id")
.or_else(|| params.get("project"))
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
document_id: loc
.get("document_id")
.or_else(|| params.get("document"))
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
source_revision: loc
.get("source_revision")
.or_else(|| loc.get("revision"))
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
generation_id: loc
.get("generation_id")
.or_else(|| loc.get("generation"))
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
artifact_id: loc
.get("artifact_id")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
node_id: loc
.get("node_id")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
source_ref_id: loc
.get("source_ref_id")
.or_else(|| loc.get("ref_id"))
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
span: loc.get("span").cloned(),
relation: loc
.get("relation")
.and_then(|v| v.as_str())
.unwrap_or("cites")
.to_string(),
})
}
pub fn to_json(&self) -> Value {
json!({
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"project_id": self.project_id,
"document_id": self.document_id,
"source_revision": self.source_revision,
"generation_id": self.generation_id,
"artifact_id": self.artifact_id,
"node_id": self.node_id,
"source_ref_id": self.source_ref_id,
"span": self.span,
"relation": self.relation,
})
}
}
/// Resolve a locator or current-document ref_id against a project snapshot.
/// Never silently falls back to "latest" when expected revision/generation mismatch.
pub fn resolve_reference(params: &Value) -> Result<Value, Value> {
let locator = KnowledgeLocator::from_params(params).map_err(|e| {
json!({"code": "locator.bad_request", "message": e})
})?;
if locator.project_id.is_empty() {
return Err(json!({
"code": "locator.bad_request",
"message": "project_id/project required"
}));
}
let root = PathBuf::from(&locator.project_id);
let root = root.canonicalize().map_err(|e| {
json!({"code": "locator.project_invalid", "message": format!("{e}")})
})?;
let options = crate::sema::CompileOptions::for_path(&root).map_err(|e| {
json!({"code": "locator.compile_failed", "message": format!("{e:#}")})
})?;
let compilation = crate::sema::compile_project(&options).map_err(|e| {
json!({"code": "locator.compile_failed", "message": format!("{e:#}")})
})?;
// Generation mismatch → version_unavailable / stale (no silent current).
if let Some(expected_gen) = &locator.generation_id {
if !expected_gen.is_empty() {
let current_gen = semantic_generation(&compilation.semantics);
if &current_gen != expected_gen {
return Ok(crate::workbench::snapshot::version_unavailable_result(
&locator.to_json(),
&format!(
"expected generation {expected_gen}, current projection is {current_gen}; refusing silent downgrade (no retained snapshot)"
),
expected_gen,
Some(&current_gen),
));
}
}
}
// Source revision check when document is known.
if let (Some(expected_rev), true) = (
&locator.source_revision,
!locator.document_id.is_empty(),
) {
if !expected_rev.is_empty() {
let abs = resolve_under(&root, &locator.document_id)?;
let raw = std::fs::read(&abs).map_err(|e| {
json!({"code": "locator.source_unreadable", "message": format!("{e}")})
})?;
use sha2::{Digest, Sha256};
let actual = format!("sha256:{:x}", Sha256::digest(&raw));
if &actual != expected_rev {
return Ok(json!({
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "stale",
"precision": "document",
"locator": locator.to_json(),
"target": {
"document_id": locator.document_id,
"source_revision": actual,
"note": "source changed since citation"
},
"candidates": [],
"diagnostics": [{
"code": "source_revision_mismatch",
"message": format!("expected {expected_rev}, actual {actual}")
}],
"label": "该来源已变化"
}));
}
}
}
let query = locator
.artifact_id
.clone()
.or_else(|| locator.node_id.clone())
.or_else(|| locator.source_ref_id.clone());
let Some(q) = query else {
if locator.document_id.is_empty() {
return Ok(json!({
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "unresolved",
"precision": "none",
"locator": locator.to_json(),
"target": Value::Null,
"candidates": [],
"diagnostics": [{"code": "missing_target", "message": "no artifact/node/ref id"}],
"label": "无法定位该知识"
}));
}
return Ok(json!({
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "resolved",
"precision": "document",
"locator": locator.to_json(),
"target": {
"document_id": locator.document_id,
"relation": locator.relation,
},
"candidates": [],
"diagnostics": [],
"label": locator.document_id,
}));
};
let mut matches: Vec<&crate::sema::SemanticArtifact> = compilation
.semantics
.artifacts
.iter()
.filter(|a| {
a.artifact_id.0 == q
|| a.display_name == q
|| a.symbol_id
.as_ref()
.map(|s| s.as_str() == q)
.unwrap_or(false)
})
.collect();
if !locator.document_id.is_empty() {
let scoped: Vec<_> = matches
.iter()
.copied()
.filter(|a| a.source.file == locator.document_id)
.collect();
if !scoped.is_empty() {
matches = scoped;
}
}
match matches.len() {
0 => Ok(json!({
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "unresolved",
"precision": "none",
"locator": locator.to_json(),
"target": Value::Null,
"candidates": [],
"diagnostics": [{"code": "not_found", "message": format!("no artifact matching {q}")}],
"label": "尚未定位到具体知识"
})),
1 => {
let a = matches[0];
Ok(json!({
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "resolved",
"precision": "artifact",
"locator": locator.to_json(),
"target": {
"artifact_id": a.artifact_id.0,
"display_name": a.display_name,
"document_id": a.source.file,
"span": {"line": a.source.line, "end_line": a.source.end_line},
"evidence": a.verification.evidence,
"relation": locator.relation,
},
"candidates": [],
"diagnostics": [],
"label": a.display_name,
}))
}
_ => {
let candidates: Vec<Value> = matches
.iter()
.map(|a| {
json!({
"artifact_id": a.artifact_id.0,
"display_name": a.display_name,
"document_id": a.source.file,
"line": a.source.line,
})
})
.collect();
Ok(json!({
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "ambiguous",
"precision": "none",
"locator": locator.to_json(),
"target": Value::Null,
"candidates": candidates,
"diagnostics": [{"code": "ambiguous", "message": format!("{} candidates for {q}", matches.len())}],
"label": "存在多个候选"
}))
}
}
}
fn semantic_generation(project: &crate::sema::SemanticProject) -> String {
use sha2::{Digest, Sha256};
let mut gh = Sha256::new();
gh.update(project.project_id.as_bytes());
for a in &project.artifacts {
gh.update(a.hashes.artifact.0.as_bytes());
}
format!("{:x}", gh.finalize())[..16].to_string()
}
fn resolve_under(root: &Path, rel: &str) -> Result<PathBuf, Value> {
let joined = root.join(rel);
let canon = joined.canonicalize().map_err(|e| {
json!({"code": "locator.document_not_found", "message": format!("{}: {e}", joined.display())})
})?;
if !canon.starts_with(root) {
return Err(json!({
"code": "locator.path_escape",
"message": "document path escapes project root"
}));
}
Ok(canon)
}
+33
View File
@@ -0,0 +1,33 @@
//! Workbench runtime contracts (M00–M05): schemas, capabilities, DocumentView,
//! KnowledgeLocator, ModelSelection / ModelCallUsage / RuntimeEvent / RunContext.
//!
//! Spec: `doc/design/qdagent-workbench-runtime-and-view-2026-10-25.md`.
//! These are v1 contracts distinct from KnowledgePresentation `schema_version:1`.
pub mod analysis_cache;
pub mod capabilities;
#[cfg(feature = "view")]
pub mod document_view;
pub mod event_sink;
pub mod locator;
pub mod run_context;
pub mod selection;
pub mod snapshot;
pub mod usage;
pub use analysis_cache::{
begin_analysis, cache_keys_isolated, cache_stats, finish_analysis, note_analysis,
project_source_digest, reset_cache_for_tests, AnalysisTicket,
};
pub use capabilities::probe_capabilities;
#[cfg(feature = "view")]
pub use document_view::{build_document_view, DocumentViewRequest};
pub use event_sink::{bind_event_sink, clear_event_sink, emit_event, emit_message_delta};
pub use locator::{resolve_reference, KnowledgeLocator};
pub use run_context::{cancel_requested, RunContext};
pub use selection::{parse_model_selection, ModelSelection};
pub use snapshot::{build_snapshot_manifest, version_unavailable_result};
pub use usage::{
model_call_usage_from_provider, model_call_usage_from_provider_rev, replace_usage_if_newer,
runtime_event, token_or_null,
};
+148
View File
@@ -0,0 +1,148 @@
//! RunContextV1 — request-local run identity for selection / usage / cancel (M01).
use serde_json::{json, Map, Value};
#[derive(Debug, Clone, Default)]
pub struct RunContext {
pub run_id: String,
pub parent_run_id: Option<String>,
pub turn_id: Option<String>,
pub workspace_scope: Option<String>,
pub selection_context: Map<String, Value>,
pub usage_sink: Option<String>,
pub event_sink: Option<String>,
pub cancellation_token: Option<String>,
}
impl RunContext {
pub fn from_value(v: &Value) -> Result<Self, String> {
let run_id = v
.get("run_id")
.and_then(|x| x.as_str())
.filter(|s| !s.is_empty())
.ok_or_else(|| "run_context.missing:run_id".to_string())?
.to_string();
Ok(Self {
run_id,
parent_run_id: v
.get("parent_run_id")
.and_then(|x| x.as_str())
.filter(|s| !s.is_empty())
.map(|s| s.to_string()),
turn_id: v
.get("turn_id")
.and_then(|x| x.as_str())
.filter(|s| !s.is_empty())
.map(|s| s.to_string()),
workspace_scope: v
.get("workspace_scope")
.and_then(|x| x.as_str())
.filter(|s| !s.is_empty())
.map(|s| s.to_string()),
selection_context: v
.get("selection_context")
.and_then(|x| x.as_object())
.cloned()
.unwrap_or_default(),
usage_sink: v
.get("usage_sink")
.and_then(|x| x.as_str())
.map(|s| s.to_string()),
event_sink: v
.get("event_sink")
.and_then(|x| x.as_str())
.map(|s| s.to_string()),
cancellation_token: v
.get("cancellation_token")
.and_then(|x| x.as_str())
.map(|s| s.to_string()),
})
}
pub fn to_json(&self) -> Value {
json!({
"schema": "RunContextV1",
"schema_version": 1,
"run_id": self.run_id,
"parent_run_id": self.parent_run_id,
"turn_id": self.turn_id,
"workspace_scope": self.workspace_scope,
"selection_context": self.selection_context,
"usage_sink": self.usage_sink,
"event_sink": self.event_sink,
"cancellation_token": self.cancellation_token,
})
}
/// Child inherits selection/event/cancel; new run_id linked to parent.
pub fn spawn_child(&self, child_run_id: &str) -> Self {
Self {
run_id: child_run_id.to_string(),
parent_run_id: Some(self.run_id.clone()),
turn_id: self.turn_id.clone(),
workspace_scope: self.workspace_scope.clone(),
selection_context: self.selection_context.clone(),
usage_sink: self.usage_sink.clone(),
event_sink: self.event_sink.clone(),
cancellation_token: self.cancellation_token.clone(),
}
}
/// Env vars for subtask injection (never includes api_key).
pub fn selection_env(&self) -> Vec<(String, String)> {
let mut out = Vec::new();
out.push(("MARQDO_RUN_ID".into(), self.run_id.clone()));
if let Some(p) = &self.parent_run_id {
out.push(("MARQDO_PARENT_RUN_ID".into(), p.clone()));
}
if let Some(t) = &self.turn_id {
out.push(("MARQDO_TURN_ID".into(), t.clone()));
}
if let Some(w) = &self.workspace_scope {
out.push(("MARQDO_WORKSPACE_SCOPE".into(), w.clone()));
}
if let Some(tok) = &self.cancellation_token {
out.push(("MARQDO_CANCEL_TOKEN".into(), tok.clone()));
}
if let Some(sink) = &self.event_sink {
out.push(("MARQDO_EVENT_FORWARD".into(), sink.clone()));
}
if let Some(model) = self
.selection_context
.get("upstream_model_id")
.or_else(|| self.selection_context.get("model"))
.and_then(|v| v.as_str())
{
out.push(("MARQDO_LLM_MODEL".into(), model.to_string()));
out.push(("OPENAI_MODEL".into(), model.to_string()));
}
if let Some(base) = self
.selection_context
.get("base_url")
.and_then(|v| v.as_str())
{
out.push(("MARQDO_LLM_BASE_URL".into(), base.to_string()));
out.push(("OPENAI_BASE_URL".into(), base.to_string()));
}
out
}
}
/// Check whether a cancel token file/env indicates cancellation.
pub fn cancel_requested(token: Option<&str>) -> bool {
let Some(tok) = token.filter(|s| !s.is_empty()) else {
return false;
};
// Token may be a path: presence of file or contents "1"/"cancelled" means stop.
let path = std::path::Path::new(tok);
if path.is_file() {
if let Ok(body) = std::fs::read_to_string(path) {
let t = body.trim();
return t.is_empty() || t == "1" || t.eq_ignore_ascii_case("cancelled");
}
return true;
}
std::env::var("MARQDO_CANCEL")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false)
}
+147
View File
@@ -0,0 +1,147 @@
//! ModelSelectionV1 — request-level model identity (M01).
use serde_json::{json, Map, Value};
#[derive(Debug, Clone)]
pub struct ModelSelection {
pub schema_version: u64,
pub selection_id: String,
pub provider_id: String,
pub model_id: String,
pub provider_revision: Option<u64>,
pub model_revision: Option<u64>,
pub adapter: String,
pub endpoint_ref: Option<String>,
pub credential_ref: Option<String>,
pub credential_version: Option<u64>,
pub upstream_model_id: String,
pub parameters: Map<String, Value>,
pub fallback_policy: String,
pub auth_mode: String,
/// Resolved call-local endpoint (never serialize API keys into events).
pub base_url: Option<String>,
/// Call-local credential material — must not appear in public snapshots.
pub api_key: Option<String>,
}
impl ModelSelection {
pub fn public_snapshot(&self) -> Value {
json!({
"schema": "ModelSelectionV1",
"schema_version": self.schema_version,
"selection_id": self.selection_id,
"provider_id": self.provider_id,
"model_id": self.model_id,
"provider_revision": self.provider_revision,
"model_revision": self.model_revision,
"adapter": self.adapter,
"endpoint_ref": self.endpoint_ref,
"credential_ref": self.credential_ref,
"credential_version": self.credential_version,
"upstream_model_id": self.upstream_model_id,
"parameters": self.parameters,
"fallback_policy": self.fallback_policy,
"auth_mode": self.auth_mode,
})
}
}
/// Parse selection. In `strict` mode, missing required fields error (no env fallback).
pub fn parse_model_selection(v: &Value, strict: bool) -> Result<ModelSelection, String> {
let schema_version = v
.get("schema_version")
.and_then(|x| x.as_u64())
.unwrap_or(1);
if schema_version != 1 {
return Err(format!("unsupported ModelSelection schema_version={schema_version}"));
}
let require = |key: &str| -> Result<String, String> {
match v.get(key).and_then(|x| x.as_str()).filter(|s| !s.is_empty()) {
Some(s) => Ok(s.to_string()),
None if strict => Err(format!("llm.selection_missing:{key}")),
None => Ok(String::new()),
}
};
let selection_id = require("selection_id")?;
let provider_id = require("provider_id")?;
let model_id = require("model_id")?;
let adapter = v
.get("adapter")
.and_then(|x| x.as_str())
.unwrap_or("openai-compatible")
.to_string();
let upstream = v
.get("upstream_model_id")
.and_then(|x| x.as_str())
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
.or_else(|| {
if model_id.is_empty() {
None
} else {
Some(model_id.clone())
}
});
let upstream_model_id = match upstream {
Some(u) => u,
None if strict => return Err("llm.selection_missing:upstream_model_id".into()),
None => String::new(),
};
let auth_mode = v
.get("auth_mode")
.and_then(|x| x.as_str())
.unwrap_or("api_key")
.to_string();
let fallback_policy = v
.get("fallback_policy")
.and_then(|x| x.as_str())
.unwrap_or("disabled")
.to_string();
let parameters = v
.get("parameters")
.and_then(|x| x.as_object())
.cloned()
.unwrap_or_default();
let base_url = v
.get("base_url")
.and_then(|x| x.as_str())
.filter(|s| !s.is_empty())
.map(|s| s.to_string());
let api_key = v
.get("api_key")
.and_then(|x| x.as_str())
.map(|s| s.to_string());
if strict && auth_mode != "none" && api_key.as_ref().map(|s| s.is_empty()).unwrap_or(true) {
// credential_ref may be resolved by host; allow missing api_key if credential_ref present
if v.get("credential_ref").and_then(|x| x.as_str()).is_none() && base_url.is_some() {
return Err("llm.selection_missing:api_key_or_credential_ref".into());
}
}
if strict && selection_id.is_empty() {
return Err("llm.selection_missing:selection_id".into());
}
Ok(ModelSelection {
schema_version,
selection_id,
provider_id,
model_id,
provider_revision: v.get("provider_revision").and_then(|x| x.as_u64()),
model_revision: v.get("model_revision").and_then(|x| x.as_u64()),
adapter,
endpoint_ref: v
.get("endpoint_ref")
.and_then(|x| x.as_str())
.map(|s| s.to_string()),
credential_ref: v
.get("credential_ref")
.and_then(|x| x.as_str())
.map(|s| s.to_string()),
credential_version: v.get("credential_version").and_then(|x| x.as_u64()),
upstream_model_id,
parameters,
fallback_policy,
auth_mode,
base_url,
api_key,
})
}
+90
View File
@@ -0,0 +1,90 @@
//! Snapshot manifest for apps to retain historical evidence (M04).
//! Framework does not store history; missing snapshot → version_unavailable.
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
use std::path::Path;
/// Build a retainable snapshot manifest for the current projection.
pub fn build_snapshot_manifest(
project_root: &Path,
document_rel: &str,
source_bytes: &[u8],
project: &crate::sema::SemanticProject,
renderer_version: &str,
) -> Value {
let source_revision = format!("sha256:{:x}", Sha256::digest(source_bytes));
let mut dep_revs = Vec::new();
for a in &project.artifacts {
if a.source.file == document_rel {
continue;
}
dep_revs.push(json!({
"document_id": a.source.file,
"artifact_id": a.artifact_id.0,
"artifact_hash": a.hashes.artifact.0,
}));
}
let mut artifact_hashes = Vec::new();
for a in &project.artifacts {
if a.source.file == document_rel {
artifact_hashes.push(json!({
"artifact_id": a.artifact_id.0,
"hash": a.hashes.artifact.0,
}));
}
}
let mut gh = Sha256::new();
gh.update(project.project_id.as_bytes());
for a in &project.artifacts {
gh.update(a.hashes.artifact.0.as_bytes());
}
let generation_id = format!("{:x}", gh.finalize())[..16].to_string();
json!({
"schema": "SnapshotManifestV1",
"schema_version": 1,
"project_id": project_root.display().to_string(),
"document_id": document_rel,
"source_revision": source_revision,
"generation_id": generation_id,
"dependency_revisions": dep_revs,
"artifact_hashes": artifact_hashes,
"renderer_version": renderer_version,
"contract_schema": "DocumentViewV1",
"retention": {
"owner": "application",
"note": "framework does not retain historical bodies; missing snapshot → version_unavailable"
}
})
}
/// When an expected revision/generation is requested but no app-provided snapshot
/// resolver is present, return the honest unavailable contract.
pub fn version_unavailable_result(
locator: &Value,
reason: &str,
expected: &str,
actual: Option<&str>,
) -> Value {
json!({
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "version_unavailable",
"precision": "none",
"locator": locator,
"target": Value::Null,
"candidates": [],
"diagnostics": [{
"code": "snapshot_not_retained",
"message": reason,
"expected": expected,
"actual": actual,
}],
"label": "引用版本不可用",
"snapshot": {
"available": false,
"owner": "application"
}
})
}
+174
View File
@@ -0,0 +1,174 @@
//! ModelCallUsageV1 — null ≠ 0; measurement_source honesty (M02 / F04).
use serde_json::{json, Value};
/// Prefer null over zero when the provider omitted a field.
pub fn token_or_null(v: Option<i64>) -> Value {
match v {
Some(n) if n >= 0 => json!(n),
_ => Value::Null,
}
}
/// Build ModelCallUsageV1 from an OpenAI-shaped `usage` object (or Null).
pub fn model_call_usage_from_provider(
call_id: &str,
attempt_id: &str,
run_id: &str,
selection: &Value,
provider_usage: &Value,
status: &str,
) -> Value {
model_call_usage_from_provider_rev(call_id, attempt_id, run_id, selection, provider_usage, status, 1)
}
/// Same as [`model_call_usage_from_provider`] with an explicit revision (F05).
pub fn model_call_usage_from_provider_rev(
call_id: &str,
attempt_id: &str,
run_id: &str,
selection: &Value,
provider_usage: &Value,
status: &str,
revision: u64,
) -> Value {
let has_usage = provider_usage.is_object();
let input = provider_usage
.get("prompt_tokens")
.or_else(|| provider_usage.get("input_tokens"))
.and_then(|v| v.as_i64());
let output = provider_usage
.get("completion_tokens")
.or_else(|| provider_usage.get("output_tokens"))
.and_then(|v| v.as_i64());
let total = provider_usage
.get("total_tokens")
.and_then(|v| v.as_i64())
.or_else(|| match (input, output) {
(Some(a), Some(b)) => Some(a + b),
_ => None,
});
let cached = provider_usage
.get("cached_input_tokens")
.or_else(|| {
provider_usage
.get("prompt_tokens_details")
.and_then(|d| d.get("cached_tokens"))
})
.and_then(|v| v.as_i64());
let reasoning = provider_usage
.get("reasoning_output_tokens")
.or_else(|| {
provider_usage
.get("completion_tokens_details")
.and_then(|d| d.get("reasoning_tokens"))
})
.and_then(|v| v.as_i64());
let (usage_state, measurement_source) = if !has_usage {
("unknown", "none")
} else if input.is_none() && output.is_none() && total.is_none() {
("unknown", "none")
} else {
("final", "provider")
};
let mut diagnostics = Vec::new();
if let (Some(t), Some(i), Some(o)) = (total, input, output) {
if t != i + o && cached.is_none() {
diagnostics.push(json!({
"code": "usage.inconsistent",
"message": format!("total_tokens={t} != input+output={}", i + o)
}));
}
}
json!({
"schema": "ModelCallUsageV1",
"schema_version": 1,
"call_id": call_id,
"attempt_id": attempt_id,
"parent_call_id": Value::Null,
"run_id": run_id,
"turn_id": selection.get("turn_id").cloned().unwrap_or(Value::Null),
"workspace_scope": selection.get("workspace_scope").cloned().unwrap_or(Value::Null),
"purpose": selection.get("purpose").cloned().unwrap_or(json!("answer")),
"selection_id": selection.get("selection_id").cloned().unwrap_or(Value::Null),
"provider_id": selection.get("provider_id").cloned().unwrap_or(Value::Null),
"model_id": selection.get("model_id").cloned().unwrap_or(Value::Null),
"requested_model": selection.get("upstream_model_id").cloned()
.or_else(|| selection.get("model").cloned())
.unwrap_or(Value::Null),
"resolved_model": selection.get("resolved_model").cloned()
.or_else(|| selection.get("upstream_model_id").cloned())
.unwrap_or(Value::Null),
"status": status,
"usage_state": usage_state,
"measurement_source": measurement_source,
"input_tokens": token_or_null(input),
"output_tokens": token_or_null(output),
"total_tokens": token_or_null(total),
"cached_input_tokens": token_or_null(cached),
"reasoning_output_tokens": token_or_null(reasoning),
"revision": revision.max(1),
"provider_request_id": provider_usage.get("provider_request_id").cloned().unwrap_or(Value::Null),
"diagnostics": diagnostics,
// Legacy fields kept but marked — apps must not treat token_cost as money.
"legacy": {
"tokens": token_or_null(total),
"token_cost": token_or_null(total),
"deprecated": true,
"unit": "tokens"
}
})
}
/// Replace prior usage for the same (call_id, attempt_id) when revision is newer (F05).
/// Returns the winning snapshot; does not sum cumulative stream counters.
pub fn replace_usage_if_newer(prior: &Value, next: &Value) -> Value {
let same_call = prior.get("call_id") == next.get("call_id")
&& prior.get("attempt_id") == next.get("attempt_id");
if !same_call {
return next.clone();
}
let pr = prior.get("revision").and_then(|v| v.as_u64()).unwrap_or(0);
let nr = next.get("revision").and_then(|v| v.as_u64()).unwrap_or(0);
if nr >= pr {
next.clone()
} else {
prior.clone()
}
}
/// RuntimeEventV1 envelope helper.
pub fn runtime_event(
event_id: &str,
seq: u64,
run_id: &str,
event_type: &str,
payload: Value,
) -> Value {
json!({
"schema": "RuntimeEventV1",
"schema_version": 1,
"event_id": event_id,
"seq": seq,
"run_id": run_id,
"parent_run_id": Value::Null,
"turn_id": Value::Null,
"workspace_scope": Value::Null,
"timestamp": chrono_like_now(),
"type": event_type,
"payload": payload
})
}
fn chrono_like_now() -> String {
// Avoid new chrono dep: RFC3339-ish from system time.
use std::time::{SystemTime, UNIX_EPOCH};
let secs = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
format!("{secs}")
}
+69
View File
@@ -5,6 +5,64 @@
use std::process::Command;
/// Build `libagent` once per test process and prefer it over a stale `~/.marqdo/ext`.
/// Same contract as `tests/gold.rs::ensure_agent_plugin_built` — EK gate must not
/// depend on a manually copied `ext/native/libagent.so`.
fn ensure_agent_plugin_built() {
use std::sync::Once;
static ONCE: Once = Once::new();
ONCE.call_once(|| {
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_agent"])
.current_dir(&root)
.status()
.expect("build marqdo_plugin_agent");
assert!(status.success(), "failed to build marqdo_plugin_agent");
let lib = if cfg!(windows) {
root.join("target").join("debug").join("agent.dll")
} else if cfg!(target_os = "macos") {
root.join("target").join("debug").join("libagent.dylib")
} else {
root.join("target").join("debug").join("libagent.so")
};
let release = if cfg!(windows) {
root.join("target").join("release").join("agent.dll")
} else if cfg!(target_os = "macos") {
root.join("target").join("release").join("libagent.dylib")
} else {
root.join("target").join("release").join("libagent.so")
};
let chosen = if lib.is_file() {
if release.is_file() {
let lib_m = std::fs::metadata(&lib).and_then(|m| m.modified()).ok();
let rel_m = std::fs::metadata(&release).and_then(|m| m.modified()).ok();
match (lib_m, rel_m) {
(Some(l), Some(r)) if r > l => release,
_ => lib,
}
} else {
lib
}
} else {
release
};
assert!(
chosen.is_file(),
"missing agent plugin at {}",
chosen.display()
);
// SAFETY: single-threaded Once; children inherit for the rest of the test process.
std::env::set_var("MARQDO_AGENT_PLUGIN", &chosen);
if std::env::var_os("MARQDO_EXT").is_none() {
let ext = root.join("ext");
if ext.join("ai").join("agent.mq.md").is_file() {
std::env::set_var("MARQDO_EXT", &ext);
}
}
});
}
fn run_backend(path: &str, backend: &str) -> (i32, String, String) {
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let out = Command::new(env!("CARGO_BIN_EXE_marqdo"))
@@ -1657,6 +1715,12 @@ fn v2_sse_accepts_packet_rejects_stale_token() {
assert!(marqdo::view::sse_accepts_packet("abc", Some("")));
assert!(marqdo::view::sse_accepts_packet("abc", Some("abc")));
assert!(!marqdo::view::sse_accepts_packet("abc", Some("zzz")));
// Embed clients must use strict: missing/empty/mismatch all reject.
assert!(!marqdo::view::sse_accepts_packet_strict("abc", None));
assert!(!marqdo::view::sse_accepts_packet_strict("abc", Some("")));
assert!(!marqdo::view::sse_accepts_packet_strict("", Some("abc")));
assert!(!marqdo::view::sse_accepts_packet_strict("abc", Some("zzz")));
assert!(marqdo::view::sse_accepts_packet_strict("abc", Some("abc")));
assert_eq!(
marqdo::view::page_token_from_source("a.mq.md", "x"),
marqdo::view::page_token_from_source("a.mq.md", "x")
@@ -1869,6 +1933,7 @@ fn f1_publish_gate_chain_via_verify_may_publish() {
#[test]
fn f1_agent_eng_bind_gate_mq_offline() {
ensure_agent_plugin_built();
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let out = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args([
@@ -1878,6 +1943,10 @@ fn f1_agent_eng_bind_gate_mq_offline() {
"tree",
])
.current_dir(&root)
.env(
"MARQDO_AGENT_PLUGIN",
std::env::var_os("MARQDO_AGENT_PLUGIN").expect("MARQDO_AGENT_PLUGIN set by ensure"),
)
.output()
.unwrap();
let stdout = String::from_utf8_lossy(&out.stdout);
+81
View File
@@ -0,0 +1,81 @@
---
markup: occlusion
title: workbench F02 agent selection (mock upstream)
description: >-
Explicit llm.create selection must drive agent.step; process env default model
must not win. Upstream URL/key/model come from env set by the Rust harness.
import:
llm: ext/ai/llm.mq.md
agent: ext/ai/agent.mq.md
sys: lib/sys.mq.md
json: lib/json.mq.md
---
# main
**base = > sys.env_get name="OPENAI_BASE_URL"**
**key = > sys.env_get name="OPENAI_API_KEY"**
**mdl = > sys.env_get name="OPENAI_MODEL"**
1. not `base`
> print text=missing-OPENAI_BASE_URL
> sys.exit code=2
2. *
**_ = 1**
1. not `key`
> print text=missing-OPENAI_API_KEY
> sys.exit code=2
2. *
**_ = 1**
1. not `mdl`
> print text=missing-OPENAI_MODEL
> sys.exit code=2
2. *
**_ = 1**
Poison ambient default — selection on the handle must still win at the plugin.
> sys.env_set name="OPENAI_MODEL" value="env-poison-model"
**model = > llm.create model=`mdl` base_url=`base` api_key=`key` backend="openai-compatible" name=f02-agent**
`tools` =
| 工具 |
|------|
**ag = > agent.agent model=`model` tools=`tools` standing="F02 selection probe; answer with one word only"**
**step = > `ag`.step task="Reply with exactly one English word pong" writeback=False max_reads=1**
**st = > json.get value=`step` key="status"**
**ans = > json.get value=`step` key="result"**
1. not `ans`
**ans = > json.get value=`step` key="answer"**
2. *
**_ = 1**
1. not `ans`
**ans = > json.get value=`step` key="text"**
2. *
**_ = 1**
> print text=`st`
> print text=`ans`
**dir = ".marqdo/workbench-f02-resume"**
**saved = > agent.resume_save id="f02-sel" goal="pong" status="running" round=1 resume_dir=`dir`**
**ok = > json.get value=`saved` key="ok"**
1. `ok`
> print text=resume-save-ok
2. *
> print text=resume-save-fail
> print text=`saved`
> sys.exit code=1
**loaded = > agent.resume id="f02-sel" resume_dir=`dir`**
**found = > json.get value=`loaded` key="found"**
1. `found`
> print text=resume-load-ok
2. *
> print text=resume-load-fail
> print text=`loaded`
> sys.exit code=1
> agent.resume_clear id="f02-sel" resume_dir=`dir`
> print text=f02-agent-selection-ok
+43
View File
@@ -0,0 +1,43 @@
---
markup: occlusion
title: workbench M01 live selection (USTC-compatible)
description: Explicit ModelSelection via llm.create; no llm.llm. Credentials from process env only.
import:
llm: ext/ai/llm.mq.md
sys: lib/sys.mq.md
json: lib/json.mq.md
---
# main
**base = > sys.env_get name="OPENAI_BASE_URL"**
**key = > sys.env_get name="OPENAI_API_KEY"**
**mdl = > sys.env_get name="OPENAI_MODEL"**
1. not `base`
> print text=missing-OPENAI_BASE_URL
> sys.exit code=2
2. *
**_ = 1**
1. not `key`
> print text=missing-OPENAI_API_KEY
> sys.exit code=2
2. *
**_ = 1**
1. not `mdl`
**mdl = "qwen3.8-chat"**
2. *
**_ = 1**
**model = > llm.create model=`mdl` base_url=`base` api_key=`key` backend="openai-compatible" name=workbench-live**
**q = "Reply with exactly one English word pong"**
**reply = > model.ask prompt=`q`**
**text = > json.get value=`reply` key="text"**
**usage = > json.get value=`reply` key="usage"**
**pt = > json.get value=`usage` key="prompt_tokens"**
> print text=`text`
1. `pt`
> print text=usage-present
2. *
> print text=usage-unknown
@@ -0,0 +1,20 @@
{
"schema": "DocumentViewV1",
"schema_version": 1,
"document": {
"id": "knowledge/example.mq.md",
"title": "知识标题",
"source_revision": "sha256:requested-source",
"semantic_digest": "sha256:semantic-facts",
"generation_id": null,
"presentation_token": "opaque-token"
},
"execution": {"state": "not_run", "run_id": null},
"render": {"format": "sanitized_html", "fragment": "<article class=\"mq-document\"></article>", "renderer_version": "1"},
"outline": [],
"nodes": [],
"refs": [],
"diagnostics": [{"code": "partial", "message": "missing dependency"}],
"completeness": "partial",
"capabilities": {"read_only": true, "has_semantics": true, "executed": false}
}
@@ -0,0 +1,8 @@
{
"ok": false,
"error": {
"code": "view.document_unsupported_format",
"message": "binary formats require extract adapter",
"retryable": false
}
}
@@ -0,0 +1,12 @@
---
title: F08 side-effect probe
status: draft
---
# F08
This document would write a marker if executed. View must never run it.
## mark
> file.write path=".workbench-f08-marker" text="executed"
@@ -0,0 +1,19 @@
{
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "stale",
"precision": "document",
"locator": {
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"project_id": "/authorized/project",
"document_id": "knowledge/example.mq.md",
"source_revision": "sha256:old",
"artifact_id": "mq://project/example/function",
"relation": "cites"
},
"target": null,
"candidates": [],
"diagnostics": [{"code": "source_revision_mismatch", "message": "expected sha256:old, actual sha256:new"}],
"label": "该来源已变化"
}
@@ -0,0 +1,19 @@
{
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"resolution": "version_unavailable",
"precision": "none",
"locator": {
"schema": "KnowledgeLocatorV1",
"schema_version": 1,
"project_id": "/authorized/project",
"document_id": "knowledge/example.mq.md",
"generation_id": "generation-missing",
"artifact_id": "mq://project/example/function",
"relation": "cites"
},
"target": null,
"candidates": [],
"diagnostics": [{"code": "generation_unavailable", "message": "snapshot not retained"}],
"label": "引用版本不可用"
}
@@ -0,0 +1,26 @@
{
"schema": "ModelCallUsageV1",
"schema_version": 1,
"call_id": "call-2",
"attempt_id": "attempt-1",
"parent_call_id": null,
"run_id": "run-1",
"turn_id": "turn-1",
"workspace_scope": "ws-1",
"purpose": "answer",
"selection_id": "sel-1",
"provider_id": "provider-1",
"model_id": "model-2",
"requested_model": "example-model",
"resolved_model": "example-model",
"status": "cancelled",
"usage_state": "partial",
"measurement_source": "provider",
"input_tokens": 100,
"output_tokens": 12,
"total_tokens": 112,
"cached_input_tokens": null,
"reasoning_output_tokens": null,
"revision": 2,
"diagnostics": []
}
@@ -0,0 +1,26 @@
{
"schema": "ModelCallUsageV1",
"schema_version": 1,
"call_id": "call-1",
"attempt_id": "attempt-1",
"parent_call_id": null,
"run_id": "run-1",
"turn_id": "turn-1",
"workspace_scope": "ws-1",
"purpose": "answer",
"selection_id": "sel-1",
"provider_id": "provider-1",
"model_id": "model-2",
"requested_model": "example-model",
"resolved_model": "example-model",
"status": "succeeded",
"usage_state": "unknown",
"measurement_source": "none",
"input_tokens": null,
"output_tokens": null,
"total_tokens": null,
"cached_input_tokens": null,
"reasoning_output_tokens": null,
"revision": 1,
"diagnostics": []
}
@@ -0,0 +1,12 @@
{
"schema": "ModelSelectionV1",
"schema_version": 1,
"error": {
"code": "llm.selection_missing",
"message": "selection_id required in strict mode",
"retryable": false
},
"partial": {
"model_id": "model-2"
}
}
@@ -0,0 +1,17 @@
{
"schema": "ModelSelectionV1",
"schema_version": 1,
"selection_id": "sel-1",
"provider_id": "provider-1",
"model_id": "model-2",
"provider_revision": 3,
"model_revision": 5,
"adapter": "openai-compatible",
"endpoint_ref": "endpoint-local-1",
"credential_ref": "credential-local-1",
"credential_version": 2,
"upstream_model_id": "example-model",
"parameters": {"max_output_tokens": 2048},
"fallback_policy": "disabled",
"auth_mode": "api_key"
}
@@ -0,0 +1,13 @@
{
"schema": "RuntimeEventV1",
"schema_version": 1,
"event_id": "evt-9",
"seq": 9,
"run_id": "run-1",
"parent_run_id": null,
"turn_id": "turn-1",
"workspace_scope": "ws-1",
"timestamp": "2026-10-25T01:00:04Z",
"type": "run.cancelled",
"payload": {"reason": "user_stop", "usage_state": "partial"}
}
@@ -0,0 +1,38 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "marqdo://contracts/DocumentViewV1",
"title": "DocumentViewV1",
"type": "object",
"required": ["schema", "schema_version", "document", "execution", "completeness"],
"properties": {
"schema": { "const": "DocumentViewV1" },
"schema_version": { "const": 1 },
"document": {
"type": "object",
"required": ["id", "source_revision"],
"properties": {
"id": { "type": "string" },
"title": { "type": "string" },
"source_revision": { "type": "string" },
"semantic_digest": { "type": "string" },
"generation_id": { "type": ["string", "null"] },
"presentation_token": { "type": "string" }
}
},
"execution": {
"type": "object",
"required": ["state"],
"properties": {
"state": { "const": "not_run" },
"run_id": { "type": ["string", "null"] }
}
},
"completeness": { "enum": ["complete", "partial"] },
"render": { "type": "object" },
"outline": { "type": "array" },
"nodes": { "type": "array" },
"refs": { "type": "array" },
"diagnostics": { "type": "array" }
},
"additionalProperties": true
}
@@ -0,0 +1,29 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "marqdo://contracts/KnowledgeLocatorV1",
"title": "KnowledgeLocatorV1",
"type": "object",
"required": ["schema_version", "project_id", "document_id"],
"properties": {
"schema": { "const": "KnowledgeLocatorV1" },
"schema_version": { "const": 1 },
"project_id": { "type": "string" },
"document_id": { "type": "string" },
"source_revision": { "type": ["string", "null"] },
"generation_id": { "type": ["string", "null"] },
"artifact_id": { "type": ["string", "null"] },
"node_id": { "type": ["string", "null"] },
"relation": { "type": "string" },
"resolution": {
"enum": [
"resolved",
"stale",
"unresolved",
"ambiguous",
"version_unavailable",
"unsupported"
]
}
},
"additionalProperties": true
}
@@ -0,0 +1,32 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "marqdo://contracts/ModelCallUsageV1",
"title": "ModelCallUsageV1",
"type": "object",
"required": [
"schema_version",
"call_id",
"attempt_id",
"run_id",
"status",
"usage_state",
"measurement_source"
],
"properties": {
"schema": { "const": "ModelCallUsageV1" },
"schema_version": { "const": 1 },
"call_id": { "type": "string", "minLength": 1 },
"attempt_id": { "type": "string", "minLength": 1 },
"run_id": { "type": "string", "minLength": 1 },
"status": {
"enum": ["started", "succeeded", "failed", "cancelled", "interrupted"]
},
"usage_state": { "enum": ["pending", "partial", "final", "unknown"] },
"measurement_source": { "enum": ["provider", "estimated", "none"] },
"input_tokens": { "type": ["integer", "null"], "minimum": 0 },
"output_tokens": { "type": ["integer", "null"], "minimum": 0 },
"total_tokens": { "type": ["integer", "null"], "minimum": 0 },
"revision": { "type": "integer", "minimum": 1 }
},
"additionalProperties": true
}
@@ -0,0 +1,32 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "marqdo://contracts/ModelSelectionV1",
"title": "ModelSelectionV1",
"type": "object",
"required": [
"schema_version",
"selection_id",
"provider_id",
"model_id",
"adapter",
"upstream_model_id"
],
"properties": {
"schema": { "const": "ModelSelectionV1" },
"schema_version": { "const": 1 },
"selection_id": { "type": "string", "minLength": 1 },
"provider_id": { "type": "string", "minLength": 1 },
"model_id": { "type": "string", "minLength": 1 },
"adapter": { "type": "string" },
"upstream_model_id": { "type": "string", "minLength": 1 },
"provider_revision": { "type": ["integer", "null"] },
"model_revision": { "type": ["integer", "null"] },
"endpoint_ref": { "type": ["string", "null"] },
"credential_ref": { "type": ["string", "null"] },
"credential_version": { "type": ["integer", "null"] },
"parameters": { "type": "object" },
"fallback_policy": { "type": "string" },
"auth_mode": { "type": "string" }
},
"additionalProperties": true
}
@@ -0,0 +1,25 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "marqdo://contracts/RuntimeEventV1",
"title": "RuntimeEventV1",
"type": "object",
"required": [
"schema_version",
"event_id",
"seq",
"run_id",
"type",
"payload"
],
"properties": {
"schema": { "const": "RuntimeEventV1" },
"schema_version": { "const": 1 },
"event_id": { "type": "string", "minLength": 1 },
"seq": { "type": "integer", "minimum": 0 },
"run_id": { "type": "string", "minLength": 1 },
"type": { "type": "string", "minLength": 1 },
"payload": { "type": "object" },
"timestamp": { "type": "string" }
},
"additionalProperties": true
}
+815
View File
@@ -0,0 +1,815 @@
//! Workbench M00–M06 contract gate (offline).
//! Spec: doc/design/qdagent-workbench-runtime-and-view-2026-10-25.md
use serde_json::Value;
use std::path::PathBuf;
fn fixture(name: &str) -> Value {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let path = root.join("tests/workbench/fixtures").join(name);
let raw = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("read {}: {e}", path.display()));
serde_json::from_str(&raw).expect("fixture json")
}
fn schema(name: &str) -> Value {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let path = root.join("tests/workbench/schemas").join(name);
let raw = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("read {}: {e}", path.display()));
serde_json::from_str(&raw).expect("schema json")
}
/// Lightweight required-field check against our JSON Schema documents (no jsonschema crate).
fn assert_required(instance: &Value, schema_doc: &Value, label: &str) {
let required = schema_doc["required"].as_array().expect("schema.required");
for key in required {
let k = key.as_str().unwrap();
assert!(
instance.get(k).is_some(),
"{label}: missing required field {k} in {instance}"
);
}
}
fn mlsp(method: &str, params: Value) -> Value {
let req = serde_json::json!({"id": 1, "method": method, "params": params});
let line = marqdo::mlsp::handle_line(&req.to_string());
serde_json::from_str(&line).expect("mlsp response json")
}
#[test]
fn m00_fixtures_consume_and_capabilities() {
let sel = fixture("model_selection_ok.json");
assert_required(&sel, &schema("ModelSelectionV1.schema.json"), "selection");
assert_eq!(sel["schema"], "ModelSelectionV1");
let parsed = marqdo::workbench::parse_model_selection(&sel, true).expect("strict ok");
assert_eq!(parsed.selection_id, "sel-1");
let snap = parsed.public_snapshot();
assert!(snap.get("api_key").is_none());
let usage = fixture("model_call_usage_unknown.json");
assert_required(&usage, &schema("ModelCallUsageV1.schema.json"), "usage");
assert!(usage["input_tokens"].is_null());
assert_ne!(usage["input_tokens"], 0);
assert_eq!(usage["usage_state"], "unknown");
let cancelled = fixture("model_call_usage_cancelled.json");
assert_eq!(cancelled["status"], "cancelled");
assert_eq!(cancelled["usage_state"], "partial");
let doc = fixture("document_view_partial.json");
assert_required(&doc, &schema("DocumentViewV1.schema.json"), "document_view");
assert_eq!(doc["execution"]["state"], "not_run");
let stale = fixture("locator_stale.json");
assert_eq!(stale["resolution"], "stale");
let loc = &stale["locator"];
assert_required(loc, &schema("KnowledgeLocatorV1.schema.json"), "locator");
let unavail = fixture("locator_version_unavailable.json");
assert_eq!(unavail["resolution"], "version_unavailable");
let ev = fixture("runtime_event_cancelled.json");
assert_required(&ev, &schema("RuntimeEventV1.schema.json"), "runtime_event");
assert_eq!(ev["type"], "run.cancelled");
let _ = fixture("model_selection_error.json");
let _ = fixture("document_view_unsupported.json");
let caps = mlsp("capabilities", serde_json::json!({}));
assert_eq!(caps["ok"], true, "{caps}");
let features = &caps["result"]["features"];
assert_eq!(features["view.document"], 1);
assert_eq!(features["view.reference_resolve"], 1);
assert!(caps["result"]["contracts"]
.as_array()
.unwrap()
.iter()
.any(|c| c.as_str() == Some("DocumentViewV1")));
// F16 honesty: unloaded plugins + forced-off stream/cancel → 0
let down = mlsp(
"capabilities",
serde_json::json!({
"force_unloaded": true,
"force_stream_off": true,
"force_cancel_off": true
}),
);
assert_eq!(down["result"]["features"]["llm.messages"], 0);
assert_eq!(down["result"]["features"]["runtime.event_stream"], 0);
assert_eq!(down["result"]["features"]["runtime.cancel"], 0);
}
#[test]
fn m00_strict_selection_rejects_missing() {
let bad = serde_json::json!({"schema_version": 1, "model_id": "x"});
let err = marqdo::workbench::parse_model_selection(&bad, true).unwrap_err();
assert!(err.contains("selection_missing"), "{err}");
}
#[test]
fn m02_usage_null_not_zero() {
let usage = marqdo::workbench::model_call_usage_from_provider(
"c1",
"a1",
"r1",
&serde_json::json!({"selection_id": "s", "provider_id": "p", "model_id": "m"}),
&serde_json::Value::Null,
"succeeded",
);
assert!(usage["input_tokens"].is_null());
assert_eq!(usage["measurement_source"], "none");
assert_eq!(usage["usage_state"], "unknown");
}
#[test]
fn m03_view_document_no_exec() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let resp = mlsp(
"view.document",
serde_json::json!({
"project": root.to_string_lossy(),
"document": "main.mq.md",
"mode": "read",
"include": ["fragment", "outline", "nodes", "diagnostics"]
}),
);
assert_eq!(resp["ok"], true, "{resp}");
let doc = &resp["result"];
assert_eq!(doc["schema"], "DocumentViewV1");
assert_eq!(doc["execution"]["state"], "not_run");
assert_eq!(doc["capabilities"]["read_only"], true);
assert_eq!(doc["capabilities"]["executed"], false);
let fragment = doc["render"]["fragment"].as_str().unwrap_or("");
assert!(fragment.contains("mq-document"), "{fragment}");
assert!(!fragment.contains("<script"), "must not embed scripts");
let outline = doc["outline"].as_array().unwrap();
assert!(
outline.iter().any(|o| o["kind"] == "heading"),
"expected heading outline from # lines: {outline:?}"
);
}
#[test]
fn m03_view_document_revision_mismatch_refuses_downgrade() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let resp = mlsp(
"view.document",
serde_json::json!({
"project": root.to_string_lossy(),
"document": "main.mq.md",
"revision": "sha256:deadbeef",
"mode": "read"
}),
);
assert_eq!(resp["ok"], false, "{resp}");
assert_eq!(resp["error"]["code"], "view.revision_mismatch");
}
#[test]
fn m04_reference_resolve_unresolved() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let resp = mlsp(
"view.reference_resolve",
serde_json::json!({
"project": root.to_string_lossy(),
"project_id": root.to_string_lossy(),
"document_id": "main.mq.md",
"artifact_id": "does-not-exist-xyz"
}),
);
assert_eq!(resp["ok"], true, "{resp}");
assert_eq!(resp["result"]["resolution"], "unresolved");
}
#[test]
fn m04_generation_mismatch_version_unavailable() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let resp = mlsp(
"view.reference_resolve",
serde_json::json!({
"project": root.to_string_lossy(),
"project_id": root.to_string_lossy(),
"document_id": "main.mq.md",
"generation_id": "not-a-real-generation",
"artifact_id": "商品小计"
}),
);
assert_eq!(resp["ok"], true, "{resp}");
assert_eq!(resp["result"]["resolution"], "version_unavailable");
}
#[test]
fn m04_source_revision_mismatch_stale() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let resp = mlsp(
"view.reference_resolve",
serde_json::json!({
"project": root.to_string_lossy(),
"project_id": root.to_string_lossy(),
"document_id": "main.mq.md",
"source_revision": "sha256:deadbeef",
"artifact_id": "商品小计"
}),
);
assert_eq!(resp["ok"], true, "{resp}");
assert_eq!(resp["result"]["resolution"], "stale");
assert_eq!(
resp["result"]["diagnostics"][0]["code"],
"source_revision_mismatch"
);
}
#[test]
fn m04_artifact_inspect_version_mismatch() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let ok = mlsp(
"artifact.inspect",
serde_json::json!({
"project": root.to_string_lossy(),
"artifact_id": "商品小计"
}),
);
assert_eq!(ok["ok"], true, "{ok}");
let bad_rev = mlsp(
"artifact.inspect",
serde_json::json!({
"project": root.to_string_lossy(),
"artifact_id": "商品小计",
"expected_source_revision": "sha256:deadbeef"
}),
);
assert_eq!(bad_rev["ok"], false, "{bad_rev}");
assert_eq!(bad_rev["error"]["code"], "artifact.version_mismatch");
let bad_gen = mlsp(
"artifact.inspect",
serde_json::json!({
"project": root.to_string_lossy(),
"artifact_id": "商品小计",
"expected_generation": "not-a-real-generation"
}),
);
assert_eq!(bad_gen["ok"], false, "{bad_gen}");
assert_eq!(bad_gen["error"]["code"], "artifact.version_mismatch");
}
#[test]
fn m05_extract_segment_byte_locator() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let path = root.join("tests/executable_knowledge/baseline/02-extract/main.mq.md");
let doc = marqdo::document_extract::extract_path(&path);
assert!(doc.status == "ok" || doc.status == "partial", "{:?}", doc.status);
assert!(
doc.segments.iter().any(|s| s.locator.contains(":B")),
"expected byte offset in locator: {:?}",
doc.segments
);
}
#[test]
fn m06_capabilities_lists_adapters_honestly() {
let caps = mlsp("capabilities", serde_json::json!({}));
let adapters = caps["result"]["adapters"].as_array().unwrap();
let native = adapters
.iter()
.find(|a| a["id"] == "anthropic-native")
.unwrap();
assert_eq!(native["available"], false);
}
#[test]
fn m01_run_context_spawn_child_inherits_selection() {
let parent = marqdo::workbench::RunContext::from_value(&serde_json::json!({
"run_id": "run-parent",
"turn_id": "turn-1",
"workspace_scope": "ws-a",
"selection_context": {
"selection_id": "sel-1",
"upstream_model_id": "model-a",
"base_url": "http://127.0.0.1:9"
},
"cancellation_token": "/tmp/mq-cancel-test"
}))
.unwrap();
let child = parent.spawn_child("run-child");
assert_eq!(child.parent_run_id.as_deref(), Some("run-parent"));
assert_eq!(child.selection_context["upstream_model_id"], "model-a");
let env = child.selection_env();
assert!(env.iter().any(|(k, v)| k == "OPENAI_MODEL" && v == "model-a"));
assert!(env.iter().any(|(k, _)| k == "MARQDO_CANCEL_TOKEN"));
// Public JSON must not invent api_key.
assert!(child.to_json().get("api_key").is_none());
}
#[test]
fn m01_f01_parallel_selections_do_not_cross() {
let a = marqdo::workbench::RunContext::from_value(&serde_json::json!({
"run_id": "run-a",
"selection_context": {
"upstream_model_id": "model-a",
"base_url": "http://127.0.0.1:7001"
}
}))
.unwrap();
let b = marqdo::workbench::RunContext::from_value(&serde_json::json!({
"run_id": "run-b",
"selection_context": {
"upstream_model_id": "model-b",
"base_url": "http://127.0.0.1:7002"
}
}))
.unwrap();
let ea = a.selection_env();
let eb = b.selection_env();
let model_a = ea.iter().find(|(k, _)| k == "OPENAI_MODEL").map(|(_, v)| v.as_str());
let model_b = eb.iter().find(|(k, _)| k == "OPENAI_MODEL").map(|(_, v)| v.as_str());
let base_a = ea.iter().find(|(k, _)| k == "OPENAI_BASE_URL").map(|(_, v)| v.as_str());
let base_b = eb.iter().find(|(k, _)| k == "OPENAI_BASE_URL").map(|(_, v)| v.as_str());
assert_eq!(model_a, Some("model-a"));
assert_eq!(model_b, Some("model-b"));
assert_ne!(base_a, base_b);
assert_ne!(a.run_id, b.run_id);
}
#[test]
fn m02_usage_revision_and_cancel_token() {
let u1 = marqdo::workbench::model_call_usage_from_provider(
"c1",
"a1",
"r1",
&serde_json::json!({"selection_id": "s"}),
&serde_json::json!({"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12}),
"succeeded",
);
assert_eq!(u1["usage_state"], "final");
assert_eq!(u1["input_tokens"], 10);
assert_eq!(u1["revision"], 1);
let dir = std::env::temp_dir().join(format!("mq-cancel-{}", std::process::id()));
let _ = std::fs::remove_file(&dir);
std::fs::write(&dir, "cancelled").unwrap();
assert!(marqdo::workbench::cancel_requested(Some(dir.to_str().unwrap())));
let _ = std::fs::remove_file(&dir);
assert!(!marqdo::workbench::cancel_requested(Some("/no/such/cancel/token")));
}
#[test]
fn m02_f05_usage_revision_replaces_same_attempt_not_sum() {
let sel = serde_json::json!({"selection_id": "s"});
let partial = marqdo::workbench::model_call_usage_from_provider_rev(
"c1",
"a1",
"r1",
&sel,
&serde_json::json!({"prompt_tokens": 10, "completion_tokens": 1, "total_tokens": 11}),
"started",
1,
);
let final_u = marqdo::workbench::model_call_usage_from_provider_rev(
"c1",
"a1",
"r1",
&sel,
&serde_json::json!({"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15}),
"succeeded",
2,
);
let won = marqdo::workbench::replace_usage_if_newer(&partial, &final_u);
assert_eq!(won["revision"], 2);
assert_eq!(won["total_tokens"], 15);
// Older revision must not overwrite.
let kept = marqdo::workbench::replace_usage_if_newer(&final_u, &partial);
assert_eq!(kept["revision"], 2);
assert_eq!(kept["total_tokens"], 15);
// New attempt is a distinct identity (caller must not merge).
let retry = marqdo::workbench::model_call_usage_from_provider_rev(
"c1",
"a2",
"r1",
&sel,
&serde_json::json!({"prompt_tokens": 3, "completion_tokens": 1, "total_tokens": 4}),
"succeeded",
1,
);
assert_ne!(retry["attempt_id"], final_u["attempt_id"]);
}
#[test]
fn m02_f07_cancel_does_not_undo_external_write() {
let marker = std::env::temp_dir().join(format!("mq-f07-{}", std::process::id()));
std::fs::write(&marker, "already-written").unwrap();
let tok = std::env::temp_dir().join(format!("mq-f07-cancel-{}", std::process::id()));
std::fs::write(&tok, "cancelled").unwrap();
assert!(marqdo::workbench::cancel_requested(Some(tok.to_str().unwrap())));
// Cancel must not claim or perform undo of prior external writes.
assert_eq!(
std::fs::read_to_string(&marker).unwrap(),
"already-written"
);
let _ = std::fs::remove_file(&marker);
let _ = std::fs::remove_file(&tok);
}
#[test]
fn m02_event_sink_emits_runtime_event() {
let path = std::env::temp_dir().join(format!("mq-evt-{}.ndjson", std::process::id()));
let _ = std::fs::remove_file(&path);
marqdo::workbench::bind_event_sink(&path);
let ev = marqdo::workbench::emit_message_delta("run-evt", "hello");
assert_eq!(ev["schema"], "RuntimeEventV1");
assert_eq!(ev["type"], "message.delta");
marqdo::workbench::clear_event_sink();
let body = std::fs::read_to_string(&path).unwrap_or_default();
assert!(body.contains("message.delta"), "{body}");
let _ = std::fs::remove_file(&path);
}
#[test]
fn m03_view_document_path_escape_rejected() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let resp = mlsp(
"view.document",
serde_json::json!({
"project": root.to_string_lossy(),
"document": "../Cargo.toml",
"mode": "read"
}),
);
assert_eq!(resp["ok"], false, "{resp}");
let code = resp["error"]["code"].as_str().unwrap_or("");
assert!(
code.contains("path") || code.contains("not_found") || code.contains("escape"),
"{resp}"
);
}
#[test]
fn m05_project_cache_isolation_f14() {
marqdo::workbench::reset_cache_for_tests();
let a = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let b = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/01-inline");
assert!(
marqdo::workbench::analysis_cache::cache_keys_isolated(&a, &b),
"distinct project roots must not share cache keys"
);
let _ = marqdo::workbench::analysis_cache::note_analysis(&a);
let hit = marqdo::workbench::analysis_cache::note_analysis(&a);
assert!(hit, "second analysis of same snapshot should hit cache");
let _ = marqdo::workbench::analysis_cache::note_analysis(&b);
}
#[test]
fn m05_f14_concurrent_100_same_snapshot() {
use std::sync::{Arc, Barrier};
use std::thread;
marqdo::workbench::reset_cache_for_tests();
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let root_s = root.to_string_lossy().to_string();
let n = 100usize;
let barrier = Arc::new(Barrier::new(n));
let mut handles = Vec::with_capacity(n);
for _ in 0..n {
let barrier = Arc::clone(&barrier);
let root_s = root_s.clone();
handles.push(thread::spawn(move || {
barrier.wait();
let resp = mlsp(
"view.document",
serde_json::json!({
"project": root_s,
"document": "main.mq.md",
"mode": "read",
"include": ["fragment"]
}),
);
assert_eq!(resp["ok"], true, "{resp}");
resp["result"]["document"]["source_revision"]
.as_str()
.unwrap_or("")
.to_string()
}));
}
let mut revs = Vec::new();
for h in handles {
revs.push(h.join().expect("thread"));
}
assert!(revs.iter().all(|r| r == &revs[0]), "all reads same revision");
assert!(revs[0].starts_with("sha256:"));
let (hits, misses, merges) = marqdo::workbench::cache_stats();
// At least one miss (leader) and either merges (coalesce) or subsequent hits.
assert!(misses >= 1, "hits={hits} misses={misses} merges={merges}");
assert!(
merges > 0 || hits > 0,
"expected coalesce or reuse: hits={hits} misses={misses} merges={merges}"
);
}
#[test]
fn m04_snapshot_manifest_on_document_view() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let resp = mlsp(
"view.document",
serde_json::json!({
"project": root.to_string_lossy(),
"document": "main.mq.md",
"mode": "read",
"include": ["fragment"]
}),
);
assert_eq!(resp["ok"], true, "{resp}");
let snap = &resp["result"]["snapshot"];
assert_eq!(snap["schema"], "SnapshotManifestV1");
assert_eq!(snap["retention"]["owner"], "application");
assert!(snap["source_revision"].as_str().unwrap().starts_with("sha256:"));
assert_eq!(
snap["source_revision"],
resp["result"]["document"]["source_revision"]
);
}
#[test]
fn m01_f02_run_context_three_level_inherit() {
let root = marqdo::workbench::RunContext::from_value(&serde_json::json!({
"run_id": "run-root",
"turn_id": "turn-1",
"workspace_scope": "ws-1",
"selection_context": {
"selection_id": "sel-1",
"upstream_model_id": "model-fixed",
"base_url": "http://127.0.0.1:7700"
},
"cancellation_token": "/tmp/mq-cancel-f02"
}))
.unwrap();
let child = root.spawn_child("run-child");
let grand = child.spawn_child("run-grand");
assert_eq!(grand.parent_run_id.as_deref(), Some("run-child"));
assert_eq!(
grand.selection_context["upstream_model_id"],
"model-fixed"
);
assert_eq!(grand.turn_id.as_deref(), Some("turn-1"));
assert_eq!(
grand.cancellation_token.as_deref(),
Some("/tmp/mq-cancel-f02")
);
let env = grand.selection_env();
assert!(env.iter().any(|(k, v)| k == "OPENAI_MODEL" && v == "model-fixed"));
// Root selection must not be mutated by descendants.
assert_eq!(root.selection_context["upstream_model_id"], "model-fixed");
assert_eq!(root.run_id, "run-root");
}
#[test]
fn m03_f08_side_effect_marker_not_created_by_view() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/workbench/fixtures");
let marker = root.join(".workbench-f08-marker");
let _ = std::fs::remove_file(&marker);
let resp = mlsp(
"view.document",
serde_json::json!({
"project": root.to_string_lossy(),
"document": "f08_write_side_effect.mq.md",
"mode": "read",
"include": ["fragment", "diagnostics"]
}),
);
assert_eq!(resp["ok"], true, "{resp}");
assert_eq!(resp["result"]["execution"]["state"], "not_run");
assert!(
!marker.exists(),
"view.document must not execute file.write side effects"
);
}
#[test]
fn m03_f09_step_kinds_baseline_projects_nodes() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/07-step-kinds");
let resp = mlsp(
"view.document",
serde_json::json!({
"project": root.to_string_lossy(),
"document": "main.mq.md",
"mode": "read",
"include": ["outline", "nodes", "fragment"]
}),
);
assert_eq!(resp["ok"], true, "{resp}");
let outline = resp["result"]["outline"].as_array().unwrap();
assert!(
outline.iter().any(|o| o["kind"] == "heading"),
"expected heading outline: {outline:?}"
);
let nodes = resp["result"]["nodes"].as_array().unwrap();
assert!(nodes.len() >= 2, "expected artifact+step nodes: {nodes:?}");
let kinds: std::collections::HashSet<&str> = nodes
.iter()
.filter_map(|n| n["kind"].as_str())
.collect();
assert!(
kinds.iter().any(|k| matches!(
*k,
"observe" | "derive" | "call" | "iterate" | "narrate" | "type" | "function"
)),
"expected step/artifact kinds in read model: {kinds:?}"
);
// Evidence statuses must not be silently upgraded to verified by the read path.
for n in nodes {
if let Some(ev) = n.get("evidence").and_then(|v| v.as_str()) {
assert_ne!(ev, "verified", "read path must not invent verified: {n}");
}
}
let fragment = resp["result"]["render"]["fragment"].as_str().unwrap_or("");
assert!(fragment.contains("mq-document"));
assert!(!fragment.contains("<script"));
}
#[test]
fn m03_f09_outline_nodes_chinese_and_static_a11y_subset() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/02-extract");
let resp = mlsp(
"view.document",
serde_json::json!({
"project": root.to_string_lossy(),
"document": "main.mq.md",
"mode": "read",
"include": ["fragment", "outline", "nodes", "refs"]
}),
);
assert_eq!(resp["ok"], true, "{resp}");
let doc = &resp["result"];
let outline = doc["outline"].as_array().unwrap();
assert!(outline.iter().any(|o| o["kind"] == "heading"));
let nodes = doc["nodes"].as_array().unwrap();
assert!(!nodes.is_empty(), "expected semantic nodes");
// Chinese / CJK labels should round-trip in outline or nodes when present in source.
let blob = format!("{outline:?}{nodes:?}");
assert!(
blob.chars().any(|c| c >= '\u{4e00}'),
"expected CJK in outline/nodes for 02-extract: {blob}"
);
let fragment = doc["render"]["fragment"].as_str().unwrap_or("");
// F12 static HTML subset (full browser → qdagent): semantic article, no script, mq namespace.
assert!(fragment.starts_with("<article") || fragment.contains("mq-document"));
assert!(!fragment.contains("<script"));
assert!(!fragment.contains("javascript:"));
}
#[test]
fn m04_f11_ambiguous_same_name() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/executable_knowledge/baseline/08-same-name");
assert!(root.is_dir(), "missing 08-same-name fixture");
// a.mq.md and b.mq.md both define 商品小计 — omit document_id so both stay in scope.
let resp = mlsp(
"view.reference_resolve",
serde_json::json!({
"project": root.to_string_lossy(),
"project_id": root.to_string_lossy(),
"artifact_id": "商品小计"
}),
);
assert_eq!(resp["ok"], true, "{resp}");
assert_eq!(resp["result"]["resolution"], "ambiguous", "{resp}");
let cands = resp["result"]["candidates"].as_array().unwrap();
assert!(cands.len() >= 2, "{resp}");
}
/// F02: agent.step + resume with explicit llm.create selection against a local mock.
/// Env poison model must not appear in the upstream request body.
#[test]
fn m01_f02_agent_step_selection_and_resume() {
use std::io::{Read, Write};
use std::net::TcpListener;
use std::process::Command;
use std::sync::{Arc, Mutex};
use std::thread;
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let ext = root.join("ext");
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_llm", "-p", "marqdo_plugin_agent", "--locked"])
.current_dir(&root)
.status()
.expect("build plugins");
assert!(status.success(), "plugin build failed");
let llm_so = root.join("target/debug/libllm.so");
let agent_so = root.join("target/debug/libagent.so");
assert!(llm_so.is_file(), "missing {llm_so:?}");
assert!(agent_so.is_file(), "missing {agent_so:?}");
let seen = Arc::new(Mutex::new(Vec::new()));
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
let seen_bg = seen.clone();
thread::spawn(move || {
// Agent may deepen once (READ) then answer — accept a few requests.
for _ in 0..4 {
let Ok((mut stream, _)) = listener.accept() else {
break;
};
let mut raw = Vec::new();
let mut buf = [0u8; 4096];
loop {
let n = stream.read(&mut buf).unwrap_or(0);
if n == 0 {
break;
}
raw.extend_from_slice(&buf[..n]);
if let Some(pos) = raw.windows(4).position(|w| w == b"\r\n\r\n") {
let header = String::from_utf8_lossy(&raw[..pos]);
let cl = header
.lines()
.find_map(|l| l.strip_prefix("Content-Length:"))
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(0);
let body_start = pos + 4;
while raw.len() < body_start + cl {
let n = stream.read(&mut buf).unwrap_or(0);
if n == 0 {
break;
}
raw.extend_from_slice(&buf[..n]);
}
break;
}
}
let req = String::from_utf8_lossy(&raw).to_string();
seen_bg.lock().unwrap().push(req);
let body = r#"{"choices":[{"message":{"content":"pong"},"finish_reason":"stop"}],"usage":{"prompt_tokens":3,"completion_tokens":1,"total_tokens":4}}"#;
let resp = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}",
body.len(),
body
);
let _ = stream.write_all(resp.as_bytes());
}
});
let base = format!("http://{addr}/v1");
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", "tests/ext/workbench-agent-selection.mq.md"])
.current_dir(&root)
.env("MARQDO_EXT", &ext)
.env("MARQDO_LLM_PLUGIN", &llm_so)
.env("MARQDO_AGENT_PLUGIN", &agent_so)
.env("OPENAI_BASE_URL", &base)
.env("OPENAI_API_KEY", "f02-key")
.env("OPENAI_MODEL", "f02-selected-model")
.output()
.expect("run workbench-agent-selection");
let stdout = String::from_utf8_lossy(&output.stdout);
let stderr = String::from_utf8_lossy(&output.stderr);
assert_eq!(
output.status.code().unwrap_or(1),
0,
"stderr={stderr}\nstdout={stdout}"
);
assert!(
stdout.contains("f02-agent-selection-ok"),
"stdout={stdout}"
);
assert!(stdout.contains("resume-save-ok") && stdout.contains("resume-load-ok"));
let captured = seen.lock().unwrap().join("\n");
assert!(
!captured.is_empty(),
"mock received no HTTP; stdout={stdout}"
);
// Body may embed the .mq.md source (which mentions the poison string in prose).
// Assert the OpenAI `model` field itself — not a raw substring of the whole HTTP dump.
let model_field = captured
.split("\r\n\r\n")
.nth(1)
.unwrap_or("")
.lines()
.next()
.unwrap_or("");
let body: Value = serde_json::from_str(model_field).unwrap_or_else(|_| {
// Some captures concatenate multiple HTTP messages; take last `{...}` blob.
let start = captured.rfind('{').expect("json body");
serde_json::from_str(&captured[start..]).expect("parse chat body")
});
assert_eq!(
body["model"].as_str(),
Some("f02-selected-model"),
"upstream model field: {body}"
);
assert_ne!(body["model"].as_str(), Some("env-poison-model"));
}
+140
View File
@@ -0,0 +1,140 @@
//! Live workbench gates — skipped unless OPENAI_API_KEY is set (never commit secrets).
use std::process::Command;
fn live_key() -> Option<String> {
for k in ["OPENAI_API_KEY", "MARQDO_LLM_API_KEY"] {
if let Ok(v) = std::env::var(k) {
let t = v.trim();
if !t.is_empty() && t != "sk-test" {
return Some(t.to_string());
}
}
}
None
}
fn repo_ext() -> std::path::PathBuf {
std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("ext")
}
fn ensure_llm_plugin() {
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_llm", "--locked"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.status()
.expect("build llm plugin");
assert!(status.success());
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let so = root.join("target/debug/libllm.so");
if so.is_file() {
std::env::set_var("MARQDO_LLM_PLUGIN", &so);
}
// Prefer in-tree ext/ over ~/.marqdo/ext (search roots list user install first).
std::env::set_var("MARQDO_EXT", repo_ext());
}
#[test]
fn live_selection_ask_pong() {
let Some(_key) = live_key() else {
eprintln!("skip live_selection_ask_pong: no API key");
return;
};
ensure_llm_plugin();
if std::env::var_os("OPENAI_BASE_URL").is_none() {
eprintln!("skip: set OPENAI_BASE_URL (e.g. https://api.llm.ustc.edu.cn/v1)");
return;
}
if std::env::var_os("OPENAI_MODEL").is_none() {
std::env::set_var("OPENAI_MODEL", "qwen3.8-chat");
}
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", "tests/ext/workbench-selection-live.mq.md"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.env("MARQDO_EXT", repo_ext())
.env("MARQDO_LLM_PLUGIN", std::env::var_os("MARQDO_LLM_PLUGIN").unwrap_or_default())
.output()
.expect("run workbench-selection-live");
let code = output.status.code().unwrap_or(1);
let stdout = String::from_utf8_lossy(&output.stdout);
let stderr = String::from_utf8_lossy(&output.stderr);
assert_eq!(code, 0, "stderr={stderr}\nstdout={stdout}");
let lower = stdout.to_lowercase();
assert!(lower.contains("pong"), "expected pong, got {stdout:?}");
assert!(
stdout.contains("usage-present") || stdout.contains("usage-unknown"),
"expected usage line, got {stdout:?}"
);
}
#[test]
fn live_dual_model_isolation_via_plugin() {
let Some(key) = live_key() else {
eprintln!("skip live_dual_model_isolation_via_plugin: no API key");
return;
};
let base = match std::env::var("OPENAI_BASE_URL") {
Ok(v) if !v.trim().is_empty() => v,
_ => {
eprintln!("skip: OPENAI_BASE_URL required");
return;
}
};
ensure_llm_plugin();
// Two sequential asks with different model ids — both must succeed without env clobber.
for model in ["qwen3.8-chat", "deepseek-flash"] {
std::env::set_var("OPENAI_MODEL", model);
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", "tests/ext/workbench-selection-live.mq.md"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.env("OPENAI_API_KEY", &key)
.env("OPENAI_BASE_URL", &base)
.env("OPENAI_MODEL", model)
.env("MARQDO_EXT", repo_ext())
.env(
"MARQDO_LLM_PLUGIN",
std::env::var_os("MARQDO_LLM_PLUGIN").unwrap_or_default(),
)
.output()
.expect("run");
let stdout = String::from_utf8_lossy(&output.stdout);
let stderr = String::from_utf8_lossy(&output.stderr);
assert_eq!(
output.status.code().unwrap_or(1),
0,
"model={model} stderr={stderr} stdout={stdout}"
);
assert!(
stdout.to_lowercase().contains("pong"),
"model={model} stdout={stdout}"
);
}
}
#[test]
fn cli_view_document_writes_embed_html() {
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let out = root.join("target/workbench-docview.html");
let _ = std::fs::remove_file(&out);
let status = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args([
"view",
"document",
"--project",
root.join("tests/executable_knowledge/baseline/02-extract")
.to_str()
.unwrap(),
"--document",
"main.mq.md",
"--out",
out.to_str().unwrap(),
])
.current_dir(&root)
.status()
.expect("view document");
assert!(status.success());
let html = std::fs::read_to_string(&out).expect("read html");
assert!(html.contains("mq-document"));
assert!(!html.contains("<script"));
assert!(html.contains("execution") || html.contains("mq-document"));
}