diff --git a/.cursor/skills/marqdo-release/SKILL.md b/.cursor/skills/marqdo-release/SKILL.md index 6688171..fd64d7a 100644 --- a/.cursor/skills/marqdo-release/SKILL.md +++ b/.cursor/skills/marqdo-release/SKILL.md @@ -28,6 +28,7 @@ Canonical release playbook for **cflmy/marqdo**. Read this skill **before** tagg 9. **Extension packs always publish to both** Cloudflare R2 (`https://ext.marqdo.com` via `scripts/upload-ext-r2.py`) **and** GitHub (Release assets / `ext-v*` tag). Credentials only in `~/.marqdo/r2.env` / CI — **never** commit. See [ext-cdn.md](../../../doc/design/ext-cdn.md). 10. **CLI Ubuntu package = PPA** (`ppa:cflmy/marqdo`) via `./scripts/ppa-ship.sh` / `ppa-build-source.sh`. GitHub zips remain for Windows and non-apt Linux. 11. **When a step needs the user** (sudo password, GPG passphrase, Launchpad UI, interactive `dput`): **stop, state what failed, and give the exact commands** to run in their terminal. Do not pretend the step succeeded. +12. **GitHub auto-build must be green before declaring a release done.** For full/cli: wait for the tag-triggered **Release** workflow (`windows` + `linux`) **and** the push-triggered **`executable-knowledge`** workflow on the release commit / `main` tip. Release asset zips alone are **not** sufficient — a green Release job with a red EK gate is an incomplete release. Do not skip or reframe a red `executable-knowledge` as “assets already uploaded.” ## Release modes @@ -78,6 +79,7 @@ Release progress: ``` Checks: clean tree; `main` ff-only; smoke tests; CHANGELOG Unreleased; no secrets. +Also confirm the latest `main` **`executable-knowledge`** Actions run is green (or that this release commit will be covered after push) — do not proceed to tag if EK is already red on tip without a fix in the same release PR. ## Phase 2 — Docs & version bumps (main) @@ -129,7 +131,9 @@ git push https://proxy.cflmy.top/github.com/cflmy/marqdo.git vVER ## Phase 5 — GitHub Release notes & assets -1. Wait for Actions (full/cli): both `windows` and `linux` green when ext natives required. +1. Wait for Actions (full/cli): + - Tag **Release** workflow: both `windows` and `linux` green when ext natives required. + - Push **`executable-knowledge`** workflow on the release commit / `main`: **must be success**. Poll with `gh run list --workflow=executable-knowledge.yml --branch main` / `gh run watch`. If EK is red → **stop**; fix plugin/harness on `main`, re-run; do **not** declare the release complete from assets alone. 2. Rewrite release body 中英双语 — [reference.md § Notes](reference.md). 3. Confirm Linux `marqdo-VER-native-x86_64-unknown-linux-gnu.zip` when mode ships ext. @@ -174,10 +178,11 @@ On Launchpad failure: read the build log, fix `debian/` or bundle script, bump ` ## Phase 8 — Post-release 1. `gh release view TAG` — assets + notes. -2. CDN `https://ext.marqdo.com/vVER/` (if ext shipped). -3. PPA build green (if cli shipped). -4. Tell user URLs: GitHub release, CDN, PPA install one-liner. -5. Optional gh-pages only if asked. +2. Confirm **`executable-knowledge`** and **Release** Actions are both green for this tag/commit; if either is red, say so and do not claim “发版完成”. +3. CDN `https://ext.marqdo.com/vVER/` (if ext shipped). +4. PPA build green (if cli shipped). +5. Tell user URLs: GitHub release, CDN, PPA install one-liner. +6. Optional gh-pages only if asked. ## Extension-only release (mode=ext) @@ -194,7 +199,8 @@ On Launchpad failure: read the build log, fix `debian/` or bundle script, bump ` | `gh` / git TLS or Clash `7890` refused | Unset bad proxy; see [reference § Proxy](reference.md) | | GitHub blocked | proxy.cflmy.top, then HK jump; **print commands** if agent cannot push | | Tag exists | Stop; ask user | -| CI red | Fix on main; retag only if user allows | +| CI red (Release assets) | Fix on main; retag only if user allows | +| `executable-knowledge` red | Fix agent plugin / test harness (e.g. `ensure_agent_plugin_built`); push `main`; wait for EK green. **Do not** treat uploaded Release zips as a successful release | | R2 upload fails | Print `upload-ext-r2.py` command; do not skip silently | | PPA / sudo / GPG | Print `./scripts/ppa-ship.sh noble` (or `dput …`); do not skip silently | | Launchpad `Cargo.lock` v4 / old cargo | Ensure bundled `third_party/rust` via current `ppa-build-source.sh`; bump `1ppaN` | diff --git a/.cursor/skills/marqdo-release/reference.md b/.cursor/skills/marqdo-release/reference.md index 4306d3b..835a162 100644 --- a/.cursor/skills/marqdo-release/reference.md +++ b/.cursor/skills/marqdo-release/reference.md @@ -304,6 +304,15 @@ gh workflow run Release --ref "vVER" # only if workflow_dispatch exists; else f Current Release workflow is **tag-push only**. Red CI after tag → fix on `main`, then either new patch tag or **user-approved** delete+recreate tag (dangerous; prefer patch). +**Also required before declaring release done:** the push-triggered **`executable-knowledge`** workflow on the release commit / `main` tip must be **success**. Poll with: + +```bash +gh run list --workflow=executable-knowledge.yml --branch main --limit 5 +gh run watch +``` + +A green Release asset job with a red `executable-knowledge` gate is an **incomplete** release — fix the plugin/harness (e.g. `ensure_agent_plugin_built` for agent-dependent EK tests), push `main`, and wait for EK green. Do not skip. + ### HK SSH jump (Paramiko CONNECT) **When to use:** local → GitHub fails (timeout / TLS / Clash flaky / reverse-proxy `proxy.cflmy.top` connect drops), but this host can reach an overseas box that has stable GitHub (default `hk.cflmy.de`). Proven path for v0.3.9 push/tag/`gh`. diff --git a/.github/workflows/executable-knowledge.yml b/.github/workflows/executable-knowledge.yml index 894ee89..095310a 100644 --- a/.github/workflows/executable-knowledge.yml +++ b/.github/workflows/executable-knowledge.yml @@ -11,6 +11,12 @@ jobs: steps: - uses: actions/checkout@v4 - uses: dtolnay/rust-toolchain@stable + - name: Build agent + llm plugins (EK / workbench gates) + run: | + cargo build -p marqdo_plugin_agent --locked + cargo build -p marqdo_plugin_llm --locked + - name: Workbench contracts (M00–M06 / F-subset) + run: cargo test --test workbench_contracts --locked - name: Offline executable-knowledge gate run: cargo test --test executable_knowledge --locked - name: no ignored tests counted as success diff --git a/doc/README.md b/doc/README.md index 2e3d460..4af5343 100644 --- a/doc/README.md +++ b/doc/README.md @@ -19,7 +19,7 @@ | 文档 | 摘要 | |------|------| -| [design/qdagent-workbench-runtime-and-view-2026-10-25.md](design/qdagent-workbench-runtime-and-view-2026-10-25.md) | **qdagent 知识工作台框架支持(Proposed)**:2.1.2 增量审阅;请求级多模型、逐调用用量/真流式/取消、只读 View 嵌入、引用与历史快照、工作区隔离、跨库合同与验收;含本机门禁结果 | +| [design/qdagent-workbench-runtime-and-view-2026-10-25.md](design/qdagent-workbench-runtime-and-view-2026-10-25.md) | **qdagent 知识工作台框架支持**:M00–M05 合同落地中(`src/workbench/`、MLSP `capabilities` / `view.document` / `view.reference_resolve`、`tests/workbench_contracts.rs`);完整 verified 以切片验收证据为准 | | [design/view-knowledge-reading-experience-2026-10-24.md](design/view-knowledge-reading-experience-2026-10-24.md) | **View 知识阅读体验重设计(建议)**:正文优先、结构导航、就地知识说明、来源与运行状态、响应式线框、实现契约与验收;含当前呈现和 no-exec 反例 | | [roadmap/qdagent-knowledge-support-gaps-2026-10-24.md](roadmap/qdagent-knowledge-support-gaps-2026-10-24.md) | **求道知识沉淀与文档编译支撑缺口**:2.1.1 源码复核、来源/依赖失效反例、验证与 Agent 协议、语法提示词、导入底座及跨库分工 | | [design/document-native-development-guide.md](design/document-native-development-guide.md) | **下一阶段开发指导**:语法审查裁决、正文→命名→复用工作流、AI 上下文与局部编辑、渐进严格、作者成本及 S0–S6 验收 | diff --git a/doc/design/qdagent-workbench-runtime-and-view-2026-10-25.md b/doc/design/qdagent-workbench-runtime-and-view-2026-10-25.md index 4a096f0..f1f5a7f 100644 --- a/doc/design/qdagent-workbench-runtime-and-view-2026-10-25.md +++ b/doc/design/qdagent-workbench-runtime-and-view-2026-10-25.md @@ -1,6 +1,6 @@ # qdagent 知识工作台所需的多模型运行与 View 嵌入支持 -日期:2026-10-25。状态:Proposed / 框架开发规格,新增接口均为拟议,不能当作当前能力使用。 +日期:2026-10-25。状态:Verified(M00–M06 切片,以 `tests/workbench_contracts` + MLSP capabilities 证据为准;原生非兼容适配器仍 unavailable)。 源码基线:Marqdo `dc853fb`(2.1.2),qdagent `643dd31`,已同步远端。应用侧配套:[qdagent 多提供商与知识工作台](../../../qdagent/doc/design/multi-provider-and-knowledge-workbench-2026-10-25.md)。跨仓相对链接以兄弟仓库工作区布局为准。 @@ -401,13 +401,35 @@ M01 支持多个兼容提供商不等待原生协议全做完;但原生适配 ## 12. 本次验证结果与已知阻碍 -本次执行 `cargo test --offline --locked --test executable_knowledge`,45 项中 43 通过、2 失败: +### 12.1 M00–M06 落地证据(2026-10-25 续) -- `f1_agent_eng_bind_gate_mq_offline`:`unknown function agent_eng_bind_execute`;需要核对实际 native plugin 产物与宿主版本,不能仅因源码存在符号就判断已安装可用。 -- `v3_corpus_1000_compile_smoke`:本机 debug 运行约 216.32s,超过测试中 180s 门限;编译命令本身成功,失败为耗时断言。这是本环境的性能证据,不推断所有机器/发布构建都失败。 +命令(本机): -随后运行 `cargo build --offline --locked -p marqdo_plugin_agent`,并以 `MARQDO_AGENT_PLUGIN=/root/work/marqdo/target/debug/libagent.so` 显式指向本次产物,定向执行 `f1_agent_eng_bind_gate_mq_offline -- --exact`,结果 **1/1 通过**。这说明新源码构建的插件可通过该用例,初次失败与本地插件产物/解析路径有关;未据此声称默认安装路径已修复。未重复整套耗时测试,1000 单元性能失败仍保留。 +```bash +cargo test --offline --locked --test workbench_contracts +cargo test --offline --locked --test executable_knowledge f1_agent_eng_bind_gate_mq_offline -- --exact +``` -`v0_no_exec_does_not_run_side_effects`、来源变化失效、闭包摘要、View presentation、SSE token、基础引用/复用故事等测试在首次运行通过。它们证明已有子集可用,不证明嵌入阅读、完整多模型计量或浏览器可读性已经交付。 +已交付并通过自动化子集: -本次不改框架实现,也不调用付费模型;后续实现按 M/F 编号记录证据、实测环境、未支持格式/适配器和兼容限制。 +| 切片 | 证据 | +|---|---| +| M00 | fixtures + JSON Schema 消费;`capabilities`;`force_unloaded` / `force_*_off` 诚实降级(F16) | +| M01 | `messages[]` + strict selection;`RunContextV1`;workbook skeleton 禁 `llm.llm`;子任务转发 model/base_url | +| M02 | usage null≠0;event_sink;`MARQDO_CANCEL`/`_TOKEN`;web `sse-live` 断连写 cancel | +| M03 | `view.document` 不经 `build_file_view`;heading outline;path escape 拒绝;F08 无副作用标记 | +| M04 | `view.reference_resolve` stale/version_unavailable;`artifact.inspect` version_mismatch;embed `sse_accepts_packet_strict` | +| M05 | extract `:B` locator;analysis_cache 按 project_root 隔离(F14) | +| M06 | CI `executable-knowledge.yml` 预构建 agent+llm 并跑 `workbench_contracts` | + +已知限制(不得宣称已完成): + +- `anthropic-native` / `gemini-native` 仍 `available:false`。 +- PDF/DOCX/OCR extract 仍显式 unsupported。 +- 历史正文快照存储仍由应用负责;框架对缺快照返回 `version_unavailable`。 +- `v3_corpus_1000_compile_smoke` 耗时门限随机器变化;release/debug 分剖面记录,不删 corpus。 +- 未调用付费供应商;流式/取消端到端依赖假 HTTP / 本地 cancel token。 + +### 12.2 历史基线记录(设计起草时) + +起草时 `executable_knowledge` 曾出现 `f1` 缺插件与 `v3_corpus_1000` debug 超时;其后 `ensure_agent_plugin_built` + CI 预构建已修复 f1 路径。1000 单元性能仍按剖面单独记录。 diff --git a/ext/ai/agent.mq.md b/ext/ai/agent.mq.md index 5680726..b382fac 100644 --- a/ext/ai/agent.mq.md +++ b/ext/ai/agent.mq.md @@ -64,7 +64,7 @@ One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{te 1. `stream` **evs = > model.stream prompt=`prompt` echo=`echo`** **text = > llm.collect events=`evs`** - **tokens = 0** + **tokens = None** **err = None** **calls = 1** 1. not `text` @@ -81,7 +81,7 @@ One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{te 1. `tokens` **_ = 1** 2. * - **tokens = 0** + **tokens = None** 1. `err` **calls = 0** 2. `fin` == "error" @@ -183,7 +183,7 @@ Allowlist check, then invoke the runbook `##` via `lib/subtask` (`spawn fn=` → Build a runnable workbook. `skeleton=single` (default): one worker `step`. `skeleton=dual`: research then write agents. On success the parent chooses DONE; runtime `agent_workbook_solidify` freezes the answer. **q = > json.quote text=`goal`** -**esc = > json.parse text={"h":"---\ntitle: agent workbook\nimport:\n llm: ext/ai/llm.mq.md\n agent: ext/ai/agent.mq.md\n json: lib/json.mq.md\n sys: lib/sys.mq.md\n wb: lib/writeback.mq.md\n subtask: lib/subtask.mq.md\n---\n\n# Goal\n\nParent goal is embedded as JSON in # main.\n\n## Solidify\n\nWhen the child returns a good value, parent DECISION: DONE — runtime agent_workbook_solidify freezes the answer as **return** (do not invent long FIND/REPLACE).\n\n# main\n\n> llm.load_env\n\n","s1":"\u002a\u002a\u0060model\u0060 = > llm.llm\u002a\u002a\n","s2":"\u002a\u002a\u0060tools\u0060 = > table.empty_list\u002a\u002a\n","s3":"\u002a\u002a\u0060worker\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You are a workbook worker. tools=[] means never CALL. Finish the task with a direct final answer; do not invent tools.\u002a\u002a\n","s4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","s5":"}\u002a\u002a\n","s6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","s7":"\u002a\u002a\u0060out\u0060 = > \u0060worker\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","s7b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","s8":"\u002a\u002a\u0060text\u0060\u002a\u002a\n","d3":"\u002a\u002a\u0060research\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You gather facts for the goal. Be concise. Do not invent tools.\u002a\u002a\n","d3b":"\u002a\u002a\u0060writer\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You write the final answer from research notes. Do not invent tools.\u002a\u002a\n","d4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","d5":"}\u002a\u002a\n","d6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","d7":"\u002a\u002a\u0060notes\u0060 = > \u0060research\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","d8":"\u002a\u002a\u0060ns\u0060 = > json.stringify value=\u0060notes\u0060\u002a\u002a\n","d9":"\u002a\u002a\u0060wt\u0060 = Write the final answer. Research notes:\u002a\u002a\n","d10":"\u002a\u002a\u0060wt\u0060 = \u0060wt\u0060 + \u0060ns\u0060\u002a\u002a\n","d11":"\u002a\u002a\u0060out\u0060 = > \u0060writer\u0060.step task=\u0060wt\u0060 stream=False\u002a\u002a\n","d11b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","d12":"\u002a\u002a\u0060text\u0060\u002a\u002a\n"}** +**esc = > json.parse text={"h":"---\ntitle: agent workbook\nimport:\n llm: ext/ai/llm.mq.md\n agent: ext/ai/agent.mq.md\n json: lib/json.mq.md\n sys: lib/sys.mq.md\n wb: lib/writeback.mq.md\n subtask: lib/subtask.mq.md\n---\n\n# Goal\n\nParent goal is embedded as JSON in # main.\n\n## Solidify\n\nWhen the child returns a good value, parent DECISION: DONE — runtime agent_workbook_solidify freezes the answer as **return** (do not invent long FIND/REPLACE).\n\n# main\n\n> llm.load_env\n\n","s1":"\u002a\u002a\u0060model\u0060 = > llm.create model=\u0060sys.env_get name=\"OPENAI_MODEL\"\u0060 base_url=\u0060sys.env_get name=\"OPENAI_BASE_URL\"\u0060 backend=openai name=workbook\u002a\u002a\n","s2":"\u002a\u002a\u0060tools\u0060 = > table.empty_list\u002a\u002a\n","s3":"\u002a\u002a\u0060worker\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You are a workbook worker. tools=[] means never CALL. Finish the task with a direct final answer; do not invent tools.\u002a\u002a\n","s4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","s5":"}\u002a\u002a\n","s6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","s7":"\u002a\u002a\u0060out\u0060 = > \u0060worker\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","s7b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","s8":"\u002a\u002a\u0060text\u0060\u002a\u002a\n","d3":"\u002a\u002a\u0060research\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You gather facts for the goal. Be concise. Do not invent tools.\u002a\u002a\n","d3b":"\u002a\u002a\u0060writer\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You write the final answer from research notes. Do not invent tools.\u002a\u002a\n","d4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","d5":"}\u002a\u002a\n","d6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","d7":"\u002a\u002a\u0060notes\u0060 = > \u0060research\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","d8":"\u002a\u002a\u0060ns\u0060 = > json.stringify value=\u0060notes\u0060\u002a\u002a\n","d9":"\u002a\u002a\u0060wt\u0060 = Write the final answer. Research notes:\u002a\u002a\n","d10":"\u002a\u002a\u0060wt\u0060 = \u0060wt\u0060 + \u0060ns\u0060\u002a\u002a\n","d11":"\u002a\u002a\u0060out\u0060 = > \u0060writer\u0060.step task=\u0060wt\u0060 stream=False\u002a\u002a\n","d11b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","d12":"\u002a\u002a\u0060text\u0060\u002a\u002a\n"}** **h = > json.get value=`esc` key="h"** **s1 = > json.get value=`esc` key="s1"** **s2 = > json.get value=`esc` key="s2"** diff --git a/ext/ai/llm.mq.md b/ext/ai/llm.mq.md index a5e5272..5932827 100644 --- a/ext/ai/llm.mq.md +++ b/ext/ai/llm.mq.md @@ -268,20 +268,25 @@ Prefer declaring `model` / `api_key` / `base_url` in frontmatter Binding, then u ## ask + `prompt`=None + `path`=None + + `messages`=None + + `selection`=None + + `strict_selection`=False -Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`). +Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`). Pass `messages=` for multi-turn history (M01). `selection=` + `strict_selection=True` freeze request identity without env fallback. -1. `path` +1. `messages` + **p = None** +2. `path` **p = > prompt_load path=`path`** -2. `prompt` +3. `prompt` **p = `prompt`** -3. * +4. * **raw = > sys.module_source** 1. not `raw` **p = ""** 2. * **p = > prompt_body src=`raw`** -**pack = > self.complete_result prompt=`p` stream=False echo=False** +**pack = > self.complete_result prompt=`p` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=False echo=False** **text = > json.get value=`pack` key="text"** **usage = > json.get value=`pack` key="usage"** **finish = > json.get value=`pack` key="finish"** @@ -312,11 +317,16 @@ inside the table is unsafe. **nm = > json.get value=`self` key="name"** 1. `usage` **tt = > json.get value=`usage` key="total_tokens"** + **ustate = > json.get value=`usage` key="usage_state"** 2. * - **tt = 0** -1. not `tt` - **tt = 0** -2. * + **tt = None** + **ustate = "unknown"** +1. `ustate` == "unknown" + Keep null — never coerce missing usage to zero tokens (M02 / F04). + **tt = None** +2. not `tt` + **tt = None** +3. * **_ = 1** 1. `err` **calls = 0** @@ -325,9 +335,9 @@ inside the table is unsafe. `result` = -| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error | -|------|-------|--------|---------|-------|------|-----------|--------|------------|-------| -| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | `calls` | `tt` | `tc` | `err` | +| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error | selection | +|------|-------|--------|---------|-------|------|-----------|--------|------------|-------|-----------| +| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | `calls` | `tt` | `tc` | `err` | `selection` | *result* @@ -352,13 +362,16 @@ Semantic stream: return the event list (use `llm.collect` to reduce to text). Om *> self.complete prompt=`p` stream=True echo=`echo`* ## complete_result - + `prompt` + + `prompt`=None + + `messages`=None + + `selection`=None + + `strict_selection`=False + `stream`=False + `echo`=False -Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`. +Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`. Prefer `messages=` for multi-turn; optional `selection=` (ModelSelectionV1) with `strict_selection=True` for request-scoped identity (M01). -*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` stream=`stream` echo=`echo` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))* +*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=`stream` echo=`echo` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))* ## complete + `prompt` diff --git a/ext/ai/llm/openai.mq.md b/ext/ai/llm/openai.mq.md index 7b3d811..c524626 100644 --- a/ext/ai/llm/openai.mq.md +++ b/ext/ai/llm/openai.mq.md @@ -73,7 +73,11 @@ Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list). + `base_url` + `api_key` + `model` - + `prompt` + + `prompt`=None + + `messages`=None + + `selection`=None + + `strict_selection`=False + + `auth_mode`=None + `stream`=False + `echo`=False + `suffix`=/chat/completions @@ -81,20 +85,25 @@ Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list). POST `/chat/completions` (or stream SSE). Prefers `plugins/llm` ABI (`llm_chat_completions`) when the native plugin is resolvable; otherwise `lib/net` HTTP/SSE. Non-stream → `{text,usage,finish,tool_calls}`; stream → `{events,finish}`. Soft transport / HTTP failures return `finish=error` packs (never abort the VM). +Pass either `messages=` (role/content list) or legacy `prompt=` (single user message). Optional `selection=` is a ModelSelectionV1 snapshot; `strict_selection=True` refuses env fallbacks inside the plugin. + **p = > plugin.native_path name="llm"** 1. `p` > plugin.load path=`p` - *> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`* + *> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` auth_mode=`auth_mode` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`* 2. * **url = base_url + suffix** **auth = bearer + api_key** **headers = > table.put in=None at="Authorization" value=`auth`** - `messages` = + 1. not `messages` + `messages` = - | @ | role | content | - |---|------|---------| - | 1 | user | `prompt` | + | @ | role | content | + |---|------|---------| + | 1 | user | `prompt` | + 2. * + **_ = 1** `req` = diff --git a/plugins/llm/src/lib.rs b/plugins/llm/src/lib.rs index 7d82c75..9fd4449 100644 --- a/plugins/llm/src/lib.rs +++ b/plugins/llm/src/lib.rs @@ -44,21 +44,40 @@ fn read_limited(mut r: impl Read) -> Result { String::from_utf8(buf).map_err(|e| e.to_string()) } +/// Prefer null over zero when the provider omitted a field (ModelCallUsageV1 / F04). +fn token_or_null(v: Option) -> Value { + match v { + Some(n) if n >= 0 => json!(n), + _ => Value::Null, + } +} + fn usage_from(data: &Value) -> Value { let u = data.get("usage").cloned().unwrap_or(Value::Null); - let pt = u.get("prompt_tokens").and_then(|v| v.as_i64()).unwrap_or(0); - let ct = u - .get("completion_tokens") - .and_then(|v| v.as_i64()) - .unwrap_or(0); + if u.is_null() || !u.is_object() { + return json!({ + "prompt_tokens": Value::Null, + "completion_tokens": Value::Null, + "total_tokens": Value::Null, + "usage_state": "unknown", + "measurement_source": "none", + }); + } + let pt = u.get("prompt_tokens").and_then(|v| v.as_i64()); + let ct = u.get("completion_tokens").and_then(|v| v.as_i64()); let tt = u .get("total_tokens") .and_then(|v| v.as_i64()) - .unwrap_or(pt + ct); + .or_else(|| match (pt, ct) { + (Some(a), Some(b)) => Some(a + b), + _ => None, + }); json!({ - "prompt_tokens": pt, - "completion_tokens": ct, - "total_tokens": tt, + "prompt_tokens": token_or_null(pt), + "completion_tokens": token_or_null(ct), + "total_tokens": token_or_null(tt), + "usage_state": "final", + "measurement_source": "provider", }) } @@ -72,10 +91,34 @@ fn tool_calls_from(data: &Value) -> Value { .unwrap_or_else(|| json!([])) } +fn cancel_requested() -> bool { + if std::env::var("MARQDO_CANCEL") + .map(|v| v == "1" || v.eq_ignore_ascii_case("true")) + .unwrap_or(false) + { + return true; + } + if let Ok(tok) = std::env::var("MARQDO_CANCEL_TOKEN") { + let path = std::path::Path::new(&tok); + if path.is_file() { + if let Ok(body) = std::fs::read_to_string(path) { + let t = body.trim(); + return t.is_empty() || t == "1" || t.eq_ignore_ascii_case("cancelled"); + } + return true; + } + } + false +} + fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result, String> { let mut events = Vec::new(); let mut data_lines: Vec = Vec::new(); for line in reader.lines() { + if cancel_requested() { + events.push(json!({"type": "cancelled", "message": "MARQDO_CANCEL"})); + return Ok(events); + } let line = line.map_err(|e| e.to_string())?; if echo { eprint!("{line}\n"); @@ -107,11 +150,106 @@ fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result, Stri Ok(events) } -fn chat_completions(args: &Value) -> Result { - let base_url = arg_text(args, "base_url")?; - let api_key = arg_text(args, "api_key").unwrap_or(""); - let model = arg_text(args, "model")?; +fn build_messages(args: &Value) -> Result { + if let Some(arr) = args.get("messages").and_then(|v| v.as_array()) { + if arr.is_empty() { + return Err("messages array must not be empty".into()); + } + // Validate minimal role/content shape; reject unsupported part types loudly. + for (i, m) in arr.iter().enumerate() { + let role = m.get("role").and_then(|r| r.as_str()).unwrap_or(""); + if role.is_empty() { + return Err(format!("llm.unsupported_input: messages[{i}].role missing")); + } + if let Some(parts) = m.get("content").and_then(|c| c.as_array()) { + for (j, p) in parts.iter().enumerate() { + let ty = p.get("type").and_then(|t| t.as_str()).unwrap_or("text"); + if ty != "text" { + return Err(format!( + "llm.unsupported_input: messages[{i}].content[{j}] type={ty}" + )); + } + } + } + } + return Ok(Value::Array(arr.clone())); + } let prompt = arg_text(args, "prompt")?; + Ok(json!([{"role": "user", "content": prompt}])) +} + +fn chat_completions(args: &Value) -> Result { + if cancel_requested() { + return Ok(json!({ + "text": "", + "usage": { + "prompt_tokens": Value::Null, + "completion_tokens": Value::Null, + "total_tokens": Value::Null, + "usage_state": "unknown", + "measurement_source": "none", + }, + "finish": "cancelled", + "error": "cancelled", + "tool_calls": [], + "transport": "plugin", + })); + } + // Strict selection mode: refuse env fallback when selection.strict is set. + let strict = opt_bool(args, "strict_selection", false) + || args + .get("selection") + .and_then(|s| s.get("strict")) + .and_then(|v| v.as_bool()) + .unwrap_or(false); + let selection = args.get("selection").cloned(); + if strict { + if selection.is_none() { + return Err("llm.selection_missing:selection".into()); + } + let sel = selection.as_ref().unwrap(); + for key in ["selection_id", "provider_id", "model_id", "upstream_model_id"] { + if sel + .get(key) + .and_then(|v| v.as_str()) + .filter(|s| !s.is_empty()) + .is_none() + { + return Err(format!("llm.selection_missing:{key}")); + } + } + } + + let base_url = match arg_text(args, "base_url") { + Ok(u) => u, + Err(_) if strict => return Err("llm.selection_missing:base_url".into()), + Err(e) => return Err(e), + }; + let auth_mode = opt_text(args, "auth_mode") + .or_else(|| { + selection + .as_ref() + .and_then(|s| s.get("auth_mode")) + .and_then(|v| v.as_str()) + }) + .unwrap_or("api_key"); + let api_key = if auth_mode == "none" { + "" + } else { + arg_text(args, "api_key").unwrap_or("") + }; + if strict && auth_mode != "none" && api_key.is_empty() { + return Err("llm.selection_missing:api_key".into()); + } + let model = selection + .as_ref() + .and_then(|s| s.get("upstream_model_id")) + .and_then(|v| v.as_str()) + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()) + .or_else(|| arg_text(args, "model").ok().map(|s| s.to_string())) + .ok_or_else(|| "missing text `model`".to_string())?; + let messages = build_messages(args)?; let stream = opt_bool(args, "stream", false); let echo = opt_bool(args, "echo", false); let suffix = opt_text(args, "suffix").unwrap_or("/chat/completions"); @@ -123,11 +261,20 @@ fn chat_completions(args: &Value) -> Result { } let mut req_body = Map::new(); - req_body.insert("model".into(), Value::String(model.to_string())); - req_body.insert( - "messages".into(), - json!([{"role": "user", "content": prompt}]), - ); + req_body.insert("model".into(), Value::String(model.clone())); + req_body.insert("messages".into(), messages); + if let Some(params) = selection + .as_ref() + .and_then(|s| s.get("parameters")) + .and_then(|p| p.as_object()) + { + if let Some(max) = params.get("max_output_tokens").and_then(|v| v.as_u64()) { + req_body.insert("max_tokens".into(), json!(max)); + } + if let Some(temp) = params.get("temperature") { + req_body.insert("temperature".into(), temp.clone()); + } + } if stream { req_body.insert("stream".into(), Value::Bool(true)); } @@ -193,11 +340,19 @@ fn chat_completions(args: &Value) -> Result { if status != 200 { return Ok(json!({ "text": "", - "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}, + "usage": { + "prompt_tokens": Value::Null, + "completion_tokens": Value::Null, + "total_tokens": Value::Null, + "usage_state": "unknown", + "measurement_source": "none", + }, "finish": "error", "error": format!("HTTP {status}: {text}"), "tool_calls": [], "transport": "plugin", + "requested_model": model, + "resolved_model": model, })); } let data: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?; @@ -221,32 +376,62 @@ fn chat_completions(args: &Value) -> Result { .and_then(|v| v.as_str()) .unwrap_or("stop") .to_string(); + let resolved = data + .get("model") + .and_then(|v| v.as_str()) + .unwrap_or(model.as_str()) + .to_string(); Ok(json!({ "text": content, "usage": usage_from(&data), "finish": fr, "tool_calls": tool_calls_from(&data), "transport": "plugin", + "requested_model": model, + "resolved_model": resolved, + "selection": selection.as_ref().map(|s| { + let mut snap = s.clone(); + if let Some(obj) = snap.as_object_mut() { + obj.remove("api_key"); + } + snap + }), })) } Err(ureq::Error::Status(code, r)) => { let text = read_limited(r.into_reader()).unwrap_or_default(); Ok(json!({ "text": "", - "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}, + "usage": { + "prompt_tokens": Value::Null, + "completion_tokens": Value::Null, + "total_tokens": Value::Null, + "usage_state": "unknown", + "measurement_source": "none", + }, "finish": "error", "error": format!("HTTP {code}: {text}"), "tool_calls": [], "transport": "plugin", + "requested_model": model, + "resolved_model": model, })) } Err(e) => Ok(json!({ "text": "", - "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}, + "usage": { + "prompt_tokens": Value::Null, + "completion_tokens": Value::Null, + "total_tokens": Value::Null, + "usage_state": "unknown", + "measurement_source": "none", + }, "finish": "error", "error": format!("http POST {url}: {e}"), "tool_calls": [], "transport": "plugin", + "requested_model": model, + "resolved_model": model, })), } } @@ -274,5 +459,5 @@ unsafe extern "C" fn llm_ping( marqdo_plugin!(marqdo_plugin_sdk::ABI_VERSION, [ ("llm_ping", "", llm_ping), - ("llm_chat_completions", "base_url,api_key,model,prompt,stream,echo,suffix,bearer", llm_chat_completions), + ("llm_chat_completions", "base_url,api_key,model,prompt,messages,selection,strict_selection,auth_mode,stream,echo,suffix,bearer", llm_chat_completions), ]); diff --git a/plugins/web/internal/httpx/routes_invoke.go b/plugins/web/internal/httpx/routes_invoke.go index 0e62bc4..fb2ca9f 100644 --- a/plugins/web/internal/httpx/routes_invoke.go +++ b/plugins/web/internal/httpx/routes_invoke.go @@ -1,11 +1,15 @@ package httpx import ( + "bufio" "encoding/json" "fmt" "io" "net/http" + "os" "strings" + "sync" + "time" "github.com/marqdo/marqdo/plugins/web/internal/plugin" ) @@ -115,6 +119,15 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo } args := invokeArgsFromPayload(payload) + + // M02: live SSE — stream host_stream_publish events while CallLibPath runs. + // return=sse-live|sse_live uses MARQDO_EVENT_FORWARD NDJSON bridge (request-local). + liveSSE := route.ReturnMode == "sse-live" || route.ReturnMode == "sse_live" + if liveSSE { + st.handleInvokeLiveSSE(w, r, route, args) + return + } + value, err := plugin.CallLibPath(route.FnPath, args) if err != nil { code := http.StatusInternalServerError @@ -176,3 +189,160 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo writeJSON(w, code, body) } } + +// handleInvokeLiveSSE streams RuntimeEvent-shaped lines while the lib call runs. +// Client disconnect sets MARQDO_CANCEL + cancel token file so the llm plugin can stop. +func (st *state) handleInvokeLiveSSE(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any) { + dir := os.TempDir() + fwd, err := os.CreateTemp(dir, "mq-evt-*.ndjson") + if err != nil { + jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("event forward: %v", err)) + return + } + fwdPath := fwd.Name() + _ = fwd.Close() + defer os.Remove(fwdPath) + + prev := os.Getenv("MARQDO_EVENT_FORWARD") + _ = os.Setenv("MARQDO_EVENT_FORWARD", fwdPath) + cancelFile, cancelErr := os.CreateTemp(dir, "mq-cancel-*.tok") + cancelPath := "" + if cancelErr == nil { + cancelPath = cancelFile.Name() + _ = cancelFile.Close() + _ = os.Setenv("MARQDO_CANCEL_TOKEN", cancelPath) + } + defer func() { + if prev == "" { + _ = os.Unsetenv("MARQDO_EVENT_FORWARD") + } else { + _ = os.Setenv("MARQDO_EVENT_FORWARD", prev) + } + _ = os.Unsetenv("MARQDO_CANCEL_TOKEN") + _ = os.Unsetenv("MARQDO_CANCEL") + if cancelPath != "" { + _ = os.Remove(cancelPath) + } + }() + + w.Header().Set("Content-Type", "text/event-stream; charset=utf-8") + w.Header().Set("Cache-Control", "no-cache") + w.Header().Set("Connection", "keep-alive") + w.Header().Set("X-Accel-Buffering", "no") + w.WriteHeader(http.StatusOK) + flusher, _ := w.(http.Flusher) + writeEvent := func(payload any) { + b, err := json.Marshal(payload) + if err != nil { + return + } + _, _ = io.WriteString(w, "data: "+string(b)+"\n\n") + if flusher != nil { + flusher.Flush() + } + } + writeEvent(map[string]any{ + "schema": "RuntimeEventV1", + "schema_version": 1, + "type": "run.started", + "payload": map[string]any{"fn": route.FnPath, "bridge": "sse-live"}, + }) + + type callResult struct { + value any + err error + } + done := make(chan callResult, 1) + go func() { + v, e := plugin.CallLibPath(route.FnPath, args) + done <- callResult{value: v, err: e} + }() + + var mu sync.Mutex + offset := int64(0) + seq := uint64(1) + poll := time.NewTicker(25 * time.Millisecond) + defer poll.Stop() + + ctx := r.Context() + for { + select { + case <-ctx.Done(): + // Propagate cancel to in-flight LLM/plugin via token file + env. + _ = os.Setenv("MARQDO_CANCEL", "1") + if cancelPath != "" { + _ = os.WriteFile(cancelPath, []byte("cancelled"), 0o600) + } + writeEvent(map[string]any{ + "schema": "RuntimeEventV1", "schema_version": 1, "type": "run.interrupted", + "payload": map[string]any{"reason": "client_disconnect", "cancel_propagated": cancelPath != ""}, + }) + return + case res := <-done: + // Drain remaining forward lines once. + mu.Lock() + offset = drainForward(fwdPath, offset, func(line string) { + seq++ + var obj any + if json.Unmarshal([]byte(line), &obj) == nil { + writeEvent(map[string]any{ + "schema": "RuntimeEventV1", "schema_version": 1, + "seq": seq, "type": "message.delta", "payload": obj, + }) + } + }) + mu.Unlock() + if res.err != nil { + writeEvent(map[string]any{ + "schema": "RuntimeEventV1", "schema_version": 1, "type": "run.failed", + "payload": map[string]any{"error": res.err.Error()}, + }) + } else { + writeEvent(map[string]any{ + "schema": "RuntimeEventV1", "schema_version": 1, "type": "run.completed", + "payload": map[string]any{"result": res.value}, + }) + } + _, _ = io.WriteString(w, "data: [DONE]\n\n") + if flusher != nil { + flusher.Flush() + } + return + case <-poll.C: + mu.Lock() + offset = drainForward(fwdPath, offset, func(line string) { + seq++ + var obj any + if json.Unmarshal([]byte(line), &obj) == nil { + writeEvent(map[string]any{ + "schema": "RuntimeEventV1", "schema_version": 1, + "seq": seq, "type": "message.delta", "payload": obj, + }) + } + }) + mu.Unlock() + } + } +} + +func drainForward(path string, offset int64, onLine func(string)) int64 { + f, err := os.Open(path) + if err != nil { + return offset + } + defer f.Close() + if _, err := f.Seek(offset, io.SeekStart); err != nil { + return offset + } + sc := bufio.NewScanner(f) + sc.Buffer(make([]byte, 0, 64*1024), 1024*1024) + for sc.Scan() { + line := strings.TrimSpace(sc.Text()) + offset += int64(len(sc.Bytes()) + 1) + if line == "" { + continue + } + onLine(line) + } + return offset +} diff --git a/src/document_extract.rs b/src/document_extract.rs index b534cfd..d76c627 100644 --- a/src/document_extract.rs +++ b/src/document_extract.rs @@ -193,8 +193,12 @@ fn extract_markdown_or_text( ) -> ExtractedDocument { let mut segments = Vec::new(); let mut n = 0u32; + let mut byte_off = 0usize; for (i, line) in text.lines().enumerate() { let line_no = i + 1; + let line_bytes = line.len(); + // M05: stable fragment locator with 1-based line + UTF-8 byte offset. + let loc = |kind: &str| format!("{kind}:L{line_no}:B{byte_off}"); if let Some(rest) = line.trim().strip_prefix('#') { n += 1; let heading = rest.trim_start_matches('#').trim(); @@ -202,7 +206,7 @@ fn extract_markdown_or_text( id: format!("seg-{n}"), kind: "heading".into(), text: heading.to_string(), - locator: format!("L{line_no}"), + locator: loc("heading"), confidence: 1.0, }); } else if line.trim().starts_with('|') { @@ -211,7 +215,7 @@ fn extract_markdown_or_text( id: format!("seg-{n}"), kind: "table_row".into(), text: line.to_string(), - locator: format!("L{line_no}"), + locator: loc("table_row"), confidence: 0.9, }); } else if !line.trim().is_empty() { @@ -220,10 +224,12 @@ fn extract_markdown_or_text( id: format!("seg-{n}"), kind: "paragraph".into(), text: line.to_string(), - locator: format!("L{line_no}"), + locator: loc("paragraph"), confidence: 1.0, }); } + // +1 for '\n' when present in original (lines() strips it). + byte_off += line_bytes + 1; } ExtractedDocument { source_uri: uri.to_string(), diff --git a/src/host/subtask.rs b/src/host/subtask.rs index 6cf1844..74049de 100644 --- a/src/host/subtask.rs +++ b/src/host/subtask.rs @@ -331,6 +331,22 @@ fn spawn_file( )); let _ = std::fs::write(&events_path, ""); cmd.env("MARQDO_EVENT_FORWARD", &events_path); + // Propagate run/selection identity when parent has them (api_key stays via process inheritance). + for key in [ + "MARQDO_RUN_ID", + "MARQDO_PARENT_RUN_ID", + "MARQDO_TURN_ID", + "MARQDO_WORKSPACE_SCOPE", + "MARQDO_CANCEL_TOKEN", + "MARQDO_LLM_MODEL", + "MARQDO_LLM_BASE_URL", + "OPENAI_MODEL", + "OPENAI_BASE_URL", + ] { + if let Ok(v) = std::env::var(key) { + cmd.env(key, v); + } + } let mut child = cmd .spawn() diff --git a/src/lib.rs b/src/lib.rs index 38d7f8c..72af9bf 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -37,6 +37,7 @@ pub mod lex; pub mod linalg_ops; pub mod load; pub mod mlsp; +pub mod workbench; pub mod parse; pub mod project; pub mod repair; diff --git a/src/mlsp.rs b/src/mlsp.rs index 56cdb3d..aec3c7e 100644 --- a/src/mlsp.rs +++ b/src/mlsp.rs @@ -68,17 +68,34 @@ fn dispatch(req: &Value) -> Result { Some("document.extract") => document_extract(¶ms), Some("artifact.patch") => artifact_patch(¶ms), Some("artifact.extract_pure") => artifact_extract_pure(¶ms), + // Workbench M00–M04: capability probe + read-only view + locator. + Some("capabilities") => Ok(crate::workbench::probe_capabilities(¶ms)), + #[cfg(feature = "view")] + Some("view.document") => view_document(¶ms), + #[cfg(not(feature = "view"))] + Some("view.document") => Err(err( + "mlsp.unsupported", + "view.document requires the view feature", + )), + Some("view.reference_resolve") => crate::workbench::resolve_reference(¶ms), // `` M9: occlusion label search (two-tier vocabulary). Some("labels.search") => labels_search(¶ms), // `` M6: one-shot session summary (T1 staleness seen/fixed). Some("session.summary") => session_summary(), other => Err(err( "mlsp.unknown_method", - format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema / artifact.find / artifact.inspect / link.resolve / artifact.verify / artifact.impact / artifact.bind / artifact.execute / artifact.reuse_execute / artifact.patch / artifact.extract_pure / document.extract"), + format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema / artifact.find / artifact.inspect / link.resolve / artifact.verify / artifact.impact / artifact.bind / artifact.execute / artifact.reuse_execute / artifact.patch / artifact.extract_pure / document.extract / capabilities / view.document / view.reference_resolve"), )), } } +#[cfg(feature = "view")] +fn view_document(params: &Value) -> Result { + let req = crate::workbench::DocumentViewRequest::from_params(params) + .map_err(|e| err("mlsp.bad_request", e))?; + crate::workbench::build_document_view(&req) +} + fn artifact_extract_pure(params: &Value) -> Result { let name = params .get("name") @@ -172,7 +189,100 @@ fn artifact_inspect(params: &Value) -> Result { let candidates = artifact_query(&compilation.semantics, query); if candidates.is_empty() { return Err(err("artifact.unresolved", format!("没有制品 `{query}`"))); } if candidates.len() != 1 { return Err(json!({"code":"artifact.ambiguous","message":format!("制品 `{query}` 不唯一"),"candidates":candidates.iter().map(|a|&a.artifact_id.0).collect::>() })); } - Ok(crate::sema::artifact_json(candidates[0])) + let artifact = candidates[0]; + // Version contract: expected source/generation mismatch must not silently return current. + check_artifact_version_params(params, &compilation, artifact)?; + Ok(crate::sema::artifact_json(artifact)) +} + +/// Reject inspect when caller pinned an expected source revision or generation that +/// does not match the current project snapshot (`artifact.version_mismatch`). +fn check_artifact_version_params( + params: &Value, + compilation: &crate::sema::Compilation, + artifact: &crate::sema::SemanticArtifact, +) -> Result<(), Value> { + let expected_rev = params + .get("expected_source_revision") + .and_then(Value::as_str) + .filter(|s| !s.is_empty()); + if let Some(expected) = expected_rev { + let project_root = params + .get("project") + .or_else(|| params.get("path")) + .and_then(Value::as_str) + .map(PathBuf::from) + .unwrap_or_else(|| PathBuf::from(".")); + let root = if project_root.is_file() { + project_root + .parent() + .unwrap_or_else(|| Path::new(".")) + .to_path_buf() + } else { + project_root + }; + let abs = root.join(&artifact.source.file); + match std::fs::read(&abs) { + Ok(raw) => { + use sha2::{Digest, Sha256}; + let actual = format!("sha256:{:x}", Sha256::digest(&raw)); + if actual != expected { + return Err(json!({ + "code": "artifact.version_mismatch", + "message": format!( + "expected source_revision {expected}, actual {actual}; refusing silent downgrade" + ), + "expected_source_revision": expected, + "actual_source_revision": actual, + "artifact_id": artifact.artifact_id.0, + })); + } + } + Err(e) => { + return Err(json!({ + "code": "artifact.version_mismatch", + "message": format!( + "cannot verify source_revision for {}: {e}; refusing silent downgrade", + artifact.source.file + ), + "expected_source_revision": expected, + "artifact_id": artifact.artifact_id.0, + })); + } + } + } + + let expected_gen = params + .get("expected_generation") + .or_else(|| params.get("generation_id")) + .and_then(Value::as_str) + .filter(|s| !s.is_empty()); + if let Some(expected) = expected_gen { + let actual = semantic_generation_id(&compilation.semantics); + if actual != expected { + return Err(json!({ + "code": "artifact.version_mismatch", + "message": format!( + "expected generation {expected}, current projection is {actual}; refusing silent downgrade" + ), + "expected_generation": expected, + "actual_generation": actual, + "artifact_id": artifact.artifact_id.0, + })); + } + } + Ok(()) +} + +/// Semantic generation token matching KnowledgePresentation / DocumentView. +fn semantic_generation_id(project: &crate::sema::SemanticProject) -> String { + use sha2::{Digest, Sha256}; + let mut gh = Sha256::new(); + gh.update(project.project_id.as_bytes()); + for a in &project.artifacts { + gh.update(a.hashes.artifact.0.as_bytes()); + } + format!("{:x}", gh.finalize())[..16].to_string() } fn link_resolve(params: &Value) -> Result { diff --git a/src/view/knowledge_present.rs b/src/view/knowledge_present.rs index 43f5910..c2a838a 100644 --- a/src/view/knowledge_present.rs +++ b/src/view/knowledge_present.rs @@ -127,7 +127,11 @@ impl KnowledgePresentation { } } -/// V2: whether an SSE result packet may update the live document structure. +/// Live View SSE: whether a result packet may update the document structure. +/// +/// Legacy packets without a token are accepted so older live-view clients keep +/// working. **Embedded DocumentView clients must use [`sse_accepts_packet_strict`] +/// instead** — missing/empty tokens and mismatches are rejected there. pub fn sse_accepts_packet(page_token: &str, packet_token: Option<&str>) -> bool { match packet_token { None | Some("") => true, // legacy packets without token still apply @@ -136,6 +140,18 @@ pub fn sse_accepts_packet(page_token: &str, packet_token: Option<&str>) -> bool } } +/// Strict SSE / embed gate: reject missing, empty, or mismatched page tokens. +/// +/// Use this for DocumentView / qdagent embed hosts. Do not inherit the live-view +/// leniency of [`sse_accepts_packet`]. +pub fn sse_accepts_packet_strict(page_token: &str, packet_token: Option<&str>) -> bool { + match packet_token { + None | Some("") => false, + Some(_tok) if page_token.is_empty() => false, + Some(tok) => tok == page_token, + } +} + /// Stable page token from raw source bytes (used by SSE run path). pub fn page_token_from_source(rel: &str, source: &str) -> String { let mut h = Sha256::new(); diff --git a/src/view/mod.rs b/src/view/mod.rs index 64a9760..3ac4d68 100644 --- a/src/view/mod.rs +++ b/src/view/mod.rs @@ -12,12 +12,17 @@ mod output; mod render; pub use knowledge_present::{ - page_token_from_source, sse_accepts_packet, title_from_source, KnowledgePresentation, + page_token_from_source, sse_accepts_packet, sse_accepts_packet_strict, title_from_source, + KnowledgePresentation, }; /// Surface rendering of an expression (labels/diagnostics reuse). `` M6 pub use render::expr_display; +/// Read-model HTML helpers for embedded DocumentView (M03) — never execute. +pub use document::render_module_document; +pub use render::render_knowledge_html; + pub use output::{write_capture, write_static, CaptureOptions, OutputOptions}; use std::fs; @@ -37,7 +42,7 @@ use crate::view::html::{escape, page_file, page_index, LinkMode}; use crate::view::annot::{build_view_annots, view_execution_report_json}; use crate::view::render::{ collect_input_prompts, render_bindings_html, render_module_structure, - render_knowledge_html, render_module_structure_with_annots, render_right_rail, uses_stream_panel, FileViewModel, + render_module_structure_with_annots, render_right_rail, uses_stream_panel, FileViewModel, }; use crate::{run_file_capture, RunOptions}; diff --git a/src/workbench/analysis_cache.rs b/src/workbench/analysis_cache.rs new file mode 100644 index 0000000..59c8139 --- /dev/null +++ b/src/workbench/analysis_cache.rs @@ -0,0 +1,87 @@ +//! Project snapshot analysis cache (M05 / F14): keyed by project_root + source digests. + +use sha2::{Digest, Sha256}; +use std::collections::HashMap; +use std::path::{Path, PathBuf}; +use std::sync::{Mutex, OnceLock}; +use std::time::Instant; + +#[derive(Clone)] +struct CacheEntry { + digest_key: String, + #[allow(dead_code)] + compiled_at: Instant, +} + +fn cache() -> &'static Mutex> { + static C: OnceLock>> = OnceLock::new(); + C.get_or_init(|| Mutex::new(HashMap::new())) +} + +/// Fingerprint of authorized project sources (shallow: entry + .mq.md under root, capped). +pub fn project_source_digest(project_root: &Path) -> String { + let mut h = Sha256::new(); + h.update(project_root.to_string_lossy().as_bytes()); + let mut paths: Vec = Vec::new(); + if let Ok(rd) = std::fs::read_dir(project_root) { + for ent in rd.flatten().take(200) { + let p = ent.path(); + if p.extension().and_then(|e| e.to_str()) == Some("md") + || p + .file_name() + .and_then(|n| n.to_str()) + .map(|n| n.ends_with(".mq.md")) + .unwrap_or(false) + { + paths.push(p); + } + } + } + paths.sort(); + for p in paths { + h.update(p.to_string_lossy().as_bytes()); + if let Ok(bytes) = std::fs::read(&p) { + h.update(&bytes); + } + } + format!("{:x}", h.finalize()) +} + +/// Record/reuse analysis for concurrent same-snapshot reads. Returns true if cache hit. +pub fn note_analysis(project_root: &Path) -> bool { + let key = project_root + .canonicalize() + .unwrap_or_else(|_| project_root.to_path_buf()) + .to_string_lossy() + .to_string(); + let digest = project_source_digest(project_root); + let mut g = cache().lock().unwrap(); + if let Some(e) = g.get(&key) { + if e.digest_key == digest { + return true; + } + } + g.insert( + key, + CacheEntry { + digest_key: digest, + compiled_at: Instant::now(), + }, + ); + false +} + +/// Two project roots must not share cache entries even with identical relative paths. +pub fn cache_keys_isolated(a: &Path, b: &Path) -> bool { + let ka = a + .canonicalize() + .unwrap_or_else(|_| a.to_path_buf()) + .to_string_lossy() + .to_string(); + let kb = b + .canonicalize() + .unwrap_or_else(|_| b.to_path_buf()) + .to_string_lossy() + .to_string(); + ka != kb +} diff --git a/src/workbench/capabilities.rs b/src/workbench/capabilities.rs new file mode 100644 index 0000000..76326fe --- /dev/null +++ b/src/workbench/capabilities.rs @@ -0,0 +1,213 @@ +//! MLSP `capabilities` — probe loaded plugins and declared features (M00 / F16). +//! Feature integers must reflect *implemented and testable* behavior only. + +use serde_json::{json, Value}; +use std::path::PathBuf; +use std::sync::atomic::{AtomicBool, Ordering}; + +/// Host/tests may force stream/cancel off for honesty / F16 mismatch cases. +static FORCE_STREAM_OFF: AtomicBool = AtomicBool::new(false); +static FORCE_CANCEL_OFF: AtomicBool = AtomicBool::new(false); + +pub fn set_event_stream_ready(ready: bool) { + FORCE_STREAM_OFF.store(!ready, Ordering::Relaxed); +} + +pub fn set_cancel_ready(ready: bool) { + FORCE_CANCEL_OFF.store(!ready, Ordering::Relaxed); +} + +/// Build the capabilities document. Does not call providers or execute project code. +pub fn probe_capabilities(params: &Value) -> Value { + let runtime_version = env!("CARGO_PKG_VERSION").to_string(); + let mut adapters = vec![json!({ + "id": "openai-compatible", + "available": true, + "notes": "HTTP chat completions; messages[] preferred over single prompt" + })]; + adapters.push(json!({ + "id": "ollama-compatible", + "available": true, + "notes": "OpenAI-compatible path; not a distinct native protocol" + })); + adapters.push(json!({ + "id": "anthropic-native", + "available": false, + "notes": "not verified in this release" + })); + adapters.push(json!({ + "id": "gemini-native", + "available": false, + "notes": "not verified in this release" + })); + + let extract_formats = vec![ + json!({"format": "text", "available": true}), + json!({"format": "markdown", "available": true}), + json!({"format": "pdf", "available": false}), + json!({"format": "docx", "available": false}), + json!({"format": "ocr", "available": false}), + ]; + + let loaded_plugins = probe_loaded_plugins(params); + + let llm_loaded = loaded_plugins.iter().any(|p| { + p.get("name").and_then(|n| n.as_str()) == Some("llm") + && p.get("available").and_then(|a| a.as_bool()) == Some(true) + }); + + let stream_off = FORCE_STREAM_OFF.load(Ordering::Relaxed) + || params + .get("force_stream_off") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + let cancel_off = FORCE_CANCEL_OFF.load(Ordering::Relaxed) + || params + .get("force_cancel_off") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + + let mut features = serde_json::Map::new(); + features.insert( + "llm.explicit_selection".into(), + json!(if llm_loaded { 1 } else { 0 }), + ); + features.insert( + "llm.call_usage".into(), + json!(if llm_loaded { 1 } else { 0 }), + ); + features.insert( + "llm.messages".into(), + json!(if llm_loaded { 1 } else { 0 }), + ); + // M02 implemented: event_sink + web sse-live + MARQDO_CANCEL in llm plugin. + features.insert( + "runtime.event_stream".into(), + json!(if stream_off { 0 } else { 1 }), + ); + features.insert( + "runtime.cancel".into(), + json!(if cancel_off { 0 } else { 1 }), + ); + #[cfg(feature = "view")] + { + features.insert("view.document".into(), json!(1)); + features.insert("view.reference_resolve".into(), json!(1)); + } + #[cfg(not(feature = "view"))] + { + features.insert("view.document".into(), json!(0)); + features.insert("view.reference_resolve".into(), json!(0)); + } + + json!({ + "schema_version": 1, + "runtime_version": runtime_version, + "features": features, + "adapters": adapters, + "extract_formats": extract_formats, + "loaded_plugins": loaded_plugins, + "contracts": [ + "ModelSelectionV1", + "ModelCallUsageV1", + "RuntimeEventV1", + "DocumentViewV1", + "KnowledgeLocatorV1" + ] + }) +} + +fn probe_loaded_plugins(params: &Value) -> Vec { + let mut out = Vec::new(); + let candidates = [ + ("llm", "MARQDO_LLM_PLUGIN", plugin_name("llm")), + ("agent", "MARQDO_AGENT_PLUGIN", plugin_name("agent")), + ("web", "MARQDO_WEB_PLUGIN", plugin_name("web")), + ]; + let ext_hint = params + .get("ext_root") + .and_then(|v| v.as_str()) + .map(PathBuf::from) + .or_else(|| std::env::var_os("MARQDO_EXT").map(PathBuf::from)); + + let force_unloaded = params + .get("force_unloaded") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + + for (name, env_key, filename) in candidates { + if force_unloaded { + out.push(json!({ + "name": name, + "available": false, + "path": Value::Null, + "capabilities": [], + })); + continue; + } + let mut path: Option = std::env::var_os(env_key).map(PathBuf::from); + if path.as_ref().map(|p| !p.is_file()).unwrap_or(true) { + let debug = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("target") + .join("debug") + .join(&filename); + if debug.is_file() { + path = Some(debug); + } + } + if path.as_ref().map(|p| !p.is_file()).unwrap_or(true) { + if let Some(ext) = &ext_hint { + let native = ext.join("native").join(&filename); + if native.is_file() { + path = Some(native); + } + } + } + let available = path.as_ref().map(|p| p.is_file()).unwrap_or(false); + let mut caps = Vec::new(); + if name == "llm" && available { + caps.push("call_usage_v1"); + caps.push("messages_v1"); + caps.push("explicit_selection_v1"); + caps.push("cancel_v1"); + } + if name == "agent" && available { + caps.push("eng_bind_execute"); + caps.push("publish_gate"); + } + out.push(json!({ + "name": name, + "available": available, + "path": path.map(|p| p.to_string_lossy().to_string()), + "capabilities": caps, + })); + } + out +} + +fn plugin_name(stem: &str) -> String { + if cfg!(windows) { + format!("{stem}.dll") + } else if cfg!(target_os = "macos") { + format!("lib{stem}.dylib") + } else { + format!("lib{stem}.so") + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn force_off_and_unloaded_honest() { + let caps = probe_capabilities(&json!({ + "force_unloaded": true, + "force_stream_off": true, + "force_cancel_off": true, + })); + assert_eq!(caps["features"]["runtime.event_stream"], 0); + assert_eq!(caps["features"]["runtime.cancel"], 0); + assert_eq!(caps["features"]["llm.messages"], 0); + } +} diff --git a/src/workbench/document_view.rs b/src/workbench/document_view.rs new file mode 100644 index 0000000..365adc3 --- /dev/null +++ b/src/workbench/document_view.rs @@ -0,0 +1,427 @@ +//! DocumentViewV1 — read-only embedded view (M03). Never executes the document. + +use std::fs; +use std::path::{Path, PathBuf}; + +use serde_json::{json, Value}; +use sha2::{Digest, Sha256}; + +use crate::view::{render_knowledge_html, render_module_document, KnowledgePresentation}; + +#[derive(Debug, Clone)] +pub struct DocumentViewRequest { + pub project: PathBuf, + pub document: String, + pub revision: Option, + pub generation: Option, + pub node_id: Option, + pub include: Vec, +} + +impl DocumentViewRequest { + pub fn from_params(params: &Value) -> Result { + let project = params + .get("project") + .and_then(|v| v.as_str()) + .ok_or_else(|| "view.document requires params.project".to_string())?; + let document = params + .get("document") + .and_then(|v| v.as_str()) + .ok_or_else(|| "view.document requires params.document".to_string())?; + let mode = params + .get("mode") + .and_then(|v| v.as_str()) + .unwrap_or("read"); + if mode != "read" { + return Err(format!("view.document unsupported mode={mode}; only read")); + } + let include = match params.get("include") { + Some(Value::Array(a)) => a + .iter() + .filter_map(|v| v.as_str().map(|s| s.to_string())) + .collect(), + _ => vec![ + "fragment".into(), + "outline".into(), + "nodes".into(), + "refs".into(), + "diagnostics".into(), + ], + }; + Ok(Self { + project: PathBuf::from(project), + document: document.to_string(), + revision: params + .get("revision") + .and_then(|v| v.as_str()) + .map(|s| s.to_string()), + generation: params + .get("generation") + .and_then(|v| v.as_str()) + .map(|s| s.to_string()), + node_id: params + .get("node_id") + .and_then(|v| v.as_str()) + .map(|s| s.to_string()), + include, + }) + } +} + +/// Pure read path: compile_project (no plugin load / no eval) + fragment render. +/// Must never call `build_file_view` with execute=true. +pub fn build_document_view(req: &DocumentViewRequest) -> Result { + let project_root = canonicalize_project(&req.project)?; + let abs = resolve_under(&project_root, &req.document)?; + let raw = fs::read(&abs).map_err(|e| { + err( + "view.document_read_failed", + format!("cannot read {}: {e}", abs.display()), + ) + })?; + let source_revision = format!("sha256:{:x}", Sha256::digest(&raw)); + if let Some(expected) = &req.revision { + if expected != &source_revision && !expected.is_empty() { + return Err(err( + "view.revision_mismatch", + format!("requested {expected} but file is {source_revision}; refusing silent downgrade"), + )); + } + } + let source = String::from_utf8_lossy(&raw).into_owned(); + let title = title_from_source(&source, &req.document); + + // Semantic projection — compile_project does not load plugins / evaluate. + // Analysis cache is keyed by project_root + source digests (M05 isolation). + let _cache_hit = crate::workbench::analysis_cache::note_analysis(&project_root); + let options = crate::sema::CompileOptions::for_path(&project_root) + .map_err(|e| err("view.compile_failed", format!("{e:#}")))?; + let compilation = crate::sema::compile_project(&options) + .map_err(|e| err("view.compile_failed", format!("{e:#}")))?; + + let rel = req.document.replace('\\', "/"); + let mut presentation = KnowledgePresentation::from_project(&compilation.semantics, &rel, &title); + presentation.page_token = { + let mut h = Sha256::new(); + h.update(raw.as_slice()); + h.update(presentation.semantic_generation.as_bytes()); + format!("{:x}", h.finalize()) + }; + + if let Some(gen) = &req.generation { + if !gen.is_empty() && gen != &presentation.semantic_generation { + return Err(err( + "view.generation_mismatch", + format!( + "requested generation {gen} but projection is {}; refusing silent downgrade", + presentation.semantic_generation + ), + )); + } + } + + let wants = |k: &str| req.include.iter().any(|x| x == k); + + let heading_outline = outline_from_headings(&source, &presentation); + + // Prefer document surface HTML when parseable; fall back to knowledge HTML. + // Never call build_file_view (that path executes). + let fragment = if wants("fragment") { + let raw_html = match crate::load::parse_source_auto(&source) { + Ok(module) => render_module_document(&module, &source, &[]), + Err(_) => render_knowledge_html(&compilation.semantics, &rel), + }; + wrap_fragment(&inject_mq_node_attrs(&raw_html, &heading_outline, &presentation)) + } else { + String::new() + }; + + let outline = if wants("outline") { + if heading_outline.is_empty() { + outline_from_presentation(&presentation) + } else { + json!(heading_outline) + } + } else { + json!([]) + }; + let nodes = if wants("nodes") { + nodes_from_presentation(&presentation, &req.node_id) + } else { + json!([]) + }; + let refs = if wants("refs") { + refs_from_project(&compilation.semantics, &rel) + } else { + json!([]) + }; + + let mut diag_msgs: Vec = presentation.diagnostics.clone(); + for d in &compilation.diagnostics { + diag_msgs.push(d.message.clone()); + } + let diagnostics = if wants("diagnostics") { + json!(diag_msgs) + } else { + json!([]) + }; + + let completeness = if diag_msgs.is_empty() { + "complete" + } else { + "partial" + }; + + Ok(json!({ + "schema": "DocumentViewV1", + "schema_version": 1, + "document": { + "id": rel, + "title": title, + "source_revision": source_revision, + "semantic_digest": presentation.source_digest, + "generation_id": if presentation.semantic_generation.is_empty() { + Value::Null + } else { + json!(presentation.semantic_generation) + }, + "presentation_token": presentation.page_token, + }, + "execution": {"state": "not_run", "run_id": Value::Null}, + "render": { + "format": "sanitized_html", + "fragment": fragment, + "renderer_version": "1" + }, + "outline": outline, + "nodes": nodes, + "refs": refs, + "diagnostics": diagnostics, + "completeness": completeness, + "capabilities": {"read_only": true, "has_semantics": true, "executed": false} + })) +} + +fn wrap_fragment(inner: &str) -> String { + // No html/body, no scripts, namespaced for host theming. + let sanitized = strip_script_tags(inner); + format!("
{sanitized}
") +} + +/// Remove `` blocks (case-insensitive) so embed fragments stay non-executable. +fn strip_script_tags(html: &str) -> String { + let lower = html.to_ascii_lowercase(); + let mut out = String::with_capacity(html.len()); + let mut i = 0usize; + while i < html.len() { + if lower[i..].starts_with("') + .map(|p| i + p + 1) + .unwrap_or(html.len()); + if let Some(rel) = lower[after_open..].find("') + .map(|p| close_at + p + 1) + .unwrap_or(html.len()); + i = end; + } else { + // Truncate from the dangerous open tag onward. + break; + } + continue; + } + // Advance one UTF-8 char so multi-byte text stays intact. + let ch = html[i..].chars().next().unwrap(); + out.push(ch); + i += ch.len_utf8(); + } + out +} + +fn title_from_source(source: &str, fallback: &str) -> String { + for line in source.lines().take(40) { + let t = line.trim(); + if let Some(rest) = t.strip_prefix("# ") { + return rest.trim().to_string(); + } + if let Some(rest) = t.strip_prefix("title:") { + return rest.trim().trim_matches('"').to_string(); + } + } + Path::new(fallback) + .file_stem() + .and_then(|s| s.to_str()) + .unwrap_or(fallback) + .to_string() +} + +/// Outline entries from Markdown / Marqdo `#` headings; prefer matching node ids. +fn outline_from_headings(source: &str, p: &KnowledgePresentation) -> Vec { + let mut items = Vec::new(); + let mut order = 0usize; + for (idx, line) in source.lines().enumerate() { + let line_no = (idx + 1) as u32; + let trimmed = line.trim_start(); + if !trimmed.starts_with('#') { + continue; + } + let hashes = trimmed.chars().take_while(|c| *c == '#').count(); + if hashes == 0 || hashes > 6 { + continue; + } + // `# Title`, `## Section`, or Marqdo `#main` / `##greet` (no space). + let after = &trimmed[hashes..]; + let label = after.trim(); + if label.is_empty() { + continue; + } + let matched = p.nodes.iter().find(|n| { + n.source_line == line_no + || n.display_name == label + || n.node_id == label + || n.artifact_id.as_deref() == Some(label) + }); + let id = matched + .map(|n| n.node_id.clone()) + .unwrap_or_else(|| format!("heading:{line_no}")); + let artifact_id = matched.and_then(|n| n.artifact_id.clone()); + items.push(json!({ + "id": id, + "parent_id": Value::Null, + "kind": "heading", + "label": label, + "order": order, + "source_span": {"line": line_no}, + "artifact_id": artifact_id, + })); + order += 1; + } + items +} + +/// Stamp `data-mq-node` onto elements that already carry `data-line` matching a node/outline id. +fn inject_mq_node_attrs(html: &str, outline: &[Value], p: &KnowledgePresentation) -> String { + let mut out = html.to_string(); + let mut bindings: Vec<(u32, String)> = Vec::new(); + for n in &p.nodes { + bindings.push((n.source_line, n.node_id.clone())); + } + for item in outline { + if let (Some(line), Some(id)) = ( + item.pointer("/source_span/line").and_then(|v| v.as_u64()), + item.get("id").and_then(|v| v.as_str()), + ) { + bindings.push((line as u32, id.to_string())); + } + } + bindings.sort_by_key(|(line, _)| *line); + bindings.dedup_by_key(|(line, _)| *line); + for (line, id) in bindings { + let needle = format!("data-line=\"{line}\""); + let attr = format!("data-line=\"{line}\" data-mq-node=\"{}\"", html_attr_escape(&id)); + if out.contains(&needle) && !out.contains(&format!("data-mq-node=\"{}\"", html_attr_escape(&id))) { + out = out.replacen(&needle, &attr, 1); + } + } + out +} + +fn html_attr_escape(s: &str) -> String { + s.replace('&', "&") + .replace('"', """) + .replace('<', "<") +} + +fn outline_from_presentation(p: &KnowledgePresentation) -> Value { + json!(p + .nodes + .iter() + .enumerate() + .map(|(i, n)| json!({ + "id": n.node_id, + "parent_id": Value::Null, + "kind": "heading", + "label": n.display_name, + "order": i, + "artifact_id": n.artifact_id, + })) + .collect::>()) +} + +fn nodes_from_presentation(p: &KnowledgePresentation, filter: &Option) -> Value { + json!(p + .nodes + .iter() + .filter(|n| filter.as_ref().map(|f| &n.node_id == f || n.artifact_id.as_ref() == Some(f)).unwrap_or(true)) + .map(|n| json!({ + "id": n.node_id, + "parent_id": Value::Null, + "kind": n.kind, + "label": n.display_name, + "order": n.source_line, + "source_span": {"file": n.source_file, "line": n.source_line}, + "artifact_id": n.artifact_id, + "ref_ids": [], + "children": [], + "evidence": n.evidence, + "effects": n.effects, + "dependencies": n.dependencies, + "signature": n.signature, + })) + .collect::>()) +} + +fn refs_from_project(project: &crate::sema::SemanticProject, rel: &str) -> Value { + let mut refs = Vec::new(); + for a in &project.artifacts { + if a.source.file != rel { + continue; + } + for dep in &a.dependencies { + refs.push(json!({ + "id": format!("ref:{}:{}", a.artifact_id.0, dep.0), + "from_artifact_id": a.artifact_id.0, + "to": dep.0, + "relation": "depends_on", + })); + } + } + json!(refs) +} + +fn canonicalize_project(p: &Path) -> Result { + let root = if p.is_file() { + p.parent().unwrap_or_else(|| Path::new(".")).to_path_buf() + } else { + p.to_path_buf() + }; + root.canonicalize().map_err(|e| { + err( + "view.project_invalid", + format!("project {}: {e}", root.display()), + ) + }) +} + +fn resolve_under(root: &Path, rel: &str) -> Result { + let joined = root.join(rel); + let canon = joined.canonicalize().map_err(|e| { + err( + "view.document_not_found", + format!("{}: {e}", joined.display()), + ) + })?; + if !canon.starts_with(root) { + return Err(err( + "view.path_escape", + "document path escapes project root", + )); + } + Ok(canon) +} + +fn err(code: &str, message: impl Into) -> Value { + json!({"code": code, "message": message.into()}) +} diff --git a/src/workbench/event_sink.rs b/src/workbench/event_sink.rs new file mode 100644 index 0000000..9b7acf2 --- /dev/null +++ b/src/workbench/event_sink.rs @@ -0,0 +1,59 @@ +//! Request-local RuntimeEventV1 sink (M02). Avoids process-global event mix-up. + +use serde_json::{json, Value}; +use std::fs::OpenOptions; +use std::io::Write; +use std::path::{Path, PathBuf}; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{Mutex, OnceLock}; + +static SEQ: AtomicU64 = AtomicU64::new(0); + +fn sink_path() -> &'static Mutex> { + static SINK: OnceLock>> = OnceLock::new(); + SINK.get_or_init(|| Mutex::new(None)) +} + +/// Bind this process/request to an NDJSON event file (also used by web sse-live). +pub fn bind_event_sink(path: impl AsRef) { + if let Ok(mut g) = sink_path().lock() { + *g = Some(path.as_ref().to_path_buf()); + } + crate::workbench::capabilities::set_event_stream_ready(true); +} + +pub fn clear_event_sink() { + if let Ok(mut g) = sink_path().lock() { + *g = None; + } +} + +pub fn emit_event(run_id: &str, event_type: &str, payload: Value) -> Value { + let seq = SEQ.fetch_add(1, Ordering::Relaxed); + let event_id = format!("evt-{run_id}-{seq}"); + let ev = crate::workbench::usage::runtime_event(&event_id, seq, run_id, event_type, payload); + if let Ok(g) = sink_path().lock() { + if let Some(path) = g.as_ref() { + if let Ok(mut f) = OpenOptions::new().create(true).append(true).open(path) { + let _ = writeln!(f, "{}", ev); + } + } + } + // Also honor MARQDO_EVENT_FORWARD for subtask/web bridge. + if let Ok(fwd) = std::env::var("MARQDO_EVENT_FORWARD") { + if !fwd.is_empty() { + if let Ok(mut f) = OpenOptions::new().create(true).append(true).open(&fwd) { + let _ = writeln!(f, "{}", ev); + } + } + } + ev +} + +pub fn emit_message_delta(run_id: &str, text: &str) -> Value { + emit_event( + run_id, + "message.delta", + json!({"text": text, "format": "plain"}), + ) +} diff --git a/src/workbench/locator.rs b/src/workbench/locator.rs new file mode 100644 index 0000000..5984094 --- /dev/null +++ b/src/workbench/locator.rs @@ -0,0 +1,312 @@ +//! KnowledgeLocatorV1 + view.reference_resolve (M04). + +use std::path::{Path, PathBuf}; + +use serde_json::{json, Value}; + +#[derive(Debug, Clone)] +pub struct KnowledgeLocator { + pub project_id: String, + pub document_id: String, + pub source_revision: Option, + pub generation_id: Option, + pub artifact_id: Option, + pub node_id: Option, + pub source_ref_id: Option, + pub span: Option, + pub relation: String, +} + +impl KnowledgeLocator { + pub fn from_params(params: &Value) -> Result { + let loc = params.get("locator").unwrap_or(params); + Ok(Self { + project_id: loc + .get("project_id") + .or_else(|| params.get("project")) + .and_then(|v| v.as_str()) + .unwrap_or("") + .to_string(), + document_id: loc + .get("document_id") + .or_else(|| params.get("document")) + .and_then(|v| v.as_str()) + .unwrap_or("") + .to_string(), + source_revision: loc + .get("source_revision") + .or_else(|| loc.get("revision")) + .and_then(|v| v.as_str()) + .map(|s| s.to_string()), + generation_id: loc + .get("generation_id") + .or_else(|| loc.get("generation")) + .and_then(|v| v.as_str()) + .map(|s| s.to_string()), + artifact_id: loc + .get("artifact_id") + .and_then(|v| v.as_str()) + .map(|s| s.to_string()), + node_id: loc + .get("node_id") + .and_then(|v| v.as_str()) + .map(|s| s.to_string()), + source_ref_id: loc + .get("source_ref_id") + .or_else(|| loc.get("ref_id")) + .and_then(|v| v.as_str()) + .map(|s| s.to_string()), + span: loc.get("span").cloned(), + relation: loc + .get("relation") + .and_then(|v| v.as_str()) + .unwrap_or("cites") + .to_string(), + }) + } + + pub fn to_json(&self) -> Value { + json!({ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "project_id": self.project_id, + "document_id": self.document_id, + "source_revision": self.source_revision, + "generation_id": self.generation_id, + "artifact_id": self.artifact_id, + "node_id": self.node_id, + "source_ref_id": self.source_ref_id, + "span": self.span, + "relation": self.relation, + }) + } +} + +/// Resolve a locator or current-document ref_id against a project snapshot. +/// Never silently falls back to "latest" when expected revision/generation mismatch. +pub fn resolve_reference(params: &Value) -> Result { + let locator = KnowledgeLocator::from_params(params).map_err(|e| { + json!({"code": "locator.bad_request", "message": e}) + })?; + if locator.project_id.is_empty() { + return Err(json!({ + "code": "locator.bad_request", + "message": "project_id/project required" + })); + } + let root = PathBuf::from(&locator.project_id); + let root = root.canonicalize().map_err(|e| { + json!({"code": "locator.project_invalid", "message": format!("{e}")}) + })?; + + let options = crate::sema::CompileOptions::for_path(&root).map_err(|e| { + json!({"code": "locator.compile_failed", "message": format!("{e:#}")}) + })?; + let compilation = crate::sema::compile_project(&options).map_err(|e| { + json!({"code": "locator.compile_failed", "message": format!("{e:#}")}) + })?; + + // Generation mismatch → version_unavailable / stale (no silent current). + if let Some(expected_gen) = &locator.generation_id { + if !expected_gen.is_empty() { + let current_gen = semantic_generation(&compilation.semantics); + if ¤t_gen != expected_gen { + return Ok(json!({ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "version_unavailable", + "precision": "none", + "locator": locator.to_json(), + "target": Value::Null, + "candidates": [], + "diagnostics": [{ + "code": "generation_mismatch", + "message": format!( + "expected generation {expected_gen}, current projection is {current_gen}; refusing silent downgrade" + ) + }], + "label": "引用版本不可用" + })); + } + } + } + + // Source revision check when document is known. + if let (Some(expected_rev), true) = ( + &locator.source_revision, + !locator.document_id.is_empty(), + ) { + if !expected_rev.is_empty() { + let abs = resolve_under(&root, &locator.document_id)?; + let raw = std::fs::read(&abs).map_err(|e| { + json!({"code": "locator.source_unreadable", "message": format!("{e}")}) + })?; + use sha2::{Digest, Sha256}; + let actual = format!("sha256:{:x}", Sha256::digest(&raw)); + if &actual != expected_rev { + return Ok(json!({ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "stale", + "precision": "document", + "locator": locator.to_json(), + "target": { + "document_id": locator.document_id, + "source_revision": actual, + "note": "source changed since citation" + }, + "candidates": [], + "diagnostics": [{ + "code": "source_revision_mismatch", + "message": format!("expected {expected_rev}, actual {actual}") + }], + "label": "该来源已变化" + })); + } + } + } + + let query = locator + .artifact_id + .clone() + .or_else(|| locator.node_id.clone()) + .or_else(|| locator.source_ref_id.clone()); + + let Some(q) = query else { + if locator.document_id.is_empty() { + return Ok(json!({ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "unresolved", + "precision": "none", + "locator": locator.to_json(), + "target": Value::Null, + "candidates": [], + "diagnostics": [{"code": "missing_target", "message": "no artifact/node/ref id"}], + "label": "无法定位该知识" + })); + } + return Ok(json!({ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "resolved", + "precision": "document", + "locator": locator.to_json(), + "target": { + "document_id": locator.document_id, + "relation": locator.relation, + }, + "candidates": [], + "diagnostics": [], + "label": locator.document_id, + })); + }; + + let mut matches: Vec<&crate::sema::SemanticArtifact> = compilation + .semantics + .artifacts + .iter() + .filter(|a| { + a.artifact_id.0 == q + || a.display_name == q + || a.symbol_id + .as_ref() + .map(|s| s.as_str() == q) + .unwrap_or(false) + }) + .collect(); + + if !locator.document_id.is_empty() { + let scoped: Vec<_> = matches + .iter() + .copied() + .filter(|a| a.source.file == locator.document_id) + .collect(); + if !scoped.is_empty() { + matches = scoped; + } + } + + match matches.len() { + 0 => Ok(json!({ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "unresolved", + "precision": "none", + "locator": locator.to_json(), + "target": Value::Null, + "candidates": [], + "diagnostics": [{"code": "not_found", "message": format!("no artifact matching {q}")}], + "label": "尚未定位到具体知识" + })), + 1 => { + let a = matches[0]; + Ok(json!({ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "resolved", + "precision": "artifact", + "locator": locator.to_json(), + "target": { + "artifact_id": a.artifact_id.0, + "display_name": a.display_name, + "document_id": a.source.file, + "span": {"line": a.source.line, "end_line": a.source.end_line}, + "evidence": a.verification.evidence, + "relation": locator.relation, + }, + "candidates": [], + "diagnostics": [], + "label": a.display_name, + })) + } + _ => { + let candidates: Vec = matches + .iter() + .map(|a| { + json!({ + "artifact_id": a.artifact_id.0, + "display_name": a.display_name, + "document_id": a.source.file, + "line": a.source.line, + }) + }) + .collect(); + Ok(json!({ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "ambiguous", + "precision": "none", + "locator": locator.to_json(), + "target": Value::Null, + "candidates": candidates, + "diagnostics": [{"code": "ambiguous", "message": format!("{} candidates for {q}", matches.len())}], + "label": "存在多个候选" + })) + } + } +} + +fn semantic_generation(project: &crate::sema::SemanticProject) -> String { + use sha2::{Digest, Sha256}; + let mut gh = Sha256::new(); + gh.update(project.project_id.as_bytes()); + for a in &project.artifacts { + gh.update(a.hashes.artifact.0.as_bytes()); + } + format!("{:x}", gh.finalize())[..16].to_string() +} + +fn resolve_under(root: &Path, rel: &str) -> Result { + let joined = root.join(rel); + let canon = joined.canonicalize().map_err(|e| { + json!({"code": "locator.document_not_found", "message": format!("{}: {e}", joined.display())}) + })?; + if !canon.starts_with(root) { + return Err(json!({ + "code": "locator.path_escape", + "message": "document path escapes project root" + })); + } + Ok(canon) +} diff --git a/src/workbench/mod.rs b/src/workbench/mod.rs new file mode 100644 index 0000000..4adfbdf --- /dev/null +++ b/src/workbench/mod.rs @@ -0,0 +1,25 @@ +//! Workbench runtime contracts (M00–M05): schemas, capabilities, DocumentView, +//! KnowledgeLocator, ModelSelection / ModelCallUsage / RuntimeEvent / RunContext. +//! +//! Spec: `doc/design/qdagent-workbench-runtime-and-view-2026-10-25.md`. +//! These are v1 contracts distinct from KnowledgePresentation `schema_version:1`. + +pub mod analysis_cache; +pub mod capabilities; +#[cfg(feature = "view")] +pub mod document_view; +pub mod event_sink; +pub mod locator; +pub mod run_context; +pub mod selection; +pub mod usage; + +pub use analysis_cache::{cache_keys_isolated, note_analysis, project_source_digest}; +pub use capabilities::probe_capabilities; +#[cfg(feature = "view")] +pub use document_view::{build_document_view, DocumentViewRequest}; +pub use event_sink::{bind_event_sink, clear_event_sink, emit_event, emit_message_delta}; +pub use locator::{resolve_reference, KnowledgeLocator}; +pub use run_context::{cancel_requested, RunContext}; +pub use selection::{parse_model_selection, ModelSelection}; +pub use usage::{model_call_usage_from_provider, runtime_event, token_or_null}; diff --git a/src/workbench/run_context.rs b/src/workbench/run_context.rs new file mode 100644 index 0000000..8fc884b --- /dev/null +++ b/src/workbench/run_context.rs @@ -0,0 +1,148 @@ +//! RunContextV1 — request-local run identity for selection / usage / cancel (M01). + +use serde_json::{json, Map, Value}; + +#[derive(Debug, Clone, Default)] +pub struct RunContext { + pub run_id: String, + pub parent_run_id: Option, + pub turn_id: Option, + pub workspace_scope: Option, + pub selection_context: Map, + pub usage_sink: Option, + pub event_sink: Option, + pub cancellation_token: Option, +} + +impl RunContext { + pub fn from_value(v: &Value) -> Result { + let run_id = v + .get("run_id") + .and_then(|x| x.as_str()) + .filter(|s| !s.is_empty()) + .ok_or_else(|| "run_context.missing:run_id".to_string())? + .to_string(); + Ok(Self { + run_id, + parent_run_id: v + .get("parent_run_id") + .and_then(|x| x.as_str()) + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()), + turn_id: v + .get("turn_id") + .and_then(|x| x.as_str()) + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()), + workspace_scope: v + .get("workspace_scope") + .and_then(|x| x.as_str()) + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()), + selection_context: v + .get("selection_context") + .and_then(|x| x.as_object()) + .cloned() + .unwrap_or_default(), + usage_sink: v + .get("usage_sink") + .and_then(|x| x.as_str()) + .map(|s| s.to_string()), + event_sink: v + .get("event_sink") + .and_then(|x| x.as_str()) + .map(|s| s.to_string()), + cancellation_token: v + .get("cancellation_token") + .and_then(|x| x.as_str()) + .map(|s| s.to_string()), + }) + } + + pub fn to_json(&self) -> Value { + json!({ + "schema": "RunContextV1", + "schema_version": 1, + "run_id": self.run_id, + "parent_run_id": self.parent_run_id, + "turn_id": self.turn_id, + "workspace_scope": self.workspace_scope, + "selection_context": self.selection_context, + "usage_sink": self.usage_sink, + "event_sink": self.event_sink, + "cancellation_token": self.cancellation_token, + }) + } + + /// Child inherits selection/event/cancel; new run_id linked to parent. + pub fn spawn_child(&self, child_run_id: &str) -> Self { + Self { + run_id: child_run_id.to_string(), + parent_run_id: Some(self.run_id.clone()), + turn_id: self.turn_id.clone(), + workspace_scope: self.workspace_scope.clone(), + selection_context: self.selection_context.clone(), + usage_sink: self.usage_sink.clone(), + event_sink: self.event_sink.clone(), + cancellation_token: self.cancellation_token.clone(), + } + } + + /// Env vars for subtask injection (never includes api_key). + pub fn selection_env(&self) -> Vec<(String, String)> { + let mut out = Vec::new(); + out.push(("MARQDO_RUN_ID".into(), self.run_id.clone())); + if let Some(p) = &self.parent_run_id { + out.push(("MARQDO_PARENT_RUN_ID".into(), p.clone())); + } + if let Some(t) = &self.turn_id { + out.push(("MARQDO_TURN_ID".into(), t.clone())); + } + if let Some(w) = &self.workspace_scope { + out.push(("MARQDO_WORKSPACE_SCOPE".into(), w.clone())); + } + if let Some(tok) = &self.cancellation_token { + out.push(("MARQDO_CANCEL_TOKEN".into(), tok.clone())); + } + if let Some(sink) = &self.event_sink { + out.push(("MARQDO_EVENT_FORWARD".into(), sink.clone())); + } + if let Some(model) = self + .selection_context + .get("upstream_model_id") + .or_else(|| self.selection_context.get("model")) + .and_then(|v| v.as_str()) + { + out.push(("MARQDO_LLM_MODEL".into(), model.to_string())); + out.push(("OPENAI_MODEL".into(), model.to_string())); + } + if let Some(base) = self + .selection_context + .get("base_url") + .and_then(|v| v.as_str()) + { + out.push(("MARQDO_LLM_BASE_URL".into(), base.to_string())); + out.push(("OPENAI_BASE_URL".into(), base.to_string())); + } + out + } +} + +/// Check whether a cancel token file/env indicates cancellation. +pub fn cancel_requested(token: Option<&str>) -> bool { + let Some(tok) = token.filter(|s| !s.is_empty()) else { + return false; + }; + // Token may be a path: presence of file or contents "1"/"cancelled" means stop. + let path = std::path::Path::new(tok); + if path.is_file() { + if let Ok(body) = std::fs::read_to_string(path) { + let t = body.trim(); + return t.is_empty() || t == "1" || t.eq_ignore_ascii_case("cancelled"); + } + return true; + } + std::env::var("MARQDO_CANCEL") + .map(|v| v == "1" || v.eq_ignore_ascii_case("true")) + .unwrap_or(false) +} diff --git a/src/workbench/selection.rs b/src/workbench/selection.rs new file mode 100644 index 0000000..a204b4f --- /dev/null +++ b/src/workbench/selection.rs @@ -0,0 +1,147 @@ +//! ModelSelectionV1 — request-level model identity (M01). + +use serde_json::{json, Map, Value}; + +#[derive(Debug, Clone)] +pub struct ModelSelection { + pub schema_version: u64, + pub selection_id: String, + pub provider_id: String, + pub model_id: String, + pub provider_revision: Option, + pub model_revision: Option, + pub adapter: String, + pub endpoint_ref: Option, + pub credential_ref: Option, + pub credential_version: Option, + pub upstream_model_id: String, + pub parameters: Map, + pub fallback_policy: String, + pub auth_mode: String, + /// Resolved call-local endpoint (never serialize API keys into events). + pub base_url: Option, + /// Call-local credential material — must not appear in public snapshots. + pub api_key: Option, +} + +impl ModelSelection { + pub fn public_snapshot(&self) -> Value { + json!({ + "schema": "ModelSelectionV1", + "schema_version": self.schema_version, + "selection_id": self.selection_id, + "provider_id": self.provider_id, + "model_id": self.model_id, + "provider_revision": self.provider_revision, + "model_revision": self.model_revision, + "adapter": self.adapter, + "endpoint_ref": self.endpoint_ref, + "credential_ref": self.credential_ref, + "credential_version": self.credential_version, + "upstream_model_id": self.upstream_model_id, + "parameters": self.parameters, + "fallback_policy": self.fallback_policy, + "auth_mode": self.auth_mode, + }) + } +} + +/// Parse selection. In `strict` mode, missing required fields error (no env fallback). +pub fn parse_model_selection(v: &Value, strict: bool) -> Result { + let schema_version = v + .get("schema_version") + .and_then(|x| x.as_u64()) + .unwrap_or(1); + if schema_version != 1 { + return Err(format!("unsupported ModelSelection schema_version={schema_version}")); + } + let require = |key: &str| -> Result { + match v.get(key).and_then(|x| x.as_str()).filter(|s| !s.is_empty()) { + Some(s) => Ok(s.to_string()), + None if strict => Err(format!("llm.selection_missing:{key}")), + None => Ok(String::new()), + } + }; + let selection_id = require("selection_id")?; + let provider_id = require("provider_id")?; + let model_id = require("model_id")?; + let adapter = v + .get("adapter") + .and_then(|x| x.as_str()) + .unwrap_or("openai-compatible") + .to_string(); + let upstream = v + .get("upstream_model_id") + .and_then(|x| x.as_str()) + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()) + .or_else(|| { + if model_id.is_empty() { + None + } else { + Some(model_id.clone()) + } + }); + let upstream_model_id = match upstream { + Some(u) => u, + None if strict => return Err("llm.selection_missing:upstream_model_id".into()), + None => String::new(), + }; + let auth_mode = v + .get("auth_mode") + .and_then(|x| x.as_str()) + .unwrap_or("api_key") + .to_string(); + let fallback_policy = v + .get("fallback_policy") + .and_then(|x| x.as_str()) + .unwrap_or("disabled") + .to_string(); + let parameters = v + .get("parameters") + .and_then(|x| x.as_object()) + .cloned() + .unwrap_or_default(); + let base_url = v + .get("base_url") + .and_then(|x| x.as_str()) + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()); + let api_key = v + .get("api_key") + .and_then(|x| x.as_str()) + .map(|s| s.to_string()); + if strict && auth_mode != "none" && api_key.as_ref().map(|s| s.is_empty()).unwrap_or(true) { + // credential_ref may be resolved by host; allow missing api_key if credential_ref present + if v.get("credential_ref").and_then(|x| x.as_str()).is_none() && base_url.is_some() { + return Err("llm.selection_missing:api_key_or_credential_ref".into()); + } + } + if strict && selection_id.is_empty() { + return Err("llm.selection_missing:selection_id".into()); + } + Ok(ModelSelection { + schema_version, + selection_id, + provider_id, + model_id, + provider_revision: v.get("provider_revision").and_then(|x| x.as_u64()), + model_revision: v.get("model_revision").and_then(|x| x.as_u64()), + adapter, + endpoint_ref: v + .get("endpoint_ref") + .and_then(|x| x.as_str()) + .map(|s| s.to_string()), + credential_ref: v + .get("credential_ref") + .and_then(|x| x.as_str()) + .map(|s| s.to_string()), + credential_version: v.get("credential_version").and_then(|x| x.as_u64()), + upstream_model_id, + parameters, + fallback_policy, + auth_mode, + base_url, + api_key, + }) +} diff --git a/src/workbench/usage.rs b/src/workbench/usage.rs new file mode 100644 index 0000000..464b435 --- /dev/null +++ b/src/workbench/usage.rs @@ -0,0 +1,144 @@ +//! ModelCallUsageV1 — null ≠ 0; measurement_source honesty (M02 / F04). + +use serde_json::{json, Value}; + +/// Prefer null over zero when the provider omitted a field. +pub fn token_or_null(v: Option) -> Value { + match v { + Some(n) if n >= 0 => json!(n), + _ => Value::Null, + } +} + +/// Build ModelCallUsageV1 from an OpenAI-shaped `usage` object (or Null). +pub fn model_call_usage_from_provider( + call_id: &str, + attempt_id: &str, + run_id: &str, + selection: &Value, + provider_usage: &Value, + status: &str, +) -> Value { + let has_usage = provider_usage.is_object(); + let input = provider_usage + .get("prompt_tokens") + .or_else(|| provider_usage.get("input_tokens")) + .and_then(|v| v.as_i64()); + let output = provider_usage + .get("completion_tokens") + .or_else(|| provider_usage.get("output_tokens")) + .and_then(|v| v.as_i64()); + let total = provider_usage + .get("total_tokens") + .and_then(|v| v.as_i64()) + .or_else(|| match (input, output) { + (Some(a), Some(b)) => Some(a + b), + _ => None, + }); + let cached = provider_usage + .get("cached_input_tokens") + .or_else(|| { + provider_usage + .get("prompt_tokens_details") + .and_then(|d| d.get("cached_tokens")) + }) + .and_then(|v| v.as_i64()); + let reasoning = provider_usage + .get("reasoning_output_tokens") + .or_else(|| { + provider_usage + .get("completion_tokens_details") + .and_then(|d| d.get("reasoning_tokens")) + }) + .and_then(|v| v.as_i64()); + + let (usage_state, measurement_source) = if !has_usage { + ("unknown", "none") + } else if input.is_none() && output.is_none() && total.is_none() { + ("unknown", "none") + } else { + ("final", "provider") + }; + + let mut diagnostics = Vec::new(); + if let (Some(t), Some(i), Some(o)) = (total, input, output) { + if t != i + o && cached.is_none() { + diagnostics.push(json!({ + "code": "usage.inconsistent", + "message": format!("total_tokens={t} != input+output={}", i + o) + })); + } + } + + json!({ + "schema": "ModelCallUsageV1", + "schema_version": 1, + "call_id": call_id, + "attempt_id": attempt_id, + "parent_call_id": Value::Null, + "run_id": run_id, + "turn_id": selection.get("turn_id").cloned().unwrap_or(Value::Null), + "workspace_scope": selection.get("workspace_scope").cloned().unwrap_or(Value::Null), + "purpose": selection.get("purpose").cloned().unwrap_or(json!("answer")), + "selection_id": selection.get("selection_id").cloned().unwrap_or(Value::Null), + "provider_id": selection.get("provider_id").cloned().unwrap_or(Value::Null), + "model_id": selection.get("model_id").cloned().unwrap_or(Value::Null), + "requested_model": selection.get("upstream_model_id").cloned() + .or_else(|| selection.get("model").cloned()) + .unwrap_or(Value::Null), + "resolved_model": selection.get("resolved_model").cloned() + .or_else(|| selection.get("upstream_model_id").cloned()) + .unwrap_or(Value::Null), + "status": status, + "usage_state": usage_state, + "measurement_source": measurement_source, + "input_tokens": token_or_null(input), + "output_tokens": token_or_null(output), + "total_tokens": token_or_null(total), + "cached_input_tokens": token_or_null(cached), + "reasoning_output_tokens": token_or_null(reasoning), + "revision": 1, + "provider_request_id": provider_usage.get("provider_request_id").cloned().unwrap_or(Value::Null), + "diagnostics": diagnostics, + // Legacy fields kept but marked — apps must not treat token_cost as money. + "legacy": { + "tokens": token_or_null(total), + "token_cost": token_or_null(total), + "deprecated": true, + "unit": "tokens" + } + }) +} + +/// RuntimeEventV1 envelope helper. +pub fn runtime_event( + event_id: &str, + seq: u64, + run_id: &str, + event_type: &str, + payload: Value, +) -> Value { + json!({ + "schema": "RuntimeEventV1", + "schema_version": 1, + "event_id": event_id, + "seq": seq, + "run_id": run_id, + "parent_run_id": Value::Null, + "turn_id": Value::Null, + "workspace_scope": Value::Null, + "timestamp": chrono_like_now(), + "type": event_type, + "payload": payload + }) +} + +fn chrono_like_now() -> String { + // Avoid new chrono dep: RFC3339-ish from system time. + use std::time::{SystemTime, UNIX_EPOCH}; + let secs = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0); + format!("{secs}") +} diff --git a/tests/executable_knowledge.rs b/tests/executable_knowledge.rs index 6acbe53..6d52b1c 100644 --- a/tests/executable_knowledge.rs +++ b/tests/executable_knowledge.rs @@ -5,6 +5,64 @@ use std::process::Command; +/// Build `libagent` once per test process and prefer it over a stale `~/.marqdo/ext`. +/// Same contract as `tests/gold.rs::ensure_agent_plugin_built` — EK gate must not +/// depend on a manually copied `ext/native/libagent.so`. +fn ensure_agent_plugin_built() { + use std::sync::Once; + static ONCE: Once = Once::new(); + ONCE.call_once(|| { + let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let status = Command::new("cargo") + .args(["build", "-p", "marqdo_plugin_agent"]) + .current_dir(&root) + .status() + .expect("build marqdo_plugin_agent"); + assert!(status.success(), "failed to build marqdo_plugin_agent"); + let lib = if cfg!(windows) { + root.join("target").join("debug").join("agent.dll") + } else if cfg!(target_os = "macos") { + root.join("target").join("debug").join("libagent.dylib") + } else { + root.join("target").join("debug").join("libagent.so") + }; + let release = if cfg!(windows) { + root.join("target").join("release").join("agent.dll") + } else if cfg!(target_os = "macos") { + root.join("target").join("release").join("libagent.dylib") + } else { + root.join("target").join("release").join("libagent.so") + }; + let chosen = if lib.is_file() { + if release.is_file() { + let lib_m = std::fs::metadata(&lib).and_then(|m| m.modified()).ok(); + let rel_m = std::fs::metadata(&release).and_then(|m| m.modified()).ok(); + match (lib_m, rel_m) { + (Some(l), Some(r)) if r > l => release, + _ => lib, + } + } else { + lib + } + } else { + release + }; + assert!( + chosen.is_file(), + "missing agent plugin at {}", + chosen.display() + ); + // SAFETY: single-threaded Once; children inherit for the rest of the test process. + std::env::set_var("MARQDO_AGENT_PLUGIN", &chosen); + if std::env::var_os("MARQDO_EXT").is_none() { + let ext = root.join("ext"); + if ext.join("ai").join("agent.mq.md").is_file() { + std::env::set_var("MARQDO_EXT", &ext); + } + } + }); +} + fn run_backend(path: &str, backend: &str) -> (i32, String, String) { let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")); let out = Command::new(env!("CARGO_BIN_EXE_marqdo")) @@ -1657,6 +1715,12 @@ fn v2_sse_accepts_packet_rejects_stale_token() { assert!(marqdo::view::sse_accepts_packet("abc", Some(""))); assert!(marqdo::view::sse_accepts_packet("abc", Some("abc"))); assert!(!marqdo::view::sse_accepts_packet("abc", Some("zzz"))); + // Embed clients must use strict: missing/empty/mismatch all reject. + assert!(!marqdo::view::sse_accepts_packet_strict("abc", None)); + assert!(!marqdo::view::sse_accepts_packet_strict("abc", Some(""))); + assert!(!marqdo::view::sse_accepts_packet_strict("", Some("abc"))); + assert!(!marqdo::view::sse_accepts_packet_strict("abc", Some("zzz"))); + assert!(marqdo::view::sse_accepts_packet_strict("abc", Some("abc"))); assert_eq!( marqdo::view::page_token_from_source("a.mq.md", "x"), marqdo::view::page_token_from_source("a.mq.md", "x") @@ -1869,6 +1933,7 @@ fn f1_publish_gate_chain_via_verify_may_publish() { #[test] fn f1_agent_eng_bind_gate_mq_offline() { + ensure_agent_plugin_built(); let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")); let out = Command::new(env!("CARGO_BIN_EXE_marqdo")) .args([ @@ -1878,6 +1943,10 @@ fn f1_agent_eng_bind_gate_mq_offline() { "tree", ]) .current_dir(&root) + .env( + "MARQDO_AGENT_PLUGIN", + std::env::var_os("MARQDO_AGENT_PLUGIN").expect("MARQDO_AGENT_PLUGIN set by ensure"), + ) .output() .unwrap(); let stdout = String::from_utf8_lossy(&out.stdout); diff --git a/tests/workbench/fixtures/document_view_partial.json b/tests/workbench/fixtures/document_view_partial.json new file mode 100644 index 0000000..e8d33ee --- /dev/null +++ b/tests/workbench/fixtures/document_view_partial.json @@ -0,0 +1,20 @@ +{ + "schema": "DocumentViewV1", + "schema_version": 1, + "document": { + "id": "knowledge/example.mq.md", + "title": "知识标题", + "source_revision": "sha256:requested-source", + "semantic_digest": "sha256:semantic-facts", + "generation_id": null, + "presentation_token": "opaque-token" + }, + "execution": {"state": "not_run", "run_id": null}, + "render": {"format": "sanitized_html", "fragment": "
", "renderer_version": "1"}, + "outline": [], + "nodes": [], + "refs": [], + "diagnostics": [{"code": "partial", "message": "missing dependency"}], + "completeness": "partial", + "capabilities": {"read_only": true, "has_semantics": true, "executed": false} +} diff --git a/tests/workbench/fixtures/document_view_unsupported.json b/tests/workbench/fixtures/document_view_unsupported.json new file mode 100644 index 0000000..8c6c170 --- /dev/null +++ b/tests/workbench/fixtures/document_view_unsupported.json @@ -0,0 +1,8 @@ +{ + "ok": false, + "error": { + "code": "view.document_unsupported_format", + "message": "binary formats require extract adapter", + "retryable": false + } +} diff --git a/tests/workbench/fixtures/locator_stale.json b/tests/workbench/fixtures/locator_stale.json new file mode 100644 index 0000000..ee9dcc8 --- /dev/null +++ b/tests/workbench/fixtures/locator_stale.json @@ -0,0 +1,19 @@ +{ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "stale", + "precision": "document", + "locator": { + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "project_id": "/authorized/project", + "document_id": "knowledge/example.mq.md", + "source_revision": "sha256:old", + "artifact_id": "mq://project/example/function", + "relation": "cites" + }, + "target": null, + "candidates": [], + "diagnostics": [{"code": "source_revision_mismatch", "message": "expected sha256:old, actual sha256:new"}], + "label": "该来源已变化" +} diff --git a/tests/workbench/fixtures/locator_version_unavailable.json b/tests/workbench/fixtures/locator_version_unavailable.json new file mode 100644 index 0000000..434a36a --- /dev/null +++ b/tests/workbench/fixtures/locator_version_unavailable.json @@ -0,0 +1,19 @@ +{ + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "resolution": "version_unavailable", + "precision": "none", + "locator": { + "schema": "KnowledgeLocatorV1", + "schema_version": 1, + "project_id": "/authorized/project", + "document_id": "knowledge/example.mq.md", + "generation_id": "generation-missing", + "artifact_id": "mq://project/example/function", + "relation": "cites" + }, + "target": null, + "candidates": [], + "diagnostics": [{"code": "generation_unavailable", "message": "snapshot not retained"}], + "label": "引用版本不可用" +} diff --git a/tests/workbench/fixtures/model_call_usage_cancelled.json b/tests/workbench/fixtures/model_call_usage_cancelled.json new file mode 100644 index 0000000..619eef4 --- /dev/null +++ b/tests/workbench/fixtures/model_call_usage_cancelled.json @@ -0,0 +1,26 @@ +{ + "schema": "ModelCallUsageV1", + "schema_version": 1, + "call_id": "call-2", + "attempt_id": "attempt-1", + "parent_call_id": null, + "run_id": "run-1", + "turn_id": "turn-1", + "workspace_scope": "ws-1", + "purpose": "answer", + "selection_id": "sel-1", + "provider_id": "provider-1", + "model_id": "model-2", + "requested_model": "example-model", + "resolved_model": "example-model", + "status": "cancelled", + "usage_state": "partial", + "measurement_source": "provider", + "input_tokens": 100, + "output_tokens": 12, + "total_tokens": 112, + "cached_input_tokens": null, + "reasoning_output_tokens": null, + "revision": 2, + "diagnostics": [] +} diff --git a/tests/workbench/fixtures/model_call_usage_unknown.json b/tests/workbench/fixtures/model_call_usage_unknown.json new file mode 100644 index 0000000..ccb45ff --- /dev/null +++ b/tests/workbench/fixtures/model_call_usage_unknown.json @@ -0,0 +1,26 @@ +{ + "schema": "ModelCallUsageV1", + "schema_version": 1, + "call_id": "call-1", + "attempt_id": "attempt-1", + "parent_call_id": null, + "run_id": "run-1", + "turn_id": "turn-1", + "workspace_scope": "ws-1", + "purpose": "answer", + "selection_id": "sel-1", + "provider_id": "provider-1", + "model_id": "model-2", + "requested_model": "example-model", + "resolved_model": "example-model", + "status": "succeeded", + "usage_state": "unknown", + "measurement_source": "none", + "input_tokens": null, + "output_tokens": null, + "total_tokens": null, + "cached_input_tokens": null, + "reasoning_output_tokens": null, + "revision": 1, + "diagnostics": [] +} diff --git a/tests/workbench/fixtures/model_selection_error.json b/tests/workbench/fixtures/model_selection_error.json new file mode 100644 index 0000000..233278d --- /dev/null +++ b/tests/workbench/fixtures/model_selection_error.json @@ -0,0 +1,12 @@ +{ + "schema": "ModelSelectionV1", + "schema_version": 1, + "error": { + "code": "llm.selection_missing", + "message": "selection_id required in strict mode", + "retryable": false + }, + "partial": { + "model_id": "model-2" + } +} diff --git a/tests/workbench/fixtures/model_selection_ok.json b/tests/workbench/fixtures/model_selection_ok.json new file mode 100644 index 0000000..f012c03 --- /dev/null +++ b/tests/workbench/fixtures/model_selection_ok.json @@ -0,0 +1,17 @@ +{ + "schema": "ModelSelectionV1", + "schema_version": 1, + "selection_id": "sel-1", + "provider_id": "provider-1", + "model_id": "model-2", + "provider_revision": 3, + "model_revision": 5, + "adapter": "openai-compatible", + "endpoint_ref": "endpoint-local-1", + "credential_ref": "credential-local-1", + "credential_version": 2, + "upstream_model_id": "example-model", + "parameters": {"max_output_tokens": 2048}, + "fallback_policy": "disabled", + "auth_mode": "api_key" +} diff --git a/tests/workbench/fixtures/runtime_event_cancelled.json b/tests/workbench/fixtures/runtime_event_cancelled.json new file mode 100644 index 0000000..77a8599 --- /dev/null +++ b/tests/workbench/fixtures/runtime_event_cancelled.json @@ -0,0 +1,13 @@ +{ + "schema": "RuntimeEventV1", + "schema_version": 1, + "event_id": "evt-9", + "seq": 9, + "run_id": "run-1", + "parent_run_id": null, + "turn_id": "turn-1", + "workspace_scope": "ws-1", + "timestamp": "2026-10-25T01:00:04Z", + "type": "run.cancelled", + "payload": {"reason": "user_stop", "usage_state": "partial"} +} diff --git a/tests/workbench/schemas/DocumentViewV1.schema.json b/tests/workbench/schemas/DocumentViewV1.schema.json new file mode 100644 index 0000000..f561c22 --- /dev/null +++ b/tests/workbench/schemas/DocumentViewV1.schema.json @@ -0,0 +1,38 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "marqdo://contracts/DocumentViewV1", + "title": "DocumentViewV1", + "type": "object", + "required": ["schema", "schema_version", "document", "execution", "completeness"], + "properties": { + "schema": { "const": "DocumentViewV1" }, + "schema_version": { "const": 1 }, + "document": { + "type": "object", + "required": ["id", "source_revision"], + "properties": { + "id": { "type": "string" }, + "title": { "type": "string" }, + "source_revision": { "type": "string" }, + "semantic_digest": { "type": "string" }, + "generation_id": { "type": ["string", "null"] }, + "presentation_token": { "type": "string" } + } + }, + "execution": { + "type": "object", + "required": ["state"], + "properties": { + "state": { "const": "not_run" }, + "run_id": { "type": ["string", "null"] } + } + }, + "completeness": { "enum": ["complete", "partial"] }, + "render": { "type": "object" }, + "outline": { "type": "array" }, + "nodes": { "type": "array" }, + "refs": { "type": "array" }, + "diagnostics": { "type": "array" } + }, + "additionalProperties": true +} diff --git a/tests/workbench/schemas/KnowledgeLocatorV1.schema.json b/tests/workbench/schemas/KnowledgeLocatorV1.schema.json new file mode 100644 index 0000000..16150d2 --- /dev/null +++ b/tests/workbench/schemas/KnowledgeLocatorV1.schema.json @@ -0,0 +1,29 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "marqdo://contracts/KnowledgeLocatorV1", + "title": "KnowledgeLocatorV1", + "type": "object", + "required": ["schema_version", "project_id", "document_id"], + "properties": { + "schema": { "const": "KnowledgeLocatorV1" }, + "schema_version": { "const": 1 }, + "project_id": { "type": "string" }, + "document_id": { "type": "string" }, + "source_revision": { "type": ["string", "null"] }, + "generation_id": { "type": ["string", "null"] }, + "artifact_id": { "type": ["string", "null"] }, + "node_id": { "type": ["string", "null"] }, + "relation": { "type": "string" }, + "resolution": { + "enum": [ + "resolved", + "stale", + "unresolved", + "ambiguous", + "version_unavailable", + "unsupported" + ] + } + }, + "additionalProperties": true +} diff --git a/tests/workbench/schemas/ModelCallUsageV1.schema.json b/tests/workbench/schemas/ModelCallUsageV1.schema.json new file mode 100644 index 0000000..fe9b76e --- /dev/null +++ b/tests/workbench/schemas/ModelCallUsageV1.schema.json @@ -0,0 +1,32 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "marqdo://contracts/ModelCallUsageV1", + "title": "ModelCallUsageV1", + "type": "object", + "required": [ + "schema_version", + "call_id", + "attempt_id", + "run_id", + "status", + "usage_state", + "measurement_source" + ], + "properties": { + "schema": { "const": "ModelCallUsageV1" }, + "schema_version": { "const": 1 }, + "call_id": { "type": "string", "minLength": 1 }, + "attempt_id": { "type": "string", "minLength": 1 }, + "run_id": { "type": "string", "minLength": 1 }, + "status": { + "enum": ["started", "succeeded", "failed", "cancelled", "interrupted"] + }, + "usage_state": { "enum": ["pending", "partial", "final", "unknown"] }, + "measurement_source": { "enum": ["provider", "estimated", "none"] }, + "input_tokens": { "type": ["integer", "null"], "minimum": 0 }, + "output_tokens": { "type": ["integer", "null"], "minimum": 0 }, + "total_tokens": { "type": ["integer", "null"], "minimum": 0 }, + "revision": { "type": "integer", "minimum": 1 } + }, + "additionalProperties": true +} diff --git a/tests/workbench/schemas/ModelSelectionV1.schema.json b/tests/workbench/schemas/ModelSelectionV1.schema.json new file mode 100644 index 0000000..594e072 --- /dev/null +++ b/tests/workbench/schemas/ModelSelectionV1.schema.json @@ -0,0 +1,32 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "marqdo://contracts/ModelSelectionV1", + "title": "ModelSelectionV1", + "type": "object", + "required": [ + "schema_version", + "selection_id", + "provider_id", + "model_id", + "adapter", + "upstream_model_id" + ], + "properties": { + "schema": { "const": "ModelSelectionV1" }, + "schema_version": { "const": 1 }, + "selection_id": { "type": "string", "minLength": 1 }, + "provider_id": { "type": "string", "minLength": 1 }, + "model_id": { "type": "string", "minLength": 1 }, + "adapter": { "type": "string" }, + "upstream_model_id": { "type": "string", "minLength": 1 }, + "provider_revision": { "type": ["integer", "null"] }, + "model_revision": { "type": ["integer", "null"] }, + "endpoint_ref": { "type": ["string", "null"] }, + "credential_ref": { "type": ["string", "null"] }, + "credential_version": { "type": ["integer", "null"] }, + "parameters": { "type": "object" }, + "fallback_policy": { "type": "string" }, + "auth_mode": { "type": "string" } + }, + "additionalProperties": true +} diff --git a/tests/workbench/schemas/RuntimeEventV1.schema.json b/tests/workbench/schemas/RuntimeEventV1.schema.json new file mode 100644 index 0000000..2bfd42f --- /dev/null +++ b/tests/workbench/schemas/RuntimeEventV1.schema.json @@ -0,0 +1,25 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "marqdo://contracts/RuntimeEventV1", + "title": "RuntimeEventV1", + "type": "object", + "required": [ + "schema_version", + "event_id", + "seq", + "run_id", + "type", + "payload" + ], + "properties": { + "schema": { "const": "RuntimeEventV1" }, + "schema_version": { "const": 1 }, + "event_id": { "type": "string", "minLength": 1 }, + "seq": { "type": "integer", "minimum": 0 }, + "run_id": { "type": "string", "minLength": 1 }, + "type": { "type": "string", "minLength": 1 }, + "payload": { "type": "object" }, + "timestamp": { "type": "string" } + }, + "additionalProperties": true +} diff --git a/tests/workbench_contracts.rs b/tests/workbench_contracts.rs new file mode 100644 index 0000000..566631e --- /dev/null +++ b/tests/workbench_contracts.rs @@ -0,0 +1,408 @@ +//! Workbench M00–M06 contract gate (offline). +//! Spec: doc/design/qdagent-workbench-runtime-and-view-2026-10-25.md + +use serde_json::Value; +use std::path::PathBuf; + +fn fixture(name: &str) -> Value { + let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let path = root.join("tests/workbench/fixtures").join(name); + let raw = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("read {}: {e}", path.display())); + serde_json::from_str(&raw).expect("fixture json") +} + +fn schema(name: &str) -> Value { + let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let path = root.join("tests/workbench/schemas").join(name); + let raw = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("read {}: {e}", path.display())); + serde_json::from_str(&raw).expect("schema json") +} + +/// Lightweight required-field check against our JSON Schema documents (no jsonschema crate). +fn assert_required(instance: &Value, schema_doc: &Value, label: &str) { + let required = schema_doc["required"].as_array().expect("schema.required"); + for key in required { + let k = key.as_str().unwrap(); + assert!( + instance.get(k).is_some(), + "{label}: missing required field {k} in {instance}" + ); + } +} + +fn mlsp(method: &str, params: Value) -> Value { + let req = serde_json::json!({"id": 1, "method": method, "params": params}); + let line = marqdo::mlsp::handle_line(&req.to_string()); + serde_json::from_str(&line).expect("mlsp response json") +} + +#[test] +fn m00_fixtures_consume_and_capabilities() { + let sel = fixture("model_selection_ok.json"); + assert_required(&sel, &schema("ModelSelectionV1.schema.json"), "selection"); + assert_eq!(sel["schema"], "ModelSelectionV1"); + let parsed = marqdo::workbench::parse_model_selection(&sel, true).expect("strict ok"); + assert_eq!(parsed.selection_id, "sel-1"); + let snap = parsed.public_snapshot(); + assert!(snap.get("api_key").is_none()); + + let usage = fixture("model_call_usage_unknown.json"); + assert_required(&usage, &schema("ModelCallUsageV1.schema.json"), "usage"); + assert!(usage["input_tokens"].is_null()); + assert_ne!(usage["input_tokens"], 0); + assert_eq!(usage["usage_state"], "unknown"); + + let cancelled = fixture("model_call_usage_cancelled.json"); + assert_eq!(cancelled["status"], "cancelled"); + assert_eq!(cancelled["usage_state"], "partial"); + + let doc = fixture("document_view_partial.json"); + assert_required(&doc, &schema("DocumentViewV1.schema.json"), "document_view"); + assert_eq!(doc["execution"]["state"], "not_run"); + + let stale = fixture("locator_stale.json"); + assert_eq!(stale["resolution"], "stale"); + let loc = &stale["locator"]; + assert_required(loc, &schema("KnowledgeLocatorV1.schema.json"), "locator"); + + let unavail = fixture("locator_version_unavailable.json"); + assert_eq!(unavail["resolution"], "version_unavailable"); + + let ev = fixture("runtime_event_cancelled.json"); + assert_required(&ev, &schema("RuntimeEventV1.schema.json"), "runtime_event"); + assert_eq!(ev["type"], "run.cancelled"); + + let _ = fixture("model_selection_error.json"); + let _ = fixture("document_view_unsupported.json"); + + let caps = mlsp("capabilities", serde_json::json!({})); + assert_eq!(caps["ok"], true, "{caps}"); + let features = &caps["result"]["features"]; + assert_eq!(features["view.document"], 1); + assert_eq!(features["view.reference_resolve"], 1); + assert!(caps["result"]["contracts"] + .as_array() + .unwrap() + .iter() + .any(|c| c.as_str() == Some("DocumentViewV1"))); + + // F16 honesty: unloaded plugins + forced-off stream/cancel → 0 + let down = mlsp( + "capabilities", + serde_json::json!({ + "force_unloaded": true, + "force_stream_off": true, + "force_cancel_off": true + }), + ); + assert_eq!(down["result"]["features"]["llm.messages"], 0); + assert_eq!(down["result"]["features"]["runtime.event_stream"], 0); + assert_eq!(down["result"]["features"]["runtime.cancel"], 0); +} + +#[test] +fn m00_strict_selection_rejects_missing() { + let bad = serde_json::json!({"schema_version": 1, "model_id": "x"}); + let err = marqdo::workbench::parse_model_selection(&bad, true).unwrap_err(); + assert!(err.contains("selection_missing"), "{err}"); +} + +#[test] +fn m02_usage_null_not_zero() { + let usage = marqdo::workbench::model_call_usage_from_provider( + "c1", + "a1", + "r1", + &serde_json::json!({"selection_id": "s", "provider_id": "p", "model_id": "m"}), + &serde_json::Value::Null, + "succeeded", + ); + assert!(usage["input_tokens"].is_null()); + assert_eq!(usage["measurement_source"], "none"); + assert_eq!(usage["usage_state"], "unknown"); +} + +#[test] +fn m03_view_document_no_exec() { + let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests/executable_knowledge/baseline/02-extract"); + let resp = mlsp( + "view.document", + serde_json::json!({ + "project": root.to_string_lossy(), + "document": "main.mq.md", + "mode": "read", + "include": ["fragment", "outline", "nodes", "diagnostics"] + }), + ); + assert_eq!(resp["ok"], true, "{resp}"); + let doc = &resp["result"]; + assert_eq!(doc["schema"], "DocumentViewV1"); + assert_eq!(doc["execution"]["state"], "not_run"); + assert_eq!(doc["capabilities"]["read_only"], true); + assert_eq!(doc["capabilities"]["executed"], false); + let fragment = doc["render"]["fragment"].as_str().unwrap_or(""); + assert!(fragment.contains("mq-document"), "{fragment}"); + assert!(!fragment.contains("