diff --git a/.cursor/skills/marqdo/SKILL.md b/.cursor/skills/marqdo/SKILL.md index ad2def4..845fb4c 100644 --- a/.cursor/skills/marqdo/SKILL.md +++ b/.cursor/skills/marqdo/SKILL.md @@ -14,68 +14,49 @@ description: >- Marqdo = **Markdown markers as syntax**. A `.mq.md` file is both documentation and a runnable program. Read this skill **before** inventing syntax from Markdown habits or Python. -Canonical design (repo): `doc/design/markdown-mapping.md`, `doc/design/keywords.md`. Deeper tables: [reference.md](reference.md). Copy-paste patterns: [examples.md](examples.md). + -**Authoring / library craft** (GFM-first, no json glue, how to develop Marqdo): use **[marqdo-dev](../marqdo-dev/SKILL.md)** whenever writing or refactoring `.mq.md`, `lib/*`, or `ext/*`. +## CORE SYNTAX (v2.1 Occlusion — mandatory) -**Product release** (tag, CHANGELOG, README, public docs, VSIX, GitHub assets): use [marqdo-release](../marqdo-release/SKILL.md) — detect latest version, **ask the user for the next SemVer**, then follow that playbook. Do not invent a version number. +These rules are the **current default**. Do **not** teach Classic `[key](map)` index or “bold is return of print”. Classic migration only: [migration-classic.md](migration-classic.md). + +1. **File suffix** `.mq.md`. Print with `> print text=…` / `> 打印 内容=…` — never use `**…**` to print. +2. **`**…**` = code / return expression.** `*…*` = statement (bind / expr). Closing markers touch the last token (no trailing space). +3. **`[描述](payload)` = occlusion:** `[]` is prose (never executed); `()` is the only executed part. Empty `()` → `None` stub. +4. **Read collections with `access(coll, key)`** (aliases `取` / `取值`). **Never** use `[key](map)` as index under default Occlusion. +5. **Foreach:** `- [描述](遍历 coll 为 item)` (`遍历`/`for`, `为`/`as`/`in`). Prefer explicit traversal when generating new code. +6. **Import / call:** frontmatter `import:` / `导入:`; call `lib.member`. One language (EN or ZH) per file. +7. **Control flow:** `#` object/type, `##`+ function, `+` params, `1.`/`2.` branches, `-` loops, `N. *` else. No invented `if`/`for`/`def`/`return` keywords. +8. **Prose vs code:** blank line separates comment paragraphs from executable lines. +9. **GFM tables** for data; prefer `table.put` over `json.set` chains. `ext/**` never calls `host_*`. +10. Body narrative `[描述](target)` is lazy (NarrativeRef) — reading does not execute side effects. + +| Marker | Meaning (v2.1) | +|--------|----------------| +| `#` / `##`+ | Object/type · function/method | +| `*…*` | Statement | +| `**…**` | Return / code expression | +| `[描述](payload)` | Occlusion step (payload executes) | +| `access(c, k)` | Collection read | +| `> fn …` | Call | +| GFM table | Collection literal | + +Canonical: `doc/design/core-surface.md`, `doc/design/document-native-development-guide.md`. Deeper tables: [reference.md](reference.md). Patterns: [examples.md](examples.md). Authoring craft: [marqdo-dev](../marqdo-dev/SKILL.md). Release: [marqdo-release](../marqdo-release/SKILL.md). + +Executable-knowledge gate: [`tests/executable_knowledge/`](../../tests/executable_knowledge/README.md). ## Hard rules (do not violate) -1. **File suffix** must be `.mq.md` (never plain `.md` for executable sources). -2. **Output is not `**bold**`.** Print with `> print text=…` (or `> 打印 内容=…`). Bold `**…**` is **return value only**. -3. **Prose vs code:** after a blank line, an unmarked first line starts a **comment paragraph**; every following non-blank line stays comment until the next blank line. Put a **blank line** between narration and executable lines. -4. **Do not invent keywords** `if` / `else` / `while` / `for` / `def` / `return`. Control flow is Markdown: **`+` params** (under heading), **`1.` `2.` … branches**, **`-` loops**, arm `N. *` = else, `#` = **object/type**, `##`+ = **function/method**, `**…**` = return. -5. **Identifiers use backticks** — params `` + `name` ``. **Exemptions:** foreach `` - [item](coll) ``; footnote `` name[^key] ``; **inside italic `*…*` / bold `**…**`**, bare ids are **variables** (Python-style unified namespace) and method-call receivers are bare too. **Drop the backticks on value-expression variable names and method receivers inside `*…*` / `**…**`** — they are redundant once the `*…*`/`**…**` markers mark the segment as code: `*分类 = 分类[^苹果]*`, `**n * 2**`, `*p = > page.主体装配 组件=home*`. **Text literals must be quoted**: `*a = > text.split value=src sep=","*`. Backticks **still required** where a bare word is *not* a variable in **standalone `>` calls** (bare = text there): `` > str `n` ``, `` > `obj`.method ``. **No trailing space** inside `*…*` / `**…**` wrapped code: the closing marker must touch the last token directly (`*a = 1*`, `**n**`), never `` *a = 1 * ``. -6. **Structure lines are not wrapped in italics.** `#` `>` `+` `-` `|` lines stand alone; use `*…*` only for general statements (bindings / expressions). -7. **Paths:** In bare *expressions* `/` is division. In **call args / param defaults / table cells**, unspaced `a/b` and quoted `".marqdo/agent-kb"` are path text — no `json.parse` needed. Numeric ratios that must stay text use quotes (`"1/5"`, `"16/9"`); bare `1/5` is division. -8. Prefer ending side-effect-only function bodies with `*None*` / `*无*` / whole-line `**` / `****` when later lines must belong to an outer function. Lone `---` / `***` are Markdown thematic breaks (skipped), not function end. -9. **`ext/**` never calls `host_*`.** Agent/OKF helpers are plugin names (`agent_kb_*`, …) after `plugin.load`. Do **not** add agent/OKF domain code to `src/host/` (core bloat). See `doc/design/ext-agent.md` §4. -10. **Browser (route C/D/E/F):** client logic is Marqdo on WASM; official bridge is host glue and **may** implement lists, routing, storage, WebSocket, canvas, file read, observers — authors **must not hand-write business JS**. Use `web.client_embed` / `data-mq-source-url` auto-mount. Client effects: prefer **GFM tables + `lib/browser`** ([marqdo-dev](../marqdo-dev/SKILL.md)). -11. **Code-as-documentation / no bag glue:** Prefer **GFM tables** for maps, lists, wire, commands. Prefer **`table.put` / named helpers** over `json.set` / `json.append` chains. `lib/json` is for parse/stringify/quote only — not a dict builder. Unreadable json pipelines are a style bug. Read collections with **`[key](map)`**, not footnote indexes for new code. In `markup: occlusion` files read with **`access(map, key)`** — there `[x](y)` is occlusion, not index. **Positive copy-paste shapes** (result map, soft-error pack, sparse `table.put`): [marqdo-dev](../marqdo-dev/SKILL.md) §2 · [examples.md](examples.md) §6e. Do **not** extend legacy `json.set` in `ext/ai` — refactor to tables. -12. **One language per file:** Do not mix EN and ZH API names in the same `.mq.md` (imports, calls, params). Pick English *or* Chinese mirrors for the whole program. -13. **Sandbox trial habit (Agent / generated `.mq.md`):** Prefer `agent.trial` / `run sandbox=True` / `sandbox.open`→`run`→`diff`→`commit` before promoting to the host tree. Host files must stay unchanged until a successful commit. Caps live in `sandbox.profile` GFM tables — do not invent a second dialect. +1. Obey **CORE SYNTAX** above — Occlusion is default; do not emit Classic index links. +2. **Identifiers:** params `` + `name` ``; inside `*…*` / `**…**` bare ids are variables. Quote text literals. Standalone `>` calls: backticks where bare would be text. +3. **Structure lines** (`#` `>` `+` `-` `|`) are not wrapped in italics. +4. **Paths:** bare `/` in expressions is division; in call args / table cells unspaced `a/b` is path text. Ratios as text need quotes (`"1/5"`). +5. End side-effect-only helpers with `*None*` / `*空*` / `**` / `****` when the outer function continues. Lone `---` is a thematic break, not function end. +6. **`ext/**` never calls `host_*`.** Prefer tables over `json.set` / `json.append`. +7. **One language per file.** Sandbox trial before promoting Agent-generated trees. +8. Browser: Marqdo on WASM + `lib/browser` tables; do not hand-write business JS. -## Markup → meaning (v0.2) - -| Marker | Meaning | -|--------|---------| -| `#` | **Object / type** (constructor body). `# main` = entry object. `# Child = > Parent` = inherit (**no** implicit parent constructor; call `` `self` = > Parent … `` explicitly when needed). | -| `##` … `######` | **Function / method** (nesting by heading depth). | -| `` + `name` `` under heading | Parameter (optional `` `name`=default ``) | -| `1.` `2.` … in body | Branch arm (condition or `N. *` else). **Same-indent restart at `1.` = new branch statement** (not more arms of the previous list). | -| `- …` inside body | Loop (`while` or `` [item](coll) `` / `` [`item`](`coll`) ``) | -| Line `N. *` | Else arm | -| `> fn args` | Call (named `k=v` or positional) | -| `` > `obj`.method args `` | Method call (`obj` must be a map with `_type`) | -| Frontmatter nested `import:` / `导入:` map | Import file (`bind: path.mq.md`) or short name (`bind: lib.member`) | -| `> lib.member …` / `> lib.Type.member …` | Call via bare dotted path (instance methods need `` `var`.m ``) | -| `*…*` | Statement (bind / expr) — closing `*` touches last token, **no trailing space**; bare ids are variables | -| `**…**` | Return value — closing `**` touches last token, **no trailing space**; bare ids are variables | -| `****` or `**` + spaces + `**` | Return `None` and end function body | -| Lone `---` / `***` | Markdown thematic break (skip; not function end) | -| GFM table after empty RHS bind | Collection; get with `access(coll, "key")` (legacy `` `m`[^key] `` remains available) | -| `- [item](遍历 coll 为 item)` | Foreach | -| `` ```lang `` | Foreign code block (via `lib/foreign`) | -| `` `"text"` `` | Quoted string (`\n` `\t` `\\` `\"`; `` `var` `` inside); bare tokens unescaped | -| Unmarked prose | Comment | - -Entry: load file (often `index.mq.md`) → collect defs → run `# main`. - -## Occlusion syntax (v2.1.0+) · Executable knowledge (v2.1.1) - -Occlusion is the default syntax. Shipped source declares `markup: occlusion` (or `标记: 遮蔽`) explicitly so imports stay deterministic. Legacy `[key](collection)` requires `markup: classic` or the one-cycle CLI flag `--legacy-markup`; `MARQDO_MARKUP` does not change parsing. - -Executable-knowledge **S0 author baseline** (six tasks, offline gate): [`tests/executable_knowledge/`](../../../tests/executable_knowledge/README.md). Design goals ≠ shipped knowledge reuse — see [`doc/design/document-native-development-guide.md`](../../../doc/design/document-native-development-guide.md). - -- `[描述](实现)` — `[]` is prose (never executed); `()` is the **only** executed part. -- `[顾客留存率](0.42)` · `[总价](access(价目表, "苹果") * 数量)` · `[功能开发中]()` → `None` (stub; `label.empty_payload` warning). -- Read maps/lists with `access(集合, 键)` (aliases `取` / `取值`): `[第二项](access(列, 2))`. -- Foreach: `- [篮子里的每个果子](遍历 篮子 为 果)` (accepts `遍历`/`for` and `为`/`as`/`in`). -- Calls inside `()`: `加(1)` / `加(1, 2)` / `加一 号(31)` / `txt.str_trim(" x ")` — `)` terminates, commas optional. -- Tooling: `marqdo labels pending` (stub ledger) · `marqdo labels --grep 词 --count` · `marqdo migrate labels . --mark --write` · `marqdo check` reports `label.missing` / `label.empty_payload` / `label.ref_dangling`. - ## Minimal program ```markdown diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c897d6b..dede631 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -126,9 +126,11 @@ jobs: ## Marqdo ${{ github.ref_name }} ### 亮点 / Highlights + - **v2.1.2**:可执行知识生产链——证据/闭包 · verify/adjudicate · reuse_execute/bind_execute · document.extract · View KnowledgePresentation + SSE page_token · 上传限流 + - **v2.1.2**: Executable-knowledge production chain — evidence/closure, verify/adjudicate, reuse_execute/bind_execute, document.extract, View KnowledgePresentation + SSE page_token, upload caps + - **v2.1.2 ext**:扩展包同步 2.1.2 / Ext pack 2.1.2 - **v2.1.1**:可执行知识主链——BoundCall · NarrativeRef · 来源/适用期 · patch/extract · 整代 CURRENT + closure · 发票示例 113 - **v2.1.1**: Executable-knowledge spine — BoundCall, NarrativeRef, sources, patch/extract, generation CURRENT + closure, invoice example 113 - - **v2.1.1 ext**:扩展包同步 2.1.1 / Ext pack 2.1.1 - **v1.5.2**:外联 v3 P0/P1——raw stdin + 每 cell 输入 · plan API · snapshot · `view capture` · Persist - **v1.5.2**: Foreign v3 P0/P1 — raw stdin + per-cell inputs, plan API, snapshot export/import/replay, `view capture`, Persist - **v1.5.2 ext**:扩展包同步 1.5.2 / Ext pack 1.5.2 diff --git a/CHANGELOG.md b/CHANGELOG.md index 02b76e8..df10674 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -75,6 +75,41 @@ marqdo ext add web ### Changed +## v2.1.2 — 2026-10-24 + +### Highlights + +**可执行知识生产消费链 + View 阅读体验发版门槛**:证据摘要进闭包/句柄;`artifact.verify` / `knowledge adjudicate` 结构化裁决;`artifact.reuse_execute` 与 `agent_eng_bind_execute` / `publish_gate` 实际复用(非 n-gram);`document.extract` 文本/Markdown + PDF/OCR 显式失败;View `KnowledgePresentation`、说明锚点、SSE `page_token` 过期包拒绝;上传 `max_bytes`;门禁 45 项(含 100/1000 单元)。CLI · wasm · ext **2.1.2**。 + +```bash +git checkout v2.1.2 && cargo build --release +marqdo ext add web +cargo test --offline --locked --test executable_knowledge +``` + +**核心表面 Δ(Core surface Δ)**:**无增删关键词**——语义/MLSP/View/Agent 宿主扩展;不改变默认 Occlusion 链接读法与表几何。 + +### Added + +- **证据与闭包(F0/M02)**:来源 digests 进入 artifact/evidence/closure;business handle 改/删来源失效;锚点/可读性诊断。 +- **验证裁决(F0/M03)**:`artifact.verify` 结构化 `may_bind` / `may_publish`;policy 枚举;`knowledge adjudicate`;CLI `--fail`。 +- **语法 Skill(F0/M05)**:CORE SYNTAX 置顶;`migration-classic.md`;`skill_core_card` 与预算截断防护。 +- **绑定执行(F1/M04)**:`artifact.reuse_execute`;`agent_eng_bind_execute` + `eng_publish_gate`(`candidate_only` / product gate);host_query `mlsp`;`tests/ext/agent-eng-bind-gate.mq.md`。 +- **发布与来源(F2/M06–M07)**:统一 `EXCLUDED_DIRS`;`publish_manifest.json`;`function_sources` 函数级来源覆盖。 +- **文档提取(F3/M01)**:`document.extract` DocumentSource→ExtractedDocument;PDF/DOCX/OCR 显式 `failed`;超限截断 `partial`;Web 上传 `MaxBytesReader` / `LimitReader`。 +- **View V0–V3(子集)**:正文优先;no-exec;`KnowledgePresentation`(`semantic_generation` / `page_token` / `local_deps`);SSE 过期包拒绝与帧选择保留;print/响应式;1000 单元 compile smoke。 +- **故事门禁(M08)**:建库→复用→来源变更失效→修复重绑。 + +### Fixed + +- View 说明链接 raw-string 截断;skill_brief / CORE SYNTAX 预算不被冲突旧规则吃掉。 +- 上传体积限制在 multipart 解析前生效(不再只靠内存 form 预算)。 + +### Changed + +- CLI / wasm / `ext/VERSION` **同步 2.1.2**。 +- `tests/executable_knowledge` 门禁扩至 45 项;README 切片进度表与发版子集对齐。 + ## v2.1.1 — 2026-10-23 ### Highlights diff --git a/Cargo.lock b/Cargo.lock index 5aef90f..973ee14 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad" [[package]] name = "marqdo" -version = "2.1.1" +version = "2.1.2" dependencies = [ "anyhow", "assert_cmd", @@ -610,7 +610,7 @@ dependencies = [ [[package]] name = "marqdo-wasm" -version = "2.1.1" +version = "2.1.2" dependencies = [ "marqdo", "serde_json", diff --git a/Cargo.toml b/Cargo.toml index ff59bc2..d7e8ae0 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "marqdo" -version = "2.1.1" +version = "2.1.2" edition = "2021" license = "Apache-2.0" description = "Marqdo — Markdown markers as an interpreted programming language" diff --git a/README.md b/README.md index c43fbf0..8b0b182 100644 --- a/README.md +++ b/README.md @@ -281,9 +281,10 @@ marqdo catalog public -o .marqdo --- -## 现状(v2.1.1) +## 现状(v2.1.2) -- **可执行知识主链(v2.1.1)**:BoundCall 双后端 · NarrativeRef · 来源/适用期 · `artifact.patch` / `extract_pure` · 整代 `CURRENT` + closure · 发票示例 `113` · 门禁 `tests/executable_knowledge`(含 100 单元)· 扩展包同步 **2.1.1** +- **可执行知识生产链(v2.1.2)**:证据/闭包失效 · `verify`/`adjudicate` · `reuse_execute` / `eng_bind_execute`+`publish_gate` · `document.extract` · View `KnowledgePresentation`+SSE `page_token` · 上传限流 · 门禁 45 项(100/1000 单元)· 扩展包同步 **2.1.2** +- **可执行知识主链(v2.1.1)**:BoundCall 双后端 · NarrativeRef · 来源/适用期 · `artifact.patch` / `extract_pure` · 整代 `CURRENT` + closure · 发票示例 `113` - **链接语法(v2.1.0)**:`[描述](目标)` 描述不执行 · `()` 与 `**代码**` 同权(`加(1)` / `加一 号(31)`)· `access` 取元与 `遍历` 链接循环 · `marqdo labels` 两级检索 / `--baseline` 债务棘轮 · `migrate labels --mark` · 诊断 `label.*` · view 描述正文 + 代码片 + 桩徽章 · **可执行知识 [link-syntax.md](doc/design/link-syntax.md)**(项目索引、LinkInfo、链接图与 lock) - **外联 v3 P0/P1(v1.5.2)**:raw stdin + 每 cell 独立输入 · plan API · snapshot export/import/replay · `view capture` · Persist 按钮 · 扩展包同步 **1.5.2** - **view 静态/动态统一 + 外联值传递(v1.5.1)**:静态导出与 live view 共用 `foreign_doc` 执行路径 · 构建时运行注入 stdout · `foreign.session` 值传递 · cell stdin;扩展包同步 **1.5.0** @@ -320,7 +321,7 @@ marqdo catalog public -o .marqdo - **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md)) - 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md) -### 如何使用最新 Marqdo(v2.1.1) +### 如何使用最新 Marqdo(v2.1.2) **安装解释器(任选其一)** @@ -335,7 +336,7 @@ marqdo catalog public -o .marqdo ```bash # 1) 源码安装(跟 tag 或 main) git clone https://github.com/cflmy/marqdo.git && cd marqdo -git checkout v2.1.1 # 或留在 main +git checkout v2.1.2 # 或留在 main cargo build --release export PATH="$PWD/target/release:$PATH" marqdo version @@ -374,7 +375,7 @@ marqdo wasm build # → dist/wasm/ … 见 examples/browser-hello/ # 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip -# https://github.com/cflmy/marqdo/releases/tag/v2.1.1 +# https://github.com/cflmy/marqdo/releases/tag/v2.1.2 ``` 开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`: diff --git a/crates/marqdo-wasm/Cargo.toml b/crates/marqdo-wasm/Cargo.toml index ccdca74..7880e09 100644 --- a/crates/marqdo-wasm/Cargo.toml +++ b/crates/marqdo-wasm/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "marqdo-wasm" -version = "2.1.1" +version = "2.1.2" edition = "2021" license = "Apache-2.0" description = "Marqdo interpreter as WebAssembly (browser host bridge)" diff --git a/debian/changelog b/debian/changelog index c43e3a2..70f51e1 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,11 @@ +marqdo (2.1.2-1ppa1) noble; urgency=medium + + * Marqdo 2.1.2: executable-knowledge F0–F3 + View V0–V3 release bar + (evidence/closure, verify gates, bind/reuse_execute, document.extract, + KnowledgePresentation, SSE page_token, upload max_bytes). + + -- cflmy Sat, 24 Oct 2026 14:50:00 +0000 + marqdo (2.1.1-1ppa1) noble; urgency=medium * Marqdo 2.1.1: executable-knowledge spine (BoundCall, NarrativeRef, diff --git a/ext/VERSION b/ext/VERSION index 3e3c2f1..eca07e4 100644 --- a/ext/VERSION +++ b/ext/VERSION @@ -1 +1 @@ -2.1.1 +2.1.2 diff --git a/ext/ai/agent.mq.md b/ext/ai/agent.mq.md index cfcab6c..5680726 100644 --- a/ext/ai/agent.mq.md +++ b/ext/ai/agent.mq.md @@ -849,25 +849,44 @@ Write a scratch tool workbook under `.marqdo/agent-runs/tools/.mq.md`. --- +## skill_core_card + +Protected v2.1 Occlusion card always prepended to skill_brief (M05). Must not be truncated by budget. + +**esc = > json.parse text={"card":"## CORE SYNTAX (v2.1 Occlusion)\n1. Print with > print / > 打印 — never use bold markers to print.\n2. Bold markers = return/code; italic markers = statement.\n3. [desc](payload) = occlusion; brackets never execute.\n4. Read with access(coll, key) — never Classic [key](map) index.\n5. Foreach: - [desc](for coll as item). import + one language per file.\n6. # type · ## function · + params · numbered branches · - loops.\n"}** +*> json.get value=`esc` key="card"* + +--- + ## skill_brief + `max_chars`=1200 + `depth`=default -Budgeted marqdo skill text for prompts. `depth=deep` raises the char budget. +Budgeted marqdo skill text for prompts. Core Occlusion card is always included; remaining budget truncates the rest. `depth=deep` raises the char budget. 1. `depth` == deep **budget = 4000** 2. * **budget = max_chars** +**core = > skill_core_card** +**core_len = > len value=`core`** +**rest_budget = budget - core_len - 2** +1. `rest_budget` < 200 + **rest_budget = 200** +2. * + **_ = 1** **skill = > agent_marqdo_skill** **full = > len value=`skill`** -**ex = > text_head_lines text=`skill` max_lines=80 max_chars=`budget`** +**rest = > text_head_lines text=`skill` max_lines=80 max_chars=`rest_budget`** +**esc = > json.parse text={"nl":"\n\n"}** +**nl = > json.get value=`esc` key="nl"** +**ex = core + nl + rest** **used = > len value=`ex`** -1. `used` < `full` - **esc = > json.parse text={"a":"\n…[truncated skill_len=","b":" budget_chars=","c":"; emit READ:skill to deepen]"}** - **a = > json.get value=`esc` key="a"** - **b = > json.get value=`esc` key="b"** - **c = > json.get value=`esc` key="c"** +1. `used` < `full` + `core_len` + **esc2 = > json.parse text={"a":"\n…[truncated skill_len=","b":" budget_chars=","c":"; emit READ:skill to deepen]"}** + **a = > json.get value=`esc2` key="a"** + **b = > json.get value=`esc2` key="b"** + **c = > json.get value=`esc2` key="c"** *ex + a + full + b + budget + c* 2. * *ex* @@ -1973,18 +1992,27 @@ O5: mark OKF skill `status: quarantine` and force `llm_free: false`. Regression + `memory_dir`=.marqdo/agent-memory + `improve_every`=3 + `promote`=True + + `candidate_only`=False + + `product_gate_ok`=True Offline learn gate: when ≥ `improve_every` successful episodes share a task hash, generalize → synthesize → compile → optional OKF promote (`llm_free`). Also attempts Policy Compilation for Adaptive Routing. O2: promote path requires sandbox proof unless `require_proof=false`. +M04: `candidate_only=True` or `product_gate_ok=False` forces promote off via eng_publish_gate. +**gate = > `self`.eng_publish_gate project="." candidate_only=`candidate_only` product_gate_ok=`product_gate_ok`** +**may_prom = > json.get value=`gate` key="may_promote"** +1. not `may_prom` + **promote = False** +2. * + **_ = 1** **skill_learn = > agent_maybe_learn task=`task` kb_dir=`kb_dir` memory_dir=`memory_dir` improve_every=`improve_every` promote=`promote`** **policy_learn = > agent_maybe_compile_policy memory_dir=`memory_dir` min_evidence=`improve_every`** **sl = > json.get value=`skill_learn` key="learned"** `out` = -| learned | skill | policy | -|---------|-------|--------| -| `sl` | `skill_learn` | `policy_learn` | +| learned | skill | policy | promote_allowed | gate | +|---------|-------|--------|-----------------|------| +| `sl` | `skill_learn` | `policy_learn` | `may_prom` | `gate` | *out* @@ -2128,7 +2156,18 @@ Primary author entry (v2 + P4): Adaptive `route` → execute (policy/kb skill) o 1. `efi_st` == "missing_knowledge" **_ = 1** 2. `efi_dec` == "REUSE" - **_ = 1** + **efi_exec = > agent_eng_bind_execute project="." query=`task` policy="pure" candidate_only=False** + **efi_ok = > json.get value=`efi_exec` key="ok"** + 1. `efi_ok` + **exec_mode = "efi_bound"** + **out = ** + + | status | goal | result | summary | efi | reuse_trace | + |--------|------|--------|---------|-----|-------------| + | ok | `task` | `efi_exec` | EFI REUSE bound+executed | `efi_pf` | `efi_exec` | + + 2. * + **_ = 1** 3. `efi_dec` == "ADAPT" **_ = 1** 4. `efi_allow` @@ -2433,13 +2472,42 @@ Hard rule: generate only when `create_allowed` is true (or decision is REUSE/ADA + `root`="." + `marqdo_dir`=None -Alias of `preflight` focused on REUSE / ADAPT / CREATE payload. +Alias of `preflight` focused on REUSE / ADAPT / CREATE payload (discovery_only). 1. `marqdo_dir` *> agent_eng_reuse task=`task` root=`root` marqdo_dir=`marqdo_dir`* 2. * *> agent_eng_reuse task=`task` root=`root`* +--- + +## eng_bind_execute + + `project`="." + + `query` + + `policy`="pure" + + `inputs`=None + + `candidate_only`=False + +M04: discover → bind → execute → record. Prefer over treating eng_reuse scores as execution. + +1. `inputs` + *> agent_eng_bind_execute project=`project` query=`query` policy=`policy` inputs=`inputs` candidate_only=`candidate_only`* +2. * + *> agent_eng_bind_execute project=`project` query=`query` policy=`policy` candidate_only=`candidate_only`* + +--- + +## eng_publish_gate + + `project`="." + + `artifact_id`=None + + `candidate_only`=False + + `product_gate_ok`=True + + `policy`="business" + +Refuse promote when candidate_only or product/verification gate fails. + +*> agent_eng_publish_gate project=`project` artifact_id=`artifact_id` candidate_only=`candidate_only` product_gate_ok=`product_gate_ok` policy=`policy`* + ## eng_record + `decision` + `root`="." diff --git a/plugins/agent/src/eng.rs b/plugins/agent/src/eng.rs index e05c210..40dfa65 100644 --- a/plugins/agent/src/eng.rs +++ b/plugins/agent/src/eng.rs @@ -501,6 +501,175 @@ pub fn eng_reuse(args: &Value) -> Result { Ok(pf) } +/// M04: discover → inspect → bind → execute → record (actual reuse, not n-gram REUSE). +/// +/// Requires host_query `mlsp`. Returns trace with artifact_id, digests, inputs, result. +/// Does **not** promote skills — callers pass `candidate_only=true` to skip learn/promote. +pub fn eng_bind_execute(args: &Value) -> Result { + let project = args + .get("project") + .or_else(|| args.get("root")) + .and_then(|v| v.as_str()) + .unwrap_or("."); + let policy = args + .get("policy") + .and_then(|v| v.as_str()) + .unwrap_or("pure"); + let inputs = args.get("inputs").cloned().unwrap_or(json!({})); + let query = if let Some(q) = args.get("artifact_id").or_else(|| args.get("query")).and_then(|v| v.as_str()) { + q.to_string() + } else { + // Fall back to first bind_target from eng_reuse. + let reuse = eng_reuse(args)?; + let targets = reuse + .get("bind_targets") + .and_then(|v| v.as_array()) + .cloned() + .unwrap_or_default(); + targets + .first() + .and_then(|t| t.get("query").and_then(|q| q.as_str())) + .ok_or_else(|| { + "eng_bind_execute: no artifact_id/query and eng_reuse returned no bind_targets" + .to_string() + })? + .to_string() + }; + + let inspect = crate::host_query_json_args( + "mlsp", + &json!({"method":"artifact.inspect","params":{"project":project,"artifact_id":&query}}), + )?; + if inspect.get("ok") != Some(&json!(true)) { + return Ok(json!({ + "ok": false, + "stage": "inspect", + "error": inspect.get("error").cloned().unwrap_or(json!("inspect failed")), + "query": query, + })); + } + let artifact_id = inspect["result"]["artifact_id"] + .as_str() + .unwrap_or(&query) + .to_string(); + + let bound = crate::host_query_json_args( + "mlsp", + &json!({"method":"artifact.bind","params":{"project":project,"artifact_id":&artifact_id,"policy":policy}}), + )?; + if bound.get("ok") != Some(&json!(true)) { + return Ok(json!({ + "ok": false, + "stage": "bind", + "error": bound.get("error").cloned().unwrap_or(json!("bind failed")), + "artifact_id": artifact_id, + "discovery_only": false, + })); + } + let handle = bound["result"]["handle"].clone(); + + let executed = crate::host_query_json_args( + "mlsp", + &json!({"method":"artifact.execute","params":{"handle":&handle,"inputs":inputs}}), + )?; + if executed.get("ok") != Some(&json!(true)) { + return Ok(json!({ + "ok": false, + "stage": "execute", + "error": executed.get("error").cloned().unwrap_or(json!("execute failed")), + "artifact_id": artifact_id, + "handle": handle, + })); + } + + // Record only after successful bind+execute (actual reuse). + let _ = eng_record(&json!({ + "decision": "REUSE", + "root": project, + "marqdo_dir": args.get("marqdo_dir"), + })); + + Ok(json!({ + "ok": true, + "stage": "executed", + "discovery_only": false, + "bind_required": false, + "actual_reuse": true, + "artifact_id": artifact_id, + "symbol_id": handle.get("symbol_id"), + "digest": handle.get("digest"), + "evidence_digest": handle.get("evidence_digest"), + "closure_digest": handle.get("closure_digest"), + "policy": policy, + "inputs": inputs, + "result": executed.get("result"), + "candidate_only": args.get("candidate_only").and_then(|v| v.as_bool()).unwrap_or(false), + "trace": { + "artifact_id": artifact_id, + "symbol_id": handle.get("symbol_id"), + "digest": handle.get("digest"), + "inputs": inputs, + "stdout": executed["result"]["stdout"], + "value": executed["result"]["result"], + }, + })) +} + +/// Gate promote/learn: when `candidate_only` or may_bind=false, refuse promotion. +pub fn eng_publish_gate(args: &Value) -> Result { + let candidate_only = matches!(args.get("candidate_only"), Some(Value::Bool(true))); + let product_ok = !matches!(args.get("product_gate_ok"), Some(Value::Bool(false))); + let project = args + .get("project") + .or_else(|| args.get("root")) + .and_then(|v| v.as_str()) + .unwrap_or("."); + let policy = args + .get("policy") + .and_then(|v| v.as_str()) + .unwrap_or("business"); + let artifact = args + .get("artifact_id") + .or_else(|| args.get("query")) + .and_then(|v| v.as_str()); + + if candidate_only { + return Ok(json!({ + "ok": true, + "may_promote": false, + "reason": "candidate_only — framework must not promote", + })); + } + if !product_ok { + return Ok(json!({ + "ok": true, + "may_promote": false, + "reason": "product gate failed — framework must not promote", + })); + } + if let Some(id) = artifact { + let adj = crate::host_query_json_args( + "mlsp", + &json!({"method":"artifact.verify","params":{"project":project,"artifact_id":id,"policy":policy}}), + )?; + let may_bind = adj + .pointer("/result/may_bind") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + return Ok(json!({ + "ok": true, + "may_promote": may_bind, + "verification": adj.get("result"), + "reason": if may_bind { "verification allows bind" } else { "verification denied bind" }, + })); + } + Ok(json!({ + "ok": true, + "may_promote": product_ok, + "reason": "no artifact_id; deferred to product_gate_ok", + })) +} + /// `agent_eng_record` — append reuse metrics under `.marqdo/agent/episodes`. pub fn eng_record(args: &Value) -> Result { let decision = arg_text(args, "decision")?; diff --git a/plugins/agent/src/lib.rs b/plugins/agent/src/lib.rs index bce6248..36fc4e3 100644 --- a/plugins/agent/src/lib.rs +++ b/plugins/agent/src/lib.rs @@ -831,6 +831,8 @@ kb_ffi!(agent_resume_clear, resume::resume_clear, "agent_resume_clear"); kb_ffi!(agent_resume_list, resume::resume_list, "agent_resume_list"); kb_ffi!(agent_eng_preflight, eng::eng_preflight, "agent_eng_preflight"); kb_ffi!(agent_eng_reuse, eng::eng_reuse, "agent_eng_reuse"); +kb_ffi!(agent_eng_bind_execute, eng::eng_bind_execute, "agent_eng_bind_execute"); +kb_ffi!(agent_eng_publish_gate, eng::eng_publish_gate, "agent_eng_publish_gate"); kb_ffi!(agent_eng_record, eng::eng_record, "agent_eng_record"); #[no_mangle] @@ -900,6 +902,8 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int ("agent_maybe_compile_policy", "memory_dir,min_evidence", agent_maybe_compile_policy), ("agent_eng_preflight", "task", agent_eng_preflight), ("agent_eng_reuse", "task", agent_eng_reuse), + ("agent_eng_bind_execute", "project,query,policy", agent_eng_bind_execute), + ("agent_eng_publish_gate", "project,artifact_id,candidate_only", agent_eng_publish_gate), ("agent_eng_record", "decision", agent_eng_record), ("sandbox_open", "", sandbox::sandbox_open), ("sandbox_write", "", sandbox::sandbox_write), diff --git a/plugins/web/internal/httpx/routes_media.go b/plugins/web/internal/httpx/routes_media.go index 6557c40..db6e17a 100644 --- a/plugins/web/internal/httpx/routes_media.go +++ b/plugins/web/internal/httpx/routes_media.go @@ -81,7 +81,16 @@ func (st *state) handleUpload(w http.ResponseWriter, r *http.Request, routePath _, hadCookie := session.IDFromCookie(r.Header.Get("Cookie")) sid, _, setCookie := resolveSession(r) - if err := r.ParseMultipartForm(32 << 20); err != nil { + // M01: hard-limit request body before multipart parse (32MiB was only in-memory + // form budget, not a request total). Cap at configured MaxBytes + 1MiB overhead. + maxBody := int64(cfg.MaxBytes) + if maxBody <= 0 { + maxBody = 32 << 20 + } + overhead := int64(1 << 20) + r.Body = http.MaxBytesReader(w, r.Body, maxBody+overhead) + + if err := r.ParseMultipartForm(maxBody + overhead); err != nil { jsonErr(w, http.StatusBadRequest, fmt.Sprintf("multipart: %v", err)) return } @@ -109,11 +118,20 @@ func (st *state) handleUpload(w http.ResponseWriter, r *http.Request, routePath return } defer f.Close() - b, err := io.ReadAll(f) + // Stream-capped read: never buffer more than MaxBytes. + lim := maxBody + if lim <= 0 { + lim = 32 << 20 + } + b, err := io.ReadAll(io.LimitReader(f, lim+1)) if err != nil { jsonErr(w, http.StatusBadRequest, fmt.Sprintf("read file: %v", err)) return } + if int64(len(b)) > lim { + jsonErr(w, http.StatusRequestEntityTooLarge, "file exceeds max_bytes") + return + } fileBytes = b } else { // Fallback: walk File map keys. diff --git a/plugins/web/internal/httpx/routes_media_test.go b/plugins/web/internal/httpx/routes_media_test.go new file mode 100644 index 0000000..7028c43 --- /dev/null +++ b/plugins/web/internal/httpx/routes_media_test.go @@ -0,0 +1,45 @@ +package httpx + +import ( + "bytes" + "mime/multipart" + "net/http" + "net/http/httptest" + "strings" + "testing" +) + +func TestHandleUploadRejectsOversizedBeforeFullBuffer(t *testing.T) { + st := &state{ + uploadRoutes: map[string]any{ + "/upload": map[string]any{ + "field": "file", + "storage_url": "memory://t", + "prefix": "u", + "max_bytes": float64(64), + }, + }, + } + cfg := parseUploadRoute(st.uploadRoutes["/upload"]) + var body bytes.Buffer + w := multipart.NewWriter(&body) + part, err := w.CreateFormFile("file", "big.txt") + if err != nil { + t.Fatal(err) + } + if _, err := part.Write([]byte(strings.Repeat("x", 200))); err != nil { + t.Fatal(err) + } + _ = w.Close() + + req := httptest.NewRequest(http.MethodPost, "/upload", &body) + req.Header.Set("Content-Type", w.FormDataContentType()) + rr := httptest.NewRecorder() + st.handleUpload(rr, req, "/upload", cfg) + if rr.Code == http.StatusOK { + t.Fatalf("expected reject for oversized upload, got %d body=%s", rr.Code, rr.Body.String()) + } + if rr.Code != http.StatusRequestEntityTooLarge && rr.Code != http.StatusBadRequest { + t.Fatalf("expected 413 or 400, got %d body=%s", rr.Code, rr.Body.String()) + } +} diff --git a/skills/marqdo/SKILL.md b/skills/marqdo/SKILL.md index 34dc04c..9417b0f 100644 --- a/skills/marqdo/SKILL.md +++ b/skills/marqdo/SKILL.md @@ -14,68 +14,49 @@ description: >- Marqdo = **Markdown markers as syntax**. A `.mq.md` file is both documentation and a runnable program. Read this skill **before** inventing syntax from Markdown habits or Python. -Canonical design (repo): `doc/design/markdown-mapping.md`, `doc/design/keywords.md`. Deeper tables: [reference.md](reference.md). Copy-paste patterns: [examples.md](examples.md). + -**Authoring / library craft** (GFM-first, no json glue, how to develop Marqdo): use **[marqdo-dev](../marqdo-dev/SKILL.md)** whenever writing or refactoring `.mq.md`, `lib/*`, or `ext/*`. +## CORE SYNTAX (v2.1 Occlusion — mandatory) -**Product release** (tag, CHANGELOG, README, public docs, VSIX, GitHub assets): use [marqdo-release](../marqdo-release/SKILL.md) — detect latest version, **ask the user for the next SemVer**, then follow that playbook. Do not invent a version number. +These rules are the **current default**. Do **not** teach Classic `[key](map)` index or “bold is return of print”. Classic migration only: [migration-classic.md](migration-classic.md). + +1. **File suffix** `.mq.md`. Print with `> print text=…` / `> 打印 内容=…` — never use `**…**` to print. +2. **`**…**` = code / return expression.** `*…*` = statement (bind / expr). Closing markers touch the last token (no trailing space). +3. **`[描述](payload)` = occlusion:** `[]` is prose (never executed); `()` is the only executed part. Empty `()` → `None` stub. +4. **Read collections with `access(coll, key)`** (aliases `取` / `取值`). **Never** use `[key](map)` as index under default Occlusion. +5. **Foreach:** `- [描述](遍历 coll 为 item)` (`遍历`/`for`, `为`/`as`/`in`). Prefer explicit traversal when generating new code. +6. **Import / call:** frontmatter `import:` / `导入:`; call `lib.member`. One language (EN or ZH) per file. +7. **Control flow:** `#` object/type, `##`+ function, `+` params, `1.`/`2.` branches, `-` loops, `N. *` else. No invented `if`/`for`/`def`/`return` keywords. +8. **Prose vs code:** blank line separates comment paragraphs from executable lines. +9. **GFM tables** for data; prefer `table.put` over `json.set` chains. `ext/**` never calls `host_*`. +10. Body narrative `[描述](target)` is lazy (NarrativeRef) — reading does not execute side effects. + +| Marker | Meaning (v2.1) | +|--------|----------------| +| `#` / `##`+ | Object/type · function/method | +| `*…*` | Statement | +| `**…**` | Return / code expression | +| `[描述](payload)` | Occlusion step (payload executes) | +| `access(c, k)` | Collection read | +| `> fn …` | Call | +| GFM table | Collection literal | + +Canonical: `doc/design/core-surface.md`, `doc/design/document-native-development-guide.md`. Deeper tables: [reference.md](reference.md). Patterns: [examples.md](examples.md). Authoring craft: [marqdo-dev](../marqdo-dev/SKILL.md). Release: [marqdo-release](../marqdo-release/SKILL.md). + +Executable-knowledge gate: [`tests/executable_knowledge/`](../../tests/executable_knowledge/README.md). ## Hard rules (do not violate) -1. **File suffix** must be `.mq.md` (never plain `.md` for executable sources). -2. **Output is not `**bold**`.** Print with `> print text=…` (or `> 打印 内容=…`). Bold `**…**` is **return value only**. -3. **Prose vs code:** after a blank line, an unmarked first line starts a **comment paragraph**; every following non-blank line stays comment until the next blank line. Put a **blank line** between narration and executable lines. -4. **Do not invent keywords** `if` / `else` / `while` / `for` / `def` / `return`. Control flow is Markdown: **`+` params** (under heading), **`1.` `2.` … branches**, **`-` loops**, arm `N. *` = else, `#` = **object/type**, `##`+ = **function/method**, `**…**` = return. -5. **Identifiers use backticks** — params `` + `name` ``. **Exemptions:** foreach `` - [item](coll) ``; footnote `` name[^key] ``; **inside italic `*…*` / bold `**…**`**, bare ids are **variables** (Python-style unified namespace) and method-call receivers are bare too. **Drop the backticks on value-expression variable names and method receivers inside `*…*` / `**…**`** — they are redundant once the `*…*`/`**…**` markers mark the segment as code: `*分类 = 分类[^苹果]*`, `**n * 2**`, `*p = > page.主体装配 组件=home*`. **Text literals must be quoted**: `*a = > text.split value=src sep=","*`. Backticks **still required** where a bare word is *not* a variable in **standalone `>` calls** (bare = text there): `` > str `n` ``, `` > `obj`.method ``. **No trailing space** inside `*…*` / `**…**` wrapped code: the closing marker must touch the last token directly (`*a = 1*`, `**n**`), never `` *a = 1 * ``. -6. **Structure lines are not wrapped in italics.** `#` `>` `+` `-` `|` lines stand alone; use `*…*` only for general statements (bindings / expressions). -7. **Paths:** In bare *expressions* `/` is division. In **call args / param defaults / table cells**, unspaced `a/b` and quoted `".marqdo/agent-kb"` are path text — no `json.parse` needed. Numeric ratios that must stay text use quotes (`"1/5"`, `"16/9"`); bare `1/5` is division. -8. Prefer ending side-effect-only function bodies with `*None*` / `*无*` / whole-line `**` / `****` when later lines must belong to an outer function. Lone `---` / `***` are Markdown thematic breaks (skipped), not function end. -9. **`ext/**` never calls `host_*`.** Agent/OKF helpers are plugin names (`agent_kb_*`, …) after `plugin.load`. Do **not** add agent/OKF domain code to `src/host/` (core bloat). See `doc/design/ext-agent.md` §4. -10. **Browser (route C/D/E/F):** client logic is Marqdo on WASM; official bridge is host glue and **may** implement lists, routing, storage, WebSocket, canvas, file read, observers — authors **must not hand-write business JS**. Use `web.client_embed` / `data-mq-source-url` auto-mount. Client effects: prefer **GFM tables + `lib/browser`** ([marqdo-dev](../marqdo-dev/SKILL.md)). -11. **Code-as-documentation / no bag glue:** Prefer **GFM tables** for maps, lists, wire, commands. Prefer **`table.put` / named helpers** over `json.set` / `json.append` chains. `lib/json` is for parse/stringify/quote only — not a dict builder. Unreadable json pipelines are a style bug. Read collections with **`[key](map)`**, not footnote indexes for new code. In `markup: occlusion` files read with **`access(map, key)`** — there `[x](y)` is occlusion, not index. **Positive copy-paste shapes** (result map, soft-error pack, sparse `table.put`): [marqdo-dev](../marqdo-dev/SKILL.md) §2 · [examples.md](examples.md) §6e. Do **not** extend legacy `json.set` in `ext/ai` — refactor to tables. -12. **One language per file:** Do not mix EN and ZH API names in the same `.mq.md` (imports, calls, params). Pick English *or* Chinese mirrors for the whole program. -13. **Sandbox trial habit (Agent / generated `.mq.md`):** Prefer `agent.trial` / `run sandbox=True` / `sandbox.open`→`run`→`diff`→`commit` before promoting to the host tree. Host files must stay unchanged until a successful commit. Caps live in `sandbox.profile` GFM tables — do not invent a second dialect. +1. Obey **CORE SYNTAX** above — Occlusion is default; do not emit Classic index links. +2. **Identifiers:** params `` + `name` ``; inside `*…*` / `**…**` bare ids are variables. Quote text literals. Standalone `>` calls: backticks where bare would be text. +3. **Structure lines** (`#` `>` `+` `-` `|`) are not wrapped in italics. +4. **Paths:** bare `/` in expressions is division; in call args / table cells unspaced `a/b` is path text. Ratios as text need quotes (`"1/5"`). +5. End side-effect-only helpers with `*None*` / `*空*` / `**` / `****` when the outer function continues. Lone `---` is a thematic break, not function end. +6. **`ext/**` never calls `host_*`.** Prefer tables over `json.set` / `json.append`. +7. **One language per file.** Sandbox trial before promoting Agent-generated trees. +8. Browser: Marqdo on WASM + `lib/browser` tables; do not hand-write business JS. -## Markup → meaning (v0.2) - -| Marker | Meaning | -|--------|---------| -| `#` | **Object / type** (constructor body). `# main` = entry object. `# Child = > Parent` = inherit (**no** implicit parent constructor; call `` `self` = > Parent … `` explicitly when needed). | -| `##` … `######` | **Function / method** (nesting by heading depth). | -| `` + `name` `` under heading | Parameter (optional `` `name`=default ``) | -| `1.` `2.` … in body | Branch arm (condition or `N. *` else). **Same-indent restart at `1.` = new branch statement** (not more arms of the previous list). | -| `- …` inside body | Loop (`while` or `` [item](coll) `` / `` [`item`](`coll`) ``) | -| Line `N. *` | Else arm | -| `> fn args` | Call (named `k=v` or positional) | -| `` > `obj`.method args `` | Method call (`obj` must be a map with `_type`) | -| Frontmatter nested `import:` / `导入:` map | Import file (`bind: path.mq.md`) or short name (`bind: lib.member`) | -| `> lib.member …` / `> lib.Type.member …` | Call via bare dotted path (instance methods need `` `var`.m ``) | -| `*…*` | Statement (bind / expr) — closing `*` touches last token, **no trailing space**; bare ids are variables | -| `**…**` | Return value — closing `**` touches last token, **no trailing space**; bare ids are variables | -| `****` or `**` + spaces + `**` | Return `None` and end function body | -| Lone `---` / `***` | Markdown thematic break (skip; not function end) | -| GFM table after empty RHS bind | Collection; get with `[key](coll)` (preferred) or legacy `` `m`[^key] `` | -| `- [item](coll)` | Foreach (unchanged) | -| `` ```lang `` | Foreign code block (via `lib/foreign`) | -| `` `"text"` `` | Quoted string (`\n` `\t` `\\` `\"`; `` `var` `` inside); bare tokens unescaped | -| Unmarked prose | Comment | - -Entry: load file (often `index.mq.md`) → collect defs → run `# main`. - -## Occlusion syntax (v2.1 default) - -**Default:** unmarked sources and `markup: occlusion` / `标记: 遮蔽` treat `[x](y)` as **occlusion** (description wraps payload), not classic index. Explicit Classic / `--legacy-markup` is a transition path only — do not teach `[键](集合)` as the default. Prefer `access(集合, 键)` for reads. - -Executable-knowledge author baseline (six tasks, offline gate): [`tests/executable_knowledge/`](../../tests/executable_knowledge/README.md). Design goals ≠ shipped knowledge reuse; see [`doc/design/document-native-development-guide.md`](../../doc/design/document-native-development-guide.md). - -- `[描述](实现)` — `[]` is prose (never executed); `()` is the **only** executed part. -- `[顾客留存率](0.42)` · `[总价](access(价目表, "苹果") * 数量)` · `[功能开发中]()` → `None` (stub; `label.empty_payload` warning). -- Read maps/lists with `access(集合, 键)` (aliases `取` / `取值`): `[第二项](access(列, 2))`. -- Foreach: `- [篮子里的每个果子](遍历 篮子 为 果)` (accepts `遍历`/`for` and `为`/`as`/`in`). -- Calls inside `()`: `加(1)` / `加(1, 2)` / `加一 号(31)` / `txt.str_trim(" x ")` — `)` terminates, commas optional. -- Tooling: `marqdo labels pending` (stub ledger) · `marqdo labels --grep 词 --count` · `marqdo migrate labels . --write` · `marqdo check` reports `label.missing` / `label.empty_payload` / `label.ref_dangling`. - ## Minimal program ```markdown diff --git a/skills/marqdo/migration-classic.md b/skills/marqdo/migration-classic.md new file mode 100644 index 0000000..79922ce --- /dev/null +++ b/skills/marqdo/migration-classic.md @@ -0,0 +1,12 @@ +# Classic markup migration card (not default) + +Use this card **only** when editing files that declare `markup: classic` / `标记: 经典`, or when the CLI is run with `--legacy-markup`. New documents must use Occlusion (see SKILL.md CORE SYNTAX). + +| Classic (legacy) | Occlusion / current | +|---|---| +| `[key](map)` index read | `access(map, "key")` / `取(map, "key")` | +| `- [item](coll)` foreach by index form | `- [描述](遍历 coll 为 item)` | +| Teach `**…**` as “the return of print” | Print with `> print`; `**…**` is return/code expression | +| Default link = index | Default `[描述](payload)` = occlusion; `[]` never executes | + +Do not place Classic rules in the front of the default Skill pack. After migration, remove `markup: classic` and verify with `marqdo run` / `marqdo check`. diff --git a/src/document_extract.rs b/src/document_extract.rs new file mode 100644 index 0000000..b534cfd --- /dev/null +++ b/src/document_extract.rs @@ -0,0 +1,293 @@ +//! Document extraction protocol (M01): DocumentSource → ExtractedDocument. +//! +//! Format parsers live here / in extensions — language core does not OCR. +//! First slice: plain text and Markdown. PDF/DOCX plug in later via the same shape. + +use std::fs; +use std::path::Path; + +use sha2::{Digest, Sha256}; +use serde_json::{json, Value}; + +pub const EXTRACTOR_VERSION: &str = "marqdo-document-extract/1"; + +#[derive(Debug, Clone)] +pub struct DocumentSource { + pub path: std::path::PathBuf, + pub format_hint: Option, +} + +#[derive(Debug, Clone)] +pub struct ExtractedSegment { + pub id: String, + pub kind: String, + pub text: String, + pub locator: String, + pub confidence: f64, +} + +#[derive(Debug, Clone)] +pub struct ExtractedDocument { + pub source_uri: String, + pub source_digest: String, + pub format: String, + pub extractor_version: String, + pub text: String, + pub segments: Vec, + pub missing: Vec, + pub diagnostics: Vec, + pub status: String, +} + +impl ExtractedDocument { + pub fn to_json(&self) -> Value { + json!({ + "schema_version": 1, + "source_uri": self.source_uri, + "source_digest": self.source_digest, + "format": self.format, + "extractor_version": self.extractor_version, + "status": self.status, + "text": self.text, + "segments": self.segments.iter().map(|s| json!({ + "id": s.id, + "kind": s.kind, + "text": s.text, + "locator": s.locator, + "confidence": s.confidence, + })).collect::>(), + "missing": self.missing, + "diagnostics": self.diagnostics, + }) + } +} + +/// Extract text/Markdown from a file. Uploaded `.mq.md` is treated as untrusted +/// source text and is never executed here. +pub fn extract_document(source: &DocumentSource) -> ExtractedDocument { + let path = &source.path; + let uri = path.to_string_lossy().replace('\\', "/"); + let ext = path + .extension() + .and_then(|e| e.to_str()) + .unwrap_or("") + .to_ascii_lowercase(); + let name = path + .file_name() + .and_then(|n| n.to_str()) + .unwrap_or("") + .to_ascii_lowercase(); + + let format = source + .format_hint + .clone() + .unwrap_or_else(|| { + if name.ends_with(".mq.md") || ext == "md" { + "markdown".into() + } else if matches!(ext.as_str(), "txt" | "text") { + "text".into() + } else if matches!(ext.as_str(), "pdf" | "docx") { + ext.clone() + } else { + "unknown".into() + } + }); + + if matches!( + format.as_str(), + "pdf" | "docx" | "ocr" | "png" | "jpg" | "jpeg" | "webp" | "tiff" + ) || matches!(ext.as_str(), "pdf" | "docx" | "png" | "jpg" | "jpeg" | "webp" | "tiff") + { + let kind = if matches!(format.as_str(), "ocr") + || matches!(ext.as_str(), "png" | "jpg" | "jpeg" | "webp" | "tiff") + { + "ocr/image".to_string() + } else { + format.clone() + }; + return ExtractedDocument { + source_uri: uri, + source_digest: String::new(), + format, + extractor_version: EXTRACTOR_VERSION.into(), + text: String::new(), + segments: vec![], + missing: vec!["full_document".into()], + diagnostics: vec![format!( + "format `{kind}` requires an external extractor plugin (not in core); incomplete extraction is explicit failure" + )], + status: "failed".into(), + }; + } + + let bytes = match fs::read(path) { + Ok(b) => b, + Err(e) => { + return ExtractedDocument { + source_uri: uri, + source_digest: String::new(), + format, + extractor_version: EXTRACTOR_VERSION.into(), + text: String::new(), + segments: vec![], + missing: vec!["full_document".into()], + diagnostics: vec![format!("read error: {e}")], + status: "failed".into(), + }; + } + }; + + if bytes.is_empty() { + return ExtractedDocument { + source_uri: uri, + source_digest: digest_bytes(&bytes), + format, + extractor_version: EXTRACTOR_VERSION.into(), + text: String::new(), + segments: vec![], + missing: vec!["content".into()], + diagnostics: vec!["empty document".into()], + status: "failed".into(), + }; + } + + // Resource limit: refuse to fully buffer huge uploads as text (M01). + const MAX_EXTRACT_BYTES: usize = 8 << 20; + if bytes.len() > MAX_EXTRACT_BYTES { + let head = &bytes[..MAX_EXTRACT_BYTES]; + let lossy = String::from_utf8_lossy(head).into_owned(); + let mut doc = extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &lossy); + doc.missing.push("tail_beyond_limit".into()); + doc.diagnostics.push(format!( + "document truncated at {MAX_EXTRACT_BYTES} bytes (got {}); extraction is partial", + bytes.len() + )); + doc.status = "partial".into(); + return doc; + } + + let text = match String::from_utf8(bytes.clone()) { + Ok(t) => t, + Err(e) => { + let lossy = String::from_utf8_lossy(&bytes).into_owned(); + let mut doc = extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &lossy); + doc.diagnostics + .push(format!("invalid utf-8 ({e}); decoded lossy")); + doc.status = if doc.text.is_empty() { + "failed".into() + } else { + "partial".into() + }; + return doc; + } + }; + + extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &text) +} + +fn extract_markdown_or_text( + uri: &str, + digest: &str, + format: &str, + text: &str, +) -> ExtractedDocument { + let mut segments = Vec::new(); + let mut n = 0u32; + for (i, line) in text.lines().enumerate() { + let line_no = i + 1; + if let Some(rest) = line.trim().strip_prefix('#') { + n += 1; + let heading = rest.trim_start_matches('#').trim(); + segments.push(ExtractedSegment { + id: format!("seg-{n}"), + kind: "heading".into(), + text: heading.to_string(), + locator: format!("L{line_no}"), + confidence: 1.0, + }); + } else if line.trim().starts_with('|') { + n += 1; + segments.push(ExtractedSegment { + id: format!("seg-{n}"), + kind: "table_row".into(), + text: line.to_string(), + locator: format!("L{line_no}"), + confidence: 0.9, + }); + } else if !line.trim().is_empty() { + n += 1; + segments.push(ExtractedSegment { + id: format!("seg-{n}"), + kind: "paragraph".into(), + text: line.to_string(), + locator: format!("L{line_no}"), + confidence: 1.0, + }); + } + } + ExtractedDocument { + source_uri: uri.to_string(), + source_digest: digest.to_string(), + format: format.to_string(), + extractor_version: EXTRACTOR_VERSION.into(), + text: text.to_string(), + segments, + missing: vec![], + diagnostics: vec![], + status: "ok".into(), + } +} + +fn digest_bytes(bytes: &[u8]) -> String { + let mut h = Sha256::new(); + h.update(bytes); + format!("sha256:{:x}", h.finalize()) +} + +/// Convenience for tests / CLI: extract if path exists. +pub fn extract_path(path: &Path) -> ExtractedDocument { + extract_document(&DocumentSource { + path: path.to_path_buf(), + format_hint: None, + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::io::Write; + + #[test] + fn extracts_markdown_headings() { + let dir = std::env::temp_dir().join(format!("mq-extract-{}", std::process::id())); + let _ = fs::create_dir_all(&dir); + let path = dir.join("note.md"); + let mut f = fs::File::create(&path).unwrap(); + writeln!(f, "# rate\n\nhello").unwrap(); + let doc = extract_path(&path); + assert_eq!(doc.status, "ok"); + assert!(doc.segments.iter().any(|s| s.kind == "heading" && s.text == "rate")); + let _ = fs::remove_dir_all(&dir); + } + + #[test] + fn pdf_reports_missing_extractor() { + let doc = extract_document(&DocumentSource { + path: Path::new("x.pdf").into(), + format_hint: Some("pdf".into()), + }); + assert_eq!(doc.status, "failed"); + assert!(!doc.diagnostics.is_empty()); + } + + #[test] + fn ocr_image_reports_missing_extractor() { + let doc = extract_document(&DocumentSource { + path: Path::new("scan.png").into(), + format_hint: Some("ocr".into()), + }); + assert_eq!(doc.status, "failed"); + assert!(doc.missing.contains(&"full_document".into())); + assert!(doc.diagnostics.iter().any(|d| d.contains("ocr") || d.contains("extractor"))); + } +} diff --git a/src/host/plugin.rs b/src/host/plugin.rs index ae3cafa..cbfff4f 100644 --- a/src/host/plugin.rs +++ b/src/host/plugin.rs @@ -460,6 +460,24 @@ unsafe extern "C" fn host_query( let cap = crate::run_file_capture(&p, &opts).map_err(|e| e.to_string())?; value_to_json(&cap.value) })(), + // M04: plugins may drive MLSP artifact.bind/execute without inventing a second protocol. + "mlsp" => (|| { + let raw = args_owned.as_deref().unwrap_or("{}"); + let args: serde_json::Value = serde_json::from_str(if raw.trim().is_empty() { + "{}" + } else { + raw + }) + .map_err(|e| format!("mlsp args: {e}"))?; + let method = args + .get("method") + .and_then(|v| v.as_str()) + .ok_or_else(|| "mlsp requires `method`".to_string())?; + let params = args.get("params").cloned().unwrap_or(serde_json::json!({})); + let line = serde_json::json!({"method": method, "params": params}).to_string(); + let resp = crate::mlsp::handle_line(&line); + serde_json::from_str(&resp).map_err(|e| format!("mlsp response: {e}")) + })(), other => Err(format!("host_query: unknown `{other}`")), }; diff --git a/src/knowledge/compile.rs b/src/knowledge/compile.rs index d7c69c2..9163021 100644 --- a/src/knowledge/compile.rs +++ b/src/knowledge/compile.rs @@ -37,6 +37,7 @@ pub fn compile(opts: KnowledgeOptions) -> Result { }; let graph = extract_graph(&extract_opts)?; write_projections(&opts.out_dir, &graph)?; + write_publish_manifest(&opts.out_dir, &semantic, &graph)?; eprintln!( "marqdo knowledge: {} module(s), {} symbol(s), {} capability(ies), {} knowledge → {} (semantic graph/; retrieval ekc/)", graph.modules.len(), @@ -48,6 +49,52 @@ pub fn compile(opts: KnowledgeOptions) -> Result { Ok(graph) } +/// M06: associate semantic generation, retrieval generation, and source digests. +fn write_publish_manifest( + out_dir: &Path, + semantic: &crate::sema::Compilation, + graph: &KnowledgeGraph, +) -> Result<()> { + let current = fs::read_to_string(out_dir.join("CURRENT")).unwrap_or_default(); + let generation = current.trim().to_string(); + let source_digest = { + use sha2::{Digest, Sha256}; + let mut h = Sha256::new(); + for a in &semantic.semantics.artifacts { + h.update(a.hashes.artifact.0.as_bytes()); + } + format!("sha256:{:x}", h.finalize()) + }; + let evidence_digest = { + use sha2::{Digest, Sha256}; + let mut h = Sha256::new(); + for a in &semantic.semantics.artifacts { + h.update(a.hashes.evidence.0.as_bytes()); + } + format!("sha256:{:x}", h.finalize()) + }; + let manifest = json!({ + "schema_version": 1, + "kind": "publish_manifest", + "semantic_generation": generation, + "retrieval": { + "ekc_graph": "ekc/graph", + "modules": graph.modules.len(), + "symbols": graph.symbols.len(), + "capabilities": graph.capabilities.len(), + "knowledge": graph.knowledge.len(), + }, + "source_digest": source_digest, + "evidence_digest": evidence_digest, + "excluded_dirs": crate::project::EXCLUDED_DIRS, + "project_id": semantic.semantics.project_id, + }); + let path = out_dir.join("publish_manifest.json"); + fs::write(&path, serde_json::to_string_pretty(&manifest)?) + .with_context(|| format!("write {}", path.display()))?; + Ok(()) +} + /// Directory for the legacy EKC retrieval projection (not execution authority). pub fn ekc_graph_dir(out_dir: &Path) -> std::path::PathBuf { out_dir.join("ekc").join("graph") diff --git a/src/knowledge/extract.rs b/src/knowledge/extract.rs index 4887776..598540d 100644 --- a/src/knowledge/extract.rs +++ b/src/knowledge/extract.rs @@ -164,13 +164,7 @@ fn collect_mq_md( let p = entry.path(); if p.is_dir() { let name = p.file_name().and_then(|s| s.to_str()).unwrap_or(""); - if name == "target" - || name == ".git" - || name == "node_modules" - || name == "docker" - || name == "data" - || name == ".cursor" - { + if crate::project::is_excluded_dir_name(name) { continue; } if let Ok(canon) = p.canonicalize() { @@ -210,13 +204,7 @@ fn walk_knowledge_md( let p = entry.path(); if p.is_dir() { let name = p.file_name().and_then(|s| s.to_str()).unwrap_or(""); - if name == "target" - || name == ".git" - || name == ".marqdo" - || name == "docker" - || name == "data" - || name == ".cursor" - { + if crate::project::is_excluded_dir_name(name) { continue; } if let Ok(canon) = p.canonicalize() { diff --git a/src/lib.rs b/src/lib.rs index 07f8414..38d7f8c 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -21,6 +21,7 @@ pub mod check; pub mod contract; pub mod debug; pub mod diagnostics; +pub mod document_extract; pub mod embedded_lib; #[cfg(feature = "cli")] pub mod ext_cli; diff --git a/src/main.rs b/src/main.rs index 2ffb69c..df87243 100644 --- a/src/main.rs +++ b/src/main.rs @@ -323,6 +323,10 @@ enum Commands { #[arg(long)] json: bool, + + /// Exit non-zero when conflicts found (CI gate; default remains report-only) + #[arg(long)] + fail: bool, }, /// List stale / deprecated / superseded knowledge Stale { @@ -334,6 +338,10 @@ enum Commands { #[arg(long)] json: bool, + + /// Exit non-zero when stale items found (CI gate; default remains report-only) + #[arg(long)] + fail: bool, }, /// Detect duplicate capabilities via symbol fingerprints Duplicate { @@ -353,7 +361,7 @@ enum Commands { #[arg(long)] json: bool, }, - /// Verify code ↔ knowledge consistency + /// Verify code ↔ knowledge consistency (EKC structural; exit 1 only on errors) Verify { #[arg(value_name = "PATH")] path: Option, @@ -363,6 +371,10 @@ enum Commands { #[arg(long)] json: bool, + + /// Also fail on warnings (does not change default success-when-no-errors) + #[arg(long)] + fail_on_warning: bool, }, /// Official extension installer (`list` / `add` / `remove`) Ext { @@ -415,6 +427,18 @@ enum KnowledgeAction { #[arg(value_name = "SYMBOL_OR_PATH")] query: String, }, + /// Machine adjudication of semantic artifacts (M03): may_bind / may_publish + Adjudicate { + /// Artifact id, symbol id, or display name (omit to report all functions) + #[arg(value_name = "ARTIFACT")] + artifact: Option, + /// Policy: pure | business | static + #[arg(long, default_value = "business")] + policy: String, + /// Exit 1 when any reported artifact has may_bind=false + #[arg(long)] + fail: bool, + }, /// Run engineering preflight for a task Preflight { #[arg(value_name = "TASK")] @@ -1324,6 +1348,51 @@ fn try_main(cli: Cli) -> Result { 1 }) } + KnowledgeAction::Adjudicate { + artifact, + policy, + fail, + } => { + let semantic_root = if path.is_file() { + path.parent() + .unwrap_or_else(|| std::path::Path::new(".")) + .to_path_buf() + } else { + path.clone() + }; + let report = marqdo::mlsp::adjudicate_project( + &semantic_root, + artifact.as_deref(), + &policy, + ) + .map_err(|e| anyhow::anyhow!("{}", e))?; + if json { + println!("{report}"); + } else { + println!( + "knowledge adjudicate policy={} may_bind_all={} count={}", + policy, + report["may_bind_all"], + report["count"] + ); + for item in report["reports"].as_array().into_iter().flatten() { + println!( + " {} evidence={} may_bind={} may_publish={}", + item["artifact_id"].as_str().unwrap_or(""), + item["verification"]["evidence"].as_str().unwrap_or(""), + item["may_bind"], + item["may_publish"] + ); + } + } + Ok( + if fail && report["may_bind_all"].as_bool() != Some(true) { + 1 + } else { + 0 + }, + ) + } KnowledgeAction::Preflight { task, min_candidates, @@ -1462,7 +1531,12 @@ fn try_main(cli: Cli) -> Result { } Ok(0) } - Commands::Conflicts { path, out, json } => { + Commands::Conflicts { + path, + out, + json, + fail, + } => { let path = path.unwrap_or_else(|| PathBuf::from(".")); let graph = ensure_compiled(&path, &out)?; let items = find_conflicts(&graph); @@ -1471,9 +1545,14 @@ fn try_main(cli: Cli) -> Result { } else { print!("{}", format_conflicts_text(&items)); } - Ok(0) + Ok(if fail && !items.is_empty() { 1 } else { 0 }) } - Commands::Stale { path, out, json } => { + Commands::Stale { + path, + out, + json, + fail, + } => { let path = path.unwrap_or_else(|| PathBuf::from(".")); let graph = ensure_compiled(&path, &out)?; let items = find_stale(&graph, &path); @@ -1482,7 +1561,7 @@ fn try_main(cli: Cli) -> Result { } else { print!("{}", format_stale_text(&items)); } - Ok(0) + Ok(if fail && !items.is_empty() { 1 } else { 0 }) } Commands::Duplicate { path, @@ -1517,7 +1596,12 @@ fn try_main(cli: Cli) -> Result { Ok(0) } } - Commands::Verify { path, out, json } => { + Commands::Verify { + path, + out, + json, + fail_on_warning, + } => { let path = path.unwrap_or_else(|| PathBuf::from(".")); let graph = ensure_compiled(&path, &out)?; let issues = verify(&graph); @@ -1527,7 +1611,12 @@ fn try_main(cli: Cli) -> Result { print!("{}", format_verify_text(&issues)); } let errors = issues.iter().filter(|i| i.severity == "error").count(); - Ok(if errors > 0 { 1 } else { 0 }) + let warnings = issues.iter().filter(|i| i.severity == "warning").count(); + Ok(if errors > 0 || (fail_on_warning && warnings > 0) { + 1 + } else { + 0 + }) } Commands::Ext { action } => { match action { diff --git a/src/mlsp.rs b/src/mlsp.rs index 326b475..56cdb3d 100644 --- a/src/mlsp.rs +++ b/src/mlsp.rs @@ -64,6 +64,8 @@ fn dispatch(req: &Value) -> Result { Some("artifact.impact") => artifact_impact(¶ms), Some("artifact.bind") => artifact_bind(¶ms), Some("artifact.execute") => artifact_execute(¶ms), + Some("artifact.reuse_execute") => artifact_reuse_execute(¶ms), + Some("document.extract") => document_extract(¶ms), Some("artifact.patch") => artifact_patch(¶ms), Some("artifact.extract_pure") => artifact_extract_pure(¶ms), // `` M9: occlusion label search (two-tier vocabulary). @@ -72,7 +74,7 @@ fn dispatch(req: &Value) -> Result { Some("session.summary") => session_summary(), other => Err(err( "mlsp.unknown_method", - format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema / artifact.find / artifact.inspect / link.resolve / artifact.verify / artifact.impact / artifact.bind / artifact.execute / artifact.patch / artifact.extract_pure"), + format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema / artifact.find / artifact.inspect / link.resolve / artifact.verify / artifact.impact / artifact.bind / artifact.execute / artifact.reuse_execute / artifact.patch / artifact.extract_pure / document.extract"), )), } } @@ -129,6 +131,22 @@ fn artifact_query<'a>(project: &'a crate::sema::SemanticProject, query: &str) -> }).collect() } +fn document_extract(params: &Value) -> Result { + let path = params + .get("path") + .and_then(Value::as_str) + .ok_or_else(|| err("mlsp.bad_request", "document.extract 需要 path"))?; + let hint = params + .get("format") + .and_then(Value::as_str) + .map(|s| s.to_string()); + let doc = crate::document_extract::extract_document(&crate::document_extract::DocumentSource { + path: PathBuf::from(path), + format_hint: hint, + }); + Ok(doc.to_json()) +} + fn artifact_find(params: &Value) -> Result { let compilation = semantic_project(params)?; let query = params.get("query").and_then(Value::as_str).unwrap_or("").to_ascii_lowercase(); @@ -173,10 +191,216 @@ fn artifact_verify(params: &Value) -> Result { let candidates = artifact_query(&compilation.semantics, query); if candidates.len() != 1 { return Err(err(if candidates.is_empty() { "artifact.unresolved" } else { "artifact.ambiguous" }, format!("无法唯一验证 `{query}`"))); } let artifact = candidates[0]; - let mut record = crate::sema::artifact_json(artifact)["verification"].clone(); - record["policy"] = params.get("policy").cloned().unwrap_or_else(|| json!("static")); - record["artifact_id"] = json!(artifact.artifact_id.0); - Ok(record) + let policy = params.get("policy").and_then(Value::as_str).unwrap_or("static"); + Ok(build_verification_report(artifact, policy)) +} + +/// CLI / product machine adjudication over semantic artifacts (M03). +pub fn adjudicate_project(project: &std::path::Path, artifact: Option<&str>, policy: &str) -> Result { + let compilation = semantic_project(&json!({"project": project}))?; + let targets: Vec<&crate::sema::SemanticArtifact> = if let Some(q) = artifact { + let c = artifact_query(&compilation.semantics, q); + if c.is_empty() { + return Err(err("artifact.unresolved", format!("没有制品 `{q}`"))); + } + c + } else { + compilation + .semantics + .artifacts + .iter() + .filter(|a| a.kind == "function") + .collect() + }; + let reports: Vec = targets + .iter() + .map(|a| build_verification_report(a, policy)) + .collect(); + let any_denied = reports.iter().any(|r| r["may_bind"] == false); + Ok(json!({ + "schema_version": 1, + "policy": policy, + "count": reports.len(), + "may_bind_all": !any_denied, + "reports": reports, + })) +} + +/// Known bind/verify policies (M03). Unknown strings are rejected. +fn parse_policy(policy: &str) -> Result<&str, Value> { + match policy { + "pure" | "business" | "static" => Ok(policy), + other => Err(json!({ + "code": "artifact.unknown_policy", + "message": format!("未知政策 `{other}`;允许 pure / business / static"), + "policy": other, + })), + } +} + +fn evidence_allows_business(status: &str) -> bool { + matches!(status, "sourced" | "verified") +} + +fn build_verification_report(artifact: &crate::sema::SemanticArtifact, policy: &str) -> Value { + let policy = match parse_policy(policy) { + Ok(p) => p, + Err(e) => { + return json!({ + "schema_version": 1, + "artifact_id": artifact.artifact_id.0, + "policy": policy, + "checks": {}, + "blockers": [e], + "may_bind": false, + "may_publish": false, + "verification": crate::sema::artifact_json(artifact)["verification"].clone(), + }); + } + }; + let region_status = if artifact.applicability.contains("region=") { + "declared" + } else { + "unknown" + }; + let checks = json!({ + "parse": {"status": artifact.verification.parse, "detail": "static_project_compile"}, + "resolve": {"status": artifact.verification.resolve, "detail": "symbol_bound"}, + "contract": {"status": artifact.verification.contract, "detail": "not_executed_by_verify"}, + "execution": {"status": artifact.verification.execution, "detail": "not_run"}, + "tests": {"status": artifact.verification.tests, "detail": "no_test_runner_invoked"}, + "evidence": {"status": artifact.verification.evidence, "detail": "compile_time_provenance"}, + "domain": {"status": artifact.verification.domain, "detail": "unreviewed"}, + "region": {"status": region_status, "detail": "input_domain_not_checked"}, + "input_domain": {"status": "unknown", "detail": "not_checked_by_static_verify"}, + }); + let mut blockers = Vec::new(); + // Never upgrade tests/execution to passed without a runner (M03). + if matches!( + artifact.verification.tests.as_str(), + "passed" | "verified" + ) && artifact + .verification + .methods + .iter() + .all(|m| m == "static_project_compile") + { + blockers.push(json!({ + "code": "artifact.tests_unproven", + "check": "tests", + "status": artifact.verification.tests, + "detail": "static compile must not claim tests passed", + })); + } + if policy == "business" && !evidence_allows_business(&artifact.verification.evidence) { + blockers.push(json!({ + "code": "artifact.evidence_denied", + "check": "evidence", + "status": artifact.verification.evidence, + })); + } + if policy == "pure" + && artifact + .effects + .values() + .iter() + .any(|e| e != "pure") + { + blockers.push(json!({ + "code": "artifact.effect_denied", + "check": "effects", + "effects": artifact.effects.values(), + })); + } + let may_bind = blockers.is_empty() + && match policy { + "business" => evidence_allows_business(&artifact.verification.evidence), + "pure" => artifact.effects.values().iter().all(|e| e == "pure"), + _ => true, + }; + // Publish requires stronger evidence than bind; static verify never claims tests passed. + let may_publish = may_bind + && artifact.verification.tests != "unknown" + && artifact.verification.tests != "not_run" + && artifact.verification.execution != "not_run" + && !matches!( + artifact.verification.evidence.as_str(), + "unverified" | "missing_source" | "anchor_missing" | "expired" | "invalid_date" + ); + json!({ + "schema_version": 1, + "artifact_id": artifact.artifact_id.0, + "symbol_id": artifact.symbol_id.as_ref().map(|id| id.as_str()), + "digest": artifact.hashes.artifact.0, + "evidence_digest": artifact.hashes.evidence.0, + "closure_digest": artifact.hashes.closure.0, + "policy": policy, + "checks": checks, + "blockers": blockers, + "may_bind": may_bind, + "may_publish": may_publish, + "verification": { + "parse": artifact.verification.parse, + "resolve": artifact.verification.resolve, + "contract": artifact.verification.contract, + "execution": artifact.verification.execution, + "tests": artifact.verification.tests, + "evidence": artifact.verification.evidence, + "domain": artifact.verification.domain, + "methods": artifact.verification.methods, + }, + }) +} + +/// M04: bind then execute in one call (actual reuse, not discovery scores). +fn artifact_reuse_execute(params: &Value) -> Result { + let project = params + .get("project") + .or_else(|| params.get("path")) + .cloned() + .ok_or_else(|| err("mlsp.bad_request", "artifact.reuse_execute 需要 project"))?; + let query = params + .get("artifact_id") + .or_else(|| params.get("symbol_id")) + .or_else(|| params.get("query")) + .and_then(Value::as_str) + .ok_or_else(|| err("mlsp.bad_request", "artifact.reuse_execute 需要 artifact_id"))?; + let policy = params.get("policy").and_then(Value::as_str).unwrap_or("pure"); + let inputs = params.get("inputs").cloned().unwrap_or(json!({})); + let bound = artifact_bind(&json!({ + "project": project, + "artifact_id": query, + "policy": policy, + "expected_digest": params.get("expected_digest"), + }))?; + let handle = bound + .get("handle") + .cloned() + .ok_or_else(|| err("artifact.bind_failed", "bind 未返回 handle"))?; + let executed = artifact_execute(&json!({ + "handle": handle, + "inputs": inputs, + "allow_effects": params.get("allow_effects"), + }))?; + Ok(json!({ + "schema_version": 1, + "actual_reuse": true, + "discovery_only": false, + "artifact_id": executed.get("artifact_id"), + "symbol_id": executed.get("symbol_id"), + "digest": executed.get("digest"), + "result": executed.get("result"), + "stdout": executed.get("stdout"), + "effects": executed.get("effects"), + "trace": { + "artifact_id": executed.get("artifact_id"), + "symbol_id": executed.get("symbol_id"), + "digest": executed.get("digest"), + "inputs": inputs, + "value": executed.get("result"), + "stdout": executed.get("stdout"), + }, + })) } fn artifact_impact(params: &Value) -> Result { @@ -197,10 +421,12 @@ fn artifact_bind(params: &Value) -> Result { if expected.is_some() && expected != Some(artifact.hashes.artifact.0.as_str()) { return Err(json!({"code":"artifact.digest_mismatch","message":"目标摘要已变化,拒绝绑定","expected":expected,"actual":artifact.hashes.artifact.0})); } - let policy = params - .get("policy") - .and_then(Value::as_str) - .unwrap_or("pure"); + let policy = parse_policy( + params + .get("policy") + .and_then(Value::as_str) + .unwrap_or("pure"), + )?; let effects = artifact.effects.values(); if policy == "pure" && effects.iter().any(|effect| effect != "pure") @@ -212,15 +438,10 @@ fn artifact_bind(params: &Value) -> Result { "effects": effects, })); } - if policy == "business" - && !matches!( - artifact.verification.evidence.as_str(), - "sourced" | "verified" - ) - { + if policy == "business" && !evidence_allows_business(&artifact.verification.evidence) { return Err(json!({ "code": "artifact.evidence_denied", - "message": "business 策略要求来源已关联;当前证据状态不足", + "message": "business 策略要求来源已关联且锚点有效、未过期;当前证据状态不足", "policy": policy, "evidence": artifact.verification.evidence, })); @@ -230,14 +451,38 @@ fn artifact_bind(params: &Value) -> Result { .as_ref() .map(|id| id.as_str().to_string()) .unwrap_or_else(|| artifact.artifact_id.0.clone()); + let valid_until = artifact + .applicability + .split(';') + .find_map(|part| { + part.trim() + .strip_prefix("valid_until=") + .map(|s| s.trim().to_string()) + }); + let provenance_digests: Vec = artifact + .provenance + .iter() + .filter(|p| p.relation == "supports") + .map(|p| { + json!({ + "uri": p.uri, + "locator": p.locator, + "content_digest": p.content_digest.as_ref().map(|d| &d.0), + }) + }) + .collect(); Ok(json!({"schema_version":1,"handle":{ "artifact_id": artifact.artifact_id.0, "symbol_id": symbol_id, "digest": artifact.hashes.artifact.0, + "closure_digest": artifact.hashes.closure.0, + "evidence_digest": artifact.hashes.evidence.0, "project": compilation.semantics.root, "policy": policy, "effects": effects, "evidence": artifact.verification.evidence, + "valid_until": valid_until, + "provenance": provenance_digests, "source": {"file": artifact.source.file, "line": artifact.source.line}, }})) } @@ -389,7 +634,27 @@ fn artifact_execute(params: &Value) -> Result { let candidates = artifact_query(&compilation.semantics, artifact_id); if candidates.len() != 1 { return Err(err("artifact.stale_handle", "制品不再可唯一解析;请重新 bind")); } let artifact = candidates[0]; - if artifact.hashes.artifact.0 != digest { return Err(json!({"code":"artifact.stale_handle","message":"源码或契约已改变;拒绝执行过期 handle","expected":digest,"actual":artifact.hashes.artifact.0})); } + if artifact.hashes.artifact.0 != digest { return Err(json!({"code":"artifact.stale_handle","message":"源码、契约或来源证据已改变;拒绝执行过期 handle","expected":digest,"actual":artifact.hashes.artifact.0})); } + if let Some(bound_closure) = handle.get("closure_digest").and_then(Value::as_str) { + if bound_closure != artifact.hashes.closure.0 { + return Err(json!({ + "code": "artifact.stale_handle", + "message": "依赖闭包已改变;拒绝执行过期 handle", + "expected": bound_closure, + "actual": artifact.hashes.closure.0, + })); + } + } + if let Some(bound_evidence) = handle.get("evidence_digest").and_then(Value::as_str) { + if bound_evidence != artifact.hashes.evidence.0 { + return Err(json!({ + "code": "artifact.stale_handle", + "message": "来源证据摘要已改变;拒绝执行过期 handle", + "expected": bound_evidence, + "actual": artifact.hashes.evidence.0, + })); + } + } if let Some(bound_symbol) = handle.get("symbol_id").and_then(Value::as_str) { let current = artifact .symbol_id @@ -405,6 +670,40 @@ fn artifact_execute(params: &Value) -> Result { })); } } + // Re-check policy at execute time (M02) — do not trust bind-time evidence string alone. + let policy = handle.get("policy").and_then(Value::as_str).unwrap_or("pure"); + let _ = parse_policy(policy)?; + if policy == "business" && !evidence_allows_business(&artifact.verification.evidence) { + return Err(json!({ + "code": "artifact.evidence_denied", + "message": "business 句柄在执行时证据状态不足(来源缺失、锚点失效或已过期)", + "policy": policy, + "evidence": artifact.verification.evidence, + })); + } + if policy == "pure" + && artifact + .effects + .values() + .iter() + .any(|effect| effect != "pure") + { + return Err(json!({ + "code": "artifact.effect_denied", + "message": "pure 句柄在执行时出现非纯效应", + "policy": policy, + "effects": artifact.effects.values(), + })); + } + if let Some(until) = handle.get("valid_until").and_then(Value::as_str) { + if crate::sema::provenance::valid_until_expired(until) { + return Err(json!({ + "code": "artifact.expired", + "message": "句柄适用期已过", + "valid_until": until, + })); + } + } if artifact.kind != "function" || artifact.callable.as_deref() == Some("instance_method") { return Err(err("artifact.not_executable", "只有非实例函数制品可由 artifact.execute 直接执行")); } let allowed: Vec<&str> = params.get("allow_effects").and_then(Value::as_array).map(|items| items.iter().filter_map(Value::as_str).collect()).unwrap_or_default(); let denied: Vec = artifact.effects.values().into_iter().filter(|effect| effect != "pure" && !allowed.contains(&effect.as_str())).collect(); diff --git a/src/project/mod.rs b/src/project/mod.rs index d982841..b648439 100644 --- a/src/project/mod.rs +++ b/src/project/mod.rs @@ -72,15 +72,26 @@ impl ProjectRoot { } } -const EXCLUDED_DIRS: &[&str] = &[ +/// Shared with EKC extract (M06): authored sources never include corpus/runtime bags. +pub const EXCLUDED_DIRS: &[&str] = &[ ".git", ".marqdo", "target", "node_modules", ".cursor", ".agents", + "data", + "docker", + "drafts", + "candidates", + "corpus", ]; +/// True when a directory basename is outside the authored source scan. +pub fn is_excluded_dir_name(name: &str) -> bool { + EXCLUDED_DIRS.iter().any(|excluded| name == *excluded) +} + /// Soft limits so discovery fails with a clear diagnosis instead of hanging (G12). pub const MAX_SOURCE_DEPTH: u32 = 32; pub const MAX_SOURCE_FILE_BYTES: u64 = 2 * 1024 * 1024; diff --git a/src/sema/ir.rs b/src/sema/ir.rs index 4fef9bc..169f039 100644 --- a/src/sema/ir.rs +++ b/src/sema/ir.rs @@ -572,9 +572,17 @@ fn artifact_for_function( content_digest: Some(implementation.clone()), relation: "defines".into(), }]; - let resolved = super::provenance::resolve_provenance(path, module_prov); - provenance.extend(resolved.refs); - let evidence = digest(contract_summary(&contract)); + let fn_sources = super::provenance::sources_for_function(module_prov, &function.name); + let mut fn_prov = module_prov.clone(); + fn_prov.sources = fn_sources; + let resolved = super::provenance::resolve_provenance_sources(path, &fn_prov, &fn_prov.sources); + provenance.extend(resolved.refs.clone()); + // M02: evidence digest includes contract + canonical provenance (URI/locator/content/applicability). + let evidence = digest(format!( + "{}|{}", + contract_summary(&contract), + resolved.evidence_record + )); let kind = match symbol.kind { SymbolKind::Type => "type", SymbolKind::Function => "function", @@ -1012,39 +1020,139 @@ fn direct_effects(calls: &[(usize, CallExpr, u32)]) -> EffectSet { /// Propagate callee effects along local `dependencies` edges until fixpoint. /// Empty effect sets become `pure` only after closure (no unknown remains). -/// Fixpoint closure digest: own artifact hash + sorted dependency closure hashes (G08). +/// +/// Deterministic SCC closure digests (M02/G08): +/// - Member seed = sorted `(artifact_id, interface, implementation)` within the SCC +/// - External = sorted closures of dependencies outside the SCC +/// - All members of an SCC share one closure digest fn close_closure_digests(artifacts: &mut [SemanticArtifact]) { let n = artifacts.len(); + if n == 0 { + return; + } let index: BTreeMap = artifacts .iter() .enumerate() .map(|(i, a)| (a.artifact_id.0.clone(), i)) .collect(); - for _ in 0..n.saturating_add(1) { - let mut changed = false; - let snap: Vec = artifacts - .iter() - .map(|a| a.hashes.closure.0.clone()) - .collect(); - for artifact in artifacts.iter_mut() { - let mut parts = vec![artifact.hashes.artifact.0.clone()]; - for dep in &artifact.dependencies { - if let Some(&i) = index.get(&dep.0) { - parts.push(snap[i].clone()); - } else { - parts.push(dep.0.clone()); + + // Tarjan SCC + let mut stack: Vec = Vec::new(); + let mut on_stack = vec![false; n]; + let mut indices = vec![-1i32; n]; + let mut lowlink = vec![0i32; n]; + let mut index_counter = 0i32; + let mut sccs: Vec> = Vec::new(); + + fn strongconnect( + v: usize, + artifacts: &[SemanticArtifact], + index_map: &BTreeMap, + stack: &mut Vec, + on_stack: &mut [bool], + indices: &mut [i32], + lowlink: &mut [i32], + index_counter: &mut i32, + sccs: &mut Vec>, + ) { + indices[v] = *index_counter; + lowlink[v] = *index_counter; + *index_counter += 1; + stack.push(v); + on_stack[v] = true; + for dep in &artifacts[v].dependencies { + if let Some(&w) = index_map.get(&dep.0) { + if indices[w] < 0 { + strongconnect( + w, + artifacts, + index_map, + stack, + on_stack, + indices, + lowlink, + index_counter, + sccs, + ); + lowlink[v] = lowlink[v].min(lowlink[w]); + } else if on_stack[w] { + lowlink[v] = lowlink[v].min(indices[w]); } } - parts.sort(); - parts.dedup(); - let next = digest(parts.join("|")); - if next.0 != artifact.hashes.closure.0 { - artifact.hashes.closure = next; - changed = true; + } + if lowlink[v] == indices[v] { + let mut scc = Vec::new(); + loop { + let w = stack.pop().expect("tarjan stack"); + on_stack[w] = false; + scc.push(w); + if w == v { + break; + } + } + scc.sort_unstable(); + sccs.push(scc); + } + } + + for v in 0..n { + if indices[v] < 0 { + strongconnect( + v, + artifacts, + &index, + &mut stack, + &mut on_stack, + &mut indices, + &mut lowlink, + &mut index_counter, + &mut sccs, + ); + } + } + + // sccs are in reverse topological order (callees before callers for DAGs). + let mut scc_of = vec![0usize; n]; + for (si, scc) in sccs.iter().enumerate() { + for &v in scc { + scc_of[v] = si; + } + } + let mut closure_done = vec![false; n]; + for scc in &sccs { + let mut member_parts = Vec::new(); + for &v in scc { + let a = &artifacts[v]; + member_parts.push(format!( + "{}|{}|{}", + a.artifact_id.0, a.hashes.interface.0, a.hashes.implementation.0 + )); + } + member_parts.sort(); + member_parts.dedup(); + let mut external = Vec::new(); + for &v in scc { + for dep in &artifacts[v].dependencies { + if let Some(&w) = index.get(&dep.0) { + if scc_of[w] != scc_of[v] && closure_done[w] { + external.push(artifacts[w].hashes.closure.0.clone()); + } else if scc_of[w] != scc_of[v] { + // External not yet closed — use artifact hash as stable seed. + external.push(artifacts[w].hashes.artifact.0.clone()); + } + } else { + external.push(dep.0.clone()); + } } } - if !changed { - break; + external.sort(); + external.dedup(); + let mut parts = member_parts; + parts.extend(external); + let scc_digest = digest(parts.join("|")); + for &v in scc { + artifacts[v].hashes.closure = scc_digest.clone(); + closure_done[v] = true; } } } diff --git a/src/sema/lock.rs b/src/sema/lock.rs index cb6fce1..fff9dff 100644 --- a/src/sema/lock.rs +++ b/src/sema/lock.rs @@ -111,7 +111,8 @@ pub fn check_links_lock(out_dir: &Path, project: &SemanticProject) -> Result Value { let mut seeds = BTreeSet::new(); for artifact in &project.artifacts { @@ -125,6 +126,15 @@ pub fn impact(project: &SemanticProject, query: &str) -> Value { .and_then(|s| s.to_str()) .is_some_and(|stem| stem == query || query.ends_with(stem)) || artifact.artifact_id.0.ends_with(query) + || artifact.provenance.iter().any(|p| { + p.relation == "supports" + && (p.uri == query + || p.uri.ends_with(query) + || query.ends_with(&p.uri) + || p.locator + .as_ref() + .is_some_and(|loc| format!("{}#{}", p.uri, loc) == query)) + }) { seeds.insert(artifact.artifact_id.0.clone()); } @@ -137,6 +147,22 @@ pub fn impact(project: &SemanticProject, query: &str) -> Value { .or_default() .push(artifact.artifact_id.0.clone()); } + // Corpus URI → citing artifact (provenance supports edges). + for prov in &artifact.provenance { + if prov.relation != "supports" { + continue; + } + reverse + .entry(prov.uri.clone()) + .or_default() + .push(artifact.artifact_id.0.clone()); + if let Some(loc) = &prov.locator { + reverse + .entry(format!("{}#{}", prov.uri, loc)) + .or_default() + .push(artifact.artifact_id.0.clone()); + } + } } // NarrativeRef edges: step → symbol; impact includes the step's owning artifact. for step in &project.steps { @@ -157,21 +183,54 @@ pub fn impact(project: &SemanticProject, query: &str) -> Value { .push(owner.clone()); } } + // Also seed reverse walk from corpus URI keys when query matches them. + if reverse.contains_key(query) { + seeds.insert(query.to_string()); + } let mut queue: VecDeque = seeds.iter().cloned().collect(); let mut affected = BTreeSet::new(); while let Some(current) = queue.pop_front() { - if !affected.insert(current.clone()) { continue; } + if !affected.insert(current.clone()) { + continue; + } if let Some(users) = reverse.get(¤t) { - for user in users { queue.push_back(user.clone()); } + for user in users { + queue.push_back(user.clone()); + } } } - let entries = affected.iter().filter_map(|id| project.artifacts.iter().find(|a| &a.artifact_id.0 == id)).map(|artifact| json!({ - "artifact_id": artifact.artifact_id.0, - "symbol_id": artifact.symbol_id.as_ref().map(|id| id.as_str()), - "display_name": artifact.display_name, - "kind": artifact.kind, - "source": {"file": artifact.source.file, "line": artifact.source.line}, - })).collect::>(); + // Drop pure corpus URI keys that are not artifacts from the affected set display. + let entries = affected + .iter() + .filter_map(|id| project.artifacts.iter().find(|a| &a.artifact_id.0 == id)) + .map(|artifact| { + let classes = { + let mut c = Vec::new(); + if !artifact.hashes.interface.0.is_empty() { + c.push("interface"); + } + if !artifact.hashes.implementation.0.is_empty() { + c.push("implementation"); + } + if !artifact.hashes.evidence.0.is_empty() { + c.push("evidence"); + } + if !artifact.dependencies.is_empty() { + c.push("dependency"); + } + c + }; + json!({ + "artifact_id": artifact.artifact_id.0, + "symbol_id": artifact.symbol_id.as_ref().map(|id| id.as_str()), + "display_name": artifact.display_name, + "kind": artifact.kind, + "source": {"file": artifact.source.file, "line": artifact.source.line}, + "hash_classes": classes, + "evidence": artifact.verification.evidence, + }) + }) + .collect::>(); json!({"schema_version":SCHEMA_VERSION,"query":query,"resolved":!seeds.is_empty(),"seeds":seeds,"affected":entries}) } diff --git a/src/sema/provenance.rs b/src/sema/provenance.rs index e5edd70..f88c5b8 100644 --- a/src/sema/provenance.rs +++ b/src/sema/provenance.rs @@ -1,4 +1,4 @@ -//! Optional source / applicability facts from module frontmatter (G05). +//! Optional source / applicability facts from module frontmatter (G05 / M02). //! //! Authors declare `sources:` and optional `applicability:` / `valid_until` in //! frontmatter. Absence means a pure tool may still run but must stay @@ -11,12 +11,19 @@ use sha2::{Digest as _, Sha256}; use super::ir::{Digest, SourceRef}; +/// Provenance extractor / verifier identity hashed into evidence digests. +pub const PROVENANCE_EXTRACTOR_VERSION: &str = "marqdo-provenance/1"; + #[derive(Debug, Clone, Default)] pub struct ModuleProvenance { /// Relative corpus paths, optional `#anchor`. pub sources: Vec, pub valid_until: Option, pub region: Option, + /// True when author wrote `sources: [...]` inline (ignored by parser — M07). + pub inline_sources_ignored: bool, + /// Optional per-function source overrides (M07): `function_sources: { name: [uri…] }`. + pub function_sources: std::collections::BTreeMap>, } /// Scan YAML-ish frontmatter for sources / applicability (no full YAML parser). @@ -28,6 +35,8 @@ pub fn parse_module_provenance(source: &str) -> ModuleProvenance { let mut out = ModuleProvenance::default(); let mut in_sources = false; let mut in_applicability = false; + let mut in_function_sources = false; + let mut current_fn: Option = None; for line in lines { let t = line.trim_end(); if t.trim() == "---" { @@ -40,21 +49,59 @@ pub fn parse_module_provenance(source: &str) -> ModuleProvenance { if !line.starts_with(' ') && !line.starts_with('\t') { in_sources = false; in_applicability = false; + in_function_sources = false; + current_fn = None; } if trimmed == "sources:" || trimmed.starts_with("sources:") { in_sources = true; in_applicability = false; + in_function_sources = false; if let Some(rest) = trimmed.strip_prefix("sources:") { let rest = rest.trim(); if !rest.is_empty() && rest != "|" { - // inline list not used; ignore + // inline list is not parsed — flag for diagnosis (M07). + out.inline_sources_ignored = true; } } continue; } + if trimmed == "function_sources:" || trimmed.starts_with("function_sources:") { + in_function_sources = true; + in_sources = false; + in_applicability = false; + current_fn = None; + continue; + } if trimmed == "applicability:" || trimmed.starts_with("applicability:") { in_applicability = true; in_sources = false; + in_function_sources = false; + continue; + } + if in_function_sources { + // Indent levels: ` name:` then ` - uri` + let indent = line.len() - line.trim_start().len(); + if trimmed.ends_with(':') && !trimmed.starts_with('-') { + let name = trimmed.trim_end_matches(':').trim().to_string(); + if !name.is_empty() { + current_fn = Some(name.clone()); + out.function_sources.entry(name).or_default(); + } + continue; + } + if let Some(item) = trimmed.strip_prefix("- ") { + let path = item.trim().trim_matches('"').to_string(); + if let Some(name) = ¤t_fn { + if !path.is_empty() { + out.function_sources + .entry(name.clone()) + .or_default() + .push(path); + } + } else if indent >= 2 && !path.is_empty() { + // ignore orphan list items + } + } continue; } if in_sources { @@ -77,26 +124,56 @@ pub fn parse_module_provenance(source: &str) -> ModuleProvenance { out } +/// Sources applicable to a named function: function_sources override, else module sources. +pub fn sources_for_function(prov: &ModuleProvenance, function_name: &str) -> Vec { + if let Some(list) = prov.function_sources.get(function_name) { + if !list.is_empty() { + return list.clone(); + } + } + prov.sources.clone() +} + #[derive(Debug, Clone)] pub enum ProvenanceDiag { Missing { path: String }, + AnchorMissing { path: String, locator: String }, + ReadError { path: String, message: String }, Expired { valid_until: String }, + InvalidDate { valid_until: String }, + InlineSourcesIgnored, } impl ProvenanceDiag { pub fn code(&self) -> &'static str { match self { Self::Missing { .. } => "source.missing", + Self::AnchorMissing { .. } => "source.anchor_missing", + Self::ReadError { .. } => "source.read_error", Self::Expired { .. } => "source.expired", + Self::InvalidDate { .. } => "source.invalid_date", + Self::InlineSourcesIgnored => "source.inline_ignored", } } pub fn message(&self) -> String { match self { Self::Missing { path } => format!("来源文件缺失:`{path}`"), + Self::AnchorMissing { path, locator } => { + format!("来源锚点不存在:`{path}` 中找不到 `#{}`", locator) + } + Self::ReadError { path, message } => { + format!("来源文件无法读取:`{path}`({message})") + } Self::Expired { valid_until } => { format!("来源适用期已过(valid_until={valid_until})") } + Self::InvalidDate { valid_until } => { + format!("适用期日期格式无效(期望 YYYY-MM-DD):`{valid_until}`") + } + Self::InlineSourcesIgnored => { + "sources 行内列表被忽略;请改用 YAML block list(`- path#anchor`)".into() + } } } } @@ -106,21 +183,35 @@ pub struct ProvenanceResolution { pub applicability: String, pub evidence_status: String, pub diagnostics: Vec, + /// Canonical record hashed into `hashes.evidence` (M02). + pub evidence_record: String, } /// Resolve corpus paths relative to the module file; never mark verified here. pub fn resolve_provenance( module_path: &Path, prov: &ModuleProvenance, +) -> ProvenanceResolution { + resolve_provenance_sources(module_path, prov, &prov.sources) +} + +/// Resolve an explicit source list (module or per-function). +pub fn resolve_provenance_sources( + module_path: &Path, + prov: &ModuleProvenance, + sources: &[String], ) -> ProvenanceResolution { let mut refs = Vec::new(); let mut diagnostics = Vec::new(); + if prov.inline_sources_ignored { + diagnostics.push(ProvenanceDiag::InlineSourcesIgnored); + } let base = module_path .parent() .map(|p| p.to_path_buf()) .unwrap_or_else(|| PathBuf::from(".")); - for raw in &prov.sources { + for raw in sources { let (path_part, locator) = match raw.split_once('#') { Some((p, a)) => (p.trim(), Some(a.trim().to_string())), None => (raw.as_str(), None), @@ -138,22 +229,56 @@ pub fn resolve_provenance( }); continue; } - let content = fs::read(&resolved).unwrap_or_default(); + let content = match fs::read(&resolved) { + Ok(bytes) => bytes, + Err(e) => { + diagnostics.push(ProvenanceDiag::ReadError { + path: path_part.to_string(), + message: e.to_string(), + }); + refs.push(SourceRef { + uri: path_part.to_string(), + locator, + content_digest: None, + relation: "supports".into(), + }); + continue; + } + }; let mut hash = Sha256::new(); hash.update(&content); + let content_digest = Digest(format!("sha256:{:x}", hash.finalize())); + if let Some(loc) = locator.as_ref() { + let text = String::from_utf8_lossy(&content); + if !anchor_exists(&text, loc) { + diagnostics.push(ProvenanceDiag::AnchorMissing { + path: path_part.to_string(), + locator: loc.clone(), + }); + } + } refs.push(SourceRef { uri: path_part.to_string(), locator, - content_digest: Some(Digest(format!("sha256:{:x}", hash.finalize()))), + content_digest: Some(content_digest), relation: "supports".into(), }); } if let Some(until) = &prov.valid_until { - if is_expired(until) { - diagnostics.push(ProvenanceDiag::Expired { - valid_until: until.clone(), - }); + match validate_date(until) { + DateCheck::Ok => { + if is_expired(until) { + diagnostics.push(ProvenanceDiag::Expired { + valid_until: until.clone(), + }); + } + } + DateCheck::Invalid => { + diagnostics.push(ProvenanceDiag::InvalidDate { + valid_until: until.clone(), + }); + } } } @@ -177,45 +302,168 @@ pub fn resolve_provenance( } }; - let evidence_status = if diagnostics.iter().any(|d| matches!(d, ProvenanceDiag::Missing { .. })) + let evidence_status = if diagnostics + .iter() + .any(|d| matches!(d, ProvenanceDiag::Missing { .. } | ProvenanceDiag::ReadError { .. })) { "missing_source".into() + } else if diagnostics + .iter() + .any(|d| matches!(d, ProvenanceDiag::AnchorMissing { .. })) + { + "anchor_missing".into() } else if diagnostics .iter() .any(|d| matches!(d, ProvenanceDiag::Expired { .. })) { "expired".into() - } else if prov.sources.is_empty() { + } else if diagnostics + .iter() + .any(|d| matches!(d, ProvenanceDiag::InvalidDate { .. })) + { + "invalid_date".into() + } else if sources.is_empty() { "unverified".into() } else { // Corpus present and in date — still not domain-verified by compile alone. "sourced".into() }; + let prov_for_record = ModuleProvenance { + sources: sources.to_vec(), + valid_until: prov.valid_until.clone(), + region: prov.region.clone(), + inline_sources_ignored: prov.inline_sources_ignored, + function_sources: Default::default(), + }; + let evidence_record = evidence_canonical_record(&prov_for_record, &refs); + ProvenanceResolution { refs, applicability, evidence_status, diagnostics, + evidence_record, } } +/// Canonical evidence inputs: extractor version, applicability, sorted source facts. +pub fn evidence_canonical_record(prov: &ModuleProvenance, refs: &[SourceRef]) -> String { + let mut parts = vec![ + format!("extractor={PROVENANCE_EXTRACTOR_VERSION}"), + format!("region={}", prov.region.as_deref().unwrap_or("")), + format!("valid_until={}", prov.valid_until.as_deref().unwrap_or("")), + ]; + let mut source_parts: Vec = refs + .iter() + .filter(|r| r.relation == "supports") + .map(|r| { + format!( + "{}#{}@{}", + r.uri, + r.locator.as_deref().unwrap_or(""), + r.content_digest + .as_ref() + .map(|d| d.0.as_str()) + .unwrap_or("none") + ) + }) + .collect(); + source_parts.sort(); + parts.extend(source_parts); + parts.join("|") +} + +/// Whether a Markdown heading / id matches the `#locator` fragment. +pub fn anchor_exists(content: &str, locator: &str) -> bool { + let loc = locator.trim().trim_start_matches('#').trim(); + if loc.is_empty() { + return true; + } + for line in content.lines() { + let t = line.trim(); + if let Some(rest) = t.strip_prefix('#') { + let heading = rest.trim_start_matches('#').trim(); + let heading = heading + .split('{') + .next() + .unwrap_or(heading) + .trim(); + if heading == loc || slugify_heading(heading) == loc { + return true; + } + } + if t.contains(&format!("id=\"{loc}\"")) + || t.contains(&format!("id='{loc}'")) + || t.contains(&format!("{{#{loc}}}")) + { + return true; + } + } + false +} + +fn slugify_heading(heading: &str) -> String { + let mut out = String::new(); + for ch in heading.chars() { + if ch.is_ascii_alphanumeric() { + out.push(ch.to_ascii_lowercase()); + } else if ch == ' ' || ch == '-' || ch == '_' { + if !out.ends_with('-') { + out.push('-'); + } + } else if !ch.is_ascii() { + // Keep non-ASCII letters (common in ZH docs) without lowercasing rules. + out.push(ch); + } + } + out.trim_matches('-').to_string() +} + +enum DateCheck { + Ok, + Invalid, +} + +fn validate_date(valid_until: &str) -> DateCheck { + let parts: Vec<&str> = valid_until.split('-').collect(); + if parts.len() != 3 { + return DateCheck::Invalid; + } + let Ok(y) = parts[0].parse::() else { + return DateCheck::Invalid; + }; + let Ok(m) = parts[1].parse::() else { + return DateCheck::Invalid; + }; + let Ok(d) = parts[2].parse::() else { + return DateCheck::Invalid; + }; + if !(1..=12).contains(&m) || !(1..=31).contains(&d) || y < 1970 { + return DateCheck::Invalid; + } + if parts[0].len() != 4 || parts[1].len() != 2 || parts[2].len() != 2 { + return DateCheck::Invalid; + } + DateCheck::Ok +} + fn is_expired(valid_until: &str) -> bool { - // Expect YYYY-MM-DD; compare lexicographically to today's UTC date. let today = chrono_today(); !valid_until.is_empty() && valid_until < today.as_str() } +/// Public recheck for handle.execute (M02). +pub fn valid_until_expired(valid_until: &str) -> bool { + matches!(validate_date(valid_until), DateCheck::Ok) && is_expired(valid_until) +} + fn chrono_today() -> String { - // Avoid new dependency: use system local date via `date` is bad; use - // SystemTime → rough days since epoch is fragile. Prefer `time` crate if - // present, else format via `js_sys` — check Cargo.toml. use std::time::{SystemTime, UNIX_EPOCH}; let secs = SystemTime::now() .duration_since(UNIX_EPOCH) .unwrap_or_default() .as_secs() as i64; - // UTC civil date from days since epoch (proleptic Gregorian). let days = secs.div_euclid(86_400); let (y, m, d) = civil_from_days(days); format!("{y:04}-{m:02}-{d:02}") @@ -248,4 +496,17 @@ mod tests { assert_eq!(p.region.as_deref(), Some("CN")); assert_eq!(p.valid_until.as_deref(), Some("2099-12-31")); } + + #[test] + fn anchor_matches_heading() { + assert!(anchor_exists("# rate\n\nrate = 0.13\n", "rate")); + assert!(!anchor_exists("# rate\n\nrate = 0.13\n", "missing-anchor")); + } + + #[test] + fn invalid_date_rejected() { + assert!(matches!(validate_date("2099-12-31"), DateCheck::Ok)); + assert!(matches!(validate_date("not-a-date"), DateCheck::Invalid)); + assert!(matches!(validate_date("2099-13-01"), DateCheck::Invalid)); + } } diff --git a/src/view/document.rs b/src/view/document.rs index e448892..22a4805 100644 --- a/src/view/document.rs +++ b/src/view/document.rs @@ -632,9 +632,10 @@ fn taken_arms_by_branch_line(annots: &[ViewAnnot]) -> HashMap { fn render_fun_heading(fun: &Function, annots: &[ViewAnnot]) -> String { let mut s = format!( - r#"
+ r##"
-

{name}

"#, +

{name}

+说明"##, line = fun.span.line, name = escape(&fun.name), ); diff --git a/src/view/html.rs b/src/view/html.rs index 9ff87a3..b74d6e5 100644 --- a/src/view/html.rs +++ b/src/view/html.rs @@ -58,6 +58,17 @@ fn stylesheet() -> &'static str { } * { box-sizing: border-box; } html { scroll-behavior: smooth; } +[id] { scroll-margin-top: 4.5rem; } +.knowledge-block summary { cursor: pointer; font-weight: 600; } +.knowledge-hint { color: var(--muted); font-size: 0.85em; margin: 0.25rem 0 0.5rem; } +.knowledge-error .err { color: #8b1a1a; } +@media print { + .topbar, .stdin-form, .stream-panel, .exec-clear, #notebook-bar, nav { display: none !important; } + .knowledge-block, .source-block { display: block; } +} +@media (prefers-reduced-motion: reduce) { + html { scroll-behavior: auto; } +} body { margin: 0; min-height: 100vh; @@ -2150,6 +2161,14 @@ pub fn layout(title: &str, nav: &str, main: &str) -> String { scrollIfPinned(); }} if (thinkingEl && (answerNode && answerNode.data)) thinkingEl.open = false; + // V2: reject stale result packets when page_token mismatches (source changed). + var pageTok = window.MARQDO_VIEW_PAGE_TOKEN || ""; + var pktTok = (typeof ev.page_token === "string") ? ev.page_token : ""; + var acceptPacket = !pktTok || !pageTok || pktTok === pageTok; + if (!acceptPacket) {{ + if (status) status.textContent = "stale result ignored"; + return; + }} if (typeof ev.bindings_html === "string") {{ var vp = document.getElementById("vars-panel"); if (vp) {{ @@ -2161,10 +2180,38 @@ pub fn layout(title: &str, nav: &str, main: &str) -> String { if (typeof ev.structure_html === "string" && ev.structure_html) {{ var structEl = document.getElementById("structure"); if (structEl) {{ + // Preserve call/loop frame selection across SSE structure swap. + var savedCall = {{}}; + structEl.querySelectorAll("select.call-picker").forEach(function(sel) {{ + if (sel.id) savedCall[sel.id] = sel.value; + }}); + var savedLoop = {{}}; + structEl.querySelectorAll("select.loop-picker").forEach(function(sel) {{ + if (sel.id) savedLoop[sel.id] = sel.value; + }}); + var focusId = (document.activeElement && document.activeElement.id) || ""; structEl.innerHTML = ev.structure_html; initAnnotLeaders(); initCallPickers(); initLoopPickers(); + Object.keys(savedCall).forEach(function(id) {{ + var el = document.getElementById(id); + if (el && savedCall[id]) {{ + el.value = savedCall[id]; + el.dispatchEvent(new Event("change", {{ bubbles: true }})); + }} + }}); + Object.keys(savedLoop).forEach(function(id) {{ + var el = document.getElementById(id); + if (el && savedLoop[id]) {{ + el.value = savedLoop[id]; + el.dispatchEvent(new Event("change", {{ bubbles: true }})); + }} + }}); + if (focusId) {{ + var fe = document.getElementById(focusId); + if (fe && fe.focus) try {{ fe.focus(); }} catch (e) {{}} + }} }} }} if (typeof ev.trace_json === "string" && ev.trace_json) {{ @@ -4153,11 +4200,12 @@ pub fn page_file(files: &[PathBuf], rel: &str, vm: &FileViewModel, links: &LinkM } } }; - let title = Path::new(rel) + let fallback_title = Path::new(rel) .file_name() .and_then(|s| s.to_str()) .unwrap_or(rel) .trim_end_matches(".mq.md"); + let title = crate::view::title_from_source(&vm.source, fallback_title); let plots_html = if vm.plots.is_empty() { String::new() } else { @@ -4224,43 +4272,59 @@ pub fn page_file(files: &[PathBuf], rel: &str, vm: &FileViewModel, links: &LinkM window.MARQDO_VIEW_LIVE = {live}; window.MARQDO_FILE_PATH = {path_js}; window.MARQDO_AUTO_STREAM = {auto_stream}; +window.MARQDO_VIEW_PAGE_TOKEN = {page_token_js};

{title}{status_label}

{rel}

{notebook} -{knowledge} -
-

Structure

+
+

文档

{structure_block}
+{knowledge}
-

Execution {exec_clear}

+

运行与结果 {exec_clear}

{stdin_panel} {stream_panel} {plan_card}
{out}
{plots}
-
-

Source

+
+ 完整源码
{source}
-
+ "#, live = live_flag, path_js = escape_js_string(rel), auto_stream = if vm.auto_stream { "true" } else { "false" }, + page_token_js = escape_js_string(&vm.page_token), trace = if trace_safe.is_empty() { "{}" } else { trace_safe.as_str() }, - title = escape(title), + title = escape(&title), status = status, status_label = status_label, rel = escape(rel), notebook = notebook, - knowledge = if vm.knowledge_html.is_empty() { String::new() } else { format!("

Executable knowledge

{}
", vm.knowledge_html) }, + knowledge = { + if !vm.knowledge_error.is_empty() { + format!( + "

知识关系

{}

", + escape(&vm.knowledge_error) + ) + } else if vm.knowledge_html.is_empty() { + String::new() + } else { + format!( + "
可执行知识(辅助){}
", + vm.knowledge_html + ) + } + }, exec_clear = exec_clear, structure_block = structure_block, stdin_panel = stdin_panel, diff --git a/src/view/knowledge_present.rs b/src/view/knowledge_present.rs new file mode 100644 index 0000000..43f5910 --- /dev/null +++ b/src/view/knowledge_present.rs @@ -0,0 +1,197 @@ +//! V1 KnowledgePresentation model — structured facts shared by body, outline, and detail. +//! +//! Semantic facts come from SemanticProject; this module does not invent verification. + +use crate::sema::{SemanticArtifact, SemanticProject}; +use serde_json::{json, Value}; +use sha2::{Digest, Sha256}; + +#[derive(Debug, Clone)] +pub struct KnowledgeNode { + pub node_id: String, + pub artifact_id: Option, + pub display_name: String, + pub kind: String, + pub source_file: String, + pub source_line: u32, + pub signature: Option, + pub effects: Vec, + pub evidence: String, + pub dependencies: Vec, + pub artifact_digest: String, +} + +#[derive(Debug, Clone)] +pub struct KnowledgePresentation { + pub document_id: String, + pub title: String, + pub source_digest: String, + /// V2: semantic generation / run matching token (empty when not compiled under CURRENT). + pub semantic_generation: String, + /// V2: page token for SSE stale-packet rejection (digest of document facts). + pub page_token: String, + pub execution_state: String, + pub nodes: Vec, + pub diagnostics: Vec, +} + +impl KnowledgePresentation { + pub fn from_project(project: &SemanticProject, rel: &str, title: &str) -> Self { + let mut nodes = Vec::new(); + for artifact in &project.artifacts { + if artifact.source.file != rel { + continue; + } + if artifact.kind == "module" { + continue; + } + nodes.push(node_from_artifact(artifact)); + } + let mut hasher_parts: Vec<&str> = project + .artifacts + .iter() + .filter(|a| a.source.file == rel) + .map(|a| a.hashes.artifact.0.as_str()) + .collect(); + hasher_parts.sort(); + let mut h = Sha256::new(); + h.update(project.project_id.as_bytes()); + h.update(rel.as_bytes()); + for part in &hasher_parts { + h.update(part.as_bytes()); + } + let digest_hex = format!("{:x}", h.finalize()); + let source_digest = format!("sha256:{digest_hex}"); + // page_token is filled by caller from raw source so SSE can match without recompile. + let page_token = String::new(); + let semantic_generation = { + let mut gh = Sha256::new(); + gh.update(project.project_id.as_bytes()); + for a in &project.artifacts { + gh.update(a.hashes.artifact.0.as_bytes()); + } + format!("{:x}", gh.finalize())[..16].to_string() + }; + let mut diagnostics = Vec::new(); + for n in &nodes { + if n.evidence == "sourced" { + // Never imply sourced == verified (V2). + diagnostics.push(format!( + "{}: 已关联来源,内容尚未核验", + n.display_name + )); + } + } + Self { + document_id: rel.to_string(), + title: title.to_string(), + source_digest, + semantic_generation, + page_token, + execution_state: "not_run".into(), + nodes, + diagnostics, + } + } + + pub fn to_json(&self) -> Value { + json!({ + "schema_version": 1, + "document_id": self.document_id, + "title": self.title, + "source_digest": self.source_digest, + "semantic_generation": self.semantic_generation, + "page_token": self.page_token, + "execution_state": self.execution_state, + "nodes": self.nodes.iter().map(|n| json!({ + "node_id": n.node_id, + "artifact_id": n.artifact_id, + "display_name": n.display_name, + "kind": n.kind, + "source": {"file": n.source_file, "line": n.source_line}, + "signature": n.signature, + "effects": n.effects, + "evidence": n.evidence, + "dependencies": n.dependencies, + "artifact_digest": n.artifact_digest, + "verified_claim": false, + })).collect::>(), + "local_deps": self.nodes.iter().flat_map(|n| { + n.dependencies.iter().map(|d| json!({ + "from": n.node_id, + "to": d, + })).collect::>() + }).collect::>(), + "diagnostics": self.diagnostics, + }) + } +} + +/// V2: whether an SSE result packet may update the live document structure. +pub fn sse_accepts_packet(page_token: &str, packet_token: Option<&str>) -> bool { + match packet_token { + None | Some("") => true, // legacy packets without token still apply + Some(_tok) if page_token.is_empty() => true, + Some(tok) => tok == page_token, + } +} + +/// Stable page token from raw source bytes (used by SSE run path). +pub fn page_token_from_source(rel: &str, source: &str) -> String { + let mut h = Sha256::new(); + h.update(rel.as_bytes()); + h.update(source.as_bytes()); + let hex = format!("{:x}", h.finalize()); + hex[..hex.len().min(32)].to_string() +} + +fn node_from_artifact(artifact: &SemanticArtifact) -> KnowledgeNode { + let signature = artifact.signature.as_ref().map(|sig| { + let params = sig + .params + .iter() + .map(|p| format!("{}:{}", p.name, p.ty)) + .collect::>() + .join(", "); + format!("({params}) → {}", sig.returns) + }); + KnowledgeNode { + node_id: artifact + .symbol_id + .as_ref() + .map(|id| id.as_str().to_string()) + .unwrap_or_else(|| artifact.artifact_id.0.clone()), + artifact_id: Some(artifact.artifact_id.0.clone()), + display_name: artifact.display_name.clone(), + kind: artifact.kind.clone(), + source_file: artifact.source.file.clone(), + source_line: artifact.source.line, + signature, + effects: artifact.effects.values(), + evidence: artifact.verification.evidence.clone(), + dependencies: artifact.dependencies.iter().map(|d| d.0.clone()).collect(), + artifact_digest: artifact.hashes.artifact.0.clone(), + } +} + +/// Pull `title:` from YAML frontmatter without a full YAML parser. +pub fn title_from_source(source: &str, fallback: &str) -> String { + let mut lines = source.lines(); + if lines.next().map(|l| l.trim_end()) != Some("---") { + return fallback.to_string(); + } + for line in lines { + let t = line.trim_end(); + if t.trim() == "---" { + break; + } + let trimmed = t.trim(); + if let Some(rest) = trimmed.strip_prefix("title:") { + let title = rest.trim().trim_matches('"').trim_matches('\'').to_string(); + if !title.is_empty() { + return title; + } + } + } + fallback.to_string() +} diff --git a/src/view/mod.rs b/src/view/mod.rs index 243aa40..64a9760 100644 --- a/src/view/mod.rs +++ b/src/view/mod.rs @@ -6,10 +6,15 @@ mod debug_api; mod debug_page; mod document; mod html; +mod knowledge_present; mod notebook_api; mod output; mod render; +pub use knowledge_present::{ + page_token_from_source, sse_accepts_packet, title_from_source, KnowledgePresentation, +}; + /// Surface rendering of an expression (labels/diagnostics reuse). `` M6 pub use render::expr_display; @@ -383,6 +388,18 @@ pub(crate) fn build_file_view( rel: &str, stdin_lines: &[String], live: bool, +) -> Result { + build_file_view_with_exec(root, abs, rel, stdin_lines, live, true) +} + +/// Like [`build_file_view`], but `execute=false` skips `run_file_capture` entirely (V0 / `--no-exec`). +pub(crate) fn build_file_view_with_exec( + root: &ViewRoot, + abs: &Path, + rel: &str, + stdin_lines: &[String], + live: bool, + execute: bool, ) -> Result { let mut source = fs::read_to_string(abs).with_context(|| format!("read {}", abs.display()))?; // Structure shows this file only — do not merge imported lib bodies into the tree. @@ -423,34 +440,43 @@ pub(crate) fn build_file_view( // Optional: sandbox to the whole view tree so sibling folders are reachable. opts.fs_root = Some(root.root.clone()); let mut captured: Option = None; - let (stdout, stderr, ok, plots, mut bindings_html) = if awaiting_input || auto_stream { - ( - String::new(), - String::new(), - true, - Vec::new(), - String::new(), - ) - } else { - match run_file_capture(abs, &opts) { - Ok(cap) => { - let stdout = cap.stdout.clone(); - let plots = cap.plots.iter().map(|p| p.svg.clone()).collect(); - let bindings_html = render_bindings_html(&cap.bindings); - captured = Some(cap); - (stdout, String::new(), true, plots, bindings_html) - } - Err(e) => ( + let (stdout, stderr, ok, plots, mut bindings_html) = + if !execute { + ( String::new(), - tidy_user_error(&format!("{e:#}"), abs, rel), - false, + "execution skipped (--no-exec)".into(), + true, + Vec::new(), + String::from(r#"

skipped

"#), + ) + } else if awaiting_input || auto_stream { + ( + String::new(), + String::new(), + true, Vec::new(), String::new(), - ), - } - }; + ) + } else { + match run_file_capture(abs, &opts) { + Ok(cap) => { + let stdout = cap.stdout.clone(); + let plots = cap.plots.iter().map(|p| p.svg.clone()).collect(); + let bindings_html = render_bindings_html(&cap.bindings); + captured = Some(cap); + (stdout, String::new(), true, plots, bindings_html) + } + Err(e) => ( + String::new(), + tidy_user_error(&format!("{e:#}"), abs, rel), + false, + Vec::new(), + String::new(), + ), + } + }; // Re-read after run so writeback slots / workbook edits show in Structure. - if !awaiting_input && !auto_stream { + if execute && !awaiting_input && !auto_stream { if let Ok(next) = fs::read_to_string(abs) { source = next; } @@ -480,9 +506,52 @@ pub(crate) fn build_file_view( }; // Analysis is intentionally independent from capture/run. If a document // has side effects, its facts remain visible while its values stay absent. - let knowledge_html = crate::sema::compile_project(&crate::sema::CompileOptions { - project_root: root.root.clone(), entry: Some(abs.to_path_buf()), lock_mode: crate::sema::LockMode::Off, - }).map(|compilation| render_knowledge_html(&compilation.semantics, rel)).unwrap_or_default(); + let title_guess = crate::view::title_from_source(&source, rel); + let (knowledge_html, knowledge_error, page_token) = + match crate::sema::compile_project(&crate::sema::CompileOptions { + project_root: root.root.clone(), + entry: Some(abs.to_path_buf()), + lock_mode: crate::sema::LockMode::Off, + }) { + Ok(compilation) => { + let html = render_knowledge_html(&compilation.semantics, rel); + // V1: also materialize structured presentation (JSON available to page scripts). + let mut presentation = crate::view::KnowledgePresentation::from_project( + &compilation.semantics, + rel, + &title_guess, + ); + presentation.page_token = crate::view::page_token_from_source(rel, &source); + let json = presentation.to_json().to_string(); + let with_model = if html.is_empty() { + format!( + r#""#, + json = json.replace("{json}"#, + html = html, + json = json.replace(" ( + String::new(), + format!("知识关系暂不可用:{}", tidy_user_error(&format!("{e:#}"), abs, rel)), + crate::view::page_token_from_source(rel, &source), + ), + }; + let page_token = if page_token.is_empty() { + crate::view::page_token_from_source(rel, &source) + } else { + page_token + }; Ok(FileViewModel { rel_path: rel.to_string(), source, @@ -500,6 +569,8 @@ pub(crate) fn build_file_view( bindings_html: std::mem::take(&mut bindings_html), trace_json, knowledge_html, + knowledge_error, + page_token, }) } @@ -752,6 +823,7 @@ fn api_run(root: &ViewRoot, mut request: Request) -> Result<()> { } Err(_) => (String::new(), String::new()), }; + let page_token = crate::view::page_token_from_source(&rel_clone, &source_now); let done = serde_json::json!({ "type": "done", "run_id": run_id, @@ -761,6 +833,7 @@ fn api_run(root: &ViewRoot, mut request: Request) -> Result<()> { "bindings_html": bindings_html, "structure_html": structure_html, "trace_json": trace_json, + "page_token": page_token, }); crate::host::event_bus::EventBus::global().publish_json(&done.to_string()); } diff --git a/src/view/output.rs b/src/view/output.rs index 0e538db..8aa894c 100644 --- a/src/view/output.rs +++ b/src/view/output.rs @@ -11,7 +11,7 @@ use crate::host::foreign_doc::DocumentSession; use crate::host::json::json_to_value; use crate::value::Value; use crate::view::html::{escape, page_file, page_index, LinkMode}; -use crate::view::{build_file_view, build_root, ViewRoot}; +use crate::view::{build_file_view, build_file_view_with_exec, build_root, ViewRoot}; pub struct OutputOptions { pub path: PathBuf, @@ -280,26 +280,8 @@ pub fn write_static(opts: OutputOptions) -> Result<()> { for rel_pb in &root.files { let rel = rel_pb.to_string_lossy().replace('\\', "/"); let abs = resolve_abs(&root, &rel)?; - let mut vm = build_file_view(&root, &abs, &rel, &[], false)?; - if opts.no_exec { - vm.stdout.clear(); - vm.stderr = "execution skipped (--no-exec)".into(); - vm.ok = true; - vm.plots.clear(); - vm.bindings_html = String::from(r#"

skipped

"#); - vm.trace_json.clear(); - // Rebuild outline + Structure without run annotations. - if let Ok(module) = crate::parse::parse_source(&vm.source) { - let wb = crate::view::render::render_writeback_outline(&vm.source); - vm.outline_html = - crate::view::render::render_right_rail(&module, &vm.bindings_html, &wb); - vm.structure_html = crate::view::render::render_module_structure_with_annots( - &module, - &vm.source, - &[], - ); - } - } + // V0: apply --no-exec before any capture run (do not execute then clear). + let vm = build_file_view_with_exec(&root, &abs, &rel, &[], false, !opts.no_exec)?; let links = LinkMode::Static { from: Some(rel.clone()), }; diff --git a/src/view/render.rs b/src/view/render.rs index 9dab63d..20179cf 100644 --- a/src/view/render.rs +++ b/src/view/render.rs @@ -44,23 +44,38 @@ pub struct FileViewModel { /// SVG plots from math lib (embedded in Execution). pub plots: Vec, /// HTML for Variables panel (entry bindings after run). + #[allow(dead_code)] pub bindings_html: String, /// ViewExecutionReport JSON for AI / call-picker (empty if no run). pub trace_json: String, /// Static semantic facts; generated without evaluating the document. pub knowledge_html: String, + /// Visible when semantic analysis fails (V0); empty when analysis succeeded or was skipped. + pub knowledge_error: String, + /// V2: page token for SSE stale-packet rejection (empty when unknown). + pub page_token: String, } /// A compact, textual (not colour-only) explanation of executable knowledge. /// View deliberately compiles semantics without running the target, so a page /// can show binding/effect/verification facts even in `--no-exec` exports. +/// +/// V0: skip module/type container cards by default; rename lock digest accurately. pub fn render_knowledge_html(project: &crate::sema::SemanticProject, rel: &str) -> String { - let artifacts: Vec<_> = project.artifacts.iter().filter(|artifact| artifact.source.file == rel).collect(); + let artifacts: Vec<_> = project + .artifacts + .iter() + .filter(|artifact| { + artifact.source.file == rel && artifact.kind != "module" && artifact.kind != "type" + }) + .collect(); let steps: Vec<_> = project.steps.iter().filter(|step| step.source.file == rel).collect(); - if artifacts.is_empty() && steps.is_empty() { return String::new(); } + if artifacts.is_empty() && steps.is_empty() { + return String::new(); + } let mut out = String::from(r#"
"#); if !steps.is_empty() { - out.push_str(r#"

Steps

    "#); + out.push_str(r#"

    文档中的步骤

    结构步骤(非实际执行时序)

      "#); for step in &steps { let label = project .refs @@ -93,12 +108,38 @@ pub fn render_knowledge_html(project: &crate::sema::SemanticProject, rel: &str) let signature = artifact.signature.as_ref().map(|sig| { let params = sig.params.iter().map(|param| format!("{}: {}{}", param.name, param.ty, if param.required { "" } else { "?" })).collect::>().join(", "); format!("({params}) → {}", sig.returns) - }).unwrap_or_else(|| "module/type declaration".into()); + }).unwrap_or_else(|| "declaration".into()); let effects = artifact.effects.values().join(", "); let deps = if artifact.dependencies.is_empty() { "none".to_string() } else { artifact.dependencies.iter().map(|id| id.0.as_str()).collect::>().join(", ") }; + // Short digest for display; full value in title for copy. + let digest = &artifact.hashes.artifact.0; + let digest_short = if digest.len() > 20 { + format!("{}…", &digest[..20]) + } else { + digest.clone() + }; + let evidence_note = match artifact.verification.evidence.as_str() { + "sourced" => "已关联来源,内容尚未核验", + "verified" => "证据已核验", + "unverified" => "尚无来源记录", + other => other, + }; out.push_str(&format!( - r#"

      {}

      Target
      {}
      Kind
      {}{}
      Signature
      {}
      Effects
      {}
      Verification
      parse: {}; resolve: {}; contract: {}; tests: {}; evidence: {}
      Lock digest
      {}
      Depends on
      {}
      Execution
      Not executed by knowledge preview.
      "#, - escape(&artifact.display_name), escape(target), escape(&artifact.kind), artifact.callable.as_ref().map(|callable| format!(" ({})", escape(callable))).unwrap_or_default(), escape(&signature), escape(&effects), escape(&artifact.verification.parse), escape(&artifact.verification.resolve), escape(&artifact.verification.contract), escape(&artifact.verification.tests), escape(&artifact.verification.evidence), escape(&artifact.hashes.artifact.0), escape(&deps), + r#"

      {name} {kind}

      Target
      {target}
      Signature
      {sig}
      Effects
      {effects}
      Verification
      parse: {parse}; resolve: {resolve}; contract: {contract}; tests: {tests}; evidence: {evidence}({evidence_note})
      制品摘要
      {digest_short} (未检查锁定状态)
      Depends on
      {deps}
      Execution
      尚无该知识的运行记录(知识预览不执行)。
      "#, + name = escape(&artifact.display_name), + kind = escape(&artifact.kind), + target = escape(target), + sig = escape(&signature), + effects = escape(&effects), + parse = escape(&artifact.verification.parse), + resolve = escape(&artifact.verification.resolve), + contract = escape(&artifact.verification.contract), + tests = escape(&artifact.verification.tests), + evidence = escape(&artifact.verification.evidence), + evidence_note = escape(evidence_note), + digest = escape(digest), + digest_short = escape(&digest_short), + deps = escape(&deps), )); } out.push_str("
"); diff --git a/tests/executable_knowledge.rs b/tests/executable_knowledge.rs index 71fa46a..6acbe53 100644 --- a/tests/executable_knowledge.rs +++ b/tests/executable_knowledge.rs @@ -302,9 +302,74 @@ fn s1_view_lists_steps_with_source_anchors() { || html.contains("data-step-kind=\"iterate\""), "expected classified step kinds in view HTML" ); + // V0: document body must appear before the knowledge auxiliary panel in main content + // (ignore stylesheet class mentions that appear earlier in