Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3cabe53da2 | ||
|
|
58e13ee6d3 | ||
|
|
1b11fe063a | ||
|
|
b792f44556 | ||
|
|
9ea5bebfc3 | ||
|
|
c7c329341b | ||
|
|
8cf5288481 | ||
|
|
69c4e74872 | ||
|
|
af0bda2e40 | ||
|
|
090b026547 | ||
|
|
03396afd02 | ||
|
|
663a8ba2c0 | ||
|
|
dc853fb580 | ||
|
|
0b80ff8e50 | ||
|
|
b8a4ddd38f | ||
|
|
352bc0d2c5 | ||
|
|
8a6d09d582 | ||
|
|
dcd3d329d4 | ||
|
|
0e1bfb3359 | ||
|
|
e9ae7ff1ae | ||
|
|
ecbdd18303 | ||
|
|
bedd53e212 | ||
|
|
7674ebb869 | ||
|
|
434be9fd52 |
No files matched your search
@@ -28,6 +28,7 @@ Canonical release playbook for **cflmy/marqdo**. Read this skill **before** tagg
|
||||
9. **Extension packs always publish to both** Cloudflare R2 (`https://ext.marqdo.com` via `scripts/upload-ext-r2.py`) **and** GitHub (Release assets / `ext-v*` tag). Credentials only in `~/.marqdo/r2.env` / CI — **never** commit. See [ext-cdn.md](../../../doc/design/ext-cdn.md).
|
||||
10. **CLI Ubuntu package = PPA** (`ppa:cflmy/marqdo`) via `./scripts/ppa-ship.sh` / `ppa-build-source.sh`. GitHub zips remain for Windows and non-apt Linux.
|
||||
11. **When a step needs the user** (sudo password, GPG passphrase, Launchpad UI, interactive `dput`): **stop, state what failed, and give the exact commands** to run in their terminal. Do not pretend the step succeeded.
|
||||
12. **GitHub auto-build must be green before declaring a release done.** For full/cli: wait for the tag-triggered **Release** workflow (`windows` + `linux`) **and** the push-triggered **`executable-knowledge`** workflow on the release commit / `main` tip. Release asset zips alone are **not** sufficient — a green Release job with a red EK gate is an incomplete release. Do not skip or reframe a red `executable-knowledge` as “assets already uploaded.”
|
||||
|
||||
## Release modes
|
||||
|
||||
@@ -78,6 +79,7 @@ Release progress:
|
||||
```
|
||||
|
||||
Checks: clean tree; `main` ff-only; smoke tests; CHANGELOG Unreleased; no secrets.
|
||||
Also confirm the latest `main` **`executable-knowledge`** Actions run is green (or that this release commit will be covered after push) — do not proceed to tag if EK is already red on tip without a fix in the same release PR.
|
||||
|
||||
## Phase 2 — Docs & version bumps (main)
|
||||
|
||||
@@ -129,7 +131,9 @@ git push https://proxy.cflmy.top/github.com/cflmy/marqdo.git vVER
|
||||
|
||||
## Phase 5 — GitHub Release notes & assets
|
||||
|
||||
1. Wait for Actions (full/cli): both `windows` and `linux` green when ext natives required.
|
||||
1. Wait for Actions (full/cli):
|
||||
- Tag **Release** workflow: both `windows` and `linux` green when ext natives required.
|
||||
- Push **`executable-knowledge`** workflow on the release commit / `main`: **must be success**. Poll with `gh run list --workflow=executable-knowledge.yml --branch main` / `gh run watch`. If EK is red → **stop**; fix plugin/harness on `main`, re-run; do **not** declare the release complete from assets alone.
|
||||
2. Rewrite release body 中英双语 — [reference.md § Notes](reference.md).
|
||||
3. Confirm Linux `marqdo-VER-native-x86_64-unknown-linux-gnu.zip` when mode ships ext.
|
||||
|
||||
@@ -174,10 +178,11 @@ On Launchpad failure: read the build log, fix `debian/` or bundle script, bump `
|
||||
## Phase 8 — Post-release
|
||||
|
||||
1. `gh release view TAG` — assets + notes.
|
||||
2. CDN `https://ext.marqdo.com/vVER/` (if ext shipped).
|
||||
3. PPA build green (if cli shipped).
|
||||
4. Tell user URLs: GitHub release, CDN, PPA install one-liner.
|
||||
5. Optional gh-pages only if asked.
|
||||
2. Confirm **`executable-knowledge`** and **Release** Actions are both green for this tag/commit; if either is red, say so and do not claim “发版完成”.
|
||||
3. CDN `https://ext.marqdo.com/vVER/` (if ext shipped).
|
||||
4. PPA build green (if cli shipped).
|
||||
5. Tell user URLs: GitHub release, CDN, PPA install one-liner.
|
||||
6. Optional gh-pages only if asked.
|
||||
|
||||
## Extension-only release (mode=ext)
|
||||
|
||||
@@ -194,7 +199,8 @@ On Launchpad failure: read the build log, fix `debian/` or bundle script, bump `
|
||||
| `gh` / git TLS or Clash `7890` refused | Unset bad proxy; see [reference § Proxy](reference.md) |
|
||||
| GitHub blocked | proxy.cflmy.top, then HK jump; **print commands** if agent cannot push |
|
||||
| Tag exists | Stop; ask user |
|
||||
| CI red | Fix on main; retag only if user allows |
|
||||
| CI red (Release assets) | Fix on main; retag only if user allows |
|
||||
| `executable-knowledge` red | Fix agent plugin / test harness (e.g. `ensure_agent_plugin_built`); push `main`; wait for EK green. **Do not** treat uploaded Release zips as a successful release |
|
||||
| R2 upload fails | Print `upload-ext-r2.py` command; do not skip silently |
|
||||
| PPA / sudo / GPG | Print `./scripts/ppa-ship.sh noble` (or `dput …`); do not skip silently |
|
||||
| Launchpad `Cargo.lock` v4 / old cargo | Ensure bundled `third_party/rust` via current `ppa-build-source.sh`; bump `1ppaN` |
|
||||
|
||||
@@ -304,6 +304,15 @@ gh workflow run Release --ref "vVER" # only if workflow_dispatch exists; else f
|
||||
|
||||
Current Release workflow is **tag-push only**. Red CI after tag → fix on `main`, then either new patch tag or **user-approved** delete+recreate tag (dangerous; prefer patch).
|
||||
|
||||
**Also required before declaring release done:** the push-triggered **`executable-knowledge`** workflow on the release commit / `main` tip must be **success**. Poll with:
|
||||
|
||||
```bash
|
||||
gh run list --workflow=executable-knowledge.yml --branch main --limit 5
|
||||
gh run watch <run-id>
|
||||
```
|
||||
|
||||
A green Release asset job with a red `executable-knowledge` gate is an **incomplete** release — fix the plugin/harness (e.g. `ensure_agent_plugin_built` for agent-dependent EK tests), push `main`, and wait for EK green. Do not skip.
|
||||
|
||||
### HK SSH jump (Paramiko CONNECT)
|
||||
|
||||
**When to use:** local → GitHub fails (timeout / TLS / Clash flaky / reverse-proxy `proxy.cflmy.top` connect drops), but this host can reach an overseas box that has stable GitHub (default `hk.cflmy.de`). Proven path for v0.3.9 push/tag/`gh`.
|
||||
|
||||
@@ -14,66 +14,49 @@ description: >-
|
||||
|
||||
Marqdo = **Markdown markers as syntax**. A `.mq.md` file is both documentation and a runnable program. Read this skill **before** inventing syntax from Markdown habits or Python.
|
||||
|
||||
Canonical design (repo): `doc/design/markdown-mapping.md`, `doc/design/keywords.md`. Deeper tables: [reference.md](reference.md). Copy-paste patterns: [examples.md](examples.md).
|
||||
<!-- CORE_SYNTAX_V2.1: must stay at the top so skill_brief budgets never truncate it -->
|
||||
|
||||
**Authoring / library craft** (GFM-first, no json glue, how to develop Marqdo): use **[marqdo-dev](../marqdo-dev/SKILL.md)** whenever writing or refactoring `.mq.md`, `lib/*`, or `ext/*`.
|
||||
## CORE SYNTAX (v2.1 Occlusion — mandatory)
|
||||
|
||||
**Product release** (tag, CHANGELOG, README, public docs, VSIX, GitHub assets): use [marqdo-release](../marqdo-release/SKILL.md) — detect latest version, **ask the user for the next SemVer**, then follow that playbook. Do not invent a version number.
|
||||
These rules are the **current default**. Do **not** teach Classic `[key](map)` index or “bold is return of print”. Classic migration only: [migration-classic.md](migration-classic.md).
|
||||
|
||||
1. **File suffix** `.mq.md`. Print with `> print text=…` / `> 打印 内容=…` — never use `**…**` to print.
|
||||
2. **`**…**` = code / return expression.** `*…*` = statement (bind / expr). Closing markers touch the last token (no trailing space).
|
||||
3. **`[描述](payload)` = occlusion:** `[]` is prose (never executed); `()` is the only executed part. Empty `()` → `None` stub.
|
||||
4. **Read collections with `access(coll, key)`** (aliases `取` / `取值`). **Never** use `[key](map)` as index under default Occlusion.
|
||||
5. **Foreach:** `- [描述](遍历 coll 为 item)` (`遍历`/`for`, `为`/`as`/`in`). Prefer explicit traversal when generating new code.
|
||||
6. **Import / call:** frontmatter `import:` / `导入:`; call `lib.member`. One language (EN or ZH) per file.
|
||||
7. **Control flow:** `#` object/type, `##`+ function, `+` params, `1.`/`2.` branches, `-` loops, `N. *` else. No invented `if`/`for`/`def`/`return` keywords.
|
||||
8. **Prose vs code:** blank line separates comment paragraphs from executable lines.
|
||||
9. **GFM tables** for data; prefer `table.put` over `json.set` chains. `ext/**` never calls `host_*`.
|
||||
10. Body narrative `[描述](target)` is lazy (NarrativeRef) — reading does not execute side effects.
|
||||
|
||||
| Marker | Meaning (v2.1) |
|
||||
|--------|----------------|
|
||||
| `#` / `##`+ | Object/type · function/method |
|
||||
| `*…*` | Statement |
|
||||
| `**…**` | Return / code expression |
|
||||
| `[描述](payload)` | Occlusion step (payload executes) |
|
||||
| `access(c, k)` | Collection read |
|
||||
| `> fn …` | Call |
|
||||
| GFM table | Collection literal |
|
||||
|
||||
Canonical: `doc/design/core-surface.md`, `doc/design/document-native-development-guide.md`. Deeper tables: [reference.md](reference.md). Patterns: [examples.md](examples.md). Authoring craft: [marqdo-dev](../marqdo-dev/SKILL.md). Release: [marqdo-release](../marqdo-release/SKILL.md).
|
||||
|
||||
Executable-knowledge gate: [`tests/executable_knowledge/`](../../tests/executable_knowledge/README.md).
|
||||
|
||||
## Hard rules (do not violate)
|
||||
|
||||
1. **File suffix** must be `.mq.md` (never plain `.md` for executable sources).
|
||||
2. **Output is not `**bold**`.** Print with `> print text=…` (or `> 打印 内容=…`). Bold `**…**` is **return value only**.
|
||||
3. **Prose vs code:** after a blank line, an unmarked first line starts a **comment paragraph**; every following non-blank line stays comment until the next blank line. Put a **blank line** between narration and executable lines.
|
||||
4. **Do not invent keywords** `if` / `else` / `while` / `for` / `def` / `return`. Control flow is Markdown: **`+` params** (under heading), **`1.` `2.` … branches**, **`-` loops**, arm `N. *` = else, `#` = **object/type**, `##`+ = **function/method**, `**…**` = return.
|
||||
5. **Identifiers use backticks** — params `` + `name` ``. **Exemptions:** foreach `` - [item](coll) ``; footnote `` name[^key] ``; **inside italic `*…*` / bold `**…**`**, bare ids are **variables** (Python-style unified namespace) and method-call receivers are bare too. **Drop the backticks on value-expression variable names and method receivers inside `*…*` / `**…**`** — they are redundant once the `*…*`/`**…**` markers mark the segment as code: `*分类 = 分类[^苹果]*`, `**n * 2**`, `*p = > page.主体装配 组件=home*`. **Text literals must be quoted**: `*a = > text.split value=src sep=","*`. Backticks **still required** where a bare word is *not* a variable in **standalone `>` calls** (bare = text there): `` > str `n` ``, `` > `obj`.method ``. **No trailing space** inside `*…*` / `**…**` wrapped code: the closing marker must touch the last token directly (`*a = 1*`, `**n**`), never `` *a = 1 * ``.
|
||||
6. **Structure lines are not wrapped in italics.** `#` `>` `+` `-` `|` lines stand alone; use `*…*` only for general statements (bindings / expressions).
|
||||
7. **Paths:** In bare *expressions* `/` is division. In **call args / param defaults / table cells**, unspaced `a/b` and quoted `".marqdo/agent-kb"` are path text — no `json.parse` needed. Numeric ratios that must stay text use quotes (`"1/5"`, `"16/9"`); bare `1/5` is division.
|
||||
8. Prefer ending side-effect-only function bodies with `*None*` / `*无*` / whole-line `**` / `****` when later lines must belong to an outer function. Lone `---` / `***` are Markdown thematic breaks (skipped), not function end.
|
||||
9. **`ext/**` never calls `host_*`.** Agent/OKF helpers are plugin names (`agent_kb_*`, …) after `plugin.load`. Do **not** add agent/OKF domain code to `src/host/` (core bloat). See `doc/design/ext-agent.md` §4.
|
||||
10. **Browser (route C/D/E/F):** client logic is Marqdo on WASM; official bridge is host glue and **may** implement lists, routing, storage, WebSocket, canvas, file read, observers — authors **must not hand-write business JS**. Use `web.client_embed` / `data-mq-source-url` auto-mount. Client effects: prefer **GFM tables + `lib/browser`** ([marqdo-dev](../marqdo-dev/SKILL.md)).
|
||||
11. **Code-as-documentation / no bag glue:** Prefer **GFM tables** for maps, lists, wire, commands. Prefer **`table.put` / named helpers** over `json.set` / `json.append` chains. `lib/json` is for parse/stringify/quote only — not a dict builder. Unreadable json pipelines are a style bug. Read collections with **`[key](map)`**, not footnote indexes for new code. In `markup: occlusion` files read with **`access(map, key)`** — there `[x](y)` is occlusion, not index. **Positive copy-paste shapes** (result map, soft-error pack, sparse `table.put`): [marqdo-dev](../marqdo-dev/SKILL.md) §2 · [examples.md](examples.md) §6e. Do **not** extend legacy `json.set` in `ext/ai` — refactor to tables.
|
||||
12. **One language per file:** Do not mix EN and ZH API names in the same `.mq.md` (imports, calls, params). Pick English *or* Chinese mirrors for the whole program.
|
||||
13. **Sandbox trial habit (Agent / generated `.mq.md`):** Prefer `agent.trial` / `run sandbox=True` / `sandbox.open`→`run`→`diff`→`commit` before promoting to the host tree. Host files must stay unchanged until a successful commit. Caps live in `sandbox.profile` GFM tables — do not invent a second dialect.
|
||||
1. Obey **CORE SYNTAX** above — Occlusion is default; do not emit Classic index links.
|
||||
2. **Identifiers:** params `` + `name` ``; inside `*…*` / `**…**` bare ids are variables. Quote text literals. Standalone `>` calls: backticks where bare would be text.
|
||||
3. **Structure lines** (`#` `>` `+` `-` `|`) are not wrapped in italics.
|
||||
4. **Paths:** bare `/` in expressions is division; in call args / table cells unspaced `a/b` is path text. Ratios as text need quotes (`"1/5"`).
|
||||
5. End side-effect-only helpers with `*None*` / `*空*` / `**` / `****` when the outer function continues. Lone `---` is a thematic break, not function end.
|
||||
6. **`ext/**` never calls `host_*`.** Prefer tables over `json.set` / `json.append`.
|
||||
7. **One language per file.** Sandbox trial before promoting Agent-generated trees.
|
||||
8. Browser: Marqdo on WASM + `lib/browser` tables; do not hand-write business JS.
|
||||
|
||||
|
||||
## Markup → meaning (v0.2)
|
||||
|
||||
| Marker | Meaning |
|
||||
|--------|---------|
|
||||
| `#` | **Object / type** (constructor body). `# main` = entry object. `# Child = > Parent` = inherit (**no** implicit parent constructor; call `` `self` = > Parent … `` explicitly when needed). |
|
||||
| `##` … `######` | **Function / method** (nesting by heading depth). |
|
||||
| `` + `name` `` under heading | Parameter (optional `` `name`=default ``) |
|
||||
| `1.` `2.` … in body | Branch arm (condition or `N. *` else). **Same-indent restart at `1.` = new branch statement** (not more arms of the previous list). |
|
||||
| `- …` inside body | Loop (`while` or `` [item](coll) `` / `` [`item`](`coll`) ``) |
|
||||
| Line `N. *` | Else arm |
|
||||
| `> fn args` | Call (named `k=v` or positional) |
|
||||
| `` > `obj`.method args `` | Method call (`obj` must be a map with `_type`) |
|
||||
| Frontmatter nested `import:` / `导入:` map | Import file (`bind: path.mq.md`) or short name (`bind: lib.member`) |
|
||||
| `> lib.member …` / `> lib.Type.member …` | Call via bare dotted path (instance methods need `` `var`.m ``) |
|
||||
| `*…*` | Statement (bind / expr) — closing `*` touches last token, **no trailing space**; bare ids are variables |
|
||||
| `**…**` | Return value — closing `**` touches last token, **no trailing space**; bare ids are variables |
|
||||
| `****` or `**` + spaces + `**` | Return `None` and end function body |
|
||||
| Lone `---` / `***` | Markdown thematic break (skip; not function end) |
|
||||
| GFM table after empty RHS bind | Collection; get with `access(coll, "key")` (legacy `` `m`[^key] `` remains available) |
|
||||
| `- [item](遍历 coll 为 item)` | Foreach |
|
||||
| `` ```lang `` | Foreign code block (via `lib/foreign`) |
|
||||
| `` `"text"` `` | Quoted string (`\n` `\t` `\\` `\"`; `` `var` `` inside); bare tokens unescaped |
|
||||
| Unmarked prose | Comment |
|
||||
|
||||
Entry: load file (often `index.mq.md`) → collect defs → run `# main`.
|
||||
|
||||
## Occlusion syntax (v2.1.0)
|
||||
|
||||
Occlusion is the default syntax. Shipped source declares `markup: occlusion` (or `标记: 遮蔽`) explicitly so imports stay deterministic. Legacy `[key](collection)` requires `markup: classic` or the one-cycle CLI flag `--legacy-markup`; `MARQDO_MARKUP` does not change parsing.
|
||||
|
||||
- `[描述](实现)` — `[]` is prose (never executed); `()` is the **only** executed part.
|
||||
- `[顾客留存率](0.42)` · `[总价](access(价目表, "苹果") * 数量)` · `[功能开发中]()` → `None` (stub; `label.empty_payload` warning).
|
||||
- Read maps/lists with `access(集合, 键)` (aliases `取` / `取值`): `[第二项](access(列, 2))`.
|
||||
- Foreach: `- [篮子里的每个果子](遍历 篮子 为 果)` (accepts `遍历`/`for` and `为`/`as`/`in`).
|
||||
- Calls inside `()`: `加(1)` / `加(1, 2)` / `加一 号(31)` / `txt.str_trim(" x ")` — `)` terminates, commas optional.
|
||||
- Tooling: `marqdo labels pending` (stub ledger) · `marqdo labels --grep 词 --count` · `marqdo migrate labels . --mark --write` · `marqdo check` reports `label.missing` / `label.empty_payload` / `label.ref_dangling`.
|
||||
|
||||
## Minimal program
|
||||
|
||||
```markdown
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
name: executable-knowledge
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
gate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- name: Build agent + llm plugins (EK / workbench gates)
|
||||
run: |
|
||||
cargo build -p marqdo_plugin_agent --locked
|
||||
cargo build -p marqdo_plugin_llm --locked
|
||||
- name: Workbench contracts (M00–M06 / F-subset)
|
||||
run: cargo test --test workbench_contracts --locked
|
||||
- name: Offline executable-knowledge gate
|
||||
run: cargo test --test executable_knowledge --locked
|
||||
- name: Isolated execute MR-01 (dual 60s SSE + short reads)
|
||||
run: |
|
||||
cargo build -p marqdo_plugin_llm --locked
|
||||
bash scripts/build-web-plugin.sh
|
||||
cargo test --test isolated_execute_mr01 --locked -- --nocapture
|
||||
timeout-minutes: 5
|
||||
- name: no ignored tests counted as success
|
||||
run: |
|
||||
if grep -n '#\[ignore\]' tests/executable_knowledge.rs; then
|
||||
echo "executable_knowledge must not use #[ignore] as success evidence" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "ok: no #[ignore] in executable_knowledge gate"
|
||||
@@ -126,9 +126,13 @@ jobs:
|
||||
## Marqdo ${{ github.ref_name }}
|
||||
|
||||
### 亮点 / Highlights
|
||||
- **v2.1.0**:文件级 Occlusion 默认语义——全仓迁移 · `migrate labels --mark` · `access` / `遍历` · 可执行知识语义投影
|
||||
- **v2.1.0**: File-scoped Occlusion by default — corpus migration, `migrate labels --mark`, `access` / `for`, executable-knowledge projections
|
||||
- **v2.1.0 ext**:扩展包同步 2.1.0 / Ext pack 2.1.0
|
||||
- **v2.1.4**:默认隔离 HTTP execute(ADR 0009)——worker 池 / `--call-lib-path` · `/_runtime/*` · MR-01 双流金样例 · `runtime.isolated_execution=1`
|
||||
- **v2.1.4**: Default isolated HTTP execute (ADR 0009) — worker pool / `--call-lib-path`, `/_runtime/*`, MR-01 dual-stream gold, `runtime.isolated_execution=1`
|
||||
- **v2.1.4 ext**:扩展包同步 2.1.4 / Ext pack 2.1.4
|
||||
- **v2.1.3**:知识工作台运行时——ModelSelection/RunContext/usage · view document 只读嵌入 · LLM selection/messages/cancel · agent selection e2e
|
||||
- **v2.1.3**: Workbench runtime — ModelSelection/RunContext/usage, view document read-only embed, LLM selection/messages/cancel, agent selection e2e
|
||||
- **v2.1.1**:可执行知识主链——BoundCall · NarrativeRef · 来源/适用期 · patch/extract · 整代 CURRENT + closure · 发票示例 113
|
||||
- **v2.1.1**: Executable-knowledge spine — BoundCall, NarrativeRef, sources, patch/extract, generation CURRENT + closure, invoice example 113
|
||||
- **v1.5.2**:外联 v3 P0/P1——raw stdin + 每 cell 输入 · plan API · snapshot · `view capture` · Persist
|
||||
- **v1.5.2**: Foreign v3 P0/P1 — raw stdin + per-cell inputs, plan API, snapshot export/import/replay, `view capture`, Persist
|
||||
- **v1.5.2 ext**:扩展包同步 1.5.2 / Ext pack 1.5.2
|
||||
|
||||
@@ -103,3 +103,4 @@ tests/ext/_sbx_cow/
|
||||
|
||||
tests/ext/_sbx_*/
|
||||
examples/**/_example_sbx/
|
||||
.secrets/
|
||||
+135
@@ -1,5 +1,67 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Added
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
|
||||
## v2.1.4 — 2026-10-26
|
||||
|
||||
### Highlights
|
||||
|
||||
**默认隔离 HTTP execute(ADR 0009)**:`class=execute` / `sse-live` / `stream:true` 经 Supervisor worker 池(`marqdo run --call-lib-path`),父进程不再 `CallLibPath` 占 `INVOKE_LOCK`;短读 / 未声明非流式仍 `legacy_serial`(ADR 0008)。`/_runtime/*` 控制面;`runtime.isolated_execution=1`;MR-01 双 60s 假 SSE + 短读金样例。CLI · wasm · ext **2.1.4**。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.4 && cargo build --release
|
||||
marqdo ext add web # CDN / Releases → 2.1.4
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——隔离运行时与 Web execute 路径,不新增 Markdown 标记。
|
||||
|
||||
### Added
|
||||
|
||||
- **默认隔离 execute(ADR 0009)**:HTTP `execute` / `sse-live` / `stream:true` 经 Supervisor worker 池(`marqdo run --call-lib-path`),父进程不再 `CallLibPath` 占 `INVOKE_LOCK`;可选 route `class`;`/_runtime/*` 控制面;`runtime.isolated_execution=1`。
|
||||
- **MR-01 金样例**:`tests/isolated_execute_mr01.rs` + `tests/ext/isolated-execute-site/`(双 60s 假 SSE + 短读并行;槽满拒绝);接入 `executable-knowledge` workflow。
|
||||
- **Worker CLI**:`marqdo run --call-lib-path` / `--fs-root` / `--args-json` / `--emit-result`;host `runtime` admit/dispatch/cancel/events。
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
|
||||
- **compile 1000 门禁**:executable-knowledge / workbench 路径恢复与能力探测诚实(isolated 默认开启)。
|
||||
- CLI / wasm / `ext/VERSION` **同步 2.1.4**。
|
||||
|
||||
## v2.1.3 — 2026-10-25
|
||||
|
||||
### Highlights
|
||||
|
||||
**知识工作台运行时合同(M00–M06)**:`ModelSelectionV1` / `RunContextV1` / `ModelCallUsageV1` / `DocumentViewV1` / locator / snapshot;严格 selection;usage null≠0;`view document` 只读嵌入;LLM messages + cancel;agent.step selection 继承假模型 e2e;USTC 兼容 live smoke(env 注入)。CLI · wasm · ext **2.1.3**。设计证据仍为 Partial(原生适配器 unavailable;完整宿主 F12 留给应用)。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.3 && cargo build --release
|
||||
marqdo ext add llm # CDN / Releases → 2.1.3
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——工作台合同与 View/LLM/Agent 扩展面,不新增 Markdown 标记。
|
||||
|
||||
### Added
|
||||
|
||||
- **Workbench M00–M06**:capabilities / selection / RunContext / usage / DocumentView / locator / SnapshotManifest;`tests/workbench_contracts`(含 F02 agent.step+resume、F14 并发);`marqdo view document --out` 只读 HTML 嵌入。
|
||||
- **LLM**:`messages=` / `selection=` / `strict_selection`;cancel token;reasoning_content 回退;双端点隔离门禁。
|
||||
- **Live**:`tests/workbench_live`(`OPENAI_*` env only;`MARQDO_EXT` 指向仓内 `ext/`)。
|
||||
- **Agent skeleton**:子工作簿禁 `llm.llm`,从 env selection `llm.create`。
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`llm.ask` 升参**:hoist `p` 避免缺参 `missing argument for parameter p`;避免 `~/.marqdo/ext` 陈旧包遮蔽仓内 `ext/`(测试强制 `MARQDO_EXT`)。
|
||||
|
||||
### Changed
|
||||
|
||||
- CLI / wasm / `ext/VERSION` **同步 2.1.3**。
|
||||
|
||||
## v2.0.0 — 2026-10-22
|
||||
|
||||
### Highlights
|
||||
@@ -69,6 +131,79 @@ marqdo ext add web
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Added
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
|
||||
## v2.1.2 — 2026-10-24
|
||||
|
||||
### Highlights
|
||||
|
||||
**可执行知识生产消费链 + View 阅读体验发版门槛**:证据摘要进闭包/句柄;`artifact.verify` / `knowledge adjudicate` 结构化裁决;`artifact.reuse_execute` 与 `agent_eng_bind_execute` / `publish_gate` 实际复用(非 n-gram);`document.extract` 文本/Markdown + PDF/OCR 显式失败;View `KnowledgePresentation`、说明锚点、SSE `page_token` 过期包拒绝;上传 `max_bytes`;门禁 45 项(含 100/1000 单元)。CLI · wasm · ext **2.1.2**。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.2 && cargo build --release
|
||||
marqdo ext add web
|
||||
cargo test --offline --locked --test executable_knowledge
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删关键词**——语义/MLSP/View/Agent 宿主扩展;不改变默认 Occlusion 链接读法与表几何。
|
||||
|
||||
### Added
|
||||
|
||||
- **证据与闭包(F0/M02)**:来源 digests 进入 artifact/evidence/closure;business handle 改/删来源失效;锚点/可读性诊断。
|
||||
- **验证裁决(F0/M03)**:`artifact.verify` 结构化 `may_bind` / `may_publish`;policy 枚举;`knowledge adjudicate`;CLI `--fail`。
|
||||
- **语法 Skill(F0/M05)**:CORE SYNTAX 置顶;`migration-classic.md`;`skill_core_card` 与预算截断防护。
|
||||
- **绑定执行(F1/M04)**:`artifact.reuse_execute`;`agent_eng_bind_execute` + `eng_publish_gate`(`candidate_only` / product gate);host_query `mlsp`;`tests/ext/agent-eng-bind-gate.mq.md`。
|
||||
- **发布与来源(F2/M06–M07)**:统一 `EXCLUDED_DIRS`;`publish_manifest.json`;`function_sources` 函数级来源覆盖。
|
||||
- **文档提取(F3/M01)**:`document.extract` DocumentSource→ExtractedDocument;PDF/DOCX/OCR 显式 `failed`;超限截断 `partial`;Web 上传 `MaxBytesReader` / `LimitReader`。
|
||||
- **View V0–V3(子集)**:正文优先;no-exec;`KnowledgePresentation`(`semantic_generation` / `page_token` / `local_deps`);SSE 过期包拒绝与帧选择保留;print/响应式;1000 单元 compile smoke。
|
||||
- **故事门禁(M08)**:建库→复用→来源变更失效→修复重绑。
|
||||
|
||||
### Fixed
|
||||
|
||||
- View 说明链接 raw-string 截断;skill_brief / CORE SYNTAX 预算不被冲突旧规则吃掉。
|
||||
- 上传体积限制在 multipart 解析前生效(不再只靠内存 form 预算)。
|
||||
|
||||
### Changed
|
||||
|
||||
- CLI / wasm / `ext/VERSION` **同步 2.1.2**。
|
||||
- `tests/executable_knowledge` 门禁扩至 45 项;README 切片进度表与发版子集对齐。
|
||||
|
||||
## v2.1.1 — 2026-10-23
|
||||
|
||||
### Highlights
|
||||
|
||||
**可执行知识主链落地(S0–S6)**:BoundCall 双后端共享解析、正文 NarrativeRef、来源/适用期、digest 保护补丁与纯表达式提取、整代 `CURRENT` 投影与闭包摘要;订单发票示例双后端输出 `113`;离线门禁含 100 单元语料与 CLI 冒烟。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.1 && cargo build --release
|
||||
marqdo ext add web
|
||||
marqdo run examples/executable-knowledge/docs/invoice-answer.mq.md
|
||||
```
|
||||
|
||||
**核心表面 Δ**:**无增删关键词**——同模块 `Type.member` 纳入 BoundCall;语义投影新增 `bound_calls` / `dynamic_calls` / `hashes.closure` / NarrativeRef 步骤;不改变默认 Occlusion 链接读法。
|
||||
|
||||
### Added
|
||||
|
||||
- **BoundCall / DynamicCall**:`sema::bind::resolve_for_exec`;Tree 与 Bytecode 静态路径对齐;`bound_calls.json` / `dynamic_calls.json`;同模块定义树路径绑定。
|
||||
- **NarrativeRef**:注释行 Markdown 链接 → `narrate` / `navigation` 步骤;围栏与行内代码不计步骤。
|
||||
- **来源与适用性**:frontmatter `sources` / `valid_until` → `evidence=sourced|expired|missing_source`;`artifact.bind policy=business`。
|
||||
- **局部编辑与提取**:MLSP `artifact.patch`(digest + wording 回滚);`artifact.extract_pure` 纯表达式预览。
|
||||
- **整代与闭包**:`generations/<id>/` + `CURRENT`;`hashes.closure` 随依赖变化;锁报告 `classes`。
|
||||
- **门禁**:`tests/executable_knowledge`(含 100 单元);CI `.github/workflows/executable-knowledge.yml`;项目扫描 depth/size/count 上限。
|
||||
- **示例**:`examples/executable-knowledge` 发票含税链可运行;负知识文档。
|
||||
|
||||
### Fixed
|
||||
|
||||
- Bytecode 裸名解析不再跨 import scope 误连同名函数;同模块 `Type.member` 写入 path_index 供兄弟调用。
|
||||
|
||||
### Changed
|
||||
|
||||
- Agent / reuse 发现结果标记 `discovery_only` + `bind_required` / `bind_targets`,不可把打分结果直接当可执行目标。
|
||||
|
||||
## v1.5.2 — 2026-10-18
|
||||
|
||||
### Highlights
|
||||
|
||||
Generated
+17
-16
@@ -1,6 +1,6 @@
|
||||
# This file is automatically @generated by Cargo.
|
||||
# It is not intended for manual editing.
|
||||
version = 4
|
||||
version = 3
|
||||
|
||||
[[package]]
|
||||
name = "adler2"
|
||||
@@ -264,9 +264,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "crossbeam-utils"
|
||||
version = "0.8.21"
|
||||
version = "0.8.23"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28"
|
||||
checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
|
||||
|
||||
[[package]]
|
||||
name = "crypto-common"
|
||||
@@ -539,9 +539,9 @@ checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
|
||||
|
||||
[[package]]
|
||||
name = "js-sys"
|
||||
version = "0.3.104"
|
||||
version = "0.3.106"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0e0c1080212aad755ea003d18543e8768dd432c48819efd73a7bf1e39b7a5a3a"
|
||||
checksum = "7883d941dae510fb2d978fc3fe018c71c9e2892fd38854de3e8b92c2e5ad9cc5"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"futures-util",
|
||||
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
|
||||
|
||||
[[package]]
|
||||
name = "marqdo"
|
||||
version = "2.1.0"
|
||||
version = "2.1.4"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"assert_cmd",
|
||||
@@ -589,6 +589,7 @@ dependencies = [
|
||||
"predicates",
|
||||
"pulldown-cmark",
|
||||
"regex",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha1",
|
||||
"sha2",
|
||||
@@ -610,7 +611,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "marqdo-wasm"
|
||||
version = "2.1.0"
|
||||
version = "2.1.4"
|
||||
dependencies = [
|
||||
"marqdo",
|
||||
"serde_json",
|
||||
@@ -1290,9 +1291,9 @@ checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen"
|
||||
version = "0.2.127"
|
||||
version = "0.2.129"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1b70935747edd64d89de3efa29d73789b806c15798f8e7dca4d8ac356b50ce70"
|
||||
checksum = "9bb54f33acc68fd454578d9820b0bde1a1a3d17aa17bb7b6595806d02886d409"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"once_cell",
|
||||
@@ -1303,9 +1304,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-macro"
|
||||
version = "0.2.127"
|
||||
version = "0.2.129"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "77775f8f3f7217702089053b94958f8f54061a3f663417df76e19cbdcca29bc1"
|
||||
checksum = "2e29d0c35b16e224a7eeb5cd2d25e3e1968fbd65604117b44d3b789d00ee8535"
|
||||
dependencies = [
|
||||
"quote",
|
||||
"wasm-bindgen-macro-support",
|
||||
@@ -1313,22 +1314,22 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-macro-support"
|
||||
version = "0.2.127"
|
||||
version = "0.2.129"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e11d33f857dc2fb11b8bc75aee111aa9cbeb12cd9f25efd3d4c2a3dd4e235284"
|
||||
checksum = "6f501a8bc3719dba86ef8ae4728879c08001bea749eb1333ac5b91e040e2a6b7"
|
||||
dependencies = [
|
||||
"bumpalo",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.119",
|
||||
"syn 3.0.3",
|
||||
"wasm-bindgen-shared",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-shared"
|
||||
version = "0.2.127"
|
||||
version = "0.2.129"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7ef64dbcc55df09c7e5a46182d181c2cfa3e925f3da937ea764728b4bbb9dcbf"
|
||||
checksum = "23f0c9c52aa7cd7d77769a4cfe2a9adb1b331f489a41d912ce14513d5ab995c6"
|
||||
dependencies = [
|
||||
"unicode-ident",
|
||||
]
|
||||
|
||||
+2
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "marqdo"
|
||||
version = "2.1.0"
|
||||
version = "2.1.4"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo — Markdown markers as an interpreted programming language"
|
||||
@@ -29,6 +29,7 @@ anyhow = "1"
|
||||
chrono = { version = "0.4", default-features = false, features = ["clock", "std"] }
|
||||
clap = { version = "=4.4.18", features = ["derive"], optional = true }
|
||||
pulldown-cmark = "0.12"
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
thiserror = "1"
|
||||
tiny_http = { version = "0.12", optional = true }
|
||||
|
||||
@@ -281,9 +281,13 @@ marqdo catalog public -o .marqdo
|
||||
|
||||
---
|
||||
|
||||
## 现状(v2.1.0)
|
||||
## 现状(v2.1.4)
|
||||
|
||||
- **链接语法(v2.1.0)**:`[描述](目标)` 描述不执行 · `()` 与 `**代码**` 同权(`加(1)` / `加一 号(31)`)· `access` 取元与 `遍历` 链接循环 · `marqdo labels` 两级检索 / `--baseline` 债务棘轮 · `migrate labels --mark` · 诊断 `label.*` · view 描述正文 + 代码片 + 桩徽章 · **可执行知识 [link-syntax.md](doc/design/link-syntax.md)**(项目索引、LinkInfo、链接图与 lock) · 扩展包同步 **2.1.0**
|
||||
- **默认隔离 HTTP execute(v2.1.4 · ADR 0009)**:`execute` / `sse-live` / `stream:true` → Supervisor worker 池(`--call-lib-path`);父进程不再占 `INVOKE_LOCK`;`/_runtime/*`;`runtime.isolated_execution=1`;MR-01 金样例;扩展包同步 **2.1.4**
|
||||
- **知识工作台运行时(v2.1.3)**:ModelSelection / RunContext / usage null≠0 · `view document` 只读嵌入 · LLM selection/messages/cancel · agent.step selection e2e · live smoke(设计 Partial:原生适配器 unavailable)
|
||||
- **可执行知识生产链(v2.1.2)**:证据/闭包失效 · `verify`/`adjudicate` · `reuse_execute` / `eng_bind_execute`+`publish_gate` · `document.extract` · View `KnowledgePresentation`+SSE `page_token` · 上传限流 · 门禁 45 项(100/1000 单元)
|
||||
- **可执行知识主链(v2.1.1)**:BoundCall 双后端 · NarrativeRef · 来源/适用期 · `artifact.patch` / `extract_pure` · 整代 `CURRENT` + closure · 发票示例 `113`
|
||||
- **链接语法(v2.1.0)**:`[描述](目标)` 描述不执行 · `()` 与 `**代码**` 同权(`加(1)` / `加一 号(31)`)· `access` 取元与 `遍历` 链接循环 · `marqdo labels` 两级检索 / `--baseline` 债务棘轮 · `migrate labels --mark` · 诊断 `label.*` · view 描述正文 + 代码片 + 桩徽章 · **可执行知识 [link-syntax.md](doc/design/link-syntax.md)**(项目索引、LinkInfo、链接图与 lock)
|
||||
- **外联 v3 P0/P1(v1.5.2)**:raw stdin + 每 cell 独立输入 · plan API · snapshot export/import/replay · `view capture` · Persist 按钮 · 扩展包同步 **1.5.2**
|
||||
- **view 静态/动态统一 + 外联值传递(v1.5.1)**:静态导出与 live view 共用 `foreign_doc` 执行路径 · 构建时运行注入 stdout · `foreign.session` 值传递 · cell stdin;扩展包同步 **1.5.0**
|
||||
- **裸自增升参(v1.4.6)**:首次粗体赋值左值即声明;`**n = n + 1**` 无反引号也升为必选形参。扩展包仍为 **1.4.5**
|
||||
@@ -319,7 +323,7 @@ marqdo catalog public -o .marqdo
|
||||
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
|
||||
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
|
||||
|
||||
### 如何使用最新 Marqdo(v2.1.0)
|
||||
### 如何使用最新 Marqdo(v2.1.4)
|
||||
|
||||
**安装解释器(任选其一)**
|
||||
|
||||
@@ -334,7 +338,7 @@ marqdo catalog public -o .marqdo
|
||||
```bash
|
||||
# 1) 源码安装(跟 tag 或 main)
|
||||
git clone https://github.com/cflmy/marqdo.git && cd marqdo
|
||||
git checkout v2.1.0 # 或留在 main
|
||||
git checkout v2.1.4 # 或留在 main
|
||||
cargo build --release
|
||||
export PATH="$PWD/target/release:$PATH"
|
||||
marqdo version
|
||||
@@ -373,7 +377,7 @@ marqdo wasm build
|
||||
# → dist/wasm/ … 见 examples/browser-hello/
|
||||
|
||||
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
|
||||
# https://github.com/cflmy/marqdo/releases/tag/v2.1.0
|
||||
# https://github.com/cflmy/marqdo/releases/tag/v2.1.4
|
||||
```
|
||||
|
||||
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "marqdo-wasm"
|
||||
version = "2.1.0"
|
||||
version = "2.1.4"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
|
||||
|
||||
Vendored
+31
@@ -1,3 +1,34 @@
|
||||
marqdo (2.1.4-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.4: default isolated HTTP execute (ADR 0009) — Supervisor
|
||||
worker pool for execute/sse-live/stream, /_runtime/* control plane,
|
||||
MR-01 dual 60s SSE gold sample, runtime.isolated_execution=1.
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Mon, 26 Oct 2026 10:00:00 +0000
|
||||
|
||||
marqdo (2.1.3-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.3: workbench M00–M06 runtime contracts (ModelSelection,
|
||||
RunContext, DocumentView, usage null≠0, LLM selection/cancel,
|
||||
agent.step selection e2e, view document CLI, live smoke).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Sun, 25 Oct 2026 15:20:00 +0000
|
||||
|
||||
marqdo (2.1.2-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.2: executable-knowledge F0–F3 + View V0–V3 release bar
|
||||
(evidence/closure, verify gates, bind/reuse_execute, document.extract,
|
||||
KnowledgePresentation, SSE page_token, upload max_bytes).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Sat, 24 Oct 2026 14:50:00 +0000
|
||||
|
||||
marqdo (2.1.1-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.1: executable-knowledge spine (BoundCall, NarrativeRef,
|
||||
sources/applicability, generation CURRENT + closure digests).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Fri, 23 Oct 2026 15:40:00 +0000
|
||||
|
||||
marqdo (2.1.0-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.0: file-scoped Occlusion is the default syntax; migrated
|
||||
|
||||
+15
-3
@@ -13,12 +13,22 @@
|
||||
|
||||
## 当前文档
|
||||
|
||||
> **可执行知识下一阶段的规范优先级**:实现 `[]()`、Semantic Artifact、知识绑定、验证、锁定、EKC/Agent 复用时,以 [next/Marqdo_Executable_Knowledge_Development.md](next/Marqdo_Executable_Knowledge_Development.md) 为实施基线。与其冲突的早期链接、遮蔽、知识分层或 EKC 设计只作历史参考。
|
||||
> **可执行知识下一阶段的阅读顺序**:[原始意图 009](next/009.md) → [文档原生开发指导](design/document-native-development-guide.md)(作者体验、语法审查结论、AI 工作协议)→ [详细开发规范](next/Marqdo_Executable_Knowledge_Development.md)(目标语义与工程约束)→ [缺口与交付路线](roadmap/executable-knowledge-gaps-2026-10-23.md)(现状证据、S0–S6)。指导文档修订作者负担、场景政策和交付次序;不得把所有内部治理字段变成作者必填项。与之冲突的早期设计只作历史参考,设计目标不等于已实现能力。
|
||||
>
|
||||
> **可执行知识门禁**:[tests/executable_knowledge/](../tests/executable_knowledge/README.md) — S0–S2 BoundCall / 效应 / MLSP bind,以及后续 NarrativeRef、来源与整代投影子集;`cargo test --test executable_knowledge --locked`(CI:`.github/workflows/executable-knowledge.yml`)。`status: design` / 预期失败与已实现分开报告。
|
||||
|
||||
| 文档 | 摘要 |
|
||||
|------|------|
|
||||
| [design/markdown-mapping.md](design/markdown-mapping.md) | **v0.2 语法宪法(现行运行时)** |
|
||||
| [design/markdown-mapping-v0.3.md](design/markdown-mapping-v0.3.md) | **v0.3 草案:叙述统一 · 粗体代码 · 斜体返回 · 形参体推断** |
|
||||
| [design/isolated-web-runtime-development-2026-10-26.md](design/isolated-web-runtime-development-2026-10-26.md) | **隔离执行与响应式 Web 运行时**:进程池、控制/读取分离、取消、事件恢复、发布一致性;实现见 `src/runtime/` |
|
||||
| [adr/0009-isolated-web-runtime.md](adr/0009-isolated-web-runtime.md) | **ADR 0009**:隔离 Web 运行时决策;金样例通过后替代 ADR 0008 的并行性结论 |
|
||||
| [跨库缺陷复核(2026-10-26)](../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md) | **最新源码与测试证据**:实际 HTTP 路由、知识发布/阅读断点、旧缺陷处置及 1000 单元编译性能失败;需同级 qdagent checkout |
|
||||
| [design/qdagent-workbench-runtime-and-view-2026-10-25.md](design/qdagent-workbench-runtime-and-view-2026-10-25.md) | **qdagent 知识工作台框架支持**:M00–M05 合同落地中(`src/workbench/`、MLSP `capabilities` / `view.document` / `view.reference_resolve`、`tests/workbench_contracts.rs`);完整 verified 以切片验收证据为准 |
|
||||
| [design/view-knowledge-reading-experience-2026-10-24.md](design/view-knowledge-reading-experience-2026-10-24.md) | **View 知识阅读体验重设计(建议)**:正文优先、结构导航、就地知识说明、来源与运行状态、响应式线框、实现契约与验收;含当前呈现和 no-exec 反例 |
|
||||
| [roadmap/qdagent-knowledge-support-gaps-2026-10-24.md](roadmap/qdagent-knowledge-support-gaps-2026-10-24.md) | **求道知识沉淀与文档编译支撑缺口**:2.1.1 源码复核、来源/依赖失效反例、验证与 Agent 协议、语法提示词、导入底座及跨库分工 |
|
||||
| [design/document-native-development-guide.md](design/document-native-development-guide.md) | **下一阶段开发指导**:语法审查裁决、正文→命名→复用工作流、AI 上下文与局部编辑、渐进严格、作者成本及 S0–S6 验收 |
|
||||
| [design/core-surface.md](design/core-surface.md) | **核心构造清单**:v2.1 默认链接/循环说明与构造守卫;不是全部实现状态清单 |
|
||||
| [design/markdown-mapping.md](design/markdown-mapping.md) | **历史 v0.2 映射**:不再代表现行默认链接语义 |
|
||||
| [design/markdown-mapping-v0.3.md](design/markdown-mapping-v0.3.md) | **v0.3 设计背景**:叙述统一 · 粗体代码 · 斜体返回 · 形参体推断;链接语义以 v2.1 规范和测试为准 |
|
||||
| [adr/0005-prose-unified-markup.md](adr/0005-prose-unified-markup.md) | **Proposed:采纳 mapping v0.3** |
|
||||
| [roadmap/markup-v0.3.md](roadmap/markup-v0.3.md) | **Markup v0.3 实现路线(M0–M5)** |
|
||||
| [design/objects.md](design/objects.md) | **`#` 对象 / `##` 函数与方法** |
|
||||
@@ -112,6 +122,8 @@
|
||||
| [design/executable-knowledge-system.md](design/executable-knowledge-system.md) | **可执行知识系统(v2.1 重构草案)**:每个 `[]()` 都是命名的可执行知识步骤 · 取元/推导/循环/调用/叙述统一建图 · 跨文件稳定寻址 · check/view/verify/impact 闭环 |
|
||||
| [next/Marqdo_Executable_Knowledge_Development.md](next/Marqdo_Executable_Knowledge_Development.md) | **可执行知识系统详细开发规范**:以 `next/009.md` 及两份附属文档为真实意图基线,冻结统一语义 IR、稳定寻址、绑定/验证/锁定、EKC/Agent 集成、迁移路线与验收矩阵 |
|
||||
| [next/Marqdo_Executable_Knowledge_Testing.md](next/Marqdo_Executable_Knowledge_Testing.md) | **可执行知识语法测试与兼容性规程**:标准库、扩展库、CLI、Tree/Bytecode 的仓库级门禁 |
|
||||
| [roadmap/executable-knowledge-gaps-2026-10-23.md](roadmap/executable-knowledge-gaps-2026-10-23.md) | **v2.1 现状与规划缺口审计(方向修订)**:保留 G01–G12 代码证据,以可读步骤、确定执行、早期 AI 复用和局部修改重排 S0–S6,严格治理按场景增加 |
|
||||
| [../tests/executable_knowledge/README.md](../tests/executable_knowledge/README.md) | **S0 作者基线门禁**:六类任务语料、作者成本计数、预期失败与 Bytecode 同名误连缺陷指纹 |
|
||||
| [design/knowledge-layers.md](design/knowledge-layers.md) | **知识分层(语法评审)**:原始语料 / 本体 / 可执行知识 / 调用面 · 单体知识类 vs 运行时实例 · `# 李白 = > 诗人` · 出处/还原验证 · EKC 映射 |
|
||||
| [design/occlusion-description-and-search.md](design/occlusion-description-and-search.md) | **遮蔽描述与两级检索**:内置词表=范例/默认,参数=扩展 · 空载荷一律 warning · 申报习惯 |
|
||||
| [design/occlusion-syntax.md](design/occlusion-syntax.md) | **遮蔽语法总纲(实施依据)**:`[描述](实现)` 的目的/语义/新鲜度校验 + M0–M8 计划 |
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
# ADR 0008:Web 热路径单解释器串行化(设计缺陷与缓解)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | **Accepted**(缺陷已记录;缓解已落地;真并行见 [ADR 0009](0009-isolated-web-runtime.md)) |
|
||||
| Date | 2026-10-26 |
|
||||
| Related | [web-invoke-serialization.md](../design/web-invoke-serialization.md) · [ADR 0009](0009-isolated-web-runtime.md) · ADR [0004](0004-web-plugin-go.md) · ADR [0007](0007-web-document-endpoint.md) · [ext-abi.md](../design/ext-abi.md) · [ext-web-go-rewrite.md](../design/ext-web-go-rewrite.md) |
|
||||
|
||||
## Context
|
||||
|
||||
- Go `plugins/web` 的 HTTP 服务是**多 goroutine** 的:每个请求可并发进入 `handleInvoke` / `handleInvokeLiveSSE`。
|
||||
- 每个 Endpoint 最终经 C ABI `host_query("call_lib_path", …)` 回到 Rust 宿主,在**同一个** `HostContext`(Marqdo 解释器状态)上执行作者 `.mq.md`。
|
||||
- `HostContext` **不是** `Sync`:共享可变会话、库装载、写回、事件总线等状态不能安全地被多线程同时求值。
|
||||
- 因此宿主用进程级 `INVOKE_LOCK`(`src/host/plugin.rs`)串行化所有来自 worker 线程的 `call_lib_path`。
|
||||
|
||||
### 常见误解
|
||||
|
||||
> 「web 都是单线程的吗?不能多线程吗?」
|
||||
|
||||
| 层 | 并发模型 |
|
||||
|----|----------|
|
||||
| Go HTTP / SSE flush / 取消 | **多线程(多 goroutine)** |
|
||||
| Marqdo 解释器 `call_lib_path` | **单飞(一把全局锁)** |
|
||||
| 浏览器 UI | 与后端锁无关;但若短 API 同步阻塞等待锁,会表现为「整页卡死」 |
|
||||
|
||||
**缺陷不在「HTTP 单线程」,而在「多 HTTP 工作线程共享一个非 Sync 解释器」。**
|
||||
真并行需要多解释器实例(或可共享的隔离运行时),不是去掉 Go 侧并发。
|
||||
|
||||
### 事故形态(已观测)
|
||||
|
||||
长对话(`return=sse-live` / `stream:true`)在 `CallLibPath` 期间一直持有 `INVOKE_LOCK`。并发的短路由(如 `/v1/models`、设置页探测)在旧实现里对锁 **无限期** `lock()`,HTTP 挂起 → 前端 `fetch` 无超时 → 页面无响应。重启进程可恢复,但属于运维掩盖,不是正确错误处理。
|
||||
|
||||
## Decision
|
||||
|
||||
1. **承认并文档化**本限制为宿主架构缺陷(单 `HostContext` + 全局 `INVOKE_LOCK`),不得再被表述为「web 插件只能单线程」。
|
||||
2. **缓解(已实现,非根治)**:
|
||||
- `call_lib_path` 从 worker 进入时用 **限时 try-lock**(默认 `MARQDO_INVOKE_WAIT_MS=1500`),超时返回错误串 `interpreter busy: …`。
|
||||
- Go `handleInvoke` 将含 `interpreter busy` 的错误映射为 **HTTP 503**,并设 `Retry-After: 2`。
|
||||
- **禁止**在 SSE 期间把进程级 `MARQDO_INVOKE_WAIT_MS` 拉长:那会让并发短请求重新卡死。
|
||||
- 长 SSE 在启动时用同一短预算争锁;一旦拿到则持有至调用结束(对话本来就要跑完)。
|
||||
3. **客户端契约**:短 API 必须有超时与 503 文案;不得假设「任意时刻都能同步进入解释器」。
|
||||
4. **根治(未决,另案)**:解释器池 / per-request isolate / 可 `Send` 的会话切片。未落地前,产品行为以本 ADR 的 503 语义为准。
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- 对话进行中,设置/模型列表等短请求约 1.5s 内失败并可读,UI 不再无限挂起。
|
||||
- 缺陷有名有姓,后续多解释器工作不会被「修好了」的假象挡住。
|
||||
|
||||
### Negative / Risks
|
||||
|
||||
- 同一进程仍无法真正并行跑两个 Marqdo Endpoint 体;第二个长 SSE 也会在短预算后失败。
|
||||
- 极短窗口内,短请求若正好撞上锁交接,可能偶发 503(可重试)。
|
||||
- `MARQDO_INVOKE_WAIT_MS` 是进程环境变量;误设过大仍会复现卡顿。
|
||||
|
||||
### Neutral
|
||||
|
||||
- Go 多 goroutine、SSE 桥(`MARQDO_EVENT_FORWARD`)、取消令牌(`MARQDO_CANCEL_TOKEN`)保持不变。
|
||||
- 不改变 Document / Endpoint 作者面(ADR 0007);只约束宿主与 web 插件错误语义。
|
||||
|
||||
## Implementation pointers
|
||||
|
||||
| 位置 | 职责 |
|
||||
|------|------|
|
||||
| `src/host/plugin.rs` → `acquire_invoke_lock` | 限时争锁;失败 `set_err("interpreter busy…")` |
|
||||
| `plugins/web/.../routes_invoke.go` → `handleInvoke` | `interpreter busy` → 503 + `Retry-After` |
|
||||
| 设计长文 | [web-invoke-serialization.md](../design/web-invoke-serialization.md) |
|
||||
@@ -0,0 +1,31 @@
|
||||
# ADR 0009: Isolated Web Runtime (worker process pool)
|
||||
|
||||
| 项目 | 约定 |
|
||||
|---|---|
|
||||
| 状态 | **Accepted**;HTTP **execute** 路径正式替代 [ADR 0008](0008-web-invoke-serialization.md) 的并行性结论。`legacy_serial`(未声明且非流式)仍引用 0008 单飞 / 父进程 `CallLibPath`。 |
|
||||
| 日期 | 2026-10-26 |
|
||||
| 设计 | [isolated-web-runtime-development-2026-10-26.md](../design/isolated-web-runtime-development-2026-10-26.md) |
|
||||
| 金样例 | `tests/isolated_execute_mr01.rs`(MR-01 双 60s SSE + short reads;同文件 MR-02 槽满拒绝) |
|
||||
|
||||
## 背景
|
||||
|
||||
ADR 0008 确认:Go HTTP 多线程,但所有 Endpoint 共享一把解释器锁与进程级 SSE 桥环境变量。1500ms busy → 503 只是缓解,不是并行。
|
||||
|
||||
## 决策
|
||||
|
||||
1. **首选架构**:预热的一次性 Marqdo worker 进程池;每个租约独占一个进程与 HostContext。
|
||||
2. **HTTP 默认**:`class=execute` 或 `return=sse-live` 或 body `stream:true` → `admit` → `dispatch`(`--call-lib-path` worker);禁止父进程 `CallLibPath` / `Setenv` / `legacySSEBridgeMu`。`short` / `read` / `control` 仍父进程短路径;未声明非流式 → `legacy_serial`。
|
||||
3. **控制/读取分离**:取消与事件订阅不进入任务解释器锁;短业务与长执行有独立容量(execute 槽默认 2)。
|
||||
4. **协议**:`InvokeRequestV2` / worker env + NDJSON `MARQDO_EVENT_FORWARD` + `--emit-result`(首期);`/_runtime/*` 控制面。
|
||||
5. **能力探测**:`runtime.isolated_execution=1`(默认)、`runtime.task_control_v2`、`runtime.event_replay`、`view.analysis_cache_v2`;`runtime.read_lane` 仍 0。
|
||||
6. **发布**:`expected_generation` CAS + publish journal;逻辑文档 ID 与物理 generation 路径解耦。
|
||||
|
||||
## 后果
|
||||
|
||||
- Execute / live SSE 不再抢进程级 SSE 桥;双流可并行(MR-01)。
|
||||
- Legacy 未声明非流式路径仍可走父进程锁(ADR 0008)。
|
||||
- 产品(qdagent)cancel / Last-Event-ID 接线仍为后续;不阻碍本框架发版。
|
||||
|
||||
## 验收(摘要)
|
||||
|
||||
MR-01 绿:双假上游 60s 流不混字、短读全 200、两 worker `--call-lib-path`、第三路槽满 429/503、`runtime.isolated_execution=1`。其余见设计文档 §12。
|
||||
@@ -1,5 +1,7 @@
|
||||
# 语义评估:参数能否从上下文推断(含叙述式调用)
|
||||
|
||||
> **2026-10-23 审阅裁决**:保留参数来源可见、避免隐藏调用者环境的边界;下文已被否决的 L0 建议不可重新启用。自然语言只连接和解释,不能成为隐式执行触发器;当前指导见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **§3 的 L0 方案已被否决**(违反「只执行标记」原则);L1/L2 结论仍有效 |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 评定:`[描述](语句)` —— 方括号读作「人话」,圆括号读作「机器话」
|
||||
|
||||
> **2026-10-23 审阅裁决**:本文的 Classic 冲突统计和兼容取舍属于迁移历史;v2.1 默认采用描述包裹 payload,不能据旧冲突恢复默认 `[键](集合)`。保留标签的阅读/检索/步骤价值,不强制所有子表达式命名,也不把标签当作目标或权限依据。参见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **已决策:走「路线二 纯度」**(落地见 [label-freshness-and-repair.md](label-freshness-and-repair.md)) |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 调用实参:具名与位置
|
||||
|
||||
> **2026-10-23 适用范围校正**:本文保留早期调用尾部规则和历史示例,不是 v2.1 全部调用语法。当前已有 `f(args)` 及具名实参;默认紧邻 `](` 是描述包裹 payload,不再是下文所述的 Classic 取元。语法审阅与开发取舍见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓),当前行为须由默认模式测试复核。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | 已定(v0.1 增补;2026-09-14:方括号标记调用) |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 语法设计:标记如何构成调用(在「只执行标记」原则下)
|
||||
|
||||
> **2026-10-23 审阅裁决**:保留显式标记和正文默认安全;性能数值是旧实验,不作为当前基准。持久跨文档接口优先具名参数,但不禁止位置参数;反引号声明与顺序改变可能是接口变化。当前调用与开发方向见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 待决策**(未改代码) |
|
||||
|
||||
@@ -3,22 +3,24 @@
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · 核心语法封闭集合(CI 守卫锁定)** |
|
||||
| 日期 | 2026-09-22 |
|
||||
| 日期 | 2026-09-22;2026-10-23 校正 v2.1 默认链接语义 |
|
||||
| 守卫 | `tests/core_surface.rs` ⇄ `src/parse/mod.rs` `CORE_CONSTRUCTS`(改动任一侧不同步 → CI 红) |
|
||||
| 相关 | [markdown-mapping-v0.3.md](markdown-mapping-v0.3.md)(语义宪法)· [layers.md](layers.md)(准入戒律)· [three-problems.md](../roadmap/three-problems.md) 问题一 · [three-problems-plan.md](../roadmap/three-problems-plan.md) T1.1/T1.3 |
|
||||
|
||||
> **戒律总纲**:核心标记是**封闭集合**(约 19 个构造)。新增任何一行必须走 ADR(见 [layers.md](layers.md) 准入戒律)并回答「为什么现有标记不够」;生态需求的默认答案是**进 ext**。本表 id 列与守卫测试强一致:改表不同步 `CORE_CONSTRUCTS`(或反之)、样例文件缺失,一律 CI 红。
|
||||
|
||||
> **版本边界**:构造 ID 是历史守卫标识,不是固定不变的语义说明。`link-index` ID 暂保留以匹配注册表,但默认 `[]()` 已是描述包裹 payload;旧索引语义仅属于显式 Classic 兼容路径。未来步骤/知识投影见[文档原生指导](document-native-development-guide.md),不要把目标能力当作当前已实现。核心数量之外,还要审查上下文例外和作者负担。
|
||||
|
||||
## 1. 行构造
|
||||
|
||||
| id | 外形 | 语义 | 戒律 | 样例 |
|
||||
|----|------|------|------|------|
|
||||
| `narrative` | 无标记文字 | 叙述(注释) | 段落内延续到空行;**默认安全**——永远不存在「叙述被执行」的路径 | `tests/markup-v03/soft-emphasis.mq.md` |
|
||||
| `narrative` | 无标记文字 | 叙述(不执行) | 段落内延续到空行;普通叙述不触发执行;未来知识引用投影也不改变此权限边界 | `tests/markup-v03/soft-emphasis.mq.md` |
|
||||
| `blank` | 空行 | 段落边界 | 叙述分段的唯一手段;不得被赋予其它语义 | `tests/markup-v03/inc-prose.mq.md` |
|
||||
| `heading-unit` | `#` / `##` | 对象 / 函数单元 | 标题即单元边界,按深度嵌套;正文禁止用 `##` 做版式(会变成函数) | `tests/markup-v03/decl-ident-only.mq.md` |
|
||||
| `call-line` | `>` | 调用行 | 输出等副作用 = **普通函数**,不是架构标记 | `tests/markup-v03/dual-call.mq.md` |
|
||||
| `branch-list` | 体内 `1.` 递增 | 分支 | 自 `1.` 起序号递增构成一个判断;重出 `1.` 开新判断;`N. *` = else;**不是所有有序列表都是 if** | `tests/markup-v03/greet-default.mq.md` |
|
||||
| `loop-list` | `-` | 循环 | 条件形 `` - `条件` `` 或遍历形 `- [项](集合)` | `tests/structure/loop.mq.md` |
|
||||
| `loop-list` | `-` | 循环 | 条件形 `` - `条件` `` 或遍历形 `- [说明](遍历 集合 为 项)`;描述不是绑定变量名 | `tests/structure/loop.mq.md` |
|
||||
| `param` | `` + `名` `` | 形参(兼容形) | 首选叙述 `` `名` `` 升参(宪法 §8);`+` 形仅兼容保留 | `tests/markup-v03/greet-default.mq.md` |
|
||||
| `table` | GFM 表 | 集合 | 几何即类型([tables-maps-footnotes.md](../roadmap/tables-maps-footnotes.md));**绑定 = 数据,未绑定 = 文档表** | `tests/structure/collection.mq.md` |
|
||||
| `fence` | ` ```lang ` | 外联 | 语言块原样外联,不进 Marqdo 语义 | `tests/lib/foreign-python.mq.md` |
|
||||
@@ -30,12 +32,12 @@
|
||||
| id | 外形 | 语义 | 戒律 | 样例 |
|
||||
|----|------|------|------|------|
|
||||
| `ident` | `` `名` `` | 声明 / 引用 | 首次未赋值 = 声明;叙述与程序共用同一视觉习惯 | `tests/markup-v03/decl-ident-only.mq.md` |
|
||||
| `bold-code` | `**…**` | 代码(赋值 / 调用 / 表达式) | 只有粗体进入执行;判别序:赋值 → 调用 → 表达式 | `tests/markup-v03/multi-bold.mq.md` |
|
||||
| `bold-code` | `**…**` | 代码(赋值 / 调用 / 表达式) | 粗体内代码判别序:赋值 → 调用 → 表达式;不取消调用行、返回等其它显式执行入口 | `tests/markup-v03/multi-bold.mq.md` |
|
||||
| `italic-return` | `*…*` | 返回 | 改变控制流(**返回是架构**);返回即结束函数体 | `tests/markup-v03/inc-prose.mq.md` |
|
||||
| `empty-return` | `*None*` / `****` | 空返回 | 返回 `None` 并结束函数体 | `tests/markup-v03/empty-return-none.mq.md` |
|
||||
| `link-index` | `[键](集合)` | 取元 / foreach | **主取元语法**;与 bracket-call 靠 `]` 后是否紧邻 `(` 消歧 | `tests/lib/table-collections.mq.md` |
|
||||
| `bracket-call` | `修饰… [函数] 实参…` | 方括号标记调用 | 前置修饰 → `名=True`;与 link-index 消歧同上 | `tests/structure/bracket-call.mq.md` |
|
||||
| `footnote-index` | `集合[^键]` | 过渡取元 | **仅兼容**;新代码一律写 link-index | `tests/structure/footnote-index.mq.md` |
|
||||
| `link-index` | `[描述](表达式)` | 描述包裹 payload(历史 ID) | 默认 Occlusion;取元可写 `[说明](access(集合, 键))`,不按描述取键;循环载荷见上 | `tests/occlusion/access-and-loop.mq.md` |
|
||||
| `bracket-call` | `修饰… [函数] 实参…` | 方括号标记调用 | 前置修饰 → `名=True`;紧邻 `](` 的默认语义为 payload 遮蔽,不是旧索引 | `tests/structure/bracket-call.mq.md` |
|
||||
| `footnote-index` | `集合[^键]` | 兼容取元 | 保留既有能力;新代码优先 `access(...)`,需要解释时包裹描述,不再推荐旧 `[键](集合)` | `tests/structure/footnote-index.mq.md` |
|
||||
| `soft-emphasis` | `*仅*` / `**说明**` | 忽略 | 非代码形强调**不执行**——默认安全的关键消歧规则 | `tests/markup-v03/soft-emphasis.mq.md` |
|
||||
|
||||
## 3. 词法大类(LineKind,`src/lex/mod.rs`)
|
||||
|
||||
@@ -0,0 +1,284 @@
|
||||
# Marqdo 文档原生开发指导:让知识的表达、执行与复用成为同一件事
|
||||
|
||||
| 项目 | 约定 |
|
||||
|---|---|
|
||||
| 状态 | 下一阶段开发指导与规划修订;不表示下述目标接口已经实现 |
|
||||
| 日期 / 审阅基线 | 2026-10-23 / `main`,`7674ebb`;继承缺口审计的 `434be9f` 代码证据 |
|
||||
| 原始意图 | [009](../next/009.md)、[附属设计](../next/Marqdo_Executable_Knowledge_Design.md)、[附属示例](../next/Marqdo_Executable_Knowledge_Example.md) |
|
||||
| 配套实施文件 | [详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md)、[缺口与交付路线](../roadmap/executable-knowledge-gaps-2026-10-23.md) |
|
||||
| 本轮边界 | 校正开发方向、作者体验、AI 工作协议和验收;不新增语法,不宣称实现新能力 |
|
||||
|
||||
## 1. 审阅结论与规范关系
|
||||
|
||||
Marqdo 的优势不在于给传统程序换一套 Markdown 符号,而在于:**读者解释知识、作者修改知识、程序执行知识、AI 查找并复用知识,能够指向同一份有结构的文档和其中的同一步。** 标题、叙述、表格、链接描述、输入输出、来源及执行行为不应分散在彼此手工同步的系统中。
|
||||
|
||||
现有规划没有放弃这一目标,但工程排序存在偏移:把身份、图、证据、锁、句柄、晋升全部铺完,再验证人和 AI 是否更容易使用。这样可能交付一个功能齐全的制品管理系统,却让文档作者承担另一套注册、声明、审核和配置工作。需要改变交付方法,不必推翻确定性编译、绑定和权限机制。
|
||||
|
||||
本指导规定**为何开发、作者维护什么、怎样证明值得开发**;详细规范规定语义模型与执行约束;路线图规定当前缺口、切片与完成证据。三者不是三套平行设计。遇到冲突,依次采用:`009` 的产品意图 → 本指导对作者负担、场景分级和交付次序的修订 → 详细规范的工程规则 → 附属概念示例。当前代码与测试证明“已实现什么”,不能把代码局限升级为永久语言禁令;原始讨论保留,不据其重新引入已被否决的方案。
|
||||
|
||||
### 1.1 方向修订清单
|
||||
|
||||
| 容易偏移的做法 | 修订后的要求 | 必须保留的技术约束 |
|
||||
|---|---|---|
|
||||
| 先做完整知识治理平台,再给作者使用 | 从一篇可读、可运行、可定位步骤的文档交付,逐层扩展 | 每个已支持路径都有准确的目标和效应边界 |
|
||||
| 为进入知识库再写 YAML、注册表和索引 | 从源码推导;只有不能推导的领域事实才需作者补充 | 生成物有版本,可重建,可检查来源 |
|
||||
| 所有逻辑先封装成可登记函数/类 | 一次性逻辑直接写;需要命名、解释或复用时逐步提取 | 作用域、形参、返回与副作用明确 |
|
||||
| 所有调用先审核、锁定、签发远程凭据 | 本地探索、项目复用、严格业务、远程执行分级 | 精确目标、运行时权限不能被标签或模型绕过 |
|
||||
| AI 只能搜文档、填工具参数 | AI 可以读步骤、组合、选择候选、局部修改、提取和验证 | 模型的选择必须转成显式目标,由编译器校验 |
|
||||
| 为“独特”拒绝普通调用或新增更多标记 | 保留简单 payload;让 Markdown 结构提供程序之外的信息 | 无第二套表达式文法,无裸词隐式执行 |
|
||||
| 核心构造数量不增长就是简单 | 同时检查上下文例外、记忆负担和诊断成本 | 新构造需 ADR;不靠把新规则藏进旧构造绕过审查 |
|
||||
|
||||
### 1.2 四个不能互相替代的完成条件
|
||||
|
||||
1. **读得懂**:不打开生成图,也能从源文件看出目的、关键步骤、参数角色和限制。
|
||||
2. **执行同一件事**:标签表达意图,payload 决定计算;运行 trace 能回到源码步骤,不能执行另一个同名目标。
|
||||
3. **改得动**:人或 AI 能修改一个明确步骤,看到影响与验证结果,不必重写整份文档或同步多份签名。
|
||||
4. **复用得对**:找到已有能力后实际调用它;不满足条件时给出具体拒绝原因,而非用复制源码掩盖失败。
|
||||
|
||||
“更适合 AI”是待验证的产品假设,不是 Markdown 文件后缀自动赋予的性质。正确性和作者成本优先于 token 下降;错误复用更便宜不算优化。
|
||||
|
||||
## 2. 语法审查的保留、修正与暂缓
|
||||
|
||||
以下裁决针对审查结论的适用范围,不重写历史实验记录。旧文档中的原型、性能数值和当时的 `Value`/AST 限制,不能直接当作 v2.1 的规范或测试结果。
|
||||
|
||||
| 审查材料 | 保留的洞见 | 需要修正的外推 / 本轮处置 |
|
||||
|---|---|---|
|
||||
| [函数即命名](function-as-naming-philosophy.md) | 从正文计算出发,命名应降低理解和复用成本;避免无意义包装 | “具名给值缺失”是历史观察,当前有具名参数;替换模型是教学解释,不意味着禁止递归、前向引用或必须文本展开 |
|
||||
| [方括号作为标签](bracket-as-label-evaluation.md) | 描述与 payload 共址;标签帮助解释、检索和拆步骤 | Classic 冲突是迁移历史,不据此恢复默认 `[键](集合)`;不强制每个常量或子表达式都有标签 |
|
||||
| [圆括号与可读调用](paren-call-and-readable-invocation.md) | 调用标记应明确;保留 `f(args)` 的局部可读性 | 原型撤回不代表现有版本缺少该能力;Marqdo 的独特性不依赖消灭熟悉的表达式形式 |
|
||||
| [从标记推导调用](call-syntax-from-marks.md) | 语义成分有明确标记;自然语言是连接与解释,不参与猜测调用 | 持久跨文档接口优先具名实参;短而稳定的位置参数仍合法。推断参数的声明次序变化属于接口变化,不承诺任意润色都无影响 |
|
||||
| [环境参数](ambient-parameters-evaluation.md) | 参数来源应可见,避免调用者隐式环境影响复用 | 文档顶部已否决 L0 裸“执行”触发,正文旧建议不能重新生效;文件级常量也须有明确绑定与依赖,不变成动态作用域 |
|
||||
| [函数作为值](functions-as-values-evaluation.md) | 区分名称数据、一等函数、有限查表;避免无需求增加核心复杂度 | “匿名函数必不能索引”“动态调用必然破坏契约”不是普遍定理。将来可研究稳定节点身份与运行时契约;本轮不引入闭包,也不永久禁止 |
|
||||
| [函数作为表格](function-as-table-semantics.md) | 数据表几何有既定含义;契约表不可另造一份参数声明 | 不把表格全部变成函数,也不把“现有实现不支持”写成“表格不能计算”;有限决策表先评估库方案,只有明确角色和歧义测试后才讨论语言扩展 |
|
||||
| [核心表面](core-surface.md)、[分层](layers.md) | 核心小、标记可解释、构造有测试与准入流程 | 清单中的旧取元/循环说明需随迁移更新;固定数量不等于低认知成本,守卫通过也不证明文档语义正确 |
|
||||
| [链接设计](link-syntax.md)、[告警疲劳](warning-fatigue-control.md) | 单一 payload 文法;标签漂移可提示;默认安静、按需严格 | 标签不是纯粹可丢弃的注释,也不是事实证明;不做每次运行重复的批准仪式,不靠词汇相似度证明描述与代码一致 |
|
||||
|
||||
### 2.1 继续冻结的语法边界
|
||||
|
||||
- 默认 `[]()` 是描述包裹 payload;旧取元只能在显式历史兼容路径下解释。现行取元用 `access(...)` 等已支持形式;循环使用明确遍历载荷。
|
||||
- `[]()` 不等于“仅函数调用”:计算、取元、调用、遍历、叙述引用都需要各自语义角色,不能只收集 call 边就宣称完成知识步骤。
|
||||
- 正文引用与可执行位置有不同调度权限。普通句子不因出现“执行”“发送”或工具名而运行;正文解析成引用也不自动求值。
|
||||
- 不把所有 `#` 解释成某种知识对象,或把继承悄悄变成实例关系。标题的既有语义必须兼容,IR 可区分 Type、Entity、Fact、Function。
|
||||
- 不给同一参数增加正文声明、签名表、工具 schema 三套作者真相。规范签名和工具 schema 是投影;契约表补充类型、约束和说明,不偷偷增加形参。
|
||||
- 可读源文件是硬要求;普通 Markdown 渲染器是否把含空格/嵌套表达式的链接渲染成可点击链接是另一件事。不能承诺全部标准渲染器理解执行语义。
|
||||
|
||||
### 2.2 哪些编辑不能当成“只是改文案”
|
||||
|
||||
不含语义标记的普通叙述润色,应保持执行结果和实现摘要;标签改名应保留 payload 行为,但更新展示/检索内容。反引号声明、首次出现顺序、标题名称/层级、粗斜体、表格几何、契约、来源和适用范围的改变则可能影响程序或知识可信性。编辑器必须展示真实差异,不能统一标为 prose-only。
|
||||
|
||||
标签漂移诊断应说明“描述或实现可能需要复核”,不能自动替作者改写业务意图。自然语言一致性检查可作辅助建议,确定性检查仍需可定位的符号、类型、契约或测试证据。
|
||||
|
||||
## 3. 作者工作流:先写清楚,再获得复用能力
|
||||
|
||||
### 3.1 一次性计算不需要预先建知识模型
|
||||
|
||||
下面是当前默认语法的完整最小程序。无模式开关、无项目注册、无知识 ID、无锁文件。
|
||||
|
||||
```markdown
|
||||
# main
|
||||
|
||||
这是演示订单,单价与数量直接写在计算处。
|
||||
|
||||
**小计 = [两件商品的小计](2 * 30)**
|
||||
> print text=`小计`
|
||||
```
|
||||
|
||||
可读意图和精确计算共址。这个标签值得保留,因为它说明本次计算的业务角色;不要求把 `2`、`30`、乘法分别命名。
|
||||
|
||||
### 3.2 需要解释或复用时才提取命名能力
|
||||
|
||||
```markdown
|
||||
## 商品小计
|
||||
|
||||
根据购买数量`数量`和商品单价`单价`计算小计。
|
||||
|
||||
*[按数量与单价计算](数量 * 单价)*
|
||||
|
||||
# main
|
||||
|
||||
计算同一笔演示订单。
|
||||
|
||||
**小计 = [两件商品的小计](商品小计(数量=2, 单价=30))**
|
||||
> print text=`小计`
|
||||
```
|
||||
|
||||
两个完整程序在本轮本地现有 `target/debug/marqdo` 上分别以 Tree 和 Bytecode 执行,均输出 `60`。这仅验证基础语法,不证明步骤身份、重构工具或 Agent 闭环已经实现;也不是全仓测试报告。可将各块保存为 `.mq.md` 后用 `marqdo run 文件 --backend tree` / `--backend bytecode` 复核。
|
||||
|
||||
**后续工具目标**:从选中表达式提取为标题单元,推导自由变量为参数、保持求值顺序与 effect 次数、生成具名调用,并给出可审阅补丁。反向内联也应保留契约和来源关联;无法证明安全的捕获变量、递归、控制流、动态目标或有副作用表达式,先拒绝自动变换并说明原因,不伪称等价。
|
||||
|
||||
### 3.3 跨文档复用与事实说明分别增长
|
||||
|
||||
当第二篇文档需要这段计算,才提取模块并显式 import;不要为十行单文件强制创建 `corpus/knowledge/apps` 目录树。项目索引按约定发现,明确边界或外部来源时再加入配置。
|
||||
|
||||
当“商品小计”依赖政策、时间、币种等条件,再就地补契约与来源。简单乘法不要求假造现实来源;“某商店当前折扣为九折”则不能仅凭函数返回 `0.9` 声称真实。
|
||||
|
||||
正文可以说明结果为何成立,目标是引用同一能力和同一次受控执行的结果,而非手抄另一份公式。正文引用 AST、结果投影和绑定身份尚需开发;完整业务链使用[目标示例](../next/Marqdo_Executable_Knowledge_Example.md)验收,不把它冒充当前全部可运行的教程。
|
||||
|
||||
### 3.4 维护时看到知识变化,而不只是文件变化
|
||||
|
||||
目标维护流程:修改上游规则 → 显示受影响的计算、正文陈述和消费者 → 执行相关测试 → 展示结果/证据/接口各自变化 → 按使用策略接受更新。普通显示文案变更不应触发全库业务审批;证据过期也不必使无关纯工具全部失效。
|
||||
|
||||
## 4. 人维护语义,编译器维护机械关联
|
||||
|
||||
| 信息 | 作者责任 | 编译器 / 工具责任 |
|
||||
|---|---|---|
|
||||
| 目的与步骤 | 有帮助的标题、叙述、标签;避免空泛重复 | 保留源位置和层级,生成步骤投影,不要求手工节点注册 |
|
||||
| 输入输出 | 一份参数来源和必要契约;领域单位不能靠猜 | 推断规范签名、检查调用、生成 schema 和签名视图 |
|
||||
| 目标与依赖 | 显式 import、可读名称、必要消歧 | 绑定唯一 SymbolId,生成依赖边与闭包;不要求手写 ID |
|
||||
| 效应 | 扩展作者声明宿主能力;业务作者必要时收窄允许范围 | 汇总直接及传递效应;未知不得推断 pure;运行时执行能力上限 |
|
||||
| 事实与来源 | 原始资料、范围、有效期、人工审阅等不可推导信息 | 关联来源、记录验证事件、传播失效;不将文件路径视为事实证明 |
|
||||
| 身份与版本 | 有意移动/更名时确认必要迁移 | 区分身份与内容摘要;报告歧义,不靠猜测维持假稳定 |
|
||||
| view / AI 索引 | 不维护第二份可执行描述 | 同一语义模型投影;显示生成代、验证政策和缺失字段 |
|
||||
|
||||
生成信息不等于免费:首次编译、缓存修复、错误定位都要可理解。失败时应回到源文档中可修改的位置,而不是只报内部 Artifact 字段名。
|
||||
|
||||
稳定身份要诚实:同一次编译内引用和 trace 必须一致;普通文本编辑不应打乱无关节点;复制相似步骤、跨文件移动等无法唯一识别时,要求确认迁移或明确生成新身份。禁止用文本相似度悄悄继承已审核的知识身份。
|
||||
|
||||
## 5. AI 原生工作协议:读步骤、改步骤、验证步骤
|
||||
|
||||
本节是**待实现的协议约束**,不是新增 CLI 命令清单。优先扩展现有 MLSP、view、EKC 和 Agent 工作簿,不另造一套 Agent 框架或知识库。
|
||||
|
||||
### 5.1 发现后的最小上下文包
|
||||
|
||||
一次检索返回可展开的上下文卡,而非强制倾倒整个文件或只给函数 schema:
|
||||
|
||||
1. 标题路径、任务相关叙述、标签和精确源码片段;
|
||||
2. SymbolId/步骤节点、source span、源码摘要、生成代;
|
||||
3. 输入角色、返回、单位、默认值和适用条件;
|
||||
4. 直接依赖、效应、必要来源及分层验证状态;
|
||||
5. 支持该能力的例子/测试,相关失败经验、禁用条件和替代能力;
|
||||
6. 还缺哪些检查、能否在当前 policy 下运行、为何被拒绝。
|
||||
|
||||
来源不可用、字段未推导、上下文被截断都须显式标识。按预算可展开依赖,但不能悄悄省略改变适用性或权限的重要信息。原始资料及标签是内容,不是可以覆盖 Agent 策略的指令。
|
||||
|
||||
### 5.2 复用、适配、新建的分界
|
||||
|
||||
- **REUSE**:AI 可根据任务选候选,提交精确目标与输入;resolver 确认目标、契约、适用性、版本和 effect,执行既有制品。复用不靠“没有生成文件”推断,必须有实际调用 trace。
|
||||
- **ADAPT**:已有能力接近但不适用;说明差异,保留依赖/来源关联,在局部副本或补丁中改变必要部分,再验证。不为了通过测试改写原始来源或放宽全局策略。
|
||||
- **CREATE**:没有合适候选或明确需要新语义;记录检索范围和失败原因,在任务预算及写权限允许时新建。不能把检索服务故障当作“仓库没有能力”。
|
||||
- **ASK / STOP**:缺业务决策、权限或关键证据时报告具体缺口。编译器无法决定的真实意图不交给近似匹配默默补全。
|
||||
|
||||
AI 并非只能发现和解释。它可以提出精确目标、组合已有能力、改进表达、重构及补测试;不能以自然语言自报正确来替代执行验证,也不能把含糊名称传给后端任意选一个目标。
|
||||
|
||||
### 5.3 局部修改的事务边界
|
||||
|
||||
编辑请求至少携带目标文件/节点、期望源码摘要、补丁范围和预期行为变化。应用前检查摘要,源已变化则重新定位和审阅,不覆盖用户改动。应用后重新解析受影响文档,生成语义 diff:参数/返回、目标、effect、控制流、证据与依赖是否改变。
|
||||
|
||||
对“只改措辞”请求,检测到接口或实现改变时停止自动合并;对“改计算”请求,运行相关正负例并展示影响。原始文本、格式与未知 metadata 尽量保留,不通过整篇重生成来完成一行修改。禁止修改生成图伪装修复源码。
|
||||
|
||||
首个试点只支持可确定定位的纯计算步骤与普通函数,不等待通用 AST 编辑器、跨项目事务或完整知识治理。安全拒绝超范围输入也属于验收。
|
||||
|
||||
### 5.4 解释和负知识必须有落点
|
||||
|
||||
解释结果应关联到“哪一步、用了哪个目标、哪些输入、哪份来源、哪次验证”,区分已执行观察和模型推断。运行 trace 可以证明程序路径,不能证明任意自然语言论证正确。
|
||||
|
||||
失败经验需要保存适用范围、触发条件、关联能力、原因与失效时间/版本。例如“此折扣不适用于退款”应在查询定价能力时可见,而不是埋在另一份聊天记录里。先由普通文档与结构化关联承载,不先新增通用推理语言;过期负知识也要撤销,不能永久拦截修复后的能力。
|
||||
|
||||
## 6. 渐进严格:不把治理门槛塞进每一行正文
|
||||
|
||||
下面是目标政策分级,不是当前已有四种 CLI 模式。政策由项目/宿主配置和调用上下文选定,源文档保持同一种语法。
|
||||
|
||||
| 场景 | 必需条件 | 默认不要求 |
|
||||
|---|---|---|
|
||||
| 本地探索、单文件运行 | 明确语法与目标、已有运行时契约、宿主能力限制;如实显示未知状态 | lock、来源、ready 晋升、远程签名句柄 |
|
||||
| 项目内自动复用 | 精确绑定、输入/适用性检查、效应许可、状态满足所选政策、执行 trace | 所有纯工具都有人审证据、每次独立确认 |
|
||||
| 严格业务/可复现任务 | 相关依赖与证据锁定、必要验证新鲜度、受控环境和审计;明确不可重放部分 | 对无关文案变化重审整个项目 |
|
||||
| 远程/多租户/不受信调用方 | 上述相关约束,加可信授权主体、范围、服务端能力上限、撤销/期限与防伪设计 | 把客户端提交的 `allow_effects` 当授权本身 |
|
||||
|
||||
`ready` 是“对某项 policy 和输入/上下文已具备复用条件”,不是永久万能认证。未验证的本地草稿可以被检索、查看和按明确权限试跑;不能冒充满足严格政策的可复用事实。测试通过与事实证实分别报告。
|
||||
|
||||
正文 `check/view static` 不执行外部效应。用户主动打开的纯预览可复用项目或会话已有政策,不要求每个链接弹窗;effectful 运行始终受宿主能力和明确执行上下文控制。锁和摘要是完整性机制,不自动等于远程授权;可信本地引用无需先搭建凭据服务。
|
||||
|
||||
## 7. 交付切片:每片都有文档作者与 AI 能看见的结果
|
||||
|
||||
编号与[缺口路线图 §7](../roadmap/executable-knowledge-gaps-2026-10-23.md#7-依赖顺序与可合并切片)一致;G 编号沿用现状证据,M0–M6 保留为工程能力映射,不再被理解为必须全部串行完成的墙。
|
||||
|
||||
### S0 作者基线与最小语料
|
||||
|
||||
- 选定单文件计算、提取函数、两文档引用、表格取元/遍历、业务政策及拒绝复用六类任务;记录当前写法、输出、诊断和作者重复维护项。
|
||||
- 优先整理 `doc/README`、核心语法卡片、教程与运行示例的版本状态;不要先扩展通用库数量。
|
||||
- 为最小样例建立无网络门禁。100 单元及更大语料逐步增长,不阻塞首个可读文档切片。
|
||||
- 完成证据:别人按当前入口能写出正确默认语法;已实现与预期失败分开报告,不能把待实现测试跳过计为成功。
|
||||
|
||||
### S1 可读步骤与唯一目标
|
||||
|
||||
- 主要落点:`src/parse/`、`src/ast.rs`、`src/sema/`、`src/knowledge/compile.rs`、`src/view/`。
|
||||
- 在最小故事内联合完成 G01/G03:保留 label、payload、上下文和跨度;每类步骤有输入输出与依赖投影;旧 EKC 查询索引不再覆盖语义图。
|
||||
- 普通导航、正文引用、可执行表达式、循环分别处理,正文保持惰性。先确保支持范围内同一引用在源码、view、图中是同一个目标,再扩展覆盖范围。
|
||||
- 完成证据:作者能从渲染步骤返回对应源码;同名目标不误连;计算/取元/循环不是伪造的 call;重复构建可对照;未支持上下文明示。
|
||||
|
||||
### S2 确定执行与早期 AI 复用试点
|
||||
|
||||
- 主要落点:`src/sema/`、`src/interp.rs`、`src/bytecode/`、`src/mlsp.rs` 与现有 Agent 入口。
|
||||
- 完成静态普通函数、显式 import、关联函数的 BoundCall 主链;G02 与 G06 的效应安全下限不可延期。小范围可先用明确纯操作白名单,未知路径必须拒绝纯证明,而非猜 pure。
|
||||
- 实现最小上下文卡与本地绑定引用;以 stub/no-LLM 测试验证协议,然后用现有 Agent 跑相同故事。无需先完成远程鉴权平台或全面来源审核。
|
||||
- 完成证据:同一需求实际调用既有函数,不新增同义实现;两个后端 trace 对上图中目标;改目标、错误输入、未知效应、权限不足不能静默执行。
|
||||
|
||||
### S3 局部编辑、提取与库可读性
|
||||
|
||||
- 主要落点:MLSP 编辑/定位、相关 sema/检查入口、`lib/`、`ext/`、教程与测试。
|
||||
- 按 §5.3 实现受摘要保护的局部修改和语义 diff;实现受限纯表达式提取,或明确标为待实现,不用人工演示冒充工具能力。
|
||||
- 迁移标准库/扩展时删掉重复包装与噪音标签,保留有意义的业务步骤、契约和负例;中英镜像含义一致。兼容旧调用接口,不用增加旧语法解释救迁移遗漏。
|
||||
- 完成证据:同义润色不改执行;改反引号声明能检测签名变化;表达式提取前后结果/effect 次数一致;冲突补丁拒绝;CLI、库、扩展与 view 既有能力继续通过。
|
||||
|
||||
### S4 业务知识的来源与适用性
|
||||
|
||||
- 主要落点:契约、SourceRef/VerificationRecord、验证事件、Agent 候选解释;完成 G05 与 G07 的项目政策部分。
|
||||
- 选定书店案例的币种、数量边界、会员范围与政策有效期;给 schema、拒绝诊断及可运行负例,不泛化为全能业务规则语言。
|
||||
- 同步补齐传递效应与相关负知识;生成契约/状态展示,减少作者重复填写。
|
||||
- 完成证据:同一计算因输入或证据不同而正确通过/拒绝;来源过期不被 `tests=passed` 覆盖;普通纯工具无需虚构 SourceRef。
|
||||
|
||||
### S5 变化传播与严格复用
|
||||
|
||||
- 主要落点:`src/sema/lock.rs`、图/impact、产物提交与消费者;完成 G04/G08 和严格 handle 复核。
|
||||
- 区分身份/接口/实现/证据/闭包,给出可读升级 diff;整代快照保证消费者不会读取混代数据。并发读写一旦开放,该一致性就是前置条件,不能等到此片才承认风险。
|
||||
- 完成证据:修改依赖政策后,下游计算和正文解释进入影响集合;文案变化不误判实现变化;并发编辑或构建崩溃不导致错目标执行;锁不声称重放网络/随机/LLM。
|
||||
|
||||
### S6 生态验收与发布
|
||||
|
||||
- G10/G11 从 S0 起每片执行;S6 汇总兼容矩阵、真实工作流和打包后冒烟,不把测试留到最后。
|
||||
- G12 随实际规模需求推进;远程凭据/多租户是有条件的独立交付。它们不阻塞受信本地试点,也不得被本地通过的测试覆盖为“已安全支持”。
|
||||
- 完成证据:当前默认语法的 lib/ext/CLI/MLSP/view/WASM 等受影响消费者有明确结果;目标示例中的已发布子集实际可运行,未发布部分保留状态;发布声明与证据一一对应。
|
||||
|
||||
## 8. 验证 Marqdo 优势,而不只验证组件存在
|
||||
|
||||
### 8.1 必须收集的产品指标
|
||||
|
||||
| 维度 | 记录方法 | 失败信号 |
|
||||
|---|---|---|
|
||||
| 作者负担 | 完成同一任务所需手写声明、重复字段、配置、跳转和确认次数 | 简单本地算式也必须填写制品清单 |
|
||||
| 阅读与定位 | 给定问题,定位正确步骤/限制的准确率、耗时与上下文量 | 只看 label 会误导,必须打开生成 JSON 才理解源文档 |
|
||||
| 可修改性 | 修改范围、无关 diff、语义变化检测率、冲突覆盖数 | 为改一步而重生成全文件;措辞调整偷偷改变接口 |
|
||||
| 复用质量 | 已有能力发现率、实际调用率、重复实现率、误用率 | 检索命中却重新写代码;正确拒绝算成任务失败而被优化掉 |
|
||||
| 知识可追溯性 | 结果关联目标、步骤、输入、来源和验证的覆盖率 | `tested` 被当作事实正确;解释引用未执行的步骤 |
|
||||
| 总成本 | 正确完成任务的 token、工具调用、轮次、总耗时及编译成本 | 只报模型 tokens,隐藏额外注册/审核/修复成本 |
|
||||
|
||||
### 8.2 对照实验与门槛
|
||||
|
||||
1. 固定任务、可访问能力、输入数据、模型版本、工具能力、预算与超时;比较普通 Markdown+代码/RAG、当前 EKC 和新方案。不能给新方案更多现成实现再声称语法更优。
|
||||
2. 任务包括成功复用、同名冲突、近似但不适用、未知 effect、过期来源、正文不执行、局部修改及库升级。加入自然语言诱导绕过 policy 的负例。
|
||||
3. 先测确定性协议,再测多次 AI 运行,保留种子(支持时)、原始 trace、版本、失败样本与分布;小样本不夸大收益。
|
||||
4. 正确目标、无越权和正文无外部 effect 是硬门槛。原有兼容测试不得降低;效率阈值先测基线再登记,不捏造提升百分比。
|
||||
5. 若作者负担增加或误用率上升,回到交互/语义设计分析原因,不能只用“更多治理更安全”或“模型更便宜”解释过去。
|
||||
|
||||
### 8.3 发布声明分层
|
||||
|
||||
- “支持新语法”需语法、迁移、双后端与库兼容证据,不意味着完整知识复用已经交付。
|
||||
- “本地确定性知识复用”需 S1/S2 支持范围内的端到端与拒绝用例;范围外路径显式不支持,不能回落为模糊执行。
|
||||
- “严格业务知识复用”还需相关 S4/S5 来源、适用性、失效、锁和一致性证据。
|
||||
- “安全远程知识执行”需额外的授权、能力上限、凭据/撤销及隔离测试;本地 JSON handle 不等于远程安全。
|
||||
|
||||
## 9. 每个开发变更必须回答的问题
|
||||
|
||||
提交设计或实现时给出一份短而具体的审阅记录:
|
||||
|
||||
- 用户要读/写/改/复用的知识是什么?附源文档前后对照,不只给内部结构图。
|
||||
- 利用了哪种文档原生信息?若只增加通用框架能力,与主线的必要关系是什么?
|
||||
- 新增了哪些作者必须手工维护的信息?为什么不能从现有语义推导?
|
||||
- 是否改变标记、作用域、求值顺序、形参顺序、表格几何或正文安全?分别有哪些测试?
|
||||
- 人、CLI、view、图和 Agent 是否看到同一目标与状态?失败能否回到源码修复?
|
||||
- AI 的正确复用/局部编辑如何证明?哪些部分仍需模型判断,哪些由编译器决定?
|
||||
- 标准库和扩展是否先迁移?是否保住既有公开接口、输出与 effect?
|
||||
- 这是已实现、受限试点还是设计目标?文档和发布声明是否一致?
|
||||
|
||||
**准入原则**:内部可以使用成熟编译器、数据库和协议技术;外部必须减少知识在“说明、程序、索引、工具描述”之间的重复翻译。独特性由这条链的实际收益体现,不由新符号数量或与其他语言不同的程度体现。
|
||||
@@ -51,7 +51,9 @@ Marqdo 把仓库投影为 **可查询、可验证、可复用** 的工程知识
|
||||
|
||||
`implements` · `uses` · `depends_on` · `tested_by` · `documented_by` · `constrained_by` · `decided_by` · `supersedes` · `conflicts_with` · `deprecated_by` · `derived_from` · `failed_by` · `related_to`
|
||||
|
||||
落盘:`.marqdo/graph/graph.json` · `edges.json` · `index.json`。
|
||||
落盘(G01 分离后):
|
||||
- **权威语义图**:`.marqdo/graph/edges.json` · `nodes.json`(`SemanticProject`,含 `authority` 字段)· `steps.json`
|
||||
- **EKC 检索投影**(非执行依据):`.marqdo/ekc/graph/graph.json` · `edges.json` · `index.json`
|
||||
|
||||
## 5. CLI
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 语法设计哲学:函数在「代码即文档」里为何水土不服,以及用什么语义取代
|
||||
|
||||
> **2026-10-23 审阅裁决**:保留“正文优先、按需命名”的方向;下文语料统计和“具名给值缺失”等判断属于当时快照。当前已有具名实参;“调用即代入”不规定文本展开实现,也不禁止递归。下一阶段以[文档原生指导 §2–3](document-native-development-guide.md#2-语法审查的保留修正与暂缓)的范围和验收为准,本文不是替换整个函数模型的已批准语法提案。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 哲学层讨论**(不涉及实现,不含语法提案细节) |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 语义探析:代码函数能否解释为一张 GFM 表
|
||||
|
||||
> **2026-10-23 审阅裁决**:下文 v1.5.2 实验不替代当前回归。保留现有数据表几何与契约表角色,不重复声明参数;“现有实现不支持函数表”不等于表格不能表达计算。有限决策表和生成式签名视图按真实用例评估,见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 待决策**(未修改任何代码) |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 方案评估:函数能否作为变量的一种(函数即表)
|
||||
|
||||
> **2026-10-23 审阅裁决**:以下是旧实现约束与方案权衡,不是“匿名函数无法索引”或“动态调用无法检查契约”的普遍结论。一等函数/闭包本轮暂缓而非永久禁止;有限决策表先评估现有库表达,文中建议的新辅助函数不因此成为已实现能力。参见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 待决策**(未修改任何代码) |
|
||||
|
||||
@@ -0,0 +1,326 @@
|
||||
# 隔离执行与响应式 Web 运行时:详细开发设计
|
||||
|
||||
| 项目 | 约定 |
|
||||
|---|---|
|
||||
| 状态 | **Verified**(execute 路径);HTTP `execute` / `sse-live` / `stream:true` 默认走 Supervisor worker 池(不再经父进程 `CallLibPath` / `INVOKE_LOCK`);`runtime.isolated_execution=1`;MR-01 金样例 `tests/isolated_execute_mr01.rs`;未声明非流式仍 `legacy_serial`(ADR 0008) |
|
||||
| 基线 | Marqdo `1b11fe0` / 2.1.3;qdagent `64b97e5`;2026-10-26 |
|
||||
| 问题依据 | [单解释器缺陷](web-invoke-serialization.md)、[ADR 0008](../adr/0008-web-invoke-serialization.md)、[本轮跨库复核 R01–R14](../../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md) |
|
||||
| 产品配套 | [qdagent 可靠工作台开发设计](../../../qdagent/doc/design/reliable-workbench-development-2026-10-26.md) |
|
||||
| 继承合同 | [工作台 M00–M06](qdagent-workbench-runtime-and-view-2026-10-25.md) 中 ModelSelection、ModelSnapshot、Usage、DocumentView、Locator;保留文档原生作者面 |
|
||||
|
||||
## 1. 目标、边界与不变量
|
||||
|
||||
目标是让用户在长对话、导入和构建过程中持续打开模型列表、读取来源、修改下一轮设置并可靠停止任务。高并发容量按资源预算显式限制;不能让所有请求进入一把解释器锁再超时。
|
||||
|
||||
Marqdo 负责通用执行隔离、资源调度、事件/取消、语义分析和制品校验。qdagent 负责用户/工作区授权、业务状态与发布政策。产品仍通过 `.mq.md` 调用 Marqdo Agent,不增加第二套模型循环或知识编译器。
|
||||
|
||||
必须保持的系统不变量:
|
||||
|
||||
1. 同一个可变 HostContext 同时最多一个求值者;不删除锁、不增加虚假的 `unsafe impl Sync` 来获得并行。
|
||||
2. 一个执行租约只属于一个 run/attempt;凭证、事件、取消、cwd、权限和模型选择不可流入另一请求。
|
||||
3. 控制与读取具有独立容量,不被 LLM 等待占满;停止不需要重新进入正在执行的解释器。
|
||||
4. 事件可以重投,外部副作用不默认重做;不宣称跨网络 exactly-once。
|
||||
5. 已确认终态不能被晚到完成/取消覆盖;取消不回滚已经完成的外部写入。
|
||||
6. 阅读不执行 `.mq.md`,缓存不绕过授权或版本校验;缓存命中必须实际复用结果。
|
||||
7. 正文、依赖、证据和编译代属于同一不可变快照;不会把两个代的结果拼成一次阅读或执行。
|
||||
8. 任何能力开关都以实际探测和测试为依据;新 API/schema 在落地前都属于本文提案。
|
||||
|
||||
## 2. 架构选择
|
||||
|
||||
| 方案 | 优点 | 当前代价 | 本轮选择 |
|
||||
|---|---|---|---|
|
||||
| 继续全局锁,只增加 Go handler | 改动小 | 长任务依然阻塞所有 Endpoint,串扰变量也未消失 | 只保留作兼容模式 |
|
||||
| 同进程多 HostContext | 潜在内存复用较好 | GLOBAL_HOST、插件回调、环境、cwd、事件 sink 生命周期需系统重做 | 后续优化,不是首个交付门槛 |
|
||||
| 独立 Marqdo worker 进程 + 控制/读取通道 | 隔离现有全局状态,可强制回收卡死任务,复用现有语言与 Agent | IPC、进程启动、内存、发布写协调 | **首选实现** |
|
||||
| 引入外部通用队列和新业务 Runtime | 容易扩到集群 | 运维复杂度及第二套编排风险 | 单机阶段不引入 |
|
||||
|
||||
首期采用**预热的一次性执行进程池**:进程可预装运行时和非用户特定模块,分配任务后独占,结束即退出并异步补充。这样不用假设清空部分字段就能安全复用 HostContext。长寿命多租约 worker 只有通过状态泄漏、插件资源与 RSS 回归后才开启。只读分析服务可以长寿命运行,但不得加载有副作用的求值插件。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
UI[浏览器 / CLI / MCP] --> WEB[Go HTTP / 认证入口 / 限流]
|
||||
WEB --> CTRL[任务控制与事件服务]
|
||||
WEB --> SHORT[短业务请求容量]
|
||||
WEB --> READ[只读分析服务]
|
||||
SHORT --> POLICY[qdagent .mq.md 授权与业务规则]
|
||||
POLICY --> CTRL
|
||||
CTRL --> DB[(任务索引 / outbox / 事件日志)]
|
||||
CTRL --> SUP[Rust Supervisor / 有界调度]
|
||||
SUP --> A[Marqdo worker A / 一个租约]
|
||||
SUP --> B[Marqdo worker B / 一个租约]
|
||||
A --> AGENT[既有 Marqdo Agent / LLM / sandbox]
|
||||
B --> AGENT
|
||||
A --> STAGE[每任务暂存目录]
|
||||
B --> STAGE
|
||||
STAGE --> COMMIT[按资源串行提交与 revision CAS]
|
||||
COMMIT --> SNAP[(不可变知识代 / 发布 manifest)]
|
||||
READ --> SNAP
|
||||
A --> CTRL
|
||||
B --> CTRL
|
||||
CTRL --> UI
|
||||
```
|
||||
|
||||
“短业务请求容量”依旧可执行产品 `.mq.md`,但有独立 worker 预算和短 deadline。控制服务只做通用接纳、状态 CAS、取消与日志投递,不把模型选型、知识晋升等业务逻辑搬进 Go。读取服务接收经过授权的资源句柄;不得直接相信浏览器提交的绝对路径。
|
||||
|
||||
## 3. 路由分类与接纳
|
||||
|
||||
### 3.1 执行类别
|
||||
|
||||
在 Endpoint 装配中新增可选 metadata(字段名待 schema 固定):
|
||||
|
||||
| 类别 | 示例 | 资源预算与规则 |
|
||||
|---|---|---|
|
||||
| `control` | 任务查询、事件订阅、取消、health | 不执行用户代码;认证后访问任务存储/租约注册表 |
|
||||
| `read` | view.document、引用解析、已发布模型列表投影 | 只读快照;不写用户源码;可缓存 |
|
||||
| `short` | 保存设置、创建导入、接纳任务 | 无 LLM 长等待;默认 deadline 2s,较慢动作转为任务 |
|
||||
| `execute` | ASK/BUILD/生成/验证 | 有界队列、专属进程、显式任务 deadline |
|
||||
| `commit` | 切换知识代、合并受控文件变更 | 按 workspace/kb 的短临界区;执行前检查 revision |
|
||||
|
||||
旧 Endpoint 未声明时仍走 legacy_serial;不能自动猜测任意函数是纯读。纯度来自静态能力和注册配置,实际写权限仍由宿主限制。文件上传字节传输不占解释器,应先落隔离存储,再进入短接纳路径。
|
||||
|
||||
### 3.2 接纳步骤
|
||||
|
||||
1. HTTP 层解析受限请求,验证身份、格式、速率与连接数量。
|
||||
2. 产品短入口解析 workspace/model/policy;授权版本与模型配置 revision 一并冻结。返回规范化任务请求,不执行 Agent。
|
||||
3. 控制服务在同一数据库事务中登记作用域化 idempotency key、payload hash、task 和 dispatch outbox;重复请求读回已有 task,同键异内容返回 409。
|
||||
4. 提交事务后才返回 202/Location。调度器消费 outbox 分配租约,租约带 attempt/epoch。
|
||||
5. 对同步兼容 API,先完成接纳,再挂接结果;等待客户端超时不等于安全重做。
|
||||
|
||||
队列入库前容量不足返回 429 + Retry-After;服务不可用返回 503;接纳失败不发送 started。控制通道须保留槽位,不能因为数据流连接占满而无法取消。
|
||||
|
||||
## 4. Supervisor、生命周期与调度
|
||||
|
||||
### 4.1 Worker 生命周期
|
||||
|
||||
`starting → ready → leased → running → draining → exited`;启动失败/心跳丢失走 `unhealthy → terminating → exited`。任务状态由控制服务维护,进程状态不能直接当作业务成功。
|
||||
|
||||
- worker 启动握手报告 CLI/ext/native、protocol_version、能力、构建摘要;不匹配不进入 ready。
|
||||
- 预热阶段不读取用户凭证、不挂载工作区写权限。分配租约后挂载当前任务的 capability roots、只读快照和独立 staging。
|
||||
- 每任务具有随机 lease_id、run_id、attempt_id、epoch、绝对 deadline 和资源限制。
|
||||
- Supervisor 持有进程句柄、子进程组、事件通道与取消句柄。worker 不能自行修改其他 worker 状态。
|
||||
- 返回业务结果后仍须 drain 事件/用量并提交最终 checkpoint,控制服务持久化终态后确认 ACK,之后释放租约。
|
||||
- 超时或断连清理不能遗留孤儿子任务;正常优雅退出后再删除临时文件。异常由独立 janitor 按租约过期回收。
|
||||
|
||||
### 4.2 有界、公平与资源预算
|
||||
|
||||
初始验证配置建议 4 vCPU/8 GiB、执行槽 2、短请求槽 1、读取服务 1。具体容量由每类任务的 RSS p95 和 CPU profile 决定,不能把这个数当所有机器的默认。执行槽上限:
|
||||
|
||||
`min(CPU约束, floor((可用内存 - 系统/控制/缓存预留) / worker_RSS_p95))`。
|
||||
|
||||
运行队列按 workspace 轮转,再按 session FIFO;一个 session 默认只允许一个活动 turn,跨 session 可并行。相同知识库的生成可并行,发布只占短提交锁。interactive 优先于后台索引,但后台使用老化提升或最低服务配额,防永久饥饿。
|
||||
|
||||
全局队列初始上限 64、单工作区 16、单会话待处理 4,均可配置;队列超限及时返回,不能悄悄丢任务。显示队列位置或“前方有任务”,没有可靠完成时间估计时不显示虚构倒计时。
|
||||
|
||||
### 4.3 副作用与重试
|
||||
|
||||
任务声明 `read_only / staged_write / external_effect`,由工具权限校验而非模型自报。read_only 可按政策重试;staged_write 丢弃未提交 staging 后重试;external_effect 出现超时/失联时标记 `effect_unknown`,等待核对,不自动重复支付、发信或写入。
|
||||
|
||||
稳定的操作键使用 `task_id + logical_step_id`,attempt_id 只标记传输尝试,不应导致同一外部动作生成新的去重键。工具不支持幂等查询时,只能明确报告不确定。工作簿记录已完成与未知副作用;“已停止”不意味着“全部撤销”。
|
||||
|
||||
## 5. IPC 与请求上下文
|
||||
|
||||
首期使用本机 Unix socket 或匿名管道,采用长度前缀 JSON 控制帧与独立事件通道。协议禁止无上限 JSON/单行日志;控制帧初始上限 1 MiB,大文档通过授权 blob handle 传递。普通 stdout/stderr 仅诊断,不与协议混流。当前 NDJSON 文件桥保留为 legacy fallback。
|
||||
|
||||
新增 `InvokeRequestV2` 示例(提案):
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "InvokeRequestV2",
|
||||
"schema_version": 2,
|
||||
"task_id": "task_01",
|
||||
"run_id": "run_01",
|
||||
"attempt_id": "attempt_01",
|
||||
"lease": {"id": "lease_01", "epoch": 3},
|
||||
"entry": {"module": "agent/qdagent.mq.md", "function": "run"},
|
||||
"args_ref": "blob:sha256:...",
|
||||
"context": {
|
||||
"workspace_id": "ws_01",
|
||||
"snapshot_id": "generation_01",
|
||||
"selection_ref": "model-snapshot_01",
|
||||
"capabilities_ref": "grant_01",
|
||||
"deadline_at": "2026-10-26T08:10:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`grant_01` 是服务端验证的短寿命授权句柄,内容包括 principal、scope、根目录、允许效应、授权 revision、失效时间。worker 不使用浏览器任意传入的 fn/path。权限撤回后,新的工具调用必须重新验证 lease/grant,已开始的外部调用记录撤回时间与实际结果。
|
||||
|
||||
`RunContextV1` 保留原有字段与序列化兼容,内部增补显式 Context 对象;扩展 V2 由能力协商启用。event_sink、usage_sink、cancel_token 使用句柄,不使用外部可控路径。子任务通过 IPC/Command.env 显式继承,禁止父进程 `setenv`/`set_current_dir` 承担请求状态。
|
||||
|
||||
凭证在租约启动时通过受保护 IPC 或继承 FD 注入;不落任务 JSON、argv、日志或通用事件。模型快照可含 credential_ref,不含 api_key。即使选择相同 provider,不同请求的授权也不得合并。
|
||||
|
||||
## 6. 取消协议与状态 CAS
|
||||
|
||||
任务基础状态:`accepted → queued → running → finalizing → succeeded/failed`;queued/running 可进入 `cancel_requested → cancelled`;worker 失联进入 `interrupted`,副作用不明时进入 `needs_reconciliation`。这些是提案,产品可以将 succeeded 映射为 completed。
|
||||
|
||||
取消顺序:
|
||||
|
||||
1. 验证调用者对 task 的权限,CAS 写 cancel_requested、revision 与 requested_at;终态返回原状态,未知 ID 返回 404。
|
||||
2. 向匹配 lease/epoch 的控制通道发送 Cancel;不经任务解释器锁。queued 任务原子移出待派发队列。
|
||||
3. worker 在每次 LLM/工具调用、重试、子任务启动和提交之前检查 token;网络请求使用可中断句柄,不能仅在一行输出结束后才检查。
|
||||
4. 收到 ACK 仍不等于任务停止。等待 worker 报 stopped、落未完成副作用和用量,再 CAS 到 cancelled。
|
||||
5. 默认 2s 协作宽限、额外 3s 进程退出宽限,之后回收进程组。强制退出记 interrupted/needs_reconciliation,不能伪造干净 cancelled。
|
||||
|
||||
完成与取消竞争的线性化点是控制存储的 CAS:终态先提交则 cancel 返回 already_terminal;取消先提交则晚到 result 只可作为证据追加,不能重新变 succeeded。若外部动作已发生但响应晚到,记录 effect outcome,不撤销取消事实。
|
||||
|
||||
兼容文件 token 统一:不存在/空/`0` 为未取消,`1`/`cancelled` 为取消;读取失败返回诊断并按明确策略处理。删除 token 只能发生在持有者退出之后。移除全局 `MARQDO_CANCEL` 对多个任务的隐式作用域。
|
||||
|
||||
## 7. 事件、SSE 与背压
|
||||
|
||||
### 7.1 事件权威
|
||||
|
||||
生产者发出 `(run_id, attempt_id, local_seq)`,控制服务去重、持久化并分配 task 内单调 seq/event_id;客户端游标来自持久化日志,不是 handler 重连后从 1 开始计数。终态事件唯一,usage 可以在终态后以更高 usage revision 补录,但不能再次结束任务。
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "RuntimeEventV2",
|
||||
"schema_version": 2,
|
||||
"event_id": "task_01:42",
|
||||
"seq": 42,
|
||||
"task_id": "task_01",
|
||||
"run_id": "run_01",
|
||||
"attempt_id": "attempt_01",
|
||||
"type": "message.delta",
|
||||
"payload": {"message_id": "msg_01", "offset": 24, "text": "结果"}
|
||||
}
|
||||
```
|
||||
|
||||
不要直接破坏 RuntimeEventV1;新增字段/新版本由能力协商确定。普通用户可见 progress 只描述工具、阶段、文件等结构化事实;模型内部 reasoning 不作为完成证据,也不默认写入公开事件。
|
||||
|
||||
### 7.2 App SSE
|
||||
|
||||
新增任务 API 的事件订阅用 `GET /tasks/{id}/events`(最终产品前缀见配套设计),发送 SSE `id`/`event`/`data`;每个 data 是一个完整 JSON。客户端支持 Last-Event-ID、CRLF、多行 data、任意 UTF-8 分块和 EOF 未完成帧。重复事件按 ID 去重;message offset 检查缺口,缺口时恢复快照。
|
||||
|
||||
心跳 15s,客户端空闲探测建议 45s;二者可配置。事件保留建议 24h,最终结果/审计按项目政策更长保留。游标早于保留边界返回 410 `event.cursor_expired` 和快照入口,不能默默从最新开始丢字。
|
||||
|
||||
订阅断开默认只关闭订阅。App 显式停止通过 cancel API,切页/移动网络重连可恢复同一任务。传统同步 OpenAI 调用可保持 disconnect=cancel 的兼容策略,但必须显式区分,不能统一把所有浏览器断流当取消。
|
||||
|
||||
### 7.3 OpenAI 兼容流
|
||||
|
||||
`/v1/chat/completions stream=true` 输出 OpenAI chunk 与 `[DONE]`;App 运行事件由独立入口或协商的协议提供。禁止仅凭任意 JSON body 中 stream=true 将不相关路由自动改为 RuntimeEvent 协议。已发 header 后的错误使用该协议允许的错误帧;发 header 前可用 429/503/401 等正确状态码。
|
||||
|
||||
token delta 与最终完整答复分开:最终事件持有结果引用/摘要和 finish reason,不再次追加整篇正文。无 token 能力的适配器报告 `evented_batch`,界面用阶段状态,不把整包人为拆字伪装成实时。
|
||||
|
||||
### 7.4 背压和旧桥修复
|
||||
|
||||
worker→事件服务与服务→订阅者均有界。建议每订阅者 256 KiB 待发送内存,超出后断开慢消费者并允许按游标重连;事件正文已持久化所以不能静默丢 token。允许合并短时间 delta,必须保留 message offset;usage/终态/错误不丢弃。磁盘压力时停止接纳新任务,已有任务使用保留终态空间并按策略中断,不能无限写事件。
|
||||
|
||||
legacy NDJSON tailer 只消费换行已完整到达的字节;保留残片,offset 按真实消费字节移动。超长行、I/O error、文件旋转显式报协议错误。测试至少分三次写一个 JSON、中文多字节边界、恰好 EOF、超过上限、取消与尾帧竞争。禁止吞掉 Scanner.Err() 继续宣告成功。
|
||||
|
||||
## 8. 只读快照与真正有效的缓存
|
||||
|
||||
### 8.1 快照一致性
|
||||
|
||||
分析服务构建 SnapshotKey:规范化 project identity、允许扫描范围、源文件闭包摘要、依赖/来源/配置摘要、compiler/lib/ext 版本、policy/schema、解析选项。输入由编译器枚举,不另造“前 200 个根目录文件”规则。
|
||||
|
||||
读取一次快照内的正文、语义和来源,完成后复核输入 manifest;文件在分析中发生变化则重试一次或返回 `snapshot.changed`。对不可变发布代直接读其 manifest。mtime/size 只用于快速发现候选变更,不能代替内容摘要;watcher 丢事件后以完整扫描修复。
|
||||
|
||||
### 8.2 缓存结构
|
||||
|
||||
- `AnalysisCache<SnapshotKey, Arc<Compilation>>`:实际编译结果。
|
||||
- `DocumentCache<(SnapshotKey, document_digest, renderer_version, view_options), DocumentView>`:安全 fragment、outline、nodes。
|
||||
- 同 key single-flight 的 follower 等待同一个 Result,不再次 compile。
|
||||
- RAII guard 在 error/panic/cancel 时清除 inflight 并唤醒 waiters;等待者有自己的 deadline。
|
||||
- compile 失败只返回给当前等待者,短暂错误缓存需显式 TTL;不得调用“finish_success”标记失败结果可复用。
|
||||
- LRU 按字节而非条数上限管理;初始预算按部署内存的 10% 且设硬上限,长文档超限不缓存或分页;记录淘汰原因。
|
||||
- 每次读取先授权再查缓存;权限敏感的响应按 scope/policy 分区。全局缓存只共享不含用户秘密且访问已验证的不可变语义数据。
|
||||
|
||||
性能证据必须统计 `compile_invocations`、输入字节、缓存结果命中、leader/follower、wait_ms、evictions;现有 hit counter 不能证明减少编译。
|
||||
|
||||
### 8.3 HTML 与来源定位
|
||||
|
||||
沿用 DocumentViewV1,明确 `render.format=plain_text|sanitized_html`。HTML 通过解析器白名单清洗标签/属性/URL scheme,剥离脚本、事件属性、危险 CSS/嵌入与可执行 SVG;相对资源经受限资源端点解析。纯文本始终转义,不以 `<` 字符判断类型。
|
||||
|
||||
node_id、revision、generation 必须从定位到读取到浏览器滚动完整传递。不存在历史快照返回 version_unavailable;返回当前版是用户显式操作且标明版本变化。no_exec 测试分别验证“无语言副作用”和“浏览器无脚本执行”,二者不能互相替代。
|
||||
|
||||
### 8.4 首次编译的性能改造
|
||||
|
||||
本轮 `v3_corpus_1000_compile_smoke` 在 debug 下 243.012s,超过既有 180s 门禁;缓存不能缩短一个全新项目的第一次编译。先为 `knowledge::compile` 的语义编译、语义落盘、检索 extract、检索落盘、manifest 分别加 span,再细分 `ProjectIndex::build`、源枚举/解析、导入挂接、绑定、效应传播、闭包摘要。记录文件数/总字节、节点/边数、parse 次数与序列化字节量。
|
||||
|
||||
源码中 `sema::compile_project` 在每个文件查找 module path、每个 module 过滤全量 symbols,属于值得测量的重复扫描;未取得 profile 前不将其认定为 243s 的唯一根因。优化候选按证据选择:建立 path→module 与 module→symbols 索引;同一输入快照共享 parsed module;绑定/依赖图用稳定索引;效应传播用 worklist;闭包按 SCC DAG 计算并保留循环依赖语义;减少每制品重复序列化和小文件写入。
|
||||
|
||||
任何优化须保持排序确定性、SymbolId/ArtifactId 与证据摘要规则,禁止通过跳过来源校验、删掉复杂 fixture 或改大超时过关。用 100/1000/10000 单元、无依赖/链/菱形/循环四种图分别测时间与 RSS;给出随 V/E 增长的曲线。release 和 debug 分开;已有 debug 门禁恢复通过只是第一步,§12 的 release 预算仍须独立验收。
|
||||
|
||||
## 9. 写入、发布与数据库边界
|
||||
|
||||
执行 worker 只写任务 staging;需要写工程时通过沙箱产生文件 diff。发布以 kb 资源锁 + expected_generation CAS 提交。锁只覆盖校验当前版本和切换 manifest,不覆盖 LLM、全文编译或网络调用。
|
||||
|
||||
完整发布代先写临时目录、校验摘要、完成必要 fsync,再原子 rename 到不可变 generation 目录;最后在同一文件系统原子替换 CURRENT。源码、编译投影和证据在该 manifest 下对齐。SQLite 中的产品“published”是投影,不参与虚构的文件/数据库跨系统原子事务。通过 publish journal + outbox 恢复:CURRENT 已切换但 DB 未更新时重放;未切换时旧代继续有效。
|
||||
|
||||
project_id 与逻辑文档 ID 不随 staging/generation 的物理目录名变化。manifest 保存稳定 project identity、逻辑相对路径和实际 blob/generation 映射;导入引用按项目规则解析,不能因复制到发布目录而重新生成全部符号身份。框架若尚不能区分逻辑身份与物理根,先补该能力,再启用 generation 目录发布;迁移 fixture 必须断言未修改的知识 ID 和跨文档绑定保持稳定。
|
||||
|
||||
任务表/幂等/CAS/outbox 需要通用事务能力:先审计 `ext/data` 是否支持事务、条件更新、受影响行数、复合唯一键;缺少则在 Marqdo 数据层补齐,不能用 `.mq.md` 的先读后写模拟原子性。SQLite 使用短事务和有界 busy timeout;WAL/连接池是否开启需按现有插件适配测试确认。单主本地盘为首期部署范围,跨主机共享 SQLite/NFS 不在首期承诺内。
|
||||
|
||||
## 10. 临时缓解与迁移
|
||||
|
||||
### 10.1 进程池完成前
|
||||
|
||||
- 保留 1500ms busy 预算和 Retry-After;展示繁忙,而非无限 spinner。
|
||||
- 在接纳任何 SSE 之前取得覆盖整个 bridge 生命周期的 legacy 单飞许可,第二条流在修改环境变量/发送 200 前失败;普通请求不得绕过许可观察被污染的状态。
|
||||
- 断连只标记该任务取消,清理由监督者等待执行结束后负责;无法确认停止时报告 stopping/interrupted。
|
||||
- 上述步骤只降低 R02/R03 风险,仍不算并行 Endpoint。
|
||||
|
||||
### 10.2 功能开关(新增提案)
|
||||
|
||||
`runtime.isolated_execution`、`runtime.task_control_v2`、`runtime.event_replay`、`view.analysis_cache_v2`、`runtime.read_lane`。探测返回 feature version 和限制。二进制版本相同不等于所有 native 插件都有该能力。
|
||||
|
||||
先离线 fixture → 本地假模型 → 新 workspace → 只读 ASK → staging BUILD → 真正发布。禁止对有副作用的请求做双执行 shadow;允许 shadow 只读分析并比较摘要。
|
||||
|
||||
回滚先停接新任务、drain 已接纳任务、保留任务/事件和新 schema;不能将活动任务交给不认识 lease 的旧进程。只读当前发布代可以保留,写入能力按版本协商关闭。ADR 0008 仅在并行金样例与回滚演练通过后由新 ADR 正式替代。
|
||||
|
||||
## 11. 开发包与依赖
|
||||
|
||||
下列新模块路径是建议,不代表文件已存在。
|
||||
|
||||
| 包 | 建议落点与改动 | 前置 | 验收 |
|
||||
|---|---|---|---|
|
||||
| M-R0 基线/缓解 | `host/plugin.rs`、`httpx/routes_invoke.go`;固定并发屏障、旧桥许可、取消清理 | 无 | 双流 busy 不改写 A 的桥;断连无提前清理 |
|
||||
| M-R1 请求上下文 | `workbench/run_context.rs`、`host/subtask.rs`、LLM 插件;统一 token 与显式 sink | M-R0 | 父子运行模型/事件/取消不串;空 token 不取消 |
|
||||
| M-R2 Worker/IPC | 新 `runtime/{supervisor,worker,protocol,lease}.rs`;独立进程与握手 | M-R1 | 两任务并行、超时回收、子进程无孤儿 |
|
||||
| M-R3 接纳/控制 | 新 `runtime/{task_store,scheduler,control}.rs` 与 ext/data 事务 primitives | M-R2 | 原子幂等、队列公平、取消与结果 CAS |
|
||||
| M-R4 事件协议 | `workbench/event_sink.rs`、Web stream/IPC、协议适配器 | M-R3 | 断点恢复/去重/半帧/慢消费/磁盘错误 |
|
||||
| M-R5 读服务/缓存 | `analysis_cache.rs`、`document_view.rs`、`locator.rs`、MLSP transport | M-R1,可独立推进 | 100 同 key 真正只编译一次;嵌套变更正确失效 |
|
||||
| M-R5b 首次编译性能 | `knowledge/compile.rs`、`sema/ir.rs`、ProjectIndex;分阶段 profile 与按热点优化 | 基线语义回归 | 现有 1000 单元门禁恢复;release 与规模曲线报告,身份和语义不变 |
|
||||
| M-R6 提交底座 | `knowledge` generation/manifest、publish journal、条件更新 | M-R3 | 崩溃点恢复,旧代不被半发布污染 |
|
||||
| M-R7 整合与发布 | 版本探测、CI、release、文档/ADR | M-R4–R6 + 产品接入 | §12 全部必要门禁与回滚演练 |
|
||||
|
||||
先以本地文件和单机进程解决明确缺陷;不要把集群、远程 worker、复杂 Actor 系统设为第一阶段依赖。
|
||||
|
||||
## 12. 验收、性能预算与可观测性
|
||||
|
||||
### 12.1 正确性矩阵
|
||||
|
||||
| 编号 | 场景 | 必须观察到的结果 |
|
||||
|---|---|---|
|
||||
| MR-01 | A/B 两个 60s 假模型流 + 20 次/s 模型/设置读取 | 两流不混字、不混模型;短请求 200;无 interpreter busy |
|
||||
| MR-02 | B 在 A 运行时接纳失败/断连 | A 事件和取消句柄完全不变 |
|
||||
| MR-03 | 同时取消 A、完成 B;取消重复/晚到 | A 正确停;B 成功;终态不回退;审计只结算一次 |
|
||||
| MR-04 | 帧拆字节、多行、EOF 半帧、重连 | 无丢字/重字;协议错误可见;游标恢复精确 |
|
||||
| MR-05 | 慢消费者、磁盘满、worker 崩溃 | 内存有界;事件不静默丢失;状态可恢复 |
|
||||
| MR-06 | 100 个相同读取并发 | 实际编译一次;后续结果复用;失败 leader 不挂死 follower |
|
||||
| MR-07 | 改子目录依赖、第 201 个文件、配置、来源 | SnapshotKey 改变;缓存失效;旧业务句柄拒绝 |
|
||||
| MR-08 | 上传有写文件/网络代码、HTML 事件属性 | 阅读不求值;浏览器脚本不执行;正文可读 |
|
||||
| MR-09 | 发布每个持久化边界 kill | 仅旧完整代或新完整代可见;DB 投影最终追上 |
|
||||
| MR-10 | 并发幂等请求、取消队列、权限撤回 | 单一 task;同键异内容 409;无越权派发 |
|
||||
| MR-11 | kill 后副作用未知 | 不自动重放,显示 needs_reconciliation |
|
||||
| MR-12 | 降级/升级、混合 CLI/ext/native | 缺能力明确禁用,不报已支持 |
|
||||
|
||||
### 12.2 性能目标(待测验收预算)
|
||||
|
||||
基准机:Linux 4 vCPU/8 GiB SSD;release 构建;1000 个知识单元、文档总量 10 MiB、500 条历史 run;假上游固定 60s、每秒 20 个增量;30s 预热、5min 采样、3 轮,报告 p50/p95/p99、错误率、CPU、RSS、磁盘。另测 10k 单元和 100 MiB 以确定容量上限,不能直接沿用小数据阈值。
|
||||
|
||||
| 指标 | 初始预算 | 说明 |
|
||||
|---|---|---|
|
||||
| 短读/任务接纳 | p95 ≤200ms,p99 ≤500ms | 两个长任务运行时;过载拒绝另统计 |
|
||||
| 取消受理 | p95 ≤200ms | 独立于任务停止耗时 |
|
||||
| 协作取消收敛 | p95 ≤2s | 假上游/可中断工具;不可中断外部动作按不确定处理 |
|
||||
| 已就绪 worker 调度 | p95 ≤100ms | 不含排队、模型首 token 时间 |
|
||||
| 事件到订阅者新增延迟 | p95 ≤100ms | 从已收到 provider delta 计,不承诺上游 TTFT |
|
||||
| 1000 单元首次分析 | p95 ≤2s | 候选目标,超标先剖析,不降低正确性 |
|
||||
| 热 DocumentView | p95 ≤100ms | 真实复用结果;长文档另测分页 |
|
||||
| 慢消费者缓冲 | 每订阅 ≤256 KiB | 总连接数也必须受限 |
|
||||
| 压测结束内存 | 稳态可回收、无持续增长 | 记录每次重复轮次 RSS;不能只看一次峰值 |
|
||||
|
||||
新指标至少有 queue_depth/wait、worker_active/recycle、admission_rejected、cancel_ack/stop、orphan_count、event_lag/replay/duplicate、compile_invocations/cache_result_hit、publish_conflict/recovery。指标标签禁止 task_id、原始路径、prompt 等高基数/敏感值;trace 中可记录脱敏 task/attempt/lease 与构建摘要。
|
||||
|
||||
现有测试继续保留:`cargo test --offline --locked --test workbench_contracts --test executable_knowledge`、LLM 插件单测、Go httpx 测试。新增集成门禁必须启动真实 HTTP→IPC→worker 路径并使用本地假模型;不能以手写 fixture 或写 token 文件代替端到端行为。本轮执行结果统一记录在[跨库复核 §5](../../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md)。
|
||||
@@ -1,5 +1,7 @@
|
||||
# 调用写法评估:`[函数名](参数)` 与更接近人读的调用形式
|
||||
|
||||
> **2026-10-23 审阅裁决**:“原型回退”仅描述当时实验,当前 payload 已支持 `f(args)` 和具名实参。旧 Classic 索引冲突不再决定默认语法;保留熟悉的调用形态与文档原生描述并不矛盾。参见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓),不要把本文备选形式当作已批准的新语法。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 待决策**(原型已验证并已回退,未合入代码) |
|
||||
|
||||
@@ -0,0 +1,435 @@
|
||||
# qdagent 知识工作台所需的多模型运行与 View 嵌入支持
|
||||
|
||||
日期:2026-10-25。状态:Partial / in progress(M00–M06 合同门禁 + F02 agent.step/resume 假模型 e2e + USTC live selection smoke + DocumentView 浏览器只读核对已落地;F 矩阵未全绿,不得整文宣称 Verified;原生非兼容适配器仍 unavailable)。
|
||||
|
||||
源码基线:Marqdo `dc853fb`(2.1.2),qdagent `643dd31`,已同步远端。应用侧配套:[qdagent 多提供商与知识工作台](../../../qdagent/doc/design/multi-provider-and-knowledge-workbench-2026-10-25.md)。跨仓相对链接以兄弟仓库工作区布局为准。
|
||||
|
||||
沿用 [文档原生开发指导](document-native-development-guide.md)、[可执行知识规范](../next/Marqdo_Executable_Knowledge_Development.md) 与 [View 阅读体验设计](view-knowledge-reading-experience-2026-10-24.md)。本规格补充嵌入、调用身份和运行观测,不另造语言或第二套知识真相。
|
||||
|
||||
## 1. 框架目标与责任边界
|
||||
|
||||
让任何宿主应用可以按请求构造模型、追踪所有真实模型调用、以只读 View 展示知识文档,并从引用定位到版本一致的知识与来源。qdagent 的设置页、模型目录持久化、账单展示、会话导航、工作区授权和画像管理由应用实现。
|
||||
|
||||
| 能力 | Marqdo 拥有 | qdagent 拥有 |
|
||||
|---|---|---|
|
||||
| 多提供商 | 协议适配、模型 handle、能力协商、调用参数、归一化错误 | 提供商账户/端点/模型 CRUD、默认选择、凭证管理 |
|
||||
| 模型路由 | Runtime 执行显式选择和允许的辅助模型策略 | 用户选择、工作区模型允许列表、路由授权 |
|
||||
| 用量 | 每次请求/重试的身份、usage 来源与完整性、子任务传播 | 持久化账本、价格版本、费用估算、筛选/导出/预算策略 |
|
||||
| 实时运行 | 逐事件输出、取消传播、背压、终态 | HTTP 会话鉴权、轮次映射、持久化事件重放、断线 UX |
|
||||
| View | 文档语义、正文/目录/步骤、纯读取、稳定定位、可嵌入输出 | 容器/主题/深链/阅读历史/产品操作 |
|
||||
| 知识引用 | 解析/歧义/版本/来源定位、验证多维事实 | 消息引用标识、授权、历史资料保存与入口 |
|
||||
| 工作区 | 按显式 project 编译/查询,缓存及输出隔离 | 工作区 registry、root 解析、用户权限、文件管理 |
|
||||
| 画像与笔记 | 通用文档渲染、来源关联、验证/编译原语 | 内容策略、提案/确认、生命周期、展示和上下文选择 |
|
||||
|
||||
不在语言核心添加 provider/workspace/profile 关键字;沿用 `.mq.md` 函数、结构化参数、扩展插件和 MLSP。不得让“可读”取决于用户手填 hash、node id、source span 等机器字段。
|
||||
|
||||
## 2. 2.1.2 已完成的基础与剩余缺口
|
||||
|
||||
### 2.1 当前可复用基础
|
||||
|
||||
| 源码 | 已有内容 | 本规格的增量 |
|
||||
|---|---|---|
|
||||
| [ext/ai/llm](../../ext/ai/llm.mq.md) | `create(model,base_url,api_key,backend,name)`、命名 handle、ask/stream | 无需重做基础 handle;增加隔离策略、协议能力、完整消息与调用身份 |
|
||||
| [LLM plugin](../../plugins/llm/src/lib.rs) | 兼容 chat completions HTTP、SSE 文本读取、非流 usage 三项 | SSE usage/异常终态/每次调用观测、可取消持续输出 |
|
||||
| [Agent](../../ext/ai/agent.mq.md) | Agent 唯一 loop、router_model、EFI、绑定/发布门闩 | 主/路由/修复/生成工作簿的模型及计量上下文传播 |
|
||||
| [语义 IR](../../src/sema/ir.rs) | Artifact、KnowledgeStep、KnowledgeRef、SourceLocation、BoundCall | View 面向阅读的完整投影及 ID/版本映射 |
|
||||
| [MLSP](../../src/mlsp.rs) | `artifact.inspect`、`link.resolve`、verify/bind/execute、document.extract | 只读 document/引用解析及 capabilities 版本握手 |
|
||||
| [KnowledgePresentation](../../src/view/knowledge_present.rs) | schema v1,artifact nodes、依赖、generation/page token | 保留兼容,新增正文/目录/refs/source spans/结构化诊断 |
|
||||
| [View output](../../src/view/output.rs) | `--no-exec` 在构建前跳过 capture | 将纯读取能力暴露给嵌入客户端,不复用会执行的 live 默认路径 |
|
||||
| [document extract](../../src/document_extract.rs) | TXT/MD,partial,PDF/DOCX/OCR 显式不支持 | 能力查询及稳定来源定位;不要求本阶段重造 OCR |
|
||||
| [数据资源](../../ext/data/db.mq.md) | SQLite migrate/exec/事务、唯一/索引声明 | 应用可直接承载 registry/账本/索引;不需另加产品专用数据库框架 |
|
||||
|
||||
2.1.2 已把知识辅助块移到正文后并折叠,跳过 module/type 卡片,显示“结构步骤非实际执行时序”。旧 View 截图中的最初顺序问题已有改进;但这不意味着已提供能直接放进 qdagent 右栏的独立阅读 SDK。
|
||||
|
||||
### 2.2 直接影响集成的事实
|
||||
|
||||
1. `llm.complete_result` 当前仍走 `openai.chat_completions`,Ollama 也是兼容路径;`backend` 名称不是任意原生协议适配器。
|
||||
2. `plugins/llm::chat_completions` 从 `prompt` 构造一条 user 消息,没有完整消息列表契约;不能靠模型选择 UI 弥补历史丢失。
|
||||
3. `usage_from` 缺字段填零;Agent `model_turn(stream=True)` 直接 `tokens=0`,最终 `token_cost=tokens`。现有输出无法支撑准确货币或流式用量显示。
|
||||
4. 原生 LLM SSE parser 返回事件数组;`ext/ai` 有流事件机制,但应用所用 [web invoke](../../plugins/web/internal/httpx/routes_invoke.go) 在 `CallLibPath` 返回后才写 `{sse}`。需要打通实际增量路径,不能只换 Content-Type。
|
||||
5. Agent 生成子工作簿的模板仍能构造默认 `llm.llm`;只传主 agent handle 不足以证明嵌套调用沿用了用户选定模型。
|
||||
6. `build_file_view` 默认传 `execute=true`;live GET 路径使用它。嵌入阅读绝不能直接暴露这个入口。
|
||||
7. Presentation 目前按 artifacts 投影,缺全量 prose/heading/step/ref 的统一阅读树;`render_knowledge_html` 另从 steps/refs 渲染,容易与 JSON 事实发生覆盖差异。
|
||||
8. `source_digest` 当前由语义 artifact hashes 汇总,`semantic_generation` 是语义散列截短,`page_token` 来自原文件。它们不能直接分别冒充原文 revision、`.marqdo/CURRENT` generation 和跨请求完整版本。
|
||||
9. `sse_accepts_packet` 对缺 token 的 legacy 包放行;正式嵌入客户端必须采用严格版本匹配,而不是继承这项兼容宽容。
|
||||
10. `artifact.inspect/link.resolve` 每次走当前 semantic_project;现有入口不保证可以读任意历史 generation。历史引用需要明确的保留与解析合同。
|
||||
|
||||
## 3. 共用合同、版本与能力发现
|
||||
|
||||
合同名字固定为 `ModelSelectionV1`、`ModelCallUsageV1`、`RuntimeEventV1`、`DocumentViewV1`、`KnowledgeLocatorV1`;它们是这次扩展的 v1,与当前 KnowledgePresentation `schema_version:1` 不混用。新增 read model 不强行给已有 JSON 改含义。
|
||||
|
||||
新增 MLSP `capabilities`(拟议):
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"runtime_version": "implementation-version",
|
||||
"features": {
|
||||
"llm.explicit_selection": 1,
|
||||
"llm.call_usage": 1,
|
||||
"runtime.event_stream": 1,
|
||||
"runtime.cancel": 1,
|
||||
"view.document": 1,
|
||||
"view.reference_resolve": 1
|
||||
},
|
||||
"adapters": [{"id": "openai-compatible", "available": true}],
|
||||
"extract_formats": [{"format": "text", "available": true}],
|
||||
"loaded_plugins": [{"name": "llm", "capabilities": ["call_usage_v1"]}]
|
||||
}
|
||||
```
|
||||
|
||||
只返回实际探测且加载可用的功能。不能仅从 CLI 版本推断 native plugin 符号存在;字段未知时应用降级或解释不支持。协议允许新增可选字段;删除/改变状态语义必须提升 major contract version。能力查询不调用供应商、不执行项目代码。
|
||||
|
||||
MLSP 继续使用现有 `{id,ok,result|error}` envelope;qdagent `/api/app` 转为自己的 `{ok,schema_version,data|error}`,不要要求它们共享同一层 envelope。所有合同提供成功、partial、unknown、unsupported、stale、cancelled、错误 fixtures 与 JSON Schema;应用和框架从相同样本跑消费测试。
|
||||
|
||||
## 4. M01:请求级模型身份与适配器
|
||||
|
||||
### 4.1 显式选择合同
|
||||
|
||||
应用将配置解析成可信 `ModelSelectionV1` 后传入 runtime,核心字段:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"selection_id": "sel-1",
|
||||
"provider_id": "provider-1",
|
||||
"model_id": "model-2",
|
||||
"provider_revision": 3,
|
||||
"model_revision": 5,
|
||||
"adapter": "openai-compatible",
|
||||
"endpoint_ref": "endpoint-local-1",
|
||||
"credential_ref": "credential-local-1",
|
||||
"credential_version": 2,
|
||||
"upstream_model_id": "example-model",
|
||||
"parameters": {"max_output_tokens": 2048},
|
||||
"fallback_policy": "disabled"
|
||||
}
|
||||
```
|
||||
|
||||
endpoint/credential 引用由宿主 resolver 解析为调用局部配置;短期直接向现有 `llm.create` 传参数亦可,但这不是可序列化的 public snapshot。API key 不能出现在 events、debug JSON、生成工作簿、prompt、错误 URI 或持久化模型对象中。
|
||||
|
||||
当前显式 args 优先的能力保留;新增 strict selection 模式:字段缺失报错,不回退 process env/frontmatter。无凭证端点通过 `auth_mode:none` 显式表达,避免 falsy key 触发环境 fallback。legacy 命令行未传 selection 时仍可沿用原有 env 行为;本地便利兼容不扩散到应用请求。
|
||||
|
||||
每个运行绑定 selection snapshot,后续设置改变不修改它。主模型与 router_model 分开但同属受控 selection 集;manual 模式所有需要 LLM 的步骤固定选择,REUSE 无模型调用仍允许。fallback 必须由宿主明确授予、列出允许目标,不能因额度/网络错误静默转到另一个供应商。
|
||||
|
||||
### 4.2 子任务和工作簿传播
|
||||
|
||||
统一 `RunContextV1 = run_id,parent_run_id?,turn_id?,workspace_scope,selection_context,usage_sink,event_sink,cancellation_token`。进入 ask/step/plan、repair、summary、learn、子工作簿时显式继承;新 run/call id 关联父节点。
|
||||
|
||||
生成工作簿只包含不可重放滥用的运行上下文引用,不写密钥;执行时宿主注入/解析。独立导出工作簿在缺凭证/选择时清晰要求配置,不偷偷取默认模型。在任何 child/spawn/resume 路径遗漏 selection 都应由 strict 模式拒绝并给出定位诊断。
|
||||
|
||||
同时审查环境变量不只是凭证:project root、temp request、输出目录、event sink 也必须请求局部,不能在并发运行中依靠进程可变状态串联。
|
||||
|
||||
### 4.3 适配器与消息
|
||||
|
||||
适配器最小职责:验证 selection、报告能力、序列化请求、解析增量结果、usage 归一化、取消传输、错误归一化。可选 discovery/probe;无模型列表 API 仍允许手工配置。
|
||||
|
||||
V1 消息包含 role 和 content parts,parts 最低 text,附件通过受控引用;tool call/result 要保持配对及 role。不支持的 part/parameter 返回 `llm.unsupported_input/parameter`,不能悄悄转成空字符串。适配器决定协议格式,Agent 决定行为;qdagent 不拥有供应商协议代码。
|
||||
|
||||
阶段一验证两个不同兼容端点,分别至少两个模型,以及无密钥本地兼容服务。阶段二增加原生非兼容协议:每个适配器需具名能力、流式/非流式/错误/取消/usage fixtures 和独立接入说明后才能在 capabilities 中置 available。不能把所有厂商默认走兼容接口后统称全支持。
|
||||
|
||||
统一参数采用 `max_output_tokens,temperature?,stop?,reasoning?` 等语义键,能力不支持时明确拒绝;不随意透传客户端未知参数。实际 upstream model 若与请求 id 不同,在 result 中保留 requested/resolved 两者。供应商版本与价格随时变化,本文仅锁定内部合同;实施原生适配器时另附当时官方协议核验记录。
|
||||
|
||||
## 5. M02:用量、事件流与取消
|
||||
|
||||
### 5.1 ModelCallUsageV1
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"call_id": "call-1",
|
||||
"attempt_id": "attempt-1",
|
||||
"parent_call_id": null,
|
||||
"run_id": "run-1",
|
||||
"turn_id": "turn-1",
|
||||
"workspace_scope": "ws-1",
|
||||
"purpose": "answer",
|
||||
"selection_id": "sel-1",
|
||||
"provider_id": "provider-1",
|
||||
"model_id": "model-2",
|
||||
"requested_model": "example-model",
|
||||
"resolved_model": "example-model",
|
||||
"status": "succeeded",
|
||||
"usage_state": "final",
|
||||
"measurement_source": "provider",
|
||||
"input_tokens": 1200,
|
||||
"output_tokens": 300,
|
||||
"total_tokens": 1500,
|
||||
"cached_input_tokens": 800,
|
||||
"reasoning_output_tokens": null,
|
||||
"revision": 2,
|
||||
"provider_request_id": "opaque-provider-id",
|
||||
"started_at": "2026-10-25T01:00:00Z",
|
||||
"finished_at": "2026-10-25T01:00:03Z"
|
||||
}
|
||||
```
|
||||
|
||||
字段要求:tokens 非负整数或 null;`usage_state = pending|partial|final|unknown`;`measurement_source = provider|estimated|none`。估算可作为单独 estimate 字段返回,不能覆盖实测字段。供应商没有报告时用 null/unknown,传输失败不能自动解释成零消耗。
|
||||
|
||||
`status = started|succeeded|failed|cancelled|interrupted`,status 和 usage_state 独立。每个真实发出的 HTTP attempt 一个唯一 id;预检拒绝只产生 call-rejected 事件,不伪造已发送请求。框架重试必须可观测:所有 attempts 包括失败/有部分输出的 attempt;重复 delivery 的 usage revision 不算新请求。
|
||||
|
||||
累计 SSE usage 在同 attempt 内按 revision 发布完整快照,不让上层猜是 delta 还是 total。provider 总量与 input/output 不一致时保留原始读数及 `usage.inconsistent` 诊断,不自行强制改账;细分字段要标 inclusion 关系以避免缓存、reasoning 与总量重复相加。只保留脱敏的必要 provider usage 扩展字段,不把完整原始响应当 public usage。
|
||||
|
||||
纯知识复用用 run summary 中 `llm_calls=0`、`measurement_source=none` 表示,而不是伪造一条耗费为零的模型请求。框架输出 `call_count,known_tokens,unknown_call_count`,宿主据此显示覆盖程度。
|
||||
|
||||
### 5.2 事件流合同
|
||||
|
||||
```text
|
||||
run.started
|
||||
→ model.resolved
|
||||
→ model.call.started
|
||||
→ message.delta / tool.progress / usage.updated(零到多次)
|
||||
→ model.call.finished(每个 attempt 各一条)
|
||||
→ document.produced / citation.produced(仅有真实对象时)
|
||||
→ run.completed | run.failed | run.cancelled | run.interrupted
|
||||
```
|
||||
|
||||
`RuntimeEventV1` 公共字段:`schema_version,event_id,seq,run_id,parent_run_id?,turn_id?,workspace_scope,timestamp,type,payload`。seq 每个 run 单调;不同子 run 可以并发,宿主按所属 turn 另分配产品流序号。runtime terminal 事件仅一次;后续补全计量用有 revision 的 `usage.updated`,不能改变内容终态。
|
||||
|
||||
事件 sink 支持有界队列和背压;慢客户端不能无限堆积 token。丢弃可合并 progress 时记录 gap;内容与终态事件不可无声丢失。连接终止不会自动重跑;持久化/重放由应用,但框架需在实时回调后仍提供可核对的 final result/call summaries。
|
||||
|
||||
同一真实调用所有数据使用同一 run/call identity,不能流与最终结果各生成一套 id。工具参数/结果按 policy 脱敏,不把 secret、完整私有文件、模型内部推理链写入默认进度事件。
|
||||
|
||||
### 5.3 打通传输而不重造 Agent
|
||||
|
||||
需要同时打通 LLM provider reader → Agent event sink → host callback → web streaming handler → 浏览器。现有 `sys.stream_publish` 可以作为底座,但要验证它是否绑定正确请求且在 `CallLibPath` 返回前可写出;不能用进程全局事件广播混流。
|
||||
|
||||
可实施路径:新增 web 流式 invoke/job primitive,由框架注册 run handle 和 event sink;调用返回 handle 后宿主可订阅/取消。采用插件 ABI 增量接口及能力检测,旧 ABI 未支持时拒绝新 streaming 模式,保留旧完整返回。qdagent 只挂产品鉴权、event id 持久化和 HTTP 端点,不添加另一个推理循环。
|
||||
|
||||
SSE parser 必须处理任意 TCP 分块、UTF-8 跨块、CRLF、多行 data、注释 heartbeat、尾帧、usage-only 帧、[DONE]、连接在结束前关闭。未知可选事件可忽略且记诊断;已知事件无效、异常截断不能静默当成功完成。公共 framing 不包含供应商私有认证信息。
|
||||
|
||||
### 5.4 取消与重试
|
||||
|
||||
取消 token 传到 HTTP reader、循环、工具边界、子任务。取消后不得启动新模型调用/新工具;正在进行的不可抢占操作返回 `cancelling` 和是否仍运行,结束后才报告 cancelled。明确“停止生成”不能撤销已经完成的外部写入。
|
||||
|
||||
调用超时、连接错误、鉴权失败、限流、协议错误分开;错误字段包含 `code,retryable,retry_after_ms?,call_id,attempt_id,usage_state`。不要将原始包含密钥/完整 prompt 的错误直接传到 UI。限流/短暂网络错误按有界策略重试;已产生输出的重试不能接在同一条文字后假装连续,也不能把未知结果的外部工具操作自动重做。
|
||||
|
||||
费用价格由应用管理。框架保留 legacy `tokens/token_cost` 并标注 deprecated/单位为 token,新功能只消费 ModelCallUsageV1,避免破坏旧脚本的同时停止误用。
|
||||
|
||||
## 6. M03:无执行的嵌入式 View
|
||||
|
||||
### 6.1 新入口与纯度约束
|
||||
|
||||
新增内部公共服务 `analyze_document → project_read_model → render_fragment`;MLSP 暴露 `view.document`,可选 CLI `view document --json --no-exec` 与它共享实现。不要让新入口先进入 `build_file_view` 再隐藏按钮/清空输出。
|
||||
|
||||
读取允许:授权范围内原文/已编译快照/已有验证记录。读取禁止:运行模块入口、加载文档声明插件、执行外国代码、LLM、自动抓取 URL、写回、重新验证执行测试、隐式发布/重编译写盘。静态分析/内存投影允许;缺完整依赖时明确 partial。任何需要执行的行为必须另走显式 execute/verify 产品流程。
|
||||
|
||||
当前 `compile_project` 声明不加载插件/不求值/不抓取来源,可复用;缓存写入若需要,作为单独的宿主管理路径,不能把“读文档”隐含写项目目录。
|
||||
|
||||
### 6.2 DocumentViewV1 结构
|
||||
|
||||
请求由宿主传入受控 project 和文档定位,宿主必须已授权;框架仍做路径边界校验。
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "view.document",
|
||||
"params": {
|
||||
"project": "/authorized/project",
|
||||
"document": "knowledge/example.mq.md",
|
||||
"revision": "sha256:requested-source",
|
||||
"generation": "generation-id-or-null",
|
||||
"mode": "read",
|
||||
"node_id": "optional-node",
|
||||
"include": ["fragment", "outline", "nodes", "refs", "diagnostics"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "DocumentViewV1",
|
||||
"schema_version": 1,
|
||||
"document": {
|
||||
"id": "project-relative-document-id",
|
||||
"title": "知识标题",
|
||||
"source_revision": "sha256:raw-bytes-digest",
|
||||
"semantic_digest": "sha256:semantic-facts",
|
||||
"generation_id": null,
|
||||
"presentation_token": "opaque-token"
|
||||
},
|
||||
"execution": {"state": "not_run", "run_id": null},
|
||||
"render": {"format": "sanitized_html", "fragment": "<article class=\"mq-document\">…</article>", "renderer_version": "1"},
|
||||
"outline": [],
|
||||
"nodes": [],
|
||||
"refs": [],
|
||||
"diagnostics": [],
|
||||
"completeness": "complete",
|
||||
"capabilities": {"read_only": true, "has_semantics": true}
|
||||
}
|
||||
```
|
||||
|
||||
`generation_id` 来自实际编译投影;即时分析但未编译发布时为 null,同时给 semantic_digest。`source_revision` 是原始文件字节摘要,不能复用当前基于 artifacts 的 `source_digest`;`presentation_token` 覆盖 source revision、generation/semantic digest、依赖版本、renderer/schema、读取策略与请求范围。应用缓存还必须含用户授权范围。
|
||||
|
||||
`completeness = complete|partial`;语法损坏时允许返回原文与结构化诊断,但不能展示猜测出来的正常知识图。失败用 error envelope,不能空字符串伪装“文档无内容”。大文档 response 支持 section cursor 和 truncated 标识,限制要可配置并在响应中标明。
|
||||
|
||||
### 6.3 统一正文、目录、节点与引用
|
||||
|
||||
节点包含 `id,parent_id,kind,label,order,source_span,artifact_id?,ref_ids[],children[]`。kind 包括 heading/prose/table/code/observe/derive/call/iterate/narrate/navigation;正文树来自 parser/classifier/sema 的共同投影,不能只枚举 functions 后把每个步骤标 call。
|
||||
|
||||
outline 描述读者文档层级,不简单把函数和对象全铺成同一级。HTML 中节点带 `data-mq-node`,引用带 `data-mq-ref`;实现 payload 按需展开,说明作为正文。动态调用/无法解析引用保持原文并解释原因,不虚构执行边。
|
||||
|
||||
首批 renderer 共享 `src/view/document.rs` 的文档表面,standalone/full page 和嵌入模式调用同一 read model;只改变容器和导航外观,不产生第二种语义解释。替换当前 `KnowledgePresentation` 与 `render_knowledge_html` 双路径重复组装。
|
||||
|
||||
稳定 ID 的保证范围要写明:
|
||||
|
||||
- 同一 revision 再次读取,node/ref id 必须相同。
|
||||
- 仅前插空行/叙述时,已有命名符号保持 symbol id;步骤 id 由结构身份映射,不能只靠行号。
|
||||
- 同一行多个节点需要 column/offset/span 区分,不能都映射成 `stmt-N`。
|
||||
- 匿名步骤重排/重复同名内容可能无法唯一映射;返回 unmapped/ambiguous,不能声称任意编辑下永恒稳定。
|
||||
- source span 明确 1-based line/column 和 UTF-8 byte offset;JavaScript UTF-16 偏移不直接当 Rust 字节偏移。
|
||||
|
||||
旧 `#stmt-{line}` 可保留为当前版本兼容锚点,但不得作为跨版本唯一身份。
|
||||
|
||||
### 6.4 嵌入渲染、主题与访问性
|
||||
|
||||
fragment 不包含 html/body、应用导航、运行按钮、内联可执行 script、全局 CSS;CSS 以 `.mq-document` namespace 限定。主题暴露 `--mq-bg/surface/text/muted/border/accent/font-body/font-code`,宿主映射自己的设计 tokens;不把独立 View 的全局 `*`/body 样式注入 qdagent。
|
||||
|
||||
首批推荐 fragment + 结构化引用 + 宿主事件代理;无需为 iframe 和 React 各重做 parser。可选 iframe embedded shell 必须只读、无重复导航,宿主/子页通信校验 origin/source/nonce/schema,仅允许 open-reference/selection/ready 等已定义消息,不能从内容发任意运行指令。
|
||||
|
||||
所有文本/属性转义,允许的 Markdown HTML 白名单净化;链接 scheme 限制;外部资源不默认跨网加载;代码/公式不得执行。静态导出、独立页面、嵌入三者都测试恶意文本/闭合标签/危险 URL。不要把 KnowledgePresentation 内 JSON 的转义当作所有正文 XSS 防护。
|
||||
|
||||
标题/列表/表格用语义 HTML;说明链接有可访问名称,焦点可见,视觉排序和键盘顺序一致。折叠只隐藏技术细节,不隐藏理解结论所需的正文。320–720px 容器宽、暗色、200% 缩放和 print 都要能读,不出现遮住正文的关系连线。无 JS 时仍能读正文和普通链接。
|
||||
|
||||
## 7. M04:知识、来源与历史引用定位
|
||||
|
||||
### 7.1 KnowledgeLocatorV1
|
||||
|
||||
统一 locator:`project_id,document_id,source_revision,generation_id?,artifact_id?,node_id?,source_ref_id?,span?,relation`。relation 区分 cites/supports/derived_from/executes。普通链接导航、NarrativeRef 和 BoundCall 都能定位,但语义不同;点击说明永不成为 BoundCall 的执行。
|
||||
|
||||
新增 `view.reference_resolve`:接受 locator 或当前文档 ref_id,返回 `resolution = resolved|stale|unresolved|ambiguous|version_unavailable|unsupported`、目标 locator、候选(若有)、定位精度、诊断、读者标签。框架内部路径不作为浏览器 href,应用根据授权映射为 document/source route。
|
||||
|
||||
扩展 `artifact.inspect` 的版本参数,旧无版本调用仍查当前;新调用带 expected source/generation mismatch 时不能自动降到当前。来源读取接口可由宿主取实际文件,再调用框架 locator 映射;框架不自动 fetch 外部网页。`mq://` 必须按框架规则解析,不能交给浏览器或把 URI 直接当文件路径。
|
||||
|
||||
### 7.2 历史版本与保留责任
|
||||
|
||||
生成 artifact/generation 元数据不等于保存了原文历史。框架须输出可保存的 snapshot manifest:原文 revision、依赖 revision、source refs、artifact hashes、renderer/schema version;应用决定哪些源版本随引用保留及保留周期。已有 Git 或内容寻址存储可提供 snapshot resolver;不强制应用部署 Git。
|
||||
|
||||
source revision 定位到存储快照后,框架基于该快照解析;缺失任何必要版本时返回 version_unavailable/partial,不读取最新文件填充旧快照。如果只留 artifact 元数据、原始资料已删除,仍可展示历史元数据但明确“原文不可读取”。
|
||||
|
||||
知识改名、移动或 source anchor 改变可返回候选映射,只有唯一可靠匹配才自动导航;否则用户选择最新版/候选。保持旧引用与新引用两个对象,防止历史回答证据在后台悄悄改变。
|
||||
|
||||
### 7.3 验证和运行事实
|
||||
|
||||
节点详情至少携带 purpose/description、signature、applicability、provenance、effects、dependencies,以及 parse/resolve/contract/tests/evidence/execution 各自状态与记录来源。sourced 只能说“已关联来源”;static verify 成功不表示执行成功或适用业务正确。
|
||||
|
||||
真实执行证据独立结构 `run_id,artifact_digest,closure_digest,source_revision,generation_id,inputs_digest,executed_at,result_state,evidence_ref`,展示时匹配版本。旧运行结果不涂成当前文档的绿色通过。知识复用必须存在实际 bind/execute trace,不能凭 find 命中显示“已执行”。
|
||||
|
||||
View 不运行验证测试以填满状态。若用户点“验证”,应用走已有 verify/policy 门禁并记录新 evidence,之后读取才刷新详情。
|
||||
|
||||
### 7.4 错误与授权协作
|
||||
|
||||
| 内部结果 | 应用建议呈现 | 不能做 |
|
||||
|---|---|---|
|
||||
| unresolved | 无法定位该知识,可读原引用 | 跳转到第一个同名 artifact |
|
||||
| ambiguous | 显示已授权候选和位置 | 自动选择或暴露不可见候选 |
|
||||
| stale | 该知识/来源已变化,提供版本选择 | 静默替换旧引用 |
|
||||
| version_unavailable | 回答所用版本不可用,可查看最新版 | 用最新版伪装历史证据 |
|
||||
| partial | 部分依赖或来源缺失,说明范围 | 默认绿色通过 |
|
||||
| permission denied(应用) | 无权访问 | 在错误中泄漏标题、路径、片段 |
|
||||
|
||||
用户权限归应用;框架服务在调用边界接受 scoped project/source resolver,不把传来一个 root 视为可遍历整个服务器。所有路径经过 canonicalize/symlink 检查;缓存的语义事实可共享,带原文的响应必须按权限复核,不能只靠 artifact id 缓存授权。
|
||||
|
||||
## 8. M05:工作区、笔记与导入所需的通用底座
|
||||
|
||||
不为 qdagent 画像加语言特殊节点。通用能力应支持普通文档元信息读取、来源范围、内容修订、diff/patch 的 expected digest、提取格式 capability。
|
||||
|
||||
document.extract 继续使用 DocumentSource→ExtractedDocument,报告 format/status/segments/source locator/diagnostics。TXT/MD 可支持;PDF/DOCX/OCR 没有适配器时明确 unsupported/failed,不能将空文本当成功。partial 保留已提取片段及原文范围、遗漏原因;发布政策由应用决定。
|
||||
|
||||
同一资料可产生多个知识单元,provenance 必须到具体片段,不能所有函数只指向文件根。原始资料 digest 改动的失效由现有 provenance/closure 保证;应用用户画像也可用普通来源引用,但不自动进入公共知识发布。
|
||||
|
||||
ProjectRoot/CompileOptions 显式传递,工作区 A/B 的 source id、输出 generation、缓存、临时目录必须隔离。避免每读一个节点就全项目编译:引入按 project snapshot 复用的分析服务,针对来源/依赖变化失效;并发同 snapshot 请求合并,不能分别写同一个 CURRENT。
|
||||
|
||||
上传 multipart/大小限制已有 web 底座,框架只提供安全提取和可消费结果;浏览器上传、作业生命周期、发布确认、权限和历史保存仍在 qdagent。多文件任务桥接采用结构化参数,禁止拼接用户文件名进 shell/JSON 或共用固定请求临时文件。
|
||||
|
||||
## 9. 实现落点与兼容策略
|
||||
|
||||
| 模块 | 改动方向 | 兼容要求 |
|
||||
|---|---|---|
|
||||
| `plugins/llm/src/lib.rs` | adapter boundary、消息数组、usage、event sink、取消 | 保留已有符号;新能力有 probe 和合同版本 |
|
||||
| `ext/ai/llm.mq.md` 与 `ext/ai/llm/*` | strict selection、结构化 result、public snapshot | 无 selection 的旧调用仍可运行;不得回显密钥 |
|
||||
| `ext/ai/agent.mq.md` | run context、子工作簿传播、逐调用观测 | 不复制 Agent loop,EFI/verify/publish_gate 不被绕过 |
|
||||
| `plugins/agent/src/*` | 子任务/恢复上下文、cancel、event identity | resume 不重发已结算请求;不可恢复说明 interrupted |
|
||||
| `src/host/*`、插件 SDK、web invoke | 请求局部 event/cancel 桥接 | ABI 旧宿主不伪支持;并发请求不混流 |
|
||||
| `src/view/knowledge_present.rs` | DocumentView read model/兼容 adapter | 旧 presentation v1 不被新字段语义污染 |
|
||||
| `src/view/document.rs,render.rs,html.rs,mod.rs,output.rs` | 共享纯读取、fragment、scope CSS、定位 | live/debug 与 read 明确分离;旧 no-exec 门禁保留 |
|
||||
| `src/mlsp.rs` | capabilities、view/document/reference、版本 inspect | 使用现有 envelope;未知 feature 可探测 |
|
||||
| `src/sema/ir.rs,provenance.rs` | span、snapshot/locator、依赖与来源失效 | 不改变核心语法,不把读请求变为执行 |
|
||||
| `src/document_extract.rs` | capability 和片段定位完整性 | 保留不支持格式的显式失败 |
|
||||
|
||||
测试分层对应 plugin HTTP fixtures、Agent 假模型、MLSP read contracts、web 持续流集成、独立/嵌入 DOM 和浏览器交互。应用共享样本文件必须注明 schema 版本;两仓 PR 在各自文档索引相互链接对应实现说明。
|
||||
|
||||
迁移顺序为新增合同/接口 → 旧输出 adapter → qdagent 按 capability 接入 → 切默认 → 后续版本再考虑弃用。不要同时移除 legacy 行为且要求应用一次完成全链升级。发布包对 CLI/ext/native 插件做组合测试,缺插件给功能不可用诊断而非 unknown function 直到用户点击才暴露。
|
||||
|
||||
## 10. 交付切片与验收门槛
|
||||
|
||||
| 切片 | 前置 | 验收输出 | 支撑应用任务 |
|
||||
|---|---|---|---|
|
||||
| M00 合同冻结/探测 | 无 | schemas、fixtures、capabilities、版本和插件清单 | Q00、所有升级探测 |
|
||||
| M01 多模型隔离 | M00 | 兼容适配器、strict handles、history、子任务选择一致 | Q01/Q02 |
|
||||
| M02 调用观测与流 | M01 | 每 attempt usage、真流式、取消、错误/重放一致 | Q03/Q06 |
|
||||
| M03 只读嵌入 View | M00,可与模型线独立 | DocumentViewV1、fragment、目录/节点、无执行证明 | Q04 |
|
||||
| M04 引用与快照 | M03 | 历史 locator、来源详情、版本失效/歧义 | Q05 |
|
||||
| M05 文档工作区底座 | M03/M04 + 已有 extract | 范围隔离、提取定位、并发与缓存 | Q07/Q08 |
|
||||
| M06 集成与性能 | 全部所选切片 | 旧门禁 + 新 contracts + 浏览器与负载报告 | Q09 |
|
||||
|
||||
M01 支持多个兼容提供商不等待原生协议全做完;但原生适配器需另列完成状态。M03 不等待计费系统。M04 未完成历史快照前,应用可以显示当前文档,但必须把历史引用标“版本不可用”,不能宣布引用链闭合。
|
||||
|
||||
每个切片的 Definition of Done:实际接口可调用、失败合同覆盖、无副作用/隔离测试、文档示例由测试消费、版本能力可探测、qdagent integration fixture 通过。只有可运行的验收证据才可将 proposed 改为 verified。
|
||||
|
||||
## 11. 框架测试矩阵
|
||||
|
||||
| 编号 | 用例 | 断言 |
|
||||
|---|---|---|
|
||||
| F01 | 两端点各两模型、并行 A/B、无密钥本地端点 | 请求 payload/端点/鉴权正确,环境不串扰 |
|
||||
| F02 | step/plan/router/repair/child/resume | strict selection 和 parent identity 完整,不退回默认模型 |
|
||||
| F03 | multi-turn、system/tool roles、超限/不支持模态 | 顺序和边界完整,不吞最新 user,不静默降级 |
|
||||
| F04 | 非流/流式/usage-only/缺 usage/错误后部分 usage | null 与 0 区分,终态和用量独立 |
|
||||
| F05 | 相同累计 usage 多次、重放、新重试 | 同 attempt revision 覆盖,不重复相加;真重试新增 attempt |
|
||||
| F06 | 慢供应商、慢消费者、断连、取消、超时 | 完成前有 delta;队列有界;取消不启动后续工具/模型 |
|
||||
| F07 | 外部工具已写后取消、恢复未知状态 | 不声称撤销,不自动重做可能有副作用的动作 |
|
||||
| F08 | 带写文件/网络/foreign/插件的文档被阅读 | read API 不加载/运行这些效果,无项目写回 |
|
||||
| F09 | prose/heading/6 类步骤、同一行多引用、中文 span | HTML/outline/节点/ref 同源,定位唯一准确 |
|
||||
| F10 | 仅说明变化、依赖变化、旧 SSE 包、无 token | token 正确失效,嵌入 strict 拒绝过期/无身份包 |
|
||||
| F11 | 来源改删、历史 generation 未留原文、同名歧义 | 明确 stale/unavailable/ambiguous,无猜测替代 |
|
||||
| F12 | no-js/390px/暗色/长表格/键盘 | 正文可读,可返回,有清晰标题和焦点 |
|
||||
| F13 | 原始 HTML/危险 URL/闭合 script/越界路径 | 无脚本执行/越权读取/泄漏;不靠应用猜净化 |
|
||||
| F14 | 两项目同相对路径/并发 100 读取 | scope 缓存隔离,同 snapshot 合并分析 |
|
||||
| F15 | 编译成功但验证/证据未知、旧执行记录 | 各状态独立;不把 static/sourced 标 verified |
|
||||
| F16 | CLI/ext/native 版本错配 | capability 早发现不可用,而非运行时静默失败 |
|
||||
|
||||
性能目标需实测而非发版文字:固定机器与 release/debug 分开记录,1000 单元分析冷/热、单文档 fragment 体积/时间、并发读取 P95/内存、UI 首屏。应用目标见 Q 文档;如果全量 compile 超时,优先分析重复全库扫描、依赖计算与缓存,不通过删除 fixture 或抬高阈值宣布可用。
|
||||
|
||||
## 12. 本次验证结果与已知阻碍
|
||||
|
||||
### 12.1 M00–M06 落地证据(2026-10-25 续)
|
||||
|
||||
命令(本机):
|
||||
|
||||
```bash
|
||||
cargo test --offline --locked --test workbench_contracts
|
||||
cargo test --offline --locked -p marqdo_plugin_llm
|
||||
cargo test --offline --locked --test executable_knowledge f1_agent_eng_bind_gate_mq_offline -- --exact
|
||||
```
|
||||
|
||||
已交付并通过自动化子集(**非** F01–F16 全绿):
|
||||
|
||||
| 切片 | 证据 |
|
||||
|---|---|
|
||||
| M00 | fixtures + JSON Schema 消费;`capabilities`;`force_unloaded` / `force_*_off`(F16) |
|
||||
| M01 | strict selection;`RunContextV1` 并行不串扰;llm plugin messages/roles;workbook 禁 `llm.llm`;**F02** `m01_f02_agent_step_selection_and_resume`(mock HTTP + `llm.create` 抗 env poison + resume save/load) |
|
||||
| M02 | usage null≠0;revision 替换(F05);cancel token(F06/F07);event_sink;web `sse-live` 断连 cancel |
|
||||
| M03 | `view.document`;CJK outline/nodes;path escape;静态 a11y 子集;F08 无副作用标记;**浏览器** `workbench-docview.html`(`article.mq-document`、0 script、无 `javascript:`) |
|
||||
| M04 | stale / version_unavailable / **ambiguous**;`SnapshotManifestV1` 输出(retention=application);`artifact.version_mismatch`;embed strict SSE |
|
||||
| M05 | extract `:B`;analysis_cache begin/finish;**F14 并发 100** 同 snapshot + 跨 project 隔离 |
|
||||
| M06 | CI 预构建 agent+llm + `workbench_contracts`;Go `drainForward` / cancel-token 桥 |
|
||||
| Live | `tests/workbench_live.rs`:USTC OpenAI-compatible(`OPENAI_*` env only)`qwen3.8-chat` / `deepseek-flash` selection ask → `pong` + usage;`MARQDO_EXT` 指向仓内 `ext/` 避免 `~/.marqdo/ext` 陈旧遮蔽 |
|
||||
|
||||
已知限制(发版前须书面接受或继续补):
|
||||
|
||||
- F02 repair/plan **全路径**假模型仍薄于 step+resume;RunContext 三级继承 + skeleton 禁 `llm.llm` + 子任务 env 已有。
|
||||
- 历史正文由应用按 SnapshotManifest 保留;框架缺快照 → `version_unavailable`(不静默降级)。
|
||||
- 完整宿主嵌入 F12 / 产品鉴权留给 qdagent;原生适配器 / PDF/OCR 仍 unavailable。
|
||||
|
||||
### 12.2 历史基线记录(设计起草时)
|
||||
|
||||
起草时 `executable_knowledge` 曾出现 `f1` 缺插件与 `v3_corpus_1000` debug 超时;其后 `ensure_agent_plugin_built` + CI 预构建已修复 f1 路径。1000 单元性能仍按剖面单独记录。
|
||||
@@ -0,0 +1,463 @@
|
||||
# View 可执行知识阅读体验重设计
|
||||
|
||||
| 项目 | 说明 |
|
||||
|---|---|
|
||||
| 日期 | 2026-10-24 |
|
||||
| 状态 | 详细设计建议,待实施;不是已交付能力声明 |
|
||||
| 审计基线 | `b8a4ddd` / Marqdo 2.1.1;使用当前源码构建的 debug CLI 复核 |
|
||||
| 输入 | 用户提供的 view 截图:Executable Knowledge 在 Structure 前,Steps、Target、Kind、Signature、Effects、Verification、Lock digest 等字段大量铺开 |
|
||||
| 范围 | Live view、静态导出、正文结构、知识说明、运行结果、来源与状态呈现;不改语言语义,不增加作者必填的展示 DSL |
|
||||
| 关联 | [文档原生开发指导](document-native-development-guide.md)、[View 总体设计](view.md)、[文档面与标注](view-document-annotations.md)、[求道相关框架缺口](../roadmap/qdagent-knowledge-support-gaps-2026-10-24.md) |
|
||||
|
||||
## 1. 设计决策
|
||||
|
||||
**View 首先呈现知识文档本身,结构帮助读者理解和定位,机器信息解释当前所读内容。** 正文、程序结构、知识关系与执行证据应由同一份源文档关联起来,而不是顺序展示几份互相脱节的报告。
|
||||
|
||||
确定以下方向:
|
||||
|
||||
1. 移除正文之前整块展开的 Executable Knowledge。主阅读流首先出现作者的标题、介绍和内容;不把完整机器报告简单搬到正文后继续平铺。
|
||||
2. 保留并改进现有 Structure 的叙述保真文档面,面向读者命名为“文档”。正文、表格、公式、函数和可执行步骤在原位置阅读,不生成一份模型改写的平行说明书。
|
||||
3. 右侧“本文结构”负责章节与知识单元定位;选中知识片段后,在同一辅助栏展示它的解释、输入输出、来源和验证依据。源码标识、摘要与完整协议放在可展开的技术详情中。
|
||||
4. 运行结果就地关联到对应步骤,明确运行实例与输入;完整日志、运行控制和源码放在正文后。看见一个数值时,读者必须能知道它是文档声明、计算结果还是历史快照。
|
||||
5. 静态分析、程序运行、测试断言、来源关联和领域确认分别表达。不能用一个 `ok`、`pure` 或绿色“已验证”覆盖不同事实。
|
||||
6. 功能渐进增强:无 JavaScript、静态导出、移动端、打印时仍能读完文档、定位知识和查看必要依据。普通短文不变成复杂仪表盘。
|
||||
|
||||
这些决策恢复 [view.md](view.md) 原有“文档面为主、Browse 与 Debug 分工”的方向,同时修订过度依赖引出线和默认展开技术面板的呈现方式。
|
||||
|
||||
## 2. 现状审计:为什么截图难读
|
||||
|
||||
### 2.1 可定位的实现依据
|
||||
|
||||
| 观察 | 当前代码 | 对阅读的影响 |
|
||||
|---|---|---|
|
||||
| 知识面板位于正文前 | [html.rs / page_file](../../src/view/html.rs) 的模板顺序为 notebook → knowledge → Structure → Execution → Source | 必须先越过辅助资料才能看到文档;模块越多越严重 |
|
||||
| 所有制品一律展开 | [render.rs / render_knowledge_html](../../src/view/render.rs) 按文件筛选 artifacts,并逐项输出完整 `<dl>` | module/type/function 同等突出,容器元信息挤占知识内容 |
|
||||
| Steps 脱离所属结构 | 同函数把当前文件所有 steps 放在单一 `<ol>` 中,展示 kind、label、完整目标 ID | 静态步骤容易被误读为实际执行顺序;调用/循环/正文引用混在一起 |
|
||||
| 内部字段未经阅读转译 | URI、签名、effects、verification 拼接文本、完整 sha256 默认可见 | 读者仍不知道这项知识解决什么问题、怎样用、依据在哪里 |
|
||||
| “Lock digest”名实不符 | 该字段实际显示 `artifact.hashes.artifact`;[mod.rs](../../src/view/mod.rs) 编译使用 `LockMode::Off` | 制品摘要被误读为已经锁定且通过复现检查 |
|
||||
| 静态说明与运行脱节 | 每张卡固定输出 `Not executed by knowledge preview.`;实际 run capture 在另一条路径 | 即使页面有运行结果,也很难理解“未执行”到底指什么 |
|
||||
| 分析失败静默消失 | `build_file_view` 对语义编译结果使用 `unwrap_or_default` | 无知识、分析失败、尚未分析在界面上都可能表现为空 |
|
||||
| 标题与层级表达较弱 | `page_file` 标题取文件名;[document.rs / render_fun_heading](../../src/view/document.rs)统一用 h3 | 作者 frontmatter 标题未用于页标题,嵌套结构主要靠其它线索判断 |
|
||||
| 知识组件缺少专用布局 | `knowledge-*` 在 HTML 中有类名,但 `html.rs` 没有与面板匹配的专用样式规则 | 大量使用默认列表/定义列表排版,空白多、关系弱,和正文主题缺少一致约束 |
|
||||
| 新旧皮肤叠加 | `html.rs` 内基础 view CSS 与 `notebook_ui` 大块 notebook 主题共存 | 正文、注释、知识区的字号和密度不一致;不能只修一个卡片颜色 |
|
||||
| 验收主要看元素存在 | [executable_knowledge.rs](../../tests/executable_knowledge.rs) 的 view 用例检查 step 类名、kind 和 `#stmt-` 链接 | 没检查正文优先、跳转目标存在、可读文案和实际布局 |
|
||||
|
||||
已有能力应保留:[document.rs](../../src/view/document.rs) 的叙述/表格保真、Occlusion 描述与实现分离、函数大纲、调用帧/循环帧选择、[annot.rs](../../src/view/annot.rs) 的真实执行标注。重新设计应基于这些能力,而不是退回 AST 卡片浏览器。
|
||||
|
||||
### 2.2 对截图的解释边界
|
||||
|
||||
截图可直接确认:机器字段先于正文、Steps 只有 `call print`、长 URI/摘要显著占位、英文状态串缺少解释,以及深色底上的孤立黑色短线。
|
||||
|
||||
黑色短线的具体来源尚未在浏览器中复现。代码中存在 annotation leader、布局重算和主题覆盖,可能需要检查,但本文件不把它们断言为唯一根因。实施时应使用用户截图对应页面复现,检查实际 DOM、computed style 和几何位置;不能用全局隐藏所有 SVG/边框来掩盖问题。
|
||||
|
||||
### 2.3 本轮实际复核
|
||||
|
||||
把 [public/features/01-phase-i-treewalk.mq.md](../../public/features/01-phase-i-treewalk.mq.md) 复制到临时目录,使用 debug 2.1.1 执行静态导出并检查生成 HTML:knowledge 确实在 structure 前,共 3 个 `knowledge-card`(含 Steps)和 2 个完整 sha256。该短文的核心只是介绍 tree-walk 流程并打印一句话,当前辅助内容明显过重。
|
||||
|
||||
另用隔离临时文件复核 `view output --no-exec`:嵌入的 `fs.write_text` 仍创建了临时标记文件。原因是 [output.rs / write_static](../../src/view/output.rs) 先调用会执行的 `build_file_view`,再在 `opts.no_exec` 分支清空结果。**这是已复现的执行/呈现不一致,必须修复,不能仅把标签改成“未运行”。** §9 给出新的构建边界。未执行线上浏览器像素比对,本文线框为目标设计。
|
||||
|
||||
## 3. 读者任务与设计成效
|
||||
|
||||
| 读者任务 | 默认应看到 | 按需展开 | 成功判据 |
|
||||
|---|---|---|---|
|
||||
| 理解这篇文档 | 作者标题、介绍、正文和核心表格/步骤 | 完整源码 | 不滚过机器报告就能开始阅读 |
|
||||
| 看懂文档结构 | 章节、类/规则/函数的真实层级,当前位置 | 参数和局部步骤树 | 能判断一个步骤属于哪个知识单元 |
|
||||
| 理解某项知识 | 描述、简洁公式/调用、条件、返回含义 | payload、契约、精确目标 | 能解释它做什么、需要什么、产生什么 |
|
||||
| 判断结果是否可信 | 来源状态和所选运行的明确结果 | 测试、来源片段、版本和诊断 | 能区分“可以运行”和“事实成立” |
|
||||
| 跟踪跨文档复用 | 友好目标名、所属文档、输入输出 | 依赖关系、完整标识 | 能到达定义并返回原阅读位置 |
|
||||
| 检查某次运行 | 本次输入、选中调用/迭代、对应结果 | 日志、历史帧 | 不把不同运行的值混在一起 |
|
||||
|
||||
不要求所有读者理解 SymbolId、IR、hash 或 effect 协议。开发者仍能找到这些信息,但它们不再成为阅读的前置条件。
|
||||
|
||||
## 4. 页面信息架构
|
||||
|
||||
### 4.1 默认桌面布局
|
||||
|
||||
```text
|
||||
┌ Marqdo 项目 / 文档路径 搜索 主题 ┐
|
||||
├───────────┬───────────────────────────────┬────────────┤
|
||||
│ 文件目录 │ Phase I — tree-walk interpreter│ 本文结构 │
|
||||
│ │ Lex · parse · check … │ 介绍 │
|
||||
│ 当前文件 │ [运行记录:尚未运行] [详情] │ main 入口 │
|
||||
│ │ │ │
|
||||
│ │ Milestone: Phase I … │ │
|
||||
│ │ Pipeline: load → … → eval │ 选中后: │
|
||||
│ │ │ 当前知识 │
|
||||
│ │ main · 入口 │ 作用 │
|
||||
│ │ 输出说明 print text=… [说明] │ 输入/返回 │
|
||||
│ │ │ 来源/条件 │
|
||||
│ │ ▸ 运行与结果 │ 检查/运行 │
|
||||
│ │ ▸ 完整源码 │ 技术详情 ▸ │
|
||||
└───────────┴───────────────────────────────┴────────────┘
|
||||
```
|
||||
|
||||
布局是文件导航、主文档、一个辅助栏,不能增加第四条永久侧栏。右栏上部保留可收起的“本文结构”;用户选中片段后,下部显示“当前知识”。空间不足时以当前知识替换下部内容,并提供“返回目录”操作。打开/关闭详情不改变正文宽度,避免重排导致阅读位置跳动。
|
||||
|
||||
主文档的默认顺序:
|
||||
|
||||
1. 文档标题、已有 description、简短状态入口;title 缺失时退回文件名。
|
||||
2. 源文档的正文、结构和轻量就地知识说明,保持作者顺序。
|
||||
3. “运行与结果”:运行按钮、输入、所选运行结果、输出图形/日志;已有重要失败有就地入口。
|
||||
4. “完整源码”及“文档技术信息”,默认折叠。
|
||||
|
||||
不在顶部强制生成内容摘要、全量函数清单、巨大状态矩阵或知识关系图。没有 description 时不调用 LLM 补写。结构大纲始终可达,正文不因折叠辅助栏而变成无结构的文章。
|
||||
|
||||
### 4.2 尺寸与响应式规则(设计目标)
|
||||
|
||||
| 可用宽度 | 布局 |
|
||||
|---|---|
|
||||
| ≥1280 CSS px | 文件栏约 220–260px;正文弹性;辅助栏约 280–320px;正文中文行长约 32–44 字 |
|
||||
| 900–1279px | 文件栏可收起;正文优先,辅助栏可切换为有标题的侧抽屉 |
|
||||
| <900px | 单列文档;“目录”“当前知识”作为明确按钮打开抽屉或块内 details |
|
||||
| 320–390px 手机 | 标题、正文、状态不横向溢出;长表格/代码局部滚动,整页不左右拖动 |
|
||||
| 打印 | 正文展开,必要来源/验证摘要转为节后附注;交互按钮和导航隐藏;技术原始字段按导出选项包含 |
|
||||
|
||||
在 1440×900、100% 缩放的短文验收样例中,首屏应包含至少一段正文和首个主要结构单元;状态头不得因每个 artifact 增长而把正文推出首屏。此约束不截断作者自己的长标题或介绍。
|
||||
|
||||
### 4.3 普通文档与复杂知识文档采用同一布局
|
||||
|
||||
- 普通说明页只有 main/print:显示原文与输出操作,技术区保留模块信息;不强制展示“1 项可执行知识”卡。
|
||||
- 可复用函数库:目录列出函数;函数标题下出现紧凑签名和“说明”入口;即使无 main,也能正常阅读。
|
||||
- 事实/规则文档:来源、条件与验证紧邻相应知识;来源问题不能藏进页尾。
|
||||
- 教程/Notebook:保留源文档的代码单元与结果;全局单元运行条收进运行区域或显式工作模式,不能把所有文章永久变为 notebook 工具台。
|
||||
- 大文档:目录按真实层级和顺序组织,可搜索;正文不因知识数量膨胀为另一份报告。
|
||||
|
||||
## 5. 文档与结构呈现规范
|
||||
|
||||
### 5.1 标题、类与函数层级
|
||||
|
||||
页标题优先来自解析后的 frontmatter `title`,其次使用合适的源文档标题,再回退文件名;不要自行用正则重解析 YAML。保留原文标题内容,不擅自把技术名称改成模型生成的名称。
|
||||
|
||||
“本文结构”同时服务阅读和程序定位,但明确区分:
|
||||
|
||||
- 作者章节按实际文档关系展示。
|
||||
- 可执行的类/类型、函数、入口附轻量类型词,例如“规则”“函数”“入口”;类型词来自编译结果或已声明元数据,不由标题猜业务类型。
|
||||
- 同一节点既是章节又是函数时只有一个目录项,旁边标类型,避免两棵树重复。
|
||||
- 嵌套函数、分支、循环在对应父节点下展开;语义归属与 Markdown 可视层级分别保留,不把所有内容都扁平化为 h3。
|
||||
- 普通段落引用不提升为函数;module 容器不自动成为一张与业务函数同权的卡。
|
||||
|
||||
DOM 标题使用连续、可访问的层级;实际语法的 `#`/`##` 层数在源码面保留。用户复制源码时取得原始合法语法,而非为了 HTML 标题级别改写过的内容。
|
||||
|
||||
### 5.2 可执行步骤:就地解释并保留上下文
|
||||
|
||||
| 编译器步骤 | 默认阅读文案 | 展开后 |
|
||||
|---|---|---|
|
||||
| observe | “读取:商品单价” | 集合、取元表达式、字段说明、当前调用下的值 |
|
||||
| derive | “计算:商品小计” | 公式、输入来源、计算条件、当前结果 |
|
||||
| call | 优先作者标签;无标签时“调用 商品小计” | 目标、所属文档、实参映射、返回和运行记录 |
|
||||
| iterate | “逐项处理:订单商品” | 集合、循环体、迭代数、所选迭代 |
|
||||
| narrate | 正文保留作者描述,附“知识引用”入口 | 定义与来源;不当作已执行步骤 |
|
||||
| navigation | 标准文档链接 | 正常导航,不显示虚构的执行结果 |
|
||||
| 空 payload / stub | 作者描述 + “待实现” | 缺失的实现和已有诊断,不能展示成功数值 |
|
||||
|
||||
名称来自当前节点的 label、声明说明、目标可读名,依次回退;没有业务描述时就显示“调用 print”,不得自动声称它是“验证知识”或“输出最终结论”。普通 print 在所属文档位置解释输出作用,放在“输出”类信息中,不作为高价值知识卡突出。
|
||||
|
||||
“步骤”列表只在当前知识详情中显示,标为“文档中的步骤”或“结构步骤”。静态序号不能暗示实际执行时序;实际轨迹另按运行事件呈现,分支和循环保留嵌套。
|
||||
|
||||
### 5.3 描述、实现与结果的三层关系
|
||||
|
||||
以 `**总价 = [计算订单小计](pricing.小计(数量, 单价))**` 为说明示意:
|
||||
|
||||
```text
|
||||
计算订单小计 [说明]
|
||||
总价 = pricing.小计(数量, 单价) ← 简短实现,次一级视觉
|
||||
本次示例:数量 2,单价 30 → 60 ← 仅有匹配运行记录时出现
|
||||
```
|
||||
|
||||
作者的上下文句、返回说明、单位和条件仍在原位。短公式/实现可次级显示,长 payload 默认折叠到详情;显式“展开实现”控制可对当前知识单元生效。不得把所有代码完全藏掉,使“代码即文档”退化成只有自然语言标签。
|
||||
|
||||
复制操作区分“复制说明”和“复制源码”;源码来自源 span,保留必要定界符、参数和 import 依赖提示。显示为漂亮文本的结果不能被复制成伪造的可执行代码。
|
||||
|
||||
### 5.4 表格、条件与关系
|
||||
|
||||
数据表按作者结构呈现,表头固定语义,单位与列说明常显;契约表按输入/返回/约束组织。显示层不能把 Marqdo 列向集合悄悄换成行向记录;需要另一种预览时明确标为预览,并能查看真实形状。
|
||||
|
||||
前提条件与失败分支不得因“简洁”隐藏。默认显示全部分支结构,所选运行只突出实际命中的分支,继续遵守 [调用帧与结构保真](view-frame-selection.md)。
|
||||
|
||||
跨文档关系默认用可读链接,例如“使用:标准税率 · tax.mq.md”,而非 `mq://…`。直接依赖与被引用者各列前几项并可展开;可选局部关系图放在详情层,不作为默认阅读画布。关系图必须有等价文字列表,并显示未知/动态关系,不能补画不存在的边。
|
||||
|
||||
## 6. 当前知识详情:回答问题,而非列出字段
|
||||
|
||||
### 6.1 内容顺序
|
||||
|
||||
```text
|
||||
商品小计 函数
|
||||
根据数量与单价计算商品金额。 ← 仅引用已有描述
|
||||
|
||||
输入 数量:整数;单价:金额 ← 类型/单位未知则明确未知
|
||||
返回 金额
|
||||
条件 数量非负 ← 仅当已声明且可定位
|
||||
实现 数量 × 单价
|
||||
|
||||
来源 定价说明 / 第 2 节 [打开]
|
||||
检查 契约未检查;测试未运行 [查看依据]
|
||||
运行记录 示例 #2:2 × 30 → 60 [定位输出]
|
||||
使用 … [查看定义]
|
||||
|
||||
▸ 技术详情:精确标识、完整签名、效应、摘要、诊断 JSON
|
||||
```
|
||||
|
||||
此处“金额”“数量非负”等只用于解释布局,不宣称当前样例具有这些元数据。没有信息就省略非关键行或写“未声明”,不得推断成“无限制”“没有来源也可靠”。缺失影响可用性时常显;纯工具没有业务来源并不自动是错误。
|
||||
|
||||
### 6.2 当前字段的去向
|
||||
|
||||
| 当前字段 | 新的主要位置与文案 |
|
||||
|---|---|
|
||||
| Target / `mq://…` | 标题处可读定义链接;完整 ID 在技术详情,可复制 |
|
||||
| Kind / callable | 标题旁简短类型词;module 信息进入文档技术信息 |
|
||||
| Signature | 输入/返回紧凑表;完整规范签名可展开 |
|
||||
| Effects | “涉及的操作”,说明文件读写、网络等;未知明确显示,非纯不等于失败 |
|
||||
| Verification 长串 | 分项检查和证据表,见 §7 |
|
||||
| Lock digest | 更名“制品摘要”,技术详情内显示;锁状态单独来自真实 lock report |
|
||||
| Depends on | 可读目标列表,指向定义与引用位置;未分析不能写“无依赖” |
|
||||
| Execution 固定句 | 匹配运行记录,或“尚无该知识的运行记录” |
|
||||
| SourceRef / applicability | 当前遗漏的关键内容:放入“来源”和“适用条件” |
|
||||
|
||||
默认不铺完整 hash。技术详情可显示缩略摘要并提供全值/复制;缩略值不用于识别唯一性。没有读取锁报告时写“未检查锁定状态”,不能把拥有摘要显示成“已锁定”。
|
||||
|
||||
### 6.3 交互、跳转与返回
|
||||
|
||||
1. 点击正文的“说明”或目录知识项,选中同一 `node_id`;正文轻量高亮,右栏同步。键盘和触屏具有等价入口,不依赖 hover。
|
||||
2. 跳转到定义时保留来源文件、锚点、滚动位置和所选运行;返回恢复阅读位置。
|
||||
3. 文件、符号与语句不能只靠显示名称匹配。同名函数的详情显示所在文档和父结构。
|
||||
4. 默认不自动抢焦点、不因滚动不停替换用户正在阅读的详情。滚动只更新目录当前位置;点击选择才改变详情。
|
||||
5. 手机抽屉有标题、关闭、返回焦点和滚动管理;无 JS 时使用正文附近的原生 details 或页尾知识附注锚点。
|
||||
6. 当前 URI 不在导出范围内时显示“定义未包含在此次导出”,提供可读路径和复制引用;禁止输出必然失效的 `mq://` 或 live `/file` 链接。
|
||||
|
||||
## 7. 状态设计:读者必须知道什么被证明了
|
||||
|
||||
### 7.1 分离三个状态来源
|
||||
|
||||
| 状态来源 | 可以说明 | 不能说明 |
|
||||
|---|---|---|
|
||||
| 编译/静态分析 | 结构解析、目标绑定、契约检查结果、已建模效应 | 程序已运行、答案正确、资料真实 |
|
||||
| 运行记录 | 指定输入、环境、源码版本下的执行与输出 | 覆盖所有输入、测试通过、来源可信 |
|
||||
| 证据/验证记录 | 哪个断言/测试/来源核验通过,何时失效 | 未执行或未核验的其它部分也可靠 |
|
||||
|
||||
顶部仅给紧凑事实,例如“静态分析完成 · 尚无运行记录”,或“有 1 项来源问题 · 查看”。普通 unknown/unchecked 不应制造全页红色警报;与当前结论直接相关的失败、过期、未解析必须就地提示且可定位。
|
||||
|
||||
### 7.2 分项文案映射
|
||||
|
||||
| 机器状态 | 阅读文案 |
|
||||
|---|---|
|
||||
| parse=passed | 文档结构可解析 |
|
||||
| resolve=passed | 此项检查报告目标已解析;覆盖范围见详情 |
|
||||
| contract=unchecked | 契约尚未检查 |
|
||||
| execution=not_run | 尚无执行记录 |
|
||||
| tests=unknown | 尚无测试记录 |
|
||||
| evidence=sourced | 已关联来源,内容尚未核验 |
|
||||
| evidence=unverified | 来源内容尚未核验;结合来源列表判断是否已关联 |
|
||||
| missing_source / expired | 来源缺失 / 已过适用期 |
|
||||
| domain=unreviewed | 尚无领域审阅记录 |
|
||||
| effects=pure | 静态分析未发现已建模的外部操作;展开可看范围 |
|
||||
| effects=unknown | 涉及的操作尚不能完全确定 |
|
||||
| 无对应字段 | 未提供该项检查结果 |
|
||||
|
||||
分项取值不能自动升级成综合“可信”。只有宿主选定的政策实际给出聚合结论,才显示“满足 ×× 政策”,同时附政策名称、版本与证据;业务条件还须明确展示。
|
||||
|
||||
### 7.3 结果来源与失效
|
||||
|
||||
每个运行值绑定 `run_id + source_digest + node_id + call_id + loop/iteration` 中适用的字段,展示本次输入或简短上下文。无节点映射时只展示整次运行输出,不硬塞到某个函数旁边。
|
||||
|
||||
静态导出有快照时标“导出快照”,附生成时间/输入;读者切换调用帧是在看历史结果,不叫“重新计算”。源码、依赖或来源与记录不匹配时标“历史结果,版本已变化”,不继续用成功样式绑定到新正文。
|
||||
|
||||
正文里的字面量可以显示为“文档声明值”;解释器实际执行得到的值才叫“运行结果”。没有值、返回 None、stdout 为空、尚未运行、执行失败必须分开,不能统一成 `(no stdout)`。
|
||||
|
||||
### 7.4 空态与错误态
|
||||
|
||||
| 情况 | 处理 |
|
||||
|---|---|
|
||||
| 没有可识别知识单元 | 正常显示文档,不插空知识面板 |
|
||||
| 语义分析失败 | 保留可读正文/源码,显示“知识关系暂不可用”及可定位诊断 |
|
||||
| 局部未解析 | 仅对应片段显示问题,保留其它已知结果 |
|
||||
| 无运行或 no-exec | 运行区说明原因,使用中性状态,无绿色运行成功标记 |
|
||||
| 运行失败 | 错误摘要定位到源片段;日志在运行区,静态知识仍可读 |
|
||||
| 证据过期/冲突 | 相关知识常显问题、来源与影响;不静默选择一个结论 |
|
||||
| 分析被截断/超过扫描范围 | 显示范围和未覆盖项;“未发现”不能替代“没有” |
|
||||
|
||||
## 8. 视觉、密度与可访问性
|
||||
|
||||
### 8.1 排版与主题
|
||||
|
||||
正文建议 16–18px、行高约 1.65;辅助说明 13–14px、行高约 1.5;代码使用等宽字体,全文不因进入 notebook 主题缩为密集工具面板。主次依赖字号、间距和位置共同表达,不能仅靠颜色。
|
||||
|
||||
使用现有系统字体和主题机制,建立统一的 view tokens:正文/辅助文字、背景、细分隔、焦点、成功/警告/错误。知识组件只依赖 tokens,不硬编码黑色线条或白底。支持明暗主题,首次尊重系统/显式设置,不为这次改版再写第三套皮肤。
|
||||
|
||||
表格与长公式允许局部滚动;ID、路径在详情中可换行;主文无整页水平滚动。重要来源和适用条件不使用极浅灰隐藏。对比度验收目标:普通文字至少 4.5:1,大字和必要非文本边界至少 3:1;这些是实现验收目标,尚非本轮测量结果。
|
||||
|
||||
### 8.2 注释和连线
|
||||
|
||||
默认使用紧邻内容的短结果行或有标签的注释块,优先通过“同一节点名 + 明确位置”关联。长 L-leader 不再是默认必需元素;复杂节点可保留短连线,必须满足端点存在、在同一块内、不交叉、不覆盖文字。
|
||||
|
||||
当目标折叠、帧过滤、元素隐藏或几何测量失败时,不绘制连线。主题/字体/容器宽度变化后只重算当前可见块;采用可取消的局部布局更新,不全页不断测量。窄屏、打印、reduced-motion 下直接使用有编号的注释。截图中的孤立线条归入专门视觉回归场景。
|
||||
|
||||
### 8.3 键盘、屏幕阅读器与退化
|
||||
|
||||
- 目录用 nav,正文用 main/article,详情用有名字的 aside/section;标题层级连续。
|
||||
- 展开按钮有 `aria-expanded/controls`;选中状态、失败和待验证都有文字,不依赖颜色或图标。
|
||||
- 深链接滚动有 `scroll-margin-top`,聚焦目标不被固定工具条遮住;关闭详情后焦点回到触发点。
|
||||
- 不使用只有鼠标悬停才能看到的来源、代码或错误。大表格有标题与表头关系,日志可复制。
|
||||
- 无 JS 时正文、目录锚点、来源链接、原生 details 和静态结果仍工作;只失去实时联动与运行操作。
|
||||
- 选择帧不隐藏源结构;隐藏实现也不能删除文档含义和必需条件。
|
||||
|
||||
## 9. 技术设计与数据契约
|
||||
|
||||
### 9.1 从预拼 HTML 改为可关联的展示模型
|
||||
|
||||
当前 `FileViewModel.knowledge_html` 是独立 HTML 字符串,难以与正文、trace、诊断同步。建议新增结构化 `KnowledgePresentation`,在 Rust 端形成同一份模型,由正文、目录、详情共同消费。兼容阶段可保留旧字段,但不让它继续成为唯一知识接口。
|
||||
|
||||
```text
|
||||
ViewDocument
|
||||
document_id, source_digest, semantic_generation, title, description
|
||||
blocks[] 保真正文结构及 source span
|
||||
outline[] 标题/知识单元父子关系
|
||||
knowledge_nodes[] 对应 artifact/ref/step,不复制语义解释器
|
||||
diagnostics[] 范围、严重性、可读说明、定位
|
||||
execution_state 明确未运行/运行中/成功/失败/快照
|
||||
run_records[] 与文档版本相匹配的证据
|
||||
|
||||
KnowledgeNode
|
||||
node_id, artifact_id?, symbol_id?, ref_id?, step_id?
|
||||
owner_id?, source_span, display_name, authored_description?
|
||||
kind, signature?, payload?, dependencies[], provenance[]
|
||||
applicability?, verification[], effects, coverage
|
||||
execution_refs[]
|
||||
```
|
||||
|
||||
这些字段为建议契约,未承诺当前 IR 已提供所有信息。语义事实来自 `SemanticProject`,文档叙述来自解析后的源结构,执行事实来自 capture;缺少关系时展示 unknown,不让 UI 用名称模糊匹配或调用模型编造。
|
||||
|
||||
### 9.2 数据流
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
S[源文档与项目边界] --> P[解析与语义分析]
|
||||
P --> B[正文块与结构]
|
||||
P --> K[知识节点与诊断]
|
||||
R[显式运行或匹配的历史快照] --> T[执行记录]
|
||||
B --> V[统一展示模型]
|
||||
K --> V
|
||||
T --> V
|
||||
V --> D[正文与就地说明]
|
||||
V --> O[目录与当前知识]
|
||||
V --> E[运行区域与技术详情]
|
||||
```
|
||||
|
||||
知识模型不通过“render HTML → 再抓取文本”生成。页面分析失败不可用空字符串吞掉,应保留有状态的结果;部分成功与整体失败分开。
|
||||
|
||||
### 9.3 源位置、稳定身份与锚点
|
||||
|
||||
目前 step 跳转用 `#stmt-{line}`,函数用 `#fn-{line}`。一行多个表达式、正文段落合并、同名函数、源码插行都会使单纯行号不够。
|
||||
|
||||
新增内部节点锚点注册表,将文档块、函数、语义 step/ref 与真实 DOM id 映射;使用 artifact/node 身份和源 span 生成页内唯一 ID,不把未经编码的 URI 当 DOM id。当前编译器没有稳定 SyntaxNodeId 的地方,退化为“源码版本 + span”,明确跨版本不可保证稳定;不能只把行号哈希一下就宣称稳定身份。
|
||||
|
||||
兼容现有 `#structure`、`#fn-N`、`#stmt-N` 链接,保留别名或定位映射;所有生成链接都验目标存在。跨文件 href 复用 [html.rs 的 LinkMode/href_file](../../src/view/html.rs),静态导出保持相对路径;路径展示解码与 URL 安全编码分离。
|
||||
|
||||
### 9.4 分离分析与执行,修正 no-exec
|
||||
|
||||
将当前 `build_file_view` 拆成可组合路径(名称为建议):
|
||||
|
||||
1. `analyze_document`:读取、解析、语义检查、生成静态展示数据,不执行目标程序。
|
||||
2. `capture_execution`:只有执行策略明确允许时调用 run/cell runtime,返回真实状态与源版本。
|
||||
3. `assemble_view`:关联正文、语义和匹配的结果,生成同一个展示模型。
|
||||
|
||||
`--no-exec` 必须在第 2 步之前分支,不能先跑后清空;snapshot 模式使用匹配快照,不为了构造 UI 又运行一次。Live 现有自动执行兼容行为在首轮不静默改变,必须通过显式 execution policy 传入并如实标注;若后续将 live 默认改为阅读时不执行,作为独立 CLI 行为变更发布和迁移。点击详情、查看来源、展开关系等阅读操作绝不能触发执行。
|
||||
|
||||
知识详情不要求通过运行得出静态信息。实现参数预览/试跑时,使用既有受控执行路径,不因“pure”字样绕过宿主能力边界。本文不新增一套求值器或权限系统。
|
||||
|
||||
### 9.5 Live 更新一致性
|
||||
|
||||
当前 SSE 会替换 structure_html/bindings_html/trace_json,而知识面板来自另一次静态编译,容易新旧混用。建议更新包包含文档摘要、语义代、run_id 和变更节点;前端仅合并兼容版本的数据。
|
||||
|
||||
运行写回导致源码变化时,重新分析后一次更新相互关联的正文、目录、知识与运行映射。原选中节点仍存在则保留选择/焦点/滚动;已消失则明确提示,不能悄悄指向同名节点。旧请求返回晚于新请求时按版本丢弃,不覆盖较新阅读状态。
|
||||
|
||||
### 9.6 性能与导出体积
|
||||
|
||||
避免每个页面重复编译整个项目。一次静态导出共享同一项目语义结果,逐页投影;live 按项目源指纹、编译器版本、配置和适用性时钟缓存,读取使用一致的语义代。来源/有效期变化必须使相关状态失效,不能无限缓存 sourced。
|
||||
|
||||
正文服务端生成,不等待 JS 初始化或全部知识图完成才出现。默认只嵌入当前文档必要知识,深层依赖按需取;静态导出则包含保证离线可用的局部数据与明确范围。大量结果采用折叠/分页,但不能让屏幕阅读器和查找失去正文。
|
||||
|
||||
性能验收目标:100 个知识单元的本地选择/展开交互 p95 ≤100ms;1000 单元目录可搜索而不阻塞正文滚动;导出项目的语义编译次数不随页面数线性重复。记录硬件、样例和体积,目标未达时先优化数据重复和布局测量,不以截断关键来源达标。
|
||||
|
||||
## 10. 实施文件与兼容策略
|
||||
|
||||
| 位置 | 建议变更 |
|
||||
|---|---|
|
||||
| [src/view/render.rs](../../src/view/render.rs) | 提取知识展示模型;替换全量 dl 面板;关联步骤归属、友好名称与诊断 |
|
||||
| [src/view/document.rs](../../src/view/document.rs) | 源块/知识节点映射、正文就地入口、层级标题、可复制原始 span |
|
||||
| [src/view/html.rs](../../src/view/html.rs) | 正文优先布局、辅助栏、详情/目录交互、状态组件、统一 tokens;保留 LinkMode |
|
||||
| [src/view/mod.rs](../../src/view/mod.rs) | 分离分析/执行、保留诊断、组装统一模型、项目分析缓存 |
|
||||
| [src/view/output.rs](../../src/view/output.rs) | 提前应用 no-exec/snapshot 策略、共享项目分析、静态详情与相对跳转 |
|
||||
| [src/view/annot.rs](../../src/view/annot.rs) | 运行标注与节点/帧/版本关联,减少重复输出;缺映射不伪绑定 |
|
||||
| [src/view/debug_page.rs](../../src/view/debug_page.rs) | 仅回归检查共享组件;不把阅读布局强加给 debug |
|
||||
| [tests/executable_knowledge.rs](../../tests/executable_knowledge.rs)、view 单测 | 从字符串存在升级到顺序、锚点、状态和执行边界断言 |
|
||||
| 文档与 public 示例 | 更新旧 View 布局说明、截图和行为描述,区分历史默认与新设计 |
|
||||
|
||||
默认展示不要求作者补特殊 frontmatter;title/description、函数说明、契约和来源已有时直接受益。尚无信息的旧文档保真显示,并在详情中如实缺省。
|
||||
|
||||
与现有文档的关系:保留 [view-document-annotations](view-document-annotations.md) 的叙述主导和真实结果,修订“长 L-leader 为默认”;保留 [view-frame-selection](view-frame-selection.md) 的源码常显与帧隔离;[view-markup-surface](view-markup-surface.md) 的历史 Classic 例子需按当前默认语法同步,不能在新 UI 继续教授旧取元。外联/WASM 交互沿既有独立路线发展,不作为完成本次阅读改版的前置。
|
||||
|
||||
## 11. 分阶段交付
|
||||
|
||||
### V0:先纠正顺序、误导状态和执行边界
|
||||
|
||||
正文提前;完整知识报告收进明确的辅助入口,module/type 容器默认不铺卡;“Lock digest”改为准确的摘要标签;状态不再共用一个 ok。修复 no-exec 先执行问题,分析错误可见。暂不等待完整节点模型,就能显著改善截图场景。
|
||||
|
||||
V0 不算最终完成:仅把面板搬到后面/折叠,仍未解决正文与知识的关联。
|
||||
|
||||
### V1:完成阅读与结构理解的主闭环
|
||||
|
||||
统一展示模型、标题/目录层级、就地说明、当前知识详情、友好定义跳转与返回。完成函数、计算、来源、空态与基本运行结果的正负例;live/static 共用。
|
||||
|
||||
### V2:贯通运行、来源和跨文档关系
|
||||
|
||||
匹配运行实例与版本、调用/循环帧、来源定位/失效、局部依赖关系、SSE 原子更新。上游证据能力尚不完整的地方明确显示未知,不能以 UI 补出“验证通过”。
|
||||
|
||||
### V3:移动端、离线和规模验收
|
||||
|
||||
完成键盘/读屏、无 JS、打印、暗色/浅色、100/1000 单元性能、视觉回归与读者测试。发布前同步 public 示例与说明,保留旧锚点兼容。
|
||||
|
||||
## 12. 验收矩阵
|
||||
|
||||
### 12.1 必测文档
|
||||
|
||||
| 样例 | 必须验证 |
|
||||
|---|---|
|
||||
| 截图对应 Phase I 页 | 正文先出现;无 full URI/hash 瀑布;print 在所属正文解释;无孤立线条 |
|
||||
| 纯叙述/少量 Markdown 页 | 阅读无知识空卡、无误导运行徽章 |
|
||||
| 单个纯函数,无 main | 参数/返回与调用方式清楚,不因无入口标全文失败 |
|
||||
| 订单定价、多文档 import | 小计/税率/总额归属和依赖可追踪;输入和结果可对应 |
|
||||
| 带来源的事实与过期/缺失来源 | sourced 不显示已证实;问题在对应知识旁可见 |
|
||||
| 同名函数、多次调用和循环分支 | 定义唯一;切换帧只改结果和高亮,不隐藏结构 |
|
||||
| 空 payload、动态调用、未解析引用 | 待实现/动态/未解析准确可辨,不能展示虚构结果 |
|
||||
| 一行多个链接、嵌套表达式、中英文/长路径 | span 对应、锚点唯一、文本不丢失、不水平溢出 |
|
||||
| 运行失败、分析失败、快照过期 | 三种情况分开,仍能读到正文与具体诊断 |
|
||||
| 外联单元、变量大表、长日志 | 单层主体与对应结果,不重复三份输出;局部滚动 |
|
||||
|
||||
### 12.2 自动化判据
|
||||
|
||||
1. DOM 中文档主体先于全量知识技术详情,标题来自既有元数据;短文没有隐藏正文的默认展开报告。
|
||||
2. 每个知识/目录链接都能定位真实节点;一行多节点和同名跨模块不冲突;live/static 路径均有效。
|
||||
3. 验证、运行、来源状态按固定真值表渲染;无证据时绝不输出综合已验证。
|
||||
4. no-exec 用隔离目录的副作用计数测试证明没有执行;snapshot 不重跑;展开详情也不执行。
|
||||
5. 帧选择和 SSE 更新保留正文结构、选中身份与焦点,拒绝过期结果包。
|
||||
6. 明暗主题/320、390、1024、1440 宽度/200% 缩放下无遮挡、截断关键条件和整页横向溢出;孤立 leader 数为零。
|
||||
7. 无 JS、file 静态打开、打印预览都保留正文、定义/来源导航和必要证据说明。
|
||||
8. 输出中的作者内容、label、URI/属性与嵌入 JSON 按原上下文转义;资料中的 HTML/脚本不能因为新详情组件被直接执行。
|
||||
|
||||
字符串断言只能覆盖结构,不能替代浏览器布局验收;需截图基线和真实键盘交互测试。可复用现有测试基础,避免测试只镜像组件实现。
|
||||
|
||||
### 12.3 读者任务测试
|
||||
|
||||
建议用至少 5 位未参与实现的读者,分别完成三个任务,并与当前页面对照:
|
||||
|
||||
- 在 30 秒内说清文档目的并找到主要规则/函数。
|
||||
- 在 60 秒内找到某个结果的输入、定义和原始来源。
|
||||
- 正确区分“来源已关联但未核验”“本次运行成功”“未运行/历史结果”,不得误认为都已验证。
|
||||
|
||||
目标为至少 4/5 完成前两项,第三项无误导性判断;小样本仅作为可用性门槛,不声称统计显著。记录失败位置并修正信息架构,不能只问“是否好看”。
|
||||
|
||||
## 13. 完成定义与本次交付边界
|
||||
|
||||
完成的 view 应让读者先读懂文档,再自然看到其中的知识与执行结构;需要深入时能沿着一个具体片段追到实现、输入、结果和证据。全量技术字段仍可查,但不要求读者先理解编译器才能读文档。
|
||||
|
||||
本次只提交设计文档与索引,不修改渲染实现。实证包括用户截图、当前模板/渲染路径、静态 HTML 顺序检查和临时 no-exec 反例;布局尺寸、交互、性能和读者任务结果均为待实施的验收目标。
|
||||
@@ -0,0 +1,163 @@
|
||||
# Web 热路径:单解释器串行化(设计缺陷说明)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted**(缺陷记录 + 缓解已落地) |
|
||||
| 日期 | 2026-10-26 |
|
||||
| 规范决策 | [ADR 0008](../adr/0008-web-invoke-serialization.md) |
|
||||
| 相关 | [ext-web-go-rewrite.md](ext-web-go-rewrite.md) · [ext-abi.md](ext-abi.md) · [ADR 0004](../adr/0004-web-plugin-go.md) · [ADR 0007](../adr/0007-web-document-endpoint.md) |
|
||||
|
||||
本文把「页面卡死 / 像单线程」拆成可验证的架构事实、错误语义与演进边界。
|
||||
**结论先行:HTTP 是多线程的;卡住是因为所有 Endpoint 共享一把解释器锁。**
|
||||
|
||||
增量实施提案见 [隔离执行与响应式 Web 运行时(2026-10-26)](isolated-web-runtime-development-2026-10-26.md):独立进程池、控制/读取容量、请求级事件与取消、缓存和恢复。提案通过验收前,本文件及 ADR 0008 的现行单飞约束不变。
|
||||
|
||||
---
|
||||
|
||||
## 1. 问题陈述
|
||||
|
||||
### 1.1 用户可见症状
|
||||
|
||||
1. 发起一次长对话(OpenAI 兼容 `POST …/v1/chat/completions` + `stream:true` / `sse-live`)。
|
||||
2. 对话尚未结束时,打开设置、拉 `/v1/models`、或其它短 JSON API。
|
||||
3. **旧行为**:浏览器请求一直 pending,整页像死机;只能杀 `marqdo run serve…` 恢复。
|
||||
4. **目标行为**:短请求在约 1.5s 内得到 **HTTP 503**,错误体标明 `interpreter busy`;对话 SSE 继续;结束后短请求立刻恢复 200。
|
||||
|
||||
### 1.2 错误归因(不要再说错)
|
||||
|
||||
| 错误说法 | 正确说法 |
|
||||
|----------|----------|
|
||||
| 「web 插件是单线程的」 | Go `net/http` 与 SSE 刷写是多 goroutine |
|
||||
| 「忘记启动进程所以卡死」 | 进程在跑;是 **锁争用** 导致短请求永不返回 |
|
||||
| 「把 SSE 的等待时间调大就能修好」 | 进程级拉长 `MARQDO_INVOKE_WAIT_MS` 会让短请求一起卡死 |
|
||||
| 「去掉 INVOKE_LOCK 即可并行」 | `HostContext` 非 Sync,无锁并行会数据竞争 / 未定义行为 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 调用栈与锁边界
|
||||
|
||||
```
|
||||
Browser
|
||||
│ GET /v1/models POST /v1/chat/completions (SSE)
|
||||
▼
|
||||
Go plugins/web (多 goroutine)
|
||||
handleInvoke / handleInvokeLiveSSE
|
||||
│
|
||||
▼ C ABI host_query("call_lib_path", {path, args})
|
||||
Rust src/host/plugin.rs
|
||||
┌─────────────────────────────┐
|
||||
│ INVOKE_LOCK (进程级 Mutex) │ ← 同一时刻只允许一个 worker 进入
|
||||
└─────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
HostContext:装载/求值作者 .mq.md、写回、事件总线、LLM 插件回调…
|
||||
```
|
||||
|
||||
要点:
|
||||
|
||||
1. **锁在宿主,不在 Go 业务路由表。** 增加更多 Go handler 不会自动并行执行 Marqdo。
|
||||
2. **SSE 持锁时间 ≈ 整次 Endpoint 求值时间**(含多次 LLM 往返),可达数十秒到数分钟。
|
||||
3. **事件转发**(`MARQDO_EVENT_FORWARD` NDJSON)与 **取消**(`MARQDO_CANCEL_TOKEN`)在持锁期间仍可由其它 goroutine 写文件/读桥;它们不进入解释器,故不受 `INVOKE_LOCK` 保护,也不应误设长等待。
|
||||
|
||||
---
|
||||
|
||||
## 3. 为何不能简单「多线程跑 Marqdo」
|
||||
|
||||
`HostContext` 持有(非穷尽):
|
||||
|
||||
- 当前模块图、函数表、写回缓冲与 FS 根;
|
||||
- 插件注册与 `CURRENT_HOST` / 全局 host 指针;
|
||||
- Agent / LLM 运行时与 stream 发布钩子;
|
||||
- 与文档写回、会话态相关的可变结构。
|
||||
|
||||
这些状态在实现上按**单线程求值**建模。把它标成 `Sync` 或去掉 `INVOKE_LOCK` 而没有隔离运行时,属于未定义行为,不是性能优化。
|
||||
|
||||
**真并行的合法方向(未实现):**
|
||||
|
||||
1. **解释器池**:每请求或每会话一个 `HostContext`(或可重置的 isolate),路由层选闲置实例;
|
||||
2. **只读快照 + 写队列**:热路径只读克隆,写回串行合并(适合纯查询 Endpoint,不适合带写回的 agent);
|
||||
3. **把长任务移出解释器**:Marqdo 只做编排,重活在独立进程/队列(与「代码即文档」主路径权衡后再定)。
|
||||
|
||||
在上述任一方案落地并经金样例验收前,**产品必须以「单飞解释器」为约束设计 UX**。
|
||||
|
||||
---
|
||||
|
||||
## 4. 缓解契约(已实现)
|
||||
|
||||
### 4.1 宿主:限时争锁
|
||||
|
||||
文件:`src/host/plugin.rs`
|
||||
|
||||
- 仅当从 worker 线程进入(`CURRENT_HOST` 为空、走全局 host)时争 `INVOKE_LOCK`。
|
||||
- `acquire_invoke_lock`:`try_lock` 循环,预算默认 **1500ms**(`MARQDO_INVOKE_WAIT_MS`,最小 1)。
|
||||
- 超时:`set_err("interpreter busy: another request holds the Marqdo invoke lock (waited …ms)")`,返回非 0。
|
||||
- **不要**在 SSE 入口把该环境变量设成数分钟:它是进程级的,会污染并发短请求。
|
||||
|
||||
### 4.2 Web 插件:503
|
||||
|
||||
文件:`plugins/web/internal/httpx/routes_invoke.go`
|
||||
|
||||
- 普通 JSON invoke:错误信息含 `interpreter busy` → **503** + `Retry-After: 2` + `{ok:false, error:…}`。
|
||||
- Live SSE:若启动时即 busy,以失败事件/错误结束该流,**不得**无限挂起握手。
|
||||
|
||||
### 4.3 客户端:超时与文案
|
||||
|
||||
短 API(模型列表、设置、registry 探测等)必须:
|
||||
|
||||
- 设置合理 `fetch` 超时(秒级,例如 8s);
|
||||
- 将 503 / `interpreter busy` 展示为「服务正忙(对话进行中)」类可读提示,而不是空白转圈。
|
||||
|
||||
(qdagent 前端 `frontend/src/lib/api.ts` 已按此契约处理。)
|
||||
|
||||
### 4.4 验收探针
|
||||
|
||||
在本地 serve 上:
|
||||
|
||||
```bash
|
||||
# 终端 A:启动长 SSE 对话(持锁)
|
||||
curl -N -m 120 -H 'Authorization: Bearer …' -H 'Content-Type: application/json' \
|
||||
-d '{"stream":true,…}' http://127.0.0.1:PORT/v1/chat/completions
|
||||
|
||||
# 终端 B:对话进行中
|
||||
curl -sS -w 'http=%{http_code} time=%{time_total}\n' --max-time 5 \
|
||||
-H 'Authorization: Bearer …' http://127.0.0.1:PORT/v1/models
|
||||
# 期望:http=503,time≈1.5,body 含 interpreter busy
|
||||
|
||||
# 对话结束后再打同一 URL → http=200,time≪1s
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 运维与调参
|
||||
|
||||
| 变量 | 默认 | 含义 |
|
||||
|------|------|------|
|
||||
| `MARQDO_INVOKE_WAIT_MS` | `1500` | worker 进入解释器前最长等待;超时 → busy |
|
||||
|
||||
建议:
|
||||
|
||||
- 开发机 / 产品默认保持 1–3s。
|
||||
- **禁止**为「让第二个聊天排队」而把默认调到分钟级——那等于恢复卡死。
|
||||
- 需要排队时,应在 **应用层**(会话队列、UI 禁用「再发一条」),而不是拉长宿主锁等待。
|
||||
|
||||
---
|
||||
|
||||
## 6. 与其它机制的边界
|
||||
|
||||
| 机制 | 与 INVOKE_LOCK 的关系 |
|
||||
|------|----------------------|
|
||||
| `MARQDO_EVENT_FORWARD` | SSE 期间写 NDJSON;不替代争锁;不延长等待 |
|
||||
| `MARQDO_CANCEL_TOKEN` | 断开连接时可取消 LLM;释放锁仍取决于解释器何时回到 FFI |
|
||||
| `web.serve` / 静态资源 | 通常不经 `call_lib_path`,不受本锁影响 |
|
||||
| 插件内部 Tokio/Go 并发 | 可以;只要不再次 `call_lib_path` 进同一 HostContext |
|
||||
|
||||
---
|
||||
|
||||
## 7. 演进检查清单(根治前勿宣称「已支持并行 Endpoint」)
|
||||
|
||||
- [ ] 多 `HostContext` 或 isolate 的生命周期与 ABI(装载、shutdown、写回冲突)有设计稿与金样例。
|
||||
- [ ] 池化后 `call_lib_path` 不再依赖进程级单锁,或锁粒度降到实例级。
|
||||
- [ ] 两个并发长 SSE + 一个短 `/v1/models` 的压力测试:短请求稳定 200,长请求互不污染会话写回。
|
||||
- [ ] 更新本文件与 ADR 0008 Status(例如改为 Superseded by ADR NNNN)。
|
||||
|
||||
在清单完成前,对外与文档中应明确:**Marqdo Web Endpoint 求值在单进程内串行;HTTP 层并发,解释器层单飞。**
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **实施基线(Draft for Implementation)** |
|
||||
| 状态 | **目标实施规范;按文档原生指导修订交付与政策分级** |
|
||||
| 日期 | 2026-10-23 |
|
||||
| 适用版本 | v2.1 及后续兼容演进 |
|
||||
| 真实意图来源 | [`009.md`](009.md)、[`Marqdo_Executable_Knowledge_Design.md`](Marqdo_Executable_Knowledge_Design.md)、[`Marqdo_Executable_Knowledge_Example.md`](Marqdo_Executable_Knowledge_Example.md) |
|
||||
@@ -10,17 +10,17 @@
|
||||
|
||||
## 0. 文档地位与执行规则
|
||||
|
||||
本文把上述三份文档中的真实产品意图整理为可编码、可测试、可迁移、可验收的开发规范。开发本轮能力时,应以本文和三份意图来源为准;仓库中更早的链接、遮蔽、知识分层、类模型、OKF 或 EKC 文档若与本文冲突,不得据其改变本轮设计。
|
||||
本文把上述三份文档中的真实产品意图整理为可编码、可测试、可迁移、可验收的开发规范。2026-10-23 的[文档原生开发指导](../design/document-native-development-guide.md)进一步约束作者负担、AI 编辑与复用、场景分级和交付顺序;本文的内部结构不能反向要求作者重复填写注册表、工具 schema 或知识图。仓库中更早的链接、遮蔽、知识分层、类模型、OKF 或 EKC 文档若与上述方向冲突,不得据其改变本轮设计。
|
||||
|
||||
优先级从高到低如下:
|
||||
|
||||
1. `009.md` 中确认的产品目标与边界;
|
||||
2. 本文冻结的工程决策、数据结构和验收条件;
|
||||
3. 两份附属文档中的概念设计与完整示例;
|
||||
4. 当前代码的兼容事实;
|
||||
2. [文档原生开发指导](../design/document-native-development-guide.md)对作者体验、政策分级和交付排序的修订;
|
||||
3. 本文的工程决策、数据结构和验收条件;
|
||||
4. 两份附属文档中的概念设计与完整示例;
|
||||
5. 其他旧文档,仅可作为历史背景,不是规范依据。
|
||||
|
||||
本文不宣称目标能力已经实现。表格中“当前”表示仓库 v2.0.0 的实际行为,“目标”表示完成相应里程碑后的行为。任何阶段只有在对应的自动化测试、双后端一致性测试和迁移测试全部通过后,才可从“目标”改标为“已实现”。
|
||||
本文不宣称目标能力已经实现。未另行标日期的“当前”表格与 §2 缺口保留最初 v2.0.0 审计背景,不能继续当作 v2.1 的实时状态:默认 Occlusion、线程局部解析作用域、关联函数与零参路径等已经演进。当前实现判断见[2026-10-23 缺口审计](../roadmap/executable-knowledge-gaps-2026-10-23.md)及对应源码/测试;代码事实证明实现范围,不自动取代产品目标。任何阶段只有在对应的自动化测试、双后端一致性测试和迁移测试全部通过后,才可改标为“已实现”。CLI/API 示意与结构定义仍是目标协议,使用前须核对实现。
|
||||
|
||||
---
|
||||
|
||||
@@ -56,7 +56,10 @@ Marqdo 不是“能运行 Markdown 中几段代码”的解释器,也不是“
|
||||
- 代码、依赖、来源或验证状态变化时,影响范围能够被程序找出;
|
||||
- Tree Interpreter 与 Bytecode VM 对同一已绑定程序产生一致结果和诊断;
|
||||
- Agent 命中已有且适用的制品时,得到的是可调用句柄,不是让模型复制源码再写一遍;
|
||||
- 系统从不把 `compiled`、`tested` 或 `source_present` 误报成“现实世界事实已证明正确”。
|
||||
- 系统从不把 `compiled`、`tested` 或 `source_present` 误报成“现实世界事实已证明正确”;
|
||||
- 作者能从正文计算开始,按需要提取命名能力与跨文档复用,不先建立知识治理工程;
|
||||
- 人和 AI 能按标题/步骤获得最小上下文、局部修改并查看语义影响,不必整篇重生成;
|
||||
- 每个交付切片同时证明阅读、修改或复用的实际收益,而不只证明内部字段存在。
|
||||
|
||||
### 1.3 不在本轮目标内
|
||||
|
||||
@@ -69,9 +72,17 @@ Marqdo 不是“能运行 Markdown 中几段代码”的解释器,也不是“
|
||||
- 不把自然语言标签当作授权、契约或事实正确性的依据;
|
||||
- 不让 EKC、view、check、Agent 各自再次解析源码并形成不同理解。
|
||||
|
||||
### 1.4 作者成本与渐进严格
|
||||
|
||||
标题、叙述、表格、链接描述和执行语句是作者的主要工作面;符号 ID、步骤 ID、规范签名、效应闭包、工具 schema、图和摘要由工具生成。不能推导的领域条件、来源和许可由作者明确补充,不要求每个纯工具函数都填写事实证据。
|
||||
|
||||
本地探索允许无锁运行草稿,但不绕过宿主能力限制;项目自动复用按所选 policy 校验;严格业务再增加必要证据、版本锁和新鲜度;远程/多租户另有可信授权与隔离门槛。这是同一语法上的政策分级,不是新的语法模式。纯预览可复用已授予的项目/会话政策,不做逐链接重复确认。详情见[指导 §6](../design/document-native-development-guide.md#6-渐进严格不把治理门槛塞进每一行正文)。
|
||||
|
||||
自然语言标签是发现、解释和局部编辑的重要信息,但不证明 payload 正确。AI 可以选候选、组合、适配、重构及补测试;编译器确定实际绑定、类型和效应,宿主决定能力上限。不得把“不用模型决定语义”误读成“模型只能搜索”。
|
||||
|
||||
---
|
||||
|
||||
## 2. 当前实现基线与真实缺口
|
||||
## 2. 初始实现基线与缺口背景(非 v2.1 实时清单)
|
||||
|
||||
### 2.1 可以复用的现有基础
|
||||
|
||||
@@ -890,6 +901,10 @@ TASK
|
||||
|
||||
REUSE 的判据不是“搜索分数超过阈值”,而是候选已经精确绑定、输入满足契约、effect 获得许可、版本策略通过。搜索分数只用于候选排序。
|
||||
|
||||
`ready` 是对所选 policy、输入与上下文的判断,不是所有知识必须先获得的全局认证。AI 可以从候选中选择明确目标;歧义不能交给执行器按扫描顺序消解。ADAPT/CREATE 必须遵守任务写权限和预算,缺少业务决策时请求补充,而非擅自改变适用条件。草稿可被发现并在明确权限下探索执行,但不得冒充已通过严格政策的事实。
|
||||
|
||||
发现结果应按需包含标题/步骤、源码片段、输入角色、契约、效应、来源、验证缺口和相关失败经验。局部编辑须携带期望源码摘要,应用后检查语义 diff 与相关测试;不以重写整份文档作为默认修复方式。协议及受限试点范围见[指导 §5](../design/document-native-development-guide.md#5-ai-原生工作协议读步骤改步骤验证步骤)。
|
||||
|
||||
### 12.4 BoundArtifactHandle
|
||||
|
||||
```rust
|
||||
@@ -906,17 +921,21 @@ pub struct BoundArtifactHandle {
|
||||
|
||||
Agent 执行必须携带 handle 和显式输入;runtime 再次核对摘要和权限,防止发现后到执行前目标变化。handle 是能力引用,不包含可由模型修改后悄悄执行的源码副本。
|
||||
|
||||
上述结构表达绑定内容,不独自构成远程授权凭据。受信本地调用可先采用精确引用和摘要复核;不受信/远程调用必须另有服务端认可的主体、能力范围、防伪及撤销/期限机制,客户端自报 `accepted_effects` 不能扩大宿主许可。handle 由工具产生,不要求作者维护;是否检查 lock 和来源新鲜度取决于明确的 policy。
|
||||
|
||||
### 12.5 Promote 门槛
|
||||
|
||||
Agent 新建或适配的知识进入可复用索引前,至少完成:
|
||||
Agent 新建或适配的知识进入**共享自动复用的 ready 集合**前,按目标 policy 至少完成:
|
||||
|
||||
- Parse/Resolve/Type/Contract 通过;
|
||||
- 声明 effect;
|
||||
- effect 已由静态绑定与宿主/扩展元数据推导;不能推导时保留 unknown 并按政策拒绝或要求明确能力许可,不要求业务作者抄写闭包;
|
||||
- pure 逻辑有测试,外部 effect 有 mock/沙箱证明或人工批准;
|
||||
- 来源型知识附 SourceRef;
|
||||
- 生命周期从 `draft` 开始;
|
||||
- 生成者、输入、工具版本、变更 diff 可审计;
|
||||
- 未达到门槛的内容只在 quarantine/draft index 中可见。
|
||||
- 未达到门槛的内容保持 draft/quarantine 状态;可在同一权威索引中过滤展示,不另造语义身份。
|
||||
|
||||
这是晋升政策,不是普通 `.mq.md` 文件被索引、查看或本地执行的前置注册流程。简单纯计算不强制现实来源;声称业务事实成立的知识不能用测试通过替代所需证据。
|
||||
|
||||
---
|
||||
|
||||
@@ -1117,7 +1136,7 @@ View 至少展示:
|
||||
|
||||
## 17. 分阶段开发计划
|
||||
|
||||
每个里程碑都必须形成可独立合并的垂直切片;不得先大规模重构再等待最后一次性验收。
|
||||
M0–M6 保留为工程能力与验收项目,不再作为顺序排期;部分迁移能力已经完成,状态见[现状审计](../roadmap/executable-knowledge-gaps-2026-10-23.md)。实际工作按[文档原生 S0–S6](../design/document-native-development-guide.md#7-交付切片每片都有文档作者与-ai-能看见的结果)形成可独立合并的垂直切片:早期联动 M1/M2/M3/M5 的安全子集,交付可读步骤和真实复用,不能等 M4 全部做完才让作者或 Agent 使用。100 单元生成器逐步增长,不阻塞最小故事;不得先大规模重构再等待最后一次性验收。
|
||||
|
||||
### M0:语义冻结与测试生成器
|
||||
|
||||
@@ -1218,7 +1237,7 @@ View 至少展示:
|
||||
- 图中 calls 边与实际运行 trace 的静态目标一致;
|
||||
- Agent 命中 ready 制品时执行 handle,不生成新函数;
|
||||
- 只有模糊候选时不能直接执行;
|
||||
- 未验证、过期、effect 不允许的制品不会被标记为 REUSE ready。
|
||||
- 不满足所选验证政策、已过相关有效期或 effect 不允许的制品不会被标记为该 policy 下的 REUSE ready;本地探索不冒充严格验证。
|
||||
|
||||
### M6:迁移默认值与发布
|
||||
|
||||
@@ -1241,6 +1260,8 @@ View 至少展示:
|
||||
|
||||
## 18. 文件级实施清单
|
||||
|
||||
下表是目标职责映射,不是“所有文件尚未创建”的任务表。`src/project/`、`src/sema/` 和部分 API 已存在,先检查对应缺口与测试,再增量修改;不要依照旧“新建”措辞重复搭建模块或移走仍被消费者使用的能力。
|
||||
|
||||
| 文件/模块 | 修改 |
|
||||
|---|---|
|
||||
| `src/ast.rs` | 增加 SyntaxNodeId;`Occluded` 迁为 `Named`;承载 Narrative AST 的最小引用 |
|
||||
@@ -1372,6 +1393,8 @@ View 至少展示:
|
||||
- 总 token、工具调用、修复轮次和耗时;
|
||||
- 回答能否追溯到 SymbolId、digest、输入和来源。
|
||||
|
||||
同时记录作者手写声明/重复字段/配置数量、阅读定位准确率、语义步骤修改范围、无关 diff、并发补丁冲突和首次写作成本。比较时各方案应获得同等现有能力与输入信息,不向新方案额外提供答案;将编译和人工准备成本计入总成本。先通过正确性与权限硬门槛,再讨论 token 和速度收益,参见[指导 §8](../design/document-native-development-guide.md#8-验证-marqdo-优势而不只验证组件存在)。
|
||||
|
||||
### 20.3 发布门槛
|
||||
|
||||
- 核心确定性套件 100% 通过;
|
||||
@@ -1385,6 +1408,8 @@ View 至少展示:
|
||||
|
||||
不要只报告平均值;保存失败样本、任务定义、环境、置信区间和版本摘要。
|
||||
|
||||
以上按发布宣称的能力范围执行。语法兼容、本地确定性复用、严格业务复用、远程执行分别有独立证据,不以一个总体 `done` 替代。M0–M6 保留为工程能力检查项,实际交付顺序采用[修订路线 S0–S6](../roadmap/executable-knowledge-gaps-2026-10-23.md#7-依赖顺序与可合并切片):早期完成一篇文档的读/执行/复用闭环,再增加治理深度;任何已开放执行路径都不能延后安全门禁。
|
||||
|
||||
---
|
||||
|
||||
## 21. 书店示例的最终验收链
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
# Marqdo 现状与规划缺口:从可执行文档到可复用知识
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | 现状审计与下一阶段规划;不是已实现能力声明 |
|
||||
| 审计基线 | `main` / `434be9f`,2026-10-23;后续改动须重新核验对应判断 |
|
||||
| 产品意图 | [009.md](../next/009.md) · [详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md) · [完整示例](../next/Marqdo_Executable_Knowledge_Example.md) |
|
||||
| 目标 | **代码即文档,文档即知识库,知识库可执行,可执行知识可复用** |
|
||||
| 方向修订 | 2026-10-23:按[文档原生开发指导](../design/document-native-development-guide.md)重排交付,G01–G12 保留为缺口证据,不等于作者必须经历的治理流程 |
|
||||
|
||||
## 1. 本文解决什么问题
|
||||
|
||||
仓库已经拥有语言运行时、双后端、标准库、扩展、文档站、EKC、Agent 和一批语义制品结构。现有[详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md)定义了目标架构和 M0–M6,但它主要写于目标能力尚未落地之时;现在照它逐项勾选,容易把“结构存在”“CLI 可调用”“目标闭环成立”混为一谈。本文只补**现状与目标之间尚未被规划成可验证闭环的工作**。语法和产品目标仍以前述意图文档为准;旧文档中的 Classic 链接取元、单体知识类、或另一套知识索引不应反向规定新语义。
|
||||
|
||||
本轮判断以源码、可定位的测试和文档状态为证据。`done` 表示有可运行入口和回归测试;`partial` 表示有结构或单一路径,仍不满足产品判据;`design` 表示主要存在于文档和目标示例。**不能把能解析、能运行、能检索、能验证事实视作同一个结论。**
|
||||
|
||||
### 1.1 对上一版规划排序的修正
|
||||
|
||||
上一版把内部工程完整度放在主要位置,直到 S6/S7 才验收 Agent 复用和作者体验,容易造成“先建治理平台,再证明文档有用”的偏差。本版改为从可读步骤、确定执行和早期本地复用向外扩展;每片同时交付源码示例、诊断和消费者可见结果。工程基础按切片所需完成,不降低已开放路径的安全要求。
|
||||
|
||||
P0/P1/P2 分类改为缺口领域,不再按编号整组串行:G06 的未知效应安全下限必须进入最早执行试点,G09 的本地复用也不再等锁和远程授权全部完成。复杂业务的证据、严格锁和远程凭据按使用场景增加,不能强加给每个单文件程序。每片还须记录作者重复声明数、局部修改范围和实际复用结果,指标见[指导 §8](../design/document-native-development-guide.md#8-验证-marqdo-优势而不只验证组件存在)。
|
||||
|
||||
## 2. 设计思想落实为四个可证伪承诺
|
||||
|
||||
| 承诺 | 必须由什么证明 | 当前判定 |
|
||||
|---|---|---|
|
||||
| 代码即文档 | 一份 `.mq.md` 的叙述、契约、示例、可执行语句由同一解析结果驱动;改变语义会让说明/示例失配可见 | 运行与 view 已有基础;正文引用尚未进入完整语义 AST(partial) |
|
||||
| 文档即知识库 | 可寻址的事实、规则、步骤、来源、适用范围和关系由编译器生成;不同消费者读取同一身份和图 | 项目索引与 Artifact 投影已起步;双图和占位字段仍存在(partial) |
|
||||
| 知识库可执行 | 引用先绑定、检查参数/契约/效应/版本,再由运行时执行**绑定的目标**;正文默认不触发副作用 | CLI 可解析、运行时可执行,但两条路径未闭合(partial) |
|
||||
| 可执行知识可复用 | Agent 发现候选后取得有条件、可失效的句柄,实际调用既有制品;trace 记录符号、摘要、输入和结果 | MLSP 有初版 handle;EKC/Agent 尚未统一到此协议(partial) |
|
||||
|
||||
这四条有语义依赖,但应在受限故事中逐步贯通,不是四个必须全部做完才能进入下一层的大项目。阅读体验和检索召回不能代替绑定;绑定不能证明知识内容正确;执行成功不能证明已经复用既有知识。
|
||||
|
||||
## 3. 仓库能力盘点与规范 M0–M6 的真实进度
|
||||
|
||||
| 既有里程碑 | 可复用成果 | 仍未满足的完成条件 |
|
||||
|---|---|---|
|
||||
| M0 语义与测试 | Occlusion 已为默认;[语法测试规程](../next/Marqdo_Executable_Knowledge_Testing.md)和部分金样存在;标准库、扩展库已经迁移 | `tests/executable_knowledge/` 系统夹具、100 单元生成器、导航/正文/效应负例矩阵未形成独立门禁 |
|
||||
| M1 项目身份 | [`src/project/mod.rs`](../../src/project/mod.rs)确定项目根和扫描顺序;[`src/sema/symbols.rs`](../../src/sema/symbols.rs)产生 ModuleId/SymbolId;CLI `knowledge inspect/resolve` 可用 | 身份保留、移动/重命名、symlink/重复扫描、大项目增量与错误聚合还需完整策略和测试 |
|
||||
| M2 统一链接与调用 | 裸零参路径和关联函数已有 Tree/Bytecode、CLI 金样;`Expr::Occluded` 已承载标签和 payload | [`src/ast.rs`](../../src/ast.rs)仍无统一 Named/正文引用节点;[`src/sema/ir.rs`](../../src/sema/ir.rs)只提取执行体部分调用;运行时仍独立按名称寻找调用目标 |
|
||||
| M3 语义制品 | SemanticArtifact、EffectSet、SourceRef、VerificationRecord、JSON 投影已存在 | 适用性、来源证据、单位、真实验证状态、完整效应闭包与规范摘要仍缺;字段存在不代表语义已实现 |
|
||||
| M4 锁与影响 | `knowledge lock/impact` 可用;[`src/sema/lock.rs`](../../src/sema/lock.rs)支持原子写入和反向依赖查询 | 锁只比较 artifact 摘要;目标、接口、实现、证据、传递闭包变更未独立报告;正文/证据边不在影响闭包内 |
|
||||
| M5 消费者 | view 显示部分语义信息;MLSP 提供 `artifact.*`;EKC/Agent 有各自检索与复用能力 | 同一图/同一绑定尚未贯通;Agent 对 ready 的判定与执行句柄未被强制;trace 不足以证明实际复用 |
|
||||
| M6 迁移与发布 | 全仓新语法迁移、`--legacy-markup` 入口、v2.1.0 发布已完成 | 发布不等于 M0–M5 已完成;目标示例仍注明“尚不能完整运行”,100 单元与 Agent 对照实验未过 |
|
||||
|
||||
`examples/executable-knowledge/README.md` 是目标示例,仍明确说明类函数、统一 `Named<T>` 和步骤图的完整运行是设计目标。它不得作为“产品闭环已完成”的发版证据。
|
||||
|
||||
### 3.1 相邻路线如何进入本规划
|
||||
|
||||
| 仓库区域 | 当前路线状态 | 与可执行知识主线的连接点 |
|
||||
|---|---|---|
|
||||
| Interpreter / 标准库 | Tree、Bytecode、`lib/` 与中英库有大量金样;[Mid2](stdlib-mid2.md)已完成 | 作为 BoundCall、效应声明、签名与迁移回归的基础;不为本路线重做通用库 |
|
||||
| Web / 浏览器 | [Web](ext-web.md) W 波与 [WASM](browser-wasm.md) C 波已落地 | 作为 view/静态投影和宿主能力消费者;知识引用不能在静态导出或浏览器预览中偷跑服务端 effect |
|
||||
| Agent / LLM / 沙盒 | [Agent](ext-agent-optimize.md) Wave B 有局部待项;[优化 O 波](ext-ai-optimize.md)已完成 | 工作簿、OKF、沙盒可复用;S2 接本地 bound handle,S4/S5 增强政策;不另建平行知识身份 |
|
||||
| 外联 / Notebook | [外联 F4](foreign-v2.md)和[会话 D4–D5](notebook-session.md)尚有余项 | 外部内核、持久状态和重放需显式建模为 effect/环境依赖;不能把会话结果当纯制品缓存 |
|
||||
| 领域扩展 | [Quantum Q8c](ext-quantum.md)等剩余项属领域体验;[宿主 H4b](ext-hosting.md)为可选 HTTP 能力 | 不阻塞语义主链;但扩展元数据与插件调用必须参加 G06/G10 的 effect 和兼容性门禁 |
|
||||
|
||||
这些路线各有自己的完成定义。本文不将其余项全部吸收到下一版核心范围;只规定它们与统一语义层交互时必须遵守的契约。
|
||||
|
||||
## 4. 语义一致性缺口:消除错误的语义真相
|
||||
|
||||
### G01 两份图与同名输出路径
|
||||
|
||||
**证据。** [`src/knowledge/compile.rs`](../../src/knowledge/compile.rs)先调用 `compile_project` 并写语义投影,随后重新用 `extract_graph` 从源码提取旧 EKC 图。后者的 `write_projections` 再写 `graph/edges.json`;[`src/knowledge/graph.rs`](../../src/knowledge/graph.rs)的调用边按短名选首个匹配符号。由此“语义边”和“检索边”既不是同一事实,也会争用同一个文件名。
|
||||
|
||||
**规划缺口。** 详细规范要求 EKC 是 `SemanticProject` 投影,却未给出**并行迁移期间的文件命名、读取优先级、冲突检测与废弃时点**。只新增语义图不解决消费者继续读旧边的问题。
|
||||
|
||||
**决策与交付。** 明确一个权威 `SemanticProject`;图边按稳定 `SymbolId/ArtifactId` 生成。旧检索索引仅含查询优化数据,不再自行确定 calls/uses。若迁移期保留旧 JSON,放入独立版本化路径并声明其非执行依据。`knowledge compile`、`view`、`check`、`impact`、MLSP、Agent 对同一引用应报告同一目标及诊断。
|
||||
|
||||
**验收。** 制造两个模块同名函数并让调用只 import 其中一个;语义图、CLI resolve、Tree/Bytecode trace、view 和 Agent handle 必须指向同一 ID。编译两次输出字节一致;生成物中没有同名文件被后续投影覆盖。
|
||||
|
||||
### G02 绑定没有成为执行的前置条件
|
||||
|
||||
**证据。** [`src/main.rs`](../../src/main.rs)的 `knowledge resolve` 走项目索引;[`src/mlsp.rs`](../../src/mlsp.rs)的 `link.resolve` 通过 artifact 名字/后缀匹配;[`src/interp.rs`](../../src/interp.rs)仍有自身的 `eval_path_call` 和函数查找;[`src/bytecode/`](../../src/bytecode/)也有独立调用实现。`compile_project` 目前只将同模块无接收者调用识别为静态依赖,其余记为 `external_or_dynamic`。这几条路径存在真实分叉。
|
||||
|
||||
**规划缺口。** 现有规范提出 Bound IR,但缺少**最小可合并的执行切面**以及动态调用的明确边界。不能先宣称图边覆盖执行调用,再等待后端改造。
|
||||
|
||||
**决策与交付。** 先让显式 import、普通函数、关联函数这三个静态路径通过唯一 resolver,生成 `BoundCall { target, signature, source_span, required_effects }`。Tree 和 Bytecode 只消费 BoundCall;确需动态解析的路径使用独立 `DynamicCall`,在图/诊断中显式标记,不能假装已绑定。`run/check/knowledge resolve/MLSP` 使用相同 import 范围和歧义规则。
|
||||
|
||||
**验收。** 同名跨模块、重命名、缺 import、实例方法误用、动态目标分别有稳定诊断;任一静态 trace 中执行的 SymbolId 与编译图边一致。错误不得因后端不同而变成另一目标。
|
||||
|
||||
### G03 `[]()` 只有标签,还没有完整知识步骤
|
||||
|
||||
**证据。** [`src/ast.rs`](../../src/ast.rs)保留 `Expr::Occluded`;[`src/parse/prose.rs`](../../src/parse/prose.rs)扫描粗体、斜体和反引号,但没有完整 Markdown 正文链接 AST。[`src/sema/ir.rs`](../../src/sema/ir.rs)当前 KnowledgeRef/KnowledgeStep 主要由调用点提取,步骤 `kind` 为 `call`;取元、推导、遍历、正文、导航尚未同等进入图。目标示例列出的 observe/derive/iterate/narrate 不能由现有投影证明。
|
||||
|
||||
**规划缺口。** 缺少正文 inline 的位置、逃逸、代码围栏、标准 URL 与知识引用的**一套无副作用解析协议**,以及 `[]()` 在表达式、循环和正文的统一节点身份。
|
||||
|
||||
**决策与交付。** 保留现有 payload 表达式解析器;为上下文建立稳定 SyntaxNodeId 与 `Named<T>`/等价 lowering。明确 `NavigationLink`、`NarrativeRef`、`ExecutableNamed`、`LoopNamed` 的互斥分类及 span。正文引用只解析和验证;主动纯预览可使用项目/会话既有政策,不要求逐链接确认;外部 effect 仍需明确执行上下文与宿主能力许可。各类型步骤的输入、输出、依赖和角色由编译器推导;标签是阅读、检索和局部编辑的上下文,但不参与授权和寻址。作者不手工填写步骤 ID 或生成图。
|
||||
|
||||
**验收。** 一份文档同时含 URL、锚点、带空格/嵌套括号载荷、围栏、行内代码、取元、计算、循环、正文引用;生成的 ref/step 数、分类和跨度稳定,正文 `run/check/view static` 不发生外部 effect。
|
||||
|
||||
### G04 生成物写入不是整代快照
|
||||
|
||||
**证据。** [`src/sema/ir.rs`](../../src/sema/ir.rs)将语义文件写到临时目录后,逐个删除并移动 `project.json`、`symbols.json`、`refs.json`、`artifacts/`、`graph/`、`verification/`;[`src/knowledge/compile.rs`](../../src/knowledge/compile.rs)再写旧投影。这不是消费者可原子读取的整代快照,构建中途可能看到混合版本。
|
||||
|
||||
**规划缺口。** 规范写了“原子写入”,未规定**整代提交、读者快照、失败恢复和旧版本保留**。
|
||||
|
||||
**决策与交付。** 使用带 schema/build-id 的代目录与原子指针或等价快照机制;所有消费者一次选择一代,读完整个结果后才释放。生成物可删除重建;崩溃后只能看到旧完整代或新完整代。
|
||||
|
||||
**验收。** 在写每个投影文件时注入故障并并发读取;不得出现 `project` 属于 A 代而 `edges` 属于 B 代,失败后旧代仍可读。
|
||||
|
||||
## 5. 可信执行缺口:按场景落实验证门槛
|
||||
|
||||
### G05 来源、适用性和验证状态仍是占位投影
|
||||
|
||||
**证据。** [`src/sema/ir.rs`](../../src/sema/ir.rs)给函数固定 `"no explicit applicability constraints"`,`SourceRef` 主要指向定义它的源码,`evidence` 摘要来自契约文本;验证记录固定 `contract=unchecked`、`tests=unknown`、`evidence=unverified`、`domain=unreviewed`。现有 [`src/knowledge/lifecycle.rs`](../../src/knowledge/lifecycle.rs)的 `verify` 是另一套 EKC 检查。因此“有 SourceRef”并不表示政策原文被关联或核验。
|
||||
|
||||
**规划缺口。** 规范列出字段和七层状态,但没有定义**作者如何写来源/适用性、如何证明测试确实覆盖此制品、谁可将 domain 从 unreviewed 提升、何时失效**。不确定现实真假的结论必须保留 unknown。
|
||||
|
||||
**决策与交付。** 用现有 Markdown/frontmatter/契约表表达可选的 `source`、有效区间、地域/单位/输入前提,形成可解析 schema;把编译、静态检查、实跑测试、来源内容、人工领域审阅作为独立证据事件。定义状态迁移、撤销和失效规则;生成物记录证据文件/测试运行摘要,不允许 CLI/Agent 自报 `verified`。
|
||||
|
||||
**验收。** 政策原文改变而函数未改时 evidence 状态下降;来源删除、过期、输入不满足分别给不同诊断;无来源的纯工具函数可运行,但不能被宣称为已证实业务事实。
|
||||
|
||||
### G06 效应识别存在“未知即纯”的漏洞
|
||||
|
||||
**证据。** [`src/sema/ir.rs`](../../src/sema/ir.rs)的 `effects_for_call` 依据名字前缀猜测 `fs/net/llm` 等,其余返回 `pure`;`direct_effects` 只汇总直接调用。未知插件、别名、包装函数和传递调用可能因此被标作纯。[`src/mlsp.rs`](../../src/mlsp.rs)以该集合做 `artifact.execute` 的效应拒绝。
|
||||
|
||||
**规划缺口。** 详细规范定义 EffectSet,却没有给出**跨宿主函数、标准库、扩展 ABI、动态调用的权威效应注册表和未知效应策略**。
|
||||
|
||||
**决策与交付。** 内核/标准库/插件元数据声明 effects,绑定后做调用图闭包;动态或缺元数据为 `unknown`,需要显式能力授予,不得降为 pure。`view` 静态、`verify pure`、Agent ready 门槛使用同一集合。实际宿主能力限制仍在执行时强制。
|
||||
|
||||
**验收。** 用自定义名字包住写文件/网络/插件调用;静态 effect 必须传递到入口,默认 pure 策略拒绝执行。循环调用图收敛;未知动态调用拒绝 pure 证明。
|
||||
|
||||
### G07 绑定句柄未表达可复用的授权与适用性
|
||||
|
||||
**证据。** [`src/mlsp.rs`](../../src/mlsp.rs)的 `artifact.bind` 主要检查目标唯一及可选 digest,返回 `artifact_id/digest/project/policy/effects`;它没有要求 ready、契约、适用性或来源状态。`artifact.execute` 重编译比对摘要后按 `display_name` 最后一段调用解释器;调用者提交的 `allow_effects` 会变为宿主能力。现阶段这是实验性执行接口,不是可信复用协议。
|
||||
|
||||
**规划缺口。** 缺少句柄的**发行者、主体、策略、输入约束、能力上限、过期/撤销、并发源变更**模型;也缺少“绑定到精确嵌套函数”在 runtime 的保证。
|
||||
|
||||
**决策与交付。** `bind` 必须验证 SymbolId、签名、输入、已声明适用性、所选验证政策及 effect 上限;政策要求锁时检查 lock。`execute` 复核同一目标与状态并按绑定符号调用。受信本地调用先采用精确绑定引用和摘要复核,不强制搭建签发服务;远程/多租户/不受信调用再加入可信发行者、主体、范围、期限/撤销及防伪机制。客户端参数不是远程授权来源;运行时能力上限始终由宿主强制。响应如实显示政策和未知状态,作者不手写 handle。
|
||||
|
||||
**验收。** 本地先覆盖改参数、改依赖、改效应策略、同名嵌套函数和并发源码变更,不得错调用或越权;严格政策增加来源与锁变更,远程增加伪造/过期/撤销测试。`ready` 必须注明相应 policy 与条件,不是全局永久认证。
|
||||
|
||||
### G08 锁定与影响分析未覆盖知识变化
|
||||
|
||||
**证据。** [`src/sema/lock.rs`](../../src/sema/lock.rs)的 lock 文件虽然写出 interface/implementation/evidence/dependencies,但检查时只比较 `artifact` 摘要。该摘要在 [`src/sema/ir.rs`](../../src/sema/ir.rs)只组合本地 interface/implementation/evidence,没有传递依赖闭包;impact 的边也只来自已识别的本地调用。`knowledge impact` 与语义 impact 是两套命令/图。
|
||||
|
||||
**规划缺口。** 未定义**锁的审阅单位、变更分类、依赖 SCC、证据变更如何传播到正文、升级冲突处理**。
|
||||
|
||||
**决策与交付。** 分别持久化 target/interface/implementation/evidence/closure 摘要,SCC 按确定规则求闭包;strict 对未审阅目标变化失败。impact 用同一图沿 calls/depends_on/supported_by/narrates 反向传播,并说明为何被影响。每次 lock update 留下人可审阅的差异报告。
|
||||
|
||||
**验收。** 本地函数源码不变而下游税率规则变了时 closure 变化;只改描述不混同事实证据变化;来源变更影响客服说明;同一个查询在 CLI、MLSP、view 展示相同受影响 ID 集合。
|
||||
|
||||
## 6. 作者与生态缺口:从早期切片持续验证
|
||||
|
||||
### G09 Agent 仍可能把候选当作可执行知识
|
||||
|
||||
**证据。** [`src/knowledge/reuse.rs`](../../src/knowledge/reuse.rs)、[`plugins/agent/`](../../plugins/agent/)和 `ext/ai/agent.mq.md` 有各自的检索、near-match、工作簿/Skill 固化流程;[`src/mlsp.rs`](../../src/mlsp.rs)另有 artifact handle。现有 [Agent 路线图](ext-agent-optimize.md)主要跟踪工作簿、路由和评测,没有把“发现→精确绑定→验证→调用既有制品→记账”设成强制路径。
|
||||
|
||||
**规划缺口。** 需要明确 **REUSE、ADAPT、CREATE 的进入条件与失败后分流**,以及“避免重复生成”用什么客观指标衡量。
|
||||
|
||||
**决策与交付。** near/soft-match 只能供候选发现;Agent 可以选择并提交精确候选,REUSE 前必须经同一 resolver 和 policy 得到 bound handle。不可用则记录原因,按权限进入 ADAPT/CREATE 或请求补充决策。新草稿可被发现、查看和在本地明确权限下试跑,进入共享自动复用的 ready 集合前按政策验证晋升。先提供含步骤、源码、约束、负知识和验证状态的最小上下文卡,再支持带期望摘要的局部补丁和语义 diff,避免整篇再生成。trace 包含任务、候选、选定 SymbolId、所有相关摘要、输入摘要、policy、实际执行结果及因果依赖;私密输入只存可审计脱敏摘要。
|
||||
|
||||
**验收。** 已有适用能力时实际调用而不新建同义函数;同名未消歧、违反所选政策的过期/未验证候选、权限不足不能自动执行。局部修改不覆盖并发编辑;只改措辞却改变签名必须可见。同样任务与预算下报告重复实现率、错误复用率、发现率、成功率、编辑范围、作者维护成本及失败样本。
|
||||
|
||||
### G10 标准库、扩展库与发布物缺少跨层一致性门禁
|
||||
|
||||
**证据。** [语法测试规程](../next/Marqdo_Executable_Knowledge_Testing.md)覆盖了部分零参路径金样;仓库有大量 `lib/`、`ext/`、`public/` 和示例。`.github/workflows/` 目前只有 release/pages,没有对每次提交运行可执行知识项目夹具的 CI;[目标示例](../../examples/executable-knowledge/README.md)仍不可端到端运行。现有 [lib/ext 路线图](lib-ext-code-as-docs.md)的历史“链接取元”已被 v2.1 默认遮蔽语义取代,`done` 不代表今天的知识引用集成已完成。
|
||||
|
||||
**规划缺口。** 对新语义没有固定的**全仓兼容矩阵、跨平台发布前检查、示例可运行状态标签、回滚规则**。
|
||||
|
||||
**决策与交付。** 分层门禁:parser/sema → Tree/Bytecode → 标准库中英镜像 → 扩展插件装载 → CLI/MLSP/view/静态导出 → 一体化书店/订单示例。每个负例也断言诊断码和副作用计数。每次语义变化先迁移 `lib/ext` 并保持 stdout 与 effect,不通过扩展旧语义来掩盖库的过时写法。CI 在 main 的 PR/push 执行无网络核心套件;需要插件/浏览器的平台测试单列结果;发布前检查包内标准库/扩展与二进制版本匹配。
|
||||
|
||||
**验收。** `cargo test --locked`、gold 单线程、可执行知识项目夹具、CLI JSON schema 和每个平台打包后冒烟通过;文档示例一旦标为可运行,CI 每次执行。没有 `skip` 可以充当核心能力成功证据。
|
||||
|
||||
### G11 文档状态本身缺少统一真相
|
||||
|
||||
**证据。** [`doc/README.md`](../README.md)仍将较早 mapping 称为“现行运行时”,将 [详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md)的 v2.0.0 基线直接当下一阶段实施依据;多个旧路线图的 `done` 和新语义目标相互交错。README 的 v2.1 链接/投影描述可能被误读为完整可执行知识系统已交付。
|
||||
|
||||
**规划缺口。** 缺少文档版本、优先级、状态和实现证据之间的**维护协议**。这会让人和 Agent 从旧文档学回被迁移掉的语义。
|
||||
|
||||
**决策与交付。** 文档顶部标明“目标规范 / 已实现 / 历史兼容 / 被取代”及适用版本;在 `doc/README.md` 提供唯一入口。每项能力把规范段落、代码位置、测试、示例、发布版本关联起来;任何 `done` 必须列出验收命令及结果。旧内容可以保留历史价值,但搜索结果与 Skill/教程默认指向当前规范。制定文档可运行片段的抽取或同步测试,降低“文字与实现分叉”。
|
||||
|
||||
**验收。** 新贡献者只读 README、doc 索引和 Skill 能写出默认语义的运行样例;旧取元写法在当前教学材料不会以默认形式出现;变更语法时 CI 能发现核心规范与构造卡片失配。
|
||||
|
||||
### G12 项目规模、缓存与权限边界尚无可量化 SLO
|
||||
|
||||
**证据。** [`src/project/mod.rs`](../../src/project/mod.rs)每次扫描目录;[`src/sema/ir.rs`](../../src/sema/ir.rs)重新读取并解析项目文件;view 和 MLSP 每次查询也可重新编译。`ProjectRoot::source_files` 特别处理 symlink,但没有明确的路径环/别名去重、资源上限与增量失效协议。当前规范虽列 1k/10k/100k 指标,却没有基线值和可接受阈值。
|
||||
|
||||
**规划缺口。** 项目发现、Symlink、外部 import、生成目录、并发编辑、overlay、缓存一致性和诊断聚合缺少一份可测试的**规模与信任边界规格**。
|
||||
|
||||
**决策与交付。** 先定义约定式项目发现、源文件唯一性、遍历深度/文件大小上限、外部来源许可和增量失效键;单文件与普通项目免额外注册,只有边界无法由约定表达时才要求显式 manifest。再建立 100/1k/10k 项目生成器、冷/热构建计时和内存基线;超过阈值的操作降级为明确诊断,不能静默跳过或返回旧图。MLSP overlay 要么被安全纳入快照,要么显式拒绝且不宣称即时编辑验证。
|
||||
|
||||
**验收。** 扫描顺序与重复路径不改变身份;symlink 环不挂起;修改单文件只失效正确下游;缓存损坏后结果与全量构建一致;目标机器上的时间与内存阈值由基准结果确定后固化进 CI。
|
||||
|
||||
## 7. 依赖顺序与可合并切片
|
||||
|
||||
| 切片 | 前置 | 作者 / AI 可见交付及工程关联 | 可复核的完成证据 |
|
||||
|---|---|---|---|
|
||||
| S0 作者基线与最小语料 | 无 | 六类实际写作/复用任务、当前语法入口、无网络最小门禁(G10/G11) | **已落地**:[tests/executable_knowledge/](../../tests/executable_knowledge/README.md) + `cargo test --test executable_knowledge` |
|
||||
| S1 可读步骤与唯一目标 | S0 | 一份文档的源码/步骤/view 联动;统一 resolver、图与上下文节点(G01/G03) | **已落地**:G01 双图分离;G03 步骤 kind + NarrativeRef(`11-narrative-ref`)+ view 导航;同名 import resolve |
|
||||
| S2 确定执行与早期 AI 复用 | S1 | BoundCall 双后端、最小上下文卡、本地 handle 与现有 Agent 试点(G02/G06/G07/G09 的安全子集) | **已落地**:G02 BoundCall;G06 效应;G07 MLSP bind/execute;eng_reuse `bind_required` |
|
||||
| S3 局部编辑与库可读性 | S1–S2 | 摘要保护的局部补丁、语义 diff、受限纯表达式提取;lib/ext 示例优化(G03/G09/G10/G11) | **已落地**:`artifact.patch` + wording 回滚;`artifact.extract_pure` 纯表达式预览;全库润色仍可选后续 |
|
||||
| S4 业务知识与适用性 | S2;与 S3 可交错 | 书店契约、来源/验证事件、传递效应及政策、负知识(G05/G06/G07/G09) | **已落地**:来源/过期;`13-bookstore` + `examples/executable-knowledge` 发票链 `113`;负知识文档 |
|
||||
| S5 变化传播与严格复用 | S1–S2/S4 | 分类摘要、SCC 闭包、整代快照、impact、严格 handle 复核(G04/G08) | **已落地**:`generations/`+`CURRENT`;锁 `classes`+`closure`;impact 含 narrates;依赖变更推动 closure |
|
||||
| S6 生态验收与发布 | 按发布声明选择 S1–S5 范围 | 全仓兼容矩阵、可运行端到端示例、平台冒烟、作者/AI 指标(G10/G11/G12) | **已落地**:CI;扫描上限;100 单元语料门禁;CLI version/check/knowledge 冒烟 |
|
||||
|
||||
每切片完成时同时更新 [详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md)的实现状态、相关示例及本表;只改文档状态不算完成。S1/S2 先用**无 LLM、无网络、无插件**测试证明确定性协议,随即接现有 Agent 验证实际复用;不等所有治理工程结束。G10/G11 是每片持续门禁,100 单元语料逐片增长;G04 的并发一致性必须在开放并发消费前完成,不能因为列在 S5 就忽略前置风险。
|
||||
|
||||
G12 大规模优化与远程凭据服务按场景安排;远程执行若开放,其授权测试就是硬前置。H4b MCP HTTP、dense embedding、更多语言糖和全面本体推理均不阻塞本地链。任务拆分、模块落点和作者验收详见[指导 §7](../design/document-native-development-guide.md#7-交付切片每片都有文档作者与-ai-能看见的结果)。
|
||||
|
||||
## 8. 横向验收场景与发布条件
|
||||
|
||||
### 8.1 必须固定的四个故事
|
||||
|
||||
1. **两文档事实**:李白出生年份定义、同名冲突、显式 import、正文惰性引用、来源变更、锁拒绝;验证“文档即知识库”。
|
||||
2. **订单定价**:来源政策→税率/折扣→多步计算→正文解释;对每个 `[]()` 生成步骤,修改上游后 `impact` 找全下游;验证“知识可执行”。
|
||||
3. **权限反例**:取一个无明显危险词的用户函数,内部经标准库/插件包装写文件或请求网络;静态未知或效应传递阻止静态预览和纯策略执行;验证默认安全。
|
||||
4. **Agent 已有能力复用**:先存在 ready 制品,再给相似需求;Agent 发现后精确绑定并执行,trace 中有 SymbolId、摘要与结果,仓库不增加同义函数;再加入过期/不适用候选并验证拒绝。
|
||||
|
||||
每个故事覆盖成功与拒绝、Tree/Bytecode(执行适用时)、CLI/MLSP、view、生成图及清缓存重建。旧版本迁移另保存输入、输出、诊断和 effect 的对照;不以“测试通过”掩盖输出语义改变。
|
||||
|
||||
### 8.2 完整可执行知识声明的门槛
|
||||
|
||||
- 100 单元确定性夹具和四个故事在 CI 全绿;正文静态路径外部 effect 计数为零。
|
||||
- 静态调用图与运行 trace 的 target ID 一致;两后端的值、目标、诊断一致;动态调用在结果中明确标记。
|
||||
- 任何模糊/同名、unknown effect、无效适用性、过期 lock、缺失必需证据都不能被自动标为 ready。
|
||||
- Agent 在已有适用能力的任务上实际执行 bound handle,且错误复用率、重复实现率与当前 EKC 基线同时报告;保存数据和失败样本。
|
||||
- `lib/ext/public/examples` 当前语义全仓通过,安装包中的 CLI、标准库、扩展库版本和示例行为一致。
|
||||
- README、教程、Skill 与目标示例明确区分已实现、实验性与设计态;运行示例由 CI 执行。
|
||||
|
||||
这些是完整链的门槛,不要求每次修补版等完整链全部完成。受限本地复用可在 S1/S2 对应正负例和兼容门禁通过后按精确范围发布;严格业务复用增加 S4/S5,远程执行另验授权和隔离,见[分层声明](../design/document-native-development-guide.md#83-发布声明分层)。未完成项不得被笼统“可执行知识已实现”覆盖。目前仍应表述为:**v2.1 已提供文件级新语法、项目索引与语义制品初版;知识验证和 Agent 复用闭环仍在建设。**
|
||||
|
||||
## 9. 明确的非目标
|
||||
|
||||
- 不要求作者为每个函数写一份重复的机器配置;优先从现有 Markdown、契约表和调用解析推导。
|
||||
- 不要求每个取元、循环或纯工具函数都附现实世界证据;只有声称业务事实成立或需要高信任复用时才提高证据门槛。
|
||||
- 不以 LLM 的模糊判断替代符号绑定、类型检查或权限授予;LLM 可选择精确候选、组合、编辑、适配和补测试,编译器校验实际目标与语义,宿主执行权限上限。
|
||||
- 不把 `Type`、`Entity`、`Fact`、`Function` 强行折叠为“类”;兼容现有 `#`/`##` 表面,语义类型在 IR 中清楚区分。
|
||||
- 不把 v2.1 的发布回退为未完成,也不把历史路线图的 `done` 自动推广为本路线的完成。
|
||||
@@ -0,0 +1,250 @@
|
||||
# Marqdo 支撑求道知识沉淀与文档编译的框架缺口
|
||||
|
||||
| 项目 | 基线 |
|
||||
|---|---|
|
||||
| 日期 | 2026-10-24 |
|
||||
| 框架源码 | `b8a4ddd`,Cargo 声明 2.1.1 |
|
||||
| 消费者源码 | 工作区 qdagent,`7f8f19d` |
|
||||
| 文档性质 | 当前源码复核、实测反例和增量建议;不重开已完成的 S0–S6 工作 |
|
||||
| 配套文档 | [求道产品侧缺口](../../../qdagent/doc/gaps/03-knowledge-ingestion-and-sedimentation-2026-10-24.md) |
|
||||
| 设计依据 | [文档原生开发指导](../design/document-native-development-guide.md)、[既有缺口路线](executable-knowledge-gaps-2026-10-23.md) |
|
||||
|
||||
## 1. 核心判断
|
||||
|
||||
Marqdo 已具备让求道采用文档原生知识的主要语言和执行基础。求道尚未接入的能力,不能统称为“框架不支持”。本轮框架侧优先工作是:**让生成知识的语言指导可信,让来源和验证真正约束发布与复用,让新语义制品进入默认 Agent 链路,并提供文档导入所需的通用适配接口。**
|
||||
|
||||
`knowledge` 编译的是已有源码和声明,不是把任意 PDF/DOCX 自动变成正确知识的转换器。上传、抽取、模型建模、语义编译、验证、发布是不同阶段。语义编译器保证程序和依赖的确定性;来源支撑、业务适用性以及领域审阅需要明确证据,不能由成功退出或 LLM 自述推导。
|
||||
|
||||
现有 [2026-10-23 路线](executable-knowledge-gaps-2026-10-23.md)的审计基线较早,之后已有多次实现提交。本文件按当前源码重新划分:已实现、有限子集、仍有断点;不照抄旧文档中的“无正文引用”“未知效应全为 pure”“两图互相覆盖”等历史判断。
|
||||
|
||||
## 2. 当前可直接复用的能力
|
||||
|
||||
| 能力 | 当前证据 | 给求道的边界说明 |
|
||||
|---|---|---|
|
||||
| 默认 Occlusion、`access`、显式遍历 | [核心表面](../design/core-surface.md)、[基线 01–04](../../tests/executable_knowledge/README.md) | 求道须迁移旧取元;无需增加新语法 |
|
||||
| 函数、契约、表格、import、关联函数 | `src/parse/`、[check.rs](../../src/check.rs)、[sema/bind.rs](../../src/sema/bind.rs) | 支持编写可复用知识;不是所有知识都要塞进 main 或建类 |
|
||||
| 项目身份、语义制品、绑定调用 | [project](../../src/project/mod.rs)、[symbols](../../src/sema/symbols.rs)、[IR](../../src/sema/ir.rs) | 已有 SymbolId/ArtifactId、BoundCall/DynamicCall;消费方不另造执行身份 |
|
||||
| 语义图与检索图隔离 | [knowledge/compile.rs](../../src/knowledge/compile.rs) | 权威语义 `graph/` 与检索 `ekc/graph/` 已分开,旧覆盖问题不再作为未实现项 |
|
||||
| 知识步骤、正文引用、view | `collect_named_steps`、NarrativeRef、现有集成测试 | 已含 observe/derive/iterate/call/narrate/navigation 子集;正文引用保持惰性 |
|
||||
| 来源与适用期初版 | [provenance.rs](../../src/sema/provenance.rs) | 已解析 `sources` block list、region、valid_until;sourced 不等于事实已经验证 |
|
||||
| 精确绑定与本地执行 | [mlsp.rs](../../src/mlsp.rs) 的 artifact.find/inspect/bind/execute | 已有摘要及 SymbolId 检查、按源行定位函数、pure 效应门槛;剩余失效问题见 M02/M03 |
|
||||
| 局部修改 | MLSP artifact.patch / artifact.extract_pure | 已有摘要保护、措辞修改守卫、纯片段提取;不应要求模型每次重写整库 |
|
||||
| 整代语义投影、锁、影响 | `write_semantic_projections`、[lock.rs](../../src/sema/lock.rs) | 已有 generations/CURRENT 与多类摘要;覆盖边界及实测失败见 M02/M06 |
|
||||
| Agent 工作簿、技能、沙箱 | [ext/ai/agent](../../ext/ai/agent.mq.md)、[plugins/agent](../../plugins/agent/src/lib.rs) | 可复用 Runtime、CoW、工具协议和诊断修复;求道不另建 Agent loop |
|
||||
| 文件上传/存储 | [storage.mq.md](../../ext/data/storage.mq.md)、[routes_media.go](../../plugins/web/internal/httpx/routes_media.go) | 已有上传路由、类型/大小验证和保存;不等于文档知识转换已存在 |
|
||||
|
||||
本轮离线执行 `cargo test --offline --locked --test executable_knowledge`:**25 passed / 1 failed / 0 ignored**。通过项包括双后端基线、绑定复用、正文引用、来源状态、局部补丁、100 单元和 CLI 冒烟。失败项为 `s5_closure_digest_moves_with_dependency`,单独重跑仍失败。存在大量有效基础,也存在需要先处理的失效缺口。
|
||||
|
||||
## 3. 框架缺口与可验收交付
|
||||
|
||||
### M01 · P1:上传底座尚未形成文档提取与来源映射协议
|
||||
|
||||
**现状。** Web 插件已注册 `web_upload_validate/save/app_upload`,`handleUpload` 接收 multipart 并存储字节。本轮在 `src/`、`plugins/`、`ext/` 的搜索未找到可供求道直接调用的 PDF/DOCX/OCR → 标准化正文 → 页段来源映射的完整链路。不要把 HTTP 文件保存或 `knowledge compile` 命令包装成已有文档转换器。
|
||||
|
||||
**建议。** 在扩展/插件层定义通用 `DocumentSource → ExtractedDocument` 协议,字段至少包括原件 URI/摘要、格式、提取器版本、段落/表格/页码定位、可提取文本、缺失部分、置信度及诊断。先支持 text/Markdown;PDF/DOCX/OCR 可通过独立插件或受控外部提取器接入。格式解析留在扩展,语言核心不引入 OCR 或模型业务。
|
||||
|
||||
知识生成由求道选模式、检索既有知识、调用模型并提供 patch;框架提供 compile/diagnostics/sandbox,而不是默默替作者推断领域真相。转换的外部程序必须服从宿主权限和资源边界。
|
||||
|
||||
**现有上传实现需补强的具体点。** `handleUpload` 先 `ParseMultipartForm(32 << 20)`、`io.ReadAll`,再按完整字节数做 Validate;32 MiB 是 multipart 内存阈值,不是请求总量硬限。应在解析/读取前限制请求与文件字节数,采用受限流式读取,并测试清理临时文件与失败路径。已有类型/路径验证应复用,不能将此描述为“没有任何上传校验”。
|
||||
|
||||
**验收。** 对文本、表格、跨页、坏编码、空文档、密码 PDF、扫描件、超限文件分别返回成功/部分/失败;每条抽取事实都可定位回原件;上传 `.mq.md` 不自动执行;相同文件及提取器版本输出可比较,重试不产生不同身份。
|
||||
|
||||
**责任。** Marqdo 提供提取与存储适配规范;求道负责上传界面、任务进度、知识库选择、发布政策与业务模型。
|
||||
|
||||
### M02 · P0:来源与依赖变化未完整进入摘要、锁与执行失效
|
||||
|
||||
**来源反例(实测)。** 2.1.1 下给函数声明 `sources: source.md#missing-anchor`,实际文件存在但锚点不存在,仍得 `evidence=sourced`;business bind 成功。修改 source.md 后 SourceRef.content_digest 变化,artifact.hashes.artifact 不变;旧 handle 继续执行。删除 source.md 后,旧 business handle 仍执行并返回原值。
|
||||
|
||||
**源码原因与边界。** [provenance.resolve_provenance](../../src/sema/provenance.rs)将 `#anchor` 存为 locator,但没有校验锚点;对来源文件计算哈希后,[IR artifact_for_function](../../src/sema/ir.rs)仍用 `digest(contract_summary(&contract))` 构造 evidence 摘要,没有包含来源内容和适用条件。`artifact.execute` 重编译并比较自身 artifact 摘要,但不重验 handle.policy 对应的证据状态。来源改变、来源删除、时间到期可以不改变函数实现,因此自身摘要检查不足。
|
||||
|
||||
**依赖反例(实测)。** [现有集成测试](../../tests/executable_knowledge.rs) `s5_closure_digest_moves_with_dependency` 中 `main` 调用 `d.rate()`,仅将被依赖函数返回值从 0.13 改为 0.14;测试读取到的 main closure 摘要未变。已重复验证。这里只确认该夹具失败:需要区分符号/模块同名选择、依赖边和闭包算法各自责任,不据此直接断言所有函数闭包都失效。
|
||||
|
||||
**其他源码缺口。** bind/execute 传递的是自身 `hashes.artifact`,没有把已有 `hashes.closure` 纳入句柄复核。`lock.impact` 从制品定义文件和依赖/正文边找反向影响,未把 `provenance` 的外部 corpus URI 纳入来源反向边。
|
||||
|
||||
**交付。**
|
||||
|
||||
1. evidence 摘要包含规范化来源 URI、定位、内容摘要、提取器/验证版本及适用性;锚点不存在返回不同于文件缺失的诊断。读取错误不能以空文件摘要冒充成功。
|
||||
2. 修复上述闭包失败,并明确 type/module/function 的身份与测试选择。依赖递归/环用确定的 SCC 或等价闭包摘要规则;无关文件增删不应导致任意漂移。
|
||||
3. 句柄记录并复核所需的依赖闭包、证据、政策及有效期;execute 重验 business/pure 等政策,不只相信 bind 时的状态。文件变化与时间变化都能使句柄失效。
|
||||
4. 将 corpus → artifact → caller/answer 的影响关系纳入同一编译结果,来源改动能查询受影响对象;报告接口、实现、证据、依赖分别发生了什么变化。
|
||||
|
||||
**验收。** 仅修改来源、删来源、改锚点、时间越过有效期、改直接/传递依赖都拒绝不再有效的句柄;原有测试恢复通过。不得为让测试变绿删除断言或改成只检查命令成功。普通无业务来源要求的纯函数仍可按相应政策执行。
|
||||
|
||||
### M03 · P0:验证 API 尚不是可发布知识的完整判据
|
||||
|
||||
**现状。** `VerificationRecord` 已分 parse/resolve/contract/execution/tests/evidence/domain,但 `semantic_artifact` 的默认值仍包括 `contract=unchecked`、`execution=not_run`、`tests=unknown`、`domain=unreviewed`、`lifecycle=draft`。MLSP `artifact.verify` 主要返回编译记录并附 policy,没有实际执行对应测试或回写证据。不能把它宣传为已经验证业务事实。
|
||||
|
||||
`artifact.bind(policy=business)` 接受 sourced/verified,尚未检查 region 与本次输入、单位、前提、输入域等;`provenance` 用日期字符串比较有效期,没有完整日期格式校验。当前“business”至多表示来源已关联且未被判过期,不等于适用性或事实成立。unknown policy 字符串也未通过枚举校验形成明确拒绝。
|
||||
|
||||
`knowledge verify` 检查的是旧 EKC 结构一致性;`conflicts/stale` 是报告命令,`duplicate` 的默认行为也是报告。它们都有价值,但消费方不能从退出码 0 得出“候选可晋升”。求道正存在这种误用。
|
||||
|
||||
**交付。** 给调用方稳定的验证结果结构:验证对象/摘要、所选政策、每个检查的 passed/failed/unknown/skipped、证据记录、阻断原因、是否允许绑定或发布。测试运行器生成执行/断言事实;来源存在、引文一致、计算正确、领域结论分别报告。政策和结果字段应有版本化 schema,未知政策拒绝。
|
||||
|
||||
保留报告命令兼容性,提供清晰的机器裁决入口或明确 fail 选项;不要悄悄改变已有 CLI 的退出语义。发布政策可渐进:纯工具先证明程序行为,业务事实额外要求来源与适用条件,领域审阅按场景要求;不为所有函数强加人工审批。
|
||||
|
||||
**验收。** `sandbox failed`、模型声称“passed”、空证据、无效日期、地区不符、输入越界、测试未运行均不能被标成对应级别的 verified;缺证据与执行失败不同;实际通过的记录绑定内容摘要,修改后自动失效。
|
||||
|
||||
### M04 · P1:新语义复用协议尚未进入默认 Agent 学习与执行闭环
|
||||
|
||||
**现状。** [plugins/agent/eng.rs](../../plugins/agent/src/eng.rs) 的 `eng_reuse`、[knowledge/reuse.rs](../../src/knowledge/reuse.rs) 已输出 `discovery_only`、`bind_required`、`bind_targets`。这是真实进展,不再列为完全缺失。
|
||||
|
||||
但 [ext/ai/agent.mq.md](../../ext/ai/agent.mq.md) 的默认 `run` 仍以 route/resolve、agent-kb workbook、plan 为主;EFI 分支主要检查 create_allowed,未在该编排中找到 artifact.bind/execute 的完整消费。`run verify=True` 依据 `status==ok` 拼接 `verification="status: passed"`;随后可 maybe_learn/promote,并非调用 M03 所需的验证器。求道 BUILD 开启 `learn=True/promote=True` 后才做自己的 docgate,因此两层发布时序有冲突风险。
|
||||
|
||||
`build_step_context` 的 CALL 白名单只有名字协议;parent CALL 支持 ARGS 的能力与 step 工具路径不同。求道若要调用精确制品与输入,需要正式适配,不能在提示词中虚构已可调用的工具。
|
||||
|
||||
**交付。** 提供现有 Runtime 内的适配步骤:discover → inspect → bind → execute → record。执行工具使用宿主选择的 project/KB/policy,LLM 只提交候选及输入。REUSE 只有在绑定执行成功且有 trace 后计为实际复用;ADAPT 用现有 patch/extract 形成可审阅修改;CREATE 保留检索范围与原因。
|
||||
|
||||
提供产品可接入的发布前验证钩子或 candidate-only 模式:产品门禁失败时框架也不能先把技能标为可复用。知识文件、技能、episode 可以保留不同存储职责,但都关联同一版本化 artifact/evidence;不要新增另一套知识语义。
|
||||
|
||||
**验收。** 用真实 `agent.run` 编排和确定性假模型/工具响应验证,而不只直调 MLSP:已有知识被调用、无同义实现新增;错误来源/错输入被拒绝;验证失败无 promote;trace 对应符号、摘要、输入与输出。不得将 n-gram 高分或字符串 REUSE 当作成功。
|
||||
|
||||
### M05 · P0/P1:提供给模型的语言 Skill 与当前语法冲突
|
||||
|
||||
**源码事实。** [skills/marqdo/SKILL.md](../../skills/marqdo/SKILL.md) 前部 Hard rules 2/4 和 v0.2 表仍写“粗体是返回”、斜体是语句,旧 `[key](map)` 取元也仍在前部。后面虽有 v2.1 默认 Occlusion 段落,不能消除同一提示包的冲突。
|
||||
|
||||
[host/agent_rt.rs](../../src/host/agent_rt.rs) 的 `load_marqdo_skill/read_skill_pack` 加载 Skill/reference/examples;[agent.skill_brief](../../ext/ai/agent.mq.md) 默认只取前 1200 字符/80 行,deep 预算 4000 字符。因此旧规则可能优先进入窗口,新的说明反而被截掉。实际读取哪个包受 MARQDO_SKILL 和搜索路径影响,部署须记录真实来源,不能假定总在读工作区文件。
|
||||
|
||||
**交付。** 从现行核心表面、解析器与金样例维护一份版本化语法卡片,最前面即当前含义:`**…**` 代码、`*…*` 返回、`[描述](payload)`、`access`、显式 import/遍历、正文惰性。旧语法迁入独立迁移卡片,不继续与默认规则并列。模型可按需取契约、来源、表格、类/函数等卡片;语言核心规则和权限边界不可被预算截掉。
|
||||
|
||||
使用已有 MLSP syntax/schema/validate/repair 能力,但明确 validate 的范围:当前主要是契约/标签检查,不等于完整事实、所有变量/调用和业务验证。诊断附 span、相关源码、目标语法版本及建议修复范围;优先局部 patch,避免整篇重生成。
|
||||
|
||||
**验收。** `dump_step_context` 和 plan 上下文的脱敏快照中没有冲突规则;基础/deep 预算均保留必需语法。卡片中可执行示例在当前 CLI 双后端通过;同一批含旧取元、错误返回、表格几何误用、缺 import 的负例给明确诊断。记录实际 skill path/hash、CLI/Ext/schema/prompt 版本。
|
||||
|
||||
### M06 · P1:整代发布已存在,但扫描范围与消费者一致性仍需约束
|
||||
|
||||
**现状。** [IR](../../src/sema/ir.rs)已将语义投影写 generations,再原子替换 CURRENT;遵循 `semantic_generation_dir` 的读者可获得完整语义代。但随后还写根目录兼容镜像,[knowledge.compile](../../src/knowledge/compile.rs)另写 engineering.yaml / ekc 等检索投影;不能把语义代原子性推广为“所有消费者、所有索引都属于同一代”。并发构建共用 CURRENT.tmp、.tmp-sema-publish,需明确串行化或独占发布规则。
|
||||
|
||||
[project 扫描器](../../src/project/mod.rs)与 [EKC extract](../../src/knowledge/extract.rs)的排除集不同,后者遍历子目录时排除 data/docker,前者不排除 data。求道有多个原文、候选和运行目录,导入 `.mq.md` 后可能在不同消费者里出现不同可见性。已有 .marqdo 排除、深度/文件大小/数量限制值得保留,但尚不能替代显式源范围和信任范围。
|
||||
|
||||
**交付。** 项目配置声明 authored sources、corpus、draft/generated 的范围;语义、EKC、Agent/view 使用同一选择规则。发布 manifest 关联语义代、检索代、源码/证据摘要和 schema;消费者拒绝混代并能回退到旧完整代。多库使用独立项目身份,索引聚合不扩大执行文件范围。
|
||||
|
||||
**验收。** 暂存上传代码不会进入活动能力;data/ 路径不会因消费者不同而忽隐忽现;并发编译/失败注入期间读者只看到一套可用版本;旧索引可重建,源文档不受损。迁移期明确旧 graph 路径兼容期限。
|
||||
|
||||
### M07 · P1:来源粒度和知识类型不足以覆盖文档沉淀
|
||||
|
||||
**现状。** 来源主要来自模块 frontmatter,容易将一整篇资料附给该文件所有函数;SourceRef 有 uri/locator/digest,但缺少标准的“具体断言 → 原文片段/表格单元/页码 → 提取版本”关联。普通 EKC `KnowledgeKind` 有 Fact/Decision/Constraint 等,而语义 artifact 主要是 module/type/function;旧 `.md` Fact 的存在不等于可调用事实函数。作者写来源 inline list 时当前解析器会忽略,未知字段也缺少强诊断。
|
||||
|
||||
**交付。** 用现有 Markdown/frontmatter/契约表定义轻量知识约定,而不增加求道专属语法:可选的函数/断言级来源、适用范围、单位、负例、冲突/取代关系。明确知识内容类型与运行时类型的区别;事实可以是返回值、规则可以是函数、决策可以是声明性文档,类只用于真实模式复用。
|
||||
|
||||
由编译器生成身份、图边和 SourceRef,作者只维护实际内容与必要来源。为 plain note → draft fact/rule → verified callable 提供金样例和机器可读投影,不能仅把文本包在一个 print 中。字段不支持或位置错误时给诊断,避免“写了来源,编译却无声忽略”。
|
||||
|
||||
**验收。** 一文多来源、一事实多来源、原文冲突、缺失字段、单位不同、来源片段变化、声明性决策和可执行规则都能区分;无依据的信息保持 unknown;不要求每个纯工具函数带业务来源。
|
||||
|
||||
### M08 · P1/P2:生成式知识验收与真实复用评估不足
|
||||
|
||||
**现状。** 既有 26 项集成测试和 100 单元是有效基础,但不覆盖资料导入/模型沉淀全过程。现有 `eng_reuse` 协议测试证明 JSON 字段,不证明默认 Agent 调用了句柄;MLSP 直调测试证明本地纯函数复用,不证明模型按正确来源建库。测试 README、旧路线和示例中存在历史状态残留;此次 closure 测试失败也说明“done”必须和当前门禁分开。
|
||||
|
||||
**交付。** 两层评测:确定性 fixtures 验证协议和失败边界,固定任务集验证模型沉淀质量。每个样本保存原文、期望知识、可执行断言、来源定位、错误复用反例、实际上下文及版本;明确事实正确率与程序运行率不同。性能指标包括编译/检索延迟、上下文规模、修复轮次、真实 token、知识重复率和误晋升率。
|
||||
|
||||
**验收。** 用同一资料完成“首次建库→再次调用→原文更新→旧知识失效→修复发布”的完整故事;包括断网/缺插件/截断/OCR 错误/资料内指令/冲突来源。只统计有目标与结果 trace 的执行复用,不按 REUSE 决策估算零推理成本。核心发布门禁不能以 skip 计成功。
|
||||
|
||||
## 4. 建议的跨库接口边界
|
||||
|
||||
下表是待实现或待稳定的契约,不暗示现在已有同名函数/CLI。
|
||||
|
||||
| 契约 | Marqdo 保证 | 求道负责 |
|
||||
|---|---|---|
|
||||
| Document extraction | 原件摘要、标准正文、定位、诊断;扩展层实现格式转换 | 上传、任务、库范围、用户预览 |
|
||||
| Knowledge draft | 当前语法卡片、检查/修复、可调用元数据 | 事实/规则选择、提示词、冲突与去重决策 |
|
||||
| Verification report | 每个检查的真实状态、证据/制品摘要、阻断原因 | 项目政策、需要人工处理的歧义、发布编排 |
|
||||
| Artifact handle | 唯一身份、依赖/证据失效、宿主能力边界 | 提交候选与输入、记录实际使用和解释 |
|
||||
| Publish manifest | 一致的语义/索引代及可回滚读取 | 知识源码发布、迁移、当前活动版本 |
|
||||
| Trace | 实际目标、输入、输出、版本与执行结果 | 回答来源展示、知识质量指标、运行记录关联 |
|
||||
|
||||
现有 `artifact.*`、sandbox、knowledge 等接口优先扩展;新增 façade 必须复用它们。工具协议、输出 schema 与提示词版本一起管理。首次本地试点可以使用固定目录与已授权项目政策;远程多租户的身份、凭据和隔离单独验收,不能把本地 MLSP 直接当公网授权协议。
|
||||
|
||||
## 5. 实施顺序
|
||||
|
||||
| 阶段 | 工作 | 完成条件 |
|
||||
|---|---|---|
|
||||
| F0 正确性基线 | M02/M03/M05:闭包反例、证据失效、验证状态、语法卡片 | 现有测试全过;来源变更/删除拒绝旧 business handle;提示词不含冲突语法 |
|
||||
| F1 最小生产消费链 | M04 + 求道文本导入:candidate-only、验证钩子、绑定执行 trace | 同一资料第一次生成,第二次实际复用,门禁失败不学习发布 |
|
||||
| F2 来源与更新 | M06/M07:统一源范围、来源定位/影响、发布 manifest | 改原文能找全受影响对象,发布失败旧版本仍可用 |
|
||||
| F3 格式覆盖与评测 | M01/M08:PDF/DOCX/OCR 适配、资源限制、模型对照 | 文档可追溯,不完整提取显式报告,效果和失败分布有数据 |
|
||||
|
||||
框架不承担求道的登录界面、笔记列表、项目业务政策;求道不复制框架的 parser、binder、effect、lock 或执行器。无需为了这条主链先完成万能本体、全格式解析或远程授权平台。
|
||||
|
||||
## 6. 本轮可复核实验
|
||||
|
||||
### 6.1 当前门禁与失败复核
|
||||
|
||||
在框架仓库执行:
|
||||
|
||||
```bash
|
||||
cargo test --offline --locked --test executable_knowledge
|
||||
cargo test --offline --locked --test executable_knowledge \
|
||||
s5_closure_digest_moves_with_dependency -- --exact
|
||||
```
|
||||
|
||||
第一条结果为 25 passed、1 failed;第二条仍在 `tests/executable_knowledge.rs:586` 报 `caller closure digest must move with dependency`。两次比较结果相同。这是既有测试的执行结果;本次未修改其实现或断言。
|
||||
|
||||
审计开始时 `target/release/marqdo --version` 与旧 debug 都是 2.1.0;上述构建生成的 debug 是 2.1.1。后续最小实验使用新 debug,不能把旧 release 输出充当当前源码能力证据。
|
||||
|
||||
### 6.2 来源变化与旧句柄复现
|
||||
|
||||
以下程序仅在新建临时目录写测试文件,通过本地 MLSP 调用,不联网、不调用模型。构建当前 debug 后在仓库根运行:
|
||||
|
||||
```python
|
||||
import json
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
binary = str(Path("target/debug/marqdo").resolve())
|
||||
root = Path(tempfile.mkdtemp(prefix="mq-source-audit-"))
|
||||
|
||||
def rpc(method, **params):
|
||||
req = json.dumps({"method": method, "params": params}) + "\n"
|
||||
proc = subprocess.run([binary, "mlsp"], input=req, text=True,
|
||||
capture_output=True, check=True)
|
||||
return json.loads(proc.stdout)
|
||||
|
||||
(root / "source.md").write_text("# rate\n\nrate = 0.13\n")
|
||||
(root / "facts.mq.md").write_text(
|
||||
"---\nsources:\n - source.md#missing-anchor\n"
|
||||
"applicability:\n region: CN\n valid_until: 2099-12-31\n"
|
||||
"---\n\n## rate\n\n*0.13*\n"
|
||||
)
|
||||
first = rpc("artifact.inspect", project=str(root), artifact_id="rate")["result"]
|
||||
handle = rpc("artifact.bind", project=str(root), artifact_id="rate",
|
||||
policy="business")["result"]["handle"]
|
||||
(root / "source.md").write_text("# rate\n\nrate = 0.14\n")
|
||||
second = rpc("artifact.inspect", project=str(root), artifact_id="rate")["result"]
|
||||
print("evidence:", first["verification"]["evidence"])
|
||||
print("source changed:", first["provenance"][-1]["content_digest"] !=
|
||||
second["provenance"][-1]["content_digest"])
|
||||
print("artifact changed:", first["hashes"]["artifact"] !=
|
||||
second["hashes"]["artifact"])
|
||||
print("after edit:", rpc("artifact.execute", handle=handle, inputs={}))
|
||||
(root / "source.md").unlink()
|
||||
print("after removal:", rpc("artifact.execute", handle=handle, inputs={}))
|
||||
```
|
||||
|
||||
本轮观察:`evidence=sourced`、`source changed=True`、`artifact changed=False`;两次 execute 均 `ok=true`、result 为 `0.13`。正确修复后,缺锚点应先阻断相应来源政策;用有效锚点重新绑定后,改/删来源也应拒绝旧句柄或要求重验。不能只修复示例中的缺锚点而遗漏来源内容失效。
|
||||
|
||||
### 6.3 求道旧取元风险
|
||||
|
||||
以下代码在默认 2.1.1 中,第一行输出整个 map,第二行输出 False:
|
||||
|
||||
```markdown
|
||||
# main
|
||||
|
||||
`m` =
|
||||
|
||||
| ok | status |
|
||||
|---|---|
|
||||
| False | denied |
|
||||
|
||||
**old = [ok](m)**
|
||||
**new = access(m, "ok")**
|
||||
> print text=`old`
|
||||
> print text=`new`
|
||||
```
|
||||
|
||||
这是默认语义变化,不是框架需要回退的新语法错误。求道需迁移;框架应提供一致的迁移指导、诊断和版本信息,而不是在 Skill 中继续教授冲突的旧默认。
|
||||
|
||||
## 7. 审计限制与后续文档同步
|
||||
|
||||
本次验证是源码审查、仓库固定测试和临时输入实验;没有执行真实 LLM 的建库质量评估,没有测试线上 Web 上传或真实 PDF/OCR。M01 的“缺少转换链”限定在所审查的主代码区域,不否定外部工具可以接入。M02 的闭包根因仍需按身份、依赖图、摘要计算分层定位。
|
||||
|
||||
完成后应同步当前 Skill、README、测试 README 和路线进度:分别标记语言能力、协议子集、产品已接入、当前测试结果。优先把失败样例转成稳定回归门禁,再扩大“可验证知识”“业务复用”“上传编译”的能力声明。此次文档提交不表示缺口已修复。
|
||||
@@ -1,35 +1,44 @@
|
||||
# 可执行知识系统示例:订单含税总额
|
||||
|
||||
这是 [可执行知识系统设计](../../doc/design/executable-knowledge-system.md) 的目标示例。
|
||||
它刻意展示 `[]()` 不只包“知识函数”:同一套语法同时命名取元、循环、推导、跨文件调用和正文结论。
|
||||
这是 [可执行知识系统设计](../../doc/design/executable-knowledge-system.md) 的目标示例,
|
||||
已按当前 occlusion 语法落地为**可运行**订单链(Tree / Bytecode)。
|
||||
|
||||
| 文件 | 图中的职责 | 关键步骤 |
|
||||
|---|---|---|
|
||||
| `corpus/tax-policy-2026.md` | E:证据 | 税率原文 |
|
||||
| `schema/order.mq.md` | C:概念 | 订单/订单行字段与单位 |
|
||||
| `knowledge/tax-2026.mq.md` | K:原子知识 | `[标准税率](0.13)` 及来源 |
|
||||
| `knowledge/tax-2026.mq.md` | K:原子知识 | `标准税率.查询` + `sources:` |
|
||||
| `knowledge/negative-refund.mq.md` | 负知识 | 折扣不适用于退款 |
|
||||
| `capabilities/order-pricing.mq.md` | D:推导知识 | 取元、调用、循环、汇总 |
|
||||
| `docs/invoice-answer.mq.md` | N:叙述知识 | 用同一个计算给人可读的回答 |
|
||||
| `docs/invoice-answer.mq.md` | N:叙述知识 | 含税结论 + NarrativeRef |
|
||||
|
||||
## 运行
|
||||
|
||||
```bash
|
||||
marqdo run examples/executable-knowledge/docs/invoice-answer.mq.md --backend tree
|
||||
marqdo run examples/executable-knowledge/docs/invoice-answer.mq.md --backend bytecode
|
||||
# → 113
|
||||
marqdo knowledge examples/executable-knowledge -o /tmp/ek
|
||||
marqdo knowledge impact "标准税率.查询" examples/executable-knowledge --json
|
||||
```
|
||||
|
||||
门禁:`cargo test --test executable_knowledge s4_examples_bookstore_chain_runs_both_backends --locked`
|
||||
|
||||
## 读法
|
||||
|
||||
`[]()` 的每一次出现都是 `Named<T>`:
|
||||
`[]()` 的每一次出现都是命名步骤(observe / derive / iterate / call / narrate):
|
||||
|
||||
```markdown
|
||||
[订单行金额](at(行, "金额")) # observe
|
||||
[本行含税价](订单行金额 * (1 + 税率)) # derive
|
||||
[逐个订单行](遍历 订单.行 为 行) # iterate
|
||||
[含税总价](pricing.订单定价.含税总额(订单)) # reuse / narrate
|
||||
[订单行金额](access(行, "金额")) # observe
|
||||
[本行含税价](金额 * (1 + 税率)) # derive
|
||||
[逐个订单行](遍历 行列表 为 行) # iterate
|
||||
这张订单的含税总额是[含税总价](总额)。 # narrate
|
||||
```
|
||||
|
||||
它们不是四种语法。`()` 中仍是原有的取元、算术、循环和调用;`[]` 让每一步同时可读、可查、
|
||||
可在图中定位。当前实现尚不能完整运行该目标形式(类函数、统一 `Named<T>`、步骤图仍是设计目标),
|
||||
但它给出了实际项目中应当写成什么样子。
|
||||
## 运行状态
|
||||
|
||||
## 预期闭环
|
||||
|
||||
1. `check` 从每个 `[]()` 提取观测、推导、遍历和调用依赖;
|
||||
2. `view` 在“含税总额”旁显示所经过的订单行、标准税率和税率原文;
|
||||
3. 税率原文变更时,`impact` 找到原子税率、订单定价、发票回答中的每一个命名步骤;
|
||||
4. `verify` 只重算 pure 步骤,不执行写入或发送等 `act` 步骤;
|
||||
5. agent 通过 catalog 得到“已有订单定价能力”和其证据链,不必从自然语言猜测。
|
||||
| 子集 | 状态 | 证据 |
|
||||
|---|---|---|
|
||||
| 发票含税链双后端 `113` | **可运行** | 上表命令 / EK 门禁 |
|
||||
| 来源 `sourced` + impact 下游 | **可运行** | knowledge 投影 + impact |
|
||||
| 负知识文档 | **已收录** | `knowledge/negative-refund.mq.md`(发现层;非执行目标) |
|
||||
@@ -1,27 +1,32 @@
|
||||
---
|
||||
markup: occlusion
|
||||
id: capability.order-pricing
|
||||
type: Capability
|
||||
import:
|
||||
tax: ../knowledge/tax-2026.mq.md
|
||||
order: ../schema/order.mq.md
|
||||
---
|
||||
|
||||
# 订单定价
|
||||
|
||||
## 含税价
|
||||
+ `行`
|
||||
按订单行取未税金额与品类,套用税率后汇总含税总额。
|
||||
|
||||
**金额 = [订单行金额](at(行, "金额"))**
|
||||
**品类 = [订单行品类](at(行, "品类"))**
|
||||
**税率 = [适用税率](tax.标准税率.查询(品类))**
|
||||
## 含税价
|
||||
|
||||
`行`
|
||||
|
||||
**金额 = [订单行金额](access(行, "金额"))**
|
||||
**品类 = [订单行品类](access(行, "品类"))**
|
||||
**税率 = > tax.标准税率.查询 品类=`品类`**
|
||||
**结果 = [本行含税价](金额 * (1 + 税率))**
|
||||
**结果**
|
||||
*结果*
|
||||
|
||||
## 含税总额
|
||||
+ `订单`
|
||||
|
||||
- [逐个订单行](遍历 订单.行 为 行)
|
||||
**行含税价 = [计算本行含税价](订单定价.含税价(行))**
|
||||
`订单`
|
||||
|
||||
**总额 = [汇总全部行含税价](sum(行含税价))**
|
||||
**总额**
|
||||
**行列表 = [订单行列表](access(订单, "行"))**
|
||||
`总额` = 0
|
||||
- [逐个订单行](遍历 行列表 为 行)
|
||||
**行含税价 = > 订单定价.含税价 行=`行`**
|
||||
**总额 = [累加](总额 + 行含税价)**
|
||||
*总额*
|
||||
@@ -4,12 +4,16 @@ id: document.invoice-answer
|
||||
type: Answer
|
||||
import:
|
||||
pricing: ../capabilities/order-pricing.mq.md
|
||||
json: lib/json.mq.md
|
||||
---
|
||||
|
||||
# 发票回答
|
||||
# main
|
||||
|
||||
**总额 = [订单的含税总价](access(pricing.订单定价.含税总额(订单), "订单的含税总价"))**
|
||||
发票回答:演示订单一行图书,未税金额 100。
|
||||
|
||||
`订单` = > json.parse text={"行":[{"金额":100,"品类":"书"}]}
|
||||
**总额 = > pricing.订单定价.含税总额 订单=`订单`**
|
||||
|
||||
这张订单的含税总额是[含税总价](总额)。
|
||||
|
||||
> view 应可展开:含税总价 → 汇总全部行含税价 → 逐个订单行 → 本行含税价 → 适用税率 → 税率原文。
|
||||
> print text=`总额`
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
markup: occlusion
|
||||
id: knowledge.negative.discount-not-for-refund
|
||||
type: Failure
|
||||
title: 折扣不适用于退款
|
||||
---
|
||||
|
||||
# 折扣不适用于退款
|
||||
|
||||
负知识:退款结算不得套用会员折扣。Agent 复用折扣能力时应拒绝该适用面。
|
||||
|
||||
## 说明
|
||||
|
||||
*[不适用](false)*
|
||||
@@ -1,18 +1,20 @@
|
||||
---
|
||||
markup: occlusion
|
||||
id: knowledge.tax.standard-2026
|
||||
type: Fact
|
||||
sources:
|
||||
- ../corpus/tax-policy-2026.md#2026 年标准增值税率
|
||||
applicability:
|
||||
region: CN
|
||||
valid_until: 2099-12-31
|
||||
---
|
||||
|
||||
# 标准税率
|
||||
|
||||
当前政策下的标准增值税率。`品类` 预留给差异化税率;本演示固定返回标准税率。
|
||||
|
||||
## 查询
|
||||
+ `品类`
|
||||
|
||||
| 返回 | 类型 | 来源 |
|
||||
|---|---|---|
|
||||
| 0.13 | number | corpus/tax-policy-2026.md#2026 年标准增值税率 |
|
||||
`品类`
|
||||
|
||||
**税率 = [标准税率](0.13)**
|
||||
**税率**
|
||||
*[标准税率](0.13)*
|
||||
+1
-1
@@ -1 +1 @@
|
||||
2.1.0
|
||||
2.1.4
|
||||
+385
-150
@@ -8,6 +8,7 @@ import:
|
||||
sys: lib/sys.mq.md
|
||||
fs: lib/fs.mq.md
|
||||
time: lib/time.mq.md
|
||||
text: lib/text.mq.md
|
||||
wb: lib/writeback.mq.md
|
||||
subtask: lib/subtask.mq.md
|
||||
plugin: lib/plugin.mq.md
|
||||
@@ -58,30 +59,45 @@ The runbook `.mq.md` is ground truth (code as documentation). Default budgets ke
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{text,llm_calls,tokens}` for episode metrics; transport failures set `error` + `llm_calls=0` (never abort).
|
||||
`stream_kind=plan` publishes protocol tokens onto the process trail (not the answer bubble).
|
||||
|
||||
1. `stream`
|
||||
**evs = > model.stream prompt=`prompt` echo=`echo`**
|
||||
**evs = > model.stream prompt=`prompt` echo=`echo` stream_kind=`stream_kind`**
|
||||
**text = > llm.collect events=`evs`**
|
||||
**tokens = 0**
|
||||
**err = None**
|
||||
**calls = 1**
|
||||
1. not `text`
|
||||
**e0 = > at value=`evs` index=0**
|
||||
**err = > json.get value=`e0` key="message"**
|
||||
1. `err`
|
||||
**calls = 0**
|
||||
**e0_s = > json.stringify value=`e0`**
|
||||
**e0_map = > text.starts_with text=`e0_s` prefix="{"**
|
||||
1. `e0_map`
|
||||
**err = > json.get value=`e0` key="message"**
|
||||
1. `err`
|
||||
**calls = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**r = > model.ask prompt=`prompt`**
|
||||
**text = > json.get value=`r` key="text"**
|
||||
**tokens = > json.get value=`r` key="tokens"**
|
||||
**err = > json.get value=`r` key="error"**
|
||||
**fin = > json.get value=`r` key="finish"**
|
||||
1. `tokens`
|
||||
**_ = 1**
|
||||
**r_s = > json.stringify value=`r`**
|
||||
**r_map = > text.starts_with text=`r_s` prefix="{"**
|
||||
1. `r_map`
|
||||
**text = > json.get value=`r` key="text"**
|
||||
**tokens = > json.get value=`r` key="tokens"**
|
||||
**err = > json.get value=`r` key="error"**
|
||||
**fin = > json.get value=`r` key="finish"**
|
||||
2. *
|
||||
**text = > str value=`r`**
|
||||
**tokens = 0**
|
||||
**err = None**
|
||||
**fin = None**
|
||||
1. not `tokens`
|
||||
**tokens = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `err`
|
||||
**calls = 0**
|
||||
2. `fin` == "error"
|
||||
@@ -183,7 +199,7 @@ Allowlist check, then invoke the runbook `##` via `lib/subtask` (`spawn fn=` →
|
||||
Build a runnable workbook. `skeleton=single` (default): one worker `step`. `skeleton=dual`: research then write agents. On success the parent chooses DONE; runtime `agent_workbook_solidify` freezes the answer.
|
||||
|
||||
**q = > json.quote text=`goal`**
|
||||
**esc = > json.parse text={"h":"---\ntitle: agent workbook\nimport:\n llm: ext/ai/llm.mq.md\n agent: ext/ai/agent.mq.md\n json: lib/json.mq.md\n sys: lib/sys.mq.md\n wb: lib/writeback.mq.md\n subtask: lib/subtask.mq.md\n---\n\n# Goal\n\nParent goal is embedded as JSON in # main.\n\n## Solidify\n\nWhen the child returns a good value, parent DECISION: DONE — runtime agent_workbook_solidify freezes the answer as **return** (do not invent long FIND/REPLACE).\n\n# main\n\n> llm.load_env\n\n","s1":"\u002a\u002a\u0060model\u0060 = > llm.llm\u002a\u002a\n","s2":"\u002a\u002a\u0060tools\u0060 = > table.empty_list\u002a\u002a\n","s3":"\u002a\u002a\u0060worker\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You are a workbook worker. tools=[] means never CALL. Finish the task with a direct final answer; do not invent tools.\u002a\u002a\n","s4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","s5":"}\u002a\u002a\n","s6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","s7":"\u002a\u002a\u0060out\u0060 = > \u0060worker\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","s7b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","s8":"\u002a\u002a\u0060text\u0060\u002a\u002a\n","d3":"\u002a\u002a\u0060research\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You gather facts for the goal. Be concise. Do not invent tools.\u002a\u002a\n","d3b":"\u002a\u002a\u0060writer\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=You write the final answer from research notes. Do not invent tools.\u002a\u002a\n","d4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","d5":"}\u002a\u002a\n","d6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=task\u002a\u002a\n","d7":"\u002a\u002a\u0060notes\u0060 = > \u0060research\u0060.step task=\u0060task\u0060 stream=False\u002a\u002a\n","d8":"\u002a\u002a\u0060ns\u0060 = > json.stringify value=\u0060notes\u0060\u002a\u002a\n","d9":"\u002a\u002a\u0060wt\u0060 = Write the final answer. Research notes:\u002a\u002a\n","d10":"\u002a\u002a\u0060wt\u0060 = \u0060wt\u0060 + \u0060ns\u0060\u002a\u002a\n","d11":"\u002a\u002a\u0060out\u0060 = > \u0060writer\u0060.step task=\u0060wt\u0060 stream=False\u002a\u002a\n","d11b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=result\u002a\u002a\n","d12":"\u002a\u002a\u0060text\u0060\u002a\u002a\n"}**
|
||||
**esc = > json.parse text={"h":"---\ntitle: agent workbook\nimport:\n llm: ext/ai/llm.mq.md\n agent: ext/ai/agent.mq.md\n json: lib/json.mq.md\n sys: lib/sys.mq.md\n table: lib/table.mq.md\n wb: lib/writeback.mq.md\n subtask: lib/subtask.mq.md\n---\n\n# Goal\n\nParent goal is embedded as JSON in # main.\n\n## Solidify\n\nWhen the child returns a good value, parent DECISION: DONE — runtime agent_workbook_solidify freezes the answer as **return** (do not invent long FIND/REPLACE).\n\n# main\n\n> llm.load_env\n\n","s1":"\u002a\u002a\u0060env_model\u0060 = > sys.env_get name=\"OPENAI_MODEL\"\u002a\u002a\n\u002a\u002a\u0060env_base\u0060 = > sys.env_get name=\"OPENAI_BASE_URL\"\u002a\u002a\n\u002a\u002a\u0060env_key\u0060 = > sys.env_get name=\"OPENAI_API_KEY\"\u002a\u002a\n\u002a\u002a\u0060model\u0060 = > llm.create model=\u0060env_model\u0060 base_url=\u0060env_base\u0060 api_key=\u0060env_key\u0060 backend=\"openai-compatible\" name=\"workbook\"\u002a\u002a\n","s2":"\u002a\u002a\u0060tools\u0060 = > table.empty_list\u002a\u002a\n","s3":"\u002a\u002a\u0060worker\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=\"You are a workbook worker. tools=[] means never CALL. Finish the task with a direct final answer; do not invent tools.\"\u002a\u002a\n","s4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","s5":"}\u002a\u002a\n","s6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=\"task\"\u002a\u002a\n","s7":"\u002a\u002a\u0060out\u0060 = > \u0060worker\u0060.step task=\u0060task\u0060 stream=True\u002a\u002a\n","s7b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=\"result\"\u002a\u002a\n","s8":"\u002a\u0060text\u0060\u002a\n","d3":"\u002a\u002a\u0060research\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=\"You gather facts for the goal. Be concise. Do not invent tools.\"\u002a\u002a\n","d3b":"\u002a\u002a\u0060writer\u0060 = > agent.agent model=\u0060model\u0060 tools=\u0060tools\u0060 standing=\"You write the final answer from research notes. Do not invent tools.\"\u002a\u002a\n","d4":"\u002a\u002a\u0060wrap\u0060 = > json.parse text={\"task\":","d5":"}\u002a\u002a\n","d6":"\u002a\u002a\u0060task\u0060 = > json.get value=\u0060wrap\u0060 key=\"task\"\u002a\u002a\n","d7":"\u002a\u002a\u0060notes\u0060 = > \u0060research\u0060.step task=\u0060task\u0060 stream=True\u002a\u002a\n","d8":"\u002a\u002a\u0060ns\u0060 = > json.stringify value=\u0060notes\u0060\u002a\u002a\n","d9":"\u002a\u002a\u0060wt\u0060 = \"Write the final answer. Research notes:\"\u002a\u002a\n","d10":"\u002a\u002a\u0060wt\u0060 = \u0060wt\u0060 + \u0060ns\u0060\u002a\u002a\n","d11":"\u002a\u002a\u0060out\u0060 = > \u0060writer\u0060.step task=\u0060wt\u0060 stream=True\u002a\u002a\n","d11b":"\u002a\u002a\u0060text\u0060 = > json.get value=\u0060out\u0060 key=\"result\"\u002a\u002a\n","d12":"\u002a\u0060text\u0060\u002a\n"}**
|
||||
**h = > json.get value=`esc` key="h"**
|
||||
**s1 = > json.get value=`esc` key="s1"**
|
||||
**s2 = > json.get value=`esc` key="s2"**
|
||||
@@ -849,25 +865,44 @@ Write a scratch tool workbook under `.marqdo/agent-runs/tools/<name>.mq.md`.
|
||||
|
||||
---
|
||||
|
||||
## skill_core_card
|
||||
|
||||
Protected v2.1 Occlusion card always prepended to skill_brief (M05). Must not be truncated by budget.
|
||||
|
||||
**esc = > json.parse text={"card":"## CORE SYNTAX (v2.1 Occlusion)\n1. Print with > print / > 打印 — never use bold markers to print.\n2. Bold markers = return/code; italic markers = statement.\n3. [desc](payload) = occlusion; brackets never execute.\n4. Read with access(coll, key) — never Classic [key](map) index.\n5. Foreach: - [desc](for coll as item). import + one language per file.\n6. # type · ## function · + params · numbered branches · - loops.\n"}**
|
||||
*> json.get value=`esc` key="card"*
|
||||
|
||||
---
|
||||
|
||||
## skill_brief
|
||||
+ `max_chars`=1200
|
||||
+ `depth`=default
|
||||
|
||||
Budgeted marqdo skill text for prompts. `depth=deep` raises the char budget.
|
||||
Budgeted marqdo skill text for prompts. Core Occlusion card is always included; remaining budget truncates the rest. `depth=deep` raises the char budget.
|
||||
|
||||
1. `depth` == deep
|
||||
**budget = 4000**
|
||||
2. *
|
||||
**budget = max_chars**
|
||||
**core = > skill_core_card**
|
||||
**core_len = > len value=`core`**
|
||||
**rest_budget = budget - core_len - 2**
|
||||
1. `rest_budget` < 200
|
||||
**rest_budget = 200**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**skill = > agent_marqdo_skill**
|
||||
**full = > len value=`skill`**
|
||||
**ex = > text_head_lines text=`skill` max_lines=80 max_chars=`budget`**
|
||||
**rest = > text_head_lines text=`skill` max_lines=80 max_chars=`rest_budget`**
|
||||
**esc = > json.parse text={"nl":"\n\n"}**
|
||||
**nl = > json.get value=`esc` key="nl"**
|
||||
**ex = core + nl + rest**
|
||||
**used = > len value=`ex`**
|
||||
1. `used` < `full`
|
||||
**esc = > json.parse text={"a":"\n…[truncated skill_len=","b":" budget_chars=","c":"; emit READ:skill to deepen]"}**
|
||||
**a = > json.get value=`esc` key="a"**
|
||||
**b = > json.get value=`esc` key="b"**
|
||||
**c = > json.get value=`esc` key="c"**
|
||||
1. `used` < `full` + `core_len`
|
||||
**esc2 = > json.parse text={"a":"\n…[truncated skill_len=","b":" budget_chars=","c":"; emit READ:skill to deepen]"}**
|
||||
**a = > json.get value=`esc2` key="a"**
|
||||
**b = > json.get value=`esc2` key="b"**
|
||||
**c = > json.get value=`esc2` key="c"**
|
||||
*ex + a + full + b + budget + c*
|
||||
2. *
|
||||
*ex*
|
||||
@@ -1061,8 +1096,15 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
|
||||
*> lib_catalog*
|
||||
4. `name` == scratch_tool_write
|
||||
**parsed = > extract_scratch_tool reply=`reply`**
|
||||
**tn = > json.get value=`parsed` key="name"**
|
||||
**tt = > json.get value=`parsed` key="text"**
|
||||
**parsed_s = > json.stringify value=`parsed`**
|
||||
**parsed_map = > text.starts_with text=`parsed_s` prefix="{"**
|
||||
**tn = None**
|
||||
**tt = None**
|
||||
1. `parsed_map`
|
||||
**tn = > json.get value=`parsed` key="name"**
|
||||
**tt = > json.get value=`parsed` key="text"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**errs = > json.parse text={"a":"scratch_tool_write needs NAME line and fenced body","b":"scratch_tool_write needs NAME id"}**
|
||||
1. `tn`
|
||||
1. `tt`
|
||||
@@ -1073,9 +1115,17 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
|
||||
*> json.get value=`errs` key="b"*
|
||||
5. `name` == corpus_search
|
||||
**args = > extract_call_args reply=`reply`**
|
||||
**q = > json.get value=`args` key="query"**
|
||||
**root = > json.get value=`args` key="root"**
|
||||
**limit = > json.get value=`args` key="limit"**
|
||||
**args_s = > json.stringify value=`args`**
|
||||
**args_map = > text.starts_with text=`args_s` prefix="{"**
|
||||
**q = None**
|
||||
**root = None**
|
||||
**limit = None**
|
||||
1. `args_map`
|
||||
**q = > json.get value=`args` key="query"**
|
||||
**root = > json.get value=`args` key="root"**
|
||||
**limit = > json.get value=`args` key="limit"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**cerrs = > json.parse text={"q":"corpus_search needs ARGS.query"}**
|
||||
1. not `q`
|
||||
*> json.get value=`cerrs` key="q"*
|
||||
@@ -1090,7 +1140,13 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
|
||||
*> corpus_search query=`q`*
|
||||
6. `name` == mcp_list_tools
|
||||
**args = > extract_call_args reply=`reply`**
|
||||
**fx = > json.get value=`args` key="fixture"**
|
||||
**args_s = > json.stringify value=`args`**
|
||||
**args_map = > text.starts_with text=`args_s` prefix="{"**
|
||||
**fx = None**
|
||||
1. `args_map`
|
||||
**fx = > json.get value=`args` key="fixture"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**merrs = > json.parse text={"f":"mcp_list_tools needs ARGS.fixture"}**
|
||||
1. `fx`
|
||||
*> mcp_list_tools fixture=`fx`*
|
||||
@@ -1098,9 +1154,17 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
|
||||
*> json.get value=`merrs` key="f"*
|
||||
7. `name` == mcp_call
|
||||
**args = > extract_call_args reply=`reply`**
|
||||
**fx = > json.get value=`args` key="fixture"**
|
||||
**tn = > json.get value=`args` key="name"**
|
||||
**targ = > json.get value=`args` key="arguments"**
|
||||
**args_s = > json.stringify value=`args`**
|
||||
**args_map = > text.starts_with text=`args_s` prefix="{"**
|
||||
**fx = None**
|
||||
**tn = None**
|
||||
**targ = None**
|
||||
1. `args_map`
|
||||
**fx = > json.get value=`args` key="fixture"**
|
||||
**tn = > json.get value=`args` key="name"**
|
||||
**targ = > json.get value=`args` key="arguments"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**xerrs = > json.parse text={"f":"mcp_call needs ARGS.fixture","n":"mcp_call needs ARGS.name"}**
|
||||
1. `fx`
|
||||
1. `tn`
|
||||
@@ -1161,14 +1225,28 @@ Parent Plan-and-Move tools (helpers + whitelist `CALL:lib…`).
|
||||
+ `observation`
|
||||
|
||||
Bounded observation for the parent prompt (actionable, short).
|
||||
Tolerates missing/non-map observation (empty compact) instead of `json get needs map`.
|
||||
Map check via stringify prefix `{` — never hard-get on None / bare text / list.
|
||||
|
||||
**obs_path = > json.get value=`observation` key="path"**
|
||||
**obs_code = > json.get value=`observation` key="exit_code"**
|
||||
**obs_val = > json.get value=`observation` key="value"**
|
||||
**obs_slots = > json.get value=`observation` key="slots"**
|
||||
**obs_src = > json.get value=`observation` key="source"**
|
||||
**obs_stdout = > json.get value=`observation` key="stdout"**
|
||||
**obs_stderr = > json.get value=`observation` key="stderr"**
|
||||
**obs_path = None**
|
||||
**obs_code = None**
|
||||
**obs_val = None**
|
||||
**obs_slots = None**
|
||||
**obs_src = None**
|
||||
**obs_stdout = None**
|
||||
**obs_stderr = None**
|
||||
**obs_s = > json.stringify value=`observation`**
|
||||
**obs_map = > text.starts_with text=`obs_s` prefix="{"**
|
||||
1. `obs_map`
|
||||
**obs_path = > json.get value=`observation` key="path"**
|
||||
**obs_code = > json.get value=`observation` key="exit_code"**
|
||||
**obs_val = > json.get value=`observation` key="value"**
|
||||
**obs_slots = > json.get value=`observation` key="slots"**
|
||||
**obs_src = > json.get value=`observation` key="source"**
|
||||
**obs_stdout = > json.get value=`observation` key="stdout"**
|
||||
**obs_stderr = > json.get value=`observation` key="stderr"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**src_len = 0**
|
||||
**has_step = False**
|
||||
1. `obs_src`
|
||||
@@ -1188,97 +1266,102 @@ Bounded observation for the parent prompt (actionable, short).
|
||||
|------|------|-----------|------------|-----------------|------------------|
|
||||
| Plan-and-Move: CALL/READ then DECISION. Success (exit 0 + has_value) → DONE. solidify_on_done. No long prose. | `obs_path` | `obs_code` | `src_len` | `has_step` | True |
|
||||
|
||||
1. `obs_src`
|
||||
**ex = > json.get value=`observation` key="source_excerpt"**
|
||||
1. `ex`
|
||||
**compact = table.put in=`compact` at="source_excerpt" value=`ex`**
|
||||
1. `obs_map`
|
||||
1. `obs_src`
|
||||
**ex = > json.get value=`observation` key="source_excerpt"**
|
||||
1. `ex`
|
||||
**compact = table.put in=`compact` at="source_excerpt" value=`ex`**
|
||||
2. *
|
||||
**ex = > workbook_source_excerpt source=`obs_src`**
|
||||
**compact = table.put in=`compact` at="source_excerpt" value=`ex`**
|
||||
2. *
|
||||
**ex = > workbook_source_excerpt source=`obs_src`**
|
||||
**compact = table.put in=`compact` at="source_excerpt" value=`ex`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**_ = 1**
|
||||
|
||||
1. `obs_stderr`
|
||||
**tail = > json.get value=`observation` key="stderr_tail"**
|
||||
1. `tail`
|
||||
**compact = table.put in=`compact` at="stderr_tail" value=`tail`**
|
||||
1. `obs_stderr`
|
||||
**tail = > json.get value=`observation` key="stderr_tail"**
|
||||
1. `tail`
|
||||
**compact = table.put in=`compact` at="stderr_tail" value=`tail`**
|
||||
2. *
|
||||
**tail = > text_tail_lines text=`obs_stderr`**
|
||||
**compact = table.put in=`compact` at="stderr_tail" value=`tail`**
|
||||
2. *
|
||||
**tail = > text_tail_lines text=`obs_stderr`**
|
||||
**compact = table.put in=`compact` at="stderr_tail" value=`tail`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `obs_stdout`
|
||||
**tail = > json.get value=`observation` key="stdout_tail"**
|
||||
1. `tail`
|
||||
**compact = table.put in=`compact` at="stdout_tail" value=`tail`**
|
||||
**_ = 1**
|
||||
1. `obs_stdout`
|
||||
**tail = > json.get value=`observation` key="stdout_tail"**
|
||||
1. `tail`
|
||||
**compact = table.put in=`compact` at="stdout_tail" value=`tail`**
|
||||
2. *
|
||||
**tail = > text_tail_lines text=`obs_stdout`**
|
||||
**compact = table.put in=`compact` at="stdout_tail" value=`tail`**
|
||||
2. *
|
||||
**tail = > text_tail_lines text=`obs_stdout`**
|
||||
**compact = table.put in=`compact` at="stdout_tail" value=`tail`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**_ = 1**
|
||||
|
||||
**val_txt = > str value=`obs_val`**
|
||||
**val_len = > len value=`val_txt`**
|
||||
1. `obs_val`
|
||||
**compact = table.put in=`compact` at="has_value" value=True**
|
||||
**compact = table.put in=`compact` at="value_len" value=`val_len`**
|
||||
1. `val_len` > 200
|
||||
**preview = > text_head_lines text=`val_txt` max_lines=4 max_chars=200**
|
||||
**compact = table.put in=`compact` at="value_preview" value=`preview`**
|
||||
**val_txt = > str value=`obs_val`**
|
||||
**val_len = > len value=`val_txt`**
|
||||
1. `obs_val`
|
||||
**compact = table.put in=`compact` at="has_value" value=True**
|
||||
**compact = table.put in=`compact` at="value_len" value=`val_len`**
|
||||
1. `val_len` > 200
|
||||
**preview = > text_head_lines text=`val_txt` max_lines=4 max_chars=200**
|
||||
**compact = table.put in=`compact` at="value_preview" value=`preview`**
|
||||
2. *
|
||||
**compact = table.put in=`compact` at="value" value=`obs_val`**
|
||||
2. *
|
||||
**compact = table.put in=`compact` at="value" value=`obs_val`**
|
||||
**compact = table.put in=`compact` at="has_value" value=False**
|
||||
|
||||
**slots_brief = > table.empty_list**
|
||||
1. `obs_slots`
|
||||
- [slot](遍历 `obs_slots` 为 `slot`)
|
||||
**sk = > json.get value=`slot` key="key"**
|
||||
**sl = > json.get value=`slot` key="line"**
|
||||
**sb = > json.get value=`slot` key="body"**
|
||||
**sb_txt = > str value=`sb`**
|
||||
**sb_len = > len value=`sb_txt`**
|
||||
`brief` =
|
||||
|
||||
| key | line | body_len |
|
||||
|-----|------|----------|
|
||||
| `sk` | `sl` | `sb_len` |
|
||||
|
||||
1. `sk` == error
|
||||
**prev = > text_head_lines text=`sb_txt` max_lines=8 max_chars=400**
|
||||
**brief = table.put in=`brief` at="body_preview" value=`prev`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**slots_brief = > table.append list=`slots_brief` item=`brief`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**compact = table.put in=`compact` at="slots" value=`slots_brief`**
|
||||
**detail = > json.get value=`observation` key="slots_detail"**
|
||||
1. `detail`
|
||||
**compact = table.put in=`compact` at="slots_detail" value=`detail`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**tool_result = > json.get value=`observation` key="tool_result"**
|
||||
1. `tool_result`
|
||||
**tr = > str value=`tool_result`**
|
||||
**tr = > text_head_lines text=`tr` max_lines=40 max_chars=3000**
|
||||
**compact = table.put in=`compact` at="tool_result" value=`tr`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**last_error = > json.get value=`observation` key="last_error"**
|
||||
1. `last_error`
|
||||
**err_txt = > str value=`last_error`**
|
||||
**err_len = > len value=`err_txt`**
|
||||
1. `err_len` > 200
|
||||
**prev = > text_head_lines text=`err_txt` max_lines=8 max_chars=400**
|
||||
**compact = table.put in=`compact` at="last_error_preview" value=`prev`**
|
||||
**compact = table.put in=`compact` at="last_error_len" value=`err_len`**
|
||||
2. *
|
||||
**compact = table.put in=`compact` at="last_error" value=`last_error`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**compact = table.put in=`compact` at="has_value" value=False**
|
||||
|
||||
**slots_brief = > table.empty_list**
|
||||
1. `obs_slots`
|
||||
- [slot](遍历 `obs_slots` 为 `slot`)
|
||||
**sk = > json.get value=`slot` key="key"**
|
||||
**sl = > json.get value=`slot` key="line"**
|
||||
**sb = > json.get value=`slot` key="body"**
|
||||
**sb_txt = > str value=`sb`**
|
||||
**sb_len = > len value=`sb_txt`**
|
||||
`brief` =
|
||||
|
||||
| key | line | body_len |
|
||||
|-----|------|----------|
|
||||
| `sk` | `sl` | `sb_len` |
|
||||
|
||||
1. `sk` == error
|
||||
**prev = > text_head_lines text=`sb_txt` max_lines=8 max_chars=400**
|
||||
**brief = table.put in=`brief` at="body_preview" value=`prev`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**slots_brief = > table.append list=`slots_brief` item=`brief`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**compact = table.put in=`compact` at="slots" value=`slots_brief`**
|
||||
**detail = > json.get value=`observation` key="slots_detail"**
|
||||
1. `detail`
|
||||
**compact = table.put in=`compact` at="slots_detail" value=`detail`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**tool_result = > json.get value=`observation` key="tool_result"**
|
||||
1. `tool_result`
|
||||
**tr = > str value=`tool_result`**
|
||||
**tr = > text_head_lines text=`tr` max_lines=40 max_chars=3000**
|
||||
**compact = table.put in=`compact` at="tool_result" value=`tr`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**last_error = > json.get value=`observation` key="last_error"**
|
||||
1. `last_error`
|
||||
**err_txt = > str value=`last_error`**
|
||||
**err_len = > len value=`err_txt`**
|
||||
1. `err_len` > 200
|
||||
**prev = > text_head_lines text=`err_txt` max_lines=8 max_chars=400**
|
||||
**compact = table.put in=`compact` at="last_error_preview" value=`prev`**
|
||||
**compact = table.put in=`compact` at="last_error_len" value=`err_len`**
|
||||
2. *
|
||||
**compact = table.put in=`compact` at="last_error" value=`last_error`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**empty_slots = > table.empty_list**
|
||||
**compact = table.put in=`compact` at="slots" value=`empty_slots`**
|
||||
|
||||
*compact*
|
||||
|
||||
@@ -1332,6 +1415,14 @@ First matching DECISION / 决定 line wins (ignore later conflicting decisions i
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `found`
|
||||
Take the first token only (`DECISION: RUN please` → RUN).
|
||||
**toks = > split value=`found` sep=" "**
|
||||
**found = > at value=`toks` index=0**
|
||||
**found = > trim value=`found`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `found` == DONE
|
||||
*"DONE"*
|
||||
2. `found` == CONTINUE
|
||||
@@ -1352,8 +1443,17 @@ First matching DECISION / 决定 line wins (ignore later conflicting decisions i
|
||||
*"REUSE"*
|
||||
10. `found` == 新建
|
||||
*"NEW"*
|
||||
11. *
|
||||
*found*
|
||||
11. `found` == PATCH
|
||||
Models sometimes invent `DECISION: PATCH` — treat as CONTINUE.
|
||||
*"CONTINUE"*
|
||||
12. *
|
||||
If the reply carries a FIND/REPLACE block, treat unknown labels as CONTINUE.
|
||||
**has_find = > split value=`text` sep="<<<"**
|
||||
**n_find = > len value=`has_find`**
|
||||
1. `n_find` > 1
|
||||
*"CONTINUE"*
|
||||
2. *
|
||||
*found*
|
||||
|
||||
---
|
||||
|
||||
@@ -1525,7 +1625,13 @@ Parent Plan-and-Move prompt. Short protocol only — no long monologues.
|
||||
2. *
|
||||
**up = None**
|
||||
|
||||
**skill_from_obs = > json.get value=`observation` key="skill_brief"**
|
||||
**skill_from_obs = None**
|
||||
**obs_probe = > json.stringify value=`observation`**
|
||||
**obs_is_map = > text.starts_with text=`obs_probe` prefix="{"**
|
||||
1. `obs_is_map`
|
||||
**skill_from_obs = > json.get value=`observation` key="skill_brief"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `skill_from_obs`
|
||||
**skill = skill_from_obs**
|
||||
2. *
|
||||
@@ -1593,22 +1699,20 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
**left = 0**
|
||||
3. *
|
||||
**ctx = > build_plan_context agent=`agent` goal=`goal` observation=`last_obs` explore_attempt=`explore_attempt` explore_n=`explore_n` phase=`phase`**
|
||||
Plan protocol streams as `stream_kind=plan` (process trail). Answer tokens
|
||||
still come from workbook `worker.step stream=True` (`stream_kind=delta`).
|
||||
1. `stream`
|
||||
**evs = > model.stream prompt=`ctx` echo=`echo`**
|
||||
**last_reply = > llm.collect events=`evs`**
|
||||
**evs_all = > plan_merge_deltas events=`evs_all` from=`evs` stream=`stream`**
|
||||
**e0 = > at value=`evs` index=0**
|
||||
**terr = > json.get value=`e0` key="message"**
|
||||
**etype = > json.get value=`e0` key="type"**
|
||||
1. `etype` == "error"
|
||||
**transport_err = `terr`**
|
||||
1. not `transport_err`
|
||||
**transport_err = "llm stream transport error"**
|
||||
**left = 0**
|
||||
2. *
|
||||
**llm_calls = llm_calls + 1**
|
||||
**turn = > model_turn model=`model` prompt=`ctx` stream=True echo=`echo` stream_kind="plan"**
|
||||
2. *
|
||||
**turn = > model_turn model=`model` prompt=`ctx` stream=False echo=`echo`**
|
||||
**turn_s = > json.stringify value=`turn`**
|
||||
**turn_map = > text.starts_with text=`turn_s` prefix="{"**
|
||||
**terr = None**
|
||||
**last_reply = ""**
|
||||
1. not `turn_map`
|
||||
**transport_err = "model_turn_not_map"**
|
||||
**left = 0**
|
||||
2. *
|
||||
**terr = > json.get value=`turn` key="error"**
|
||||
1. `terr`
|
||||
**transport_err = `terr`**
|
||||
@@ -1617,6 +1721,14 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
**last_reply = > json.get value=`turn` key="text"**
|
||||
**c = > json.get value=`turn` key="llm_calls"**
|
||||
**t = > json.get value=`turn` key="tokens"**
|
||||
1. not `c`
|
||||
**c = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `t`
|
||||
**t = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
1. `transport_err`
|
||||
@@ -1624,16 +1736,49 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
**last_reply = `transport_err`**
|
||||
**left = 0**
|
||||
2. *
|
||||
**last_reply = > trim value=`last_reply`**
|
||||
**act = > extract_plan_act reply=`last_reply`**
|
||||
**kind = > json.get value=`act` key="kind"**
|
||||
**name = > json.get value=`act` key="name"**
|
||||
1. not `last_reply`
|
||||
**last_reply = ""**
|
||||
2. *
|
||||
**last_reply = > trim value=`last_reply`**
|
||||
Plan replies often embed CALL: inside PATCH bodies — prefer an explicit
|
||||
DECISION line when present so protocol exit is not stolen by a false CALL.
|
||||
**dec_line = > extract_plan_decision reply=`last_reply`**
|
||||
**kind = "unknown"**
|
||||
**name = None**
|
||||
1. `dec_line`
|
||||
**kind = "decision"**
|
||||
**name = `dec_line`**
|
||||
2. *
|
||||
**act = > extract_plan_act reply=`last_reply`**
|
||||
**act_s = > json.stringify value=`act`**
|
||||
**act_map = > text.starts_with text=`act_s` prefix="{"**
|
||||
1. `act_map`
|
||||
**kind = > json.get value=`act` key="kind"**
|
||||
**name = > json.get value=`act` key="name"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
Live act marker so the UI never goes silent between plan tokens and the next step.
|
||||
1. `stream`
|
||||
`act_ev` =
|
||||
|
||||
| type | kind | name |
|
||||
|------|------|------|
|
||||
| plan_act | `kind` | `name` |
|
||||
|
||||
> sys.stream_publish event=`act_ev`
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `kind` == call
|
||||
**evs_all = > plan_append_tool events=`evs_all` type="tool_start" name=`name` kind="call" stream=`stream`**
|
||||
**tool_out = > run_parent_tool name=`name` path=`path` observation=`last_obs` reply=`last_reply`**
|
||||
**brief = > tool_result_brief value=`tool_out`**
|
||||
**evs_all = > plan_append_tool events=`evs_all` type="tool_end" name=`name` kind="call" result=`brief` stream=`stream`**
|
||||
**last_obs = table.put in=`last_obs` at="tool_result" value=`tool_out`**
|
||||
**obs_s = > json.stringify value=`last_obs`**
|
||||
**obs_map = > text.starts_with text=`obs_s` prefix="{"**
|
||||
1. `obs_map`
|
||||
**last_obs = table.put in=`last_obs` at="tool_result" value=`tool_out`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**left = left - 1**
|
||||
2. `kind` == read
|
||||
**evs_all = > plan_append_tool events=`evs_all` type="tool_start" name=`name` kind="read" stream=`stream`**
|
||||
@@ -1973,18 +2118,27 @@ O5: mark OKF skill `status: quarantine` and force `llm_free: false`. Regression
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `improve_every`=3
|
||||
+ `promote`=True
|
||||
+ `candidate_only`=False
|
||||
+ `product_gate_ok`=True
|
||||
|
||||
Offline learn gate: when ≥ `improve_every` successful episodes share a task hash, generalize → synthesize → compile → optional OKF promote (`llm_free`). Also attempts Policy Compilation for Adaptive Routing.
|
||||
O2: promote path requires sandbox proof unless `require_proof=false`.
|
||||
M04: `candidate_only=True` or `product_gate_ok=False` forces promote off via eng_publish_gate.
|
||||
|
||||
**gate = > `self`.eng_publish_gate project="." candidate_only=`candidate_only` product_gate_ok=`product_gate_ok`**
|
||||
**may_prom = > json.get value=`gate` key="may_promote"**
|
||||
1. not `may_prom`
|
||||
**promote = False**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**skill_learn = > agent_maybe_learn task=`task` kb_dir=`kb_dir` memory_dir=`memory_dir` improve_every=`improve_every` promote=`promote`**
|
||||
**policy_learn = > agent_maybe_compile_policy memory_dir=`memory_dir` min_evidence=`improve_every`**
|
||||
**sl = > json.get value=`skill_learn` key="learned"**
|
||||
`out` =
|
||||
|
||||
| learned | skill | policy |
|
||||
|---------|-------|--------|
|
||||
| `sl` | `skill_learn` | `policy_learn` |
|
||||
| learned | skill | policy | promote_allowed | gate |
|
||||
|---------|-------|--------|-----------------|------|
|
||||
| `sl` | `skill_learn` | `policy_learn` | `may_prom` | `gate` |
|
||||
|
||||
*out*
|
||||
|
||||
@@ -2075,6 +2229,12 @@ Primary author entry (v2 + P4): Adaptive `route` → execute (policy/kb skill) o
|
||||
| ok | `task` | `child_val` | `path` | hit | `rmode` | `sk` | `last_obs` | adaptive route hit | `routed` |
|
||||
|
||||
**llm_calls = 0**
|
||||
1. `stream`
|
||||
**_ev = > table.empty_list**
|
||||
**_ev = > plan_append_decision events=`_ev` decision="REUSE" stream=True summary="adaptive route hit"**
|
||||
**_ev = > plan_append_round events=`_ev` round=1 workbook=`path` exit_code=`code` result=`child_val` stream=True**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
@@ -2128,7 +2288,18 @@ Primary author entry (v2 + P4): Adaptive `route` → execute (policy/kb skill) o
|
||||
1. `efi_st` == "missing_knowledge"
|
||||
**_ = 1**
|
||||
2. `efi_dec` == "REUSE"
|
||||
**_ = 1**
|
||||
**efi_exec = > agent_eng_bind_execute project="." query=`task` policy="pure" candidate_only=False**
|
||||
**efi_ok = > json.get value=`efi_exec` key="ok"**
|
||||
1. `efi_ok`
|
||||
**exec_mode = "efi_bound"**
|
||||
**out = **
|
||||
|
||||
| status | goal | result | summary | efi | reuse_trace |
|
||||
|--------|------|--------|---------|-----|-------------|
|
||||
| ok | `task` | `efi_exec` | EFI REUSE bound+executed | `efi_pf` | `efi_exec` |
|
||||
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. `efi_dec` == "ADAPT"
|
||||
**_ = 1**
|
||||
4. `efi_allow`
|
||||
@@ -2302,6 +2473,14 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**reply = > json.get value=`turn` key="text"**
|
||||
**c = > json.get value=`turn` key="llm_calls"**
|
||||
**t = > json.get value=`turn` key="tokens"**
|
||||
1. not `c`
|
||||
**c = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `t`
|
||||
**t = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**reply = > trim value=`reply`**
|
||||
@@ -2354,6 +2533,14 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**reply = > json.get value=`turn2` key="text"**
|
||||
**c = > json.get value=`turn2` key="llm_calls"**
|
||||
**t = > json.get value=`turn2` key="tokens"**
|
||||
1. not `c`
|
||||
**c = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `t`
|
||||
**t = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**out = table.put in=`out` at="denied_tool" value=`tool_name`**
|
||||
@@ -2371,6 +2558,14 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**reply = > json.get value=`turn2` key="text"**
|
||||
**c = > json.get value=`turn2` key="llm_calls"**
|
||||
**t = > json.get value=`turn2` key="tokens"**
|
||||
1. not `c`
|
||||
**c = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `t`
|
||||
**t = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**out = table.put in=`out` at="tool" value=`tool_name`**
|
||||
@@ -2433,13 +2628,42 @@ Hard rule: generate only when `create_allowed` is true (or decision is REUSE/ADA
|
||||
+ `root`="."
|
||||
+ `marqdo_dir`=None
|
||||
|
||||
Alias of `preflight` focused on REUSE / ADAPT / CREATE payload.
|
||||
Alias of `preflight` focused on REUSE / ADAPT / CREATE payload (discovery_only).
|
||||
|
||||
1. `marqdo_dir`
|
||||
*> agent_eng_reuse task=`task` root=`root` marqdo_dir=`marqdo_dir`*
|
||||
2. *
|
||||
*> agent_eng_reuse task=`task` root=`root`*
|
||||
|
||||
---
|
||||
|
||||
## eng_bind_execute
|
||||
+ `project`="."
|
||||
+ `query`
|
||||
+ `policy`="pure"
|
||||
+ `inputs`=None
|
||||
+ `candidate_only`=False
|
||||
|
||||
M04: discover → bind → execute → record. Prefer over treating eng_reuse scores as execution.
|
||||
|
||||
1. `inputs`
|
||||
*> agent_eng_bind_execute project=`project` query=`query` policy=`policy` inputs=`inputs` candidate_only=`candidate_only`*
|
||||
2. *
|
||||
*> agent_eng_bind_execute project=`project` query=`query` policy=`policy` candidate_only=`candidate_only`*
|
||||
|
||||
---
|
||||
|
||||
## eng_publish_gate
|
||||
+ `project`="."
|
||||
+ `artifact_id`=None
|
||||
+ `candidate_only`=False
|
||||
+ `product_gate_ok`=True
|
||||
+ `policy`="business"
|
||||
|
||||
Refuse promote when candidate_only or product/verification gate fails.
|
||||
|
||||
*> agent_eng_publish_gate project=`project` artifact_id=`artifact_id` candidate_only=`candidate_only` product_gate_ok=`product_gate_ok` policy=`policy`*
|
||||
|
||||
## eng_record
|
||||
+ `decision`
|
||||
+ `root`="."
|
||||
@@ -2798,17 +3022,28 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
|
||||
> plan_echo_decompose workbook=`path` stream=`stream` echo=`echo`
|
||||
**last_obs = > inspect_workbook path=`path`**
|
||||
**turn = > plan_llm_act_loop agent=`self` goal=`goal` path=`path` observation=`last_obs` phase="decompose" explore_attempt=`explore_attempt` explore_n=`explore_n` stream=`stream` echo=`echo` events=`events`**
|
||||
**dec = > json.get value=`turn` key="decision"**
|
||||
**last_reply = > json.get value=`turn` key="reply"**
|
||||
**last_obs = > json.get value=`turn` key="observation"**
|
||||
**events = > json.get value=`turn` key="events"**
|
||||
**turn_s = > json.stringify value=`turn`**
|
||||
**turn_map = > text.starts_with text=`turn_s` prefix="{"**
|
||||
**dec = None**
|
||||
**last_reply = None**
|
||||
1. `turn_map`
|
||||
**dec = > json.get value=`turn` key="decision"**
|
||||
**last_reply = > json.get value=`turn` key="reply"**
|
||||
**last_obs = > json.get value=`turn` key="observation"**
|
||||
**events = > json.get value=`turn` key="events"**
|
||||
2. *
|
||||
**dec = "TRANSPORT_ERROR"**
|
||||
**last_reply = "plan_llm_act_loop_not_map"**
|
||||
**err = `last_reply`**
|
||||
|
||||
1. `dec` == "TRANSPORT_ERROR"
|
||||
**done = 1**
|
||||
**status = "error"**
|
||||
**err = > json.get value=`turn` key="error"**
|
||||
1. not `err`
|
||||
**err = `last_reply`**
|
||||
**err = `last_reply`**
|
||||
1. `turn_map`
|
||||
**e2 = > json.get value=`turn` key="error"**
|
||||
1. `e2`
|
||||
**err = `e2`**
|
||||
**summary = "llm transport error"**
|
||||
**skip_loop = 1**
|
||||
2. `dec` == CONTINUE
|
||||
|
||||
+90
-23
@@ -23,16 +23,27 @@ Load `.env` from cwd (optional `path=`). Does not override existing variables.
|
||||
## collect
|
||||
+ `events`
|
||||
|
||||
Collapse a stream event list to final answer text (`done.result` or `error.message`). Prefer this over raw SSE fields. Alias: `stream_result`.
|
||||
Collapse a stream event list to final answer text.
|
||||
Accepts semantic events (`delta` / `text` / `done.result` / `error.message`) and
|
||||
native-plugin raw SSE frames (`type=data` with OpenAI chunk JSON in `data`).
|
||||
Alias: `stream_result`.
|
||||
|
||||
**answer = ""**
|
||||
|
||||
- [ev](遍历 events 为 ev)
|
||||
**t = > json.get value=`ev` key="type"**
|
||||
1. `t` == "done"
|
||||
**answer = > json.get value=`ev` key="result"**
|
||||
**done_r = > json.get value=`ev` key="result"**
|
||||
1. `done_r`
|
||||
**answer = `done_r`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. `t` == "error"
|
||||
**answer = > json.get value=`ev` key="message"**
|
||||
**em = > json.get value=`ev` key="message"**
|
||||
1. `em`
|
||||
**answer = `em`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. `t` == "delta"
|
||||
4. `t` == "text"
|
||||
**chunk = > json.get value=`ev` key="text"**
|
||||
@@ -40,9 +51,45 @@ Collapse a stream event list to final answer text (`done.result` or `error.messa
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
5. `t` == "data"
|
||||
**raw = > json.get value=`ev` key="data"**
|
||||
1. `raw`
|
||||
1. `raw` == "[DONE]"
|
||||
**_ = 1**
|
||||
2. *
|
||||
**parsed = > json.parse text=`raw`**
|
||||
1. `parsed`
|
||||
**choices = > json.get value=`parsed` key="choices"**
|
||||
1. `choices`
|
||||
**c0 = [1](access(choices, "1"))**
|
||||
**delta = > json.get value=`c0` key="delta"**
|
||||
1. `delta`
|
||||
**piece = > json.get value=`delta` key="content"**
|
||||
1. `piece`
|
||||
**answer = answer + piece**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**msg = > json.get value=`c0` key="message"**
|
||||
1. `msg`
|
||||
**piece = > json.get value=`msg` key="content"**
|
||||
1. `piece`
|
||||
**answer = answer + piece**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
6. *
|
||||
**_ = 1**
|
||||
|
||||
1. not `answer`
|
||||
**answer = ""**
|
||||
*answer*
|
||||
|
||||
## stream_result
|
||||
@@ -156,6 +203,7 @@ Prompt artifact helper: `path=` loads a file; `text=` strips `type: prompt` fron
|
||||
|
||||
Module convenience: create a default handle and return **answer text**. Pass `path=` to load a prompt document, `prompt=` text, or omit both to use the entry `.mq.md` as a prompt artifact (`type: prompt` body via `sys.module_source`).
|
||||
|
||||
**p = `prompt`**
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
@@ -268,20 +316,26 @@ Prefer declaring `model` / `api_key` / `base_url` in frontmatter Binding, then u
|
||||
## ask
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
+ `messages`=None
|
||||
+ `selection`=None
|
||||
+ `strict_selection`=False
|
||||
|
||||
Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`).
|
||||
Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`). Pass `messages=` for multi-turn history (M01). `selection=` + `strict_selection=True` freeze request identity without env fallback.
|
||||
|
||||
1. `path`
|
||||
**p = `prompt`**
|
||||
1. `messages`
|
||||
**p = None**
|
||||
2. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
3. `prompt`
|
||||
**p = `prompt`**
|
||||
3. *
|
||||
4. *
|
||||
**raw = > sys.module_source**
|
||||
1. not `raw`
|
||||
**p = ""**
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
**pack = > self.complete_result prompt=`p` stream=False echo=False**
|
||||
**pack = > self.complete_result prompt=`p` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=False echo=False**
|
||||
**text = > json.get value=`pack` key="text"**
|
||||
**usage = > json.get value=`pack` key="usage"**
|
||||
**finish = > json.get value=`pack` key="finish"**
|
||||
@@ -312,11 +366,16 @@ inside the table is unsafe.
|
||||
**nm = > json.get value=`self` key="name"**
|
||||
1. `usage`
|
||||
**tt = > json.get value=`usage` key="total_tokens"**
|
||||
**ustate = > json.get value=`usage` key="usage_state"**
|
||||
2. *
|
||||
**tt = 0**
|
||||
1. not `tt`
|
||||
**tt = 0**
|
||||
2. *
|
||||
**tt = None**
|
||||
**ustate = "unknown"**
|
||||
1. `ustate` == "unknown"
|
||||
Keep null — never coerce missing usage to zero tokens (M02 / F04).
|
||||
**tt = None**
|
||||
2. not `tt`
|
||||
**tt = None**
|
||||
3. *
|
||||
**_ = 1**
|
||||
1. `err`
|
||||
**calls = 0**
|
||||
@@ -325,9 +384,9 @@ inside the table is unsafe.
|
||||
|
||||
`result` =
|
||||
|
||||
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error |
|
||||
|------|-------|--------|---------|-------|------|-----------|--------|------------|-------|
|
||||
| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | `calls` | `tt` | `tc` | `err` |
|
||||
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error | selection |
|
||||
|------|-------|--------|---------|-------|------|-----------|--------|------------|-------|-----------|
|
||||
| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | `calls` | `tt` | `tc` | `err` | `selection` |
|
||||
|
||||
*result*
|
||||
|
||||
@@ -335,8 +394,10 @@ inside the table is unsafe.
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
Semantic stream: return the event list (use `llm.collect` to reduce to text). Omit `prompt=` / `path=` to use the entry document.
|
||||
`stream_kind=plan` tags EventBus frames for the parent plan protocol (process trail).
|
||||
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
@@ -349,30 +410,35 @@ Semantic stream: return the event list (use `llm.collect` to reduce to text). Om
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
|
||||
*> self.complete prompt=`p` stream=True echo=`echo`*
|
||||
*> self.complete prompt=`p` stream=True echo=`echo` stream_kind=`stream_kind`*
|
||||
|
||||
## complete_result
|
||||
+ `prompt`
|
||||
+ `prompt`=None
|
||||
+ `messages`=None
|
||||
+ `selection`=None
|
||||
+ `strict_selection`=False
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`.
|
||||
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`. Prefer `messages=` for multi-turn; optional `selection=` (ModelSelectionV1) with `strict_selection=True` for request-scoped identity (M01).
|
||||
|
||||
*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` stream=`stream` echo=`echo` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))*
|
||||
*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=`stream` echo=`echo` stream_kind=`stream_kind` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))*
|
||||
|
||||
## complete
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
Internal / compatibility primitive. Prefer `ask` / `stream` in new code.
|
||||
|
||||
1. `stream`
|
||||
**pack = > self.complete_result prompt=`prompt` stream=True echo=`echo`**
|
||||
**pack = > self.complete_result prompt=`prompt` stream=True echo=`echo` stream_kind=`stream_kind`**
|
||||
**_ret = > json.get value=`pack` key="events"**
|
||||
*_ret*
|
||||
2. *
|
||||
**pack = > self.complete_result prompt=`prompt` stream=False echo=`echo`**
|
||||
**pack = > self.complete_result prompt=`prompt` stream=False echo=`echo` stream_kind=`stream_kind`**
|
||||
**_ret = > json.get value=`pack` key="text"**
|
||||
*_ret*
|
||||
|
||||
@@ -380,7 +446,8 @@ Internal / compatibility primitive. Prefer `ask` / `stream` in new code.
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
Alias for `complete` (compat).
|
||||
|
||||
*> self.complete prompt=`prompt` stream=`stream` echo=`echo`*
|
||||
*> self.complete prompt=`prompt` stream=`stream` echo=`echo` stream_kind=`stream_kind`*
|
||||
+17
-6
@@ -73,28 +73,39 @@ Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list).
|
||||
+ `base_url`
|
||||
+ `api_key`
|
||||
+ `model`
|
||||
+ `prompt`
|
||||
+ `prompt`=None
|
||||
+ `messages`=None
|
||||
+ `selection`=None
|
||||
+ `strict_selection`=False
|
||||
+ `auth_mode`=None
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
+ `suffix`=/chat/completions
|
||||
+ `bearer`="Bearer "
|
||||
|
||||
POST `/chat/completions` (or stream SSE). Prefers `plugins/llm` ABI (`llm_chat_completions`) when the native plugin is resolvable; otherwise `lib/net` HTTP/SSE. Non-stream → `{text,usage,finish,tool_calls}`; stream → `{events,finish}`. Soft transport / HTTP failures return `finish=error` packs (never abort the VM).
|
||||
|
||||
Pass either `messages=` (role/content list) or legacy `prompt=` (single user message). Optional `selection=` is a ModelSelectionV1 snapshot; `strict_selection=True` refuses env fallbacks inside the plugin.
|
||||
`stream_kind` tags live EventBus frames: `delta` (answer tokens) or `plan` (parent protocol — UI process trail, not the bubble).
|
||||
|
||||
**p = > plugin.native_path name="llm"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`*
|
||||
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` auth_mode=`auth_mode` stream=`stream` echo=`echo` stream_kind=`stream_kind` suffix=`suffix` bearer=`bearer`*
|
||||
2. *
|
||||
**url = base_url + suffix**
|
||||
**auth = bearer + api_key**
|
||||
**headers = > table.put in=None at="Authorization" value=`auth`**
|
||||
|
||||
`messages` =
|
||||
1. not `messages`
|
||||
`messages` =
|
||||
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `prompt` |
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `prompt` |
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
`req` =
|
||||
|
||||
|
||||
+50
-3
@@ -20,14 +20,25 @@ description: >-
|
||||
## 收集
|
||||
+ `事件`
|
||||
|
||||
Collapse stream events to text. Supports semantic `delta`/`done.result` and
|
||||
native-plugin raw frames (`type=data` + OpenAI chunk JSON).
|
||||
|
||||
**answer = ""**
|
||||
|
||||
- [ev](遍历 事件 为 ev)
|
||||
**t = [type](access(ev, "type"))**
|
||||
1. `t` == "done"
|
||||
**answer = [result](access(ev, "result"))**
|
||||
**done_r = [result](access(ev, "result"))**
|
||||
1. `done_r`
|
||||
**answer = `done_r`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. `t` == "error"
|
||||
**answer = [message](access(ev, "message"))**
|
||||
**em = [message](access(ev, "message"))**
|
||||
1. `em`
|
||||
**answer = `em`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. `t` == "delta"
|
||||
4. `t` == "text"
|
||||
**chunk = [text](access(ev, "text"))**
|
||||
@@ -35,9 +46,45 @@ description: >-
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
5. `t` == "data"
|
||||
**raw = [data](access(ev, "data"))**
|
||||
1. `raw`
|
||||
1. `raw` == "[DONE]"
|
||||
**_ = 1**
|
||||
2. *
|
||||
**parsed = > json.parse text=`raw`**
|
||||
1. `parsed`
|
||||
**choices = [choices](access(parsed, "choices"))**
|
||||
1. `choices`
|
||||
**c0 = [1](access(choices, "1"))**
|
||||
**delta = [delta](access(c0, "delta"))**
|
||||
1. `delta`
|
||||
**piece = [content](access(delta, "content"))**
|
||||
1. `piece`
|
||||
**answer = answer + piece**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**msg = [message](access(c0, "message"))**
|
||||
1. `msg`
|
||||
**piece = [content](access(msg, "content"))**
|
||||
1. `piece`
|
||||
**answer = answer + piece**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
6. *
|
||||
**_ = 1**
|
||||
|
||||
1. not `answer`
|
||||
**answer = ""**
|
||||
*answer*
|
||||
|
||||
## 流式结果
|
||||
|
||||
@@ -86,6 +86,19 @@ Like `select` but returns `{ rows, total }` — the total counts rows matching `
|
||||
**url = [url](access(self, "url"))**
|
||||
*> web_db_update url=`url` table=`table` id=`id` row=`row` txn=`txn`*
|
||||
|
||||
## update_if
|
||||
+ `table`
|
||||
+ `id`
|
||||
+ `row`
|
||||
+ `expect_col`
|
||||
+ `expect_val`
|
||||
+ `txn`=None
|
||||
|
||||
Conditional update (CAS) for task/revision control: `UPDATE … WHERE id=? AND expect_col=expect_val`. Returns `{ ok, updated, matched }`; `updated=0` means conflict — do not simulate atomicity with read-then-write in `.mq.md`.
|
||||
|
||||
**url = [url](access(self, "url"))**
|
||||
*> web_db_update_if url=`url` table=`table` id=`id` row=`row` expect_col=`expect_col` expect_val=`expect_val` txn=`txn`*
|
||||
|
||||
## delete
|
||||
+ `table`
|
||||
+ `id`
|
||||
|
||||
+7
-4
@@ -16,15 +16,17 @@ import:
|
||||
+ `request`="json"
|
||||
+ `response`="json"
|
||||
+ `auth`=""
|
||||
+ `class`=""
|
||||
|
||||
Declared route handle. Prefer `web.route` / Artifact `type: endpoint` discovery via `web.serve`.
|
||||
Optional `class`: `execute` | `short` | `read` | `control` (empty → heuristic / `legacy_serial`).
|
||||
|
||||
> cap.load
|
||||
`out` =
|
||||
|
||||
| _type | app | path | method | page | fn | request | response | auth |
|
||||
|-------|-----|------|--------|------|----|---------|----------|------|
|
||||
| route | `app` | `path` | `method` | `page` | `fn` | `request` | `response` | `auth` |
|
||||
| _type | app | path | method | page | fn | request | response | auth | class |
|
||||
|-------|-----|------|--------|------|----|---------|----------|------|-------|
|
||||
| route | `app` | `path` | `method` | `page` | `fn` | `request` | `response` | `auth` | `class` |
|
||||
|
||||
*out*
|
||||
|
||||
@@ -57,6 +59,7 @@ Register this declared route on an app handle.
|
||||
**method = [method](access(self, "method"))**
|
||||
**request = [request](access(self, "request"))**
|
||||
**response = [response](access(self, "response"))**
|
||||
*> web_app_invoke app=`a` path=`path` method=`method` fn=`fn` body=`request` return=`response`*
|
||||
**class = [class](access(self, "class"))**
|
||||
*> web_app_invoke app=`a` path=`path` method=`method` fn=`fn` body=`request` return=`response` class=`class`*
|
||||
3. *
|
||||
*> table.put in=None at="ok" value=False*
|
||||
+3
-1
@@ -38,10 +38,12 @@ Construct a Document handle. Prefer `type: web` Artifact files discovered by `se
|
||||
+ `request`="json"
|
||||
+ `response`="json"
|
||||
+ `auth`=""
|
||||
+ `class`=""
|
||||
|
||||
Register a declared Document page or Endpoint (`fn` = `lib.member`).
|
||||
Optional `class`: `execute` | `short` | `read` | `control`.
|
||||
|
||||
**r = > route_mod.route app=`app` path=`path` method=`method` page=`page` fn=`fn` request=`request` response=`response` auth=`auth`**
|
||||
**r = > route_mod.route app=`app` path=`path` method=`method` page=`page` fn=`fn` request=`request` response=`response` auth=`auth` class=`class`**
|
||||
*> `r`.register app=`app`*
|
||||
|
||||
## render
|
||||
|
||||
+241
-7
@@ -77,7 +77,10 @@ fn resolve_marqdo_root(start: &str) -> PathBuf {
|
||||
}
|
||||
for _ in 0..8 {
|
||||
let cand = cur.join(".marqdo");
|
||||
if cand.join("engineering.yaml").exists() || cand.join("graph/graph.json").exists() {
|
||||
if cand.join("engineering.yaml").exists()
|
||||
|| cand.join("ekc/graph/graph.json").exists()
|
||||
|| cand.join("graph/graph.json").exists()
|
||||
{
|
||||
return cand;
|
||||
}
|
||||
if !cur.pop() {
|
||||
@@ -289,7 +292,10 @@ pub fn eng_preflight(args: &Value) -> Result<Value, String> {
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| resolve_marqdo_root(root));
|
||||
|
||||
if !marqdo.join("engineering.yaml").exists() && !marqdo.join("graph/graph.json").exists() {
|
||||
if !marqdo.join("engineering.yaml").exists()
|
||||
&& !marqdo.join("ekc/graph/graph.json").exists()
|
||||
&& !marqdo.join("graph/graph.json").exists()
|
||||
{
|
||||
return Ok(json!({
|
||||
"status": "missing_knowledge",
|
||||
"create_allowed": false,
|
||||
@@ -424,18 +430,246 @@ pub fn eng_preflight(args: &Value) -> Result<Value, String> {
|
||||
}))
|
||||
}
|
||||
|
||||
/// `agent_eng_reuse` — alias of preflight focused on decision payload.
|
||||
/// `agent_eng_reuse` — discovery + REUSE|ADAPT|CREATE decision.
|
||||
///
|
||||
/// Scores and capability names are **not** executable targets (G02/G07).
|
||||
/// When decision is REUSE/ADAPT, callers must `artifact.bind` a
|
||||
/// `bind_targets[]` entry before `artifact.execute`.
|
||||
pub fn eng_reuse(args: &Value) -> Result<Value, String> {
|
||||
let mut pf = eng_preflight(args)?;
|
||||
if let Some(obj) = pf.as_object_mut() {
|
||||
obj.insert(
|
||||
"protocol".into(),
|
||||
json!("REUSE|ADAPT|CREATE"),
|
||||
);
|
||||
obj.insert("protocol".into(), json!("REUSE|ADAPT|CREATE"));
|
||||
obj.insert("discovery_only".into(), json!(true));
|
||||
let decision = obj
|
||||
.get("decision")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_ascii_uppercase();
|
||||
let bind_required = matches!(decision.as_str(), "REUSE" | "ADAPT");
|
||||
obj.insert("bind_required".into(), json!(bind_required));
|
||||
let mut targets = Vec::new();
|
||||
if let Some(cands) = obj.get("candidates").and_then(|v| v.as_array()) {
|
||||
for c in cands {
|
||||
if let Some(impls) = c.get("implemented_by").and_then(|v| v.as_array()) {
|
||||
for t in impls {
|
||||
if let Some(s) = t.as_str() {
|
||||
if !s.is_empty() {
|
||||
targets.push(json!({
|
||||
"query": s,
|
||||
"via": "implemented_by",
|
||||
"capability": c.get("name"),
|
||||
"note": "pass query to artifact.bind then artifact.execute",
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(id) = c.get("id").and_then(|v| v.as_str()) {
|
||||
if id.contains(':') || id.contains('.') {
|
||||
targets.push(json!({
|
||||
"query": id,
|
||||
"via": "candidate_id",
|
||||
"capability": c.get("name"),
|
||||
"note": "pass query to artifact.bind then artifact.execute",
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// Dedup by query
|
||||
let mut seen = std::collections::HashSet::new();
|
||||
targets.retain(|t| {
|
||||
t.get("query")
|
||||
.and_then(|q| q.as_str())
|
||||
.map(|q| seen.insert(q.to_string()))
|
||||
.unwrap_or(false)
|
||||
});
|
||||
obj.insert("bind_targets".into(), json!(targets));
|
||||
if bind_required {
|
||||
let mut reasons = obj
|
||||
.get("reason")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!([]));
|
||||
if let Some(arr) = reasons.as_array_mut() {
|
||||
arr.push(json!(
|
||||
"scores are discovery-only; bind SymbolId/handle before execute (G02/G07)"
|
||||
));
|
||||
}
|
||||
obj.insert("reason".into(), reasons);
|
||||
}
|
||||
}
|
||||
Ok(pf)
|
||||
}
|
||||
|
||||
/// M04: discover → inspect → bind → execute → record (actual reuse, not n-gram REUSE).
|
||||
///
|
||||
/// Requires host_query `mlsp`. Returns trace with artifact_id, digests, inputs, result.
|
||||
/// Does **not** promote skills — callers pass `candidate_only=true` to skip learn/promote.
|
||||
pub fn eng_bind_execute(args: &Value) -> Result<Value, String> {
|
||||
let project = args
|
||||
.get("project")
|
||||
.or_else(|| args.get("root"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(".");
|
||||
let policy = args
|
||||
.get("policy")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("pure");
|
||||
let inputs = args.get("inputs").cloned().unwrap_or(json!({}));
|
||||
let query = if let Some(q) = args.get("artifact_id").or_else(|| args.get("query")).and_then(|v| v.as_str()) {
|
||||
q.to_string()
|
||||
} else {
|
||||
// Fall back to first bind_target from eng_reuse.
|
||||
let reuse = eng_reuse(args)?;
|
||||
let targets = reuse
|
||||
.get("bind_targets")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
targets
|
||||
.first()
|
||||
.and_then(|t| t.get("query").and_then(|q| q.as_str()))
|
||||
.ok_or_else(|| {
|
||||
"eng_bind_execute: no artifact_id/query and eng_reuse returned no bind_targets"
|
||||
.to_string()
|
||||
})?
|
||||
.to_string()
|
||||
};
|
||||
|
||||
let inspect = crate::host_query_json_args(
|
||||
"mlsp",
|
||||
&json!({"method":"artifact.inspect","params":{"project":project,"artifact_id":&query}}),
|
||||
)?;
|
||||
if inspect.get("ok") != Some(&json!(true)) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"stage": "inspect",
|
||||
"error": inspect.get("error").cloned().unwrap_or(json!("inspect failed")),
|
||||
"query": query,
|
||||
}));
|
||||
}
|
||||
let artifact_id = inspect["result"]["artifact_id"]
|
||||
.as_str()
|
||||
.unwrap_or(&query)
|
||||
.to_string();
|
||||
|
||||
let bound = crate::host_query_json_args(
|
||||
"mlsp",
|
||||
&json!({"method":"artifact.bind","params":{"project":project,"artifact_id":&artifact_id,"policy":policy}}),
|
||||
)?;
|
||||
if bound.get("ok") != Some(&json!(true)) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"stage": "bind",
|
||||
"error": bound.get("error").cloned().unwrap_or(json!("bind failed")),
|
||||
"artifact_id": artifact_id,
|
||||
"discovery_only": false,
|
||||
}));
|
||||
}
|
||||
let handle = bound["result"]["handle"].clone();
|
||||
|
||||
let executed = crate::host_query_json_args(
|
||||
"mlsp",
|
||||
&json!({"method":"artifact.execute","params":{"handle":&handle,"inputs":inputs}}),
|
||||
)?;
|
||||
if executed.get("ok") != Some(&json!(true)) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"stage": "execute",
|
||||
"error": executed.get("error").cloned().unwrap_or(json!("execute failed")),
|
||||
"artifact_id": artifact_id,
|
||||
"handle": handle,
|
||||
}));
|
||||
}
|
||||
|
||||
// Record only after successful bind+execute (actual reuse).
|
||||
let _ = eng_record(&json!({
|
||||
"decision": "REUSE",
|
||||
"root": project,
|
||||
"marqdo_dir": args.get("marqdo_dir"),
|
||||
}));
|
||||
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"stage": "executed",
|
||||
"discovery_only": false,
|
||||
"bind_required": false,
|
||||
"actual_reuse": true,
|
||||
"artifact_id": artifact_id,
|
||||
"symbol_id": handle.get("symbol_id"),
|
||||
"digest": handle.get("digest"),
|
||||
"evidence_digest": handle.get("evidence_digest"),
|
||||
"closure_digest": handle.get("closure_digest"),
|
||||
"policy": policy,
|
||||
"inputs": inputs,
|
||||
"result": executed.get("result"),
|
||||
"candidate_only": args.get("candidate_only").and_then(|v| v.as_bool()).unwrap_or(false),
|
||||
"trace": {
|
||||
"artifact_id": artifact_id,
|
||||
"symbol_id": handle.get("symbol_id"),
|
||||
"digest": handle.get("digest"),
|
||||
"inputs": inputs,
|
||||
"stdout": executed["result"]["stdout"],
|
||||
"value": executed["result"]["result"],
|
||||
},
|
||||
}))
|
||||
}
|
||||
|
||||
/// Gate promote/learn: when `candidate_only` or may_bind=false, refuse promotion.
|
||||
pub fn eng_publish_gate(args: &Value) -> Result<Value, String> {
|
||||
let candidate_only = matches!(args.get("candidate_only"), Some(Value::Bool(true)));
|
||||
let product_ok = !matches!(args.get("product_gate_ok"), Some(Value::Bool(false)));
|
||||
let project = args
|
||||
.get("project")
|
||||
.or_else(|| args.get("root"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(".");
|
||||
let policy = args
|
||||
.get("policy")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("business");
|
||||
let artifact = args
|
||||
.get("artifact_id")
|
||||
.or_else(|| args.get("query"))
|
||||
.and_then(|v| v.as_str());
|
||||
|
||||
if candidate_only {
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"may_promote": false,
|
||||
"reason": "candidate_only — framework must not promote",
|
||||
}));
|
||||
}
|
||||
if !product_ok {
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"may_promote": false,
|
||||
"reason": "product gate failed — framework must not promote",
|
||||
}));
|
||||
}
|
||||
if let Some(id) = artifact {
|
||||
let adj = crate::host_query_json_args(
|
||||
"mlsp",
|
||||
&json!({"method":"artifact.verify","params":{"project":project,"artifact_id":id,"policy":policy}}),
|
||||
)?;
|
||||
let may_bind = adj
|
||||
.pointer("/result/may_bind")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"may_promote": may_bind,
|
||||
"verification": adj.get("result"),
|
||||
"reason": if may_bind { "verification allows bind" } else { "verification denied bind" },
|
||||
}));
|
||||
}
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"may_promote": product_ok,
|
||||
"reason": "no artifact_id; deferred to product_gate_ok",
|
||||
}))
|
||||
}
|
||||
|
||||
/// `agent_eng_record` — append reuse metrics under `.marqdo/agent/episodes`.
|
||||
pub fn eng_record(args: &Value) -> Result<Value, String> {
|
||||
let decision = arg_text(args, "decision")?;
|
||||
|
||||
@@ -831,6 +831,8 @@ kb_ffi!(agent_resume_clear, resume::resume_clear, "agent_resume_clear");
|
||||
kb_ffi!(agent_resume_list, resume::resume_list, "agent_resume_list");
|
||||
kb_ffi!(agent_eng_preflight, eng::eng_preflight, "agent_eng_preflight");
|
||||
kb_ffi!(agent_eng_reuse, eng::eng_reuse, "agent_eng_reuse");
|
||||
kb_ffi!(agent_eng_bind_execute, eng::eng_bind_execute, "agent_eng_bind_execute");
|
||||
kb_ffi!(agent_eng_publish_gate, eng::eng_publish_gate, "agent_eng_publish_gate");
|
||||
kb_ffi!(agent_eng_record, eng::eng_record, "agent_eng_record");
|
||||
|
||||
#[no_mangle]
|
||||
@@ -900,6 +902,8 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
|
||||
("agent_maybe_compile_policy", "memory_dir,min_evidence", agent_maybe_compile_policy),
|
||||
("agent_eng_preflight", "task", agent_eng_preflight),
|
||||
("agent_eng_reuse", "task", agent_eng_reuse),
|
||||
("agent_eng_bind_execute", "project,query,policy", agent_eng_bind_execute),
|
||||
("agent_eng_publish_gate", "project,artifact_id,candidate_only", agent_eng_publish_gate),
|
||||
("agent_eng_record", "decision", agent_eng_record),
|
||||
("sandbox_open", "", sandbox::sandbox_open),
|
||||
("sandbox_write", "", sandbox::sandbox_write),
|
||||
|
||||
+476
-24
@@ -44,21 +44,40 @@ fn read_limited(mut r: impl Read) -> Result<String, String> {
|
||||
String::from_utf8(buf).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Prefer null over zero when the provider omitted a field (ModelCallUsageV1 / F04).
|
||||
fn token_or_null(v: Option<i64>) -> Value {
|
||||
match v {
|
||||
Some(n) if n >= 0 => json!(n),
|
||||
_ => Value::Null,
|
||||
}
|
||||
}
|
||||
|
||||
fn usage_from(data: &Value) -> Value {
|
||||
let u = data.get("usage").cloned().unwrap_or(Value::Null);
|
||||
let pt = u.get("prompt_tokens").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
let ct = u
|
||||
.get("completion_tokens")
|
||||
.and_then(|v| v.as_i64())
|
||||
.unwrap_or(0);
|
||||
if u.is_null() || !u.is_object() {
|
||||
return json!({
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
});
|
||||
}
|
||||
let pt = u.get("prompt_tokens").and_then(|v| v.as_i64());
|
||||
let ct = u.get("completion_tokens").and_then(|v| v.as_i64());
|
||||
let tt = u
|
||||
.get("total_tokens")
|
||||
.and_then(|v| v.as_i64())
|
||||
.unwrap_or(pt + ct);
|
||||
.or_else(|| match (pt, ct) {
|
||||
(Some(a), Some(b)) => Some(a + b),
|
||||
_ => None,
|
||||
});
|
||||
json!({
|
||||
"prompt_tokens": pt,
|
||||
"completion_tokens": ct,
|
||||
"total_tokens": tt,
|
||||
"prompt_tokens": token_or_null(pt),
|
||||
"completion_tokens": token_or_null(ct),
|
||||
"total_tokens": token_or_null(tt),
|
||||
"usage_state": "final",
|
||||
"measurement_source": "provider",
|
||||
})
|
||||
}
|
||||
|
||||
@@ -72,10 +91,101 @@ fn tool_calls_from(data: &Value) -> Value {
|
||||
.unwrap_or_else(|| json!([]))
|
||||
}
|
||||
|
||||
fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, String> {
|
||||
fn cancel_requested() -> bool {
|
||||
// Prefer request-scoped token handle. Empty / "0" ≠ cancel (M-R1).
|
||||
// When a token path is set, ignore process-global MARQDO_CANCEL so parallel
|
||||
// tasks cannot cancel each other via shared env.
|
||||
if let Ok(tok) = std::env::var("MARQDO_CANCEL_TOKEN") {
|
||||
if !tok.is_empty() {
|
||||
let path = std::path::Path::new(&tok);
|
||||
if path.is_file() {
|
||||
if let Ok(body) = std::fs::read_to_string(path) {
|
||||
let t = body.trim();
|
||||
return t == "1" || t.eq_ignore_ascii_case("cancelled");
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
// Legacy single-task CLI only when no token handle is bound.
|
||||
std::env::var("MARQDO_CANCEL")
|
||||
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// Mirror host `net::push_openai_delta_events`: append semantic frames to
|
||||
/// `MARQDO_EVENT_FORWARD` so subtask/web sse-live can flush tokens before the
|
||||
/// lib call returns (OpenAI-style incremental SSE).
|
||||
fn forward_event_bus_frame(ev: &Value) {
|
||||
let Ok(path) = std::env::var("MARQDO_EVENT_FORWARD") else {
|
||||
return;
|
||||
};
|
||||
if path.is_empty() {
|
||||
return;
|
||||
}
|
||||
use std::io::Write;
|
||||
if let Ok(mut f) = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(&path)
|
||||
{
|
||||
let _ = writeln!(f, "{ev}");
|
||||
let _ = f.flush();
|
||||
}
|
||||
}
|
||||
|
||||
/// Publish `{type:delta|plan|reasoning|error, text|message}` from one OpenAI chunk.
|
||||
/// `content_type` is usually `delta` (answer tokens) or `plan` (parent protocol LLM).
|
||||
fn forward_openai_chunk(data: &str, content_type: &str) {
|
||||
let data = data.trim();
|
||||
if data.is_empty() || data == "[DONE]" {
|
||||
return;
|
||||
}
|
||||
let Ok(v) = serde_json::from_str::<Value>(data) else {
|
||||
return;
|
||||
};
|
||||
if let Some(msg) = v.get("error").and_then(|e| {
|
||||
e.get("message")
|
||||
.and_then(|m| m.as_str())
|
||||
.or_else(|| e.as_str())
|
||||
}) {
|
||||
forward_event_bus_frame(&json!({"type": "error", "message": msg}));
|
||||
return;
|
||||
}
|
||||
let reasoning = v
|
||||
.pointer("/choices/0/delta/reasoning_content")
|
||||
.and_then(|c| c.as_str())
|
||||
.unwrap_or("");
|
||||
if !reasoning.is_empty() {
|
||||
forward_event_bus_frame(&json!({"type": "reasoning", "text": reasoning}));
|
||||
}
|
||||
let content = v
|
||||
.pointer("/choices/0/delta/content")
|
||||
.and_then(|c| c.as_str())
|
||||
.unwrap_or("");
|
||||
if !content.is_empty() {
|
||||
let ty = if content_type.is_empty() {
|
||||
"delta"
|
||||
} else {
|
||||
content_type
|
||||
};
|
||||
forward_event_bus_frame(&json!({"type": ty, "text": content}));
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_sse_events(
|
||||
reader: impl BufRead,
|
||||
echo: bool,
|
||||
content_type: &str,
|
||||
) -> Result<Vec<Value>, String> {
|
||||
let mut events = Vec::new();
|
||||
let mut data_lines: Vec<String> = Vec::new();
|
||||
let mut accumulated = String::new();
|
||||
for line in reader.lines() {
|
||||
if cancel_requested() {
|
||||
events.push(json!({"type": "cancelled", "message": "MARQDO_CANCEL"}));
|
||||
return Ok(events);
|
||||
}
|
||||
let line = line.map_err(|e| e.to_string())?;
|
||||
if echo {
|
||||
eprint!("{line}\n");
|
||||
@@ -85,9 +195,23 @@ fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, Stri
|
||||
let data = data_lines.join("\n");
|
||||
data_lines.clear();
|
||||
if data.trim() == "[DONE]" {
|
||||
// Parent owns run lifecycle `done`; only forward for answer streams.
|
||||
if content_type == "delta" || content_type.is_empty() {
|
||||
let done = json!({"type": "done", "result": accumulated});
|
||||
forward_event_bus_frame(&done);
|
||||
}
|
||||
events.push(json!({"type": "done", "data": "[DONE]"}));
|
||||
break;
|
||||
}
|
||||
forward_openai_chunk(&data, content_type);
|
||||
if let Ok(v) = serde_json::from_str::<Value>(&data) {
|
||||
if let Some(c) = v
|
||||
.pointer("/choices/0/delta/content")
|
||||
.and_then(|x| x.as_str())
|
||||
{
|
||||
accumulated.push_str(c);
|
||||
}
|
||||
}
|
||||
events.push(json!({"type": "data", "data": data}));
|
||||
}
|
||||
continue;
|
||||
@@ -99,21 +223,123 @@ fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, Stri
|
||||
if !data_lines.is_empty() {
|
||||
let data = data_lines.join("\n");
|
||||
if data.trim() == "[DONE]" {
|
||||
if content_type == "delta" || content_type.is_empty() {
|
||||
let done = json!({"type": "done", "result": accumulated});
|
||||
forward_event_bus_frame(&done);
|
||||
}
|
||||
events.push(json!({"type": "done", "data": "[DONE]"}));
|
||||
} else {
|
||||
forward_openai_chunk(&data, content_type);
|
||||
events.push(json!({"type": "data", "data": data}));
|
||||
}
|
||||
}
|
||||
Ok(events)
|
||||
}
|
||||
|
||||
fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
let base_url = arg_text(args, "base_url")?;
|
||||
let api_key = arg_text(args, "api_key").unwrap_or("");
|
||||
let model = arg_text(args, "model")?;
|
||||
fn build_messages(args: &Value) -> Result<Value, String> {
|
||||
if let Some(arr) = args.get("messages").and_then(|v| v.as_array()) {
|
||||
if arr.is_empty() {
|
||||
return Err("messages array must not be empty".into());
|
||||
}
|
||||
// Validate minimal role/content shape; reject unsupported part types loudly.
|
||||
for (i, m) in arr.iter().enumerate() {
|
||||
let role = m.get("role").and_then(|r| r.as_str()).unwrap_or("");
|
||||
if role.is_empty() {
|
||||
return Err(format!("llm.unsupported_input: messages[{i}].role missing"));
|
||||
}
|
||||
if let Some(parts) = m.get("content").and_then(|c| c.as_array()) {
|
||||
for (j, p) in parts.iter().enumerate() {
|
||||
let ty = p.get("type").and_then(|t| t.as_str()).unwrap_or("text");
|
||||
if ty != "text" {
|
||||
return Err(format!(
|
||||
"llm.unsupported_input: messages[{i}].content[{j}] type={ty}"
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return Ok(Value::Array(arr.clone()));
|
||||
}
|
||||
let prompt = arg_text(args, "prompt")?;
|
||||
Ok(json!([{"role": "user", "content": prompt}]))
|
||||
}
|
||||
|
||||
fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
if cancel_requested() {
|
||||
return Ok(json!({
|
||||
"text": "",
|
||||
"usage": {
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
},
|
||||
"finish": "cancelled",
|
||||
"error": "cancelled",
|
||||
"tool_calls": [],
|
||||
"transport": "plugin",
|
||||
}));
|
||||
}
|
||||
// Strict selection mode: refuse env fallback when selection.strict is set.
|
||||
let strict = opt_bool(args, "strict_selection", false)
|
||||
|| args
|
||||
.get("selection")
|
||||
.and_then(|s| s.get("strict"))
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
let selection = args.get("selection").cloned();
|
||||
if strict {
|
||||
if selection.is_none() {
|
||||
return Err("llm.selection_missing:selection".into());
|
||||
}
|
||||
let sel = selection.as_ref().unwrap();
|
||||
for key in ["selection_id", "provider_id", "model_id", "upstream_model_id"] {
|
||||
if sel
|
||||
.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.is_none()
|
||||
{
|
||||
return Err(format!("llm.selection_missing:{key}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let base_url = match arg_text(args, "base_url") {
|
||||
Ok(u) => u,
|
||||
Err(_) if strict => return Err("llm.selection_missing:base_url".into()),
|
||||
Err(e) => return Err(e),
|
||||
};
|
||||
let auth_mode = opt_text(args, "auth_mode")
|
||||
.or_else(|| {
|
||||
selection
|
||||
.as_ref()
|
||||
.and_then(|s| s.get("auth_mode"))
|
||||
.and_then(|v| v.as_str())
|
||||
})
|
||||
.unwrap_or("api_key");
|
||||
let api_key = if auth_mode == "none" {
|
||||
""
|
||||
} else {
|
||||
arg_text(args, "api_key").unwrap_or("")
|
||||
};
|
||||
if strict && auth_mode != "none" && api_key.is_empty() {
|
||||
return Err("llm.selection_missing:api_key".into());
|
||||
}
|
||||
let model = selection
|
||||
.as_ref()
|
||||
.and_then(|s| s.get("upstream_model_id"))
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(|s| s.to_string())
|
||||
.or_else(|| arg_text(args, "model").ok().map(|s| s.to_string()))
|
||||
.ok_or_else(|| "missing text `model`".to_string())?;
|
||||
let messages = build_messages(args)?;
|
||||
let stream = opt_bool(args, "stream", false);
|
||||
let echo = opt_bool(args, "echo", false);
|
||||
// Answer tokens → "delta"; parent plan protocol → "plan" (UI process trail).
|
||||
let stream_kind = opt_text(args, "stream_kind").unwrap_or("delta");
|
||||
let suffix = opt_text(args, "suffix").unwrap_or("/chat/completions");
|
||||
let bearer = opt_text(args, "bearer").unwrap_or("Bearer ");
|
||||
|
||||
@@ -123,11 +349,20 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
}
|
||||
|
||||
let mut req_body = Map::new();
|
||||
req_body.insert("model".into(), Value::String(model.to_string()));
|
||||
req_body.insert(
|
||||
"messages".into(),
|
||||
json!([{"role": "user", "content": prompt}]),
|
||||
);
|
||||
req_body.insert("model".into(), Value::String(model.clone()));
|
||||
req_body.insert("messages".into(), messages);
|
||||
if let Some(params) = selection
|
||||
.as_ref()
|
||||
.and_then(|s| s.get("parameters"))
|
||||
.and_then(|p| p.as_object())
|
||||
{
|
||||
if let Some(max) = params.get("max_output_tokens").and_then(|v| v.as_u64()) {
|
||||
req_body.insert("max_tokens".into(), json!(max));
|
||||
}
|
||||
if let Some(temp) = params.get("temperature") {
|
||||
req_body.insert("temperature".into(), temp.clone());
|
||||
}
|
||||
}
|
||||
if stream {
|
||||
req_body.insert("stream".into(), Value::Bool(true));
|
||||
}
|
||||
@@ -162,7 +397,8 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
"transport": "plugin",
|
||||
}));
|
||||
}
|
||||
let events = parse_sse_events(BufReader::new(r.into_reader()), echo)?;
|
||||
let events =
|
||||
parse_sse_events(BufReader::new(r.into_reader()), echo, stream_kind)?;
|
||||
Ok(json!({
|
||||
"events": events,
|
||||
"finish": "stop",
|
||||
@@ -193,11 +429,19 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
if status != 200 {
|
||||
return Ok(json!({
|
||||
"text": "",
|
||||
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
|
||||
"usage": {
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
},
|
||||
"finish": "error",
|
||||
"error": format!("HTTP {status}: {text}"),
|
||||
"tool_calls": [],
|
||||
"transport": "plugin",
|
||||
"requested_model": model,
|
||||
"resolved_model": model,
|
||||
}));
|
||||
}
|
||||
let data: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?;
|
||||
@@ -208,9 +452,12 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
.and_then(|c0| c0.get("message"))
|
||||
.cloned()
|
||||
.unwrap_or(json!({}));
|
||||
// Some gateways (reasoning models) return null content + reasoning_content.
|
||||
let content = msg
|
||||
.get("content")
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.or_else(|| msg.get("reasoning_content").and_then(|v| v.as_str()))
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let fr = data
|
||||
@@ -221,32 +468,62 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("stop")
|
||||
.to_string();
|
||||
let resolved = data
|
||||
.get("model")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(model.as_str())
|
||||
.to_string();
|
||||
Ok(json!({
|
||||
"text": content,
|
||||
"usage": usage_from(&data),
|
||||
"finish": fr,
|
||||
"tool_calls": tool_calls_from(&data),
|
||||
"transport": "plugin",
|
||||
"requested_model": model,
|
||||
"resolved_model": resolved,
|
||||
"selection": selection.as_ref().map(|s| {
|
||||
let mut snap = s.clone();
|
||||
if let Some(obj) = snap.as_object_mut() {
|
||||
obj.remove("api_key");
|
||||
}
|
||||
snap
|
||||
}),
|
||||
}))
|
||||
}
|
||||
Err(ureq::Error::Status(code, r)) => {
|
||||
let text = read_limited(r.into_reader()).unwrap_or_default();
|
||||
Ok(json!({
|
||||
"text": "",
|
||||
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
|
||||
"usage": {
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
},
|
||||
"finish": "error",
|
||||
"error": format!("HTTP {code}: {text}"),
|
||||
"tool_calls": [],
|
||||
"transport": "plugin",
|
||||
"requested_model": model,
|
||||
"resolved_model": model,
|
||||
}))
|
||||
}
|
||||
Err(e) => Ok(json!({
|
||||
"text": "",
|
||||
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
|
||||
"usage": {
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
},
|
||||
"finish": "error",
|
||||
"error": format!("http POST {url}: {e}"),
|
||||
"tool_calls": [],
|
||||
"transport": "plugin",
|
||||
"requested_model": model,
|
||||
"resolved_model": model,
|
||||
})),
|
||||
}
|
||||
}
|
||||
@@ -274,5 +551,180 @@ unsafe extern "C" fn llm_ping(
|
||||
|
||||
marqdo_plugin!(marqdo_plugin_sdk::ABI_VERSION, [
|
||||
("llm_ping", "", llm_ping),
|
||||
("llm_chat_completions", "base_url,api_key,model,prompt,stream,echo,suffix,bearer", llm_chat_completions),
|
||||
("llm_chat_completions", "base_url,api_key,model,prompt,messages,selection,strict_selection,auth_mode,stream,echo,suffix,bearer", llm_chat_completions),
|
||||
]);
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn f03_messages_preserve_roles_and_reject_image() {
|
||||
let msgs = build_messages(&json!({
|
||||
"messages": [
|
||||
{"role": "system", "content": "rules"},
|
||||
{"role": "user", "content": "你好"},
|
||||
{"role": "assistant", "content": "先前"},
|
||||
{"role": "user", "content": "最新用户"}
|
||||
]
|
||||
}))
|
||||
.unwrap();
|
||||
let arr = msgs.as_array().unwrap();
|
||||
assert_eq!(arr.len(), 4);
|
||||
assert_eq!(arr[3]["role"], "user");
|
||||
assert_eq!(arr[3]["content"], "最新用户");
|
||||
|
||||
let err = build_messages(&json!({
|
||||
"messages": [{"role": "user", "content": [{"type": "image", "url": "x"}]}]
|
||||
}))
|
||||
.unwrap_err();
|
||||
assert!(err.contains("unsupported_input"), "{err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f01_strict_selection_does_not_invent_env_model() {
|
||||
std::env::remove_var("MARQDO_CANCEL");
|
||||
std::env::remove_var("MARQDO_CANCEL_TOKEN");
|
||||
let err = chat_completions(&json!({
|
||||
"strict_selection": true,
|
||||
"base_url": "http://127.0.0.1:9",
|
||||
"api_key": "k",
|
||||
"prompt": "hi"
|
||||
// missing selection → must fail before any HTTP
|
||||
}))
|
||||
.unwrap_err();
|
||||
assert!(err.contains("selection_missing"), "{err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f06_cancel_short_circuits_before_http() {
|
||||
// Prefer token file over process-global MARQDO_CANCEL to avoid parallel test races.
|
||||
let tok = std::env::temp_dir().join(format!("llm-cancel-{}", std::process::id()));
|
||||
std::fs::write(&tok, "cancelled").unwrap();
|
||||
std::env::set_var("MARQDO_CANCEL_TOKEN", &tok);
|
||||
let out = chat_completions(&json!({
|
||||
"base_url": "http://127.0.0.1:9",
|
||||
"api_key": "k",
|
||||
"model": "m",
|
||||
"prompt": "hi"
|
||||
}));
|
||||
std::env::remove_var("MARQDO_CANCEL_TOKEN");
|
||||
let _ = std::fs::remove_file(&tok);
|
||||
let out = out.unwrap();
|
||||
assert_eq!(out["finish"], "cancelled");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f06_empty_cancel_token_is_not_cancelled() {
|
||||
// sse-live creates an empty token file; that must NOT abort the turn.
|
||||
let tok = std::env::temp_dir().join(format!("llm-cancel-empty-{}", std::process::id()));
|
||||
std::fs::write(&tok, "").unwrap();
|
||||
std::env::set_var("MARQDO_CANCEL_TOKEN", &tok);
|
||||
std::env::remove_var("MARQDO_CANCEL");
|
||||
assert!(!cancel_requested());
|
||||
std::env::remove_var("MARQDO_CANCEL_TOKEN");
|
||||
let _ = std::fs::remove_file(&tok);
|
||||
}
|
||||
|
||||
/// F01: two local endpoints receive distinct models/auth — no cross-talk.
|
||||
#[test]
|
||||
fn f01_dual_endpoint_isolation() {
|
||||
use std::io::{Read, Write};
|
||||
use std::net::TcpListener;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::thread;
|
||||
|
||||
fn spawn_mock(tag: &'static str, seen: Arc<Mutex<Vec<String>>>) -> String {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
thread::spawn(move || {
|
||||
if let Ok((mut stream, _)) = listener.accept() {
|
||||
let mut raw = Vec::new();
|
||||
let mut buf = [0u8; 4096];
|
||||
loop {
|
||||
let n = stream.read(&mut buf).unwrap_or(0);
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
raw.extend_from_slice(&buf[..n]);
|
||||
if let Some(pos) = raw.windows(4).position(|w| w == b"\r\n\r\n") {
|
||||
let header = String::from_utf8_lossy(&raw[..pos]);
|
||||
let cl = header
|
||||
.lines()
|
||||
.find_map(|l| l.strip_prefix("Content-Length:"))
|
||||
.and_then(|v| v.trim().parse::<usize>().ok())
|
||||
.unwrap_or(0);
|
||||
let body_start = pos + 4;
|
||||
while raw.len() < body_start + cl {
|
||||
let n = stream.read(&mut buf).unwrap_or(0);
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
raw.extend_from_slice(&buf[..n]);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
let req = String::from_utf8_lossy(&raw).to_string();
|
||||
seen.lock().unwrap().push(format!("{tag}:{req}"));
|
||||
let body = r#"{"choices":[{"message":{"content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1,"total_tokens":2}}"#;
|
||||
let resp = format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}",
|
||||
body.len(),
|
||||
body
|
||||
);
|
||||
let _ = stream.write_all(resp.as_bytes());
|
||||
}
|
||||
});
|
||||
format!("http://{addr}")
|
||||
}
|
||||
|
||||
let seen_a = Arc::new(Mutex::new(Vec::new()));
|
||||
let seen_b = Arc::new(Mutex::new(Vec::new()));
|
||||
let url_a = spawn_mock("A", seen_a.clone());
|
||||
let url_b = spawn_mock("B", seen_b.clone());
|
||||
|
||||
let out_a = chat_completions(&json!({
|
||||
"base_url": url_a,
|
||||
"api_key": "key-a",
|
||||
"model": "model-a",
|
||||
"auth_mode": "api_key",
|
||||
"messages": [{"role":"user","content":"from-a"}],
|
||||
"selection": {
|
||||
"selection_id": "sel-a",
|
||||
"provider_id": "p-a",
|
||||
"model_id": "m-a",
|
||||
"upstream_model_id": "model-a",
|
||||
"strict": true
|
||||
},
|
||||
"strict_selection": true
|
||||
}))
|
||||
.unwrap();
|
||||
let out_b = chat_completions(&json!({
|
||||
"base_url": url_b,
|
||||
"api_key": "key-b",
|
||||
"model": "model-b",
|
||||
"auth_mode": "api_key",
|
||||
"messages": [{"role":"user","content":"from-b"}],
|
||||
"selection": {
|
||||
"selection_id": "sel-b",
|
||||
"provider_id": "p-b",
|
||||
"model_id": "m-b",
|
||||
"upstream_model_id": "model-b",
|
||||
"strict": true
|
||||
},
|
||||
"strict_selection": true
|
||||
}))
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(out_a["finish"], "stop", "{out_a}");
|
||||
assert_eq!(out_b["finish"], "stop", "{out_b}");
|
||||
let a = seen_a.lock().unwrap().join("\n");
|
||||
let b = seen_b.lock().unwrap().join("\n");
|
||||
assert!(a.contains("model-a"), "A saw wrong model: {a}");
|
||||
assert!(a.contains("key-a") || a.contains("Authorization"), "{a}");
|
||||
assert!(!a.contains("model-b"), "A must not see B's model: {a}");
|
||||
assert!(b.contains("model-b"), "B saw wrong model: {b}");
|
||||
assert!(!b.contains("model-a"), "B must not see A's model: {b}");
|
||||
}
|
||||
}
|
||||
@@ -63,6 +63,11 @@ func ParseFrontmatter(src string) (meta map[string]string, body string) {
|
||||
body = body[i+3:]
|
||||
body = strings.TrimPrefix(body, "\r")
|
||||
body = strings.TrimPrefix(body, "\n")
|
||||
// Drop blank line(s) between closing --- and first content.
|
||||
for strings.HasPrefix(body, "\n") || strings.HasPrefix(body, "\r\n") {
|
||||
body = strings.TrimPrefix(body, "\r\n")
|
||||
body = strings.TrimPrefix(body, "\n")
|
||||
}
|
||||
}
|
||||
for _, line := range strings.Split(fm, "\n") {
|
||||
line = strings.TrimSpace(strings.TrimSuffix(line, "\r"))
|
||||
@@ -254,6 +259,10 @@ func SystemRoutes() []map[string]any {
|
||||
return []map[string]any{
|
||||
{"method": "GET", "path": "/_part/*", "kind": "system"},
|
||||
{"method": "GET|POST", "path": "/_form/*", "kind": "system"},
|
||||
{"method": "POST", "path": "/_runtime/tasks", "kind": "system"},
|
||||
{"method": "GET", "path": "/_runtime/tasks/{id}", "kind": "system"},
|
||||
{"method": "POST", "path": "/_runtime/tasks/{id}/cancel", "kind": "system"},
|
||||
{"method": "GET", "path": "/_runtime/tasks/{id}/events", "kind": "system"},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -275,8 +284,20 @@ func InspectBag(app map[string]any, declared []ArtifactFile) map[string]any {
|
||||
}
|
||||
}
|
||||
if inv, ok := app["invoke_routes"].(map[string]any); ok {
|
||||
for key := range inv {
|
||||
dec = append(dec, map[string]any{"path": key, "kind": "endpoint"})
|
||||
for key, v := range inv {
|
||||
row := map[string]any{"path": key, "kind": "endpoint"}
|
||||
if m, ok := v.(map[string]any); ok {
|
||||
if c, ok := m["class"].(string); ok && c != "" {
|
||||
row["class"] = c
|
||||
}
|
||||
if method, ok := m["method"].(string); ok && method != "" {
|
||||
row["method"] = method
|
||||
}
|
||||
if fn, ok := m["fn"].(string); ok && fn != "" {
|
||||
row["fn"] = fn
|
||||
}
|
||||
}
|
||||
dec = append(dec, row)
|
||||
}
|
||||
}
|
||||
if art, ok := app["artifact_routes"].(map[string]any); ok {
|
||||
|
||||
@@ -968,6 +968,54 @@ func Update(url, tableName, id string, row any, txnID ...string) (map[string]any
|
||||
return map[string]any{"ok": true, "updated": float64(n)}, nil
|
||||
}
|
||||
|
||||
// UpdateIf is a conditional update (CAS): SET row WHERE id=? AND expect_col=expect_val.
|
||||
// Returns {"ok":true,"updated":N,"matched":N}. Callers treat updated==0 as conflict.
|
||||
// Optional txnID routes through an open Begin transaction.
|
||||
func UpdateIf(url, tableName, id string, row any, expectCol string, expectVal any, txnID ...string) (map[string]any, error) {
|
||||
tn, err := ident(tableName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ecol, err := ident(expectCol)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
obj, ok := row.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("row must be a map")
|
||||
}
|
||||
c, err := connFor(url, optTxn(txnID))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var sets []string
|
||||
var vals []any
|
||||
for k, v := range obj {
|
||||
if k == "id" {
|
||||
continue
|
||||
}
|
||||
if _, err := ident(k); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sets = append(sets, fmt.Sprintf(`"%s" = ?`, k))
|
||||
vals = append(vals, toSQLArg(v))
|
||||
}
|
||||
if len(sets) == 0 {
|
||||
return nil, fmt.Errorf("nothing to update")
|
||||
}
|
||||
vals = append(vals, id, toSQLArg(expectVal))
|
||||
sqlStmt := fmt.Sprintf(
|
||||
`UPDATE "%s" SET %s WHERE "id" = ? AND "%s" = ?`,
|
||||
tn, strings.Join(sets, ", "), ecol,
|
||||
)
|
||||
res, err := c.Exec(sqlStmt, vals...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
n, _ := res.RowsAffected()
|
||||
return map[string]any{"ok": true, "updated": float64(n), "matched": float64(n)}, nil
|
||||
}
|
||||
|
||||
// Delete removes a row by id. Returns {"ok":true,"deleted":N}.
|
||||
// Optional txnID routes through an open Begin transaction.
|
||||
func Delete(url, tableName, id string, txnID ...string) (map[string]any, error) {
|
||||
|
||||
@@ -100,6 +100,24 @@ func TestInitInsertSelectGet(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpdateIfCAS(t *testing.T) {
|
||||
url := setupArticles(t)
|
||||
ok, err := db.UpdateIf(url, "articles", "1", map[string]any{"title": "CAS"}, "title", "Alpha")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ok["updated"] != float64(1) {
|
||||
t.Fatalf("cas hit=%v", ok)
|
||||
}
|
||||
miss, err := db.UpdateIf(url, "articles", "1", map[string]any{"title": "Nope"}, "title", "Alpha")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if miss["updated"] != float64(0) {
|
||||
t.Fatalf("cas miss should be 0, got %v", miss)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpdateDelete(t *testing.T) {
|
||||
url := setupArticles(t)
|
||||
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// F06: client-disconnect cancel bridge writes token file + MARQDO_CANCEL for llm plugin.
|
||||
func TestPropagateCancelSignal(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
tok := filepath.Join(dir, "cancel.tok")
|
||||
if err := os.WriteFile(tok, []byte("cancelled"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("MARQDO_CANCEL_TOKEN", tok)
|
||||
t.Setenv("MARQDO_CANCEL", "1")
|
||||
|
||||
if os.Getenv("MARQDO_CANCEL") != "1" {
|
||||
t.Fatalf("MARQDO_CANCEL not set")
|
||||
}
|
||||
body, err := os.ReadFile(tok)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(body) != "cancelled" {
|
||||
t.Fatalf("token body=%q", body)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestDrainForwardReadsIncrementalLines(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "evt.ndjson")
|
||||
if err := os.WriteFile(path, []byte("{\"type\":\"a\"}\n{\"type\":\"b\"}\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var lines []string
|
||||
off := drainForward(path, 0, func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 2 {
|
||||
t.Fatalf("got %d lines %#v", len(lines), lines)
|
||||
}
|
||||
if off <= 0 {
|
||||
t.Fatalf("offset=%d", off)
|
||||
}
|
||||
// Append after first drain — second call with offset must only see new line.
|
||||
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.WriteString("{\"type\":\"c\"}\n"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = f.Close()
|
||||
var more []string
|
||||
off2 := drainForward(path, off, func(line string) {
|
||||
more = append(more, line)
|
||||
})
|
||||
if len(more) != 1 || more[0] != "{\"type\":\"c\"}" {
|
||||
t.Fatalf("incremental got %#v off=%d→%d", more, off, off2)
|
||||
}
|
||||
}
|
||||
@@ -198,6 +198,35 @@ func invokeArgsFromPayload(payload map[string]any) map[string]any {
|
||||
return args
|
||||
}
|
||||
|
||||
// payloadWantsStream detects OpenAI-compatible stream:true on the request body.
|
||||
func payloadWantsStream(payload map[string]any) bool {
|
||||
if payload == nil {
|
||||
return false
|
||||
}
|
||||
if v, ok := payload["stream"]; ok {
|
||||
return boolishAny(v)
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func boolishAny(v any) bool {
|
||||
switch t := v.(type) {
|
||||
case bool:
|
||||
return t
|
||||
case string:
|
||||
s := strings.ToLower(strings.TrimSpace(t))
|
||||
return s == "1" || s == "true" || s == "yes" || s == "on"
|
||||
case float64:
|
||||
return t != 0
|
||||
case int:
|
||||
return t != 0
|
||||
case int64:
|
||||
return t != 0
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func readJSONBody(r *http.Request, max int64) (map[string]any, error) {
|
||||
r.Body = http.MaxBytesReader(nil, r.Body, max)
|
||||
defer r.Body.Close()
|
||||
|
||||
@@ -239,6 +239,7 @@ func NewHandler(appBag map[string]any, entryDir string) (http.Handler, error) {
|
||||
st.mountGalleryRoutes(mux)
|
||||
st.mountProxyRoutes(mux)
|
||||
st.mountInvokeRoutes(mux)
|
||||
st.mountRuntimeRoutes(mux)
|
||||
st.mountArtifactRoutes(mux)
|
||||
st.mountAuthRoutes(mux)
|
||||
|
||||
@@ -799,6 +800,13 @@ func (g *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
|
||||
return nil, nil, fmt.Errorf("underlying ResponseWriter does not support hijacking")
|
||||
}
|
||||
|
||||
func (g *gzipResponseWriter) Flush() {
|
||||
_ = g.w.Flush()
|
||||
if f, ok := g.ResponseWriter.(http.Flusher); ok {
|
||||
f.Flush()
|
||||
}
|
||||
}
|
||||
|
||||
type statusRecorder struct {
|
||||
http.ResponseWriter
|
||||
status int
|
||||
@@ -816,6 +824,14 @@ func (s *statusRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
|
||||
return nil, nil, fmt.Errorf("underlying ResponseWriter does not support hijacking")
|
||||
}
|
||||
|
||||
// Flush must be forwarded so sse-live / EventBus token streams are not buffered
|
||||
// behind access-log wrapping (otherwise clients see 0 bytes until the invoke ends).
|
||||
func (s *statusRecorder) Flush() {
|
||||
if f, ok := s.ResponseWriter.(http.Flusher); ok {
|
||||
f.Flush()
|
||||
}
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||
w.WriteHeader(code)
|
||||
|
||||
@@ -5,16 +5,39 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/plugin"
|
||||
)
|
||||
|
||||
// legacySSEBridgeMu serializes the process-global MARQDO_EVENT_FORWARD /
|
||||
// MARQDO_CANCEL_TOKEN bridge for the legacy_serial path only. Isolated execute
|
||||
// never acquires this lock (ADR 0009).
|
||||
var legacySSEBridgeMu sync.Mutex
|
||||
|
||||
// legacySSEBridgeActive is 1 while a holder has mutated process env for the bridge.
|
||||
var legacySSEBridgeActive atomic.Int32
|
||||
|
||||
func tryAcquireLegacySSEBridge() bool {
|
||||
return legacySSEBridgeMu.TryLock()
|
||||
}
|
||||
|
||||
func releaseLegacySSEBridge() {
|
||||
legacySSEBridgeActive.Store(0)
|
||||
legacySSEBridgeMu.Unlock()
|
||||
}
|
||||
|
||||
type invokeRoute struct {
|
||||
Method string
|
||||
FnPath string
|
||||
BodyMode string
|
||||
ReturnMode string
|
||||
Class string
|
||||
}
|
||||
|
||||
func parseInvokeRoute(v any) invokeRoute {
|
||||
@@ -26,11 +49,16 @@ func parseInvokeRoute(v any) invokeRoute {
|
||||
if fn == "" {
|
||||
fn = strOpt(m, "function", "")
|
||||
}
|
||||
class := strings.ToLower(strOpt(m, "class", ""))
|
||||
if class == "" {
|
||||
class = strings.ToLower(strOpt(m, "类别", ""))
|
||||
}
|
||||
return invokeRoute{
|
||||
Method: method,
|
||||
FnPath: fn,
|
||||
BodyMode: body,
|
||||
ReturnMode: ret,
|
||||
Class: class,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -102,7 +130,6 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
|
||||
}
|
||||
}
|
||||
|
||||
// Bind common auth / client headers for OpenAI-compatible and API-key surfaces.
|
||||
if auth := strings.TrimSpace(r.Header.Get("Authorization")); auth != "" {
|
||||
if _, exists := payload["authorization"]; !exists {
|
||||
payload["authorization"] = auth
|
||||
@@ -115,10 +142,51 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
|
||||
}
|
||||
|
||||
args := invokeArgsFromPayload(payload)
|
||||
|
||||
liveSSE := route.ReturnMode == "sse-live" || route.ReturnMode == "sse_live"
|
||||
if !liveSSE && payloadWantsStream(payload) {
|
||||
liveSSE = true
|
||||
}
|
||||
|
||||
class := resolveInvokeClass(route, liveSSE)
|
||||
switch class {
|
||||
case "execute":
|
||||
st.handleInvokeIsolated(w, r, route, args, liveSSE)
|
||||
return
|
||||
case "short", "read", "control":
|
||||
st.handleInvokeParent(w, r, route, args)
|
||||
return
|
||||
default:
|
||||
// legacy_serial: parent CallLibPath (ADR 0008). Never isolate undeclared non-stream.
|
||||
st.handleInvokeParent(w, r, route, args)
|
||||
}
|
||||
}
|
||||
|
||||
func resolveInvokeClass(route invokeRoute, liveSSE bool) string {
|
||||
c := strings.ToLower(strings.TrimSpace(route.Class))
|
||||
switch c {
|
||||
case "execute", "short", "read", "control":
|
||||
return c
|
||||
case "legacy", "legacy_serial":
|
||||
return "legacy_serial"
|
||||
}
|
||||
if liveSSE {
|
||||
return "execute"
|
||||
}
|
||||
return "legacy_serial"
|
||||
}
|
||||
|
||||
func (st *state) handleInvokeParent(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any) {
|
||||
_ = r
|
||||
value, err := plugin.CallLibPath(route.FnPath, args)
|
||||
if err != nil {
|
||||
code := http.StatusInternalServerError
|
||||
msg := err.Error()
|
||||
if strings.Contains(msg, "interpreter busy") {
|
||||
w.Header().Set("Retry-After", "2")
|
||||
jsonErr(w, http.StatusServiceUnavailable, msg)
|
||||
return
|
||||
}
|
||||
if strings.Contains(msg, "unknown library") ||
|
||||
strings.Contains(msg, "call_lib_path: need") ||
|
||||
strings.Contains(msg, "not allowed") ||
|
||||
@@ -133,8 +201,6 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
|
||||
return
|
||||
}
|
||||
|
||||
// OpenAI-compatible handlers may return `{sse: "..."}` to force event-stream
|
||||
// even when the route's declared return mode is json.
|
||||
if m, ok := value.(map[string]any); ok {
|
||||
if s, ok := m["sse"].(string); ok && strings.TrimSpace(s) != "" {
|
||||
writeSSE(w, m)
|
||||
@@ -176,3 +242,441 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
|
||||
writeJSON(w, code, body)
|
||||
}
|
||||
}
|
||||
|
||||
// handleInvokeIsolated runs execute-class invokes on a Supervisor worker (no
|
||||
// parent CallLibPath, no process Setenv, no legacySSEBridgeMu).
|
||||
func (st *state) handleInvokeIsolated(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any, liveSSE bool) {
|
||||
taskID := fmt.Sprintf("exec_%d", time.Now().UnixNano())
|
||||
adm, err := runtimeQuery("admit", map[string]any{
|
||||
"task_id": taskID,
|
||||
"idempotency_key": taskID,
|
||||
"class": "execute",
|
||||
"payload": map[string]any{"fn": route.FnPath},
|
||||
})
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("admit: %v", err))
|
||||
return
|
||||
}
|
||||
if ok, _ := adm["ok"].(bool); !ok {
|
||||
status := http.StatusServiceUnavailable
|
||||
if s, ok := adm["status"].(float64); ok && s >= 400 {
|
||||
status = int(s)
|
||||
}
|
||||
w.Header().Set("Retry-After", "2")
|
||||
jsonErr(w, status, fmt.Sprint(adm["error"]))
|
||||
return
|
||||
}
|
||||
cancelTok, _ := adm["cancel_token"].(string)
|
||||
eventPath, _ := adm["event_path"].(string)
|
||||
|
||||
entryModule, fsRoot, err := resolveSiteEntry(st)
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
argsFile, err := materializeArgsJSON(args)
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("args: %v", err))
|
||||
return
|
||||
}
|
||||
defer os.Remove(argsFile)
|
||||
|
||||
disp, err := runtimeQuery("dispatch", map[string]any{
|
||||
"task_id": taskID,
|
||||
"entry": map[string]any{
|
||||
"module": entryModule,
|
||||
"function": route.FnPath,
|
||||
},
|
||||
"fs_root": fsRoot,
|
||||
"args_json": argsFile,
|
||||
})
|
||||
if err != nil {
|
||||
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
|
||||
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("dispatch: %v", err))
|
||||
return
|
||||
}
|
||||
if ok, _ := disp["ok"].(bool); !ok {
|
||||
status := http.StatusTooManyRequests
|
||||
if s, ok := disp["status"].(float64); ok && s >= 400 {
|
||||
status = int(s)
|
||||
}
|
||||
if ra, ok := disp["Retry-After"]; ok {
|
||||
w.Header().Set("Retry-After", fmt.Sprint(ra))
|
||||
} else {
|
||||
w.Header().Set("Retry-After", "2")
|
||||
}
|
||||
jsonErr(w, status, fmt.Sprint(disp["error"]))
|
||||
return
|
||||
}
|
||||
if ep, ok := disp["event_path"].(string); ok && ep != "" {
|
||||
eventPath = ep
|
||||
}
|
||||
if tok, ok := disp["cancel_token"].(string); ok && tok != "" {
|
||||
cancelTok = tok
|
||||
}
|
||||
emitResult, _ := disp["emit_result"].(string)
|
||||
|
||||
if liveSSE {
|
||||
st.streamIsolatedSSE(w, r, route, taskID, eventPath, cancelTok, emitResult)
|
||||
return
|
||||
}
|
||||
st.waitIsolatedJSON(w, r, route, taskID, eventPath, cancelTok, emitResult)
|
||||
}
|
||||
|
||||
func (st *state) streamIsolatedSSE(w http.ResponseWriter, r *http.Request, route invokeRoute, taskID, eventPath, cancelTok, emitResult string) {
|
||||
w.Header().Set("Content-Type", "text/event-stream; charset=utf-8")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
w.Header().Set("X-Accel-Buffering", "no")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
flusher, _ := w.(http.Flusher)
|
||||
writeEvent := func(payload any) {
|
||||
b, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
_, _ = io.WriteString(w, "data: "+string(b)+"\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
writeEvent(map[string]any{
|
||||
"schema": "RuntimeEventV1",
|
||||
"schema_version": 1,
|
||||
"type": "run.started",
|
||||
"payload": map[string]any{
|
||||
"fn": route.FnPath,
|
||||
"bridge": "isolated",
|
||||
"task_id": taskID,
|
||||
},
|
||||
})
|
||||
|
||||
offset := int64(0)
|
||||
carry := ""
|
||||
seq := uint64(1)
|
||||
poll := time.NewTicker(25 * time.Millisecond)
|
||||
defer poll.Stop()
|
||||
ctx := r.Context()
|
||||
|
||||
finish := func(clientGone bool) {
|
||||
offset, carry = drainForwardCarry(eventPath, offset, carry, func(line string) {
|
||||
seq++
|
||||
writeEvent(normalizeLiveSSELine(seq, line))
|
||||
})
|
||||
if !clientGone {
|
||||
result := readEmitResult(emitResult)
|
||||
if result == nil {
|
||||
if g, err := runtimeQuery("get", map[string]any{"task_id": taskID}); err == nil {
|
||||
if t, ok := g["task"].(map[string]any); ok {
|
||||
result = t["result"]
|
||||
}
|
||||
}
|
||||
}
|
||||
writeEvent(map[string]any{
|
||||
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.completed",
|
||||
"payload": map[string]any{"result": result, "task_id": taskID},
|
||||
})
|
||||
_, _ = io.WriteString(w, "data: [DONE]\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
|
||||
_ = writeCancelToken(cancelTok)
|
||||
writeEvent(map[string]any{
|
||||
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.interrupted",
|
||||
"payload": map[string]any{
|
||||
"reason": "client_disconnect",
|
||||
"cancel_propagated": cancelTok != "",
|
||||
"status": "stopping",
|
||||
"task_id": taskID,
|
||||
},
|
||||
})
|
||||
// Brief wait so worker can notice cancel and flush.
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for time.Now().Before(deadline) {
|
||||
if workerDone(taskID, emitResult) {
|
||||
break
|
||||
}
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
}
|
||||
finish(true)
|
||||
return
|
||||
case <-poll.C:
|
||||
offset, carry = drainForwardCarry(eventPath, offset, carry, func(line string) {
|
||||
seq++
|
||||
writeEvent(normalizeLiveSSELine(seq, line))
|
||||
})
|
||||
if workerDone(taskID, emitResult) {
|
||||
finish(false)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (st *state) waitIsolatedJSON(w http.ResponseWriter, r *http.Request, route invokeRoute, taskID, eventPath, cancelTok, emitResult string) {
|
||||
_ = eventPath
|
||||
poll := time.NewTicker(25 * time.Millisecond)
|
||||
defer poll.Stop()
|
||||
ctx := r.Context()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
|
||||
_ = writeCancelToken(cancelTok)
|
||||
jsonErr(w, http.StatusRequestTimeout, "client disconnected")
|
||||
return
|
||||
case <-poll.C:
|
||||
if !workerDone(taskID, emitResult) {
|
||||
continue
|
||||
}
|
||||
value := readEmitResult(emitResult)
|
||||
if value == nil {
|
||||
jsonErr(w, http.StatusInternalServerError, "worker produced no result")
|
||||
return
|
||||
}
|
||||
if m, ok := value.(map[string]any); ok {
|
||||
if s, ok := m["sse"].(string); ok && strings.TrimSpace(s) != "" {
|
||||
writeSSE(w, m)
|
||||
return
|
||||
}
|
||||
}
|
||||
switch route.ReturnMode {
|
||||
case "text", "html", "text/html":
|
||||
text := fmt.Sprint(value)
|
||||
if s, ok := value.(string); ok {
|
||||
text = s
|
||||
}
|
||||
ct := "text/plain; charset=utf-8"
|
||||
if route.ReturnMode == "html" || route.ReturnMode == "text/html" {
|
||||
ct = "text/html; charset=utf-8"
|
||||
}
|
||||
w.Header().Set("Content-Type", ct)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = io.WriteString(w, text)
|
||||
case "status":
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
default:
|
||||
code, body, err := jsonWrapInvokeValue(value)
|
||||
if err != nil {
|
||||
jsonErr(w, code, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, code, body)
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func workerDone(taskID, emitResult string) bool {
|
||||
if emitResult != "" {
|
||||
if st, err := os.Stat(emitResult); err == nil && st.Size() > 0 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
// Sibling of event file written by Supervisor even if emit_result was omitted.
|
||||
if taskID != "" {
|
||||
if dir := os.Getenv("MARQDO_RUNTIME_DIR"); dir != "" {
|
||||
p := filepath.Join(dir, "events", taskID+".result.json")
|
||||
if st, err := os.Stat(p); err == nil && st.Size() > 0 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
g, err := runtimeQuery("get", map[string]any{"task_id": taskID})
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
if _, ok := g["exit_code"]; ok {
|
||||
return true
|
||||
}
|
||||
if t, ok := g["task"].(map[string]any); ok {
|
||||
switch fmt.Sprint(t["state"]) {
|
||||
case "succeeded", "failed", "cancelled", "interrupted":
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func readEmitResult(path string) any {
|
||||
if path == "" {
|
||||
return nil
|
||||
}
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil || len(b) == 0 {
|
||||
return nil
|
||||
}
|
||||
var v any
|
||||
if err := json.Unmarshal(b, &v); err != nil {
|
||||
return string(b)
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
func materializeArgsJSON(args map[string]any) (string, error) {
|
||||
f, err := os.CreateTemp("", "mq-args-*.json")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
path := f.Name()
|
||||
enc := json.NewEncoder(f)
|
||||
if err := enc.Encode(args); err != nil {
|
||||
_ = f.Close()
|
||||
_ = os.Remove(path)
|
||||
return "", err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
_ = os.Remove(path)
|
||||
return "", err
|
||||
}
|
||||
return path, nil
|
||||
}
|
||||
|
||||
func resolveSiteEntry(st *state) (module string, fsRoot string, err error) {
|
||||
fsRoot = st.entryDir
|
||||
if raw, qerr := plugin.Query("entry_dir", "{}"); qerr == nil {
|
||||
var s string
|
||||
if json.Unmarshal([]byte(raw), &s) == nil && s != "" {
|
||||
fsRoot = s
|
||||
}
|
||||
}
|
||||
if raw, qerr := plugin.Query("entry_path", "{}"); qerr == nil {
|
||||
var s string
|
||||
if json.Unmarshal([]byte(raw), &s) == nil && s != "" {
|
||||
module = s
|
||||
}
|
||||
}
|
||||
if module == "" {
|
||||
// Fallback: common serve entry names under fs_root.
|
||||
for _, name := range []string{"serve.mq.md", "index.mq.md", "main.mq.md"} {
|
||||
p := filepath.Join(fsRoot, name)
|
||||
if st, e := os.Stat(p); e == nil && !st.IsDir() {
|
||||
module = p
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if module == "" {
|
||||
return "", "", fmt.Errorf("isolated execute: cannot resolve site entry module")
|
||||
}
|
||||
if !filepath.IsAbs(module) {
|
||||
abs, aerr := filepath.Abs(module)
|
||||
if aerr == nil {
|
||||
module = abs
|
||||
}
|
||||
}
|
||||
if fsRoot == "" {
|
||||
fsRoot = filepath.Dir(module)
|
||||
}
|
||||
if !filepath.IsAbs(fsRoot) {
|
||||
abs, aerr := filepath.Abs(fsRoot)
|
||||
if aerr == nil {
|
||||
fsRoot = abs
|
||||
}
|
||||
}
|
||||
return module, fsRoot, nil
|
||||
}
|
||||
|
||||
// normalizeLiveSSELine maps Marqdo EventBus frames onto RuntimeEventV1.
|
||||
func normalizeLiveSSELine(seq uint64, line string) map[string]any {
|
||||
base := map[string]any{
|
||||
"schema": "RuntimeEventV1", "schema_version": 1, "seq": seq,
|
||||
}
|
||||
var obj map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &obj); err != nil || obj == nil {
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = map[string]any{"raw": line}
|
||||
return base
|
||||
}
|
||||
if t, _ := obj["type"].(string); t == "message.delta" || t == "turn.started" || t == "model.resolved" || t == "usage.updated" || t == "turn.completed" || t == "turn.failed" {
|
||||
for k, v := range obj {
|
||||
base[k] = v
|
||||
}
|
||||
base["seq"] = seq
|
||||
return base
|
||||
}
|
||||
if t, _ := obj["type"].(string); t == "delta" || t == "plan" || t == "reasoning" {
|
||||
text, _ := obj["text"].(string)
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = map[string]any{"delta": text, "kind": t}
|
||||
return base
|
||||
}
|
||||
if t, _ := obj["type"].(string); t == "decompose" || t == "decision" || t == "await" || t == "round" || t == "done" || t == "tool_start" || t == "tool_end" || t == "plan_act" {
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = obj
|
||||
return base
|
||||
}
|
||||
if text, ok := obj["text"].(string); ok && text != "" {
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = map[string]any{"delta": text}
|
||||
return base
|
||||
}
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = obj
|
||||
return base
|
||||
}
|
||||
|
||||
func drainForward(path string, offset int64, onLine func(string)) int64 {
|
||||
off, _ := drainForwardCarry(path, offset, "", onLine)
|
||||
return off
|
||||
}
|
||||
|
||||
const drainForwardMaxLine = 1024 * 1024
|
||||
|
||||
func drainForwardCarry(path string, offset int64, carry string, onLine func(string)) (int64, string) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return offset, carry
|
||||
}
|
||||
defer f.Close()
|
||||
if _, err := f.Seek(offset, io.SeekStart); err != nil {
|
||||
return offset, carry
|
||||
}
|
||||
chunk, err := io.ReadAll(io.LimitReader(f, drainForwardMaxLine+1))
|
||||
if err != nil && err != io.EOF {
|
||||
onLine(fmt.Sprintf(`{"type":"protocol.error","error":%q}`, err.Error()))
|
||||
return offset, carry
|
||||
}
|
||||
if len(chunk) == 0 {
|
||||
return offset, carry
|
||||
}
|
||||
offset += int64(len(chunk))
|
||||
buf := carry + string(chunk)
|
||||
for {
|
||||
i := strings.IndexByte(buf, '\n')
|
||||
if i < 0 {
|
||||
if len(buf) > drainForwardMaxLine {
|
||||
onLine(`{"type":"protocol.error","error":"ndjson line exceeds 1MiB"}`)
|
||||
return offset, ""
|
||||
}
|
||||
return offset, buf
|
||||
}
|
||||
lineBytes := buf[:i]
|
||||
line := strings.TrimSpace(strings.TrimSuffix(lineBytes, "\r"))
|
||||
buf = buf[i+1:]
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
onLine(line)
|
||||
}
|
||||
}
|
||||
|
||||
func legacySSEBridgeSnapshot() (active bool, forward string, cancelTok string) {
|
||||
return legacySSEBridgeActive.Load() != 0, os.Getenv("MARQDO_EVENT_FORWARD"), os.Getenv("MARQDO_CANCEL_TOKEN")
|
||||
}
|
||||
|
||||
func writeCancelToken(path string) error {
|
||||
if path == "" {
|
||||
return nil
|
||||
}
|
||||
return os.WriteFile(path, []byte("cancelled"), 0o600)
|
||||
}
|
||||
@@ -81,7 +81,16 @@ func (st *state) handleUpload(w http.ResponseWriter, r *http.Request, routePath
|
||||
_, hadCookie := session.IDFromCookie(r.Header.Get("Cookie"))
|
||||
sid, _, setCookie := resolveSession(r)
|
||||
|
||||
if err := r.ParseMultipartForm(32 << 20); err != nil {
|
||||
// M01: hard-limit request body before multipart parse (32MiB was only in-memory
|
||||
// form budget, not a request total). Cap at configured MaxBytes + 1MiB overhead.
|
||||
maxBody := int64(cfg.MaxBytes)
|
||||
if maxBody <= 0 {
|
||||
maxBody = 32 << 20
|
||||
}
|
||||
overhead := int64(1 << 20)
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxBody+overhead)
|
||||
|
||||
if err := r.ParseMultipartForm(maxBody + overhead); err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("multipart: %v", err))
|
||||
return
|
||||
}
|
||||
@@ -109,11 +118,20 @@ func (st *state) handleUpload(w http.ResponseWriter, r *http.Request, routePath
|
||||
return
|
||||
}
|
||||
defer f.Close()
|
||||
b, err := io.ReadAll(f)
|
||||
// Stream-capped read: never buffer more than MaxBytes.
|
||||
lim := maxBody
|
||||
if lim <= 0 {
|
||||
lim = 32 << 20
|
||||
}
|
||||
b, err := io.ReadAll(io.LimitReader(f, lim+1))
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("read file: %v", err))
|
||||
return
|
||||
}
|
||||
if int64(len(b)) > lim {
|
||||
jsonErr(w, http.StatusRequestEntityTooLarge, "file exceeds max_bytes")
|
||||
return
|
||||
}
|
||||
fileBytes = b
|
||||
} else {
|
||||
// Fallback: walk File map keys.
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestHandleUploadRejectsOversizedBeforeFullBuffer(t *testing.T) {
|
||||
st := &state{
|
||||
uploadRoutes: map[string]any{
|
||||
"/upload": map[string]any{
|
||||
"field": "file",
|
||||
"storage_url": "memory://t",
|
||||
"prefix": "u",
|
||||
"max_bytes": float64(64),
|
||||
},
|
||||
},
|
||||
}
|
||||
cfg := parseUploadRoute(st.uploadRoutes["/upload"])
|
||||
var body bytes.Buffer
|
||||
w := multipart.NewWriter(&body)
|
||||
part, err := w.CreateFormFile("file", "big.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := part.Write([]byte(strings.Repeat("x", 200))); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = w.Close()
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/upload", &body)
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
rr := httptest.NewRecorder()
|
||||
st.handleUpload(rr, req, "/upload", cfg)
|
||||
if rr.Code == http.StatusOK {
|
||||
t.Fatalf("expected reject for oversized upload, got %d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if rr.Code != http.StatusRequestEntityTooLarge && rr.Code != http.StatusBadRequest {
|
||||
t.Fatalf("expected 413 or 400, got %d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,207 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/plugin"
|
||||
)
|
||||
|
||||
// System runtime control routes (M-R3 / M-R4). Cancel and event replay do not
|
||||
// enter the Marqdo interpreter lock.
|
||||
func (st *state) mountRuntimeRoutes(mux *http.ServeMux) {
|
||||
mux.HandleFunc("POST /_runtime/tasks", st.handleRuntimeAdmit)
|
||||
mux.HandleFunc("GET /_runtime/tasks/{id}", st.handleRuntimeGet)
|
||||
mux.HandleFunc("POST /_runtime/tasks/{id}/cancel", st.handleRuntimeCancel)
|
||||
mux.HandleFunc("GET /_runtime/tasks/{id}/events", st.handleRuntimeEvents)
|
||||
}
|
||||
|
||||
func runtimeQuery(op string, extra map[string]any) (map[string]any, error) {
|
||||
payload := map[string]any{"op": op}
|
||||
for k, v := range extra {
|
||||
payload[k] = v
|
||||
}
|
||||
raw, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out, err := plugin.Query("runtime", string(raw))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal([]byte(out), &m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func (st *state) handleRuntimeAdmit(w http.ResponseWriter, r *http.Request) {
|
||||
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 1<<20))
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
var req map[string]any
|
||||
if len(body) > 0 {
|
||||
if err := json.Unmarshal(body, &req); err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("json: %v", err))
|
||||
return
|
||||
}
|
||||
}
|
||||
if req == nil {
|
||||
req = map[string]any{}
|
||||
}
|
||||
taskID, _ := req["task_id"].(string)
|
||||
if taskID == "" {
|
||||
taskID = fmt.Sprintf("task_%d", time.Now().UnixNano())
|
||||
req["task_id"] = taskID
|
||||
}
|
||||
if _, ok := req["idempotency_key"]; !ok {
|
||||
req["idempotency_key"] = taskID
|
||||
}
|
||||
if _, ok := req["class"]; !ok {
|
||||
req["class"] = "execute"
|
||||
}
|
||||
m, err := runtimeQuery("admit", req)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "conflict") || strings.Contains(fmt.Sprint(m["error"]), "conflict") {
|
||||
jsonErr(w, http.StatusConflict, err.Error())
|
||||
return
|
||||
}
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
if ok, _ := m["ok"].(bool); !ok {
|
||||
jsonErr(w, http.StatusConflict, fmt.Sprint(m["error"]))
|
||||
return
|
||||
}
|
||||
w.Header().Set("Location", "/_runtime/tasks/"+taskID)
|
||||
writeJSON(w, http.StatusAccepted, m)
|
||||
}
|
||||
|
||||
func (st *state) handleRuntimeGet(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
m, err := runtimeQuery("get", map[string]any{"task_id": id})
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
|
||||
jsonErr(w, http.StatusNotFound, "task not found")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
}
|
||||
|
||||
func (st *state) handleRuntimeCancel(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
m, err := runtimeQuery("cancel", map[string]any{"task_id": id})
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
|
||||
jsonErr(w, http.StatusNotFound, "task not found")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
}
|
||||
|
||||
func (st *state) handleRuntimeEvents(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
after := int64(0)
|
||||
if s := r.URL.Query().Get("after_seq"); s != "" {
|
||||
fmt.Sscanf(s, "%d", &after)
|
||||
}
|
||||
if last := strings.TrimSpace(r.Header.Get("Last-Event-ID")); last != "" {
|
||||
// Accept "task:seq" or bare seq.
|
||||
if i := strings.LastIndexByte(last, ':'); i >= 0 {
|
||||
fmt.Sscanf(last[i+1:], "%d", &after)
|
||||
} else {
|
||||
fmt.Sscanf(last, "%d", &after)
|
||||
}
|
||||
}
|
||||
wantSSE := strings.Contains(r.Header.Get("Accept"), "text/event-stream") ||
|
||||
r.URL.Query().Get("stream") == "1"
|
||||
|
||||
if !wantSSE {
|
||||
m, err := runtimeQuery("events", map[string]any{"task_id": id, "after_seq": after})
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
|
||||
jsonErr(w, http.StatusNotFound, "task not found")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/event-stream; charset=utf-8")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
flusher, _ := w.(http.Flusher)
|
||||
cursor := after
|
||||
ticker := time.NewTicker(25 * time.Millisecond)
|
||||
defer ticker.Stop()
|
||||
heartbeat := time.NewTicker(15 * time.Second)
|
||||
defer heartbeat.Stop()
|
||||
ctx := r.Context()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-heartbeat.C:
|
||||
_, _ = io.WriteString(w, ": ping\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
case <-ticker.C:
|
||||
m, err := runtimeQuery("events", map[string]any{"task_id": id, "after_seq": cursor})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if st, _ := m["status"].(float64); st == 404 {
|
||||
_, _ = io.WriteString(w, "event: error\ndata: {\"error\":\"not_found\"}\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
return
|
||||
}
|
||||
evs, _ := m["events"].([]any)
|
||||
for _, raw := range evs {
|
||||
b, err := json.Marshal(raw)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
var obj map[string]any
|
||||
_ = json.Unmarshal(b, &obj)
|
||||
eid := ""
|
||||
if s, ok := obj["event_id"].(string); ok {
|
||||
eid = s
|
||||
}
|
||||
if seq, ok := obj["seq"].(float64); ok {
|
||||
cursor = int64(seq)
|
||||
}
|
||||
etype, _ := obj["type"].(string)
|
||||
if etype == "" {
|
||||
etype = "message"
|
||||
}
|
||||
if eid != "" {
|
||||
_, _ = io.WriteString(w, "id: "+eid+"\n")
|
||||
}
|
||||
_, _ = io.WriteString(w, "event: "+etype+"\ndata: "+string(b)+"\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSystemRuntimeRoutesListed(t *testing.T) {
|
||||
// Compile-time presence: handlers must be addressable.
|
||||
var st state
|
||||
_ = st.handleRuntimeAdmit
|
||||
_ = st.handleRuntimeCancel
|
||||
_ = st.handleRuntimeEvents
|
||||
_ = st.handleRuntimeGet
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// M-R0: second stream must fail before rewriting A's forward/cancel env.
|
||||
func TestLegacySSEBridgeSingleFlight(t *testing.T) {
|
||||
if !tryAcquireLegacySSEBridge() {
|
||||
t.Fatal("expected first acquire to succeed")
|
||||
}
|
||||
fwdA := filepath.Join(t.TempDir(), "a.ndjson")
|
||||
tokA := filepath.Join(t.TempDir(), "a.tok")
|
||||
if err := os.WriteFile(fwdA, nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(tokA, nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("MARQDO_EVENT_FORWARD", fwdA)
|
||||
t.Setenv("MARQDO_CANCEL_TOKEN", tokA)
|
||||
legacySSEBridgeActive.Store(1)
|
||||
|
||||
if tryAcquireLegacySSEBridge() {
|
||||
releaseLegacySSEBridge()
|
||||
t.Fatal("second acquire must fail while bridge held")
|
||||
}
|
||||
active, fwd, tok := legacySSEBridgeSnapshot()
|
||||
if !active {
|
||||
t.Fatal("bridge should remain active for holder A")
|
||||
}
|
||||
if fwd != fwdA || tok != tokA {
|
||||
t.Fatalf("rejected second stream rewrote env: fwd=%q tok=%q", fwd, tok)
|
||||
}
|
||||
|
||||
releaseLegacySSEBridge()
|
||||
if tryAcquireLegacySSEBridge() {
|
||||
releaseLegacySSEBridge()
|
||||
} else {
|
||||
t.Fatal("bridge should be free after release")
|
||||
}
|
||||
}
|
||||
|
||||
// M-R0: disconnect marks cancel; token file must not be deleted while holder waits.
|
||||
func TestCancelTokenSurvivesUntilHolderExits(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
tok := filepath.Join(dir, "cancel.tok")
|
||||
if err := os.WriteFile(tok, nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := writeCancelToken(tok); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body, err := os.ReadFile(tok)
|
||||
if err != nil {
|
||||
t.Fatalf("token must still exist after cancel mark: %v", err)
|
||||
}
|
||||
if string(body) != "cancelled" {
|
||||
t.Fatalf("body=%q", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDrainForwardCarryHalfFrames(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "evt.ndjson")
|
||||
if err := os.WriteFile(path, []byte(`{"t":"部`), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var lines []string
|
||||
off, carry := drainForwardCarry(path, 0, "", func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 0 {
|
||||
t.Fatalf("incomplete UTF-8 frame must stay in carry, got %#v", lines)
|
||||
}
|
||||
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.WriteString("分\"}\n{\"t\":\"ok\"}\n"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = f.Close()
|
||||
off, carry = drainForwardCarry(path, off, carry, func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 2 {
|
||||
t.Fatalf("got %d lines %#v carry=%q off=%d", len(lines), lines, carry, off)
|
||||
}
|
||||
if carry != "" {
|
||||
t.Fatalf("carry should be empty, got %q", carry)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDrainForwardCarryCRLFAndSplit(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "evt.ndjson")
|
||||
part1 := []byte("{\"a\":1}\r\n{\"b\":")
|
||||
if err := os.WriteFile(path, part1, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var lines []string
|
||||
off, carry := drainForwardCarry(path, 0, "", func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 1 || lines[0] != `{"a":1}` {
|
||||
t.Fatalf("first frame %#v", lines)
|
||||
}
|
||||
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.WriteString("2}\n"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = f.Close()
|
||||
_, carry = drainForwardCarry(path, off, carry, func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 2 || lines[1] != `{"b":2}` {
|
||||
t.Fatalf("split frame %#v carry=%q", lines, carry)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacySSEBridgeConcurrentReject(t *testing.T) {
|
||||
var wg sync.WaitGroup
|
||||
acquired := 0
|
||||
var mu sync.Mutex
|
||||
for i := 0; i < 8; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if tryAcquireLegacySSEBridge() {
|
||||
mu.Lock()
|
||||
acquired++
|
||||
mu.Unlock()
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
releaseLegacySSEBridge()
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if acquired < 1 {
|
||||
t.Fatal("expected at least one acquirer")
|
||||
}
|
||||
}
|
||||
@@ -16,9 +16,10 @@ type Options struct {
|
||||
Fn string
|
||||
Body string
|
||||
Return string
|
||||
Class string
|
||||
}
|
||||
|
||||
// RoutesFromTable normalizes `|路径|方法|函数|正文|返回|` into invoke_routes map keys.
|
||||
// RoutesFromTable normalizes `|路径|方法|函数|正文|返回|类别|` into invoke_routes map keys.
|
||||
func RoutesFromTable(tableV any) map[string]any {
|
||||
rows, _ := table.AsRows(tableV).([]any)
|
||||
out := map[string]any{}
|
||||
@@ -47,7 +48,8 @@ func RoutesFromTable(tableV any) map[string]any {
|
||||
if ret == "" {
|
||||
ret = "json"
|
||||
}
|
||||
out[path] = routeEntry(method, fnPath, body, ret)
|
||||
class := strings.ToLower(strings.TrimSpace(col(m, "类别", "class", "Class")))
|
||||
out[path] = routeEntry(method, fnPath, body, ret, class)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -78,12 +80,13 @@ func Register(appBag map[string]any, opts Options) (map[string]any, error) {
|
||||
if ret == "" {
|
||||
ret = "json"
|
||||
}
|
||||
class := strings.ToLower(strings.TrimSpace(opts.Class))
|
||||
out := clone(appBag)
|
||||
routes := map[string]any{}
|
||||
if r, ok := out["invoke_routes"].(map[string]any); ok {
|
||||
routes = clone(r)
|
||||
}
|
||||
routes[key] = routeEntry(method, fnPath, body, ret)
|
||||
routes[key] = routeEntry(method, fnPath, body, ret, class)
|
||||
out["invoke_routes"] = routes
|
||||
return out, nil
|
||||
}
|
||||
@@ -101,22 +104,28 @@ func OptionsFromArgs(args map[string]any) (Options, error) {
|
||||
method, _ := argStr(args, "method", "方法")
|
||||
body, _ := argStr(args, "body", "正文")
|
||||
ret, _ := argStr(args, "return", "返回")
|
||||
class, _ := argStr(args, "class", "类别")
|
||||
return Options{
|
||||
Path: path,
|
||||
Method: method,
|
||||
Fn: fnPath,
|
||||
Body: body,
|
||||
Return: ret,
|
||||
Class: class,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func routeEntry(method, fnPath, body, ret string) map[string]any {
|
||||
return map[string]any{
|
||||
func routeEntry(method, fnPath, body, ret, class string) map[string]any {
|
||||
m := map[string]any{
|
||||
"method": method,
|
||||
"fn": fnPath,
|
||||
"body": body,
|
||||
"return": ret,
|
||||
}
|
||||
if class != "" {
|
||||
m["class"] = class
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func col(m map[string]any, keys ...string) string {
|
||||
|
||||
+37
-1
@@ -80,6 +80,7 @@ extern int web_compose_form(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_app_new(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_app_route(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_update(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_update_if(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_delete(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_query(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_count(char *args_json, char **out_json, char **err_msg);
|
||||
@@ -183,6 +184,7 @@ static int register_core(void) {
|
||||
if (host_register((char *)"web_db_select", (char *)"url,table,where,limit,order,offset,txn", web_db_select) != 0) return 1;
|
||||
if (host_register((char *)"web_db_get", (char *)"url,table,id,txn", web_db_get) != 0) return 1;
|
||||
if (host_register((char *)"web_db_update", (char *)"url,table,id,row,txn", web_db_update) != 0) return 1;
|
||||
if (host_register((char *)"web_db_update_if", (char *)"url,table,id,row,expect_col,expect_val,txn", web_db_update_if) != 0) return 1;
|
||||
if (host_register((char *)"web_db_delete", (char *)"url,table,id,txn", web_db_delete) != 0) return 1;
|
||||
if (host_register((char *)"web_db_exec", (char *)"url,sql,args,txn", web_db_exec) != 0) return 1;
|
||||
if (host_register((char *)"web_db_query", (char *)"url,sql,args,txn", web_db_query) != 0) return 1;
|
||||
@@ -264,7 +266,7 @@ static int register_core(void) {
|
||||
if (host_register((char *)"web_cache_exists", (char *)"url,key", web_cache_exists) != 0) return 1;
|
||||
if (host_register((char *)"web_cache_ttl", (char *)"url,key", web_cache_ttl) != 0) return 1;
|
||||
if (host_register((char *)"web_app_proxy", (char *)"app,path,upstream,stream,strip_prefix,methods,headers_from_env,timeout_ms", web_app_proxy) != 0) return 1;
|
||||
if (host_register((char *)"web_app_invoke", (char *)"app,path,method,fn,body,return", web_app_invoke) != 0) return 1;
|
||||
if (host_register((char *)"web_app_invoke", (char *)"app,path,method,fn,body,return,class", web_app_invoke) != 0) return 1;
|
||||
if (host_register((char *)"web_api_key_check", (char *)"key,authorization,pepper,keys", web_api_key_check) != 0) return 1;
|
||||
if (host_register((char *)"web_render_nodes", (char *)"page,nodes,db,title", web_render_nodes) != 0) return 1;
|
||||
if (host_register((char *)"web_render_document", (char *)"page,body,db,title", web_render_document) != 0) return 1;
|
||||
@@ -994,6 +996,40 @@ func web_db_update(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
return replyJSON(outJSON, errMsg, out, err)
|
||||
}
|
||||
|
||||
//export web_db_update_if
|
||||
func web_db_update_if(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
url, err := dbURLOf(args)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
tableName, err := argStrReq(args, "table")
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
id, err := argStrReq(args, "id")
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
expectCol, err := argStrReq(args, "expect_col")
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
expectVal, ok := args["expect_val"]
|
||||
if !ok {
|
||||
return replyJSON(outJSON, errMsg, nil, fmt.Errorf("missing expect_val"))
|
||||
}
|
||||
row := args["row"]
|
||||
if row == nil {
|
||||
row = map[string]any{}
|
||||
}
|
||||
out, err := db.UpdateIf(url, tableName, id, row, expectCol, expectVal, txnOpt(args))
|
||||
return replyJSON(outJSON, errMsg, out, err)
|
||||
}
|
||||
|
||||
//export web_db_delete
|
||||
func web_db_delete(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
|
||||
+36
-53
@@ -14,66 +14,49 @@ description: >-
|
||||
|
||||
Marqdo = **Markdown markers as syntax**. A `.mq.md` file is both documentation and a runnable program. Read this skill **before** inventing syntax from Markdown habits or Python.
|
||||
|
||||
Canonical design (repo): `doc/design/markdown-mapping.md`, `doc/design/keywords.md`. Deeper tables: [reference.md](reference.md). Copy-paste patterns: [examples.md](examples.md).
|
||||
<!-- CORE_SYNTAX_V2.1: must stay at the top so skill_brief budgets never truncate it -->
|
||||
|
||||
**Authoring / library craft** (GFM-first, no json glue, how to develop Marqdo): use **[marqdo-dev](../marqdo-dev/SKILL.md)** whenever writing or refactoring `.mq.md`, `lib/*`, or `ext/*`.
|
||||
## CORE SYNTAX (v2.1 Occlusion — mandatory)
|
||||
|
||||
**Product release** (tag, CHANGELOG, README, public docs, VSIX, GitHub assets): use [marqdo-release](../marqdo-release/SKILL.md) — detect latest version, **ask the user for the next SemVer**, then follow that playbook. Do not invent a version number.
|
||||
These rules are the **current default**. Do **not** teach Classic `[key](map)` index or “bold is return of print”. Classic migration only: [migration-classic.md](migration-classic.md).
|
||||
|
||||
1. **File suffix** `.mq.md`. Print with `> print text=…` / `> 打印 内容=…` — never use `**…**` to print.
|
||||
2. **`**…**` = code / return expression.** `*…*` = statement (bind / expr). Closing markers touch the last token (no trailing space).
|
||||
3. **`[描述](payload)` = occlusion:** `[]` is prose (never executed); `()` is the only executed part. Empty `()` → `None` stub.
|
||||
4. **Read collections with `access(coll, key)`** (aliases `取` / `取值`). **Never** use `[key](map)` as index under default Occlusion.
|
||||
5. **Foreach:** `- [描述](遍历 coll 为 item)` (`遍历`/`for`, `为`/`as`/`in`). Prefer explicit traversal when generating new code.
|
||||
6. **Import / call:** frontmatter `import:` / `导入:`; call `lib.member`. One language (EN or ZH) per file.
|
||||
7. **Control flow:** `#` object/type, `##`+ function, `+` params, `1.`/`2.` branches, `-` loops, `N. *` else. No invented `if`/`for`/`def`/`return` keywords.
|
||||
8. **Prose vs code:** blank line separates comment paragraphs from executable lines.
|
||||
9. **GFM tables** for data; prefer `table.put` over `json.set` chains. `ext/**` never calls `host_*`.
|
||||
10. Body narrative `[描述](target)` is lazy (NarrativeRef) — reading does not execute side effects.
|
||||
|
||||
| Marker | Meaning (v2.1) |
|
||||
|--------|----------------|
|
||||
| `#` / `##`+ | Object/type · function/method |
|
||||
| `*…*` | Statement |
|
||||
| `**…**` | Return / code expression |
|
||||
| `[描述](payload)` | Occlusion step (payload executes) |
|
||||
| `access(c, k)` | Collection read |
|
||||
| `> fn …` | Call |
|
||||
| GFM table | Collection literal |
|
||||
|
||||
Canonical: `doc/design/core-surface.md`, `doc/design/document-native-development-guide.md`. Deeper tables: [reference.md](reference.md). Patterns: [examples.md](examples.md). Authoring craft: [marqdo-dev](../marqdo-dev/SKILL.md). Release: [marqdo-release](../marqdo-release/SKILL.md).
|
||||
|
||||
Executable-knowledge gate: [`tests/executable_knowledge/`](../../tests/executable_knowledge/README.md).
|
||||
|
||||
## Hard rules (do not violate)
|
||||
|
||||
1. **File suffix** must be `.mq.md` (never plain `.md` for executable sources).
|
||||
2. **Output is not `**bold**`.** Print with `> print text=…` (or `> 打印 内容=…`). Bold `**…**` is **return value only**.
|
||||
3. **Prose vs code:** after a blank line, an unmarked first line starts a **comment paragraph**; every following non-blank line stays comment until the next blank line. Put a **blank line** between narration and executable lines.
|
||||
4. **Do not invent keywords** `if` / `else` / `while` / `for` / `def` / `return`. Control flow is Markdown: **`+` params** (under heading), **`1.` `2.` … branches**, **`-` loops**, arm `N. *` = else, `#` = **object/type**, `##`+ = **function/method**, `**…**` = return.
|
||||
5. **Identifiers use backticks** — params `` + `name` ``. **Exemptions:** foreach `` - [item](coll) ``; footnote `` name[^key] ``; **inside italic `*…*` / bold `**…**`**, bare ids are **variables** (Python-style unified namespace) and method-call receivers are bare too. **Drop the backticks on value-expression variable names and method receivers inside `*…*` / `**…**`** — they are redundant once the `*…*`/`**…**` markers mark the segment as code: `*分类 = 分类[^苹果]*`, `**n * 2**`, `*p = > page.主体装配 组件=home*`. **Text literals must be quoted**: `*a = > text.split value=src sep=","*`. Backticks **still required** where a bare word is *not* a variable in **standalone `>` calls** (bare = text there): `` > str `n` ``, `` > `obj`.method ``. **No trailing space** inside `*…*` / `**…**` wrapped code: the closing marker must touch the last token directly (`*a = 1*`, `**n**`), never `` *a = 1 * ``.
|
||||
6. **Structure lines are not wrapped in italics.** `#` `>` `+` `-` `|` lines stand alone; use `*…*` only for general statements (bindings / expressions).
|
||||
7. **Paths:** In bare *expressions* `/` is division. In **call args / param defaults / table cells**, unspaced `a/b` and quoted `".marqdo/agent-kb"` are path text — no `json.parse` needed. Numeric ratios that must stay text use quotes (`"1/5"`, `"16/9"`); bare `1/5` is division.
|
||||
8. Prefer ending side-effect-only function bodies with `*None*` / `*无*` / whole-line `**` / `****` when later lines must belong to an outer function. Lone `---` / `***` are Markdown thematic breaks (skipped), not function end.
|
||||
9. **`ext/**` never calls `host_*`.** Agent/OKF helpers are plugin names (`agent_kb_*`, …) after `plugin.load`. Do **not** add agent/OKF domain code to `src/host/` (core bloat). See `doc/design/ext-agent.md` §4.
|
||||
10. **Browser (route C/D/E/F):** client logic is Marqdo on WASM; official bridge is host glue and **may** implement lists, routing, storage, WebSocket, canvas, file read, observers — authors **must not hand-write business JS**. Use `web.client_embed` / `data-mq-source-url` auto-mount. Client effects: prefer **GFM tables + `lib/browser`** ([marqdo-dev](../marqdo-dev/SKILL.md)).
|
||||
11. **Code-as-documentation / no bag glue:** Prefer **GFM tables** for maps, lists, wire, commands. Prefer **`table.put` / named helpers** over `json.set` / `json.append` chains. `lib/json` is for parse/stringify/quote only — not a dict builder. Unreadable json pipelines are a style bug. Read collections with **`[key](map)`**, not footnote indexes for new code. In `markup: occlusion` files read with **`access(map, key)`** — there `[x](y)` is occlusion, not index. **Positive copy-paste shapes** (result map, soft-error pack, sparse `table.put`): [marqdo-dev](../marqdo-dev/SKILL.md) §2 · [examples.md](examples.md) §6e. Do **not** extend legacy `json.set` in `ext/ai` — refactor to tables.
|
||||
12. **One language per file:** Do not mix EN and ZH API names in the same `.mq.md` (imports, calls, params). Pick English *or* Chinese mirrors for the whole program.
|
||||
13. **Sandbox trial habit (Agent / generated `.mq.md`):** Prefer `agent.trial` / `run sandbox=True` / `sandbox.open`→`run`→`diff`→`commit` before promoting to the host tree. Host files must stay unchanged until a successful commit. Caps live in `sandbox.profile` GFM tables — do not invent a second dialect.
|
||||
1. Obey **CORE SYNTAX** above — Occlusion is default; do not emit Classic index links.
|
||||
2. **Identifiers:** params `` + `name` ``; inside `*…*` / `**…**` bare ids are variables. Quote text literals. Standalone `>` calls: backticks where bare would be text.
|
||||
3. **Structure lines** (`#` `>` `+` `-` `|`) are not wrapped in italics.
|
||||
4. **Paths:** bare `/` in expressions is division; in call args / table cells unspaced `a/b` is path text. Ratios as text need quotes (`"1/5"`).
|
||||
5. End side-effect-only helpers with `*None*` / `*空*` / `**` / `****` when the outer function continues. Lone `---` is a thematic break, not function end.
|
||||
6. **`ext/**` never calls `host_*`.** Prefer tables over `json.set` / `json.append`.
|
||||
7. **One language per file.** Sandbox trial before promoting Agent-generated trees.
|
||||
8. Browser: Marqdo on WASM + `lib/browser` tables; do not hand-write business JS.
|
||||
|
||||
|
||||
## Markup → meaning (v0.2)
|
||||
|
||||
| Marker | Meaning |
|
||||
|--------|---------|
|
||||
| `#` | **Object / type** (constructor body). `# main` = entry object. `# Child = > Parent` = inherit (**no** implicit parent constructor; call `` `self` = > Parent … `` explicitly when needed). |
|
||||
| `##` … `######` | **Function / method** (nesting by heading depth). |
|
||||
| `` + `name` `` under heading | Parameter (optional `` `name`=default ``) |
|
||||
| `1.` `2.` … in body | Branch arm (condition or `N. *` else). **Same-indent restart at `1.` = new branch statement** (not more arms of the previous list). |
|
||||
| `- …` inside body | Loop (`while` or `` [item](coll) `` / `` [`item`](`coll`) ``) |
|
||||
| Line `N. *` | Else arm |
|
||||
| `> fn args` | Call (named `k=v` or positional) |
|
||||
| `` > `obj`.method args `` | Method call (`obj` must be a map with `_type`) |
|
||||
| Frontmatter nested `import:` / `导入:` map | Import file (`bind: path.mq.md`) or short name (`bind: lib.member`) |
|
||||
| `> lib.member …` / `> lib.Type.member …` | Call via bare dotted path (instance methods need `` `var`.m ``) |
|
||||
| `*…*` | Statement (bind / expr) — closing `*` touches last token, **no trailing space**; bare ids are variables |
|
||||
| `**…**` | Return value — closing `**` touches last token, **no trailing space**; bare ids are variables |
|
||||
| `****` or `**` + spaces + `**` | Return `None` and end function body |
|
||||
| Lone `---` / `***` | Markdown thematic break (skip; not function end) |
|
||||
| GFM table after empty RHS bind | Collection; get with `[key](coll)` (preferred) or legacy `` `m`[^key] `` |
|
||||
| `- [item](coll)` | Foreach (unchanged) |
|
||||
| `` ```lang `` | Foreign code block (via `lib/foreign`) |
|
||||
| `` `"text"` `` | Quoted string (`\n` `\t` `\\` `\"`; `` `var` `` inside); bare tokens unescaped |
|
||||
| Unmarked prose | Comment |
|
||||
|
||||
Entry: load file (often `index.mq.md`) → collect defs → run `# main`.
|
||||
|
||||
## Occlusion syntax (v2.0.0)
|
||||
|
||||
A file with frontmatter `markup: occlusion` (or `标记: 遮蔽`) reads `[x](y)` as **occlusion**, not index — in that file only (`MARQDO_MARKUP=occlusion` sets the process default; classic files are unchanged).
|
||||
|
||||
- `[描述](实现)` — `[]` is prose (never executed); `()` is the **only** executed part.
|
||||
- `[顾客留存率](0.42)` · `[总价](at(价目表, "苹果") * 数量)` · `[功能开发中]()` → `None` (stub; `label.empty_payload` warning).
|
||||
- Read maps/lists with `access(集合, 键)` (aliases `取` / `取值`): `[第二项](access(列, 2))`.
|
||||
- Foreach: `- [篮子里的每个果子](遍历 篮子 为 果)` (accepts `遍历`/`for` and `为`/`as`/`in`).
|
||||
- Calls inside `()`: `加(1)` / `加(1, 2)` / `加一 号(31)` / `txt.str_trim(" x ")` — `)` terminates, commas optional.
|
||||
- Tooling: `marqdo labels pending` (stub ledger) · `marqdo labels --grep 词 --count` · `marqdo migrate labels . --write` · `marqdo check` reports `label.missing` / `label.empty_payload` / `label.ref_dangling`.
|
||||
|
||||
## Minimal program
|
||||
|
||||
```markdown
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
# Classic markup migration card (not default)
|
||||
|
||||
Use this card **only** when editing files that declare `markup: classic` / `标记: 经典`, or when the CLI is run with `--legacy-markup`. New documents must use Occlusion (see SKILL.md CORE SYNTAX).
|
||||
|
||||
| Classic (legacy) | Occlusion / current |
|
||||
|---|---|
|
||||
| `[key](map)` index read | `access(map, "key")` / `取(map, "key")` |
|
||||
| `- [item](coll)` foreach by index form | `- [描述](遍历 coll 为 item)` |
|
||||
| Teach `**…**` as “the return of print” | Print with `> print`; `**…**` is return/code expression |
|
||||
| Default link = index | Default `[描述](payload)` = occlusion; `[]` never executes |
|
||||
|
||||
Do not place Classic rules in the front of the default Skill pack. After migration, remove `markup: classic` and verify with `marqdo run` / `marqdo check`.
|
||||
+71
-18
@@ -28,11 +28,14 @@ struct FlatFun {
|
||||
path_index: HashMap<Vec<String>, usize>,
|
||||
/// Short name from frontmatter `use` → path segments.
|
||||
uses: HashMap<String, Vec<String>>,
|
||||
/// Module collection scope (G02): bare-name resolve must not cross imports.
|
||||
scope: u32,
|
||||
}
|
||||
|
||||
pub fn compile_module(path: Option<&Path>, module: &Module) -> Result<Program> {
|
||||
let mut flat = Vec::new();
|
||||
let _exports = collect_module_tree(module, &mut flat);
|
||||
let mut next_scope = 1u32;
|
||||
let _exports = collect_module_tree(module, &mut flat, 0, &mut next_scope);
|
||||
if flat.is_empty() {
|
||||
bail!("bytecode: no functions to compile");
|
||||
}
|
||||
@@ -106,10 +109,21 @@ pub fn compile_module(path: Option<&Path>, module: &Module) -> Result<Program> {
|
||||
|
||||
/// Collect `module` and its imports into `flat`.
|
||||
/// Returns relative export paths within this module (e.g. `["parse"]`, `["agent","step"]`).
|
||||
fn collect_module_tree(module: &Module, flat: &mut Vec<FlatFun>) -> HashMap<Vec<String>, usize> {
|
||||
///
|
||||
/// Each module gets a distinct `scope` id. Bare-name `resolve_call` only searches
|
||||
/// within the caller's scope so imported same-named functions cannot be silently
|
||||
/// selected (BoundCall / G02). Cross-module calls must use `bind.member` paths.
|
||||
fn collect_module_tree(
|
||||
module: &Module,
|
||||
flat: &mut Vec<FlatFun>,
|
||||
scope: u32,
|
||||
next_scope: &mut u32,
|
||||
) -> HashMap<Vec<String>, usize> {
|
||||
let mut path_index = HashMap::new();
|
||||
for (bind, dep) in &module.import_modules {
|
||||
let dep_exports = collect_module_tree(dep, flat);
|
||||
let child_scope = *next_scope;
|
||||
*next_scope += 1;
|
||||
let dep_exports = collect_module_tree(dep, flat, child_scope, next_scope);
|
||||
for (rel, idx) in dep_exports {
|
||||
let mut full = Vec::with_capacity(rel.len() + 1);
|
||||
full.push(bind.clone());
|
||||
@@ -124,11 +138,23 @@ fn collect_module_tree(module: &Module, flat: &mut Vec<FlatFun>) -> HashMap<Vec<
|
||||
.map(|u| (u.bind.clone(), u.path.clone()))
|
||||
.collect();
|
||||
|
||||
let start = flat.len();
|
||||
let mut exports = HashMap::new();
|
||||
for fun in &module.functions {
|
||||
let id = collect_fun(fun, None, flat, &path_index, &uses);
|
||||
let id = collect_fun(fun, None, flat, &path_index, &uses, scope);
|
||||
export_tree(&mut exports, &[], id, flat);
|
||||
}
|
||||
// Same-module BoundCall paths (`Type.member`) must be visible to siblings.
|
||||
for (rel, idx) in &exports {
|
||||
path_index.insert(rel.clone(), *idx);
|
||||
}
|
||||
for f in &mut flat[start..] {
|
||||
if f.scope == scope {
|
||||
for (rel, idx) in &exports {
|
||||
f.path_index.insert(rel.clone(), *idx);
|
||||
}
|
||||
}
|
||||
}
|
||||
exports
|
||||
}
|
||||
|
||||
@@ -152,6 +178,7 @@ fn collect_fun(
|
||||
flat: &mut Vec<FlatFun>,
|
||||
path_index: &HashMap<Vec<String>, usize>,
|
||||
uses: &HashMap<String, Vec<String>>,
|
||||
scope: u32,
|
||||
) -> usize {
|
||||
let id = flat.len();
|
||||
flat.push(FlatFun {
|
||||
@@ -166,10 +193,11 @@ fn collect_fun(
|
||||
span: fun.span,
|
||||
path_index: path_index.clone(),
|
||||
uses: uses.clone(),
|
||||
scope,
|
||||
});
|
||||
let mut child_ids = Vec::new();
|
||||
for child in &fun.children {
|
||||
child_ids.push(collect_fun(child, Some(id), flat, path_index, uses));
|
||||
child_ids.push(collect_fun(child, Some(id), flat, path_index, uses, scope));
|
||||
}
|
||||
flat[id].children = child_ids;
|
||||
id
|
||||
@@ -323,28 +351,42 @@ impl<'a> FnCompiler<'a> {
|
||||
s
|
||||
}
|
||||
|
||||
/// BoundCall-equivalent bare-name resolve (G02): only the caller's module
|
||||
/// scope. Imported functions are reachable solely via `path_index` paths.
|
||||
fn resolve_call(&self, name: &str) -> Result<usize> {
|
||||
let fun = &self.flat[self.fn_id];
|
||||
let scope = fun.scope;
|
||||
for &cid in &fun.children {
|
||||
if self.flat[cid].name == name {
|
||||
if self.flat[cid].name == name && self.flat[cid].scope == scope {
|
||||
return Ok(cid);
|
||||
}
|
||||
}
|
||||
let mut parent = fun.parent;
|
||||
while let Some(pid) = parent {
|
||||
if self.flat[pid].scope != scope {
|
||||
break;
|
||||
}
|
||||
for &cid in &self.flat[pid].children {
|
||||
if self.flat[cid].name == name {
|
||||
if self.flat[cid].name == name && self.flat[cid].scope == scope {
|
||||
return Ok(cid);
|
||||
}
|
||||
}
|
||||
parent = self.flat[pid].parent;
|
||||
}
|
||||
let mut matches = Vec::new();
|
||||
for (i, f) in self.flat.iter().enumerate() {
|
||||
if f.parent.is_none() && f.name == name {
|
||||
return Ok(i);
|
||||
if f.parent.is_none() && f.scope == scope && f.name == name {
|
||||
matches.push(i);
|
||||
}
|
||||
}
|
||||
Err(self.err(format!("unknown function `{name}`")))
|
||||
match matches.as_slice() {
|
||||
[only] => Ok(*only),
|
||||
[] => Err(self.err(format!("unknown function `{name}`"))),
|
||||
_ => Err(self.err(format!(
|
||||
"ambiguous function `{name}`: {} candidates in this module; use an explicit path",
|
||||
matches.len()
|
||||
))),
|
||||
}
|
||||
}
|
||||
|
||||
fn compile_stmt(&mut self, stmt: &Stmt) -> Result<()> {
|
||||
@@ -470,13 +512,20 @@ impl<'a> FnCompiler<'a> {
|
||||
}
|
||||
|
||||
fn compile_call(&mut self, call: &CallExpr, as_stmt: bool) -> Result<()> {
|
||||
// Mirror `sema::bind::resolve_for_exec` (G02 BoundCall):
|
||||
// static → Op::Call(fid); dynamic → MethodCall / CallFn / PluginCall.
|
||||
let mut call = call
|
||||
.with_modifiers_expanded()
|
||||
.map_err(|m| self.err(m))?;
|
||||
if let Some(recv) = &call.receiver {
|
||||
return self.compile_method_call(recv, &call, as_stmt);
|
||||
}
|
||||
if let Some(path) = &call.path {
|
||||
// Bare dotted callee `a.b`: if `a` is a local variable, it is a method
|
||||
// receiver (Python-style unified namespace); otherwise it's a library path.
|
||||
if path.len() == 2 && self.locals.contains_key(&path[0]) {
|
||||
// Local-var method sugar before import path (same as Tree / resolve_for_exec).
|
||||
if path.len() == 2
|
||||
&& self.locals.contains_key(&path[0])
|
||||
&& !self.flat[self.fn_id].path_index.keys().any(|p| p.first() == Some(&path[0]))
|
||||
{
|
||||
let recv = path[0].clone();
|
||||
return self.compile_method_call(&recv, &call, as_stmt);
|
||||
}
|
||||
@@ -488,10 +537,6 @@ impl<'a> FnCompiler<'a> {
|
||||
call.callee =
|
||||
crate::aliases::normalize_call_callee_and_args(&call.callee, &mut call.args);
|
||||
|
||||
if let Some(recv) = &call.receiver {
|
||||
return self.compile_method_call(recv, &call, as_stmt);
|
||||
}
|
||||
|
||||
match call.callee.as_str() {
|
||||
"print" => {
|
||||
let text_expr = call
|
||||
@@ -648,9 +693,17 @@ impl<'a> FnCompiler<'a> {
|
||||
_ => {}
|
||||
}
|
||||
|
||||
// BoundCall bare name (same-module scope only) or DynamicCall plugin.
|
||||
match self.resolve_call(&call.callee) {
|
||||
Ok(fid) => self.compile_user_call(fid, &call, as_stmt),
|
||||
Err(_) => self.compile_plugin_call(&call, as_stmt),
|
||||
Err(err) => {
|
||||
let msg = format!("{err:#}");
|
||||
if msg.contains("ambiguous function") {
|
||||
return Err(err);
|
||||
}
|
||||
// DynamicCall(Plugin): unresolved bare names may be runtime plugins.
|
||||
self.compile_plugin_call(&call, as_stmt)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,299 @@
|
||||
//! Document extraction protocol (M01): DocumentSource → ExtractedDocument.
|
||||
//!
|
||||
//! Format parsers live here / in extensions — language core does not OCR.
|
||||
//! First slice: plain text and Markdown. PDF/DOCX plug in later via the same shape.
|
||||
|
||||
use std::fs;
|
||||
use std::path::Path;
|
||||
|
||||
use sha2::{Digest, Sha256};
|
||||
use serde_json::{json, Value};
|
||||
|
||||
pub const EXTRACTOR_VERSION: &str = "marqdo-document-extract/1";
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DocumentSource {
|
||||
pub path: std::path::PathBuf,
|
||||
pub format_hint: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ExtractedSegment {
|
||||
pub id: String,
|
||||
pub kind: String,
|
||||
pub text: String,
|
||||
pub locator: String,
|
||||
pub confidence: f64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ExtractedDocument {
|
||||
pub source_uri: String,
|
||||
pub source_digest: String,
|
||||
pub format: String,
|
||||
pub extractor_version: String,
|
||||
pub text: String,
|
||||
pub segments: Vec<ExtractedSegment>,
|
||||
pub missing: Vec<String>,
|
||||
pub diagnostics: Vec<String>,
|
||||
pub status: String,
|
||||
}
|
||||
|
||||
impl ExtractedDocument {
|
||||
pub fn to_json(&self) -> Value {
|
||||
json!({
|
||||
"schema_version": 1,
|
||||
"source_uri": self.source_uri,
|
||||
"source_digest": self.source_digest,
|
||||
"format": self.format,
|
||||
"extractor_version": self.extractor_version,
|
||||
"status": self.status,
|
||||
"text": self.text,
|
||||
"segments": self.segments.iter().map(|s| json!({
|
||||
"id": s.id,
|
||||
"kind": s.kind,
|
||||
"text": s.text,
|
||||
"locator": s.locator,
|
||||
"confidence": s.confidence,
|
||||
})).collect::<Vec<_>>(),
|
||||
"missing": self.missing,
|
||||
"diagnostics": self.diagnostics,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Extract text/Markdown from a file. Uploaded `.mq.md` is treated as untrusted
|
||||
/// source text and is never executed here.
|
||||
pub fn extract_document(source: &DocumentSource) -> ExtractedDocument {
|
||||
let path = &source.path;
|
||||
let uri = path.to_string_lossy().replace('\\', "/");
|
||||
let ext = path
|
||||
.extension()
|
||||
.and_then(|e| e.to_str())
|
||||
.unwrap_or("")
|
||||
.to_ascii_lowercase();
|
||||
let name = path
|
||||
.file_name()
|
||||
.and_then(|n| n.to_str())
|
||||
.unwrap_or("")
|
||||
.to_ascii_lowercase();
|
||||
|
||||
let format = source
|
||||
.format_hint
|
||||
.clone()
|
||||
.unwrap_or_else(|| {
|
||||
if name.ends_with(".mq.md") || ext == "md" {
|
||||
"markdown".into()
|
||||
} else if matches!(ext.as_str(), "txt" | "text") {
|
||||
"text".into()
|
||||
} else if matches!(ext.as_str(), "pdf" | "docx") {
|
||||
ext.clone()
|
||||
} else {
|
||||
"unknown".into()
|
||||
}
|
||||
});
|
||||
|
||||
if matches!(
|
||||
format.as_str(),
|
||||
"pdf" | "docx" | "ocr" | "png" | "jpg" | "jpeg" | "webp" | "tiff"
|
||||
) || matches!(ext.as_str(), "pdf" | "docx" | "png" | "jpg" | "jpeg" | "webp" | "tiff")
|
||||
{
|
||||
let kind = if matches!(format.as_str(), "ocr")
|
||||
|| matches!(ext.as_str(), "png" | "jpg" | "jpeg" | "webp" | "tiff")
|
||||
{
|
||||
"ocr/image".to_string()
|
||||
} else {
|
||||
format.clone()
|
||||
};
|
||||
return ExtractedDocument {
|
||||
source_uri: uri,
|
||||
source_digest: String::new(),
|
||||
format,
|
||||
extractor_version: EXTRACTOR_VERSION.into(),
|
||||
text: String::new(),
|
||||
segments: vec![],
|
||||
missing: vec!["full_document".into()],
|
||||
diagnostics: vec![format!(
|
||||
"format `{kind}` requires an external extractor plugin (not in core); incomplete extraction is explicit failure"
|
||||
)],
|
||||
status: "failed".into(),
|
||||
};
|
||||
}
|
||||
|
||||
let bytes = match fs::read(path) {
|
||||
Ok(b) => b,
|
||||
Err(e) => {
|
||||
return ExtractedDocument {
|
||||
source_uri: uri,
|
||||
source_digest: String::new(),
|
||||
format,
|
||||
extractor_version: EXTRACTOR_VERSION.into(),
|
||||
text: String::new(),
|
||||
segments: vec![],
|
||||
missing: vec!["full_document".into()],
|
||||
diagnostics: vec![format!("read error: {e}")],
|
||||
status: "failed".into(),
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
if bytes.is_empty() {
|
||||
return ExtractedDocument {
|
||||
source_uri: uri,
|
||||
source_digest: digest_bytes(&bytes),
|
||||
format,
|
||||
extractor_version: EXTRACTOR_VERSION.into(),
|
||||
text: String::new(),
|
||||
segments: vec![],
|
||||
missing: vec!["content".into()],
|
||||
diagnostics: vec!["empty document".into()],
|
||||
status: "failed".into(),
|
||||
};
|
||||
}
|
||||
|
||||
// Resource limit: refuse to fully buffer huge uploads as text (M01).
|
||||
const MAX_EXTRACT_BYTES: usize = 8 << 20;
|
||||
if bytes.len() > MAX_EXTRACT_BYTES {
|
||||
let head = &bytes[..MAX_EXTRACT_BYTES];
|
||||
let lossy = String::from_utf8_lossy(head).into_owned();
|
||||
let mut doc = extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &lossy);
|
||||
doc.missing.push("tail_beyond_limit".into());
|
||||
doc.diagnostics.push(format!(
|
||||
"document truncated at {MAX_EXTRACT_BYTES} bytes (got {}); extraction is partial",
|
||||
bytes.len()
|
||||
));
|
||||
doc.status = "partial".into();
|
||||
return doc;
|
||||
}
|
||||
|
||||
let text = match String::from_utf8(bytes.clone()) {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
let lossy = String::from_utf8_lossy(&bytes).into_owned();
|
||||
let mut doc = extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &lossy);
|
||||
doc.diagnostics
|
||||
.push(format!("invalid utf-8 ({e}); decoded lossy"));
|
||||
doc.status = if doc.text.is_empty() {
|
||||
"failed".into()
|
||||
} else {
|
||||
"partial".into()
|
||||
};
|
||||
return doc;
|
||||
}
|
||||
};
|
||||
|
||||
extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &text)
|
||||
}
|
||||
|
||||
fn extract_markdown_or_text(
|
||||
uri: &str,
|
||||
digest: &str,
|
||||
format: &str,
|
||||
text: &str,
|
||||
) -> ExtractedDocument {
|
||||
let mut segments = Vec::new();
|
||||
let mut n = 0u32;
|
||||
let mut byte_off = 0usize;
|
||||
for (i, line) in text.lines().enumerate() {
|
||||
let line_no = i + 1;
|
||||
let line_bytes = line.len();
|
||||
// M05: stable fragment locator with 1-based line + UTF-8 byte offset.
|
||||
let loc = |kind: &str| format!("{kind}:L{line_no}:B{byte_off}");
|
||||
if let Some(rest) = line.trim().strip_prefix('#') {
|
||||
n += 1;
|
||||
let heading = rest.trim_start_matches('#').trim();
|
||||
segments.push(ExtractedSegment {
|
||||
id: format!("seg-{n}"),
|
||||
kind: "heading".into(),
|
||||
text: heading.to_string(),
|
||||
locator: loc("heading"),
|
||||
confidence: 1.0,
|
||||
});
|
||||
} else if line.trim().starts_with('|') {
|
||||
n += 1;
|
||||
segments.push(ExtractedSegment {
|
||||
id: format!("seg-{n}"),
|
||||
kind: "table_row".into(),
|
||||
text: line.to_string(),
|
||||
locator: loc("table_row"),
|
||||
confidence: 0.9,
|
||||
});
|
||||
} else if !line.trim().is_empty() {
|
||||
n += 1;
|
||||
segments.push(ExtractedSegment {
|
||||
id: format!("seg-{n}"),
|
||||
kind: "paragraph".into(),
|
||||
text: line.to_string(),
|
||||
locator: loc("paragraph"),
|
||||
confidence: 1.0,
|
||||
});
|
||||
}
|
||||
// +1 for '\n' when present in original (lines() strips it).
|
||||
byte_off += line_bytes + 1;
|
||||
}
|
||||
ExtractedDocument {
|
||||
source_uri: uri.to_string(),
|
||||
source_digest: digest.to_string(),
|
||||
format: format.to_string(),
|
||||
extractor_version: EXTRACTOR_VERSION.into(),
|
||||
text: text.to_string(),
|
||||
segments,
|
||||
missing: vec![],
|
||||
diagnostics: vec![],
|
||||
status: "ok".into(),
|
||||
}
|
||||
}
|
||||
|
||||
fn digest_bytes(bytes: &[u8]) -> String {
|
||||
let mut h = Sha256::new();
|
||||
h.update(bytes);
|
||||
format!("sha256:{:x}", h.finalize())
|
||||
}
|
||||
|
||||
/// Convenience for tests / CLI: extract if path exists.
|
||||
pub fn extract_path(path: &Path) -> ExtractedDocument {
|
||||
extract_document(&DocumentSource {
|
||||
path: path.to_path_buf(),
|
||||
format_hint: None,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::io::Write;
|
||||
|
||||
#[test]
|
||||
fn extracts_markdown_headings() {
|
||||
let dir = std::env::temp_dir().join(format!("mq-extract-{}", std::process::id()));
|
||||
let _ = fs::create_dir_all(&dir);
|
||||
let path = dir.join("note.md");
|
||||
let mut f = fs::File::create(&path).unwrap();
|
||||
writeln!(f, "# rate\n\nhello").unwrap();
|
||||
let doc = extract_path(&path);
|
||||
assert_eq!(doc.status, "ok");
|
||||
assert!(doc.segments.iter().any(|s| s.kind == "heading" && s.text == "rate"));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pdf_reports_missing_extractor() {
|
||||
let doc = extract_document(&DocumentSource {
|
||||
path: Path::new("x.pdf").into(),
|
||||
format_hint: Some("pdf".into()),
|
||||
});
|
||||
assert_eq!(doc.status, "failed");
|
||||
assert!(!doc.diagnostics.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ocr_image_reports_missing_extractor() {
|
||||
let doc = extract_document(&DocumentSource {
|
||||
path: Path::new("scan.png").into(),
|
||||
format_hint: Some("ocr".into()),
|
||||
});
|
||||
assert_eq!(doc.status, "failed");
|
||||
assert!(doc.missing.contains(&"full_document".into()));
|
||||
assert!(doc.diagnostics.iter().any(|d| d.contains("ocr") || d.contains("extractor")));
|
||||
}
|
||||
}
|
||||
+5
-2
@@ -326,8 +326,11 @@ pub fn apply_patch_blocks(
|
||||
};
|
||||
match apply_patch_blocks_inner(ctx, path, text) {
|
||||
Ok(n) => Ok(n),
|
||||
// soft only swallows unique-FIND miss — never parse / multi-match / whole-file.
|
||||
Err(e) if soft && e.contains("FIND not found") => Ok(Value::Int(0)),
|
||||
// soft swallows unique-FIND miss and empty FIND (model protocol mistakes).
|
||||
// Never softens parse / multi-match / whole-file rewrite.
|
||||
Err(e) if soft && (e.contains("FIND not found") || e.contains("empty FIND")) => {
|
||||
Ok(Value::Int(0))
|
||||
}
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
}
|
||||
|
||||
+113
-11
@@ -6,7 +6,8 @@ use std::collections::HashMap;
|
||||
use std::ffi::{CStr, CString};
|
||||
use std::os::raw::{c_char, c_int, c_void};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Mutex, OnceLock};
|
||||
use std::sync::{Mutex, MutexGuard, OnceLock};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use libloading::{Library, Symbol};
|
||||
|
||||
@@ -75,6 +76,8 @@ static GLOBAL_HOST: OnceLock<Mutex<SendPtr<HostContext>>> = OnceLock::new();
|
||||
/// Process-wide lib-path hook (same lifetime as GLOBAL_HOST).
|
||||
static GLOBAL_LIB_PATH: OnceLock<Mutex<SendLibPath>> = OnceLock::new();
|
||||
/// Serialize interpreter re-entry from HTTP worker threads.
|
||||
/// HostContext is not Sync — one Marqdo invoke at a time. Short routes must
|
||||
/// fail fast (`interpreter busy`) so a long SSE chat cannot freeze the UI.
|
||||
static INVOKE_LOCK: OnceLock<Mutex<()>> = OnceLock::new();
|
||||
|
||||
fn global_host_slot() -> &'static Mutex<SendPtr<HostContext>> {
|
||||
@@ -89,6 +92,41 @@ fn invoke_lock() -> &'static Mutex<()> {
|
||||
INVOKE_LOCK.get_or_init(|| Mutex::new(()))
|
||||
}
|
||||
|
||||
/// Default wait for short HTTP invokes (`MARQDO_INVOKE_WAIT_MS`, default 1500ms).
|
||||
/// Do not raise this process-wide during SSE — concurrent short routes would hang again.
|
||||
/// See doc/adr/0008-web-invoke-serialization.md.
|
||||
fn invoke_wait_budget() -> Duration {
|
||||
const DEFAULT_MS: u64 = 1500;
|
||||
match std::env::var("MARQDO_INVOKE_WAIT_MS") {
|
||||
Ok(s) => {
|
||||
let ms = s.trim().parse::<u64>().unwrap_or(DEFAULT_MS);
|
||||
Duration::from_millis(ms.max(1))
|
||||
}
|
||||
Err(_) => Duration::from_millis(DEFAULT_MS),
|
||||
}
|
||||
}
|
||||
|
||||
fn acquire_invoke_lock() -> Result<MutexGuard<'static, ()>, String> {
|
||||
let budget = invoke_wait_budget();
|
||||
let deadline = Instant::now() + budget;
|
||||
let lock = invoke_lock();
|
||||
loop {
|
||||
match lock.try_lock() {
|
||||
Ok(g) => return Ok(g),
|
||||
Err(std::sync::TryLockError::Poisoned(p)) => return Ok(p.into_inner()),
|
||||
Err(std::sync::TryLockError::WouldBlock) => {
|
||||
if Instant::now() >= deadline {
|
||||
return Err(format!(
|
||||
"interpreter busy: another request holds the Marqdo invoke lock (waited {}ms)",
|
||||
budget.as_millis()
|
||||
));
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(15));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Hook: resolve bare `lib.member` (and longer paths) in the site module's import tree.
|
||||
#[derive(Clone, Copy)]
|
||||
pub struct LibPathCall {
|
||||
@@ -284,6 +322,42 @@ unsafe extern "C" fn host_query(
|
||||
}
|
||||
};
|
||||
|
||||
let args_owned: Option<String> = if args_json.is_null() {
|
||||
None
|
||||
} else {
|
||||
CStr::from_ptr(args_json)
|
||||
.to_str()
|
||||
.ok()
|
||||
.map(|s| s.to_string())
|
||||
};
|
||||
|
||||
// Control plane (admit/cancel/events) must not take INVOKE_LOCK or require
|
||||
// an evaluating HostContext — otherwise cancel cannot interrupt a busy run.
|
||||
if name == "runtime" {
|
||||
let raw = args_owned.as_deref().unwrap_or("{}");
|
||||
let args: serde_json::Value = match serde_json::from_str(if raw.trim().is_empty() {
|
||||
"{}"
|
||||
} else {
|
||||
raw
|
||||
}) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
set_err(&format!("runtime args: {e}"));
|
||||
return 1;
|
||||
}
|
||||
};
|
||||
match crate::runtime::host_api::handle(&args) {
|
||||
Ok(j) => {
|
||||
set_out(&j.to_string());
|
||||
return 0;
|
||||
}
|
||||
Err(e) => {
|
||||
set_err(&e);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let ctx_ptr = CURRENT_HOST.with(|c| c.get());
|
||||
let ctx_ptr = if ctx_ptr.is_null() {
|
||||
global_host_slot()
|
||||
@@ -300,19 +374,16 @@ unsafe extern "C" fn host_query(
|
||||
let _invoke_guard;
|
||||
let from_worker = CURRENT_HOST.with(|c| c.get().is_null());
|
||||
if from_worker {
|
||||
_invoke_guard = invoke_lock().lock().unwrap_or_else(|e| e.into_inner());
|
||||
match acquire_invoke_lock() {
|
||||
Ok(g) => _invoke_guard = g,
|
||||
Err(msg) => {
|
||||
set_err(&msg);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
let ctx = &mut *ctx_ptr;
|
||||
|
||||
let args_owned: Option<String> = if args_json.is_null() {
|
||||
None
|
||||
} else {
|
||||
CStr::from_ptr(args_json)
|
||||
.to_str()
|
||||
.ok()
|
||||
.map(|s| s.to_string())
|
||||
};
|
||||
|
||||
let result = match name {
|
||||
"module_source" => agent_rt::module_source(ctx).and_then(|v| value_to_json(&v)),
|
||||
"call_site" => agent_rt::call_site(ctx, Some(ctx.current_line)).and_then(|v| value_to_json(&v)),
|
||||
@@ -320,6 +391,19 @@ unsafe extern "C" fn host_query(
|
||||
"cwd" => Ok(serde_json::Value::String(
|
||||
ctx.cwd.to_string_lossy().into_owned(),
|
||||
)),
|
||||
"entry_path" => {
|
||||
let path = ctx.entry_path.clone().unwrap_or_else(|| ctx.cwd.clone());
|
||||
let abs = if path.is_absolute() {
|
||||
path
|
||||
} else {
|
||||
std::env::current_dir()
|
||||
.unwrap_or_else(|_| PathBuf::from("."))
|
||||
.join(path)
|
||||
};
|
||||
Ok(serde_json::Value::String(
|
||||
abs.to_string_lossy().into_owned(),
|
||||
))
|
||||
}
|
||||
"entry_dir" => {
|
||||
// Script directory: parent of entry file (for_run also sets ctx.cwd to this).
|
||||
let dir = ctx
|
||||
@@ -460,6 +544,24 @@ unsafe extern "C" fn host_query(
|
||||
let cap = crate::run_file_capture(&p, &opts).map_err(|e| e.to_string())?;
|
||||
value_to_json(&cap.value)
|
||||
})(),
|
||||
// M04: plugins may drive MLSP artifact.bind/execute without inventing a second protocol.
|
||||
"mlsp" => (|| {
|
||||
let raw = args_owned.as_deref().unwrap_or("{}");
|
||||
let args: serde_json::Value = serde_json::from_str(if raw.trim().is_empty() {
|
||||
"{}"
|
||||
} else {
|
||||
raw
|
||||
})
|
||||
.map_err(|e| format!("mlsp args: {e}"))?;
|
||||
let method = args
|
||||
.get("method")
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or_else(|| "mlsp requires `method`".to_string())?;
|
||||
let params = args.get("params").cloned().unwrap_or(serde_json::json!({}));
|
||||
let line = serde_json::json!({"method": method, "params": params}).to_string();
|
||||
let resp = crate::mlsp::handle_line(&line);
|
||||
serde_json::from_str(&resp).map_err(|e| format!("mlsp response: {e}"))
|
||||
})(),
|
||||
other => Err(format!("host_query: unknown `{other}`")),
|
||||
};
|
||||
|
||||
|
||||
@@ -331,6 +331,7 @@ fn spawn_file(
|
||||
));
|
||||
let _ = std::fs::write(&events_path, "");
|
||||
cmd.env("MARQDO_EVENT_FORWARD", &events_path);
|
||||
apply_run_selection_env(&mut cmd);
|
||||
|
||||
let mut child = cmd
|
||||
.spawn()
|
||||
@@ -693,3 +694,86 @@ fn require<'a>(bound: &'a HashMap<String, Value>, key: &str) -> Result<&'a Value
|
||||
.get(key)
|
||||
.ok_or_else(|| format!("host call missing `{key}`"))
|
||||
}
|
||||
|
||||
/// Propagate run/selection identity onto the *child* Command.env map.
|
||||
/// Reads parent env only as a fallback when an explicit RunContext was not
|
||||
/// applied; never uses parent `setenv`/`set_current_dir` as request state (M-R1).
|
||||
fn apply_run_selection_env(cmd: &mut Command) {
|
||||
for key in [
|
||||
"MARQDO_RUN_ID",
|
||||
"MARQDO_PARENT_RUN_ID",
|
||||
"MARQDO_TURN_ID",
|
||||
"MARQDO_WORKSPACE_SCOPE",
|
||||
"MARQDO_CANCEL_TOKEN",
|
||||
"MARQDO_EVENT_FORWARD",
|
||||
"MARQDO_USAGE_SINK",
|
||||
"MARQDO_CREDENTIAL_REF",
|
||||
"MARQDO_WORKSPACE_ID",
|
||||
"MARQDO_SNAPSHOT_ID",
|
||||
"MARQDO_LEASE_ID",
|
||||
"MARQDO_ATTEMPT_ID",
|
||||
"MARQDO_LLM_MODEL",
|
||||
"MARQDO_LLM_BASE_URL",
|
||||
"OPENAI_MODEL",
|
||||
"OPENAI_BASE_URL",
|
||||
] {
|
||||
if let Ok(v) = std::env::var(key) {
|
||||
cmd.env(key, v);
|
||||
}
|
||||
}
|
||||
// Do not forward process-global MARQDO_CANCEL into children — request cancel
|
||||
// must use MARQDO_CANCEL_TOKEN so leases stay isolated.
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::collections::HashMap;
|
||||
use std::process::Command;
|
||||
|
||||
#[test]
|
||||
fn f02_apply_run_selection_env_forwards_parent_identity() {
|
||||
std::env::set_var("MARQDO_RUN_ID", "run-parent");
|
||||
std::env::set_var("OPENAI_MODEL", "child-model");
|
||||
std::env::set_var("OPENAI_BASE_URL", "http://127.0.0.1:7010");
|
||||
// Must never require writing secrets into child env map for selection.
|
||||
std::env::remove_var("OPENAI_API_KEY");
|
||||
|
||||
let mut cmd = Command::new("true");
|
||||
apply_run_selection_env(&mut cmd);
|
||||
// Command doesn't expose env for inspection portably; re-check via helper contract:
|
||||
let keys: HashMap<&str, String> = [
|
||||
"MARQDO_RUN_ID",
|
||||
"OPENAI_MODEL",
|
||||
"OPENAI_BASE_URL",
|
||||
]
|
||||
.into_iter()
|
||||
.filter_map(|k| std::env::var(k).ok().map(|v| (k, v)))
|
||||
.collect();
|
||||
assert_eq!(keys.get("MARQDO_RUN_ID").map(String::as_str), Some("run-parent"));
|
||||
assert_eq!(keys.get("OPENAI_MODEL").map(String::as_str), Some("child-model"));
|
||||
assert_eq!(
|
||||
keys.get("OPENAI_BASE_URL").map(String::as_str),
|
||||
Some("http://127.0.0.1:7010")
|
||||
);
|
||||
|
||||
std::env::remove_var("MARQDO_RUN_ID");
|
||||
std::env::remove_var("OPENAI_MODEL");
|
||||
std::env::remove_var("OPENAI_BASE_URL");
|
||||
let _ = cmd; // ensure apply didn't panic
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f02_agent_skeleton_has_no_llm_llm() {
|
||||
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
|
||||
let agent = std::fs::read_to_string(root.join("ext/ai/agent.mq.md")).unwrap();
|
||||
assert!(
|
||||
!agent.contains("llm.llm"),
|
||||
"render_workbook_skeleton must not hardcode llm.llm"
|
||||
);
|
||||
assert!(
|
||||
agent.contains("llm.create") && agent.contains("OPENAI_MODEL"),
|
||||
"skeleton should create model from env selection"
|
||||
);
|
||||
}
|
||||
}
|
||||
+170
-100
@@ -15,7 +15,7 @@ use crate::builtin::{
|
||||
};
|
||||
use crate::debug::{emit_trace, DebugController};
|
||||
use crate::diagnostics::{bail_at, Span};
|
||||
use crate::host::{call_host, HostContext, HostFn};
|
||||
use crate::host::{call_host, HostContext};
|
||||
use crate::input_feed::InputFeed;
|
||||
use crate::capture::{
|
||||
BranchHit, CallFrame, LoopFrame, LoopHit, LoopIter, StmtEvent, StmtEventKind, ViewTrace,
|
||||
@@ -349,6 +349,17 @@ impl Interpreter {
|
||||
) -> Result<Value> {
|
||||
let fun = find_function_anywhere(module, name)
|
||||
.ok_or_else(|| self.err(format!("unknown function `{name}`")))?;
|
||||
self.invoke_bound_function(module, fun, args)
|
||||
}
|
||||
|
||||
/// Run an already-bound function node (SymbolId / source-line identity).
|
||||
/// Prefer this over name lookup so nested same-named functions cannot drift (G07).
|
||||
pub fn invoke_bound_function(
|
||||
&mut self,
|
||||
module: &Module,
|
||||
fun: &Function,
|
||||
args: &[(String, Value)],
|
||||
) -> Result<Value> {
|
||||
self.run_function(module, fun, Env::new(), args, false)
|
||||
}
|
||||
|
||||
@@ -534,6 +545,20 @@ impl Interpreter {
|
||||
])
|
||||
}
|
||||
|
||||
/// Pin `module` as site and call `lib.member` (isolated worker / CLI `--call-lib-path`).
|
||||
pub fn run_site_lib_path(
|
||||
&mut self,
|
||||
module: &Module,
|
||||
path: &str,
|
||||
args: Option<&Value>,
|
||||
) -> std::result::Result<Value, String> {
|
||||
let prev = self.site_module;
|
||||
self.site_module = Some(module as *const Module);
|
||||
let out = self.call_site_lib_path(path, args);
|
||||
self.site_module = prev;
|
||||
out
|
||||
}
|
||||
|
||||
/// Resolve `lib.member` on the site (entry) module — used by ABI `call_lib_path`.
|
||||
pub(crate) fn call_site_lib_path(
|
||||
&mut self,
|
||||
@@ -1169,34 +1194,150 @@ impl Interpreter {
|
||||
return self.eval_method_call(module, fun, env, recv_name, &call.callee, &ev_args);
|
||||
}
|
||||
|
||||
// Local-var method sugar (`a.b` when `a` is in env) before BoundCall path resolve.
|
||||
if let Some(path) = &call.path {
|
||||
// Bare dotted callee `a.b`: if `a` is a local variable, it is a method
|
||||
// receiver (Python-style unified namespace); otherwise it's a library path.
|
||||
if path.len() == 2 && env.get(&path[0]).is_some() {
|
||||
if path.len() == 2
|
||||
&& env.get(&path[0]).is_some()
|
||||
&& !module.import_modules.contains_key(&path[0])
|
||||
{
|
||||
let recv = path[0].clone();
|
||||
let method = path[1].clone();
|
||||
return self.eval_method_call(module, fun, env, &recv, &method, &ev_args);
|
||||
}
|
||||
return self.eval_path_call(module, path, &ev_args);
|
||||
}
|
||||
|
||||
if let Some(u) = module.uses.iter().find(|u| u.bind == call.callee) {
|
||||
return self.eval_path_call(module, &u.path, &ev_args);
|
||||
use crate::sema::bind::{resolve_for_exec, DynamicReason, ExecTarget};
|
||||
match resolve_for_exec(module, fun, &call) {
|
||||
ExecTarget::Local(target) => {
|
||||
self.eval_bound_function_call(module, fun, env, target, &ev_args)
|
||||
}
|
||||
ExecTarget::Import {
|
||||
lib: lib_mod,
|
||||
fun: target,
|
||||
..
|
||||
} => self.eval_bound_function_call(lib_mod, fun, env, target, &ev_args),
|
||||
ExecTarget::Builtin(name) => self.eval_builtin_call(name, &ev_args),
|
||||
ExecTarget::Host(hf) => {
|
||||
let mut params: Vec<String> =
|
||||
hf.required_params().iter().map(|s| (*s).to_string()).collect();
|
||||
for p in hf.optional_params() {
|
||||
params.push((*p).to_string());
|
||||
}
|
||||
let bound = bind_args(¶ms, &ev_args, true).map_err(|m| self.err(m))?;
|
||||
for req in hf.required_params() {
|
||||
if !bound.contains_key(*req) {
|
||||
return Err(self.err(format!("{} requires `{req}`", hf.name())));
|
||||
}
|
||||
}
|
||||
call_host(&mut self.host, hf, &bound).map_err(|m| self.err(m))
|
||||
}
|
||||
ExecTarget::Dynamic {
|
||||
reason: DynamicReason::CallFn,
|
||||
..
|
||||
} => {
|
||||
let bound = bind_args(&["name".into()], &ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let name = match bound.get("name") {
|
||||
Some(Value::Text(s)) => s.as_str(),
|
||||
_ => return Err(self.err("call_fn requires text name")),
|
||||
};
|
||||
let target = lookup_function(module, fun, name).ok_or_else(|| {
|
||||
self.err(format!("call_fn: unknown function `{name}`"))
|
||||
})?;
|
||||
if !target.params.is_empty() {
|
||||
return Err(self.err(format!(
|
||||
"call_fn: `{name}` must have no parameters"
|
||||
)));
|
||||
}
|
||||
self.run_function(module, target, Env::new(), &[], false)
|
||||
}
|
||||
ExecTarget::Dynamic {
|
||||
reason: DynamicReason::Plugin,
|
||||
name,
|
||||
} => {
|
||||
if let Some(reg) = self.host.plugins.get(&name).cloned() {
|
||||
let bound =
|
||||
bind_args(®.params, &ev_args, false).map_err(|m| self.err(m))?;
|
||||
return self
|
||||
.call_plugin_registered(&name, &bound)
|
||||
.map_err(|m| self.err(m));
|
||||
}
|
||||
Err(self.err(format!("unknown function `{name}`")))
|
||||
}
|
||||
ExecTarget::Dynamic {
|
||||
reason: DynamicReason::MethodReceiver,
|
||||
name,
|
||||
} => {
|
||||
if let Some(path) = &call.path {
|
||||
if path.len() == 2 && env.get(&path[0]).is_some() {
|
||||
return self.eval_method_call(
|
||||
module,
|
||||
fun,
|
||||
env,
|
||||
&path[0],
|
||||
&path[1],
|
||||
&ev_args,
|
||||
);
|
||||
}
|
||||
}
|
||||
Err(self.err(format!("unresolved dynamic call `{name}`")))
|
||||
}
|
||||
ExecTarget::Unresolved { message, .. } => Err(self.err(message)),
|
||||
}
|
||||
}
|
||||
|
||||
match call.callee.as_str() {
|
||||
/// Invoke a function already resolved via [`crate::sema::bind::resolve_for_exec`].
|
||||
fn eval_bound_function_call(
|
||||
&mut self,
|
||||
module: &Module,
|
||||
caller: &Function,
|
||||
env: &mut Env,
|
||||
target: &Function,
|
||||
ev_args: &[EvArg],
|
||||
) -> Result<Value> {
|
||||
let bound =
|
||||
bind_function_args(self, module, caller, env, &target.params, ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
if let Some(c) = &target.contract {
|
||||
if let Some(d) = crate::contract::call_arg_diag(
|
||||
c,
|
||||
&target.name,
|
||||
&bound,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
let mut call_env = Env::new();
|
||||
for (k, v) in bound {
|
||||
call_env.set(k, v);
|
||||
}
|
||||
self.host.push_call_site_line(self.host.current_line);
|
||||
let result = self.run_function(module, target, call_env, &[], false);
|
||||
self.host.pop_call_site_line();
|
||||
let result = result?;
|
||||
if target.is_object() {
|
||||
Ok(tag_instance(&target.name, result))
|
||||
} else {
|
||||
Ok(result)
|
||||
}
|
||||
}
|
||||
|
||||
fn eval_builtin_call(&mut self, name: &str, ev_args: &[EvArg]) -> Result<Value> {
|
||||
match name {
|
||||
"print" => {
|
||||
let bound = bind_args(&["text".into()], &ev_args, false)
|
||||
let bound = bind_args(&["text".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let text = bound
|
||||
.get("text")
|
||||
.map(|v| v.as_display())
|
||||
.ok_or_else(|| self.err("print requires text (named or positional)"))?;
|
||||
self.emit_line(&text);
|
||||
return Ok(Value::None);
|
||||
Ok(Value::None)
|
||||
}
|
||||
"input" => {
|
||||
let bound = bind_args(&["prompt".into()], &ev_args, true)
|
||||
let bound = bind_args(&["prompt".into()], ev_args, true)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let prompt = bound
|
||||
.get("prompt")
|
||||
@@ -1221,52 +1362,52 @@ impl Interpreter {
|
||||
e
|
||||
}
|
||||
})?;
|
||||
return Ok(Value::Text(line));
|
||||
Ok(Value::Text(line))
|
||||
}
|
||||
"len" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("len requires value"))?;
|
||||
let n = builtin_len(v).map_err(|m| self.err(m))?;
|
||||
return Ok(Value::Int(n));
|
||||
Ok(Value::Int(n))
|
||||
}
|
||||
"str" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("str requires value"))?;
|
||||
return Ok(builtin_str(v));
|
||||
Ok(builtin_str(v))
|
||||
}
|
||||
"int" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("int requires value"))?;
|
||||
let n = builtin_int(v).map_err(|m| self.err(m))?;
|
||||
return Ok(Value::Int(n));
|
||||
Ok(Value::Int(n))
|
||||
}
|
||||
"type" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("type requires value"))?;
|
||||
return Ok(builtin_type(v));
|
||||
Ok(builtin_type(v))
|
||||
}
|
||||
"trim" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("trim requires value"))?;
|
||||
return builtin_trim(v).map_err(|m| self.err(m));
|
||||
builtin_trim(v).map_err(|m| self.err(m))
|
||||
}
|
||||
"split" => {
|
||||
let bound = bind_args(&["value".into(), "sep".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into(), "sep".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let value = bound
|
||||
.get("value")
|
||||
@@ -1274,10 +1415,10 @@ impl Interpreter {
|
||||
let sep = bound
|
||||
.get("sep")
|
||||
.ok_or_else(|| self.err("split requires sep"))?;
|
||||
return builtin_split(value, sep).map_err(|m| self.err(m));
|
||||
builtin_split(value, sep).map_err(|m| self.err(m))
|
||||
}
|
||||
"join" => {
|
||||
let bound = bind_args(&["value".into(), "sep".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into(), "sep".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let value = bound
|
||||
.get("value")
|
||||
@@ -1285,10 +1426,10 @@ impl Interpreter {
|
||||
let sep = bound
|
||||
.get("sep")
|
||||
.ok_or_else(|| self.err("join requires sep"))?;
|
||||
return builtin_join(value, sep).map_err(|m| self.err(m));
|
||||
builtin_join(value, sep).map_err(|m| self.err(m))
|
||||
}
|
||||
"at" => {
|
||||
let bound = bind_args(&["value".into(), "index".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into(), "index".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let value = bound
|
||||
.get("value")
|
||||
@@ -1296,80 +1437,9 @@ impl Interpreter {
|
||||
let index = bound
|
||||
.get("index")
|
||||
.ok_or_else(|| self.err("at requires index"))?;
|
||||
return builtin_at(value, index).map_err(|m| self.err(m));
|
||||
builtin_at(value, index).map_err(|m| self.err(m))
|
||||
}
|
||||
"call_fn" => {
|
||||
let bound = bind_args(&["name".into()], &ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let name = match bound.get("name") {
|
||||
Some(Value::Text(s)) => s.as_str(),
|
||||
_ => return Err(self.err("call_fn requires text name")),
|
||||
};
|
||||
let target = lookup_function(module, fun, name).ok_or_else(|| {
|
||||
self.err(format!("call_fn: unknown function `{name}`"))
|
||||
})?;
|
||||
if !target.params.is_empty() {
|
||||
return Err(self.err(format!(
|
||||
"call_fn: `{name}` must have no parameters"
|
||||
)));
|
||||
}
|
||||
return self.run_function(module, target, Env::new(), &[], false);
|
||||
}
|
||||
other if HostFn::from_name(other).is_some() => {
|
||||
let hf = HostFn::from_name(other).unwrap();
|
||||
let mut params: Vec<String> =
|
||||
hf.required_params().iter().map(|s| (*s).to_string()).collect();
|
||||
for p in hf.optional_params() {
|
||||
params.push((*p).to_string());
|
||||
}
|
||||
let bound = bind_args(¶ms, &ev_args, true).map_err(|m| self.err(m))?;
|
||||
for req in hf.required_params() {
|
||||
if !bound.contains_key(*req) {
|
||||
return Err(self.err(format!("{} requires `{req}`", hf.name())));
|
||||
}
|
||||
}
|
||||
return call_host(&mut self.host, hf, &bound).map_err(|m| self.err(m));
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
if let Some(reg) = self.host.plugins.get(&call.callee).cloned() {
|
||||
let bound = bind_args(®.params, &ev_args, false).map_err(|m| self.err(m))?;
|
||||
return self
|
||||
.call_plugin_registered(&call.callee, &bound)
|
||||
.map_err(|m| self.err(m));
|
||||
}
|
||||
|
||||
let target = lookup_function(module, fun, &call.callee)
|
||||
.ok_or_else(|| self.err(format!("unknown function `{}`", call.callee)))?;
|
||||
|
||||
let bound = bind_function_args(self, module, fun, env, &target.params, &ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
// 契约边界 1(T2.2):调用实参 vs 形参契约。
|
||||
if let Some(c) = &target.contract {
|
||||
if let Some(d) = crate::contract::call_arg_diag(
|
||||
c,
|
||||
&target.name,
|
||||
&bound,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
let mut call_env = Env::new();
|
||||
for (k, v) in bound {
|
||||
call_env.set(k, v);
|
||||
}
|
||||
|
||||
self.host.push_call_site_line(self.host.current_line);
|
||||
let result = self.run_function(module, target, call_env, &[], false);
|
||||
self.host.pop_call_site_line();
|
||||
let result = result?;
|
||||
if target.is_object() {
|
||||
Ok(tag_instance(&target.name, result))
|
||||
} else {
|
||||
Ok(result)
|
||||
other => Err(self.err(format!("unknown builtin `{other}`"))),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+82
-20
@@ -17,43 +17,109 @@ pub struct KnowledgeOptions {
|
||||
}
|
||||
|
||||
pub fn compile(opts: KnowledgeOptions) -> Result<KnowledgeGraph> {
|
||||
// The semantic projection is the authoritative machine-facing graph. The
|
||||
// older EKC catalog below remains intentionally available as a compatible
|
||||
// human/retrieval projection, but never substitutes for semantic binding.
|
||||
// Authoritative execution/binding graph: SemanticProject under `graph/`.
|
||||
// Legacy EKC retrieval catalog writes only under `ekc/` and must never
|
||||
// overwrite `graph/edges.json` (G01).
|
||||
let semantic_root = if opts.path.is_file() {
|
||||
opts.path.parent().unwrap_or_else(|| Path::new(".")).to_path_buf()
|
||||
} else {
|
||||
opts.path.clone()
|
||||
};
|
||||
let t0 = std::time::Instant::now();
|
||||
let semantic = crate::sema::compile_project(&crate::sema::CompileOptions {
|
||||
project_root: semantic_root,
|
||||
entry: opts.path.is_file().then(|| opts.path.clone()),
|
||||
lock_mode: crate::sema::LockMode::Off,
|
||||
})?;
|
||||
let sema_ms = t0.elapsed().as_millis();
|
||||
let t1 = std::time::Instant::now();
|
||||
crate::sema::write_semantic_projections(&opts.out_dir, &semantic)?;
|
||||
let sema_write_ms = t1.elapsed().as_millis();
|
||||
let extract_opts = ExtractOptions {
|
||||
path: opts.path.clone(),
|
||||
out_dir: opts.out_dir.clone(),
|
||||
};
|
||||
let t2 = std::time::Instant::now();
|
||||
let graph = extract_graph(&extract_opts)?;
|
||||
let extract_ms = t2.elapsed().as_millis();
|
||||
let t3 = std::time::Instant::now();
|
||||
write_projections(&opts.out_dir, &graph)?;
|
||||
let retrieve_write_ms = t3.elapsed().as_millis();
|
||||
let t4 = std::time::Instant::now();
|
||||
write_publish_manifest(&opts.out_dir, &semantic, &graph)?;
|
||||
let manifest_ms = t4.elapsed().as_millis();
|
||||
eprintln!(
|
||||
"marqdo knowledge: {} module(s), {} symbol(s), {} capability(ies), {} knowledge → {}",
|
||||
"marqdo knowledge: {} module(s), {} symbol(s), {} capability(ies), {} knowledge → {} (semantic graph/; retrieval ekc/)",
|
||||
graph.modules.len(),
|
||||
graph.symbols.len(),
|
||||
graph.capabilities.len(),
|
||||
graph.knowledge.len(),
|
||||
opts.out_dir.display()
|
||||
);
|
||||
eprintln!(
|
||||
"marqdo knowledge timing_ms: sema={sema_ms} sema_write={sema_write_ms} extract={extract_ms} retrieve_write={retrieve_write_ms} manifest={manifest_ms} total={}",
|
||||
t0.elapsed().as_millis()
|
||||
);
|
||||
Ok(graph)
|
||||
}
|
||||
|
||||
/// M06: associate semantic generation, retrieval generation, and source digests.
|
||||
fn write_publish_manifest(
|
||||
out_dir: &Path,
|
||||
semantic: &crate::sema::Compilation,
|
||||
graph: &KnowledgeGraph,
|
||||
) -> Result<()> {
|
||||
let current = fs::read_to_string(out_dir.join("CURRENT")).unwrap_or_default();
|
||||
let generation = current.trim().to_string();
|
||||
let source_digest = {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut h = Sha256::new();
|
||||
for a in &semantic.semantics.artifacts {
|
||||
h.update(a.hashes.artifact.0.as_bytes());
|
||||
}
|
||||
format!("sha256:{:x}", h.finalize())
|
||||
};
|
||||
let evidence_digest = {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut h = Sha256::new();
|
||||
for a in &semantic.semantics.artifacts {
|
||||
h.update(a.hashes.evidence.0.as_bytes());
|
||||
}
|
||||
format!("sha256:{:x}", h.finalize())
|
||||
};
|
||||
let manifest = json!({
|
||||
"schema_version": 1,
|
||||
"kind": "publish_manifest",
|
||||
"semantic_generation": generation,
|
||||
"retrieval": {
|
||||
"ekc_graph": "ekc/graph",
|
||||
"modules": graph.modules.len(),
|
||||
"symbols": graph.symbols.len(),
|
||||
"capabilities": graph.capabilities.len(),
|
||||
"knowledge": graph.knowledge.len(),
|
||||
},
|
||||
"source_digest": source_digest,
|
||||
"evidence_digest": evidence_digest,
|
||||
"excluded_dirs": crate::project::EXCLUDED_DIRS,
|
||||
"project_id": semantic.semantics.project_id,
|
||||
});
|
||||
let path = out_dir.join("publish_manifest.json");
|
||||
fs::write(&path, serde_json::to_string_pretty(&manifest)?)
|
||||
.with_context(|| format!("write {}", path.display()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Directory for the legacy EKC retrieval projection (not execution authority).
|
||||
pub fn ekc_graph_dir(out_dir: &Path) -> std::path::PathBuf {
|
||||
out_dir.join("ekc").join("graph")
|
||||
}
|
||||
|
||||
pub fn load_graph(out_dir: &Path) -> Result<KnowledgeGraph> {
|
||||
let graph_path = out_dir.join("graph").join("graph.json");
|
||||
let edges_path = out_dir.join("graph").join("edges.json");
|
||||
let graph_path = ekc_graph_dir(out_dir).join("graph.json");
|
||||
let edges_path = ekc_graph_dir(out_dir).join("edges.json");
|
||||
if !graph_path.exists() {
|
||||
anyhow::bail!(
|
||||
"no knowledge graph at {} — run `marqdo knowledge` first",
|
||||
"no EKC retrieval graph at {} — run `marqdo knowledge` first",
|
||||
graph_path.display()
|
||||
);
|
||||
}
|
||||
@@ -139,14 +205,7 @@ pub fn load_graph(out_dir: &Path) -> Result<KnowledgeGraph> {
|
||||
}
|
||||
|
||||
pub fn ensure_compiled(path: &Path, out_dir: &Path) -> Result<KnowledgeGraph> {
|
||||
let graph_path = out_dir.join("graph").join("graph.json");
|
||||
if graph_path.exists() {
|
||||
// Always refresh from source when path given
|
||||
return compile(KnowledgeOptions {
|
||||
path: path.to_path_buf(),
|
||||
out_dir: out_dir.to_path_buf(),
|
||||
});
|
||||
}
|
||||
// Always refresh from source when path is given.
|
||||
compile(KnowledgeOptions {
|
||||
path: path.to_path_buf(),
|
||||
out_dir: out_dir.to_path_buf(),
|
||||
@@ -164,7 +223,7 @@ fn write_projections(out_dir: &Path, graph: &KnowledgeGraph) -> Result<()> {
|
||||
"knowledge/patterns",
|
||||
"knowledge/failures",
|
||||
"knowledge/migrations",
|
||||
"graph",
|
||||
"ekc/graph",
|
||||
"agent/policies",
|
||||
"agent/skills",
|
||||
"agent/contexts",
|
||||
@@ -234,20 +293,23 @@ fn write_projections(out_dir: &Path, graph: &KnowledgeGraph) -> Result<()> {
|
||||
fs::write(&path, body)?;
|
||||
}
|
||||
|
||||
// Graph JSON
|
||||
// Retrieval-only EKC graph (must not touch authoritative `graph/`).
|
||||
let ekc = ekc_graph_dir(out_dir);
|
||||
let graph_json = serde_json::to_string_pretty(&graph.to_graph_json())?;
|
||||
fs::write(out_dir.join("graph/graph.json"), graph_json)?;
|
||||
fs::write(ekc.join("graph.json"), graph_json)?;
|
||||
let edges_json = serde_json::to_string_pretty(&graph.to_edges_json())?;
|
||||
fs::write(out_dir.join("graph/edges.json"), edges_json)?;
|
||||
fs::write(ekc.join("edges.json"), edges_json)?;
|
||||
|
||||
// Search index
|
||||
let index = json!({
|
||||
"type": "Marqdo Knowledge Index",
|
||||
"role": "retrieval_only",
|
||||
"note": "Not an execution authority; binding uses graph/edges.json (SemanticProject).",
|
||||
"generated": { "by": graph.generated_by },
|
||||
"entries": graph.index_map(),
|
||||
});
|
||||
fs::write(
|
||||
out_dir.join("graph/index.json"),
|
||||
ekc.join("index.json"),
|
||||
serde_json::to_string_pretty(&index)?,
|
||||
)?;
|
||||
|
||||
|
||||
@@ -164,13 +164,7 @@ fn collect_mq_md(
|
||||
let p = entry.path();
|
||||
if p.is_dir() {
|
||||
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
|
||||
if name == "target"
|
||||
|| name == ".git"
|
||||
|| name == "node_modules"
|
||||
|| name == "docker"
|
||||
|| name == "data"
|
||||
|| name == ".cursor"
|
||||
{
|
||||
if crate::project::is_excluded_dir_name(name) {
|
||||
continue;
|
||||
}
|
||||
if let Ok(canon) = p.canonicalize() {
|
||||
@@ -210,13 +204,7 @@ fn walk_knowledge_md(
|
||||
let p = entry.path();
|
||||
if p.is_dir() {
|
||||
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
|
||||
if name == "target"
|
||||
|| name == ".git"
|
||||
|| name == ".marqdo"
|
||||
|| name == "docker"
|
||||
|| name == "data"
|
||||
|| name == ".cursor"
|
||||
{
|
||||
if crate::project::is_excluded_dir_name(name) {
|
||||
continue;
|
||||
}
|
||||
if let Ok(canon) = p.canonicalize() {
|
||||
|
||||
@@ -52,7 +52,9 @@ pub fn find_stale(graph: &KnowledgeGraph, root: &Path) -> Vec<StaleItem> {
|
||||
if src.exists() {
|
||||
if let (Ok(meta), Ok(proj_meta)) = (
|
||||
fs::metadata(&src),
|
||||
fs::metadata(root.join(".marqdo/graph/graph.json"))
|
||||
fs::metadata(root.join(".marqdo/ekc/graph/graph.json"))
|
||||
.or_else(|_| fs::metadata(root.join(".marqdo/graph/graph.json")))
|
||||
.or_else(|_| fs::metadata(root.join("ekc/graph/graph.json")))
|
||||
.or_else(|_| fs::metadata(root.join("graph/graph.json"))),
|
||||
) {
|
||||
if let (Ok(st), Ok(pt)) = (meta.modified(), proj_meta.modified()) {
|
||||
|
||||
@@ -20,7 +20,7 @@ pub mod preflight;
|
||||
pub mod reuse;
|
||||
pub mod util;
|
||||
|
||||
pub use compile::{compile, ensure_compiled, load_graph, KnowledgeOptions};
|
||||
pub use compile::{compile, ekc_graph_dir, ensure_compiled, load_graph, KnowledgeOptions};
|
||||
pub use duplicate::{find_duplicates, format_duplicates_text};
|
||||
pub use find::{find, find_json, format_find_text};
|
||||
pub use impact::{format_impact_text, impact, impact_json};
|
||||
|
||||
@@ -274,6 +274,25 @@ pub fn format_reuse_text(task: &str, result: &ReuseResult) -> String {
|
||||
}
|
||||
|
||||
pub fn reuse_json(task: &str, result: &ReuseResult) -> Value {
|
||||
let bind_required = matches!(
|
||||
result.decision,
|
||||
ReuseDecision::Reuse | ReuseDecision::Adapt
|
||||
);
|
||||
let mut bind_targets = Vec::new();
|
||||
if let Some(sym) = &result.symbol {
|
||||
bind_targets.push(json!({
|
||||
"query": sym,
|
||||
"via": "reuse.symbol",
|
||||
"note": "pass query to artifact.bind then artifact.execute",
|
||||
}));
|
||||
}
|
||||
if let Some(src) = &result.source {
|
||||
bind_targets.push(json!({
|
||||
"query": src,
|
||||
"via": "reuse.source",
|
||||
"note": "pass query to artifact.bind then artifact.execute",
|
||||
}));
|
||||
}
|
||||
json!({
|
||||
"task": task,
|
||||
"decision": result.decision.as_str(),
|
||||
@@ -287,6 +306,9 @@ pub fn reuse_json(task: &str, result: &ReuseResult) -> Value {
|
||||
"constraints": result.constraints,
|
||||
"decisions": result.decisions,
|
||||
"failures": result.failures,
|
||||
"discovery_only": true,
|
||||
"bind_required": bind_required,
|
||||
"bind_targets": bind_targets,
|
||||
"candidates": result.candidates.iter().map(|c| json!({
|
||||
"id": c.id,
|
||||
"kind": c.kind,
|
||||
|
||||
+53
-1
@@ -21,6 +21,7 @@ pub mod check;
|
||||
pub mod contract;
|
||||
pub mod debug;
|
||||
pub mod diagnostics;
|
||||
pub mod document_extract;
|
||||
pub mod embedded_lib;
|
||||
#[cfg(feature = "cli")]
|
||||
pub mod ext_cli;
|
||||
@@ -36,6 +37,8 @@ pub mod lex;
|
||||
pub mod linalg_ops;
|
||||
pub mod load;
|
||||
pub mod mlsp;
|
||||
pub mod runtime;
|
||||
pub mod workbench;
|
||||
pub mod parse;
|
||||
pub mod project;
|
||||
pub mod repair;
|
||||
@@ -110,6 +113,10 @@ pub struct RunOptions {
|
||||
pub binds: Vec<(String, String)>,
|
||||
/// Attach CoW session for `ext/ai/sandbox` nested runs.
|
||||
pub sandbox_session: Option<String>,
|
||||
/// Isolated worker: call `lib.member` on the site entry instead of `# main`.
|
||||
pub call_lib_path: Option<String>,
|
||||
/// JSON object of named args for [`Self::call_lib_path`] (file path or inline).
|
||||
pub args_json: Option<std::path::PathBuf>,
|
||||
}
|
||||
|
||||
impl Default for RunOptions {
|
||||
@@ -133,6 +140,8 @@ impl Default for RunOptions {
|
||||
emit_result: None,
|
||||
binds: Vec::new(),
|
||||
sandbox_session: None,
|
||||
call_lib_path: None,
|
||||
args_json: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -227,7 +236,11 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
|
||||
if opts.trace_eval {
|
||||
eprintln!("=== marqdo: trace-eval ({path_label}) ===");
|
||||
}
|
||||
let value = interp.run_module(&module)?;
|
||||
let value = if let Some(lib_path) = &opts.call_lib_path {
|
||||
run_call_lib_path(&mut interp, &module, lib_path, opts)?
|
||||
} else {
|
||||
interp.run_module(&module)?
|
||||
};
|
||||
if opts.trace_eval {
|
||||
eprintln!("=== marqdo: end trace-eval ===");
|
||||
}
|
||||
@@ -273,6 +286,45 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
|
||||
Ok(0)
|
||||
}
|
||||
|
||||
/// Worker mode: resolve `lib.member` on the loaded site module (imports already wired).
|
||||
fn run_call_lib_path(
|
||||
interp: &mut Interpreter,
|
||||
module: &crate::ast::Module,
|
||||
lib_path: &str,
|
||||
opts: &RunOptions,
|
||||
) -> Result<crate::value::Value> {
|
||||
let args_val = load_args_json(opts)?;
|
||||
// Keep site_module pinned for the duration of the lib call (plugins may re-enter).
|
||||
let value = interp
|
||||
.run_site_lib_path(module, lib_path, args_val.as_ref())
|
||||
.map_err(|e| anyhow::anyhow!(e))?;
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
fn load_args_json(opts: &RunOptions) -> Result<Option<crate::value::Value>> {
|
||||
let Some(path) = &opts.args_json else {
|
||||
return Ok(None);
|
||||
};
|
||||
let text = if path.as_os_str() == "-" {
|
||||
use std::io::Read;
|
||||
let mut buf = String::new();
|
||||
std::io::stdin()
|
||||
.read_to_string(&mut buf)
|
||||
.map_err(|e| anyhow::anyhow!("args-json stdin: {e}"))?;
|
||||
buf
|
||||
} else {
|
||||
std::fs::read_to_string(path)
|
||||
.map_err(|e| anyhow::anyhow!("args-json {}: {e}", path.display()))?
|
||||
};
|
||||
if text.trim().is_empty() {
|
||||
return Ok(None);
|
||||
}
|
||||
let json: serde_json::Value =
|
||||
serde_json::from_str(&text).map_err(|e| anyhow::anyhow!("args-json parse: {e}"))?;
|
||||
let v = crate::host::json::json_to_value(&json).map_err(|e| anyhow::anyhow!(e))?;
|
||||
Ok(Some(v))
|
||||
}
|
||||
|
||||
fn emit_result_file(opts: &RunOptions, value: &crate::value::Value) -> Result<()> {
|
||||
let Some(path) = &opts.emit_result else {
|
||||
return Ok(());
|
||||
|
||||
+177
-7
@@ -120,6 +120,14 @@ enum Commands {
|
||||
#[arg(long = "fs-root", value_name = "DIR")]
|
||||
fs_root: Option<PathBuf>,
|
||||
|
||||
/// Isolated worker: call `lib.member` on the site entry instead of `# main`.
|
||||
#[arg(long = "call-lib-path", value_name = "LIB.MEMBER")]
|
||||
call_lib_path: Option<String>,
|
||||
|
||||
/// Named args JSON for `--call-lib-path` (file path, or `-` for stdin).
|
||||
#[arg(long = "args-json", value_name = "FILE")]
|
||||
args_json: Option<PathBuf>,
|
||||
|
||||
/// Emit diagnostics as JSON on stderr (machine-readable; AI/MLSP 面).
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
@@ -323,6 +331,10 @@ enum Commands {
|
||||
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
|
||||
/// Exit non-zero when conflicts found (CI gate; default remains report-only)
|
||||
#[arg(long)]
|
||||
fail: bool,
|
||||
},
|
||||
/// List stale / deprecated / superseded knowledge
|
||||
Stale {
|
||||
@@ -334,6 +346,10 @@ enum Commands {
|
||||
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
|
||||
/// Exit non-zero when stale items found (CI gate; default remains report-only)
|
||||
#[arg(long)]
|
||||
fail: bool,
|
||||
},
|
||||
/// Detect duplicate capabilities via symbol fingerprints
|
||||
Duplicate {
|
||||
@@ -353,7 +369,7 @@ enum Commands {
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
},
|
||||
/// Verify code ↔ knowledge consistency
|
||||
/// Verify code ↔ knowledge consistency (EKC structural; exit 1 only on errors)
|
||||
Verify {
|
||||
#[arg(value_name = "PATH")]
|
||||
path: Option<PathBuf>,
|
||||
@@ -363,6 +379,10 @@ enum Commands {
|
||||
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
|
||||
/// Also fail on warnings (does not change default success-when-no-errors)
|
||||
#[arg(long)]
|
||||
fail_on_warning: bool,
|
||||
},
|
||||
/// Official extension installer (`list` / `add` / `remove`)
|
||||
Ext {
|
||||
@@ -415,6 +435,18 @@ enum KnowledgeAction {
|
||||
#[arg(value_name = "SYMBOL_OR_PATH")]
|
||||
query: String,
|
||||
},
|
||||
/// Machine adjudication of semantic artifacts (M03): may_bind / may_publish
|
||||
Adjudicate {
|
||||
/// Artifact id, symbol id, or display name (omit to report all functions)
|
||||
#[arg(value_name = "ARTIFACT")]
|
||||
artifact: Option<String>,
|
||||
/// Policy: pure | business | static
|
||||
#[arg(long, default_value = "business")]
|
||||
policy: String,
|
||||
/// Exit 1 when any reported artifact has may_bind=false
|
||||
#[arg(long)]
|
||||
fail: bool,
|
||||
},
|
||||
/// Run engineering preflight for a task
|
||||
Preflight {
|
||||
#[arg(value_name = "TASK")]
|
||||
@@ -505,6 +537,23 @@ enum ViewAction {
|
||||
#[arg(short = 'o', long = "out", required = true)]
|
||||
out: PathBuf,
|
||||
},
|
||||
/// Read-only DocumentViewV1 (never executes). JSON to stdout or `--out`.
|
||||
Document {
|
||||
/// Project root (defaults to cwd / CLI path)
|
||||
#[arg(long)]
|
||||
project: Option<PathBuf>,
|
||||
|
||||
/// Document path relative to project
|
||||
#[arg(long)]
|
||||
document: String,
|
||||
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
|
||||
/// Optional HTML wrapper for browser inspection (embed fragment only)
|
||||
#[arg(long, value_name = "FILE")]
|
||||
out: Option<PathBuf>,
|
||||
},
|
||||
}
|
||||
|
||||
fn main() -> ExitCode {
|
||||
@@ -647,6 +696,8 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
emit_result,
|
||||
binds,
|
||||
fs_root,
|
||||
call_lib_path,
|
||||
args_json,
|
||||
json: _,
|
||||
legacy_markup,
|
||||
} => {
|
||||
@@ -667,6 +718,8 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
emit_result: emit_result.clone(),
|
||||
binds: bind_pairs.clone(),
|
||||
fs_root: fs_root.clone(),
|
||||
call_lib_path: call_lib_path.clone(),
|
||||
args_json: args_json.clone(),
|
||||
..RunOptions::dump_all()
|
||||
}
|
||||
} else {
|
||||
@@ -682,6 +735,8 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
emit_result,
|
||||
binds: bind_pairs,
|
||||
fs_root,
|
||||
call_lib_path,
|
||||
args_json,
|
||||
..RunOptions::default()
|
||||
}
|
||||
};
|
||||
@@ -1081,6 +1136,56 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
})?;
|
||||
Ok(0)
|
||||
}
|
||||
Some(ViewAction::Document {
|
||||
project,
|
||||
document,
|
||||
json,
|
||||
out,
|
||||
}) => {
|
||||
let project = project
|
||||
.or(path)
|
||||
.unwrap_or_else(|| PathBuf::from("."));
|
||||
let req = marqdo::workbench::DocumentViewRequest {
|
||||
project,
|
||||
document,
|
||||
revision: None,
|
||||
generation: None,
|
||||
node_id: None,
|
||||
include: vec![
|
||||
"fragment".into(),
|
||||
"outline".into(),
|
||||
"nodes".into(),
|
||||
"refs".into(),
|
||||
"diagnostics".into(),
|
||||
],
|
||||
};
|
||||
let view = marqdo::workbench::build_document_view(&req).map_err(|e| {
|
||||
anyhow::anyhow!("{}", e)
|
||||
})?;
|
||||
if let Some(out_path) = out {
|
||||
let fragment = view
|
||||
.pointer("/render/fragment")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("");
|
||||
let title = view
|
||||
.pointer("/document/title")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("DocumentView");
|
||||
let html = format!(
|
||||
"<!doctype html><html><head><meta charset=\"utf-8\"/><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"/><title>{title}</title><style>:root{{--mq-bg:#f7f5f0;--mq-text:#1a1a1a;--mq-muted:#5c5c5c;--mq-border:#d0cbc2;--mq-accent:#0b5fff;--mq-font-body:\"Iowan Old Style\",\"Palatino Linotype\",Palatino,serif;--mq-font-code:ui-monospace,Consolas,monospace}}body{{margin:0;background:var(--mq-bg);color:var(--mq-text);font:16px/1.75 var(--mq-font-body)}}.wrap{{max-width:46rem;margin:0 auto;padding:1.5rem}} .mq-document a{{color:var(--mq-accent)}}</style></head><body><div class=\"wrap\">{fragment}</div></body></html>"
|
||||
);
|
||||
std::fs::write(&out_path, html)?;
|
||||
if json {
|
||||
println!("{}", serde_json::to_string_pretty(&view)?);
|
||||
} else {
|
||||
println!("wrote {}", out_path.display());
|
||||
}
|
||||
} else {
|
||||
println!("{}", serde_json::to_string_pretty(&view)?);
|
||||
}
|
||||
let _ = json;
|
||||
Ok(0)
|
||||
}
|
||||
None => {
|
||||
serve(ViewOptions {
|
||||
path: path.unwrap_or_else(|| PathBuf::from(".")),
|
||||
@@ -1324,6 +1429,51 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
1
|
||||
})
|
||||
}
|
||||
KnowledgeAction::Adjudicate {
|
||||
artifact,
|
||||
policy,
|
||||
fail,
|
||||
} => {
|
||||
let semantic_root = if path.is_file() {
|
||||
path.parent()
|
||||
.unwrap_or_else(|| std::path::Path::new("."))
|
||||
.to_path_buf()
|
||||
} else {
|
||||
path.clone()
|
||||
};
|
||||
let report = marqdo::mlsp::adjudicate_project(
|
||||
&semantic_root,
|
||||
artifact.as_deref(),
|
||||
&policy,
|
||||
)
|
||||
.map_err(|e| anyhow::anyhow!("{}", e))?;
|
||||
if json {
|
||||
println!("{report}");
|
||||
} else {
|
||||
println!(
|
||||
"knowledge adjudicate policy={} may_bind_all={} count={}",
|
||||
policy,
|
||||
report["may_bind_all"],
|
||||
report["count"]
|
||||
);
|
||||
for item in report["reports"].as_array().into_iter().flatten() {
|
||||
println!(
|
||||
" {} evidence={} may_bind={} may_publish={}",
|
||||
item["artifact_id"].as_str().unwrap_or(""),
|
||||
item["verification"]["evidence"].as_str().unwrap_or(""),
|
||||
item["may_bind"],
|
||||
item["may_publish"]
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(
|
||||
if fail && report["may_bind_all"].as_bool() != Some(true) {
|
||||
1
|
||||
} else {
|
||||
0
|
||||
},
|
||||
)
|
||||
}
|
||||
KnowledgeAction::Preflight {
|
||||
task,
|
||||
min_candidates,
|
||||
@@ -1462,7 +1612,12 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
}
|
||||
Ok(0)
|
||||
}
|
||||
Commands::Conflicts { path, out, json } => {
|
||||
Commands::Conflicts {
|
||||
path,
|
||||
out,
|
||||
json,
|
||||
fail,
|
||||
} => {
|
||||
let path = path.unwrap_or_else(|| PathBuf::from("."));
|
||||
let graph = ensure_compiled(&path, &out)?;
|
||||
let items = find_conflicts(&graph);
|
||||
@@ -1471,9 +1626,14 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
} else {
|
||||
print!("{}", format_conflicts_text(&items));
|
||||
}
|
||||
Ok(0)
|
||||
Ok(if fail && !items.is_empty() { 1 } else { 0 })
|
||||
}
|
||||
Commands::Stale { path, out, json } => {
|
||||
Commands::Stale {
|
||||
path,
|
||||
out,
|
||||
json,
|
||||
fail,
|
||||
} => {
|
||||
let path = path.unwrap_or_else(|| PathBuf::from("."));
|
||||
let graph = ensure_compiled(&path, &out)?;
|
||||
let items = find_stale(&graph, &path);
|
||||
@@ -1482,7 +1642,7 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
} else {
|
||||
print!("{}", format_stale_text(&items));
|
||||
}
|
||||
Ok(0)
|
||||
Ok(if fail && !items.is_empty() { 1 } else { 0 })
|
||||
}
|
||||
Commands::Duplicate {
|
||||
path,
|
||||
@@ -1517,7 +1677,12 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
Ok(0)
|
||||
}
|
||||
}
|
||||
Commands::Verify { path, out, json } => {
|
||||
Commands::Verify {
|
||||
path,
|
||||
out,
|
||||
json,
|
||||
fail_on_warning,
|
||||
} => {
|
||||
let path = path.unwrap_or_else(|| PathBuf::from("."));
|
||||
let graph = ensure_compiled(&path, &out)?;
|
||||
let issues = verify(&graph);
|
||||
@@ -1527,7 +1692,12 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
print!("{}", format_verify_text(&issues));
|
||||
}
|
||||
let errors = issues.iter().filter(|i| i.severity == "error").count();
|
||||
Ok(if errors > 0 { 1 } else { 0 })
|
||||
let warnings = issues.iter().filter(|i| i.severity == "warning").count();
|
||||
Ok(if errors > 0 || (fail_on_warning && warnings > 0) {
|
||||
1
|
||||
} else {
|
||||
0
|
||||
})
|
||||
}
|
||||
Commands::Ext { action } => {
|
||||
match action {
|
||||
|
||||
+699
-12
@@ -64,17 +64,62 @@ fn dispatch(req: &Value) -> Result<Value, Value> {
|
||||
Some("artifact.impact") => artifact_impact(¶ms),
|
||||
Some("artifact.bind") => artifact_bind(¶ms),
|
||||
Some("artifact.execute") => artifact_execute(¶ms),
|
||||
Some("artifact.reuse_execute") => artifact_reuse_execute(¶ms),
|
||||
Some("document.extract") => document_extract(¶ms),
|
||||
Some("artifact.patch") => artifact_patch(¶ms),
|
||||
Some("artifact.extract_pure") => artifact_extract_pure(¶ms),
|
||||
// Workbench M00–M04: capability probe + read-only view + locator.
|
||||
Some("capabilities") => Ok(crate::workbench::probe_capabilities(¶ms)),
|
||||
#[cfg(feature = "view")]
|
||||
Some("view.document") => view_document(¶ms),
|
||||
#[cfg(not(feature = "view"))]
|
||||
Some("view.document") => Err(err(
|
||||
"mlsp.unsupported",
|
||||
"view.document requires the view feature",
|
||||
)),
|
||||
Some("view.reference_resolve") => crate::workbench::resolve_reference(¶ms),
|
||||
// `<TAB>` M9: occlusion label search (two-tier vocabulary).
|
||||
Some("labels.search") => labels_search(¶ms),
|
||||
// `<TAB>` M6: one-shot session summary (T1 staleness seen/fixed).
|
||||
Some("session.summary") => session_summary(),
|
||||
other => Err(err(
|
||||
"mlsp.unknown_method",
|
||||
format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema / artifact.find / artifact.inspect / link.resolve / artifact.verify / artifact.impact / artifact.bind / artifact.execute"),
|
||||
format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema / artifact.find / artifact.inspect / link.resolve / artifact.verify / artifact.impact / artifact.bind / artifact.execute / artifact.reuse_execute / artifact.patch / artifact.extract_pure / document.extract / capabilities / view.document / view.reference_resolve"),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "view")]
|
||||
fn view_document(params: &Value) -> Result<Value, Value> {
|
||||
let req = crate::workbench::DocumentViewRequest::from_params(params)
|
||||
.map_err(|e| err("mlsp.bad_request", e))?;
|
||||
crate::workbench::build_document_view(&req)
|
||||
}
|
||||
|
||||
fn artifact_extract_pure(params: &Value) -> Result<Value, Value> {
|
||||
let name = params
|
||||
.get("name")
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| err("mlsp.bad_request", "artifact.extract_pure 需要 name"))?;
|
||||
let expression = params
|
||||
.get("expression")
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| err("mlsp.bad_request", "artifact.extract_pure 需要 expression"))?;
|
||||
match crate::sema::plan_pure_extract(name, expression) {
|
||||
Ok(plan) => Ok(json!({
|
||||
"schema_version": 1,
|
||||
"name": plan.name,
|
||||
"params": plan.params,
|
||||
"expression": plan.expression,
|
||||
"heading": plan.heading,
|
||||
"call": plan.call,
|
||||
"effects": ["pure"],
|
||||
"note": "预览补丁;写入请配合 artifact.patch / 人工审阅",
|
||||
})),
|
||||
Err(e) => Err(json!({"code": e.code, "message": e.message})),
|
||||
}
|
||||
}
|
||||
|
||||
// ---------- Executable-knowledge artifact protocol ----------
|
||||
|
||||
/// All artifact methods compile the same semantic project as the CLI. They
|
||||
@@ -103,6 +148,22 @@ fn artifact_query<'a>(project: &'a crate::sema::SemanticProject, query: &str) ->
|
||||
}).collect()
|
||||
}
|
||||
|
||||
fn document_extract(params: &Value) -> Result<Value, Value> {
|
||||
let path = params
|
||||
.get("path")
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| err("mlsp.bad_request", "document.extract 需要 path"))?;
|
||||
let hint = params
|
||||
.get("format")
|
||||
.and_then(Value::as_str)
|
||||
.map(|s| s.to_string());
|
||||
let doc = crate::document_extract::extract_document(&crate::document_extract::DocumentSource {
|
||||
path: PathBuf::from(path),
|
||||
format_hint: hint,
|
||||
});
|
||||
Ok(doc.to_json())
|
||||
}
|
||||
|
||||
fn artifact_find(params: &Value) -> Result<Value, Value> {
|
||||
let compilation = semantic_project(params)?;
|
||||
let query = params.get("query").and_then(Value::as_str).unwrap_or("").to_ascii_lowercase();
|
||||
@@ -128,7 +189,100 @@ fn artifact_inspect(params: &Value) -> Result<Value, Value> {
|
||||
let candidates = artifact_query(&compilation.semantics, query);
|
||||
if candidates.is_empty() { return Err(err("artifact.unresolved", format!("没有制品 `{query}`"))); }
|
||||
if candidates.len() != 1 { return Err(json!({"code":"artifact.ambiguous","message":format!("制品 `{query}` 不唯一"),"candidates":candidates.iter().map(|a|&a.artifact_id.0).collect::<Vec<_>>() })); }
|
||||
Ok(crate::sema::artifact_json(candidates[0]))
|
||||
let artifact = candidates[0];
|
||||
// Version contract: expected source/generation mismatch must not silently return current.
|
||||
check_artifact_version_params(params, &compilation, artifact)?;
|
||||
Ok(crate::sema::artifact_json(artifact))
|
||||
}
|
||||
|
||||
/// Reject inspect when caller pinned an expected source revision or generation that
|
||||
/// does not match the current project snapshot (`artifact.version_mismatch`).
|
||||
fn check_artifact_version_params(
|
||||
params: &Value,
|
||||
compilation: &crate::sema::Compilation,
|
||||
artifact: &crate::sema::SemanticArtifact,
|
||||
) -> Result<(), Value> {
|
||||
let expected_rev = params
|
||||
.get("expected_source_revision")
|
||||
.and_then(Value::as_str)
|
||||
.filter(|s| !s.is_empty());
|
||||
if let Some(expected) = expected_rev {
|
||||
let project_root = params
|
||||
.get("project")
|
||||
.or_else(|| params.get("path"))
|
||||
.and_then(Value::as_str)
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| PathBuf::from("."));
|
||||
let root = if project_root.is_file() {
|
||||
project_root
|
||||
.parent()
|
||||
.unwrap_or_else(|| Path::new("."))
|
||||
.to_path_buf()
|
||||
} else {
|
||||
project_root
|
||||
};
|
||||
let abs = root.join(&artifact.source.file);
|
||||
match std::fs::read(&abs) {
|
||||
Ok(raw) => {
|
||||
use sha2::{Digest, Sha256};
|
||||
let actual = format!("sha256:{:x}", Sha256::digest(&raw));
|
||||
if actual != expected {
|
||||
return Err(json!({
|
||||
"code": "artifact.version_mismatch",
|
||||
"message": format!(
|
||||
"expected source_revision {expected}, actual {actual}; refusing silent downgrade"
|
||||
),
|
||||
"expected_source_revision": expected,
|
||||
"actual_source_revision": actual,
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
}));
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
return Err(json!({
|
||||
"code": "artifact.version_mismatch",
|
||||
"message": format!(
|
||||
"cannot verify source_revision for {}: {e}; refusing silent downgrade",
|
||||
artifact.source.file
|
||||
),
|
||||
"expected_source_revision": expected,
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let expected_gen = params
|
||||
.get("expected_generation")
|
||||
.or_else(|| params.get("generation_id"))
|
||||
.and_then(Value::as_str)
|
||||
.filter(|s| !s.is_empty());
|
||||
if let Some(expected) = expected_gen {
|
||||
let actual = semantic_generation_id(&compilation.semantics);
|
||||
if actual != expected {
|
||||
return Err(json!({
|
||||
"code": "artifact.version_mismatch",
|
||||
"message": format!(
|
||||
"expected generation {expected}, current projection is {actual}; refusing silent downgrade"
|
||||
),
|
||||
"expected_generation": expected,
|
||||
"actual_generation": actual,
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
}));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Semantic generation token matching KnowledgePresentation / DocumentView.
|
||||
fn semantic_generation_id(project: &crate::sema::SemanticProject) -> String {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut gh = Sha256::new();
|
||||
gh.update(project.project_id.as_bytes());
|
||||
for a in &project.artifacts {
|
||||
gh.update(a.hashes.artifact.0.as_bytes());
|
||||
}
|
||||
format!("{:x}", gh.finalize())[..16].to_string()
|
||||
}
|
||||
|
||||
fn link_resolve(params: &Value) -> Result<Value, Value> {
|
||||
@@ -147,10 +301,216 @@ fn artifact_verify(params: &Value) -> Result<Value, Value> {
|
||||
let candidates = artifact_query(&compilation.semantics, query);
|
||||
if candidates.len() != 1 { return Err(err(if candidates.is_empty() { "artifact.unresolved" } else { "artifact.ambiguous" }, format!("无法唯一验证 `{query}`"))); }
|
||||
let artifact = candidates[0];
|
||||
let mut record = crate::sema::artifact_json(artifact)["verification"].clone();
|
||||
record["policy"] = params.get("policy").cloned().unwrap_or_else(|| json!("static"));
|
||||
record["artifact_id"] = json!(artifact.artifact_id.0);
|
||||
Ok(record)
|
||||
let policy = params.get("policy").and_then(Value::as_str).unwrap_or("static");
|
||||
Ok(build_verification_report(artifact, policy))
|
||||
}
|
||||
|
||||
/// CLI / product machine adjudication over semantic artifacts (M03).
|
||||
pub fn adjudicate_project(project: &std::path::Path, artifact: Option<&str>, policy: &str) -> Result<Value, Value> {
|
||||
let compilation = semantic_project(&json!({"project": project}))?;
|
||||
let targets: Vec<&crate::sema::SemanticArtifact> = if let Some(q) = artifact {
|
||||
let c = artifact_query(&compilation.semantics, q);
|
||||
if c.is_empty() {
|
||||
return Err(err("artifact.unresolved", format!("没有制品 `{q}`")));
|
||||
}
|
||||
c
|
||||
} else {
|
||||
compilation
|
||||
.semantics
|
||||
.artifacts
|
||||
.iter()
|
||||
.filter(|a| a.kind == "function")
|
||||
.collect()
|
||||
};
|
||||
let reports: Vec<Value> = targets
|
||||
.iter()
|
||||
.map(|a| build_verification_report(a, policy))
|
||||
.collect();
|
||||
let any_denied = reports.iter().any(|r| r["may_bind"] == false);
|
||||
Ok(json!({
|
||||
"schema_version": 1,
|
||||
"policy": policy,
|
||||
"count": reports.len(),
|
||||
"may_bind_all": !any_denied,
|
||||
"reports": reports,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Known bind/verify policies (M03). Unknown strings are rejected.
|
||||
fn parse_policy(policy: &str) -> Result<&str, Value> {
|
||||
match policy {
|
||||
"pure" | "business" | "static" => Ok(policy),
|
||||
other => Err(json!({
|
||||
"code": "artifact.unknown_policy",
|
||||
"message": format!("未知政策 `{other}`;允许 pure / business / static"),
|
||||
"policy": other,
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
fn evidence_allows_business(status: &str) -> bool {
|
||||
matches!(status, "sourced" | "verified")
|
||||
}
|
||||
|
||||
fn build_verification_report(artifact: &crate::sema::SemanticArtifact, policy: &str) -> Value {
|
||||
let policy = match parse_policy(policy) {
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
return json!({
|
||||
"schema_version": 1,
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
"policy": policy,
|
||||
"checks": {},
|
||||
"blockers": [e],
|
||||
"may_bind": false,
|
||||
"may_publish": false,
|
||||
"verification": crate::sema::artifact_json(artifact)["verification"].clone(),
|
||||
});
|
||||
}
|
||||
};
|
||||
let region_status = if artifact.applicability.contains("region=") {
|
||||
"declared"
|
||||
} else {
|
||||
"unknown"
|
||||
};
|
||||
let checks = json!({
|
||||
"parse": {"status": artifact.verification.parse, "detail": "static_project_compile"},
|
||||
"resolve": {"status": artifact.verification.resolve, "detail": "symbol_bound"},
|
||||
"contract": {"status": artifact.verification.contract, "detail": "not_executed_by_verify"},
|
||||
"execution": {"status": artifact.verification.execution, "detail": "not_run"},
|
||||
"tests": {"status": artifact.verification.tests, "detail": "no_test_runner_invoked"},
|
||||
"evidence": {"status": artifact.verification.evidence, "detail": "compile_time_provenance"},
|
||||
"domain": {"status": artifact.verification.domain, "detail": "unreviewed"},
|
||||
"region": {"status": region_status, "detail": "input_domain_not_checked"},
|
||||
"input_domain": {"status": "unknown", "detail": "not_checked_by_static_verify"},
|
||||
});
|
||||
let mut blockers = Vec::new();
|
||||
// Never upgrade tests/execution to passed without a runner (M03).
|
||||
if matches!(
|
||||
artifact.verification.tests.as_str(),
|
||||
"passed" | "verified"
|
||||
) && artifact
|
||||
.verification
|
||||
.methods
|
||||
.iter()
|
||||
.all(|m| m == "static_project_compile")
|
||||
{
|
||||
blockers.push(json!({
|
||||
"code": "artifact.tests_unproven",
|
||||
"check": "tests",
|
||||
"status": artifact.verification.tests,
|
||||
"detail": "static compile must not claim tests passed",
|
||||
}));
|
||||
}
|
||||
if policy == "business" && !evidence_allows_business(&artifact.verification.evidence) {
|
||||
blockers.push(json!({
|
||||
"code": "artifact.evidence_denied",
|
||||
"check": "evidence",
|
||||
"status": artifact.verification.evidence,
|
||||
}));
|
||||
}
|
||||
if policy == "pure"
|
||||
&& artifact
|
||||
.effects
|
||||
.values()
|
||||
.iter()
|
||||
.any(|e| e != "pure")
|
||||
{
|
||||
blockers.push(json!({
|
||||
"code": "artifact.effect_denied",
|
||||
"check": "effects",
|
||||
"effects": artifact.effects.values(),
|
||||
}));
|
||||
}
|
||||
let may_bind = blockers.is_empty()
|
||||
&& match policy {
|
||||
"business" => evidence_allows_business(&artifact.verification.evidence),
|
||||
"pure" => artifact.effects.values().iter().all(|e| e == "pure"),
|
||||
_ => true,
|
||||
};
|
||||
// Publish requires stronger evidence than bind; static verify never claims tests passed.
|
||||
let may_publish = may_bind
|
||||
&& artifact.verification.tests != "unknown"
|
||||
&& artifact.verification.tests != "not_run"
|
||||
&& artifact.verification.execution != "not_run"
|
||||
&& !matches!(
|
||||
artifact.verification.evidence.as_str(),
|
||||
"unverified" | "missing_source" | "anchor_missing" | "expired" | "invalid_date"
|
||||
);
|
||||
json!({
|
||||
"schema_version": 1,
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
"symbol_id": artifact.symbol_id.as_ref().map(|id| id.as_str()),
|
||||
"digest": artifact.hashes.artifact.0,
|
||||
"evidence_digest": artifact.hashes.evidence.0,
|
||||
"closure_digest": artifact.hashes.closure.0,
|
||||
"policy": policy,
|
||||
"checks": checks,
|
||||
"blockers": blockers,
|
||||
"may_bind": may_bind,
|
||||
"may_publish": may_publish,
|
||||
"verification": {
|
||||
"parse": artifact.verification.parse,
|
||||
"resolve": artifact.verification.resolve,
|
||||
"contract": artifact.verification.contract,
|
||||
"execution": artifact.verification.execution,
|
||||
"tests": artifact.verification.tests,
|
||||
"evidence": artifact.verification.evidence,
|
||||
"domain": artifact.verification.domain,
|
||||
"methods": artifact.verification.methods,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
/// M04: bind then execute in one call (actual reuse, not discovery scores).
|
||||
fn artifact_reuse_execute(params: &Value) -> Result<Value, Value> {
|
||||
let project = params
|
||||
.get("project")
|
||||
.or_else(|| params.get("path"))
|
||||
.cloned()
|
||||
.ok_or_else(|| err("mlsp.bad_request", "artifact.reuse_execute 需要 project"))?;
|
||||
let query = params
|
||||
.get("artifact_id")
|
||||
.or_else(|| params.get("symbol_id"))
|
||||
.or_else(|| params.get("query"))
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| err("mlsp.bad_request", "artifact.reuse_execute 需要 artifact_id"))?;
|
||||
let policy = params.get("policy").and_then(Value::as_str).unwrap_or("pure");
|
||||
let inputs = params.get("inputs").cloned().unwrap_or(json!({}));
|
||||
let bound = artifact_bind(&json!({
|
||||
"project": project,
|
||||
"artifact_id": query,
|
||||
"policy": policy,
|
||||
"expected_digest": params.get("expected_digest"),
|
||||
}))?;
|
||||
let handle = bound
|
||||
.get("handle")
|
||||
.cloned()
|
||||
.ok_or_else(|| err("artifact.bind_failed", "bind 未返回 handle"))?;
|
||||
let executed = artifact_execute(&json!({
|
||||
"handle": handle,
|
||||
"inputs": inputs,
|
||||
"allow_effects": params.get("allow_effects"),
|
||||
}))?;
|
||||
Ok(json!({
|
||||
"schema_version": 1,
|
||||
"actual_reuse": true,
|
||||
"discovery_only": false,
|
||||
"artifact_id": executed.get("artifact_id"),
|
||||
"symbol_id": executed.get("symbol_id"),
|
||||
"digest": executed.get("digest"),
|
||||
"result": executed.get("result"),
|
||||
"stdout": executed.get("stdout"),
|
||||
"effects": executed.get("effects"),
|
||||
"trace": {
|
||||
"artifact_id": executed.get("artifact_id"),
|
||||
"symbol_id": executed.get("symbol_id"),
|
||||
"digest": executed.get("digest"),
|
||||
"inputs": inputs,
|
||||
"value": executed.get("result"),
|
||||
"stdout": executed.get("stdout"),
|
||||
},
|
||||
}))
|
||||
}
|
||||
|
||||
fn artifact_impact(params: &Value) -> Result<Value, Value> {
|
||||
@@ -171,7 +531,208 @@ fn artifact_bind(params: &Value) -> Result<Value, Value> {
|
||||
if expected.is_some() && expected != Some(artifact.hashes.artifact.0.as_str()) {
|
||||
return Err(json!({"code":"artifact.digest_mismatch","message":"目标摘要已变化,拒绝绑定","expected":expected,"actual":artifact.hashes.artifact.0}));
|
||||
}
|
||||
Ok(json!({"schema_version":1,"handle":{"artifact_id":artifact.artifact_id.0,"digest":artifact.hashes.artifact.0,"project":compilation.semantics.root,"policy":params.get("policy").cloned().unwrap_or_else(||json!("pure")),"effects":artifact.effects.values()}}))
|
||||
let policy = parse_policy(
|
||||
params
|
||||
.get("policy")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("pure"),
|
||||
)?;
|
||||
let effects = artifact.effects.values();
|
||||
if policy == "pure"
|
||||
&& effects.iter().any(|effect| effect != "pure")
|
||||
{
|
||||
return Err(json!({
|
||||
"code": "artifact.effect_denied",
|
||||
"message": "pure 策略下不能绑定含非纯/未知效应的制品",
|
||||
"policy": policy,
|
||||
"effects": effects,
|
||||
}));
|
||||
}
|
||||
if policy == "business" && !evidence_allows_business(&artifact.verification.evidence) {
|
||||
return Err(json!({
|
||||
"code": "artifact.evidence_denied",
|
||||
"message": "business 策略要求来源已关联且锚点有效、未过期;当前证据状态不足",
|
||||
"policy": policy,
|
||||
"evidence": artifact.verification.evidence,
|
||||
}));
|
||||
}
|
||||
let symbol_id = artifact
|
||||
.symbol_id
|
||||
.as_ref()
|
||||
.map(|id| id.as_str().to_string())
|
||||
.unwrap_or_else(|| artifact.artifact_id.0.clone());
|
||||
let valid_until = artifact
|
||||
.applicability
|
||||
.split(';')
|
||||
.find_map(|part| {
|
||||
part.trim()
|
||||
.strip_prefix("valid_until=")
|
||||
.map(|s| s.trim().to_string())
|
||||
});
|
||||
let provenance_digests: Vec<Value> = artifact
|
||||
.provenance
|
||||
.iter()
|
||||
.filter(|p| p.relation == "supports")
|
||||
.map(|p| {
|
||||
json!({
|
||||
"uri": p.uri,
|
||||
"locator": p.locator,
|
||||
"content_digest": p.content_digest.as_ref().map(|d| &d.0),
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
Ok(json!({"schema_version":1,"handle":{
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
"symbol_id": symbol_id,
|
||||
"digest": artifact.hashes.artifact.0,
|
||||
"closure_digest": artifact.hashes.closure.0,
|
||||
"evidence_digest": artifact.hashes.evidence.0,
|
||||
"project": compilation.semantics.root,
|
||||
"policy": policy,
|
||||
"effects": effects,
|
||||
"evidence": artifact.verification.evidence,
|
||||
"valid_until": valid_until,
|
||||
"provenance": provenance_digests,
|
||||
"source": {"file": artifact.source.file, "line": artifact.source.line},
|
||||
}}))
|
||||
}
|
||||
|
||||
/// Digest-guarded local patch (S3/G09). `intent: wording` refuses signature/effect drift.
|
||||
fn artifact_patch(params: &Value) -> Result<Value, Value> {
|
||||
let compilation = semantic_project(params)?;
|
||||
let query = params
|
||||
.get("artifact_id")
|
||||
.or_else(|| params.get("symbol_id"))
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| err("mlsp.bad_request", "artifact.patch 需要 artifact_id 或 symbol_id"))?;
|
||||
let candidates = artifact_query(&compilation.semantics, query);
|
||||
if candidates.len() != 1 {
|
||||
return Err(err(
|
||||
if candidates.is_empty() {
|
||||
"artifact.unresolved"
|
||||
} else {
|
||||
"artifact.ambiguous"
|
||||
},
|
||||
format!("无法唯一修补 `{query}`"),
|
||||
));
|
||||
}
|
||||
let before = candidates[0];
|
||||
let expected = params
|
||||
.get("expected_digest")
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| {
|
||||
err(
|
||||
"mlsp.bad_request",
|
||||
"artifact.patch 需要 expected_digest(hashes.implementation)",
|
||||
)
|
||||
})?;
|
||||
if expected != before.hashes.implementation.0 {
|
||||
return Err(json!({
|
||||
"code": "artifact.digest_mismatch",
|
||||
"message": "源码实现摘要已变化,拒绝覆盖",
|
||||
"expected": expected,
|
||||
"actual": before.hashes.implementation.0,
|
||||
}));
|
||||
}
|
||||
let path = compilation.semantics.root.join(&before.source.file);
|
||||
let original = std::fs::read_to_string(&path)
|
||||
.map_err(|e| err("artifact.load", format!("读源失败: {e}")))?;
|
||||
let edits_v = params
|
||||
.get("edits")
|
||||
.and_then(Value::as_array)
|
||||
.ok_or_else(|| err("mlsp.bad_request", "artifact.patch 需要 edits"))?;
|
||||
let mut edits = Vec::new();
|
||||
for e in edits_v {
|
||||
let line = e
|
||||
.get("line")
|
||||
.and_then(Value::as_u64)
|
||||
.ok_or_else(|| err("mlsp.bad_request", "edit.line 必填"))? as u32;
|
||||
let kind = e.get("op").and_then(Value::as_str).unwrap_or("replace");
|
||||
match kind {
|
||||
"replace" => {
|
||||
let text = e
|
||||
.get("text")
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(|| err("mlsp.bad_request", "replace 需要 text"))?
|
||||
.to_string();
|
||||
edits.push(crate::repair::Edit::Replace { line, text });
|
||||
}
|
||||
"delete" => edits.push(crate::repair::Edit::Delete { line }),
|
||||
"insert_before" => {
|
||||
let text = e
|
||||
.get("text")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
edits.push(crate::repair::Edit::InsertBefore { line, text });
|
||||
}
|
||||
other => {
|
||||
return Err(err(
|
||||
"mlsp.bad_request",
|
||||
format!("未知 edit.op `{other}`"),
|
||||
))
|
||||
}
|
||||
}
|
||||
}
|
||||
let start = before.source.line;
|
||||
let end = before.source.end_line.max(start);
|
||||
let ranges = [(start, end.max(start + 40))];
|
||||
let patched = crate::repair::apply_edits(&original, &ranges, &edits).map_err(|o| {
|
||||
json!({
|
||||
"code": "mlsp.repair_out_of_scope",
|
||||
"message": "补丁超出制品源码范围",
|
||||
"lines": o.lines,
|
||||
"ranges": o.ranges,
|
||||
})
|
||||
})?;
|
||||
std::fs::write(&path, &patched)
|
||||
.map_err(|e| err("artifact.write", format!("写源失败: {e}")))?;
|
||||
let after_comp = semantic_project(params)?;
|
||||
let after_cands = artifact_query(&after_comp.semantics, query);
|
||||
if after_cands.len() != 1 {
|
||||
// roll back
|
||||
let _ = std::fs::write(&path, &original);
|
||||
return Err(err(
|
||||
"artifact.patch_ambiguous",
|
||||
"补丁后制品无法唯一解析;已回滚",
|
||||
));
|
||||
}
|
||||
let after = after_cands[0];
|
||||
let signature_changed = before.hashes.interface.0 != after.hashes.interface.0;
|
||||
let effects_changed = before.effects.values() != after.effects.values();
|
||||
let implementation_changed =
|
||||
before.hashes.implementation.0 != after.hashes.implementation.0;
|
||||
let intent = params.get("intent").and_then(Value::as_str).unwrap_or("edit");
|
||||
if intent == "wording" && (signature_changed || effects_changed) {
|
||||
let _ = std::fs::write(&path, &original);
|
||||
return Err(json!({
|
||||
"code": "artifact.wording_broke_contract",
|
||||
"message": "intent=wording 但签名或效应已变;已回滚",
|
||||
"signature_changed": signature_changed,
|
||||
"effects_changed": effects_changed,
|
||||
}));
|
||||
}
|
||||
Ok(json!({
|
||||
"schema_version": 1,
|
||||
"artifact_id": after.artifact_id.0,
|
||||
"symbol_id": after.symbol_id.as_ref().map(|id| id.as_str()),
|
||||
"applied": true,
|
||||
"diff": {
|
||||
"signature_changed": signature_changed,
|
||||
"effects_changed": effects_changed,
|
||||
"implementation_changed": implementation_changed,
|
||||
"before": {
|
||||
"interface": before.hashes.interface.0,
|
||||
"implementation": before.hashes.implementation.0,
|
||||
"effects": before.effects.values(),
|
||||
},
|
||||
"after": {
|
||||
"interface": after.hashes.interface.0,
|
||||
"implementation": after.hashes.implementation.0,
|
||||
"effects": after.effects.values(),
|
||||
},
|
||||
},
|
||||
}))
|
||||
}
|
||||
|
||||
fn artifact_execute(params: &Value) -> Result<Value, Value> {
|
||||
@@ -183,7 +744,76 @@ fn artifact_execute(params: &Value) -> Result<Value, Value> {
|
||||
let candidates = artifact_query(&compilation.semantics, artifact_id);
|
||||
if candidates.len() != 1 { return Err(err("artifact.stale_handle", "制品不再可唯一解析;请重新 bind")); }
|
||||
let artifact = candidates[0];
|
||||
if artifact.hashes.artifact.0 != digest { return Err(json!({"code":"artifact.stale_handle","message":"源码或契约已改变;拒绝执行过期 handle","expected":digest,"actual":artifact.hashes.artifact.0})); }
|
||||
if artifact.hashes.artifact.0 != digest { return Err(json!({"code":"artifact.stale_handle","message":"源码、契约或来源证据已改变;拒绝执行过期 handle","expected":digest,"actual":artifact.hashes.artifact.0})); }
|
||||
if let Some(bound_closure) = handle.get("closure_digest").and_then(Value::as_str) {
|
||||
if bound_closure != artifact.hashes.closure.0 {
|
||||
return Err(json!({
|
||||
"code": "artifact.stale_handle",
|
||||
"message": "依赖闭包已改变;拒绝执行过期 handle",
|
||||
"expected": bound_closure,
|
||||
"actual": artifact.hashes.closure.0,
|
||||
}));
|
||||
}
|
||||
}
|
||||
if let Some(bound_evidence) = handle.get("evidence_digest").and_then(Value::as_str) {
|
||||
if bound_evidence != artifact.hashes.evidence.0 {
|
||||
return Err(json!({
|
||||
"code": "artifact.stale_handle",
|
||||
"message": "来源证据摘要已改变;拒绝执行过期 handle",
|
||||
"expected": bound_evidence,
|
||||
"actual": artifact.hashes.evidence.0,
|
||||
}));
|
||||
}
|
||||
}
|
||||
if let Some(bound_symbol) = handle.get("symbol_id").and_then(Value::as_str) {
|
||||
let current = artifact
|
||||
.symbol_id
|
||||
.as_ref()
|
||||
.map(|id| id.as_str())
|
||||
.unwrap_or(artifact.artifact_id.0.as_str());
|
||||
if bound_symbol != current {
|
||||
return Err(json!({
|
||||
"code": "artifact.stale_handle",
|
||||
"message": "handle 绑定的 SymbolId 与当前制品不一致",
|
||||
"expected": bound_symbol,
|
||||
"actual": current,
|
||||
}));
|
||||
}
|
||||
}
|
||||
// Re-check policy at execute time (M02) — do not trust bind-time evidence string alone.
|
||||
let policy = handle.get("policy").and_then(Value::as_str).unwrap_or("pure");
|
||||
let _ = parse_policy(policy)?;
|
||||
if policy == "business" && !evidence_allows_business(&artifact.verification.evidence) {
|
||||
return Err(json!({
|
||||
"code": "artifact.evidence_denied",
|
||||
"message": "business 句柄在执行时证据状态不足(来源缺失、锚点失效或已过期)",
|
||||
"policy": policy,
|
||||
"evidence": artifact.verification.evidence,
|
||||
}));
|
||||
}
|
||||
if policy == "pure"
|
||||
&& artifact
|
||||
.effects
|
||||
.values()
|
||||
.iter()
|
||||
.any(|effect| effect != "pure")
|
||||
{
|
||||
return Err(json!({
|
||||
"code": "artifact.effect_denied",
|
||||
"message": "pure 句柄在执行时出现非纯效应",
|
||||
"policy": policy,
|
||||
"effects": artifact.effects.values(),
|
||||
}));
|
||||
}
|
||||
if let Some(until) = handle.get("valid_until").and_then(Value::as_str) {
|
||||
if crate::sema::provenance::valid_until_expired(until) {
|
||||
return Err(json!({
|
||||
"code": "artifact.expired",
|
||||
"message": "句柄适用期已过",
|
||||
"valid_until": until,
|
||||
}));
|
||||
}
|
||||
}
|
||||
if artifact.kind != "function" || artifact.callable.as_deref() == Some("instance_method") { return Err(err("artifact.not_executable", "只有非实例函数制品可由 artifact.execute 直接执行")); }
|
||||
let allowed: Vec<&str> = params.get("allow_effects").and_then(Value::as_array).map(|items| items.iter().filter_map(Value::as_str).collect()).unwrap_or_default();
|
||||
let denied: Vec<String> = artifact.effects.values().into_iter().filter(|effect| effect != "pure" && !allowed.contains(&effect.as_str())).collect();
|
||||
@@ -197,10 +827,44 @@ fn artifact_execute(params: &Value) -> Result<Value, Value> {
|
||||
let caps = crate::host::HostCaps { fs_write: allowed.contains(&"fs.write"), exec: allowed.iter().any(|e| matches!(*e, "process.exec" | "plugin.call" | "llm.call")), net: allowed.iter().any(|e| matches!(*e, "network.read" | "network.write" | "llm.call")), plugin: allowed.contains(&"plugin.call") };
|
||||
let host = crate::host::HostContext::for_capture(Some(&path), caps);
|
||||
let mut interpreter = crate::interp::Interpreter::with_capture(Some(&path), false).with_host(host);
|
||||
let name = artifact.display_name.rsplit('.').next().unwrap_or(&artifact.display_name);
|
||||
let value = interpreter.invoke_function(&module, name, &args).map_err(|e| diagnostic_err("artifact.execution_failed", &e))?;
|
||||
// Bind by source line / declaration identity — never by bare display_name suffix (G07).
|
||||
let fun = find_function_at_line(&module.functions, artifact.source.line).ok_or_else(|| {
|
||||
err(
|
||||
"artifact.unresolved",
|
||||
format!(
|
||||
"无法在 {}:{} 定位绑定函数 {}",
|
||||
artifact.source.file, artifact.source.line, artifact.display_name
|
||||
),
|
||||
)
|
||||
})?;
|
||||
let value = interpreter
|
||||
.invoke_bound_function(&module, fun, &args)
|
||||
.map_err(|e| diagnostic_err("artifact.execution_failed", &e))?;
|
||||
let stdout = std::mem::take(&mut interpreter.captured_stdout);
|
||||
Ok(json!({"schema_version":1,"artifact_id":artifact.artifact_id.0,"digest":digest,"result":crate::value_as_json(&value).map_err(|e|err("artifact.serialization",e))?,"stdout":stdout,"effects":artifact.effects.values()}))
|
||||
Ok(json!({
|
||||
"schema_version": 1,
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
"symbol_id": artifact.symbol_id.as_ref().map(|id| id.as_str()),
|
||||
"digest": digest,
|
||||
"result": crate::value_as_json(&value).map_err(|e| err("artifact.serialization", e))?,
|
||||
"stdout": stdout,
|
||||
"effects": artifact.effects.values(),
|
||||
}))
|
||||
}
|
||||
|
||||
fn find_function_at_line<'a>(
|
||||
functions: &'a [crate::ast::Function],
|
||||
line: u32,
|
||||
) -> Option<&'a crate::ast::Function> {
|
||||
for fun in functions {
|
||||
if fun.span.line == line {
|
||||
return Some(fun);
|
||||
}
|
||||
if let Some(found) = find_function_at_line(&fun.children, line) {
|
||||
return Some(found);
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn err(code: &str, message: impl Into<String>) -> Value {
|
||||
@@ -658,9 +1322,32 @@ mod tests {
|
||||
let candidate = found["candidates"].as_array().unwrap().iter().find(|v| v["kind"] == "function").unwrap();
|
||||
let id = candidate["artifact_id"].as_str().unwrap();
|
||||
let bound = dispatch(&json!({"method":"artifact.bind","params":{"project":root,"artifact_id":id}})).unwrap();
|
||||
let handle = bound["handle"].clone();
|
||||
let handle = &bound["handle"];
|
||||
assert!(handle.get("symbol_id").and_then(|v| v.as_str()).is_some());
|
||||
assert!(handle.get("source").is_some());
|
||||
let result = dispatch(&json!({"method":"artifact.execute","params":{"handle":handle,"inputs":{"x":4}}})).unwrap();
|
||||
assert_eq!(result["result"], 5);
|
||||
assert_eq!(result["symbol_id"], handle["symbol_id"]);
|
||||
fs::remove_dir_all(root).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn artifact_bind_rejects_non_pure_under_pure_policy() {
|
||||
let stamp = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos();
|
||||
let root = std::env::temp_dir().join(format!("marqdo-mlsp-effect-{stamp}"));
|
||||
fs::create_dir_all(&root).unwrap();
|
||||
fs::write(
|
||||
root.join("main.mq.md"),
|
||||
"# main\n\n## save\n\n> fs.write_text path=t.txt text=x\n\n*1*\n",
|
||||
)
|
||||
.unwrap();
|
||||
let found = dispatch(&json!({"method":"artifact.find","params":{"project":root,"query":"save"}})).unwrap();
|
||||
let id = found["candidates"].as_array().unwrap().iter().find(|v| v["kind"] == "function").unwrap()["artifact_id"]
|
||||
.as_str()
|
||||
.unwrap();
|
||||
let err = dispatch(&json!({"method":"artifact.bind","params":{"project":root,"artifact_id":id,"policy":"pure"}}))
|
||||
.unwrap_err();
|
||||
assert_eq!(err["code"], "artifact.effect_denied");
|
||||
fs::remove_dir_all(root).unwrap();
|
||||
}
|
||||
|
||||
|
||||
+4
-1
@@ -7,7 +7,10 @@ pub use expr::{
|
||||
parse_call_after_gt, parse_call_arg_value, parse_expr, parse_expr_prefer_var,
|
||||
parse_value_or_interp,
|
||||
};
|
||||
pub use prose::{scan_prose_line, ProseBit};
|
||||
pub use prose::{
|
||||
classify_narrative_target, scan_md_links_in_line, scan_narrative_links, scan_prose_line,
|
||||
NarrativeLink, NarrativeLinkKind, ProseBit,
|
||||
};
|
||||
|
||||
use anyhow::{bail, Result};
|
||||
|
||||
|
||||
+186
-1
@@ -2,8 +2,11 @@
|
||||
//!
|
||||
//! Soft Markdown emphasis that does not look like code/return is skipped (G2 —
|
||||
//! [code-as-docs-gaps.md](../../../doc/design/code-as-docs-gaps.md)).
|
||||
//!
|
||||
//! Comment-line Markdown links `[label](target)` are NarrativeRef / navigation
|
||||
//! sites (G03); they are never executed.
|
||||
|
||||
use crate::lex::{matching_bold_close_inline, matching_italic_close_inline};
|
||||
use crate::lex::{matching_bold_close_inline, matching_italic_close_inline, LineKind};
|
||||
|
||||
pub use crate::lex::looks_like_bold_code;
|
||||
|
||||
@@ -22,6 +25,162 @@ pub enum ProseBit {
|
||||
ItalicReturn { inner: String, at: usize },
|
||||
}
|
||||
|
||||
/// Why a comment-line `[]()` is not an executable BoundCall (G03).
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum NarrativeLinkKind {
|
||||
/// In-project symbol / binding / `#anchor` — inert explanation.
|
||||
Narrate,
|
||||
/// URL or file path — navigation only; never fetched by compile/run.
|
||||
Navigation,
|
||||
}
|
||||
|
||||
impl NarrativeLinkKind {
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Narrate => "narrate",
|
||||
Self::Navigation => "navigation",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One Markdown link found on a comment (narrative) line.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct NarrativeLink {
|
||||
pub label: String,
|
||||
pub target: String,
|
||||
pub kind: NarrativeLinkKind,
|
||||
/// 1-based source line.
|
||||
pub line: u32,
|
||||
pub at: usize,
|
||||
}
|
||||
|
||||
/// Classify a link target without parsing it as an expression.
|
||||
pub fn classify_narrative_target(target: &str) -> NarrativeLinkKind {
|
||||
let t = target.trim();
|
||||
if t.starts_with("http://")
|
||||
|| t.starts_with("https://")
|
||||
|| t.starts_with("mailto:")
|
||||
|| t.starts_with("//")
|
||||
{
|
||||
return NarrativeLinkKind::Navigation;
|
||||
}
|
||||
if t.contains('/') || t.ends_with(".md") || t.ends_with(".mq.md") || t.ends_with(".html")
|
||||
{
|
||||
return NarrativeLinkKind::Navigation;
|
||||
}
|
||||
NarrativeLinkKind::Narrate
|
||||
}
|
||||
|
||||
/// Scan whole-source comment lines for Markdown links; skip fences and code lines.
|
||||
pub fn scan_narrative_links(source: &str) -> Vec<NarrativeLink> {
|
||||
let classified = crate::lex::classify_source(source);
|
||||
let mut out = Vec::new();
|
||||
let mut in_fence = false;
|
||||
for (i, cl) in classified.iter().enumerate() {
|
||||
let line_no = (i + 1) as u32;
|
||||
let trimmed = cl.text.trim();
|
||||
if trimmed.starts_with("```") {
|
||||
in_fence = !in_fence;
|
||||
continue;
|
||||
}
|
||||
if in_fence {
|
||||
continue;
|
||||
}
|
||||
if cl.kind != LineKind::Comment {
|
||||
continue;
|
||||
}
|
||||
out.extend(scan_md_links_in_line(&cl.text, line_no));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Scan one line for `[label](target)`, skipping `` `inline` `` spans.
|
||||
pub fn scan_md_links_in_line(line: &str, line_no: u32) -> Vec<NarrativeLink> {
|
||||
let bytes = line.as_bytes();
|
||||
let mut i = 0usize;
|
||||
let mut out = Vec::new();
|
||||
while i < bytes.len() {
|
||||
if bytes[i] == b'`' {
|
||||
if let Some(rel) = line[i + 1..].find('`') {
|
||||
i = i + 1 + rel + 1;
|
||||
continue;
|
||||
}
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
if bytes[i] != b'[' {
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
let Some((label, target, next)) = parse_md_link(line, i) else {
|
||||
i += 1;
|
||||
continue;
|
||||
};
|
||||
let kind = classify_narrative_target(&target);
|
||||
out.push(NarrativeLink {
|
||||
label,
|
||||
target,
|
||||
kind,
|
||||
line: line_no,
|
||||
at: i,
|
||||
});
|
||||
i = next;
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn parse_md_link(line: &str, start: usize) -> Option<(String, String, usize)> {
|
||||
if !line[start..].starts_with('[') {
|
||||
return None;
|
||||
}
|
||||
let mut depth = 0i32;
|
||||
let mut label_end = None;
|
||||
for (rel, ch) in line[start..].char_indices() {
|
||||
match ch {
|
||||
'[' => depth += 1,
|
||||
']' => {
|
||||
depth -= 1;
|
||||
if depth == 0 {
|
||||
label_end = Some(start + rel);
|
||||
break;
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
let label_end = label_end?;
|
||||
let after = line.get(label_end + 1..)?;
|
||||
if !after.starts_with('(') {
|
||||
return None;
|
||||
}
|
||||
let mut depth = 0i32;
|
||||
let mut target_end = None;
|
||||
for (rel, ch) in after.char_indices() {
|
||||
match ch {
|
||||
'(' => depth += 1,
|
||||
')' => {
|
||||
depth -= 1;
|
||||
if depth == 0 {
|
||||
target_end = Some(rel);
|
||||
break;
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
let target_end = target_end?;
|
||||
let label = line[start + 1..label_end].to_string();
|
||||
let target = after[1..target_end].trim().to_string();
|
||||
if label.is_empty() || target.is_empty() {
|
||||
return None;
|
||||
}
|
||||
// Executable occlusion payloads look like calls (`foo(…)` / `a.b(…)`); those
|
||||
// live on Code lines already. If a comment link target is expression-shaped
|
||||
// with `(`, still treat as narrate/navigation by target class — do not eval.
|
||||
let next = label_end + 1 + target_end + 1;
|
||||
Some((label, target, next))
|
||||
}
|
||||
|
||||
/// Scan a narrative (or mixed) line for v0.3 inline bits, left to right.
|
||||
pub fn scan_prose_line(line: &str) -> Vec<ProseBit> {
|
||||
let bytes = line.as_bytes();
|
||||
@@ -255,4 +414,30 @@ mod tests {
|
||||
"glob-like ticks must not yield italic: {bits:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn narrative_links_skip_fence_and_classify() {
|
||||
let src = "---\nmarkup: occlusion\n---\n\n# main\n\n见[文档](https://example.com/a)与[总额](总额)。\n\n```\n[ignored](https://x.test)\n```\n\n行内跳过 `[not](x)` 标记。\n";
|
||||
let links = scan_narrative_links(src);
|
||||
assert!(
|
||||
links.iter().any(|l| {
|
||||
l.kind == NarrativeLinkKind::Navigation && l.target.contains("example.com")
|
||||
}),
|
||||
"{links:?}"
|
||||
);
|
||||
assert!(
|
||||
links
|
||||
.iter()
|
||||
.any(|l| l.kind == NarrativeLinkKind::Narrate && l.target == "总额"),
|
||||
"{links:?}"
|
||||
);
|
||||
assert!(
|
||||
!links.iter().any(|l| l.target.contains("x.test")),
|
||||
"fenced link must not become a step: {links:?}"
|
||||
);
|
||||
assert!(
|
||||
!links.iter().any(|l| l.target == "x"),
|
||||
"inline-code link must not become a step: {links:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
+64
-5
@@ -72,16 +72,42 @@ impl ProjectRoot {
|
||||
}
|
||||
}
|
||||
|
||||
const EXCLUDED_DIRS: &[&str] = &[
|
||||
/// Shared with EKC extract (M06): authored sources never include corpus/runtime bags.
|
||||
pub const EXCLUDED_DIRS: &[&str] = &[
|
||||
".git",
|
||||
".marqdo",
|
||||
"target",
|
||||
"node_modules",
|
||||
".cursor",
|
||||
".agents",
|
||||
"data",
|
||||
"docker",
|
||||
"drafts",
|
||||
"candidates",
|
||||
"corpus",
|
||||
];
|
||||
|
||||
/// True when a directory basename is outside the authored source scan.
|
||||
pub fn is_excluded_dir_name(name: &str) -> bool {
|
||||
EXCLUDED_DIRS.iter().any(|excluded| name == *excluded)
|
||||
}
|
||||
|
||||
/// Soft limits so discovery fails with a clear diagnosis instead of hanging (G12).
|
||||
pub const MAX_SOURCE_DEPTH: u32 = 32;
|
||||
pub const MAX_SOURCE_FILE_BYTES: u64 = 2 * 1024 * 1024;
|
||||
pub const MAX_SOURCE_FILES: usize = 10_000;
|
||||
|
||||
fn collect_sources(root: &Path, dir: &Path, out: &mut Vec<PathBuf>) -> Result<()> {
|
||||
collect_sources_at(root, dir, out, 0)
|
||||
}
|
||||
|
||||
fn collect_sources_at(root: &Path, dir: &Path, out: &mut Vec<PathBuf>, depth: u32) -> Result<()> {
|
||||
if depth > MAX_SOURCE_DEPTH {
|
||||
bail!(
|
||||
"project.scan_depth_exceeded: directory depth > {MAX_SOURCE_DEPTH} under {}",
|
||||
dir.display()
|
||||
);
|
||||
}
|
||||
let mut entries: Vec<_> = fs::read_dir(dir)
|
||||
.with_context(|| format!("read project directory {}", dir.display()))?
|
||||
.collect::<std::result::Result<Vec<_>, _>>()?;
|
||||
@@ -100,9 +126,9 @@ fn collect_sources(root: &Path, dir: &Path, out: &mut Vec<PathBuf>) -> Result<()
|
||||
continue;
|
||||
}
|
||||
if canonical.is_dir() {
|
||||
collect_sources(root, &canonical, out)?;
|
||||
collect_sources_at(root, &canonical, out, depth + 1)?;
|
||||
} else if is_mq_md(&canonical) {
|
||||
out.push(canonical);
|
||||
push_source(out, &canonical)?;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
@@ -111,14 +137,33 @@ fn collect_sources(root: &Path, dir: &Path, out: &mut Vec<PathBuf>) -> Result<()
|
||||
if EXCLUDED_DIRS.iter().any(|excluded| name == *excluded) {
|
||||
continue;
|
||||
}
|
||||
collect_sources(root, &path, out)?;
|
||||
collect_sources_at(root, &path, out, depth + 1)?;
|
||||
} else if file_type.is_file() && is_mq_md(&path) {
|
||||
out.push(path);
|
||||
push_source(out, &path)?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn push_source(out: &mut Vec<PathBuf>, path: &Path) -> Result<()> {
|
||||
if out.len() >= MAX_SOURCE_FILES {
|
||||
bail!(
|
||||
"project.scan_count_exceeded: more than {MAX_SOURCE_FILES} `.mq.md` sources"
|
||||
);
|
||||
}
|
||||
let meta = fs::metadata(path)
|
||||
.with_context(|| format!("stat {}", path.display()))?;
|
||||
if meta.len() > MAX_SOURCE_FILE_BYTES {
|
||||
bail!(
|
||||
"project.scan_file_too_large: {} is {} bytes (max {MAX_SOURCE_FILE_BYTES})",
|
||||
path.display(),
|
||||
meta.len()
|
||||
);
|
||||
}
|
||||
out.push(path.to_path_buf());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn is_mq_md(path: &Path) -> bool {
|
||||
path.file_name()
|
||||
.and_then(|name| name.to_str())
|
||||
@@ -173,6 +218,20 @@ mod tests {
|
||||
fs::remove_dir_all(root).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn oversized_source_is_diagnosed() {
|
||||
let root = fixture_root("too-large");
|
||||
let big = "x".repeat((MAX_SOURCE_FILE_BYTES as usize) + 8);
|
||||
fs::write(root.join("big.mq.md"), format!("# main\n\n{big}\n")).unwrap();
|
||||
let project = ProjectRoot::new(&root).unwrap();
|
||||
let err = project.source_files().unwrap_err().to_string();
|
||||
assert!(
|
||||
err.contains("project.scan_file_too_large"),
|
||||
"expected size diagnostic, got {err}"
|
||||
);
|
||||
fs::remove_dir_all(root).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nearest_marker_wins() {
|
||||
let root = fixture_root("discover");
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
//! Task control plane: cancel without interpreter lock (M-R3).
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::runtime::protocol::RuntimeEventV2;
|
||||
use crate::runtime::scheduler::{AdmissionClass, ScheduleError, Scheduler};
|
||||
use crate::runtime::task_store::{StoreError, TaskRecord, TaskState, TaskStore};
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum ControlError {
|
||||
Store(StoreError),
|
||||
Schedule(ScheduleError),
|
||||
Forbidden,
|
||||
}
|
||||
|
||||
impl From<StoreError> for ControlError {
|
||||
fn from(e: StoreError) -> Self {
|
||||
Self::Store(e)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<ScheduleError> for ControlError {
|
||||
fn from(e: ScheduleError) -> Self {
|
||||
Self::Schedule(e)
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for ControlError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::Store(e) => write!(f, "{e}"),
|
||||
Self::Schedule(e) => write!(f, "{e}"),
|
||||
Self::Forbidden => write!(f, "forbidden"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct ControlService {
|
||||
pub store: Arc<TaskStore>,
|
||||
pub scheduler: Arc<Scheduler>,
|
||||
/// Per-lease cancel token directory (token files, not process-global env).
|
||||
cancel_dir: PathBuf,
|
||||
}
|
||||
|
||||
impl ControlService {
|
||||
pub fn new(store: Arc<TaskStore>, scheduler: Arc<Scheduler>, cancel_dir: impl AsRef<Path>) -> Self {
|
||||
let cancel_dir = cancel_dir.as_ref().to_path_buf();
|
||||
let _ = std::fs::create_dir_all(&cancel_dir);
|
||||
Self {
|
||||
store,
|
||||
scheduler,
|
||||
cancel_dir,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn admit(
|
||||
&self,
|
||||
task_id: &str,
|
||||
run_id: &str,
|
||||
workspace_id: &str,
|
||||
session_id: Option<String>,
|
||||
idempotency_key: &str,
|
||||
payload: &Value,
|
||||
class: AdmissionClass,
|
||||
) -> Result<(TaskRecord, usize), ControlError> {
|
||||
let rec = self.store.admit(
|
||||
task_id,
|
||||
run_id,
|
||||
workspace_id,
|
||||
session_id,
|
||||
idempotency_key,
|
||||
payload,
|
||||
)?;
|
||||
let depth = self.scheduler.enqueue(&rec, class)?;
|
||||
let queued = self.store.cas_state(
|
||||
&rec.task_id,
|
||||
rec.revision,
|
||||
TaskState::Queued,
|
||||
|_| {},
|
||||
)?;
|
||||
Ok((queued, depth))
|
||||
}
|
||||
|
||||
/// Cancel path never enters the Marqdo interpreter lock.
|
||||
pub fn cancel(&self, task_id: &str) -> Result<TaskRecord, ControlError> {
|
||||
let rec = self.store.request_cancel(task_id)?;
|
||||
let tok = self.cancel_token_path(task_id);
|
||||
let _ = std::fs::write(&tok, b"cancelled");
|
||||
let _ = self.store.append_event(
|
||||
task_id,
|
||||
&rec.run_id,
|
||||
&rec.attempt_id,
|
||||
"run.cancel_requested",
|
||||
serde_json::json!({"revision": rec.revision}),
|
||||
);
|
||||
Ok(rec)
|
||||
}
|
||||
|
||||
pub fn cancel_token_path(&self, task_id: &str) -> PathBuf {
|
||||
self.cancel_dir.join(format!("{task_id}.tok"))
|
||||
}
|
||||
|
||||
pub fn mark_cancelled(&self, task_id: &str, expected_revision: u64) -> Result<TaskRecord, ControlError> {
|
||||
Ok(self.store.cas_state(
|
||||
task_id,
|
||||
expected_revision,
|
||||
TaskState::Cancelled,
|
||||
|_| {},
|
||||
)?)
|
||||
}
|
||||
|
||||
pub fn complete(
|
||||
&self,
|
||||
task_id: &str,
|
||||
expected_revision: u64,
|
||||
result: Value,
|
||||
) -> Result<TaskRecord, ControlError> {
|
||||
match self.store.cas_state(task_id, expected_revision, TaskState::Succeeded, |r| {
|
||||
r.result = Some(result.clone());
|
||||
}) {
|
||||
Ok(rec) => Ok(rec),
|
||||
Err(StoreError::AlreadyTerminal(state)) => {
|
||||
// Late result after cancel: append evidence only.
|
||||
let rec = self.store.get(task_id)?;
|
||||
let _ = self.store.append_event(
|
||||
task_id,
|
||||
&rec.run_id,
|
||||
&rec.attempt_id,
|
||||
"run.late_result",
|
||||
serde_json::json!({"ignored_terminal": format!("{state:?}"), "result": result}),
|
||||
);
|
||||
Err(ControlError::Store(StoreError::AlreadyTerminal(state)))
|
||||
}
|
||||
Err(e) => Err(ControlError::Store(e)),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn events_since(
|
||||
&self,
|
||||
task_id: &str,
|
||||
after_seq: u64,
|
||||
) -> Result<Vec<RuntimeEventV2>, ControlError> {
|
||||
Ok(self.store.events_after(task_id, after_seq)?)
|
||||
}
|
||||
|
||||
/// Collaboration grace + force-kill budget from the design (§6).
|
||||
pub fn cancel_deadlines() -> (Duration, Duration) {
|
||||
(Duration::from_secs(2), Duration::from_secs(3))
|
||||
}
|
||||
}
|
||||
|
||||
/// Convenience wrapper used by HTTP/control adapters.
|
||||
pub fn cancel_task(svc: &ControlService, task_id: &str) -> Result<TaskRecord, ControlError> {
|
||||
svc.cancel(task_id)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::runtime::scheduler::SchedulerConfig;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn cancel_writes_token_not_interpreter() {
|
||||
let dir = std::env::temp_dir().join(format!("mq-ctrl-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
let store = Arc::new(TaskStore::memory());
|
||||
let sched = Arc::new(Scheduler::new(SchedulerConfig::default()));
|
||||
let ctl = ControlService::new(store, sched, &dir);
|
||||
let (t, _) = ctl
|
||||
.admit("t1", "r1", "ws", None, "k", &json!({}), AdmissionClass::Execute)
|
||||
.unwrap();
|
||||
let c = ctl.cancel(&t.task_id).unwrap();
|
||||
assert_eq!(c.state, TaskState::CancelRequested);
|
||||
let body = std::fs::read_to_string(ctl.cancel_token_path(&t.task_id)).unwrap();
|
||||
assert_eq!(body, "cancelled");
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn late_complete_after_cancel_is_evidence_only() {
|
||||
let dir = std::env::temp_dir().join(format!("mq-ctrl2-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
let store = Arc::new(TaskStore::memory());
|
||||
let sched = Arc::new(Scheduler::new(SchedulerConfig::default()));
|
||||
let ctl = ControlService::new(Arc::clone(&store), sched, &dir);
|
||||
let (t, _) = ctl
|
||||
.admit("t1", "r1", "ws", None, "k", &json!({}), AdmissionClass::Execute)
|
||||
.unwrap();
|
||||
let c = ctl.cancel(&t.task_id).unwrap();
|
||||
ctl.mark_cancelled(&t.task_id, c.revision).unwrap();
|
||||
let err = ctl.complete(&t.task_id, c.revision + 1, json!("late")).unwrap_err();
|
||||
assert!(matches!(err, ControlError::Store(StoreError::AlreadyTerminal(_))));
|
||||
let evs = store.events_after(&t.task_id, 0).unwrap();
|
||||
assert!(evs.iter().any(|e| e.event_type == "run.late_result"));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,457 @@
|
||||
//! Process-global control plane via `host_query("runtime", …)` — no INVOKE_LOCK.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
use std::thread;
|
||||
use std::time::Duration;
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use crate::runtime::control::ControlService;
|
||||
use crate::runtime::protocol::{
|
||||
EntryRef, InvokeContext, InvokeRequestV2, LeaseRef, PROTOCOL_VERSION,
|
||||
};
|
||||
use crate::runtime::scheduler::{AdmissionClass, Scheduler, SchedulerConfig};
|
||||
use crate::runtime::supervisor::{Supervisor, SupervisorConfig};
|
||||
use crate::runtime::task_store::{TaskState, TaskStore};
|
||||
|
||||
struct RuntimeHost {
|
||||
control: ControlService,
|
||||
supervisor: Arc<Supervisor>,
|
||||
/// task_id → lease_id
|
||||
leases: Mutex<std::collections::HashMap<String, String>>,
|
||||
base: PathBuf,
|
||||
}
|
||||
|
||||
fn runtime_dir() -> PathBuf {
|
||||
std::env::var_os("MARQDO_RUNTIME_DIR")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| std::env::temp_dir().join("marqdo-runtime"))
|
||||
}
|
||||
|
||||
fn marqdo_bin() -> PathBuf {
|
||||
std::env::var_os("MARQDO_BIN")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| std::env::current_exe().unwrap_or_else(|_| PathBuf::from("marqdo")))
|
||||
}
|
||||
|
||||
fn ensure_host() -> &'static RuntimeHost {
|
||||
static H: OnceLock<RuntimeHost> = OnceLock::new();
|
||||
static JANITOR: OnceLock<()> = OnceLock::new();
|
||||
let host = H.get_or_init(|| {
|
||||
let base = runtime_dir();
|
||||
let _ = std::fs::create_dir_all(&base);
|
||||
let store = match TaskStore::with_persist(base.join("store")) {
|
||||
Ok(s) => Arc::new(s),
|
||||
Err(_) => Arc::new(TaskStore::memory()),
|
||||
};
|
||||
let scheduler = Arc::new(Scheduler::new(SchedulerConfig::default()));
|
||||
let control = ControlService::new(
|
||||
Arc::clone(&store),
|
||||
Arc::clone(&scheduler),
|
||||
base.join("cancel"),
|
||||
);
|
||||
let supervisor = Arc::new(Supervisor::new(
|
||||
SupervisorConfig::default(),
|
||||
base.join("stage"),
|
||||
marqdo_bin(),
|
||||
base.join("cancel"),
|
||||
base.join("events"),
|
||||
));
|
||||
let _ = supervisor.warm();
|
||||
RuntimeHost {
|
||||
control,
|
||||
supervisor,
|
||||
leases: Mutex::new(std::collections::HashMap::new()),
|
||||
base,
|
||||
}
|
||||
});
|
||||
JANITOR.get_or_init(|| {
|
||||
let sup = Arc::clone(&host.supervisor);
|
||||
let _ = thread::Builder::new()
|
||||
.name("mq-runtime-janitor".into())
|
||||
.spawn(move || loop {
|
||||
thread::sleep(Duration::from_millis(500));
|
||||
let _ = sup.janitor_tick();
|
||||
});
|
||||
});
|
||||
host
|
||||
}
|
||||
|
||||
/// Dispatch `host_query("runtime", {op, ...})`. Never blocks on INVOKE_LOCK.
|
||||
pub fn handle(args: &Value) -> Result<Value, String> {
|
||||
let op = args
|
||||
.get("op")
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or_else(|| "runtime requires op".to_string())?;
|
||||
let host = ensure_host();
|
||||
let ctl = &host.control;
|
||||
match op {
|
||||
"ping" | "capabilities" => Ok(json!({
|
||||
"ok": true,
|
||||
"protocol_version": PROTOCOL_VERSION,
|
||||
"features": {
|
||||
"runtime.task_control_v2": 1,
|
||||
"runtime.event_replay": 1,
|
||||
"runtime.isolated_execution": 1,
|
||||
},
|
||||
"runtime_dir": host.base.to_string_lossy(),
|
||||
})),
|
||||
"admit" => {
|
||||
let task_id = req_str(args, "task_id")?;
|
||||
let run_id = args
|
||||
.get("run_id")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(task_id);
|
||||
let workspace_id = args
|
||||
.get("workspace_id")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("default");
|
||||
let session_id = args
|
||||
.get("session_id")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|s| s.to_string());
|
||||
let idem = args
|
||||
.get("idempotency_key")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(task_id);
|
||||
let class = AdmissionClass::parse(
|
||||
args.get("class").and_then(|v| v.as_str()).unwrap_or("execute"),
|
||||
);
|
||||
let payload = args.get("payload").cloned().unwrap_or(json!({}));
|
||||
let (rec, depth) = ctl
|
||||
.admit(
|
||||
task_id,
|
||||
run_id,
|
||||
workspace_id,
|
||||
session_id,
|
||||
idem,
|
||||
&payload,
|
||||
class,
|
||||
)
|
||||
.map_err(|e| e.to_string())?;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"task": task_json(&rec),
|
||||
"queue_depth": depth,
|
||||
"cancel_token": ctl.cancel_token_path(&rec.task_id).to_string_lossy(),
|
||||
"event_path": host.supervisor.event_path(&rec.task_id).to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
"dispatch" => dispatch_op(host, args),
|
||||
"cancel" => {
|
||||
let task_id = req_str(args, "task_id")?;
|
||||
match ctl.cancel(task_id) {
|
||||
Ok(rec) => {
|
||||
if let Some(lease) = host.leases.lock().unwrap().get(task_id) {
|
||||
let _ = host.supervisor.signal_cancel(lease, task_id);
|
||||
} else {
|
||||
let _ = std::fs::write(ctl.cancel_token_path(task_id), b"cancelled");
|
||||
}
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"task": task_json(&rec),
|
||||
"cancel_token": ctl.cancel_token_path(task_id).to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
Err(crate::runtime::control::ControlError::Store(
|
||||
crate::runtime::task_store::StoreError::NotFound,
|
||||
)) => Ok(json!({"ok": false, "error": "not_found", "status": 404})),
|
||||
Err(crate::runtime::control::ControlError::Store(
|
||||
crate::runtime::task_store::StoreError::AlreadyTerminal(s),
|
||||
)) => Ok(json!({
|
||||
"ok": true,
|
||||
"already_terminal": true,
|
||||
"state": format!("{s:?}").to_ascii_lowercase(),
|
||||
})),
|
||||
Err(e) => Err(e.to_string()),
|
||||
}
|
||||
}
|
||||
"get" => {
|
||||
let task_id = req_str(args, "task_id")?;
|
||||
match ctl.store.get(task_id) {
|
||||
Ok(rec) => {
|
||||
let mut out = json!({"ok": true, "task": task_json(&rec)});
|
||||
if let Some(lease) = host.leases.lock().unwrap().get(task_id).cloned() {
|
||||
out["lease_id"] = json!(lease);
|
||||
if let Some(p) = host.supervisor.emit_result_path(&lease) {
|
||||
out["emit_result"] = json!(p.to_string_lossy());
|
||||
}
|
||||
if let Ok(Some(code)) = host.supervisor.try_reap(&lease) {
|
||||
out["exit_code"] = json!(code);
|
||||
host.control
|
||||
.scheduler
|
||||
.release_slot(AdmissionClass::Execute);
|
||||
let _ = ctl.store.cas_state(
|
||||
task_id,
|
||||
rec.revision,
|
||||
if code == 0 {
|
||||
TaskState::Succeeded
|
||||
} else {
|
||||
TaskState::Failed
|
||||
},
|
||||
|_| {},
|
||||
);
|
||||
}
|
||||
}
|
||||
out["event_path"] =
|
||||
json!(host.supervisor.event_path(task_id).to_string_lossy());
|
||||
Ok(out)
|
||||
}
|
||||
Err(_) => Ok(json!({"ok": false, "error": "not_found", "status": 404})),
|
||||
}
|
||||
}
|
||||
"events" => {
|
||||
let task_id = req_str(args, "task_id")?;
|
||||
let after = args.get("after_seq").and_then(|v| v.as_u64()).unwrap_or(0);
|
||||
let path = host.supervisor.event_path(task_id);
|
||||
if path.is_file() {
|
||||
let lines = read_ndjson_lines(&path, after)?;
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"events": lines,
|
||||
"after_seq": after,
|
||||
"source": "worker_ndjson",
|
||||
"event_path": path.to_string_lossy(),
|
||||
}));
|
||||
}
|
||||
match ctl.events_since(task_id, after) {
|
||||
Ok(evs) => Ok(json!({
|
||||
"ok": true,
|
||||
"events": evs,
|
||||
"after_seq": after,
|
||||
"source": "task_store",
|
||||
})),
|
||||
Err(_) => Ok(json!({"ok": false, "error": "not_found", "status": 404})),
|
||||
}
|
||||
}
|
||||
"append_event" => {
|
||||
let task_id = req_str(args, "task_id")?;
|
||||
let rec = ctl.store.get(task_id).map_err(|_| "not_found".to_string())?;
|
||||
let et = args
|
||||
.get("type")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("message.delta");
|
||||
let payload = args.get("payload").cloned().unwrap_or(json!({}));
|
||||
let ev = ctl
|
||||
.store
|
||||
.append_event(task_id, &rec.run_id, &rec.attempt_id, et, payload)
|
||||
.map_err(|e| e.to_string())?;
|
||||
Ok(json!({"ok": true, "event": ev}))
|
||||
}
|
||||
other => Err(format!("runtime.unknown_op:{other}")),
|
||||
}
|
||||
}
|
||||
|
||||
fn dispatch_op(host: &RuntimeHost, args: &Value) -> Result<Value, String> {
|
||||
let task_id = req_str(args, "task_id")?;
|
||||
let rec = host
|
||||
.control
|
||||
.store
|
||||
.get(task_id)
|
||||
.map_err(|_| "not_found".to_string())?;
|
||||
let entry = args.get("entry").ok_or("dispatch.entry_required")?;
|
||||
let module = entry
|
||||
.get("module")
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or("dispatch.module_required")?;
|
||||
let function = entry
|
||||
.get("function")
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or("dispatch.function_required")?;
|
||||
let fs_root = args
|
||||
.get("fs_root")
|
||||
.and_then(|v| v.as_str())
|
||||
.or_else(|| {
|
||||
args.get("context")
|
||||
.and_then(|c| c.get("fs_root"))
|
||||
.and_then(|v| v.as_str())
|
||||
})
|
||||
.ok_or("dispatch.fs_root_required")?;
|
||||
let args_ref = args
|
||||
.get("args_ref")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|s| s.to_string())
|
||||
.or_else(|| {
|
||||
args.get("args_json")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|p| format!("file:{p}"))
|
||||
});
|
||||
|
||||
let tok = host.control.cancel_token_path(task_id);
|
||||
if !tok.exists() {
|
||||
std::fs::write(&tok, b"").map_err(|e| e.to_string())?;
|
||||
}
|
||||
|
||||
let epoch = host.control.store.next_lease_epoch();
|
||||
let req = InvokeRequestV2 {
|
||||
schema: "InvokeRequestV2".into(),
|
||||
schema_version: PROTOCOL_VERSION,
|
||||
task_id: task_id.into(),
|
||||
run_id: rec.run_id.clone(),
|
||||
attempt_id: rec.attempt_id.clone(),
|
||||
lease: LeaseRef {
|
||||
id: format!("lease_{task_id}"),
|
||||
epoch,
|
||||
},
|
||||
entry: EntryRef {
|
||||
module: module.into(),
|
||||
function: function.into(),
|
||||
},
|
||||
args_ref,
|
||||
context: InvokeContext {
|
||||
workspace_id: rec.workspace_id.clone(),
|
||||
snapshot_id: None,
|
||||
selection_ref: None,
|
||||
capabilities_ref: None,
|
||||
deadline_at: "2099-01-01T00:00:00Z".into(),
|
||||
fs_root: Some(fs_root.into()),
|
||||
},
|
||||
};
|
||||
|
||||
if host
|
||||
.control
|
||||
.scheduler
|
||||
.try_acquire_slot(AdmissionClass::Execute)
|
||||
.is_err()
|
||||
{
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"error": "no_slot",
|
||||
"status": 429,
|
||||
"Retry-After": 2,
|
||||
}));
|
||||
}
|
||||
|
||||
match host.supervisor.dispatch(req) {
|
||||
Ok(lease_id) => {
|
||||
host.leases
|
||||
.lock()
|
||||
.unwrap()
|
||||
.insert(task_id.to_string(), lease_id.clone());
|
||||
let _ = host.control.store.cas_state(
|
||||
task_id,
|
||||
rec.revision,
|
||||
TaskState::Running,
|
||||
|r| {
|
||||
r.lease_id = Some(lease_id.clone());
|
||||
r.lease_epoch = Some(epoch);
|
||||
},
|
||||
);
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"lease_id": lease_id,
|
||||
"task_id": task_id,
|
||||
"cancel_token": tok.to_string_lossy(),
|
||||
"event_path": host.supervisor.event_path(task_id).to_string_lossy(),
|
||||
"emit_result": host.supervisor.emit_result_path(&lease_id)
|
||||
.map(|p| p.to_string_lossy().to_string()),
|
||||
}))
|
||||
}
|
||||
Err(e) => {
|
||||
host.control
|
||||
.scheduler
|
||||
.release_slot(AdmissionClass::Execute);
|
||||
if e.contains("no_slot") {
|
||||
Ok(json!({
|
||||
"ok": false,
|
||||
"error": "no_slot",
|
||||
"status": 429,
|
||||
"Retry-After": 2,
|
||||
}))
|
||||
} else {
|
||||
Err(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn read_ndjson_lines(path: &Path, after_seq: u64) -> Result<Vec<Value>, String> {
|
||||
let text = std::fs::read_to_string(path).map_err(|e| e.to_string())?;
|
||||
let mut out = Vec::new();
|
||||
let mut seq = 0u64;
|
||||
for line in text.lines() {
|
||||
let line = line.trim();
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
seq += 1;
|
||||
if seq <= after_seq {
|
||||
continue;
|
||||
}
|
||||
match serde_json::from_str::<Value>(line) {
|
||||
Ok(mut v) => {
|
||||
if v.get("seq").is_none() {
|
||||
if let Some(obj) = v.as_object_mut() {
|
||||
obj.insert("seq".into(), json!(seq));
|
||||
}
|
||||
}
|
||||
out.push(v);
|
||||
}
|
||||
Err(_) => out.push(json!({"seq": seq, "raw": line})),
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn req_str<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.ok_or_else(|| format!("runtime missing `{key}`"))
|
||||
}
|
||||
|
||||
fn task_json(rec: &crate::runtime::task_store::TaskRecord) -> Value {
|
||||
json!({
|
||||
"task_id": rec.task_id,
|
||||
"run_id": rec.run_id,
|
||||
"attempt_id": rec.attempt_id,
|
||||
"workspace_id": rec.workspace_id,
|
||||
"session_id": rec.session_id,
|
||||
"state": state_str(rec.state),
|
||||
"revision": rec.revision,
|
||||
"idempotency_key": rec.idempotency_key,
|
||||
"created_at": rec.created_at,
|
||||
"updated_at": rec.updated_at,
|
||||
"cancel_requested_at": rec.cancel_requested_at,
|
||||
"result": rec.result,
|
||||
"error": rec.error,
|
||||
"lease_id": rec.lease_id,
|
||||
})
|
||||
}
|
||||
|
||||
fn state_str(s: TaskState) -> &'static str {
|
||||
match s {
|
||||
TaskState::Accepted => "accepted",
|
||||
TaskState::Queued => "queued",
|
||||
TaskState::Running => "running",
|
||||
TaskState::Finalizing => "finalizing",
|
||||
TaskState::Succeeded => "succeeded",
|
||||
TaskState::Failed => "failed",
|
||||
TaskState::CancelRequested => "cancel_requested",
|
||||
TaskState::Cancelled => "cancelled",
|
||||
TaskState::Interrupted => "interrupted",
|
||||
TaskState::NeedsReconciliation => "needs_reconciliation",
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn admit_cancel_without_host_context() {
|
||||
let id = format!("rt-{}", std::process::id());
|
||||
let adm = handle(&json!({
|
||||
"op": "admit",
|
||||
"task_id": id,
|
||||
"idempotency_key": id,
|
||||
"payload": {"n": 1}
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(adm["ok"], true);
|
||||
let can = handle(&json!({"op": "cancel", "task_id": id})).unwrap();
|
||||
assert_eq!(can["ok"], true);
|
||||
assert_eq!(can["task"]["state"], "cancel_requested");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
//! Worker lease identity and process lifecycle states (M-R2).
|
||||
|
||||
use std::time::{Duration, SystemTime};
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
||||
pub struct LeaseId(pub String);
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum WorkerState {
|
||||
Starting,
|
||||
Ready,
|
||||
Leased,
|
||||
Running,
|
||||
Draining,
|
||||
Unhealthy,
|
||||
Terminating,
|
||||
Exited,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Lease {
|
||||
pub id: LeaseId,
|
||||
pub epoch: u64,
|
||||
pub task_id: String,
|
||||
pub run_id: String,
|
||||
pub attempt_id: String,
|
||||
pub deadline: SystemTime,
|
||||
pub state: WorkerState,
|
||||
}
|
||||
|
||||
impl Lease {
|
||||
pub fn new(
|
||||
id: impl Into<String>,
|
||||
epoch: u64,
|
||||
task_id: impl Into<String>,
|
||||
run_id: impl Into<String>,
|
||||
attempt_id: impl Into<String>,
|
||||
ttl: Duration,
|
||||
) -> Self {
|
||||
Self {
|
||||
id: LeaseId(id.into()),
|
||||
epoch,
|
||||
task_id: task_id.into(),
|
||||
run_id: run_id.into(),
|
||||
attempt_id: attempt_id.into(),
|
||||
deadline: SystemTime::now() + ttl,
|
||||
state: WorkerState::Leased,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn expired(&self) -> bool {
|
||||
SystemTime::now() > self.deadline
|
||||
}
|
||||
|
||||
pub fn matches(&self, lease_id: &str, epoch: u64) -> bool {
|
||||
self.id.0 == lease_id && self.epoch == epoch
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
//! Isolated web runtime (M-R2–M-R4): supervisor, worker IPC, leases, task
|
||||
//! admission/control, and persistent event log.
|
||||
//!
|
||||
//! Design: `doc/design/isolated-web-runtime-development-2026-10-26.md`.
|
||||
//! Until capability probes report version ≥1, callers must treat this as opt-in.
|
||||
|
||||
pub mod control;
|
||||
pub mod host_api;
|
||||
pub mod lease;
|
||||
pub mod protocol;
|
||||
pub mod publish;
|
||||
pub mod scheduler;
|
||||
pub mod supervisor;
|
||||
pub mod task_store;
|
||||
pub mod worker;
|
||||
|
||||
pub use control::{cancel_task, ControlError, ControlService};
|
||||
pub use lease::{Lease, LeaseId, WorkerState};
|
||||
pub use protocol::{
|
||||
InvokeRequestV2, RuntimeEventV2, CONTROL_FRAME_MAX_BYTES, PROTOCOL_VERSION,
|
||||
};
|
||||
pub use publish::{PublishJournal, PublishOutcome, PublishPhase};
|
||||
pub use scheduler::{AdmissionClass, Scheduler, SchedulerConfig};
|
||||
pub use supervisor::{Supervisor, SupervisorConfig};
|
||||
pub use task_store::{TaskRecord, TaskState, TaskStore};
|
||||
pub use worker::{WorkerHandle, WorkerPool};
|
||||
@@ -0,0 +1,158 @@
|
||||
//! Length-prefixed JSON control frames + event channel types (M-R2 / M-R4).
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
pub const PROTOCOL_VERSION: u32 = 2;
|
||||
pub const CONTROL_FRAME_MAX_BYTES: usize = 1024 * 1024;
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct InvokeRequestV2 {
|
||||
pub schema: String,
|
||||
pub schema_version: u32,
|
||||
pub task_id: String,
|
||||
pub run_id: String,
|
||||
pub attempt_id: String,
|
||||
pub lease: LeaseRef,
|
||||
pub entry: EntryRef,
|
||||
#[serde(default)]
|
||||
pub args_ref: Option<String>,
|
||||
pub context: InvokeContext,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct LeaseRef {
|
||||
pub id: String,
|
||||
pub epoch: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct EntryRef {
|
||||
pub module: String,
|
||||
pub function: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct InvokeContext {
|
||||
pub workspace_id: String,
|
||||
#[serde(default)]
|
||||
pub snapshot_id: Option<String>,
|
||||
#[serde(default)]
|
||||
pub selection_ref: Option<String>,
|
||||
#[serde(default)]
|
||||
pub capabilities_ref: Option<String>,
|
||||
pub deadline_at: String,
|
||||
/// Absolute site filesystem root (`--fs-root` for the worker).
|
||||
#[serde(default)]
|
||||
pub fs_root: Option<String>,
|
||||
}
|
||||
|
||||
impl InvokeRequestV2 {
|
||||
pub fn validate(&self) -> Result<(), String> {
|
||||
if self.schema != "InvokeRequestV2" {
|
||||
return Err(format!("protocol.schema: {}", self.schema));
|
||||
}
|
||||
if self.schema_version != PROTOCOL_VERSION {
|
||||
return Err(format!(
|
||||
"protocol.version: got {} want {PROTOCOL_VERSION}",
|
||||
self.schema_version
|
||||
));
|
||||
}
|
||||
if self.task_id.is_empty() || self.run_id.is_empty() || self.attempt_id.is_empty() {
|
||||
return Err("protocol.ids_required".into());
|
||||
}
|
||||
if self.entry.module.is_empty() || self.entry.function.is_empty() {
|
||||
return Err("protocol.entry_required".into());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct RuntimeEventV2 {
|
||||
pub schema: String,
|
||||
pub schema_version: u32,
|
||||
pub event_id: String,
|
||||
pub seq: u64,
|
||||
pub task_id: String,
|
||||
pub run_id: String,
|
||||
pub attempt_id: String,
|
||||
#[serde(rename = "type")]
|
||||
pub event_type: String,
|
||||
pub payload: Value,
|
||||
}
|
||||
|
||||
impl RuntimeEventV2 {
|
||||
pub fn new(
|
||||
task_id: &str,
|
||||
run_id: &str,
|
||||
attempt_id: &str,
|
||||
seq: u64,
|
||||
event_type: &str,
|
||||
payload: Value,
|
||||
) -> Self {
|
||||
Self {
|
||||
schema: "RuntimeEventV2".into(),
|
||||
schema_version: PROTOCOL_VERSION,
|
||||
event_id: format!("{task_id}:{seq}"),
|
||||
seq,
|
||||
task_id: task_id.into(),
|
||||
run_id: run_id.into(),
|
||||
attempt_id: attempt_id.into(),
|
||||
event_type: event_type.into(),
|
||||
payload,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Encode a control frame: u32 LE length + UTF-8 JSON (≤ CONTROL_FRAME_MAX_BYTES).
|
||||
pub fn encode_frame(value: &Value) -> Result<Vec<u8>, String> {
|
||||
let body = serde_json::to_vec(value).map_err(|e| format!("protocol.encode: {e}"))?;
|
||||
if body.len() > CONTROL_FRAME_MAX_BYTES {
|
||||
return Err("protocol.frame_too_large".into());
|
||||
}
|
||||
let mut out = Vec::with_capacity(4 + body.len());
|
||||
out.extend_from_slice(&(body.len() as u32).to_le_bytes());
|
||||
out.extend_from_slice(&body);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Decode one frame from the front of `buf`; returns (value, bytes_consumed).
|
||||
pub fn decode_frame(buf: &[u8]) -> Result<Option<(Value, usize)>, String> {
|
||||
if buf.len() < 4 {
|
||||
return Ok(None);
|
||||
}
|
||||
let len = u32::from_le_bytes([buf[0], buf[1], buf[2], buf[3]]) as usize;
|
||||
if len > CONTROL_FRAME_MAX_BYTES {
|
||||
return Err("protocol.frame_too_large".into());
|
||||
}
|
||||
if buf.len() < 4 + len {
|
||||
return Ok(None);
|
||||
}
|
||||
let value: Value = serde_json::from_slice(&buf[4..4 + len])
|
||||
.map_err(|e| format!("protocol.decode: {e}"))?;
|
||||
Ok(Some((value, 4 + len)))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn frame_round_trip() {
|
||||
let v = json!({"hello": "世界"});
|
||||
let enc = encode_frame(&v).unwrap();
|
||||
let (got, n) = decode_frame(&enc).unwrap().unwrap();
|
||||
assert_eq!(n, enc.len());
|
||||
assert_eq!(got, v);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn incomplete_frame_waits() {
|
||||
let v = json!({"a": 1});
|
||||
let enc = encode_frame(&v).unwrap();
|
||||
assert!(decode_frame(&enc[..3]).unwrap().is_none());
|
||||
assert!(decode_frame(&enc[..enc.len() - 1]).unwrap().is_none());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,274 @@
|
||||
//! Publish journal + expected_generation CAS (M-R6).
|
||||
|
||||
use std::fs;
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum PublishPhase {
|
||||
Staging,
|
||||
Validated,
|
||||
GenerationRenamed,
|
||||
CurrentSwitched,
|
||||
DbProjected,
|
||||
Aborted,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct PublishJournal {
|
||||
pub kb_id: String,
|
||||
pub expected_generation: Option<String>,
|
||||
pub new_generation: String,
|
||||
pub phase: PublishPhase,
|
||||
pub staging_dir: PathBuf,
|
||||
pub generation_dir: PathBuf,
|
||||
pub current_path: PathBuf,
|
||||
pub source_digest: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum PublishOutcome {
|
||||
Published { generation: String },
|
||||
Conflict { current: String, expected: String },
|
||||
Recovered { generation: String },
|
||||
Aborted { reason: String },
|
||||
}
|
||||
|
||||
impl PublishJournal {
|
||||
pub fn begin(
|
||||
out_dir: impl AsRef<Path>,
|
||||
kb_id: impl Into<String>,
|
||||
expected_generation: Option<String>,
|
||||
new_generation: impl Into<String>,
|
||||
source_digest: impl Into<String>,
|
||||
) -> Result<Self, String> {
|
||||
let out_dir = out_dir.as_ref();
|
||||
let new_generation = new_generation.into();
|
||||
let staging_dir = out_dir.join(".publish-staging").join(&new_generation);
|
||||
let generation_dir = out_dir.join("generations").join(&new_generation);
|
||||
let current_path = out_dir.join("CURRENT");
|
||||
fs::create_dir_all(&staging_dir).map_err(|e| e.to_string())?;
|
||||
let j = Self {
|
||||
kb_id: kb_id.into(),
|
||||
expected_generation,
|
||||
new_generation,
|
||||
phase: PublishPhase::Staging,
|
||||
staging_dir,
|
||||
generation_dir,
|
||||
current_path,
|
||||
source_digest: source_digest.into(),
|
||||
};
|
||||
j.write_journal(out_dir)?;
|
||||
Ok(j)
|
||||
}
|
||||
|
||||
fn journal_path(out_dir: &Path) -> PathBuf {
|
||||
out_dir.join("publish_journal.json")
|
||||
}
|
||||
|
||||
fn write_journal(&self, out_dir: &Path) -> Result<(), String> {
|
||||
let path = Self::journal_path(out_dir);
|
||||
let tmp = path.with_extension("json.tmp");
|
||||
let body = serde_json::to_vec_pretty(self).map_err(|e| e.to_string())?;
|
||||
fs::write(&tmp, body).map_err(|e| e.to_string())?;
|
||||
// Best-effort fsync via File.
|
||||
let f = fs::OpenOptions::new()
|
||||
.write(true)
|
||||
.open(&tmp)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let _ = f.sync_all();
|
||||
drop(f);
|
||||
fs::rename(&tmp, &path).map_err(|e| e.to_string())?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn load(out_dir: impl AsRef<Path>) -> Result<Option<Self>, String> {
|
||||
let path = Self::journal_path(out_dir.as_ref());
|
||||
if !path.is_file() {
|
||||
return Ok(None);
|
||||
}
|
||||
let body = fs::read_to_string(&path).map_err(|e| e.to_string())?;
|
||||
let j: Self = serde_json::from_str(&body).map_err(|e| e.to_string())?;
|
||||
Ok(Some(j))
|
||||
}
|
||||
|
||||
/// CAS: refuse if CURRENT does not match expected_generation.
|
||||
pub fn assert_expected_current(&self) -> Result<String, PublishOutcome> {
|
||||
let current = fs::read_to_string(&self.current_path)
|
||||
.unwrap_or_default()
|
||||
.trim()
|
||||
.to_string();
|
||||
if let Some(exp) = &self.expected_generation {
|
||||
if ¤t != exp {
|
||||
return Err(PublishOutcome::Conflict {
|
||||
current,
|
||||
expected: exp.clone(),
|
||||
});
|
||||
}
|
||||
}
|
||||
Ok(current)
|
||||
}
|
||||
|
||||
pub fn mark(&mut self, out_dir: &Path, phase: PublishPhase) -> Result<(), String> {
|
||||
self.phase = phase;
|
||||
self.write_journal(out_dir)
|
||||
}
|
||||
|
||||
/// Atomic publish: staging tree already complete → rename into generations → CURRENT.
|
||||
pub fn commit_filesystem(&mut self, out_dir: &Path) -> Result<PublishOutcome, String> {
|
||||
match self.assert_expected_current() {
|
||||
Err(o) => return Ok(o),
|
||||
Ok(_) => {}
|
||||
}
|
||||
self.mark(out_dir, PublishPhase::Validated)?;
|
||||
|
||||
if self.generation_dir.exists() {
|
||||
fs::remove_dir_all(&self.generation_dir).map_err(|e| e.to_string())?;
|
||||
}
|
||||
fs::create_dir_all(self.generation_dir.parent().unwrap()).map_err(|e| e.to_string())?;
|
||||
fs::rename(&self.staging_dir, &self.generation_dir).map_err(|e| e.to_string())?;
|
||||
self.mark(out_dir, PublishPhase::GenerationRenamed)?;
|
||||
|
||||
let tmp = self.current_path.with_extension("tmp");
|
||||
{
|
||||
let mut f = fs::File::create(&tmp).map_err(|e| e.to_string())?;
|
||||
writeln!(f, "{}", self.new_generation).map_err(|e| e.to_string())?;
|
||||
let _ = f.sync_all();
|
||||
}
|
||||
fs::rename(&tmp, &self.current_path).map_err(|e| e.to_string())?;
|
||||
self.mark(out_dir, PublishPhase::CurrentSwitched)?;
|
||||
|
||||
// Clear journal after successful CURRENT switch; DB projection is product-side.
|
||||
let _ = fs::remove_file(Self::journal_path(out_dir));
|
||||
Ok(PublishOutcome::Published {
|
||||
generation: self.new_generation.clone(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Crash recovery: if CURRENT already points at new_generation, treat as done;
|
||||
/// if generation exists but CURRENT old, finish CURRENT switch; else abort staging.
|
||||
pub fn recover(out_dir: impl AsRef<Path>) -> Result<PublishOutcome, String> {
|
||||
let out_dir = out_dir.as_ref();
|
||||
let Some(mut j) = Self::load(out_dir)? else {
|
||||
return Ok(PublishOutcome::Aborted {
|
||||
reason: "no_journal".into(),
|
||||
});
|
||||
};
|
||||
let current = fs::read_to_string(&j.current_path)
|
||||
.unwrap_or_default()
|
||||
.trim()
|
||||
.to_string();
|
||||
if current == j.new_generation {
|
||||
let _ = fs::remove_file(Self::journal_path(out_dir));
|
||||
return Ok(PublishOutcome::Recovered {
|
||||
generation: j.new_generation,
|
||||
});
|
||||
}
|
||||
match j.phase {
|
||||
PublishPhase::GenerationRenamed | PublishPhase::Validated
|
||||
if j.generation_dir.is_dir() =>
|
||||
{
|
||||
let tmp = j.current_path.with_extension("tmp");
|
||||
fs::write(&tmp, format!("{}\n", j.new_generation)).map_err(|e| e.to_string())?;
|
||||
fs::rename(&tmp, &j.current_path).map_err(|e| e.to_string())?;
|
||||
let _ = fs::remove_file(Self::journal_path(out_dir));
|
||||
Ok(PublishOutcome::Recovered {
|
||||
generation: j.new_generation,
|
||||
})
|
||||
}
|
||||
PublishPhase::Staging | PublishPhase::Aborted => {
|
||||
let _ = fs::remove_dir_all(&j.staging_dir);
|
||||
j.mark(out_dir, PublishPhase::Aborted)?;
|
||||
let _ = fs::remove_file(Self::journal_path(out_dir));
|
||||
Ok(PublishOutcome::Aborted {
|
||||
reason: "incomplete_staging".into(),
|
||||
})
|
||||
}
|
||||
other => Ok(PublishOutcome::Aborted {
|
||||
reason: format!("unhandled_phase:{other:?}"),
|
||||
}),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Stable logical document identity independent of generation directory name.
|
||||
pub fn logical_document_id(project_id: &str, relative_path: &str) -> String {
|
||||
let mut h = Sha256::new();
|
||||
h.update(project_id.as_bytes());
|
||||
h.update(b"\0");
|
||||
h.update(relative_path.replace('\\', "/").as_bytes());
|
||||
format!("doc:{:x}", h.finalize())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn cas_conflict_and_commit() {
|
||||
let root = std::env::temp_dir().join(format!("mq-pub-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
fs::create_dir_all(root.join("generations")).unwrap();
|
||||
fs::write(root.join("CURRENT"), "gen_old\n").unwrap();
|
||||
|
||||
let mut j = PublishJournal::begin(&root, "kb1", Some("gen_old".into()), "gen_new", "digest")
|
||||
.unwrap();
|
||||
fs::write(j.staging_dir.join("manifest.json"), "{}").unwrap();
|
||||
let out = j.commit_filesystem(&root).unwrap();
|
||||
assert_eq!(
|
||||
out,
|
||||
PublishOutcome::Published {
|
||||
generation: "gen_new".into()
|
||||
}
|
||||
);
|
||||
assert_eq!(
|
||||
fs::read_to_string(root.join("CURRENT")).unwrap().trim(),
|
||||
"gen_new"
|
||||
);
|
||||
|
||||
let mut j2 =
|
||||
PublishJournal::begin(&root, "kb1", Some("gen_old".into()), "gen_x", "d").unwrap();
|
||||
fs::write(j2.staging_dir.join("x"), "1").unwrap();
|
||||
let conflict = j2.commit_filesystem(&root).unwrap();
|
||||
assert!(matches!(conflict, PublishOutcome::Conflict { .. }));
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recover_after_generation_rename() {
|
||||
let root = std::env::temp_dir().join(format!("mq-pub2-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
fs::create_dir_all(root.join("generations/gen_r")).unwrap();
|
||||
fs::write(root.join("CURRENT"), "old\n").unwrap();
|
||||
let mut j =
|
||||
PublishJournal::begin(&root, "kb", Some("old".into()), "gen_r", "d").unwrap();
|
||||
// Simulate crash after generation rename: staging already moved.
|
||||
let _ = fs::remove_dir_all(&j.staging_dir);
|
||||
j.phase = PublishPhase::GenerationRenamed;
|
||||
j.write_journal(&root).unwrap();
|
||||
let out = PublishJournal::recover(&root).unwrap();
|
||||
assert_eq!(
|
||||
out,
|
||||
PublishOutcome::Recovered {
|
||||
generation: "gen_r".into()
|
||||
}
|
||||
);
|
||||
assert_eq!(
|
||||
fs::read_to_string(root.join("CURRENT")).unwrap().trim(),
|
||||
"gen_r"
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn logical_id_stable_across_generation_paths() {
|
||||
let a = logical_document_id("proj", "docs/a.mq.md");
|
||||
let b = logical_document_id("proj", "docs/a.mq.md");
|
||||
assert_eq!(a, b);
|
||||
assert_ne!(a, logical_document_id("proj", "docs/b.mq.md"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,249 @@
|
||||
//! Bounded admission classes and fair queueing (M-R3).
|
||||
|
||||
use std::collections::{HashMap, VecDeque};
|
||||
use std::sync::Mutex;
|
||||
|
||||
use crate::runtime::task_store::TaskRecord;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum AdmissionClass {
|
||||
Control,
|
||||
Read,
|
||||
Short,
|
||||
Execute,
|
||||
Commit,
|
||||
/// Undeclared endpoints stay on the legacy single-interpreter path.
|
||||
LegacySerial,
|
||||
}
|
||||
|
||||
impl AdmissionClass {
|
||||
pub fn parse(s: &str) -> Self {
|
||||
match s.to_ascii_lowercase().as_str() {
|
||||
"control" => Self::Control,
|
||||
"read" => Self::Read,
|
||||
"short" => Self::Short,
|
||||
"execute" => Self::Execute,
|
||||
"commit" => Self::Commit,
|
||||
_ => Self::LegacySerial,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SchedulerConfig {
|
||||
pub execute_slots: usize,
|
||||
pub short_slots: usize,
|
||||
pub read_slots: usize,
|
||||
pub global_queue_limit: usize,
|
||||
pub workspace_queue_limit: usize,
|
||||
pub session_pending_limit: usize,
|
||||
}
|
||||
|
||||
impl Default for SchedulerConfig {
|
||||
fn default() -> Self {
|
||||
// 4 vCPU / 8 GiB verification machine defaults from the design.
|
||||
Self {
|
||||
execute_slots: 2,
|
||||
short_slots: 1,
|
||||
read_slots: 1,
|
||||
global_queue_limit: 64,
|
||||
workspace_queue_limit: 16,
|
||||
session_pending_limit: 4,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum ScheduleError {
|
||||
QueueFull,
|
||||
WorkspaceFull,
|
||||
SessionFull,
|
||||
NoSlot,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for ScheduleError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::QueueFull => write!(f, "queue_full"),
|
||||
Self::WorkspaceFull => write!(f, "workspace_full"),
|
||||
Self::SessionFull => write!(f, "session_full"),
|
||||
Self::NoSlot => write!(f, "no_slot"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct QueueState {
|
||||
global: VecDeque<String>,
|
||||
by_workspace: HashMap<String, VecDeque<String>>,
|
||||
by_session: HashMap<String, usize>,
|
||||
active_execute: usize,
|
||||
active_short: usize,
|
||||
active_read: usize,
|
||||
/// workspace rotation cursor
|
||||
workspace_order: VecDeque<String>,
|
||||
}
|
||||
|
||||
pub struct Scheduler {
|
||||
cfg: SchedulerConfig,
|
||||
inner: Mutex<QueueState>,
|
||||
}
|
||||
|
||||
impl Scheduler {
|
||||
pub fn new(cfg: SchedulerConfig) -> Self {
|
||||
Self {
|
||||
cfg,
|
||||
inner: Mutex::new(QueueState {
|
||||
global: VecDeque::new(),
|
||||
by_workspace: HashMap::new(),
|
||||
by_session: HashMap::new(),
|
||||
active_execute: 0,
|
||||
active_short: 0,
|
||||
active_read: 0,
|
||||
workspace_order: VecDeque::new(),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn enqueue(&self, task: &TaskRecord, class: AdmissionClass) -> Result<usize, ScheduleError> {
|
||||
if class == AdmissionClass::Control || class == AdmissionClass::LegacySerial {
|
||||
return Ok(0);
|
||||
}
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
if g.global.len() >= self.cfg.global_queue_limit {
|
||||
return Err(ScheduleError::QueueFull);
|
||||
}
|
||||
let ws_len = g
|
||||
.by_workspace
|
||||
.get(&task.workspace_id)
|
||||
.map(|q| q.len())
|
||||
.unwrap_or(0);
|
||||
if ws_len >= self.cfg.workspace_queue_limit {
|
||||
return Err(ScheduleError::WorkspaceFull);
|
||||
}
|
||||
if let Some(sid) = &task.session_id {
|
||||
let n = g.by_session.entry(sid.clone()).or_insert(0);
|
||||
if *n >= self.cfg.session_pending_limit {
|
||||
return Err(ScheduleError::SessionFull);
|
||||
}
|
||||
*n += 1;
|
||||
}
|
||||
if !g.workspace_order.iter().any(|w| w == &task.workspace_id) {
|
||||
g.workspace_order.push_back(task.workspace_id.clone());
|
||||
}
|
||||
g.by_workspace
|
||||
.entry(task.workspace_id.clone())
|
||||
.or_default()
|
||||
.push_back(task.task_id.clone());
|
||||
g.global.push_back(task.task_id.clone());
|
||||
Ok(g.global.len())
|
||||
}
|
||||
|
||||
pub fn try_acquire_slot(&self, class: AdmissionClass) -> Result<(), ScheduleError> {
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
match class {
|
||||
AdmissionClass::Execute => {
|
||||
if g.active_execute >= self.cfg.execute_slots {
|
||||
return Err(ScheduleError::NoSlot);
|
||||
}
|
||||
g.active_execute += 1;
|
||||
}
|
||||
AdmissionClass::Short | AdmissionClass::Commit => {
|
||||
if g.active_short >= self.cfg.short_slots {
|
||||
return Err(ScheduleError::NoSlot);
|
||||
}
|
||||
g.active_short += 1;
|
||||
}
|
||||
AdmissionClass::Read => {
|
||||
if g.active_read >= self.cfg.read_slots {
|
||||
return Err(ScheduleError::NoSlot);
|
||||
}
|
||||
g.active_read += 1;
|
||||
}
|
||||
AdmissionClass::Control | AdmissionClass::LegacySerial => {}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn release_slot(&self, class: AdmissionClass) {
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
match class {
|
||||
AdmissionClass::Execute => {
|
||||
g.active_execute = g.active_execute.saturating_sub(1);
|
||||
}
|
||||
AdmissionClass::Short | AdmissionClass::Commit => {
|
||||
g.active_short = g.active_short.saturating_sub(1);
|
||||
}
|
||||
AdmissionClass::Read => {
|
||||
g.active_read = g.active_read.saturating_sub(1);
|
||||
}
|
||||
AdmissionClass::Control | AdmissionClass::LegacySerial => {}
|
||||
}
|
||||
}
|
||||
|
||||
/// Fair pop: rotate workspaces, then session FIFO within workspace.
|
||||
pub fn pop_next(&self) -> Option<String> {
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
if g.workspace_order.is_empty() {
|
||||
return None;
|
||||
}
|
||||
for _ in 0..g.workspace_order.len() {
|
||||
let ws = g.workspace_order.pop_front()?;
|
||||
if let Some(q) = g.by_workspace.get_mut(&ws) {
|
||||
if let Some(tid) = q.pop_front() {
|
||||
g.workspace_order.push_back(ws);
|
||||
g.global.retain(|id| id != &tid);
|
||||
return Some(tid);
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
pub fn queue_depth(&self) -> usize {
|
||||
self.inner.lock().unwrap().global.len()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::runtime::task_store::{TaskRecord, TaskState};
|
||||
|
||||
fn task(id: &str, ws: &str, session: Option<&str>) -> TaskRecord {
|
||||
TaskRecord {
|
||||
task_id: id.into(),
|
||||
run_id: format!("r-{id}"),
|
||||
attempt_id: format!("a-{id}"),
|
||||
workspace_id: ws.into(),
|
||||
session_id: session.map(|s| s.into()),
|
||||
state: TaskState::Queued,
|
||||
revision: 1,
|
||||
idempotency_key: id.into(),
|
||||
payload_hash: "h".into(),
|
||||
lease_id: None,
|
||||
lease_epoch: None,
|
||||
created_at: "0".into(),
|
||||
updated_at: "0".into(),
|
||||
cancel_requested_at: None,
|
||||
result: None,
|
||||
error: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn workspace_rotation_fairness() {
|
||||
let sched = Scheduler::new(SchedulerConfig::default());
|
||||
sched
|
||||
.enqueue(&task("a1", "wsA", None), AdmissionClass::Execute)
|
||||
.unwrap();
|
||||
sched
|
||||
.enqueue(&task("a2", "wsA", None), AdmissionClass::Execute)
|
||||
.unwrap();
|
||||
sched
|
||||
.enqueue(&task("b1", "wsB", None), AdmissionClass::Execute)
|
||||
.unwrap();
|
||||
assert_eq!(sched.pop_next().as_deref(), Some("a1"));
|
||||
assert_eq!(sched.pop_next().as_deref(), Some("b1"));
|
||||
assert_eq!(sched.pop_next().as_deref(), Some("a2"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,303 @@
|
||||
//! Supervisor: lease workers, enforce deadlines, reap orphans (M-R2).
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use crate::runtime::lease::{Lease, WorkerState};
|
||||
use crate::runtime::protocol::InvokeRequestV2;
|
||||
use crate::runtime::worker::{WorkerHandle, WorkerPool};
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SupervisorConfig {
|
||||
pub execute_slots: usize,
|
||||
pub worker_ttl: Duration,
|
||||
pub cancel_grace: Duration,
|
||||
pub force_kill_after: Duration,
|
||||
}
|
||||
|
||||
impl Default for SupervisorConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
execute_slots: 2,
|
||||
worker_ttl: Duration::from_secs(600),
|
||||
cancel_grace: Duration::from_secs(2),
|
||||
force_kill_after: Duration::from_secs(3),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct ActiveWorker {
|
||||
handle: WorkerHandle,
|
||||
cancel_since: Option<Instant>,
|
||||
}
|
||||
|
||||
pub struct Supervisor {
|
||||
cfg: SupervisorConfig,
|
||||
pool: Mutex<WorkerPool>,
|
||||
active: Mutex<HashMap<String, ActiveWorker>>,
|
||||
cancel_dir: PathBuf,
|
||||
event_dir: PathBuf,
|
||||
}
|
||||
|
||||
impl Supervisor {
|
||||
pub fn new(
|
||||
cfg: SupervisorConfig,
|
||||
staging_root: impl AsRef<Path>,
|
||||
marqdo_bin: impl AsRef<Path>,
|
||||
cancel_dir: impl AsRef<Path>,
|
||||
event_dir: impl AsRef<Path>,
|
||||
) -> Self {
|
||||
let cancel_dir = cancel_dir.as_ref().to_path_buf();
|
||||
let event_dir = event_dir.as_ref().to_path_buf();
|
||||
let _ = std::fs::create_dir_all(&cancel_dir);
|
||||
let _ = std::fs::create_dir_all(&event_dir);
|
||||
Self {
|
||||
pool: Mutex::new(WorkerPool::new(
|
||||
staging_root,
|
||||
marqdo_bin,
|
||||
cfg.execute_slots,
|
||||
)),
|
||||
cfg,
|
||||
active: Mutex::new(HashMap::new()),
|
||||
cancel_dir,
|
||||
event_dir,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn warm(&self) -> Result<(), String> {
|
||||
self.pool.lock().unwrap().warm()
|
||||
}
|
||||
|
||||
pub fn dispatch(&self, req: InvokeRequestV2) -> Result<String, String> {
|
||||
req.validate()?;
|
||||
{
|
||||
let active = self.active.lock().unwrap();
|
||||
if active.len() >= self.cfg.execute_slots {
|
||||
return Err("supervisor.no_slot".into());
|
||||
}
|
||||
}
|
||||
let mut pool = self.pool.lock().unwrap();
|
||||
let worker = pool.acquire()?;
|
||||
drop(pool);
|
||||
let epoch = Instant::now().elapsed().as_nanos() as u64;
|
||||
let lease = Lease::new(
|
||||
format!("lease_{}", req.task_id),
|
||||
epoch,
|
||||
&req.task_id,
|
||||
&req.run_id,
|
||||
&req.attempt_id,
|
||||
self.cfg.worker_ttl,
|
||||
);
|
||||
let cancel = self.cancel_dir.join(format!("{}.tok", req.task_id));
|
||||
if !cancel.exists() {
|
||||
std::fs::write(&cancel, b"").map_err(|e| e.to_string())?;
|
||||
}
|
||||
let events = self.event_dir.join(format!("{}.ndjson", req.task_id));
|
||||
std::fs::write(&events, b"").map_err(|e| e.to_string())?;
|
||||
// Durable emit path (not under staging — WorkerHandle::Drop removes staging).
|
||||
let emit = self.event_dir.join(format!("{}.result.json", req.task_id));
|
||||
let fs_root = req
|
||||
.context
|
||||
.fs_root
|
||||
.as_ref()
|
||||
.map(PathBuf::from)
|
||||
.filter(|p| p.is_dir())
|
||||
.ok_or_else(|| "supervisor.fs_root_required".to_string())?;
|
||||
let args_path = req.args_ref.as_ref().and_then(|r| {
|
||||
r.strip_prefix("file:")
|
||||
.map(PathBuf::from)
|
||||
.or_else(|| {
|
||||
let p = PathBuf::from(r);
|
||||
p.is_file().then_some(p)
|
||||
})
|
||||
});
|
||||
let handle = {
|
||||
if std::env::var_os("MARQDO_RUNTIME_DRY_RUN").is_some() {
|
||||
let mut w = worker;
|
||||
w.lease = Some(lease.clone());
|
||||
w.emit_result = Some(emit.clone());
|
||||
w.state = WorkerState::Running;
|
||||
w
|
||||
} else {
|
||||
self.pool.lock().unwrap().lease_and_start(
|
||||
worker,
|
||||
lease.clone(),
|
||||
&req,
|
||||
&cancel,
|
||||
&events,
|
||||
&fs_root,
|
||||
args_path.as_deref(),
|
||||
&emit,
|
||||
)?
|
||||
}
|
||||
};
|
||||
let lease_id = lease.id.0.clone();
|
||||
self.active.lock().unwrap().insert(
|
||||
lease_id.clone(),
|
||||
ActiveWorker {
|
||||
handle,
|
||||
cancel_since: None,
|
||||
},
|
||||
);
|
||||
Ok(lease_id)
|
||||
}
|
||||
|
||||
pub fn event_path(&self, task_id: &str) -> PathBuf {
|
||||
self.event_dir.join(format!("{task_id}.ndjson"))
|
||||
}
|
||||
|
||||
pub fn cancel_path(&self, task_id: &str) -> PathBuf {
|
||||
self.cancel_dir.join(format!("{task_id}.tok"))
|
||||
}
|
||||
|
||||
pub fn emit_result_path(&self, lease_id: &str) -> Option<PathBuf> {
|
||||
self.active
|
||||
.lock()
|
||||
.unwrap()
|
||||
.get(lease_id)
|
||||
.and_then(|w| w.handle.emit_result.clone())
|
||||
}
|
||||
|
||||
/// Non-blocking poll: if worker exited, remove from active and return exit code.
|
||||
pub fn try_reap(&self, lease_id: &str) -> Result<Option<i32>, String> {
|
||||
let mut active = self.active.lock().unwrap();
|
||||
let Some(w) = active.get_mut(lease_id) else {
|
||||
return Ok(None);
|
||||
};
|
||||
match w.handle.try_wait()? {
|
||||
Some(st) => {
|
||||
let code = st.code().unwrap_or(-1);
|
||||
active.remove(lease_id);
|
||||
let _ = self.pool.lock().unwrap().warm();
|
||||
Ok(Some(code))
|
||||
}
|
||||
None => Ok(None),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn signal_cancel(&self, lease_id: &str, task_id: &str) -> Result<(), String> {
|
||||
let tok = self.cancel_dir.join(format!("{task_id}.tok"));
|
||||
std::fs::write(&tok, b"cancelled").map_err(|e| e.to_string())?;
|
||||
let mut active = self.active.lock().unwrap();
|
||||
if let Some(w) = active.get_mut(lease_id) {
|
||||
w.cancel_since = Some(Instant::now());
|
||||
w.handle.state = WorkerState::Draining;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Reap expired / force-kill after cancel grace. Returns orphan count.
|
||||
pub fn janitor_tick(&self) -> usize {
|
||||
let mut orphans = 0usize;
|
||||
let mut active = self.active.lock().unwrap();
|
||||
let mut done = Vec::new();
|
||||
for (id, w) in active.iter_mut() {
|
||||
if let Some(lease) = &w.handle.lease {
|
||||
if lease.expired() {
|
||||
w.handle.kill_process_group();
|
||||
orphans += 1;
|
||||
done.push(id.clone());
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if let Some(since) = w.cancel_since {
|
||||
if since.elapsed() > self.cfg.cancel_grace + self.cfg.force_kill_after {
|
||||
w.handle.kill_process_group();
|
||||
orphans += 1;
|
||||
done.push(id.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
for id in done {
|
||||
active.remove(&id);
|
||||
}
|
||||
orphans
|
||||
}
|
||||
|
||||
pub fn active_count(&self) -> usize {
|
||||
self.active.lock().unwrap().len()
|
||||
}
|
||||
|
||||
pub fn release(&self, lease_id: &str) {
|
||||
let mut active = self.active.lock().unwrap();
|
||||
if let Some(mut w) = active.remove(lease_id) {
|
||||
w.handle.state = WorkerState::Draining;
|
||||
w.handle.kill_process_group();
|
||||
}
|
||||
// Async replenish hint.
|
||||
let _ = self.pool.lock().unwrap().warm();
|
||||
}
|
||||
}
|
||||
|
||||
/// Shared supervisor handle for HTTP/control wiring.
|
||||
pub type SharedSupervisor = Arc<Supervisor>;
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::runtime::protocol::{EntryRef, InvokeContext, LeaseRef};
|
||||
|
||||
fn req(task: &str) -> InvokeRequestV2 {
|
||||
InvokeRequestV2 {
|
||||
schema: "InvokeRequestV2".into(),
|
||||
schema_version: 2,
|
||||
task_id: task.into(),
|
||||
run_id: format!("run_{task}"),
|
||||
attempt_id: format!("attempt_{task}"),
|
||||
lease: LeaseRef {
|
||||
id: format!("lease_{task}"),
|
||||
epoch: 1,
|
||||
},
|
||||
entry: EntryRef {
|
||||
module: "agent/qdagent.mq.md".into(),
|
||||
function: "run".into(),
|
||||
},
|
||||
args_ref: None,
|
||||
context: InvokeContext {
|
||||
workspace_id: "ws".into(),
|
||||
snapshot_id: None,
|
||||
selection_ref: None,
|
||||
capabilities_ref: None,
|
||||
deadline_at: "2099-01-01T00:00:00Z".into(),
|
||||
fs_root: Some("/tmp".into()),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dry_run_two_parallel_leases() {
|
||||
std::env::set_var("MARQDO_RUNTIME_DRY_RUN", "1");
|
||||
let root = std::env::temp_dir().join(format!("mq-sup-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
let bin = std::env::current_exe().unwrap();
|
||||
let _ = std::fs::create_dir_all(root.join("fs"));
|
||||
let mut ra = req("a");
|
||||
ra.context.fs_root = Some(root.join("fs").to_string_lossy().into());
|
||||
let mut rb = req("b");
|
||||
rb.context.fs_root = ra.context.fs_root.clone();
|
||||
let mut rc = req("c");
|
||||
rc.context.fs_root = ra.context.fs_root.clone();
|
||||
let sup = Supervisor::new(
|
||||
SupervisorConfig {
|
||||
execute_slots: 2,
|
||||
..SupervisorConfig::default()
|
||||
},
|
||||
root.join("stage"),
|
||||
&bin,
|
||||
root.join("cancel"),
|
||||
root.join("events"),
|
||||
);
|
||||
sup.warm().unwrap();
|
||||
let a = sup.dispatch(ra).unwrap();
|
||||
let b = sup.dispatch(rb).unwrap();
|
||||
assert_ne!(a, b);
|
||||
assert_eq!(sup.active_count(), 2);
|
||||
assert!(sup.dispatch(rc).is_err());
|
||||
sup.release(&a);
|
||||
assert_eq!(sup.active_count(), 1);
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
std::env::remove_var("MARQDO_RUNTIME_DRY_RUN");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,385 @@
|
||||
//! Task index with idempotency, revision CAS, and event log (M-R3 / M-R4).
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
use crate::runtime::protocol::RuntimeEventV2;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum TaskState {
|
||||
Accepted,
|
||||
Queued,
|
||||
Running,
|
||||
Finalizing,
|
||||
Succeeded,
|
||||
Failed,
|
||||
CancelRequested,
|
||||
Cancelled,
|
||||
Interrupted,
|
||||
NeedsReconciliation,
|
||||
}
|
||||
|
||||
impl TaskState {
|
||||
pub fn is_terminal(self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
Self::Succeeded
|
||||
| Self::Failed
|
||||
| Self::Cancelled
|
||||
| Self::Interrupted
|
||||
| Self::NeedsReconciliation
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct TaskRecord {
|
||||
pub task_id: String,
|
||||
pub run_id: String,
|
||||
pub attempt_id: String,
|
||||
pub workspace_id: String,
|
||||
pub session_id: Option<String>,
|
||||
pub state: TaskState,
|
||||
pub revision: u64,
|
||||
pub idempotency_key: String,
|
||||
pub payload_hash: String,
|
||||
pub lease_id: Option<String>,
|
||||
pub lease_epoch: Option<u64>,
|
||||
pub created_at: String,
|
||||
pub updated_at: String,
|
||||
pub cancel_requested_at: Option<String>,
|
||||
pub result: Option<Value>,
|
||||
pub error: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct Inner {
|
||||
tasks: HashMap<String, TaskRecord>,
|
||||
/// idempotency_key → task_id
|
||||
idem: HashMap<String, String>,
|
||||
/// task_id → events (monotonic seq)
|
||||
events: HashMap<String, Vec<RuntimeEventV2>>,
|
||||
next_epoch: u64,
|
||||
}
|
||||
|
||||
pub struct TaskStore {
|
||||
inner: Arc<Mutex<Inner>>,
|
||||
persist_dir: Option<PathBuf>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum StoreError {
|
||||
Conflict(String),
|
||||
NotFound,
|
||||
AlreadyTerminal(TaskState),
|
||||
Io(String),
|
||||
}
|
||||
|
||||
impl std::fmt::Display for StoreError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::Conflict(s) => write!(f, "conflict: {s}"),
|
||||
Self::NotFound => write!(f, "not_found"),
|
||||
Self::AlreadyTerminal(s) => write!(f, "already_terminal:{s:?}"),
|
||||
Self::Io(s) => write!(f, "io: {s}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl TaskStore {
|
||||
pub fn memory() -> Self {
|
||||
Self {
|
||||
inner: Arc::new(Mutex::new(Inner::default())),
|
||||
persist_dir: None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_persist(dir: impl AsRef<Path>) -> Result<Self, StoreError> {
|
||||
let dir = dir.as_ref().to_path_buf();
|
||||
std::fs::create_dir_all(&dir).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
let store = Self {
|
||||
inner: Arc::new(Mutex::new(Inner::default())),
|
||||
persist_dir: Some(dir.clone()),
|
||||
};
|
||||
store.load()?;
|
||||
Ok(store)
|
||||
}
|
||||
|
||||
fn now() -> String {
|
||||
let ms = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_millis();
|
||||
format!("{ms}")
|
||||
}
|
||||
|
||||
pub fn payload_hash(payload: &Value) -> String {
|
||||
let bytes = serde_json::to_vec(payload).unwrap_or_default();
|
||||
format!("{:x}", Sha256::digest(&bytes))
|
||||
}
|
||||
|
||||
/// Admit or return existing task for the same idempotency key.
|
||||
/// Same key + different payload hash → Conflict (409).
|
||||
pub fn admit(
|
||||
&self,
|
||||
task_id: impl Into<String>,
|
||||
run_id: impl Into<String>,
|
||||
workspace_id: impl Into<String>,
|
||||
session_id: Option<String>,
|
||||
idempotency_key: impl Into<String>,
|
||||
payload: &Value,
|
||||
) -> Result<TaskRecord, StoreError> {
|
||||
let task_id = task_id.into();
|
||||
let idempotency_key = idempotency_key.into();
|
||||
let payload_hash = Self::payload_hash(payload);
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
if let Some(existing_id) = g.idem.get(&idempotency_key).cloned() {
|
||||
let existing = g.tasks.get(&existing_id).cloned().ok_or(StoreError::NotFound)?;
|
||||
if existing.payload_hash != payload_hash {
|
||||
return Err(StoreError::Conflict(
|
||||
"idempotency key reuse with different payload".into(),
|
||||
));
|
||||
}
|
||||
return Ok(existing);
|
||||
}
|
||||
let now = Self::now();
|
||||
let rec = TaskRecord {
|
||||
task_id: task_id.clone(),
|
||||
run_id: run_id.into(),
|
||||
attempt_id: format!("attempt_{}", task_id),
|
||||
workspace_id: workspace_id.into(),
|
||||
session_id,
|
||||
state: TaskState::Accepted,
|
||||
revision: 1,
|
||||
idempotency_key: idempotency_key.clone(),
|
||||
payload_hash,
|
||||
lease_id: None,
|
||||
lease_epoch: None,
|
||||
created_at: now.clone(),
|
||||
updated_at: now,
|
||||
cancel_requested_at: None,
|
||||
result: None,
|
||||
error: None,
|
||||
};
|
||||
g.idem.insert(idempotency_key, task_id.clone());
|
||||
g.tasks.insert(task_id, rec.clone());
|
||||
g.events.insert(rec.task_id.clone(), Vec::new());
|
||||
drop(g);
|
||||
self.persist_task(&rec)?;
|
||||
Ok(rec)
|
||||
}
|
||||
|
||||
pub fn get(&self, task_id: &str) -> Result<TaskRecord, StoreError> {
|
||||
self.inner
|
||||
.lock()
|
||||
.unwrap()
|
||||
.tasks
|
||||
.get(task_id)
|
||||
.cloned()
|
||||
.ok_or(StoreError::NotFound)
|
||||
}
|
||||
|
||||
/// CAS state transition. `expected_revision` must match.
|
||||
pub fn cas_state(
|
||||
&self,
|
||||
task_id: &str,
|
||||
expected_revision: u64,
|
||||
next: TaskState,
|
||||
patch: impl FnOnce(&mut TaskRecord),
|
||||
) -> Result<TaskRecord, StoreError> {
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
let rec = g.tasks.get_mut(task_id).ok_or(StoreError::NotFound)?;
|
||||
if rec.revision != expected_revision {
|
||||
return Err(StoreError::Conflict(format!(
|
||||
"revision {} != {}",
|
||||
rec.revision, expected_revision
|
||||
)));
|
||||
}
|
||||
if rec.state.is_terminal() && next != rec.state {
|
||||
return Err(StoreError::AlreadyTerminal(rec.state));
|
||||
}
|
||||
rec.state = next;
|
||||
rec.revision += 1;
|
||||
rec.updated_at = Self::now();
|
||||
patch(rec);
|
||||
let out = rec.clone();
|
||||
drop(g);
|
||||
self.persist_task(&out)?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
pub fn request_cancel(&self, task_id: &str) -> Result<TaskRecord, StoreError> {
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
let rec = g.tasks.get_mut(task_id).ok_or(StoreError::NotFound)?;
|
||||
if rec.state.is_terminal() {
|
||||
return Err(StoreError::AlreadyTerminal(rec.state));
|
||||
}
|
||||
let rev = rec.revision;
|
||||
rec.state = TaskState::CancelRequested;
|
||||
rec.revision = rev + 1;
|
||||
rec.cancel_requested_at = Some(Self::now());
|
||||
rec.updated_at = Self::now();
|
||||
let out = rec.clone();
|
||||
drop(g);
|
||||
self.persist_task(&out)?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
pub fn append_event(
|
||||
&self,
|
||||
task_id: &str,
|
||||
run_id: &str,
|
||||
attempt_id: &str,
|
||||
event_type: &str,
|
||||
payload: Value,
|
||||
) -> Result<RuntimeEventV2, StoreError> {
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
if !g.tasks.contains_key(task_id) {
|
||||
return Err(StoreError::NotFound);
|
||||
}
|
||||
let list = g.events.entry(task_id.to_string()).or_default();
|
||||
let seq = list.last().map(|e| e.seq + 1).unwrap_or(1);
|
||||
let ev = RuntimeEventV2::new(task_id, run_id, attempt_id, seq, event_type, payload);
|
||||
list.push(ev.clone());
|
||||
drop(g);
|
||||
self.persist_event(&ev)?;
|
||||
Ok(ev)
|
||||
}
|
||||
|
||||
/// Events with seq > after_seq (Last-Event-ID resume).
|
||||
pub fn events_after(
|
||||
&self,
|
||||
task_id: &str,
|
||||
after_seq: u64,
|
||||
) -> Result<Vec<RuntimeEventV2>, StoreError> {
|
||||
let g = self.inner.lock().unwrap();
|
||||
let list = g.events.get(task_id).ok_or(StoreError::NotFound)?;
|
||||
Ok(list.iter().filter(|e| e.seq > after_seq).cloned().collect())
|
||||
}
|
||||
|
||||
pub fn next_lease_epoch(&self) -> u64 {
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
g.next_epoch += 1;
|
||||
g.next_epoch
|
||||
}
|
||||
|
||||
fn persist_task(&self, rec: &TaskRecord) -> Result<(), StoreError> {
|
||||
let Some(dir) = &self.persist_dir else {
|
||||
return Ok(());
|
||||
};
|
||||
let path = dir.join("tasks").join(format!("{}.json", rec.task_id));
|
||||
std::fs::create_dir_all(path.parent().unwrap()).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
let tmp = path.with_extension("json.tmp");
|
||||
let body = serde_json::to_vec_pretty(rec).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
std::fs::write(&tmp, body).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
std::fs::rename(&tmp, &path).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn persist_event(&self, ev: &RuntimeEventV2) -> Result<(), StoreError> {
|
||||
let Some(dir) = &self.persist_dir else {
|
||||
return Ok(());
|
||||
};
|
||||
let path = dir.join("events").join(format!("{}.ndjson", ev.task_id));
|
||||
std::fs::create_dir_all(path.parent().unwrap()).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
use std::io::Write;
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(&path)
|
||||
.map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
let line = serde_json::to_string(ev).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
writeln!(f, "{line}").map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn load(&self) -> Result<(), StoreError> {
|
||||
let Some(dir) = &self.persist_dir else {
|
||||
return Ok(());
|
||||
};
|
||||
let tasks_dir = dir.join("tasks");
|
||||
if !tasks_dir.is_dir() {
|
||||
return Ok(());
|
||||
}
|
||||
let mut g = self.inner.lock().unwrap();
|
||||
for ent in std::fs::read_dir(&tasks_dir).map_err(|e| StoreError::Io(e.to_string()))? {
|
||||
let ent = ent.map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
if ent.path().extension().and_then(|e| e.to_str()) != Some("json") {
|
||||
continue;
|
||||
}
|
||||
let body =
|
||||
std::fs::read_to_string(ent.path()).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
let rec: TaskRecord =
|
||||
serde_json::from_str(&body).map_err(|e| StoreError::Io(e.to_string()))?;
|
||||
g.idem
|
||||
.insert(rec.idempotency_key.clone(), rec.task_id.clone());
|
||||
g.events.entry(rec.task_id.clone()).or_default();
|
||||
g.tasks.insert(rec.task_id.clone(), rec);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn idempotent_admit_and_conflict() {
|
||||
let store = TaskStore::memory();
|
||||
let p = json!({"a": 1});
|
||||
let a = store
|
||||
.admit("t1", "r1", "ws", None, "key-1", &p)
|
||||
.unwrap();
|
||||
let b = store
|
||||
.admit("t2", "r2", "ws", None, "key-1", &p)
|
||||
.unwrap();
|
||||
assert_eq!(a.task_id, b.task_id);
|
||||
let err = store
|
||||
.admit("t3", "r3", "ws", None, "key-1", &json!({"a": 2}))
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, StoreError::Conflict(_)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cancel_cas_terminal() {
|
||||
let store = TaskStore::memory();
|
||||
let t = store
|
||||
.admit("t1", "r1", "ws", None, "k", &json!({}))
|
||||
.unwrap();
|
||||
store
|
||||
.cas_state(&t.task_id, t.revision, TaskState::Succeeded, |r| {
|
||||
r.result = Some(json!("ok"));
|
||||
})
|
||||
.unwrap();
|
||||
let err = store.request_cancel(&t.task_id).unwrap_err();
|
||||
assert!(matches!(err, StoreError::AlreadyTerminal(TaskState::Succeeded)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn event_seq_monotonic_and_resume() {
|
||||
let store = TaskStore::memory();
|
||||
let t = store
|
||||
.admit("t1", "r1", "ws", None, "k", &json!({}))
|
||||
.unwrap();
|
||||
let e1 = store
|
||||
.append_event(&t.task_id, &t.run_id, &t.attempt_id, "message.delta", json!({"n":1}))
|
||||
.unwrap();
|
||||
let e2 = store
|
||||
.append_event(&t.task_id, &t.run_id, &t.attempt_id, "message.delta", json!({"n":2}))
|
||||
.unwrap();
|
||||
assert_eq!(e1.seq, 1);
|
||||
assert_eq!(e2.seq, 2);
|
||||
let after = store.events_after(&t.task_id, 1).unwrap();
|
||||
assert_eq!(after.len(), 1);
|
||||
assert_eq!(after[0].seq, 2);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,259 @@
|
||||
//! One-shot Marqdo worker process handles (M-R2).
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Child, Command, Stdio};
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use crate::runtime::lease::{Lease, WorkerState};
|
||||
use crate::runtime::protocol::{InvokeRequestV2, PROTOCOL_VERSION};
|
||||
|
||||
static WORKER_SEQ: AtomicU64 = AtomicU64::new(1);
|
||||
|
||||
pub struct WorkerHandle {
|
||||
pub id: String,
|
||||
pub state: WorkerState,
|
||||
pub lease: Option<Lease>,
|
||||
child: Option<Child>,
|
||||
staging: PathBuf,
|
||||
/// Where the child writes `--emit-result` JSON.
|
||||
pub emit_result: Option<PathBuf>,
|
||||
}
|
||||
|
||||
impl WorkerHandle {
|
||||
pub fn staging(&self) -> &Path {
|
||||
&self.staging
|
||||
}
|
||||
|
||||
pub fn take_child(&mut self) -> Option<Child> {
|
||||
self.child.take()
|
||||
}
|
||||
|
||||
pub fn kill_process_group(&mut self) {
|
||||
if let Some(mut child) = self.child.take() {
|
||||
let _ = child.kill();
|
||||
let _ = child.wait();
|
||||
}
|
||||
self.state = WorkerState::Exited;
|
||||
}
|
||||
|
||||
pub fn try_wait(&mut self) -> Result<Option<std::process::ExitStatus>, String> {
|
||||
let Some(child) = self.child.as_mut() else {
|
||||
return Ok(None);
|
||||
};
|
||||
child.try_wait().map_err(|e| format!("worker.wait: {e}"))
|
||||
}
|
||||
|
||||
pub fn wait(&mut self) -> Result<std::process::ExitStatus, String> {
|
||||
let Some(mut child) = self.child.take() else {
|
||||
return Err("worker.no_child".into());
|
||||
};
|
||||
let st = child.wait().map_err(|e| format!("worker.wait: {e}"))?;
|
||||
self.state = WorkerState::Exited;
|
||||
Ok(st)
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for WorkerHandle {
|
||||
fn drop(&mut self) {
|
||||
self.kill_process_group();
|
||||
let _ = std::fs::remove_dir_all(&self.staging);
|
||||
}
|
||||
}
|
||||
|
||||
pub struct WorkerPool {
|
||||
staging_root: PathBuf,
|
||||
marqdo_bin: PathBuf,
|
||||
ready: Vec<WorkerHandle>,
|
||||
target_ready: usize,
|
||||
}
|
||||
|
||||
impl WorkerPool {
|
||||
pub fn new(staging_root: impl AsRef<Path>, marqdo_bin: impl AsRef<Path>, target_ready: usize) -> Self {
|
||||
let staging_root = staging_root.as_ref().to_path_buf();
|
||||
let _ = std::fs::create_dir_all(&staging_root);
|
||||
Self {
|
||||
staging_root,
|
||||
marqdo_bin: marqdo_bin.as_ref().to_path_buf(),
|
||||
ready: Vec::new(),
|
||||
target_ready,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn spawn_ready(&mut self) -> Result<WorkerHandle, String> {
|
||||
let seq = WORKER_SEQ.fetch_add(1, Ordering::Relaxed);
|
||||
let id = format!("worker_{seq}");
|
||||
let staging = self.staging_root.join(&id);
|
||||
std::fs::create_dir_all(&staging).map_err(|e| e.to_string())?;
|
||||
if !self.marqdo_bin.exists() {
|
||||
return Err(format!("worker.binary_missing: {}", self.marqdo_bin.display()));
|
||||
}
|
||||
Ok(WorkerHandle {
|
||||
id,
|
||||
state: WorkerState::Ready,
|
||||
lease: None,
|
||||
child: None,
|
||||
staging,
|
||||
emit_result: None,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn warm(&mut self) -> Result<(), String> {
|
||||
while self.ready.len() < self.target_ready {
|
||||
let w = self.spawn_ready()?;
|
||||
self.ready.push(w);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn acquire(&mut self) -> Result<WorkerHandle, String> {
|
||||
if self.ready.is_empty() {
|
||||
self.warm()?;
|
||||
}
|
||||
self.ready
|
||||
.pop()
|
||||
.ok_or_else(|| "worker.pool_empty".to_string())
|
||||
}
|
||||
|
||||
pub fn replenish_async_hint(&self) -> usize {
|
||||
self.target_ready.saturating_sub(self.ready.len())
|
||||
}
|
||||
|
||||
/// Lease a worker: spawn `marqdo run --call-lib-path …` with request-scoped env.
|
||||
/// `emit_result` must be outside staging so Drop does not erase the parent's read path.
|
||||
pub fn lease_and_start(
|
||||
&self,
|
||||
mut worker: WorkerHandle,
|
||||
lease: Lease,
|
||||
req: &InvokeRequestV2,
|
||||
cancel_token: &Path,
|
||||
event_sink: &Path,
|
||||
fs_root: &Path,
|
||||
args_json: Option<&Path>,
|
||||
emit_result: &Path,
|
||||
) -> Result<WorkerHandle, String> {
|
||||
req.validate()?;
|
||||
let entry = PathBuf::from(&req.entry.module);
|
||||
if !entry.is_file() {
|
||||
return Err(format!(
|
||||
"worker.entry_missing: {} (need absolute site entry .mq.md)",
|
||||
entry.display()
|
||||
));
|
||||
}
|
||||
if let Some(parent) = emit_result.parent() {
|
||||
std::fs::create_dir_all(parent).map_err(|e| e.to_string())?;
|
||||
}
|
||||
let emit = emit_result.to_path_buf();
|
||||
let mut cmd = Command::new(&self.marqdo_bin);
|
||||
cmd.arg("run")
|
||||
.arg(&entry)
|
||||
.arg("--call-lib-path")
|
||||
.arg(&req.entry.function)
|
||||
.arg("--fs-root")
|
||||
.arg(fs_root)
|
||||
.arg("--emit-result")
|
||||
.arg(&emit)
|
||||
// Worker cwd = site root so relative imports resolve; staging only for lease bookkeeping.
|
||||
.current_dir(fs_root)
|
||||
.env("MARQDO_RUN_ID", &req.run_id)
|
||||
.env("MARQDO_TASK_ID", &req.task_id)
|
||||
.env("MARQDO_ATTEMPT_ID", &req.attempt_id)
|
||||
.env("MARQDO_LEASE_ID", &lease.id.0)
|
||||
.env("MARQDO_LEASE_EPOCH", lease.epoch.to_string())
|
||||
.env("MARQDO_CANCEL_TOKEN", cancel_token)
|
||||
.env("MARQDO_EVENT_FORWARD", event_sink)
|
||||
.env("MARQDO_WORKSPACE_ID", &req.context.workspace_id)
|
||||
.env("MARQDO_PROTOCOL_VERSION", PROTOCOL_VERSION.to_string())
|
||||
.env("MARQDO_FS_ROOT", fs_root)
|
||||
.stdin(Stdio::null())
|
||||
// Avoid pipe-buffer deadlock: nobody drains worker stdout/stderr.
|
||||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::null());
|
||||
if let Some(args) = args_json {
|
||||
cmd.arg("--args-json").arg(args);
|
||||
} else if let Some(r) = &req.args_ref {
|
||||
if let Some(path) = r.strip_prefix("file:") {
|
||||
cmd.arg("--args-json").arg(path);
|
||||
}
|
||||
}
|
||||
if let Some(sel) = &req.context.selection_ref {
|
||||
cmd.env("MARQDO_SELECTION_REF", sel);
|
||||
}
|
||||
if let Some(snap) = &req.context.snapshot_id {
|
||||
cmd.env("MARQDO_SNAPSHOT_ID", snap);
|
||||
}
|
||||
if let Some(cap) = &req.context.capabilities_ref {
|
||||
cmd.env("MARQDO_CAPABILITIES_REF", cap);
|
||||
}
|
||||
let child = cmd.spawn().map_err(|e| format!("worker.spawn: {e}"))?;
|
||||
worker.child = Some(child);
|
||||
worker.lease = Some(lease);
|
||||
worker.emit_result = Some(emit);
|
||||
worker.state = WorkerState::Running;
|
||||
Ok(worker)
|
||||
}
|
||||
}
|
||||
|
||||
pub fn handshake_ok(report: &Value) -> Result<(), String> {
|
||||
let ver = report
|
||||
.get("protocol_version")
|
||||
.and_then(|v| v.as_u64())
|
||||
.ok_or("handshake.protocol_version")?;
|
||||
if ver != PROTOCOL_VERSION as u64 {
|
||||
return Err(format!("handshake.version_mismatch: {ver}"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn fake_handshake() -> Value {
|
||||
json!({
|
||||
"cli": true,
|
||||
"protocol_version": PROTOCOL_VERSION,
|
||||
"capabilities": ["runtime.isolated_execution"],
|
||||
"build": env!("CARGO_PKG_VERSION"),
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn handshake_version_gate() {
|
||||
assert!(handshake_ok(&fake_handshake()).is_ok());
|
||||
assert!(handshake_ok(&json!({"protocol_version": 1})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pool_warm_requires_binary() {
|
||||
let dir = std::env::temp_dir().join(format!("mq-pool-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
let mut pool = WorkerPool::new(&dir, "/no/such/marqdo-bin", 1);
|
||||
assert!(pool.warm().is_err());
|
||||
let self_bin = std::env::current_exe().unwrap();
|
||||
let mut pool = WorkerPool::new(&dir, &self_bin, 1);
|
||||
pool.warm().unwrap();
|
||||
assert_eq!(pool.ready.len(), 1);
|
||||
let w = pool.acquire().unwrap();
|
||||
assert_eq!(w.state, WorkerState::Ready);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn kill_clears_child() {
|
||||
let dir = std::env::temp_dir().join(format!("mq-wh-{}", std::process::id()));
|
||||
let _ = std::fs::create_dir_all(&dir);
|
||||
let mut w = WorkerHandle {
|
||||
id: "w".into(),
|
||||
state: WorkerState::Running,
|
||||
lease: None,
|
||||
child: None,
|
||||
staging: dir.clone(),
|
||||
emit_result: None,
|
||||
};
|
||||
w.kill_process_group();
|
||||
assert_eq!(w.state, WorkerState::Exited);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,241 @@
|
||||
//! Shared call binding for execution (G02 BoundCall).
|
||||
//!
|
||||
//! Tree and Bytecode must agree: explicit import paths and same-module bare
|
||||
//! names bind; bare names never silently pick an imported symbol; true
|
||||
//! dynamics are marked separately.
|
||||
|
||||
use crate::ast::{CallExpr, Function, Module};
|
||||
use crate::diagnostics::Span;
|
||||
use crate::host::HostFn;
|
||||
|
||||
/// Why a call site is not a static BoundCall.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum DynamicReason {
|
||||
MethodReceiver,
|
||||
CallFn,
|
||||
Plugin,
|
||||
}
|
||||
|
||||
impl DynamicReason {
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::MethodReceiver => "method_receiver",
|
||||
Self::CallFn => "call_fn",
|
||||
Self::Plugin => "plugin",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Runtime resolution shared by Tree (and mirrored by Bytecode rules).
|
||||
#[derive(Debug)]
|
||||
pub enum ExecTarget<'a> {
|
||||
/// Function in the calling module (or nested children).
|
||||
Local(&'a Function),
|
||||
/// `bind.member…` through `module.import_modules`.
|
||||
Import {
|
||||
bind: String,
|
||||
lib: &'a Module,
|
||||
fun: &'a Function,
|
||||
path: Vec<String>,
|
||||
},
|
||||
Builtin(&'static str),
|
||||
Host(HostFn),
|
||||
/// Plugin / method / call_fn — not a static BoundCall.
|
||||
Dynamic {
|
||||
reason: DynamicReason,
|
||||
name: String,
|
||||
},
|
||||
Unresolved {
|
||||
message: String,
|
||||
#[allow(dead_code)]
|
||||
span_hint: Option<Span>,
|
||||
},
|
||||
}
|
||||
|
||||
pub fn call_display(call: &CallExpr) -> String {
|
||||
if let Some(path) = &call.path {
|
||||
return path.join(".");
|
||||
}
|
||||
if let Some(recv) = &call.receiver {
|
||||
return format!("{recv}.{}", call.callee);
|
||||
}
|
||||
call.callee.clone()
|
||||
}
|
||||
|
||||
fn find_in_children<'a>(fun: &'a Function, name: &str) -> Option<&'a Function> {
|
||||
for child in &fun.children {
|
||||
if child.name == name {
|
||||
return Some(child);
|
||||
}
|
||||
if let Some(f) = find_in_children(child, name) {
|
||||
return Some(f);
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Same-module bare-name lookup (never searches imports).
|
||||
pub fn lookup_local<'a>(
|
||||
module: &'a Module,
|
||||
current: &'a Function,
|
||||
name: &str,
|
||||
) -> Option<&'a Function> {
|
||||
if let Some(f) = find_in_children(current, name) {
|
||||
return Some(f);
|
||||
}
|
||||
module.functions.iter().find(|f| f.name == name)
|
||||
}
|
||||
|
||||
fn walk_path<'a>(lib: &'a Module, path: &[String]) -> Result<&'a Function, String> {
|
||||
let display = path.join(".");
|
||||
if path.is_empty() {
|
||||
return Err("empty knowledge path".into());
|
||||
}
|
||||
let mut parent_is_object = false;
|
||||
let mut node = lib
|
||||
.functions
|
||||
.iter()
|
||||
.find(|f| f.name == path[0])
|
||||
.ok_or_else(|| format!("unknown `{display}`"))?;
|
||||
for seg in &path[1..] {
|
||||
parent_is_object = node.is_object();
|
||||
node = node
|
||||
.children
|
||||
.iter()
|
||||
.find(|c| c.name == *seg)
|
||||
.ok_or_else(|| format!("unknown `{display}`"))?;
|
||||
}
|
||||
if parent_is_object && node.level != 1 && crate::ast::function_uses_receiver(node) {
|
||||
return Err(format!(
|
||||
"`{display}` is an instance method; call it as `var`.{}",
|
||||
node.name
|
||||
));
|
||||
}
|
||||
Ok(node)
|
||||
}
|
||||
|
||||
fn resolve_import_path<'a>(module: &'a Module, path: &[String]) -> ExecTarget<'a> {
|
||||
let display = path.join(".");
|
||||
if path.len() < 2 {
|
||||
return ExecTarget::Unresolved {
|
||||
message: format!("library path `{display}`: need at least `lib.member`"),
|
||||
span_hint: None,
|
||||
};
|
||||
}
|
||||
let bind = &path[0];
|
||||
let Some(lib) = module.import_modules.get(bind) else {
|
||||
return ExecTarget::Unresolved {
|
||||
message: format!("unknown library `{bind}` (import it in frontmatter)"),
|
||||
span_hint: None,
|
||||
};
|
||||
};
|
||||
match walk_path(lib, &path[1..]) {
|
||||
Ok(fun) => ExecTarget::Import {
|
||||
bind: bind.clone(),
|
||||
lib,
|
||||
fun,
|
||||
path: path.to_vec(),
|
||||
},
|
||||
Err(message) => ExecTarget::Unresolved {
|
||||
message,
|
||||
span_hint: None,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolve a call for execution. Import bare-name leakage is forbidden (G02).
|
||||
pub fn resolve_for_exec<'a>(
|
||||
module: &'a Module,
|
||||
current: &'a Function,
|
||||
call: &CallExpr,
|
||||
) -> ExecTarget<'a> {
|
||||
if call.receiver.is_some() {
|
||||
return ExecTarget::Dynamic {
|
||||
reason: DynamicReason::MethodReceiver,
|
||||
name: call_display(call),
|
||||
};
|
||||
}
|
||||
|
||||
if let Some(path) = &call.path {
|
||||
if module.import_modules.contains_key(&path[0]) {
|
||||
return resolve_import_path(module, path);
|
||||
}
|
||||
// Same-module definition-tree path (`Type.member` / nested), BoundCall.
|
||||
if path.len() >= 2 {
|
||||
match walk_path(module, path) {
|
||||
Ok(fun) => return ExecTarget::Local(fun),
|
||||
Err(_) if path.len() == 2 => {
|
||||
// Possible local-variable method; caller checks env.
|
||||
return ExecTarget::Dynamic {
|
||||
reason: DynamicReason::MethodReceiver,
|
||||
name: call_display(call),
|
||||
};
|
||||
}
|
||||
Err(message) => {
|
||||
return ExecTarget::Unresolved {
|
||||
message,
|
||||
span_hint: None,
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
return ExecTarget::Unresolved {
|
||||
message: format!(
|
||||
"library path `{}`: need at least `lib.member`",
|
||||
path.join(".")
|
||||
),
|
||||
span_hint: None,
|
||||
};
|
||||
}
|
||||
|
||||
if let Some(u) = module.uses.iter().find(|u| u.bind == call.callee) {
|
||||
return resolve_import_path(module, &u.path);
|
||||
}
|
||||
|
||||
match call.callee.as_str() {
|
||||
"print" | "input" | "len" | "str" | "int" | "type" | "trim" | "split" | "join" | "at" => {
|
||||
return ExecTarget::Builtin(match call.callee.as_str() {
|
||||
"print" => "print",
|
||||
"input" => "input",
|
||||
"len" => "len",
|
||||
"str" => "str",
|
||||
"int" => "int",
|
||||
"type" => "type",
|
||||
"trim" => "trim",
|
||||
"split" => "split",
|
||||
"join" => "join",
|
||||
"at" => "at",
|
||||
_ => unreachable!(),
|
||||
});
|
||||
}
|
||||
"call_fn" => {
|
||||
return ExecTarget::Dynamic {
|
||||
reason: DynamicReason::CallFn,
|
||||
name: "call_fn".into(),
|
||||
};
|
||||
}
|
||||
other if HostFn::from_name(other).is_some() => {
|
||||
return ExecTarget::Host(HostFn::from_name(other).unwrap());
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
if let Some(fun) = lookup_local(module, current, &call.callee) {
|
||||
return ExecTarget::Local(fun);
|
||||
}
|
||||
|
||||
ExecTarget::Dynamic {
|
||||
reason: DynamicReason::Plugin,
|
||||
name: call.callee.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_static_bound(target: &ExecTarget<'_>) -> bool {
|
||||
matches!(
|
||||
target,
|
||||
ExecTarget::Local(_)
|
||||
| ExecTarget::Import { .. }
|
||||
| ExecTarget::Builtin(_)
|
||||
| ExecTarget::Host(_)
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
//! Restricted pure-expression extract (S3).
|
||||
//!
|
||||
//! Only Literal / Var / Unary / Binary / List / Map / Index payloads may be
|
||||
//! lifted. Calls, host, and plugins are refused so effect counts stay zero.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
|
||||
use crate::ast::Expr;
|
||||
use crate::parse::expr::parse_expr_prefer_var;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ExtractPlan {
|
||||
pub name: String,
|
||||
pub params: Vec<String>,
|
||||
pub expression: String,
|
||||
pub heading: String,
|
||||
pub call: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ExtractError {
|
||||
pub code: &'static str,
|
||||
pub message: String,
|
||||
}
|
||||
|
||||
/// Build an extract plan from a pure expression string (no file I/O).
|
||||
pub fn plan_pure_extract(name: &str, expression: &str) -> Result<ExtractPlan, ExtractError> {
|
||||
let name = name.trim();
|
||||
if name.is_empty()
|
||||
|| name.contains(|c: char| c.is_whitespace() || c == '.' || c == '/')
|
||||
{
|
||||
return Err(ExtractError {
|
||||
code: "extract.bad_name",
|
||||
message: "提取目标名必须是单个标识符".into(),
|
||||
});
|
||||
}
|
||||
let expr = parse_expr_prefer_var(expression).map_err(|e| ExtractError {
|
||||
code: "extract.parse",
|
||||
message: format!("无法解析表达式: {e}"),
|
||||
})?;
|
||||
if !is_pure_payload(&expr) {
|
||||
return Err(ExtractError {
|
||||
code: "extract.not_pure",
|
||||
message: "仅允许字面量/变量/算术/列表等纯载荷;拒绝调用与宿主效应".into(),
|
||||
});
|
||||
}
|
||||
let params: Vec<String> = free_vars(&expr).into_iter().collect();
|
||||
let call = if params.is_empty() {
|
||||
format!("{name}()")
|
||||
} else {
|
||||
let args = params
|
||||
.iter()
|
||||
.map(|p| format!("{p}=`{p}`"))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
format!("{name}({args})")
|
||||
};
|
||||
let heading = if params.is_empty() {
|
||||
format!("## {name}\n\n*{expression}*\n")
|
||||
} else {
|
||||
let decls = params
|
||||
.iter()
|
||||
.map(|p| format!("`{p}`"))
|
||||
.collect::<Vec<_>>()
|
||||
.join("、");
|
||||
format!("## {name}\n\n参数:{decls}\n\n*{expression}*\n")
|
||||
};
|
||||
Ok(ExtractPlan {
|
||||
name: name.into(),
|
||||
params,
|
||||
expression: expression.trim().into(),
|
||||
heading,
|
||||
call,
|
||||
})
|
||||
}
|
||||
|
||||
fn is_pure_payload(expr: &Expr) -> bool {
|
||||
match expr {
|
||||
Expr::Literal(_) | Expr::Var(_) => true,
|
||||
Expr::Unary { expr, .. } => is_pure_payload(expr),
|
||||
Expr::Binary { left, right, .. } => is_pure_payload(left) && is_pure_payload(right),
|
||||
Expr::List(items) => items.iter().all(is_pure_payload),
|
||||
Expr::Map(items) => items.iter().all(|(_, v)| is_pure_payload(v)),
|
||||
Expr::Index { base, .. } => is_pure_payload(base),
|
||||
Expr::Occluded { payload, .. } => payload
|
||||
.as_ref()
|
||||
.map(|p| is_pure_payload(p))
|
||||
.unwrap_or(true),
|
||||
Expr::Call(_) | Expr::Interp(_) | Expr::Formula(_) | Expr::Code(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn free_vars(expr: &Expr) -> BTreeSet<String> {
|
||||
let mut out = BTreeSet::new();
|
||||
walk(expr, &mut out);
|
||||
out
|
||||
}
|
||||
|
||||
fn walk(expr: &Expr, out: &mut BTreeSet<String>) {
|
||||
match expr {
|
||||
Expr::Var(name) => {
|
||||
out.insert(name.clone());
|
||||
}
|
||||
Expr::Unary { expr, .. } => walk(expr, out),
|
||||
Expr::Binary { left, right, .. } => {
|
||||
walk(left, out);
|
||||
walk(right, out);
|
||||
}
|
||||
Expr::List(items) => {
|
||||
for item in items {
|
||||
walk(item, out);
|
||||
}
|
||||
}
|
||||
Expr::Map(items) => {
|
||||
for (_, item) in items {
|
||||
walk(item, out);
|
||||
}
|
||||
}
|
||||
Expr::Index { base, .. } => walk(base, out),
|
||||
Expr::Occluded { payload, .. } => {
|
||||
if let Some(p) = payload {
|
||||
walk(p, out);
|
||||
}
|
||||
}
|
||||
Expr::Literal(_)
|
||||
| Expr::Call(_)
|
||||
| Expr::Interp(_)
|
||||
| Expr::Formula(_)
|
||||
| Expr::Code(_) => {}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn extracts_product() {
|
||||
let plan = plan_pure_extract("商品小计", "数量 * 单价").unwrap();
|
||||
assert_eq!(plan.params, vec!["单价".to_string(), "数量".to_string()]);
|
||||
assert!(plan.heading.contains("## 商品小计"));
|
||||
assert!(plan.call.contains("商品小计("));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_call() {
|
||||
let err = plan_pure_extract("x", "save()").unwrap_err();
|
||||
assert_eq!(err.code, "extract.not_pure");
|
||||
}
|
||||
}
|
||||
+1106
-76
File diff suppressed because it is too large.
Load diff
+151
-20
@@ -15,6 +15,8 @@ pub struct LockReport {
|
||||
pub missing: Vec<String>,
|
||||
pub changed: Vec<String>,
|
||||
pub removed: Vec<String>,
|
||||
/// Per-artifact which digest class drifted (G08).
|
||||
pub classes: Vec<Value>,
|
||||
}
|
||||
|
||||
impl LockReport {
|
||||
@@ -29,6 +31,7 @@ impl LockReport {
|
||||
"missing": self.missing,
|
||||
"changed": self.changed,
|
||||
"removed": self.removed,
|
||||
"classes": self.classes,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -63,26 +66,53 @@ pub fn check_links_lock(out_dir: &Path, project: &SemanticProject) -> Result<Loc
|
||||
..LockReport::default()
|
||||
});
|
||||
}
|
||||
let expected = artifact_lock_map(project);
|
||||
let expected = artifact_lock_entries(project);
|
||||
let actual = raw.get("artifacts").and_then(Value::as_object).cloned().unwrap_or_default();
|
||||
let mut report = LockReport::default();
|
||||
for (id, digest) in &expected {
|
||||
for (id, entry) in &expected {
|
||||
match actual.get(id).and_then(Value::as_object) {
|
||||
None => report.missing.push(id.clone()),
|
||||
Some(found) if found.get("artifact").and_then(Value::as_str) != Some(digest.as_str()) => report.changed.push(id.clone()),
|
||||
_ => {}
|
||||
Some(found) => {
|
||||
let mut drifted = Vec::new();
|
||||
for key in ["interface", "implementation", "evidence", "artifact", "closure"] {
|
||||
let want = entry.get(key).and_then(Value::as_str);
|
||||
let got = found.get(key).and_then(Value::as_str);
|
||||
if want != got {
|
||||
drifted.push(key.to_string());
|
||||
}
|
||||
}
|
||||
if !drifted.is_empty() {
|
||||
report.changed.push(id.clone());
|
||||
report.classes.push(json!({
|
||||
"artifact_id": id,
|
||||
"changed": drifted,
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for id in actual.keys() {
|
||||
if !expected.contains_key(id) { report.removed.push(id.clone()); }
|
||||
if !expected.contains_key(id) {
|
||||
report.removed.push(id.clone());
|
||||
}
|
||||
}
|
||||
report.missing.sort(); report.changed.sort(); report.removed.sort();
|
||||
report.status = if report.missing.is_empty() && report.changed.is_empty() && report.removed.is_empty() { "clean".into() } else { "outdated".into() };
|
||||
report.missing.sort();
|
||||
report.changed.sort();
|
||||
report.removed.sort();
|
||||
report.status = if report.missing.is_empty()
|
||||
&& report.changed.is_empty()
|
||||
&& report.removed.is_empty()
|
||||
{
|
||||
"clean".into()
|
||||
} else {
|
||||
"outdated".into()
|
||||
};
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
/// Static reverse dependency closure. Queries can be an artifact/symbol URI,
|
||||
/// a declaration suffix, or a project-relative source filename.
|
||||
/// a declaration suffix, a project-relative source filename, or a corpus
|
||||
/// provenance URI (`supports` relation — M02).
|
||||
pub fn impact(project: &SemanticProject, query: &str) -> Value {
|
||||
let mut seeds = BTreeSet::new();
|
||||
for artifact in &project.artifacts {
|
||||
@@ -90,7 +120,21 @@ pub fn impact(project: &SemanticProject, query: &str) -> Value {
|
||||
|| artifact.symbol_id.as_ref().is_some_and(|id| id.as_str() == query)
|
||||
|| artifact.display_name == query
|
||||
|| artifact.source.file == query
|
||||
|| artifact.source.file.ends_with(query)
|
||||
|| std::path::Path::new(&artifact.source.file)
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.is_some_and(|stem| stem == query || query.ends_with(stem))
|
||||
|| artifact.artifact_id.0.ends_with(query)
|
||||
|| artifact.provenance.iter().any(|p| {
|
||||
p.relation == "supports"
|
||||
&& (p.uri == query
|
||||
|| p.uri.ends_with(query)
|
||||
|| query.ends_with(&p.uri)
|
||||
|| p.locator
|
||||
.as_ref()
|
||||
.is_some_and(|loc| format!("{}#{}", p.uri, loc) == query))
|
||||
})
|
||||
{
|
||||
seeds.insert(artifact.artifact_id.0.clone());
|
||||
}
|
||||
@@ -98,29 +142,115 @@ pub fn impact(project: &SemanticProject, query: &str) -> Value {
|
||||
let mut reverse: BTreeMap<String, Vec<String>> = BTreeMap::new();
|
||||
for artifact in &project.artifacts {
|
||||
for dependency in &artifact.dependencies {
|
||||
reverse.entry(dependency.0.clone()).or_default().push(artifact.artifact_id.0.clone());
|
||||
reverse
|
||||
.entry(dependency.0.clone())
|
||||
.or_default()
|
||||
.push(artifact.artifact_id.0.clone());
|
||||
}
|
||||
// Corpus URI → citing artifact (provenance supports edges).
|
||||
for prov in &artifact.provenance {
|
||||
if prov.relation != "supports" {
|
||||
continue;
|
||||
}
|
||||
reverse
|
||||
.entry(prov.uri.clone())
|
||||
.or_default()
|
||||
.push(artifact.artifact_id.0.clone());
|
||||
if let Some(loc) = &prov.locator {
|
||||
reverse
|
||||
.entry(format!("{}#{}", prov.uri, loc))
|
||||
.or_default()
|
||||
.push(artifact.artifact_id.0.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
// NarrativeRef edges: step → symbol; impact includes the step's owning artifact.
|
||||
for step in &project.steps {
|
||||
if step.kind != "narrate" {
|
||||
continue;
|
||||
}
|
||||
let owner = step
|
||||
.id
|
||||
.0
|
||||
.split("#step-")
|
||||
.next()
|
||||
.unwrap_or(step.id.0.as_str())
|
||||
.to_string();
|
||||
for target in &step.calls {
|
||||
reverse
|
||||
.entry(target.as_str().to_string())
|
||||
.or_default()
|
||||
.push(owner.clone());
|
||||
}
|
||||
}
|
||||
// Also seed reverse walk from corpus URI keys when query matches them.
|
||||
if reverse.contains_key(query) {
|
||||
seeds.insert(query.to_string());
|
||||
}
|
||||
let mut queue: VecDeque<String> = seeds.iter().cloned().collect();
|
||||
let mut affected = BTreeSet::new();
|
||||
while let Some(current) = queue.pop_front() {
|
||||
if !affected.insert(current.clone()) { continue; }
|
||||
if !affected.insert(current.clone()) {
|
||||
continue;
|
||||
}
|
||||
if let Some(users) = reverse.get(¤t) {
|
||||
for user in users { queue.push_back(user.clone()); }
|
||||
for user in users {
|
||||
queue.push_back(user.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
let entries = affected.iter().filter_map(|id| project.artifacts.iter().find(|a| &a.artifact_id.0 == id)).map(|artifact| json!({
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
"symbol_id": artifact.symbol_id.as_ref().map(|id| id.as_str()),
|
||||
"display_name": artifact.display_name,
|
||||
"kind": artifact.kind,
|
||||
"source": {"file": artifact.source.file, "line": artifact.source.line},
|
||||
})).collect::<Vec<_>>();
|
||||
// Drop pure corpus URI keys that are not artifacts from the affected set display.
|
||||
let entries = affected
|
||||
.iter()
|
||||
.filter_map(|id| project.artifacts.iter().find(|a| &a.artifact_id.0 == id))
|
||||
.map(|artifact| {
|
||||
let classes = {
|
||||
let mut c = Vec::new();
|
||||
if !artifact.hashes.interface.0.is_empty() {
|
||||
c.push("interface");
|
||||
}
|
||||
if !artifact.hashes.implementation.0.is_empty() {
|
||||
c.push("implementation");
|
||||
}
|
||||
if !artifact.hashes.evidence.0.is_empty() {
|
||||
c.push("evidence");
|
||||
}
|
||||
if !artifact.dependencies.is_empty() {
|
||||
c.push("dependency");
|
||||
}
|
||||
c
|
||||
};
|
||||
json!({
|
||||
"artifact_id": artifact.artifact_id.0,
|
||||
"symbol_id": artifact.symbol_id.as_ref().map(|id| id.as_str()),
|
||||
"display_name": artifact.display_name,
|
||||
"kind": artifact.kind,
|
||||
"source": {"file": artifact.source.file, "line": artifact.source.line},
|
||||
"hash_classes": classes,
|
||||
"evidence": artifact.verification.evidence,
|
||||
})
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
json!({"schema_version":SCHEMA_VERSION,"query":query,"resolved":!seeds.is_empty(),"seeds":seeds,"affected":entries})
|
||||
}
|
||||
|
||||
fn artifact_lock_map(project: &SemanticProject) -> BTreeMap<String, String> {
|
||||
project.artifacts.iter().map(|artifact| (artifact.artifact_id.0.clone(), artifact.hashes.artifact.0.clone())).collect()
|
||||
fn artifact_lock_entries(project: &SemanticProject) -> BTreeMap<String, Value> {
|
||||
project
|
||||
.artifacts
|
||||
.iter()
|
||||
.map(|artifact| {
|
||||
(
|
||||
artifact.artifact_id.0.clone(),
|
||||
json!({
|
||||
"interface": artifact.hashes.interface.0,
|
||||
"implementation": artifact.hashes.implementation.0,
|
||||
"evidence": artifact.hashes.evidence.0,
|
||||
"artifact": artifact.hashes.artifact.0,
|
||||
"closure": artifact.hashes.closure.0,
|
||||
}),
|
||||
)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn lock_json(project: &SemanticProject) -> Value {
|
||||
@@ -131,6 +261,7 @@ fn lock_json(project: &SemanticProject) -> Value {
|
||||
"implementation": artifact.hashes.implementation.0,
|
||||
"evidence": artifact.hashes.evidence.0,
|
||||
"artifact": artifact.hashes.artifact.0,
|
||||
"closure": artifact.hashes.closure.0,
|
||||
"dependencies": artifact.dependencies.iter().map(|item| &item.0).collect::<Vec<_>>(),
|
||||
}));
|
||||
}
|
||||
|
||||
+10
-2
@@ -4,13 +4,21 @@
|
||||
//! binding facts so runtime, EKC, view, and agents do not independently infer
|
||||
//! the meaning of the same source text.
|
||||
|
||||
pub mod bind;
|
||||
pub mod extract;
|
||||
pub mod ir;
|
||||
pub mod lock;
|
||||
pub mod provenance;
|
||||
pub mod symbols;
|
||||
|
||||
pub use bind::{
|
||||
call_display, is_static_bound, lookup_local, resolve_for_exec, DynamicReason, ExecTarget,
|
||||
};
|
||||
pub use extract::{plan_pure_extract, ExtractError, ExtractPlan};
|
||||
pub use ir::{
|
||||
artifact_json, compile_project, write_semantic_projections, ArtifactId, CompileOptions, Compilation,
|
||||
Digest, EffectSet, LockMode, SemanticArtifact, SemanticProject,
|
||||
artifact_json, compile_project, semantic_generation_dir, write_semantic_projections, ArtifactId,
|
||||
BoundCall, CompileOptions, Compilation, Digest, DynamicCall, EffectSet, LockMode,
|
||||
SemanticArtifact, SemanticProject,
|
||||
};
|
||||
pub use lock::{check_links_lock, impact as semantic_impact, update_links_lock, LockReport};
|
||||
pub use symbols::{CallableKind, ModuleId, ProjectIndex, SymbolDecl, SymbolId, SymbolKind};
|
||||
@@ -0,0 +1,512 @@
|
||||
//! Optional source / applicability facts from module frontmatter (G05 / M02).
|
||||
//!
|
||||
//! Authors declare `sources:` and optional `applicability:` / `valid_until` in
|
||||
//! frontmatter. Absence means a pure tool may still run but must stay
|
||||
//! `evidence=unverified` — never invent a business SourceRef.
|
||||
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use sha2::{Digest as _, Sha256};
|
||||
|
||||
use super::ir::{Digest, SourceRef};
|
||||
|
||||
/// Provenance extractor / verifier identity hashed into evidence digests.
|
||||
pub const PROVENANCE_EXTRACTOR_VERSION: &str = "marqdo-provenance/1";
|
||||
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct ModuleProvenance {
|
||||
/// Relative corpus paths, optional `#anchor`.
|
||||
pub sources: Vec<String>,
|
||||
pub valid_until: Option<String>,
|
||||
pub region: Option<String>,
|
||||
/// True when author wrote `sources: [...]` inline (ignored by parser — M07).
|
||||
pub inline_sources_ignored: bool,
|
||||
/// Optional per-function source overrides (M07): `function_sources: { name: [uri…] }`.
|
||||
pub function_sources: std::collections::BTreeMap<String, Vec<String>>,
|
||||
}
|
||||
|
||||
/// Scan YAML-ish frontmatter for sources / applicability (no full YAML parser).
|
||||
pub fn parse_module_provenance(source: &str) -> ModuleProvenance {
|
||||
let mut lines = source.lines();
|
||||
if lines.next().map(|l| l.trim_end()) != Some("---") {
|
||||
return ModuleProvenance::default();
|
||||
}
|
||||
let mut out = ModuleProvenance::default();
|
||||
let mut in_sources = false;
|
||||
let mut in_applicability = false;
|
||||
let mut in_function_sources = false;
|
||||
let mut current_fn: Option<String> = None;
|
||||
for line in lines {
|
||||
let t = line.trim_end();
|
||||
if t.trim() == "---" {
|
||||
break;
|
||||
}
|
||||
let trimmed = t.trim();
|
||||
if trimmed.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if !line.starts_with(' ') && !line.starts_with('\t') {
|
||||
in_sources = false;
|
||||
in_applicability = false;
|
||||
in_function_sources = false;
|
||||
current_fn = None;
|
||||
}
|
||||
if trimmed == "sources:" || trimmed.starts_with("sources:") {
|
||||
in_sources = true;
|
||||
in_applicability = false;
|
||||
in_function_sources = false;
|
||||
if let Some(rest) = trimmed.strip_prefix("sources:") {
|
||||
let rest = rest.trim();
|
||||
if !rest.is_empty() && rest != "|" {
|
||||
// inline list is not parsed — flag for diagnosis (M07).
|
||||
out.inline_sources_ignored = true;
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if trimmed == "function_sources:" || trimmed.starts_with("function_sources:") {
|
||||
in_function_sources = true;
|
||||
in_sources = false;
|
||||
in_applicability = false;
|
||||
current_fn = None;
|
||||
continue;
|
||||
}
|
||||
if trimmed == "applicability:" || trimmed.starts_with("applicability:") {
|
||||
in_applicability = true;
|
||||
in_sources = false;
|
||||
in_function_sources = false;
|
||||
continue;
|
||||
}
|
||||
if in_function_sources {
|
||||
// Indent levels: ` name:` then ` - uri`
|
||||
let indent = line.len() - line.trim_start().len();
|
||||
if trimmed.ends_with(':') && !trimmed.starts_with('-') {
|
||||
let name = trimmed.trim_end_matches(':').trim().to_string();
|
||||
if !name.is_empty() {
|
||||
current_fn = Some(name.clone());
|
||||
out.function_sources.entry(name).or_default();
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if let Some(item) = trimmed.strip_prefix("- ") {
|
||||
let path = item.trim().trim_matches('"').to_string();
|
||||
if let Some(name) = ¤t_fn {
|
||||
if !path.is_empty() {
|
||||
out.function_sources
|
||||
.entry(name.clone())
|
||||
.or_default()
|
||||
.push(path);
|
||||
}
|
||||
} else if indent >= 2 && !path.is_empty() {
|
||||
// ignore orphan list items
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if in_sources {
|
||||
if let Some(item) = trimmed.strip_prefix("- ") {
|
||||
let path = item.trim().trim_matches('"').to_string();
|
||||
if !path.is_empty() {
|
||||
out.sources.push(path);
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if in_applicability {
|
||||
if let Some(rest) = trimmed.strip_prefix("valid_until:") {
|
||||
out.valid_until = Some(rest.trim().trim_matches('"').to_string());
|
||||
} else if let Some(rest) = trimmed.strip_prefix("region:") {
|
||||
out.region = Some(rest.trim().trim_matches('"').to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Sources applicable to a named function: function_sources override, else module sources.
|
||||
pub fn sources_for_function(prov: &ModuleProvenance, function_name: &str) -> Vec<String> {
|
||||
if let Some(list) = prov.function_sources.get(function_name) {
|
||||
if !list.is_empty() {
|
||||
return list.clone();
|
||||
}
|
||||
}
|
||||
prov.sources.clone()
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum ProvenanceDiag {
|
||||
Missing { path: String },
|
||||
AnchorMissing { path: String, locator: String },
|
||||
ReadError { path: String, message: String },
|
||||
Expired { valid_until: String },
|
||||
InvalidDate { valid_until: String },
|
||||
InlineSourcesIgnored,
|
||||
}
|
||||
|
||||
impl ProvenanceDiag {
|
||||
pub fn code(&self) -> &'static str {
|
||||
match self {
|
||||
Self::Missing { .. } => "source.missing",
|
||||
Self::AnchorMissing { .. } => "source.anchor_missing",
|
||||
Self::ReadError { .. } => "source.read_error",
|
||||
Self::Expired { .. } => "source.expired",
|
||||
Self::InvalidDate { .. } => "source.invalid_date",
|
||||
Self::InlineSourcesIgnored => "source.inline_ignored",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn message(&self) -> String {
|
||||
match self {
|
||||
Self::Missing { path } => format!("来源文件缺失:`{path}`"),
|
||||
Self::AnchorMissing { path, locator } => {
|
||||
format!("来源锚点不存在:`{path}` 中找不到 `#{}`", locator)
|
||||
}
|
||||
Self::ReadError { path, message } => {
|
||||
format!("来源文件无法读取:`{path}`({message})")
|
||||
}
|
||||
Self::Expired { valid_until } => {
|
||||
format!("来源适用期已过(valid_until={valid_until})")
|
||||
}
|
||||
Self::InvalidDate { valid_until } => {
|
||||
format!("适用期日期格式无效(期望 YYYY-MM-DD):`{valid_until}`")
|
||||
}
|
||||
Self::InlineSourcesIgnored => {
|
||||
"sources 行内列表被忽略;请改用 YAML block list(`- path#anchor`)".into()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct ProvenanceResolution {
|
||||
pub refs: Vec<SourceRef>,
|
||||
pub applicability: String,
|
||||
pub evidence_status: String,
|
||||
pub diagnostics: Vec<ProvenanceDiag>,
|
||||
/// Canonical record hashed into `hashes.evidence` (M02).
|
||||
pub evidence_record: String,
|
||||
}
|
||||
|
||||
/// Resolve corpus paths relative to the module file; never mark verified here.
|
||||
pub fn resolve_provenance(
|
||||
module_path: &Path,
|
||||
prov: &ModuleProvenance,
|
||||
) -> ProvenanceResolution {
|
||||
resolve_provenance_sources(module_path, prov, &prov.sources)
|
||||
}
|
||||
|
||||
/// Resolve an explicit source list (module or per-function).
|
||||
pub fn resolve_provenance_sources(
|
||||
module_path: &Path,
|
||||
prov: &ModuleProvenance,
|
||||
sources: &[String],
|
||||
) -> ProvenanceResolution {
|
||||
let mut refs = Vec::new();
|
||||
let mut diagnostics = Vec::new();
|
||||
if prov.inline_sources_ignored {
|
||||
diagnostics.push(ProvenanceDiag::InlineSourcesIgnored);
|
||||
}
|
||||
let base = module_path
|
||||
.parent()
|
||||
.map(|p| p.to_path_buf())
|
||||
.unwrap_or_else(|| PathBuf::from("."));
|
||||
|
||||
for raw in sources {
|
||||
let (path_part, locator) = match raw.split_once('#') {
|
||||
Some((p, a)) => (p.trim(), Some(a.trim().to_string())),
|
||||
None => (raw.as_str(), None),
|
||||
};
|
||||
let resolved = base.join(path_part);
|
||||
if !resolved.is_file() {
|
||||
diagnostics.push(ProvenanceDiag::Missing {
|
||||
path: raw.clone(),
|
||||
});
|
||||
refs.push(SourceRef {
|
||||
uri: path_part.to_string(),
|
||||
locator,
|
||||
content_digest: None,
|
||||
relation: "supports".into(),
|
||||
});
|
||||
continue;
|
||||
}
|
||||
let content = match fs::read(&resolved) {
|
||||
Ok(bytes) => bytes,
|
||||
Err(e) => {
|
||||
diagnostics.push(ProvenanceDiag::ReadError {
|
||||
path: path_part.to_string(),
|
||||
message: e.to_string(),
|
||||
});
|
||||
refs.push(SourceRef {
|
||||
uri: path_part.to_string(),
|
||||
locator,
|
||||
content_digest: None,
|
||||
relation: "supports".into(),
|
||||
});
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let mut hash = Sha256::new();
|
||||
hash.update(&content);
|
||||
let content_digest = Digest(format!("sha256:{:x}", hash.finalize()));
|
||||
if let Some(loc) = locator.as_ref() {
|
||||
let text = String::from_utf8_lossy(&content);
|
||||
if !anchor_exists(&text, loc) {
|
||||
diagnostics.push(ProvenanceDiag::AnchorMissing {
|
||||
path: path_part.to_string(),
|
||||
locator: loc.clone(),
|
||||
});
|
||||
}
|
||||
}
|
||||
refs.push(SourceRef {
|
||||
uri: path_part.to_string(),
|
||||
locator,
|
||||
content_digest: Some(content_digest),
|
||||
relation: "supports".into(),
|
||||
});
|
||||
}
|
||||
|
||||
if let Some(until) = &prov.valid_until {
|
||||
match validate_date(until) {
|
||||
DateCheck::Ok => {
|
||||
if is_expired(until) {
|
||||
diagnostics.push(ProvenanceDiag::Expired {
|
||||
valid_until: until.clone(),
|
||||
});
|
||||
}
|
||||
}
|
||||
DateCheck::Invalid => {
|
||||
diagnostics.push(ProvenanceDiag::InvalidDate {
|
||||
valid_until: until.clone(),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let applicability = match (&prov.region, &prov.valid_until) {
|
||||
(None, None) if prov.sources.is_empty() => {
|
||||
"no explicit applicability constraints".into()
|
||||
}
|
||||
(region, until) => {
|
||||
let mut parts = Vec::new();
|
||||
if let Some(r) = region {
|
||||
parts.push(format!("region={r}"));
|
||||
}
|
||||
if let Some(u) = until {
|
||||
parts.push(format!("valid_until={u}"));
|
||||
}
|
||||
if parts.is_empty() {
|
||||
"sources declared".into()
|
||||
} else {
|
||||
parts.join("; ")
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
let evidence_status = if diagnostics
|
||||
.iter()
|
||||
.any(|d| matches!(d, ProvenanceDiag::Missing { .. } | ProvenanceDiag::ReadError { .. }))
|
||||
{
|
||||
"missing_source".into()
|
||||
} else if diagnostics
|
||||
.iter()
|
||||
.any(|d| matches!(d, ProvenanceDiag::AnchorMissing { .. }))
|
||||
{
|
||||
"anchor_missing".into()
|
||||
} else if diagnostics
|
||||
.iter()
|
||||
.any(|d| matches!(d, ProvenanceDiag::Expired { .. }))
|
||||
{
|
||||
"expired".into()
|
||||
} else if diagnostics
|
||||
.iter()
|
||||
.any(|d| matches!(d, ProvenanceDiag::InvalidDate { .. }))
|
||||
{
|
||||
"invalid_date".into()
|
||||
} else if sources.is_empty() {
|
||||
"unverified".into()
|
||||
} else {
|
||||
// Corpus present and in date — still not domain-verified by compile alone.
|
||||
"sourced".into()
|
||||
};
|
||||
|
||||
let prov_for_record = ModuleProvenance {
|
||||
sources: sources.to_vec(),
|
||||
valid_until: prov.valid_until.clone(),
|
||||
region: prov.region.clone(),
|
||||
inline_sources_ignored: prov.inline_sources_ignored,
|
||||
function_sources: Default::default(),
|
||||
};
|
||||
let evidence_record = evidence_canonical_record(&prov_for_record, &refs);
|
||||
|
||||
ProvenanceResolution {
|
||||
refs,
|
||||
applicability,
|
||||
evidence_status,
|
||||
diagnostics,
|
||||
evidence_record,
|
||||
}
|
||||
}
|
||||
|
||||
/// Canonical evidence inputs: extractor version, applicability, sorted source facts.
|
||||
pub fn evidence_canonical_record(prov: &ModuleProvenance, refs: &[SourceRef]) -> String {
|
||||
let mut parts = vec![
|
||||
format!("extractor={PROVENANCE_EXTRACTOR_VERSION}"),
|
||||
format!("region={}", prov.region.as_deref().unwrap_or("")),
|
||||
format!("valid_until={}", prov.valid_until.as_deref().unwrap_or("")),
|
||||
];
|
||||
let mut source_parts: Vec<String> = refs
|
||||
.iter()
|
||||
.filter(|r| r.relation == "supports")
|
||||
.map(|r| {
|
||||
format!(
|
||||
"{}#{}@{}",
|
||||
r.uri,
|
||||
r.locator.as_deref().unwrap_or(""),
|
||||
r.content_digest
|
||||
.as_ref()
|
||||
.map(|d| d.0.as_str())
|
||||
.unwrap_or("none")
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
source_parts.sort();
|
||||
parts.extend(source_parts);
|
||||
parts.join("|")
|
||||
}
|
||||
|
||||
/// Whether a Markdown heading / id matches the `#locator` fragment.
|
||||
pub fn anchor_exists(content: &str, locator: &str) -> bool {
|
||||
let loc = locator.trim().trim_start_matches('#').trim();
|
||||
if loc.is_empty() {
|
||||
return true;
|
||||
}
|
||||
for line in content.lines() {
|
||||
let t = line.trim();
|
||||
if let Some(rest) = t.strip_prefix('#') {
|
||||
let heading = rest.trim_start_matches('#').trim();
|
||||
let heading = heading
|
||||
.split('{')
|
||||
.next()
|
||||
.unwrap_or(heading)
|
||||
.trim();
|
||||
if heading == loc || slugify_heading(heading) == loc {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
if t.contains(&format!("id=\"{loc}\""))
|
||||
|| t.contains(&format!("id='{loc}'"))
|
||||
|| t.contains(&format!("{{#{loc}}}"))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
fn slugify_heading(heading: &str) -> String {
|
||||
let mut out = String::new();
|
||||
for ch in heading.chars() {
|
||||
if ch.is_ascii_alphanumeric() {
|
||||
out.push(ch.to_ascii_lowercase());
|
||||
} else if ch == ' ' || ch == '-' || ch == '_' {
|
||||
if !out.ends_with('-') {
|
||||
out.push('-');
|
||||
}
|
||||
} else if !ch.is_ascii() {
|
||||
// Keep non-ASCII letters (common in ZH docs) without lowercasing rules.
|
||||
out.push(ch);
|
||||
}
|
||||
}
|
||||
out.trim_matches('-').to_string()
|
||||
}
|
||||
|
||||
enum DateCheck {
|
||||
Ok,
|
||||
Invalid,
|
||||
}
|
||||
|
||||
fn validate_date(valid_until: &str) -> DateCheck {
|
||||
let parts: Vec<&str> = valid_until.split('-').collect();
|
||||
if parts.len() != 3 {
|
||||
return DateCheck::Invalid;
|
||||
}
|
||||
let Ok(y) = parts[0].parse::<i32>() else {
|
||||
return DateCheck::Invalid;
|
||||
};
|
||||
let Ok(m) = parts[1].parse::<u32>() else {
|
||||
return DateCheck::Invalid;
|
||||
};
|
||||
let Ok(d) = parts[2].parse::<u32>() else {
|
||||
return DateCheck::Invalid;
|
||||
};
|
||||
if !(1..=12).contains(&m) || !(1..=31).contains(&d) || y < 1970 {
|
||||
return DateCheck::Invalid;
|
||||
}
|
||||
if parts[0].len() != 4 || parts[1].len() != 2 || parts[2].len() != 2 {
|
||||
return DateCheck::Invalid;
|
||||
}
|
||||
DateCheck::Ok
|
||||
}
|
||||
|
||||
fn is_expired(valid_until: &str) -> bool {
|
||||
let today = chrono_today();
|
||||
!valid_until.is_empty() && valid_until < today.as_str()
|
||||
}
|
||||
|
||||
/// Public recheck for handle.execute (M02).
|
||||
pub fn valid_until_expired(valid_until: &str) -> bool {
|
||||
matches!(validate_date(valid_until), DateCheck::Ok) && is_expired(valid_until)
|
||||
}
|
||||
|
||||
fn chrono_today() -> String {
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
let secs = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs() as i64;
|
||||
let days = secs.div_euclid(86_400);
|
||||
let (y, m, d) = civil_from_days(days);
|
||||
format!("{y:04}-{m:02}-{d:02}")
|
||||
}
|
||||
|
||||
/// Algorithm from Howard Hinnant (public domain).
|
||||
fn civil_from_days(z: i64) -> (i32, u32, u32) {
|
||||
let z = z + 719_468;
|
||||
let era = if z >= 0 { z } else { z - 146_096 }.div_euclid(146_097);
|
||||
let doe = (z - era * 146_097) as u64;
|
||||
let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146_096) / 365;
|
||||
let y = (yoe as i64 + era * 400) as i32;
|
||||
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
|
||||
let mp = (5 * doy + 2) / 153;
|
||||
let d = doy - (153 * mp + 2) / 5 + 1;
|
||||
let m = if mp < 10 { mp + 3 } else { mp - 9 };
|
||||
let y = if m <= 2 { y + 1 } else { y };
|
||||
(y, m as u32, d as u32)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn parse_sources_and_valid_until() {
|
||||
let src = "---\nsources:\n - ../corpus/tax.md#rate\napplicability:\n region: CN\n valid_until: 2099-12-31\n---\n\n# main\n";
|
||||
let p = parse_module_provenance(src);
|
||||
assert_eq!(p.sources, vec!["../corpus/tax.md#rate"]);
|
||||
assert_eq!(p.region.as_deref(), Some("CN"));
|
||||
assert_eq!(p.valid_until.as_deref(), Some("2099-12-31"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn anchor_matches_heading() {
|
||||
assert!(anchor_exists("# rate\n\nrate = 0.13\n", "rate"));
|
||||
assert!(!anchor_exists("# rate\n\nrate = 0.13\n", "missing-anchor"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalid_date_rejected() {
|
||||
assert!(matches!(validate_date("2099-12-31"), DateCheck::Ok));
|
||||
assert!(matches!(validate_date("not-a-date"), DateCheck::Invalid));
|
||||
assert!(matches!(validate_date("2099-13-01"), DateCheck::Invalid));
|
||||
}
|
||||
}
|
||||
@@ -92,15 +92,29 @@ pub struct SymbolDecl {
|
||||
pub line: u32,
|
||||
}
|
||||
|
||||
/// Parsed module retained so `compile_project` does not re-parse (M-R5b).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ParsedModule {
|
||||
pub path: PathBuf,
|
||||
pub text: String,
|
||||
pub module: crate::ast::Module,
|
||||
}
|
||||
|
||||
/// Deterministic read-only view of all declarations in one project.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ProjectIndex {
|
||||
project_id: String,
|
||||
root: ProjectRoot,
|
||||
modules: BTreeMap<ModuleId, PathBuf>,
|
||||
/// path → module id (avoids linear scan per file in compile_project).
|
||||
path_to_module: BTreeMap<PathBuf, ModuleId>,
|
||||
symbols: Vec<SymbolDecl>,
|
||||
by_id: BTreeMap<SymbolId, usize>,
|
||||
by_name: BTreeMap<String, Vec<usize>>,
|
||||
/// module → symbol indexes (avoids filtering the full symbol table per module).
|
||||
symbols_by_module: BTreeMap<ModuleId, Vec<usize>>,
|
||||
/// Shared parsed AST/text for the same input snapshot.
|
||||
parsed: BTreeMap<ModuleId, ParsedModule>,
|
||||
}
|
||||
|
||||
impl ProjectIndex {
|
||||
@@ -108,7 +122,9 @@ impl ProjectIndex {
|
||||
let project_id = project_id(root.path());
|
||||
let files = root.source_files()?;
|
||||
let mut modules = BTreeMap::new();
|
||||
let mut path_to_module = BTreeMap::new();
|
||||
let mut symbols = Vec::new();
|
||||
let mut parsed = BTreeMap::new();
|
||||
|
||||
for file in files {
|
||||
let source = fs::read_to_string(&file)
|
||||
@@ -121,6 +137,7 @@ impl ProjectIndex {
|
||||
file.display()
|
||||
);
|
||||
}
|
||||
path_to_module.insert(file.clone(), module_id.clone());
|
||||
let module = crate::load::parse_source_auto(&source)
|
||||
.with_context(|| format!("parse source {}", file.display()))?;
|
||||
for function in &module.functions {
|
||||
@@ -134,28 +151,47 @@ impl ProjectIndex {
|
||||
&mut symbols,
|
||||
)?;
|
||||
}
|
||||
parsed.insert(
|
||||
module_id.clone(),
|
||||
ParsedModule {
|
||||
path: file,
|
||||
text: source,
|
||||
module,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
symbols.sort_by(|a, b| a.id.cmp(&b.id));
|
||||
let mut by_id = BTreeMap::new();
|
||||
let mut by_name: BTreeMap<String, Vec<usize>> = BTreeMap::new();
|
||||
let mut symbols_by_module: BTreeMap<ModuleId, Vec<usize>> = BTreeMap::new();
|
||||
for (index, symbol) in symbols.iter().enumerate() {
|
||||
if by_id.insert(symbol.id.clone(), index).is_some() {
|
||||
bail!("duplicate symbol id `{}`", symbol.id);
|
||||
}
|
||||
by_name.entry(symbol.name.clone()).or_default().push(index);
|
||||
symbols_by_module
|
||||
.entry(symbol.module.clone())
|
||||
.or_default()
|
||||
.push(index);
|
||||
}
|
||||
for indexes in by_name.values_mut() {
|
||||
indexes.sort_by(|a, b| symbols[*a].id.cmp(&symbols[*b].id));
|
||||
}
|
||||
for indexes in symbols_by_module.values_mut() {
|
||||
indexes.sort_by(|a, b| symbols[*a].id.cmp(&symbols[*b].id));
|
||||
}
|
||||
|
||||
Ok(Self {
|
||||
project_id,
|
||||
root,
|
||||
modules,
|
||||
path_to_module,
|
||||
symbols,
|
||||
by_id,
|
||||
by_name,
|
||||
symbols_by_module,
|
||||
parsed,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -171,6 +207,21 @@ impl ProjectIndex {
|
||||
&self.modules
|
||||
}
|
||||
|
||||
pub fn module_id_for_path(&self, path: &Path) -> Option<&ModuleId> {
|
||||
self.path_to_module.get(path)
|
||||
}
|
||||
|
||||
pub fn symbols_in_module(&self, module_id: &ModuleId) -> Vec<&SymbolDecl> {
|
||||
self.symbols_by_module
|
||||
.get(module_id)
|
||||
.map(|idxs| idxs.iter().map(|i| &self.symbols[*i]).collect())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
pub fn parsed_modules(&self) -> &BTreeMap<ModuleId, ParsedModule> {
|
||||
&self.parsed
|
||||
}
|
||||
|
||||
pub fn symbols(&self) -> &[SymbolDecl] {
|
||||
&self.symbols
|
||||
}
|
||||
|
||||
Loaded 100 of 169 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user