Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
58791fc7d9 | ||
|
|
1730f26fc7 | ||
|
|
4018e9928a | ||
|
|
628e3b6e5d | ||
|
|
5c6c612edd | ||
|
|
41587f319e | ||
|
|
298fbd51ea | ||
|
|
265dfdbcc4 | ||
|
|
42a43f5ffd | ||
|
|
037a963547 | ||
|
|
a32638e935 | ||
|
|
43eebe5671 | ||
|
|
e833dc05e2 | ||
|
|
11bb7ef277 | ||
|
|
6bd90bb2d0 | ||
|
|
4683f2e067 | ||
|
|
5912164039 | ||
|
|
303c06f773 | ||
|
|
a1086fd6c3 | ||
|
|
6ffe5ceb19 | ||
|
|
6b568aff3d | ||
|
|
16f26d7d60 | ||
|
|
b8265a92aa | ||
|
|
9af929971c | ||
|
|
2c4fa4bb05 | ||
|
|
3cabe53da2 | ||
|
|
58e13ee6d3 | ||
|
|
1b11fe063a | ||
|
|
b792f44556 | ||
|
|
9ea5bebfc3 | ||
|
|
c7c329341b | ||
|
|
8cf5288481 | ||
|
|
69c4e74872 | ||
|
|
af0bda2e40 | ||
|
|
090b026547 | ||
|
|
03396afd02 | ||
|
|
663a8ba2c0 | ||
|
|
dc853fb580 | ||
|
|
0b80ff8e50 | ||
|
|
b8a4ddd38f | ||
|
|
352bc0d2c5 | ||
|
|
8a6d09d582 | ||
|
|
dcd3d329d4 | ||
|
|
0e1bfb3359 | ||
|
|
e9ae7ff1ae | ||
|
|
ecbdd18303 | ||
|
|
bedd53e212 | ||
|
|
7674ebb869 | ||
|
|
434be9fd52 |
No files matched your search
@@ -28,6 +28,7 @@ Canonical release playbook for **cflmy/marqdo**. Read this skill **before** tagg
|
||||
9. **Extension packs always publish to both** Cloudflare R2 (`https://ext.marqdo.com` via `scripts/upload-ext-r2.py`) **and** GitHub (Release assets / `ext-v*` tag). Credentials only in `~/.marqdo/r2.env` / CI — **never** commit. See [ext-cdn.md](../../../doc/design/ext-cdn.md).
|
||||
10. **CLI Ubuntu package = PPA** (`ppa:cflmy/marqdo`) via `./scripts/ppa-ship.sh` / `ppa-build-source.sh`. GitHub zips remain for Windows and non-apt Linux.
|
||||
11. **When a step needs the user** (sudo password, GPG passphrase, Launchpad UI, interactive `dput`): **stop, state what failed, and give the exact commands** to run in their terminal. Do not pretend the step succeeded.
|
||||
12. **GitHub auto-build must be green before declaring a release done.** For full/cli: wait for the tag-triggered **Release** workflow (`windows` + `linux`) **and** the push-triggered **`executable-knowledge`** workflow on the release commit / `main` tip. Release asset zips alone are **not** sufficient — a green Release job with a red EK gate is an incomplete release. Do not skip or reframe a red `executable-knowledge` as “assets already uploaded.”
|
||||
|
||||
## Release modes
|
||||
|
||||
@@ -78,6 +79,7 @@ Release progress:
|
||||
```
|
||||
|
||||
Checks: clean tree; `main` ff-only; smoke tests; CHANGELOG Unreleased; no secrets.
|
||||
Also confirm the latest `main` **`executable-knowledge`** Actions run is green (or that this release commit will be covered after push) — do not proceed to tag if EK is already red on tip without a fix in the same release PR.
|
||||
|
||||
## Phase 2 — Docs & version bumps (main)
|
||||
|
||||
@@ -129,7 +131,9 @@ git push https://proxy.cflmy.top/github.com/cflmy/marqdo.git vVER
|
||||
|
||||
## Phase 5 — GitHub Release notes & assets
|
||||
|
||||
1. Wait for Actions (full/cli): both `windows` and `linux` green when ext natives required.
|
||||
1. Wait for Actions (full/cli):
|
||||
- Tag **Release** workflow: both `windows` and `linux` green when ext natives required.
|
||||
- Push **`executable-knowledge`** workflow on the release commit / `main`: **must be success**. Poll with `gh run list --workflow=executable-knowledge.yml --branch main` / `gh run watch`. If EK is red → **stop**; fix plugin/harness on `main`, re-run; do **not** declare the release complete from assets alone.
|
||||
2. Rewrite release body 中英双语 — [reference.md § Notes](reference.md).
|
||||
3. Confirm Linux `marqdo-VER-native-x86_64-unknown-linux-gnu.zip` when mode ships ext.
|
||||
|
||||
@@ -174,10 +178,11 @@ On Launchpad failure: read the build log, fix `debian/` or bundle script, bump `
|
||||
## Phase 8 — Post-release
|
||||
|
||||
1. `gh release view TAG` — assets + notes.
|
||||
2. CDN `https://ext.marqdo.com/vVER/` (if ext shipped).
|
||||
3. PPA build green (if cli shipped).
|
||||
4. Tell user URLs: GitHub release, CDN, PPA install one-liner.
|
||||
5. Optional gh-pages only if asked.
|
||||
2. Confirm **`executable-knowledge`** and **Release** Actions are both green for this tag/commit; if either is red, say so and do not claim “发版完成”.
|
||||
3. CDN `https://ext.marqdo.com/vVER/` (if ext shipped).
|
||||
4. PPA build green (if cli shipped).
|
||||
5. Tell user URLs: GitHub release, CDN, PPA install one-liner.
|
||||
6. Optional gh-pages only if asked.
|
||||
|
||||
## Extension-only release (mode=ext)
|
||||
|
||||
@@ -194,7 +199,8 @@ On Launchpad failure: read the build log, fix `debian/` or bundle script, bump `
|
||||
| `gh` / git TLS or Clash `7890` refused | Unset bad proxy; see [reference § Proxy](reference.md) |
|
||||
| GitHub blocked | proxy.cflmy.top, then HK jump; **print commands** if agent cannot push |
|
||||
| Tag exists | Stop; ask user |
|
||||
| CI red | Fix on main; retag only if user allows |
|
||||
| CI red (Release assets) | Fix on main; retag only if user allows |
|
||||
| `executable-knowledge` red | Fix agent plugin / test harness (e.g. `ensure_agent_plugin_built`); push `main`; wait for EK green. **Do not** treat uploaded Release zips as a successful release |
|
||||
| R2 upload fails | Print `upload-ext-r2.py` command; do not skip silently |
|
||||
| PPA / sudo / GPG | Print `./scripts/ppa-ship.sh noble` (or `dput …`); do not skip silently |
|
||||
| Launchpad `Cargo.lock` v4 / old cargo | Ensure bundled `third_party/rust` via current `ppa-build-source.sh`; bump `1ppaN` |
|
||||
|
||||
@@ -304,6 +304,15 @@ gh workflow run Release --ref "vVER" # only if workflow_dispatch exists; else f
|
||||
|
||||
Current Release workflow is **tag-push only**. Red CI after tag → fix on `main`, then either new patch tag or **user-approved** delete+recreate tag (dangerous; prefer patch).
|
||||
|
||||
**Also required before declaring release done:** the push-triggered **`executable-knowledge`** workflow on the release commit / `main` tip must be **success**. Poll with:
|
||||
|
||||
```bash
|
||||
gh run list --workflow=executable-knowledge.yml --branch main --limit 5
|
||||
gh run watch <run-id>
|
||||
```
|
||||
|
||||
A green Release asset job with a red `executable-knowledge` gate is an **incomplete** release — fix the plugin/harness (e.g. `ensure_agent_plugin_built` for agent-dependent EK tests), push `main`, and wait for EK green. Do not skip.
|
||||
|
||||
### HK SSH jump (Paramiko CONNECT)
|
||||
|
||||
**When to use:** local → GitHub fails (timeout / TLS / Clash flaky / reverse-proxy `proxy.cflmy.top` connect drops), but this host can reach an overseas box that has stable GitHub (default `hk.cflmy.de`). Proven path for v0.3.9 push/tag/`gh`.
|
||||
|
||||
@@ -14,65 +14,66 @@ description: >-
|
||||
|
||||
Marqdo = **Markdown markers as syntax**. A `.mq.md` file is both documentation and a runnable program. Read this skill **before** inventing syntax from Markdown habits or Python.
|
||||
|
||||
Canonical design (repo): `doc/design/markdown-mapping.md`, `doc/design/keywords.md`. Deeper tables: [reference.md](reference.md). Copy-paste patterns: [examples.md](examples.md).
|
||||
<!-- CORE_SYNTAX_V2.1: must stay at the top so skill_brief budgets never truncate it -->
|
||||
|
||||
**Authoring / library craft** (GFM-first, no json glue, how to develop Marqdo): use **[marqdo-dev](../marqdo-dev/SKILL.md)** whenever writing or refactoring `.mq.md`, `lib/*`, or `ext/*`.
|
||||
## CORE SYNTAX (v2.1 Occlusion — mandatory)
|
||||
|
||||
**Product release** (tag, CHANGELOG, README, public docs, VSIX, GitHub assets): use [marqdo-release](../marqdo-release/SKILL.md) — detect latest version, **ask the user for the next SemVer**, then follow that playbook. Do not invent a version number.
|
||||
These rules are the **current default**. Do **not** teach Classic `[key](map)` index or “bold is return of print”. Classic migration only: [migration-classic.md](migration-classic.md).
|
||||
|
||||
1. **File suffix** `.mq.md`. Print with `> print text=…` / `> 打印 内容=…` — never use `**…**` to print.
|
||||
2. **`**…**` = statement** (assign / call / expression). `*…*` = return. Closing markers touch the last token (no trailing space).
|
||||
3. **`[描述](payload)` = occlusion:** `[]` is prose (never executed); `()` is the only executed part. Empty `()` → `None` stub.
|
||||
4. **Read collections with `access(coll, key)`** (aliases `取` / `取值`). **Never** use `[key](map)` as index under default Occlusion.
|
||||
5. **Foreach:** `- [描述](遍历 coll 为 item)` (`遍历`/`for`, `为`/`as`/`in`). Prefer explicit traversal when generating new code.
|
||||
6. **Import / call:** frontmatter `import:` / `导入:`; call `lib.member`. One language (EN or ZH) per file.
|
||||
7. **Control flow:** `#` object/type, `##`+ function, `+` params, `1.`/`2.` branches, `-` loops, `N. *` else. No invented `if`/`for`/`def`/`return` keywords.
|
||||
8. **Prose vs code:** blank line separates comment paragraphs from executable lines.
|
||||
9. **GFM tables** for data; prefer `table.put` over `json.set` chains. `ext/**` never calls `host_*`.
|
||||
10. Body narrative `[描述](target)` is lazy (NarrativeRef) — reading does not execute side effects.
|
||||
|
||||
| Marker | Meaning (v2.1) |
|
||||
|--------|----------------|
|
||||
| `#` / `##`+ | Object/type · function/method |
|
||||
| `**…**` | Statement (assign / call / expression) |
|
||||
| `*…*` | Return |
|
||||
| `[描述](payload)` | Occlusion step (payload executes) |
|
||||
| `access(c, k)` | Collection read |
|
||||
| `> fn …` | Call |
|
||||
| GFM table | Collection literal |
|
||||
|
||||
Canonical: `doc/design/core-surface.md`, `doc/design/document-native-development-guide.md`. Deeper tables: [reference.md](reference.md). Patterns: [examples.md](examples.md). Authoring craft: [marqdo-dev](../marqdo-dev/SKILL.md). Release: [marqdo-release](../marqdo-release/SKILL.md).
|
||||
|
||||
Executable-knowledge gate: [`tests/executable_knowledge/`](../../tests/executable_knowledge/README.md).
|
||||
|
||||
## Hard rules (do not violate)
|
||||
|
||||
1. **File suffix** must be `.mq.md` (never plain `.md` for executable sources).
|
||||
2. **Output is not `**bold**`.** Print with `> print text=…` (or `> 打印 内容=…`). Bold `**…**` is **return value only**.
|
||||
3. **Prose vs code:** after a blank line, an unmarked first line starts a **comment paragraph**; every following non-blank line stays comment until the next blank line. Put a **blank line** between narration and executable lines.
|
||||
4. **Do not invent keywords** `if` / `else` / `while` / `for` / `def` / `return`. Control flow is Markdown: **`+` params** (under heading), **`1.` `2.` … branches**, **`-` loops**, arm `N. *` = else, `#` = **object/type**, `##`+ = **function/method**, `**…**` = return.
|
||||
5. **Identifiers use backticks** — params `` + `name` ``. **Exemptions:** foreach `` - [item](coll) ``; footnote `` name[^key] ``; **inside italic `*…*` / bold `**…**`**, bare ids are **variables** (Python-style unified namespace) and method-call receivers are bare too. **Drop the backticks on value-expression variable names and method receivers inside `*…*` / `**…**`** — they are redundant once the `*…*`/`**…**` markers mark the segment as code: `*分类 = 分类[^苹果]*`, `**n * 2**`, `*p = > page.主体装配 组件=home*`. **Text literals must be quoted**: `*a = > text.split value=src sep=","*`. Backticks **still required** where a bare word is *not* a variable in **standalone `>` calls** (bare = text there): `` > str `n` ``, `` > `obj`.method ``. **No trailing space** inside `*…*` / `**…**` wrapped code: the closing marker must touch the last token directly (`*a = 1*`, `**n**`), never `` *a = 1 * ``.
|
||||
6. **Structure lines are not wrapped in italics.** `#` `>` `+` `-` `|` lines stand alone; use `*…*` only for general statements (bindings / expressions).
|
||||
7. **Paths:** In bare *expressions* `/` is division. In **call args / param defaults / table cells**, unspaced `a/b` and quoted `".marqdo/agent-kb"` are path text — no `json.parse` needed. Numeric ratios that must stay text use quotes (`"1/5"`, `"16/9"`); bare `1/5` is division.
|
||||
8. Prefer ending side-effect-only function bodies with `*None*` / `*无*` / whole-line `**` / `****` when later lines must belong to an outer function. Lone `---` / `***` are Markdown thematic breaks (skipped), not function end.
|
||||
9. **`ext/**` never calls `host_*`.** Agent/OKF helpers are plugin names (`agent_kb_*`, …) after `plugin.load`. Do **not** add agent/OKF domain code to `src/host/` (core bloat). See `doc/design/ext-agent.md` §4.
|
||||
10. **Browser (route C/D/E/F):** client logic is Marqdo on WASM; official bridge is host glue and **may** implement lists, routing, storage, WebSocket, canvas, file read, observers — authors **must not hand-write business JS**. Use `web.client_embed` / `data-mq-source-url` auto-mount. Client effects: prefer **GFM tables + `lib/browser`** ([marqdo-dev](../marqdo-dev/SKILL.md)).
|
||||
11. **Code-as-documentation / no bag glue:** Prefer **GFM tables** for maps, lists, wire, commands. Prefer **`table.put` / named helpers** over `json.set` / `json.append` chains. `lib/json` is for parse/stringify/quote only — not a dict builder. Unreadable json pipelines are a style bug. Read collections with **`[key](map)`**, not footnote indexes for new code. In `markup: occlusion` files read with **`access(map, key)`** — there `[x](y)` is occlusion, not index. **Positive copy-paste shapes** (result map, soft-error pack, sparse `table.put`): [marqdo-dev](../marqdo-dev/SKILL.md) §2 · [examples.md](examples.md) §6e. Do **not** extend legacy `json.set` in `ext/ai` — refactor to tables.
|
||||
12. **One language per file:** Do not mix EN and ZH API names in the same `.mq.md` (imports, calls, params). Pick English *or* Chinese mirrors for the whole program.
|
||||
13. **Sandbox trial habit (Agent / generated `.mq.md`):** Prefer `agent.trial` / `run sandbox=True` / `sandbox.open`→`run`→`diff`→`commit` before promoting to the host tree. Host files must stay unchanged until a successful commit. Caps live in `sandbox.profile` GFM tables — do not invent a second dialect.
|
||||
1. Obey **CORE SYNTAX** above — Occlusion is default; do not emit Classic index links.
|
||||
2. **Identifiers:** params `` + `name` ``; inside `*…*` / `**…**` bare ids are variables. Quote text literals. Standalone `>` calls: backticks where bare would be text.
|
||||
3. **Structure lines** (`#` `>` `+` `-` `|`) are not wrapped in italics.
|
||||
4. **Paths:** bare `/` in expressions is division; in call args / table cells unspaced `a/b` is path text. Ratios as text need quotes (`"1/5"`).
|
||||
5. End side-effect-only helpers with `*None*` / `*空*` / `**` / `****` when the outer function continues. Lone `---` is a thematic break, not function end.
|
||||
6. **`ext/**` never calls `host_*`.** Prefer tables over `json.set` / `json.append`.
|
||||
7. **One language per file.** Sandbox trial before promoting Agent-generated trees.
|
||||
8. Browser: Marqdo on WASM + `lib/browser` tables; do not hand-write business JS.
|
||||
|
||||
|
||||
## Markup → meaning (v0.2)
|
||||
## Call one function
|
||||
|
||||
| Marker | Meaning |
|
||||
|--------|---------|
|
||||
| `#` | **Object / type** (constructor body). `# main` = entry object. `# Child = > Parent` = inherit (**no** implicit parent constructor; call `` `self` = > Parent … `` explicitly when needed). |
|
||||
| `##` … `######` | **Function / method** (nesting by heading depth). |
|
||||
| `` + `name` `` under heading | Parameter (optional `` `name`=default ``) |
|
||||
| `1.` `2.` … in body | Branch arm (condition or `N. *` else). **Same-indent restart at `1.` = new branch statement** (not more arms of the previous list). |
|
||||
| `- …` inside body | Loop (`while` or `` [item](coll) `` / `` [`item`](`coll`) ``) |
|
||||
| Line `N. *` | Else arm |
|
||||
| `> fn args` | Call (named `k=v` or positional) |
|
||||
| `` > `obj`.method args `` | Method call (`obj` must be a map with `_type`) |
|
||||
| Frontmatter nested `import:` / `导入:` map | Import file (`bind: path.mq.md`) or short name (`bind: lib.member`) |
|
||||
| `> lib.member …` / `> lib.Type.member …` | Call via bare dotted path (instance methods need `` `var`.m ``) |
|
||||
| `*…*` | Statement (bind / expr) — closing `*` touches last token, **no trailing space**; bare ids are variables |
|
||||
| `**…**` | Return value — closing `**` touches last token, **no trailing space**; bare ids are variables |
|
||||
| `****` or `**` + spaces + `**` | Return `None` and end function body |
|
||||
| Lone `---` / `***` | Markdown thematic break (skip; not function end) |
|
||||
| GFM table after empty RHS bind | Collection; get with `access(coll, "key")` (legacy `` `m`[^key] `` remains available) |
|
||||
| `- [item](遍历 coll 为 item)` | Foreach |
|
||||
| `` ```lang `` | Foreign code block (via `lib/foreign`) |
|
||||
| `` `"text"` `` | Quoted string (`\n` `\t` `\\` `\"`; `` `var` `` inside); bare tokens unescaped |
|
||||
| Unmarked prose | Comment |
|
||||
```markdown
|
||||
# 计算器
|
||||
|
||||
Entry: load file (often `index.mq.md`) → collect defs → run `# main`.
|
||||
## 加法
|
||||
|
||||
## Occlusion syntax (v2.1.0)
|
||||
计算两个数字`a`和`b`的和。
|
||||
|
||||
Occlusion is the default syntax. Shipped source declares `markup: occlusion` (or `标记: 遮蔽`) explicitly so imports stay deterministic. Legacy `[key](collection)` requires `markup: classic` or the one-cycle CLI flag `--legacy-markup`; `MARQDO_MARKUP` does not change parsing.
|
||||
*[计算结果](a + b)*
|
||||
```
|
||||
|
||||
- `[描述](实现)` — `[]` is prose (never executed); `()` is the **only** executed part.
|
||||
- `[顾客留存率](0.42)` · `[总价](access(价目表, "苹果") * 数量)` · `[功能开发中]()` → `None` (stub; `label.empty_payload` warning).
|
||||
- Read maps/lists with `access(集合, 键)` (aliases `取` / `取值`): `[第二项](access(列, 2))`.
|
||||
- Foreach: `- [篮子里的每个果子](遍历 篮子 为 果)` (accepts `遍历`/`for` and `为`/`as`/`in`).
|
||||
- Calls inside `()`: `加(1)` / `加(1, 2)` / `加一 号(31)` / `txt.str_trim(" x ")` — `)` terminates, commas optional.
|
||||
- Tooling: `marqdo labels pending` (stub ledger) · `marqdo labels --grep 词 --count` · `marqdo migrate labels . --mark --write` · `marqdo check` reports `label.missing` / `label.empty_payload` / `label.ref_dangling`.
|
||||
```
|
||||
marqdo call calculator.mq.md 计算器.加法 a=2 b=3
|
||||
```
|
||||
|
||||
prints `5`. The same command works on any `.mq.md`; the file does not need a special skill name.
|
||||
|
||||
## Minimal program
|
||||
|
||||
@@ -113,13 +114,13 @@ Chinese builtins (same functions, no import):
|
||||
## Statements, returns, branches
|
||||
|
||||
```markdown
|
||||
*x = 1*
|
||||
*y = x + 2*
|
||||
**x = 1**
|
||||
**y = x + 2**
|
||||
|
||||
## add_one
|
||||
+ `n`
|
||||
|
||||
**n + 1**
|
||||
*n + 1*
|
||||
```
|
||||
|
||||
```markdown
|
||||
@@ -147,9 +148,9 @@ import:
|
||||
|
||||
# main
|
||||
|
||||
*xs = > text.split value="a,b,c" sep=","*
|
||||
**xs = > text.split value="a,b,c" sep=","**
|
||||
> print text=`xs`
|
||||
*t = > clock.now_unix*
|
||||
**t = > clock.now_unix**
|
||||
```
|
||||
|
||||
- Import **English** or **Chinese** library file; call that file’s API names via `lib.member` (do not mix languages).
|
||||
@@ -208,8 +209,8 @@ Install: `marqdo ext add quantum` (ZH id: `量子`). Build: `cargo build --relea
|
||||
**Q8 themes (circuit / probs / bloch):** `theme="dark"|"light"|"bw"` (ZH `主题=`; **default `dark`** — slate lab look, gate-family colors, label chips so wires never cross `q0` text). View plot chrome follows `data-theme` on the SVG. Advanced kinds (hinton/…) still Q8c pending full re-skin.
|
||||
|
||||
```markdown
|
||||
*_ = > `qc`.draw kind="circuit" theme="dark"*
|
||||
*_ = > `qc`.draw kind="probs" theme="light"*
|
||||
**_ = > `qc`.draw kind="circuit" theme="dark"**
|
||||
**_ = > `qc`.draw kind="probs" theme="light"**
|
||||
```
|
||||
|
||||
Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7.md](../../doc/design/ext-quantum-q7.md) · Q8 viz: [ext-quantum-viz-style.md](../../doc/design/ext-quantum-viz-style.md) · examples: [quantum-bell](../../examples/quantum-bell/) · [quantum-entanglement](../../examples/quantum-entanglement/).
|
||||
@@ -240,8 +241,8 @@ Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7
|
||||
| `# main` then prose then code with no blank line | Blank line before code |
|
||||
| `if x > 0:` | `1. `x` > 0` |
|
||||
| `* > print text=hi *` | `> print text=hi` |
|
||||
| `` *`a` = 1 * `` (trailing space + backticks) | `*a = 1*` — bare bind, no trailing space before `*` / `**` |
|
||||
| `*xs = > text.split value=a,b sep=,*` (bare text args) | `*xs = > text.split value="a,b" sep=","*` — quote text literals; bare words are variables |
|
||||
| `` *`a` = 1 * `` (trailing space + backticks) | `**a = 1**` — bare bind, no trailing space before `*` / `**` |
|
||||
| `*xs = > text.split value=a,b sep=,*` (italic is return, and bare text args) | `**xs = > text.split value="a,b" sep=","**` — statement; quote text literals |
|
||||
| Forgetting empty return after nested `##` helper when outer continues | Add `*None*` / `**` / `****` (not `---`) |
|
||||
| Import `lib/text` then call bare `split` | `> text.split …` (qualified) |
|
||||
| Import `lib/text` then call `拆分` | Match file language (`text.split`, not 文本) |
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
name: executable-knowledge
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
gate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- name: Build agent + llm plugins (EK / workbench gates)
|
||||
run: |
|
||||
cargo build -p marqdo_plugin_agent --locked
|
||||
cargo build -p marqdo_plugin_llm --locked
|
||||
- name: Workbench contracts (M00–M06 / F-subset)
|
||||
run: cargo test --test workbench_contracts --locked
|
||||
- name: Offline executable-knowledge gate
|
||||
run: cargo test --test executable_knowledge --locked
|
||||
- name: Isolated execute MR-01 (dual 60s SSE + short reads)
|
||||
run: |
|
||||
cargo build -p marqdo_plugin_llm --locked
|
||||
bash scripts/build-web-plugin.sh
|
||||
cargo test --test isolated_execute_mr01 --locked -- --nocapture
|
||||
timeout-minutes: 5
|
||||
- name: no ignored tests counted as success
|
||||
run: |
|
||||
if grep -n '#\[ignore\]' tests/executable_knowledge.rs; then
|
||||
echo "executable_knowledge must not use #[ignore] as success evidence" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "ok: no #[ignore] in executable_knowledge gate"
|
||||
@@ -126,9 +126,21 @@ jobs:
|
||||
## Marqdo ${{ github.ref_name }}
|
||||
|
||||
### 亮点 / Highlights
|
||||
- **v2.1.0**:文件级 Occlusion 默认语义——全仓迁移 · `migrate labels --mark` · `access` / `遍历` · 可执行知识语义投影
|
||||
- **v2.1.0**: File-scoped Occlusion by default — corpus migration, `migrate labels --mark`, `access` / `for`, executable-knowledge projections
|
||||
- **v2.1.0 ext**:扩展包同步 2.1.0 / Ext pack 2.1.0
|
||||
- **v2.2.1**:view / 静态导出——修复 occlusion 与中等宽度下大纲侧栏掉到正文下方
|
||||
- **v2.2.1**: view / static export — fix outline rail nested under document (occlusion + mid-width layout)
|
||||
- **v2.2.1 ext**:扩展包同步 2.2.1 / Ext pack 2.2.1
|
||||
- **v2.2.0**:函数级调用 `marqdo call`——同一份 .mq.md 就是文档、知识和工具;Tree/Bytecode 执行被点名的函数;外部调用默认只跑纯函数
|
||||
- **v2.2.0**: `marqdo call` runs one function in any .mq.md — the file is the document, the knowledge and the tool; Tree and Bytecode execute that function; external calls stay pure
|
||||
- **v2.2.0 ext**:扩展包同步 2.2.0 / Ext pack 2.2.0
|
||||
- **v2.1.5**:Agent 可靠工作台——ResourceView / ToolSpec / CheckpointV2 / EffectLedger · EKC ir.json 暖机 · Web 短读槽隔离
|
||||
- **v2.1.5**: Agent reliability workbench — ResourceView / ToolSpec / CheckpointV2 / EffectLedger, EKC ir.json warm-load, web short/read lane isolation
|
||||
- **v2.1.5 ext**:扩展包同步 2.1.5 / Ext pack 2.1.5
|
||||
- **v2.1.4**:默认隔离 HTTP execute(ADR 0009)——worker 池 / `--call-lib-path` · `/_runtime/*` · MR-01 双流金样例 · `runtime.isolated_execution=1`
|
||||
- **v2.1.4**: Default isolated HTTP execute (ADR 0009) — worker pool / `--call-lib-path`, `/_runtime/*`, MR-01 dual-stream gold, `runtime.isolated_execution=1`
|
||||
- **v2.1.3**:知识工作台运行时——ModelSelection/RunContext/usage · view document 只读嵌入 · LLM selection/messages/cancel · agent selection e2e
|
||||
- **v2.1.3**: Workbench runtime — ModelSelection/RunContext/usage, view document read-only embed, LLM selection/messages/cancel, agent selection e2e
|
||||
- **v2.1.1**:可执行知识主链——BoundCall · NarrativeRef · 来源/适用期 · patch/extract · 整代 CURRENT + closure · 发票示例 113
|
||||
- **v2.1.1**: Executable-knowledge spine — BoundCall, NarrativeRef, sources, patch/extract, generation CURRENT + closure, invoice example 113
|
||||
- **v1.5.2**:外联 v3 P0/P1——raw stdin + 每 cell 输入 · plan API · snapshot · `view capture` · Persist
|
||||
- **v1.5.2**: Foreign v3 P0/P1 — raw stdin + per-cell inputs, plan API, snapshot export/import/replay, `view capture`, Persist
|
||||
- **v1.5.2 ext**:扩展包同步 1.5.2 / Ext pack 1.5.2
|
||||
|
||||
@@ -103,3 +103,4 @@ tests/ext/_sbx_cow/
|
||||
|
||||
tests/ext/_sbx_*/
|
||||
examples/**/_example_sbx/
|
||||
.secrets/
|
||||
+217
@@ -1,5 +1,149 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Added
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
|
||||
## v2.2.1 — 2026-10-30
|
||||
|
||||
### Highlights
|
||||
|
||||
**view / 静态导出大纲侧栏修复**:带遮蔽(occlusion)语句的文档页里,Functions / Variables 右侧栏会掉到正文下面;中等窗口宽度下也会被错误叠成单列。CLI · wasm · ext **2.2.1**。
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——布局与 HTML 闭合修复,不改 Markdown 标记。
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`marqdo view` / `view output`**:occlusion 行提前返回时未关闭 `.doc-block`,把 `#outline-rail` 嵌进 `document-surface`,侧栏排到文档下方;现已闭合并保留标注带。
|
||||
- **view 壳 CSS**:大纲折叠改为按主栏 `@container mq-panel` 宽度(32rem),不再用视口 `960px` 媒体查询把右侧栏压到正文下;极窄时大纲上移到文档上方。
|
||||
|
||||
## v2.2.0 — 2026-10-30
|
||||
|
||||
### Highlights
|
||||
|
||||
**函数级调用 `marqdo call`**:同一份 `.mq.md` 已经是文档、知识和程序。这一版补上从外部调用其中某个函数的入口。人继续读这一页;开发者、测试和 Agent 用同一条命令传入参数并执行。知识复用、把函数当工具、逐个函数做测试、改完再跑,都是这一入口的用法,不是另一套 Skill 系统。CLI · wasm · ext **2.2.0**。
|
||||
|
||||
```bash
|
||||
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3
|
||||
# 5
|
||||
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3 --json
|
||||
```
|
||||
|
||||
**作用**:最小可执行单位从「整份文件」扩展到「文件里的一个函数」。不必为了试一个函数再写 `# main`、打印和包装程序。
|
||||
|
||||
**价值**:说明和实现仍在同一页。Agent 读函数旁边的叙述,定位函数,传入参数,得到可检查的结果。正式发布和权限仍走已有的项目验证;`call` 默认不开放写文件、进程、网络和插件。
|
||||
|
||||
说明页:[public/features/22-函数调用.mq.md](public/features/22-函数调用.mq.md) · [public/features/22-call.mq.md](public/features/22-call.mq.md)
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——新增的是 CLI 入口,不新增 Markdown 标记。`**…**` 仍是语句,`*…*` 仍是返回。
|
||||
|
||||
### Added
|
||||
|
||||
- **`marqdo call <文件.mq.md> <函数> [名字=值 ...]`**:加载该文件及其 import,按显示路径、SymbolId 或唯一短名定位函数,用现有签名和契约绑定参数,再用现有运行时执行。
|
||||
- **`--json`**:成功 `{"ok":true,"result":...}`,失败 `{"ok":false,"error":...}`,错误带诊断 `code`。`--backend tree|bytecode` 与 `marqdo run` 相同;指定函数时两个后端都不会改去跑 `# main`。
|
||||
- **默认纯函数**:效应含写入、网络、进程或插件时,执行前返回 `artifact.effect_denied`,副作用不会发生。
|
||||
- **参照文档** `examples/calculator.mq.md`,以及用户说明页 `public/features/22-*`。
|
||||
|
||||
### Fixed
|
||||
|
||||
- 官方语法说明与 Agent 语法卡的标记极性与解析器对齐:粗体是语句,斜体是返回。同步测试会实际运行说明里的调用示例。
|
||||
- Bytecode 的 `marqdo run --call-lib-path` 执行被点名的导入函数,不再忽略该参数去跑入口。
|
||||
- 隔离运行时冒烟测试补上 `InvokeContext.fs_root`,`--tests` 可以编过。
|
||||
|
||||
### Changed
|
||||
|
||||
- CLI / wasm / `ext/VERSION` **同步 2.2.0**。扩展包版本号对齐;这一版的新能力在 CLI。
|
||||
|
||||
## v2.1.5 — 2026-10-27
|
||||
|
||||
### Highlights
|
||||
|
||||
**Agent 可靠工作台 + EKC 热路径**:ResourceView(JSON/CSV/PDF/image `blob_ref`、locator 页/行)· ToolSpec 授权先于调用 · CheckpointV2 / EffectLedger / ContextPackV2 · EKC `ir.json` 暖机与 `--refresh` · Web 短读/控制面与 execute 槽隔离。CLI · wasm · ext **2.1.5**。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.5 && cargo build --release
|
||||
marqdo ext add web # CDN / Releases → 2.1.5
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——工作台协议与 Agent 运行时合同,不新增 Markdown 标记。
|
||||
|
||||
### Added
|
||||
|
||||
- **ResourceView v1**:有界 JSON/CSV 渲染;PDF/image `blob_ref`;`locator.page` / 行定位;CSV `text` 供 TablePreview。
|
||||
- **Agent 合同**:ToolSpec 注册与 authorize-before-call;CheckpointV2 原子 resume;AgentErrorV1;输出通道;ContextPackV2;EffectLedger(含 mutating `run_tool` 水印)。
|
||||
- **Workbench fixtures/schemas**:`tests/workbench_contracts` 协议金样例。
|
||||
- **EKC `ir.json` 快照**:`marqdo knowledge` 写入 `.marqdo/ekc/graph/ir.json`;查询命令暖机热加载,免全量重编译。
|
||||
- **`--refresh`**:`knowledge preflight` / `find` / `reuse` / `impact` / `conflicts` / `stale` / `duplicate` / `verify` 可强制重编译。
|
||||
|
||||
### Fixed
|
||||
|
||||
- **工程预检热路径**:`knowledge preflight`(及 find/reuse 等)默认不再每次 `ensure_compiled` 全量 compile,修复聊天「工程预检」墙钟过长。
|
||||
- **Web 槽隔离**:`short`/`read`/`control` 不再填满 execute 槽;disconnect≠cancel。
|
||||
- **CSV ResourceView**:去掉尾部空行伪影。
|
||||
|
||||
### Changed
|
||||
|
||||
- 查询类 knowledge CLI:**缺投影才 compile**;CI / 源变更请先 `marqdo knowledge` 或 `--refresh`。
|
||||
- CLI / wasm / `ext/VERSION` **同步 2.1.5**。
|
||||
|
||||
## v2.1.4 — 2026-10-26
|
||||
|
||||
### Highlights
|
||||
|
||||
**默认隔离 HTTP execute(ADR 0009)**:`class=execute` / `sse-live` / `stream:true` 经 Supervisor worker 池(`marqdo run --call-lib-path`),父进程不再 `CallLibPath` 占 `INVOKE_LOCK`;短读 / 未声明非流式仍 `legacy_serial`(ADR 0008)。`/_runtime/*` 控制面;`runtime.isolated_execution=1`;MR-01 双 60s 假 SSE + 短读金样例。CLI · wasm · ext **2.1.4**。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.4 && cargo build --release
|
||||
marqdo ext add web # CDN / Releases → 2.1.4
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——隔离运行时与 Web execute 路径,不新增 Markdown 标记。
|
||||
|
||||
### Added
|
||||
|
||||
- **默认隔离 execute(ADR 0009)**:HTTP `execute` / `sse-live` / `stream:true` 经 Supervisor worker 池(`marqdo run --call-lib-path`),父进程不再 `CallLibPath` 占 `INVOKE_LOCK`;可选 route `class`;`/_runtime/*` 控制面;`runtime.isolated_execution=1`。
|
||||
- **MR-01 金样例**:`tests/isolated_execute_mr01.rs` + `tests/ext/isolated-execute-site/`(双 60s 假 SSE + 短读并行;槽满拒绝);接入 `executable-knowledge` workflow。
|
||||
- **Worker CLI**:`marqdo run --call-lib-path` / `--fs-root` / `--args-json` / `--emit-result`;host `runtime` admit/dispatch/cancel/events。
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
|
||||
- **compile 1000 门禁**:executable-knowledge / workbench 路径恢复与能力探测诚实(isolated 默认开启)。
|
||||
- CLI / wasm / `ext/VERSION` **同步 2.1.4**。
|
||||
|
||||
## v2.1.3 — 2026-10-25
|
||||
|
||||
### Highlights
|
||||
|
||||
**知识工作台运行时合同(M00–M06)**:`ModelSelectionV1` / `RunContextV1` / `ModelCallUsageV1` / `DocumentViewV1` / locator / snapshot;严格 selection;usage null≠0;`view document` 只读嵌入;LLM messages + cancel;agent.step selection 继承假模型 e2e;USTC 兼容 live smoke(env 注入)。CLI · wasm · ext **2.1.3**。设计证据仍为 Partial(原生适配器 unavailable;完整宿主 F12 留给应用)。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.3 && cargo build --release
|
||||
marqdo ext add llm # CDN / Releases → 2.1.3
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**(仍 19 个构造)——工作台合同与 View/LLM/Agent 扩展面,不新增 Markdown 标记。
|
||||
|
||||
### Added
|
||||
|
||||
- **Workbench M00–M06**:capabilities / selection / RunContext / usage / DocumentView / locator / SnapshotManifest;`tests/workbench_contracts`(含 F02 agent.step+resume、F14 并发);`marqdo view document --out` 只读 HTML 嵌入。
|
||||
- **LLM**:`messages=` / `selection=` / `strict_selection`;cancel token;reasoning_content 回退;双端点隔离门禁。
|
||||
- **Live**:`tests/workbench_live`(`OPENAI_*` env only;`MARQDO_EXT` 指向仓内 `ext/`)。
|
||||
- **Agent skeleton**:子工作簿禁 `llm.llm`,从 env selection `llm.create`。
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`llm.ask` 升参**:hoist `p` 避免缺参 `missing argument for parameter p`;避免 `~/.marqdo/ext` 陈旧包遮蔽仓内 `ext/`(测试强制 `MARQDO_EXT`)。
|
||||
|
||||
### Changed
|
||||
|
||||
- CLI / wasm / `ext/VERSION` **同步 2.1.3**。
|
||||
|
||||
## v2.0.0 — 2026-10-22
|
||||
|
||||
### Highlights
|
||||
@@ -69,6 +213,79 @@ marqdo ext add web
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Added
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
|
||||
## v2.1.2 — 2026-10-24
|
||||
|
||||
### Highlights
|
||||
|
||||
**可执行知识生产消费链 + View 阅读体验发版门槛**:证据摘要进闭包/句柄;`artifact.verify` / `knowledge adjudicate` 结构化裁决;`artifact.reuse_execute` 与 `agent_eng_bind_execute` / `publish_gate` 实际复用(非 n-gram);`document.extract` 文本/Markdown + PDF/OCR 显式失败;View `KnowledgePresentation`、说明锚点、SSE `page_token` 过期包拒绝;上传 `max_bytes`;门禁 45 项(含 100/1000 单元)。CLI · wasm · ext **2.1.2**。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.2 && cargo build --release
|
||||
marqdo ext add web
|
||||
cargo test --offline --locked --test executable_knowledge
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删关键词**——语义/MLSP/View/Agent 宿主扩展;不改变默认 Occlusion 链接读法与表几何。
|
||||
|
||||
### Added
|
||||
|
||||
- **证据与闭包(F0/M02)**:来源 digests 进入 artifact/evidence/closure;business handle 改/删来源失效;锚点/可读性诊断。
|
||||
- **验证裁决(F0/M03)**:`artifact.verify` 结构化 `may_bind` / `may_publish`;policy 枚举;`knowledge adjudicate`;CLI `--fail`。
|
||||
- **语法 Skill(F0/M05)**:CORE SYNTAX 置顶;`migration-classic.md`;`skill_core_card` 与预算截断防护。
|
||||
- **绑定执行(F1/M04)**:`artifact.reuse_execute`;`agent_eng_bind_execute` + `eng_publish_gate`(`candidate_only` / product gate);host_query `mlsp`;`tests/ext/agent-eng-bind-gate.mq.md`。
|
||||
- **发布与来源(F2/M06–M07)**:统一 `EXCLUDED_DIRS`;`publish_manifest.json`;`function_sources` 函数级来源覆盖。
|
||||
- **文档提取(F3/M01)**:`document.extract` DocumentSource→ExtractedDocument;PDF/DOCX/OCR 显式 `failed`;超限截断 `partial`;Web 上传 `MaxBytesReader` / `LimitReader`。
|
||||
- **View V0–V3(子集)**:正文优先;no-exec;`KnowledgePresentation`(`semantic_generation` / `page_token` / `local_deps`);SSE 过期包拒绝与帧选择保留;print/响应式;1000 单元 compile smoke。
|
||||
- **故事门禁(M08)**:建库→复用→来源变更失效→修复重绑。
|
||||
|
||||
### Fixed
|
||||
|
||||
- View 说明链接 raw-string 截断;skill_brief / CORE SYNTAX 预算不被冲突旧规则吃掉。
|
||||
- 上传体积限制在 multipart 解析前生效(不再只靠内存 form 预算)。
|
||||
|
||||
### Changed
|
||||
|
||||
- CLI / wasm / `ext/VERSION` **同步 2.1.2**。
|
||||
- `tests/executable_knowledge` 门禁扩至 45 项;README 切片进度表与发版子集对齐。
|
||||
|
||||
## v2.1.1 — 2026-10-23
|
||||
|
||||
### Highlights
|
||||
|
||||
**可执行知识主链落地(S0–S6)**:BoundCall 双后端共享解析、正文 NarrativeRef、来源/适用期、digest 保护补丁与纯表达式提取、整代 `CURRENT` 投影与闭包摘要;订单发票示例双后端输出 `113`;离线门禁含 100 单元语料与 CLI 冒烟。
|
||||
|
||||
```bash
|
||||
git checkout v2.1.1 && cargo build --release
|
||||
marqdo ext add web
|
||||
marqdo run examples/executable-knowledge/docs/invoice-answer.mq.md
|
||||
```
|
||||
|
||||
**核心表面 Δ**:**无增删关键词**——同模块 `Type.member` 纳入 BoundCall;语义投影新增 `bound_calls` / `dynamic_calls` / `hashes.closure` / NarrativeRef 步骤;不改变默认 Occlusion 链接读法。
|
||||
|
||||
### Added
|
||||
|
||||
- **BoundCall / DynamicCall**:`sema::bind::resolve_for_exec`;Tree 与 Bytecode 静态路径对齐;`bound_calls.json` / `dynamic_calls.json`;同模块定义树路径绑定。
|
||||
- **NarrativeRef**:注释行 Markdown 链接 → `narrate` / `navigation` 步骤;围栏与行内代码不计步骤。
|
||||
- **来源与适用性**:frontmatter `sources` / `valid_until` → `evidence=sourced|expired|missing_source`;`artifact.bind policy=business`。
|
||||
- **局部编辑与提取**:MLSP `artifact.patch`(digest + wording 回滚);`artifact.extract_pure` 纯表达式预览。
|
||||
- **整代与闭包**:`generations/<id>/` + `CURRENT`;`hashes.closure` 随依赖变化;锁报告 `classes`。
|
||||
- **门禁**:`tests/executable_knowledge`(含 100 单元);CI `.github/workflows/executable-knowledge.yml`;项目扫描 depth/size/count 上限。
|
||||
- **示例**:`examples/executable-knowledge` 发票含税链可运行;负知识文档。
|
||||
|
||||
### Fixed
|
||||
|
||||
- Bytecode 裸名解析不再跨 import scope 误连同名函数;同模块 `Type.member` 写入 path_index 供兄弟调用。
|
||||
|
||||
### Changed
|
||||
|
||||
- Agent / reuse 发现结果标记 `discovery_only` + `bind_required` / `bind_targets`,不可把打分结果直接当可执行目标。
|
||||
|
||||
## v1.5.2 — 2026-10-18
|
||||
|
||||
### Highlights
|
||||
|
||||
Generated
+17
-16
@@ -1,6 +1,6 @@
|
||||
# This file is automatically @generated by Cargo.
|
||||
# It is not intended for manual editing.
|
||||
version = 4
|
||||
version = 3
|
||||
|
||||
[[package]]
|
||||
name = "adler2"
|
||||
@@ -264,9 +264,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "crossbeam-utils"
|
||||
version = "0.8.21"
|
||||
version = "0.8.23"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28"
|
||||
checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
|
||||
|
||||
[[package]]
|
||||
name = "crypto-common"
|
||||
@@ -539,9 +539,9 @@ checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
|
||||
|
||||
[[package]]
|
||||
name = "js-sys"
|
||||
version = "0.3.104"
|
||||
version = "0.3.106"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0e0c1080212aad755ea003d18543e8768dd432c48819efd73a7bf1e39b7a5a3a"
|
||||
checksum = "7883d941dae510fb2d978fc3fe018c71c9e2892fd38854de3e8b92c2e5ad9cc5"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"futures-util",
|
||||
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
|
||||
|
||||
[[package]]
|
||||
name = "marqdo"
|
||||
version = "2.1.0"
|
||||
version = "2.2.1"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"assert_cmd",
|
||||
@@ -589,6 +589,7 @@ dependencies = [
|
||||
"predicates",
|
||||
"pulldown-cmark",
|
||||
"regex",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha1",
|
||||
"sha2",
|
||||
@@ -610,7 +611,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "marqdo-wasm"
|
||||
version = "2.1.0"
|
||||
version = "2.2.1"
|
||||
dependencies = [
|
||||
"marqdo",
|
||||
"serde_json",
|
||||
@@ -1290,9 +1291,9 @@ checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen"
|
||||
version = "0.2.127"
|
||||
version = "0.2.129"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1b70935747edd64d89de3efa29d73789b806c15798f8e7dca4d8ac356b50ce70"
|
||||
checksum = "9bb54f33acc68fd454578d9820b0bde1a1a3d17aa17bb7b6595806d02886d409"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"once_cell",
|
||||
@@ -1303,9 +1304,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-macro"
|
||||
version = "0.2.127"
|
||||
version = "0.2.129"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "77775f8f3f7217702089053b94958f8f54061a3f663417df76e19cbdcca29bc1"
|
||||
checksum = "2e29d0c35b16e224a7eeb5cd2d25e3e1968fbd65604117b44d3b789d00ee8535"
|
||||
dependencies = [
|
||||
"quote",
|
||||
"wasm-bindgen-macro-support",
|
||||
@@ -1313,22 +1314,22 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-macro-support"
|
||||
version = "0.2.127"
|
||||
version = "0.2.129"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e11d33f857dc2fb11b8bc75aee111aa9cbeb12cd9f25efd3d4c2a3dd4e235284"
|
||||
checksum = "6f501a8bc3719dba86ef8ae4728879c08001bea749eb1333ac5b91e040e2a6b7"
|
||||
dependencies = [
|
||||
"bumpalo",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.119",
|
||||
"syn 3.0.3",
|
||||
"wasm-bindgen-shared",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen-shared"
|
||||
version = "0.2.127"
|
||||
version = "0.2.129"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7ef64dbcc55df09c7e5a46182d181c2cfa3e925f3da937ea764728b4bbb9dcbf"
|
||||
checksum = "23f0c9c52aa7cd7d77769a4cfe2a9adb1b331f489a41d912ce14513d5ab995c6"
|
||||
dependencies = [
|
||||
"unicode-ident",
|
||||
]
|
||||
|
||||
+2
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "marqdo"
|
||||
version = "2.1.0"
|
||||
version = "2.2.1"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo — Markdown markers as an interpreted programming language"
|
||||
@@ -29,6 +29,7 @@ anyhow = "1"
|
||||
chrono = { version = "0.4", default-features = false, features = ["clock", "std"] }
|
||||
clap = { version = "=4.4.18", features = ["derive"], optional = true }
|
||||
pulldown-cmark = "0.12"
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
thiserror = "1"
|
||||
tiny_http = { version = "0.12", optional = true }
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,既是给人读的文稿,也是给机器跑的程序。写文档的过程就是在写可执行逻辑——大型项目里,说明、示例与实现不再分叉成三份真相。它不是「又一个 Markdown 编程语言」,而是**文档原生的编程运行时(Document-native runtime)**——一门关于**可执行知识(Executable Knowledge)**的语言。
|
||||
|
||||
v2.2.0 起,`marqdo call` 可以直接调用这份文件里的一个函数。读的是同一页说明,跑的是同一个实现。知识因此可以当工具用,测试也不必再包一层 `# main`。
|
||||
|
||||
欢迎访问 [marqdo 官方网站](https://www.marqdo.com/) 阅读更多特性与可执行文档。
|
||||
|
||||
**链接语法**:[doc/design/link-syntax.md](doc/design/link-syntax.md) · 语法宪法:[doc/design/markdown-mapping.md](doc/design/markdown-mapping.md) · 对象:[objects.md](doc/design/objects.md) · 调用实参:[call-arguments.md](doc/design/call-arguments.md) · 用户站:[user-site.md](doc/design/user-site.md) · 浏览:[view.md](doc/design/view.md) · 调试:[view-debug.md](doc/design/view-debug.md) · OKF 清单:[catalog-cli.md](doc/design/catalog-cli.md) · Engineering Knowledge:[engineering-knowledge.md](doc/design/engineering-knowledge.md) · VS Code 扩展:[vscode-extension.md](doc/design/vscode-extension.md)(分支 **`vscode-extension`**;提交约定 [vscode-extension-commit.md](doc/design/vscode-extension-commit.md)) · **AI Skill**:[skills/marqdo/](skills/marqdo/)(说明 [ai-skill.md](doc/design/ai-skill.md)) · 官方扩展:[ext-llm.md](doc/design/ext-llm.md)(`ext/llm`)· [ext-agent.md](doc/design/ext-agent.md)(智能体开发框架)· [ext-cli.md](doc/design/ext-cli.md)(`marqdo ext list/add/remove`) · 原生插件 ABI:[ext-abi.md](doc/design/ext-abi.md)(`lib/plugin`) · 金样例:[tests/](tests/) · 用户文档:[public/](public/) · 变更:[CHANGELOG.md](CHANGELOG.md)
|
||||
@@ -29,6 +31,7 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
|
||||
| 真正的核心优势 | 说明 |
|
||||
|------|------|
|
||||
| **知识-代码一致性**<br>Knowledge-Code Coherence | 说明、数据、示例与实现**同一份源文件**:文档里的表格就是程序的数据,文档里的调用就是真实执行。「文档过期」不再是悄悄漂移,而是可被机器发现的**知识不一致**——知识陈述本身可以被测试。 |
|
||||
| **函数即工具**<br>`marqdo call` | 最小可执行单位是文件中的函数,不只是整份文件。人阅读函数旁边的叙述;开发者、测试和 Agent 用 `marqdo call 文件 函数 名字=值` 执行它。知识复用、工具调用、函数级测试和局部改进是同一入口的四种用法。 |
|
||||
| **默认安全,AI 友好** | 无标记文字默认是叙述(注释),只有显式标记才进入执行。AI 自由发挥**不会污染**文档与代码的边界——这比「自然语言编程」安全得多,是 AI 原生语言的重要前提。 |
|
||||
| **multi-view artifact**<br>`.mq.md` 双后缀即开放接口 | `.mq` 告诉解释器「可执行」,`.md` 告诉整个 Markdown 生态「仍是文档」。GitHub 直接读、VS Code 直接改、搜索引擎直接索引、LLM 直接理解、Marqdo 直接运行。**Marqdo 不占有知识,只是给 Markdown 增加执行语义。** |
|
||||
| **表格即数据结构** | 程序的数据结构直接就是文档的数据结构:单列竖表 → `List`;横表(1 数据行)→ `Map`;横表(多数据行)→ `Map` of `List`(列向聚合);首列写 `@` / `行` / `row` → 行向记录 `List` of `Map`。取元使用 `access(集合, "键")`,链接 `[描述](表达式)` 只承载叙述与执行体。这是 literate programming 的下一步。 |
|
||||
@@ -281,9 +284,17 @@ marqdo catalog public -o .marqdo
|
||||
|
||||
---
|
||||
|
||||
## 现状(v2.1.0)
|
||||
## 现状(v2.2.1)
|
||||
|
||||
- **链接语法(v2.1.0)**:`[描述](目标)` 描述不执行 · `()` 与 `**代码**` 同权(`加(1)` / `加一 号(31)`)· `access` 取元与 `遍历` 链接循环 · `marqdo labels` 两级检索 / `--baseline` 债务棘轮 · `migrate labels --mark` · 诊断 `label.*` · view 描述正文 + 代码片 + 桩徽章 · **可执行知识 [link-syntax.md](doc/design/link-syntax.md)**(项目索引、LinkInfo、链接图与 lock) · 扩展包同步 **2.1.0**
|
||||
- **view 大纲侧栏(v2.2.1)**:修复 occlusion 文档与中等宽度下 Functions / Variables 掉到正文下方的问题;静态 `view output` 同源。
|
||||
- **函数级调用(v2.2.0)**:`marqdo call <文件.mq.md> <函数> 名字=值`。同一份文档给人读,也按函数执行;Tree 与 Bytecode 都跑被点名的函数,不回退到 `# main`。默认只执行纯函数。`--json` 给 Agent 读取结果或诊断。说明与价值见 [22-函数调用](public/features/22-函数调用.mq.md)。示例:`marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3` → `5`。扩展包同步 **2.2.1**
|
||||
- **Agent 可靠工作台(v2.1.5)**:ResourceView(JSON/CSV/PDF/image)· ToolSpec 授权先于调用 · CheckpointV2 / EffectLedger / ContextPackV2 · EKC `ir.json` 暖机与 `--refresh` · Web 短读/控制与 execute 槽隔离;扩展包同步 **2.1.5**
|
||||
- **默认隔离 HTTP execute(v2.1.4 · ADR 0009)**:`execute` / `sse-live` / `stream:true` → Supervisor worker 池(`--call-lib-path`);父进程不再占 `INVOKE_LOCK`;`/_runtime/*`;`runtime.isolated_execution=1`;MR-01 金样例
|
||||
- **知识工作台运行时(v2.1.3)**:ModelSelection / RunContext / usage null≠0 · `view document` 只读嵌入 · LLM selection/messages/cancel · agent.step selection e2e · live smoke(设计 Partial:原生适配器 unavailable)
|
||||
|
||||
- **可执行知识生产链(v2.1.2)**:证据/闭包失效 · `verify`/`adjudicate` · `reuse_execute` / `eng_bind_execute`+`publish_gate` · `document.extract` · View `KnowledgePresentation`+SSE `page_token` · 上传限流 · 门禁 45 项(100/1000 单元)
|
||||
- **可执行知识主链(v2.1.1)**:BoundCall 双后端 · NarrativeRef · 来源/适用期 · `artifact.patch` / `extract_pure` · 整代 `CURRENT` + closure · 发票示例 `113`
|
||||
- **链接语法(v2.1.0)**:`[描述](目标)` 描述不执行 · `()` 与 `**代码**` 同权(`加(1)` / `加一 号(31)`)· `access` 取元与 `遍历` 链接循环 · `marqdo labels` 两级检索 / `--baseline` 债务棘轮 · `migrate labels --mark` · 诊断 `label.*` · view 描述正文 + 代码片 + 桩徽章 · **可执行知识 [link-syntax.md](doc/design/link-syntax.md)**(项目索引、LinkInfo、链接图与 lock)
|
||||
- **外联 v3 P0/P1(v1.5.2)**:raw stdin + 每 cell 独立输入 · plan API · snapshot export/import/replay · `view capture` · Persist 按钮 · 扩展包同步 **1.5.2**
|
||||
- **view 静态/动态统一 + 外联值传递(v1.5.1)**:静态导出与 live view 共用 `foreign_doc` 执行路径 · 构建时运行注入 stdout · `foreign.session` 值传递 · cell stdin;扩展包同步 **1.5.0**
|
||||
- **裸自增升参(v1.4.6)**:首次粗体赋值左值即声明;`**n = n + 1**` 无反引号也升为必选形参。扩展包仍为 **1.4.5**
|
||||
@@ -319,7 +330,7 @@ marqdo catalog public -o .marqdo
|
||||
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
|
||||
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
|
||||
|
||||
### 如何使用最新 Marqdo(v2.1.0)
|
||||
### 如何使用最新 Marqdo(v2.2.1)
|
||||
|
||||
**安装解释器(任选其一)**
|
||||
|
||||
@@ -334,14 +345,16 @@ marqdo catalog public -o .marqdo
|
||||
```bash
|
||||
# 1) 源码安装(跟 tag 或 main)
|
||||
git clone https://github.com/cflmy/marqdo.git && cd marqdo
|
||||
git checkout v2.1.0 # 或留在 main
|
||||
git checkout v2.2.1 # 或留在 main
|
||||
cargo build --release
|
||||
export PATH="$PWD/target/release:$PATH"
|
||||
marqdo version
|
||||
marqdo version --check
|
||||
|
||||
# 2) 跑解释器 / 文档站 / catalog
|
||||
# 2) 跑整个文件,或只调用其中一个函数
|
||||
marqdo run tests/structure/hello.mq.md
|
||||
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3
|
||||
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3 --json
|
||||
marqdo run tests/structure/hello.mq.md --backend bytecode
|
||||
marqdo view public --no-open
|
||||
marqdo debug public --no-open
|
||||
@@ -373,7 +386,7 @@ marqdo wasm build
|
||||
# → dist/wasm/ … 见 examples/browser-hello/
|
||||
|
||||
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
|
||||
# https://github.com/cflmy/marqdo/releases/tag/v2.1.0
|
||||
# https://github.com/cflmy/marqdo/releases/tag/v2.2.1
|
||||
```
|
||||
|
||||
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "marqdo-wasm"
|
||||
version = "2.1.0"
|
||||
version = "2.2.1"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
|
||||
|
||||
Vendored
+60
@@ -1,3 +1,63 @@
|
||||
marqdo (2.2.1-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.2.1: fix view/static-export outline rail under document
|
||||
(occlusion doc-block close + panel container query layout).
|
||||
|
||||
-- Marqdo Maintainers <noreply@marqdo.com> Fri, 30 Oct 2026 11:45:00 +0800
|
||||
|
||||
marqdo (2.2.0-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.2.0: marqdo call runs one named function in any .mq.md.
|
||||
The same file stays the document, the knowledge and the program.
|
||||
Tree and bytecode both execute that function. External calls stay
|
||||
pure: write, process, network and plugin stay off by default.
|
||||
|
||||
-- Marqdo Maintainers <noreply@marqdo.com> Fri, 30 Oct 2026 09:20:00 +0800
|
||||
|
||||
marqdo (2.1.5-1ppa2) noble; urgency=medium
|
||||
|
||||
* Rebuild 2.1.5 against Launchpad (previous 1ppa1 failed with no build log).
|
||||
|
||||
-- Marqdo Maintainers <noreply@marqdo.com> Tue, 27 Oct 2026 09:35:49 +0000
|
||||
|
||||
marqdo (2.1.5-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.5: agent reliability workbench — ResourceView / ToolSpec /
|
||||
CheckpointV2 / EffectLedger; EKC ir.json warm-load; web short/read lanes.
|
||||
|
||||
-- Marqdo Maintainers <noreply@marqdo.com> Tue, 27 Oct 2026 07:39:08 +0000
|
||||
|
||||
marqdo (2.1.4-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.4: default isolated HTTP execute (ADR 0009) — Supervisor
|
||||
worker pool for execute/sse-live/stream, /_runtime/* control plane,
|
||||
MR-01 dual 60s SSE gold sample, runtime.isolated_execution=1.
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Mon, 26 Oct 2026 10:00:00 +0000
|
||||
|
||||
marqdo (2.1.3-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.3: workbench M00–M06 runtime contracts (ModelSelection,
|
||||
RunContext, DocumentView, usage null≠0, LLM selection/cancel,
|
||||
agent.step selection e2e, view document CLI, live smoke).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Sun, 25 Oct 2026 15:20:00 +0000
|
||||
|
||||
marqdo (2.1.2-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.2: executable-knowledge F0–F3 + View V0–V3 release bar
|
||||
(evidence/closure, verify gates, bind/reuse_execute, document.extract,
|
||||
KnowledgePresentation, SSE page_token, upload max_bytes).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Sat, 24 Oct 2026 14:50:00 +0000
|
||||
|
||||
marqdo (2.1.1-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.1: executable-knowledge spine (BoundCall, NarrativeRef,
|
||||
sources/applicability, generation CURRENT + closure digests).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Fri, 23 Oct 2026 15:40:00 +0000
|
||||
|
||||
marqdo (2.1.0-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 2.1.0: file-scoped Occlusion is the default syntax; migrated
|
||||
|
||||
+18
-3
@@ -13,12 +13,25 @@
|
||||
|
||||
## 当前文档
|
||||
|
||||
> **可执行知识下一阶段的规范优先级**:实现 `[]()`、Semantic Artifact、知识绑定、验证、锁定、EKC/Agent 复用时,以 [next/Marqdo_Executable_Knowledge_Development.md](next/Marqdo_Executable_Knowledge_Development.md) 为实施基线。与其冲突的早期链接、遮蔽、知识分层或 EKC 设计只作历史参考。
|
||||
> **可执行知识下一阶段的阅读顺序**:[原始意图 009](next/009.md) → [文档原生开发指导](design/document-native-development-guide.md)(作者体验、语法审查结论、AI 工作协议)→ [详细开发规范](next/Marqdo_Executable_Knowledge_Development.md)(目标语义与工程约束)→ [缺口与交付路线](roadmap/executable-knowledge-gaps-2026-10-23.md)(现状证据、S0–S6)。指导文档修订作者负担、场景政策和交付次序;不得把所有内部治理字段变成作者必填项。与之冲突的早期设计只作历史参考,设计目标不等于已实现能力。
|
||||
>
|
||||
> **可执行知识门禁**:[tests/executable_knowledge/](../tests/executable_knowledge/README.md) — S0–S2 BoundCall / 效应 / MLSP bind,以及后续 NarrativeRef、来源与整代投影子集;`cargo test --test executable_knowledge --locked`(CI:`.github/workflows/executable-knowledge.yml`)。`status: design` / 预期失败与已实现分开报告。
|
||||
|
||||
| 文档 | 摘要 |
|
||||
|------|------|
|
||||
| [design/markdown-mapping.md](design/markdown-mapping.md) | **v0.2 语法宪法(现行运行时)** |
|
||||
| [design/markdown-mapping-v0.3.md](design/markdown-mapping-v0.3.md) | **v0.3 草案:叙述统一 · 粗体代码 · 斜体返回 · 形参体推断** |
|
||||
| [roadmap/agent-reliability-crossrepo-2026-10-27.md](roadmap/agent-reliability-crossrepo-2026-10-27.md) | **Proposed:跨仓 Agent 可靠性开发计划**;M-A0–M-A6 ∥ Q-A0–Q-A7 排期、依赖图、F01–F20 映射、MA-T*/QA-* 验收与第一刀切片;协议真源仍见下方设计文档 |
|
||||
| [design/agent-reliability-tools-resources-2026-10-27.md](design/agent-reliability-tools-resources-2026-10-27.md) | **Proposed:智能体能力补足**;基于 `16f26d7`,增量设计错误恢复、检查点/效应账本、结构化工具、输出通道、上下文与只读资源视图;含跨仓边界、阶段计划和故障验收 |
|
||||
| [research/agent-capabilities-2026-10-27.md](research/agent-capabilities-2026-10-27.md) | **官方能力调研与源码复核**;OpenAI/Claude/Gemini/LangGraph/Deep Agents/MCP,求道当前问题 F01–F20;区分现有能力、设计推导与待实测项 |
|
||||
| [design/isolated-web-runtime-development-2026-10-26.md](design/isolated-web-runtime-development-2026-10-26.md) | **隔离执行与响应式 Web 运行时**:进程池、控制/读取分离、取消、事件恢复、发布一致性;实现见 `src/runtime/` |
|
||||
| [adr/0009-isolated-web-runtime.md](adr/0009-isolated-web-runtime.md) | **ADR 0009**:隔离 Web 运行时决策;金样例通过后替代 ADR 0008 的并行性结论 |
|
||||
| [跨库缺陷复核(2026-10-26)](../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md) | **最新源码与测试证据**:实际 HTTP 路由、知识发布/阅读断点、旧缺陷处置及 1000 单元编译性能失败;需同级 qdagent checkout |
|
||||
| [design/qdagent-workbench-runtime-and-view-2026-10-25.md](design/qdagent-workbench-runtime-and-view-2026-10-25.md) | **qdagent 知识工作台框架支持**:M00–M05 合同落地中(`src/workbench/`、MLSP `capabilities` / `view.document` / `view.reference_resolve`、`tests/workbench_contracts.rs`);完整 verified 以切片验收证据为准 |
|
||||
| [design/view-knowledge-reading-experience-2026-10-24.md](design/view-knowledge-reading-experience-2026-10-24.md) | **View 知识阅读体验重设计(建议)**:正文优先、结构导航、就地知识说明、来源与运行状态、响应式线框、实现契约与验收;含当前呈现和 no-exec 反例 |
|
||||
| [roadmap/qdagent-knowledge-support-gaps-2026-10-24.md](roadmap/qdagent-knowledge-support-gaps-2026-10-24.md) | **求道知识沉淀与文档编译支撑缺口**:2.1.1 源码复核、来源/依赖失效反例、验证与 Agent 协议、语法提示词、导入底座及跨库分工 |
|
||||
| [design/document-native-development-guide.md](design/document-native-development-guide.md) | **下一阶段开发指导**:语法审查裁决、正文→命名→复用工作流、AI 上下文与局部编辑、渐进严格、作者成本及 S0–S6 验收 |
|
||||
| [design/core-surface.md](design/core-surface.md) | **核心构造清单**:v2.1 默认链接/循环说明与构造守卫;不是全部实现状态清单 |
|
||||
| [design/markdown-mapping.md](design/markdown-mapping.md) | **历史 v0.2 映射**:不再代表现行默认链接语义 |
|
||||
| [design/markdown-mapping-v0.3.md](design/markdown-mapping-v0.3.md) | **v0.3 设计背景**:叙述统一 · 粗体代码 · 斜体返回 · 形参体推断;链接语义以 v2.1 规范和测试为准 |
|
||||
| [adr/0005-prose-unified-markup.md](adr/0005-prose-unified-markup.md) | **Proposed:采纳 mapping v0.3** |
|
||||
| [roadmap/markup-v0.3.md](roadmap/markup-v0.3.md) | **Markup v0.3 实现路线(M0–M5)** |
|
||||
| [design/objects.md](design/objects.md) | **`#` 对象 / `##` 函数与方法** |
|
||||
@@ -112,6 +125,8 @@
|
||||
| [design/executable-knowledge-system.md](design/executable-knowledge-system.md) | **可执行知识系统(v2.1 重构草案)**:每个 `[]()` 都是命名的可执行知识步骤 · 取元/推导/循环/调用/叙述统一建图 · 跨文件稳定寻址 · check/view/verify/impact 闭环 |
|
||||
| [next/Marqdo_Executable_Knowledge_Development.md](next/Marqdo_Executable_Knowledge_Development.md) | **可执行知识系统详细开发规范**:以 `next/009.md` 及两份附属文档为真实意图基线,冻结统一语义 IR、稳定寻址、绑定/验证/锁定、EKC/Agent 集成、迁移路线与验收矩阵 |
|
||||
| [next/Marqdo_Executable_Knowledge_Testing.md](next/Marqdo_Executable_Knowledge_Testing.md) | **可执行知识语法测试与兼容性规程**:标准库、扩展库、CLI、Tree/Bytecode 的仓库级门禁 |
|
||||
| [roadmap/executable-knowledge-gaps-2026-10-23.md](roadmap/executable-knowledge-gaps-2026-10-23.md) | **v2.1 现状与规划缺口审计(方向修订)**:保留 G01–G12 代码证据,以可读步骤、确定执行、早期 AI 复用和局部修改重排 S0–S6,严格治理按场景增加 |
|
||||
| [../tests/executable_knowledge/README.md](../tests/executable_knowledge/README.md) | **S0 作者基线门禁**:六类任务语料、作者成本计数、预期失败与 Bytecode 同名误连缺陷指纹 |
|
||||
| [design/knowledge-layers.md](design/knowledge-layers.md) | **知识分层(语法评审)**:原始语料 / 本体 / 可执行知识 / 调用面 · 单体知识类 vs 运行时实例 · `# 李白 = > 诗人` · 出处/还原验证 · EKC 映射 |
|
||||
| [design/occlusion-description-and-search.md](design/occlusion-description-and-search.md) | **遮蔽描述与两级检索**:内置词表=范例/默认,参数=扩展 · 空载荷一律 warning · 申报习惯 |
|
||||
| [design/occlusion-syntax.md](design/occlusion-syntax.md) | **遮蔽语法总纲(实施依据)**:`[描述](实现)` 的目的/语义/新鲜度校验 + M0–M8 计划 |
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
# ADR 0008:Web 热路径单解释器串行化(设计缺陷与缓解)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | **Accepted**(缺陷已记录;缓解已落地;真并行见 [ADR 0009](0009-isolated-web-runtime.md)) |
|
||||
| Date | 2026-10-26 |
|
||||
| Related | [web-invoke-serialization.md](../design/web-invoke-serialization.md) · [ADR 0009](0009-isolated-web-runtime.md) · ADR [0004](0004-web-plugin-go.md) · ADR [0007](0007-web-document-endpoint.md) · [ext-abi.md](../design/ext-abi.md) · [ext-web-go-rewrite.md](../design/ext-web-go-rewrite.md) |
|
||||
|
||||
## Context
|
||||
|
||||
- Go `plugins/web` 的 HTTP 服务是**多 goroutine** 的:每个请求可并发进入 `handleInvoke` / `handleInvokeLiveSSE`。
|
||||
- 每个 Endpoint 最终经 C ABI `host_query("call_lib_path", …)` 回到 Rust 宿主,在**同一个** `HostContext`(Marqdo 解释器状态)上执行作者 `.mq.md`。
|
||||
- `HostContext` **不是** `Sync`:共享可变会话、库装载、写回、事件总线等状态不能安全地被多线程同时求值。
|
||||
- 因此宿主用进程级 `INVOKE_LOCK`(`src/host/plugin.rs`)串行化所有来自 worker 线程的 `call_lib_path`。
|
||||
|
||||
### 常见误解
|
||||
|
||||
> 「web 都是单线程的吗?不能多线程吗?」
|
||||
|
||||
| 层 | 并发模型 |
|
||||
|----|----------|
|
||||
| Go HTTP / SSE flush / 取消 | **多线程(多 goroutine)** |
|
||||
| Marqdo 解释器 `call_lib_path` | **单飞(一把全局锁)** |
|
||||
| 浏览器 UI | 与后端锁无关;但若短 API 同步阻塞等待锁,会表现为「整页卡死」 |
|
||||
|
||||
**缺陷不在「HTTP 单线程」,而在「多 HTTP 工作线程共享一个非 Sync 解释器」。**
|
||||
真并行需要多解释器实例(或可共享的隔离运行时),不是去掉 Go 侧并发。
|
||||
|
||||
### 事故形态(已观测)
|
||||
|
||||
长对话(`return=sse-live` / `stream:true`)在 `CallLibPath` 期间一直持有 `INVOKE_LOCK`。并发的短路由(如 `/v1/models`、设置页探测)在旧实现里对锁 **无限期** `lock()`,HTTP 挂起 → 前端 `fetch` 无超时 → 页面无响应。重启进程可恢复,但属于运维掩盖,不是正确错误处理。
|
||||
|
||||
## Decision
|
||||
|
||||
1. **承认并文档化**本限制为宿主架构缺陷(单 `HostContext` + 全局 `INVOKE_LOCK`),不得再被表述为「web 插件只能单线程」。
|
||||
2. **缓解(已实现,非根治)**:
|
||||
- `call_lib_path` 从 worker 进入时用 **限时 try-lock**(默认 `MARQDO_INVOKE_WAIT_MS=1500`),超时返回错误串 `interpreter busy: …`。
|
||||
- Go `handleInvoke` 将含 `interpreter busy` 的错误映射为 **HTTP 503**,并设 `Retry-After: 2`。
|
||||
- **禁止**在 SSE 期间把进程级 `MARQDO_INVOKE_WAIT_MS` 拉长:那会让并发短请求重新卡死。
|
||||
- 长 SSE 在启动时用同一短预算争锁;一旦拿到则持有至调用结束(对话本来就要跑完)。
|
||||
3. **客户端契约**:短 API 必须有超时与 503 文案;不得假设「任意时刻都能同步进入解释器」。
|
||||
4. **根治(未决,另案)**:解释器池 / per-request isolate / 可 `Send` 的会话切片。未落地前,产品行为以本 ADR 的 503 语义为准。
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- 对话进行中,设置/模型列表等短请求约 1.5s 内失败并可读,UI 不再无限挂起。
|
||||
- 缺陷有名有姓,后续多解释器工作不会被「修好了」的假象挡住。
|
||||
|
||||
### Negative / Risks
|
||||
|
||||
- 同一进程仍无法真正并行跑两个 Marqdo Endpoint 体;第二个长 SSE 也会在短预算后失败。
|
||||
- 极短窗口内,短请求若正好撞上锁交接,可能偶发 503(可重试)。
|
||||
- `MARQDO_INVOKE_WAIT_MS` 是进程环境变量;误设过大仍会复现卡顿。
|
||||
|
||||
### Neutral
|
||||
|
||||
- Go 多 goroutine、SSE 桥(`MARQDO_EVENT_FORWARD`)、取消令牌(`MARQDO_CANCEL_TOKEN`)保持不变。
|
||||
- 不改变 Document / Endpoint 作者面(ADR 0007);只约束宿主与 web 插件错误语义。
|
||||
|
||||
## Implementation pointers
|
||||
|
||||
| 位置 | 职责 |
|
||||
|------|------|
|
||||
| `src/host/plugin.rs` → `acquire_invoke_lock` | 限时争锁;失败 `set_err("interpreter busy…")` |
|
||||
| `plugins/web/.../routes_invoke.go` → `handleInvoke` | `interpreter busy` → 503 + `Retry-After` |
|
||||
| 设计长文 | [web-invoke-serialization.md](../design/web-invoke-serialization.md) |
|
||||
@@ -0,0 +1,33 @@
|
||||
# ADR 0009: Isolated Web Runtime (worker process pool)
|
||||
|
||||
| 项目 | 约定 |
|
||||
|---|---|
|
||||
| 状态 | **Accepted**;HTTP **execute** 路径正式替代 [ADR 0008](0008-web-invoke-serialization.md) 的并行性结论。`legacy_serial`(未声明且非流式)仍引用 0008 单飞 / 父进程 `CallLibPath`。 |
|
||||
| 日期 | 2026-10-26(修订 2026-10-27:disconnect≠cancel;control/read/short 无父锁) |
|
||||
| 设计 | [isolated-web-runtime-development-2026-10-26.md](../design/isolated-web-runtime-development-2026-10-26.md) |
|
||||
| 金样例 | `tests/isolated_execute_mr01.rs`(MR-01 双 60s SSE + short reads;同文件 MR-02 槽满拒绝;disconnect-keeps-running;control-under-dual-execute) |
|
||||
|
||||
## 背景
|
||||
|
||||
ADR 0008 确认:Go HTTP 多线程,但所有 Endpoint 共享一把解释器锁与进程级 SSE 桥环境变量。1500ms busy → 503 只是缓解,不是并行。
|
||||
|
||||
## 决策
|
||||
|
||||
1. **首选架构**:预热的一次性 Marqdo worker 进程池;每个租约独占一个进程与 HostContext。
|
||||
2. **HTTP 默认**:`class=execute` 或 `return=sse-live` / body `stream:true` → `admit` → `dispatch`(`--call-lib-path` worker);**禁止**父进程 `CallLibPath` / `Setenv` / `legacySSEBridgeMu`。声明的 `short`/`read`/`control` 走父进程短路径(execute 已离进程,父锁通常空闲);未声明非流式 → `legacy_serial`。
|
||||
3. **控制/读取分离**:`/_runtime/*`(cancel、events、task get)不进入解释器锁。短读勿挤占 execute worker 槽(池默认 2);产品 cancel 优先 `/_runtime`。
|
||||
4. **订阅 ≠ 取消**:`sse-live` 观察者 `ctx.Done()` 默认**只结束订阅**,不 `runtime.cancel`;后台仍 reap worker 以释放槽。仅显式 header `X-Marqdo-Cancel-On-Disconnect: 1` 才 disconnect=cancel。同步 `waitIsolatedJSON` 可保留 disconnect 取消。重连:`GET /_runtime/tasks/{id}/events` + `Last-Event-ID` / `after_seq`。
|
||||
5. **协议**:`InvokeRequestV2` / worker env + NDJSON `MARQDO_EVENT_FORWARD` + `--emit-result`(首期);`/_runtime/*` 控制面。
|
||||
6. **能力探测**:`runtime.isolated_execution=1`、`runtime.task_control_v2`、`runtime.event_replay`、`view.analysis_cache_v2`、`runtime.read_lane=1`(短读不与 execute 抢父锁)。
|
||||
7. **发布**:`expected_generation` CAS + publish journal;逻辑文档 ID 与物理 generation 路径解耦。
|
||||
|
||||
## 后果
|
||||
|
||||
- Execute / live SSE 不再抢进程级 SSE 桥;双流可并行(MR-01)。
|
||||
- 短读走父进程:在 dual execute 下通常仍 200(父锁空闲);勿把 short/read/control 塞进 2 槽 execute 池。
|
||||
- Legacy 未声明非流式路径仍可走父进程锁(ADR 0008)。
|
||||
- 产品(qdagent)`turn_events` / Last-Event-ID / cancel CAS 接线由 qdagent 验收;框架侧 disconnect≠cancel 与 `/_runtime` 无锁控制面已落地。
|
||||
|
||||
## 验收(摘要)
|
||||
|
||||
MR-01 绿:双假上游 60s 流不混字、短读全 200、两 worker `--call-lib-path`、第三路槽满 429/503、`runtime.isolated_execution=1`、`runtime.read_lane=1`。观察者断连后任务仍跑完并可经 `/_runtime` 重放。其余见设计文档 §12。
|
||||
@@ -0,0 +1,348 @@
|
||||
# Marqdo 智能体能力补足:恢复、工具、上下文与资源视图
|
||||
|
||||
| 项目 | 约定 |
|
||||
|---|---|
|
||||
| 状态 | **Proposed**;本次交付设计,不代表以下新接口已实现 |
|
||||
| 日期 / 基线 | 2026-10-27(Asia/Shanghai);Marqdo `16f26d7` / 2.1.4;qdagent `68b757e` |
|
||||
| 调研与现状 | [官方资料与源码证据 F01–F20](../research/agent-capabilities-2026-10-27.md) |
|
||||
| 产品配套 | [qdagent 智能体能力与预览设计](../../../qdagent/doc/design/agent-capabilities-and-preview-2026-10-27.md)(同级 checkout) |
|
||||
| 继承 | [ext/agent](ext-agent.md)、[隔离 Web 运行时](isolated-web-runtime-development-2026-10-26.md)、[工作台合同](qdagent-workbench-runtime-and-view-2026-10-25.md) |
|
||||
|
||||
## 1. 目标与架构边界
|
||||
|
||||
让同一套文档原生 Agent 在模型错误、工具失败、网络中断后有界恢复;让工具可发现、可验证、可审计;让结果同时服务模型上下文与用户阅读。保留 `.mq.md` 工作簿、EKC 与验证证据的权威地位,不把框架改成第二套产品聊天服务。
|
||||
|
||||
现有 `src/runtime/` 已实现隔离进程、任务/事件与控制通道,本设计在其上补 Agent 步骤语义。现有 `ext/ai/agent.mq.md` 已有 step/plan/run、CALL/READ、观察压缩,继续扩展。框架不持有求道“笔记”业务表、中文按钮、默认回答字数或具体供应商账户。
|
||||
|
||||
| 议题 | Marqdo 所有 | qdagent 所有 |
|
||||
|---|---|---|
|
||||
| 恢复 | 错误分类、预算、检查点、工具效应、运行/事件语义 | 恢复按钮、会话关联、凭证修复入口、产品政策 |
|
||||
| 工具 | ToolSpec/Call/Result、schema、权限执行、并发、MCP 传输 | 业务工具、默认集合、连接配置、用户授权范围 |
|
||||
| 回答 | 结果分通道、输出预算、角色消息、finish reason | 简洁/标准/详细策略、摘要与产物卡、ASK/BUILD 行为 |
|
||||
| 资源 | ResourceRef、只读内容/派生预览、定位、转换插件接口 | 笔记/文件/产物 ID 映射、owner 权限、预览组件和布局 |
|
||||
| 知识 | preflight/reuse/verify/compile、记忆原语 | 何时提议沉淀、用户画像与可见管理 |
|
||||
| 调试 | span、结构化事件、用量与错误码 | 可读过程、运行详情、产品评测语料 |
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
APP[qdagent 产品策略与授权] --> RUN[ext/ai/agent step / plan / run]
|
||||
RUN --> CTX[ContextPack 与预算]
|
||||
RUN --> LLM[LLM Adapter]
|
||||
RUN --> DISPATCH[Tool Dispatcher]
|
||||
DISPATCH --> LOCAL[受控 Marqdo 函数]
|
||||
DISPATCH --> MCP[MCP Client]
|
||||
DISPATCH --> RES[资源读取 / 转换]
|
||||
RUN --> REC[Checkpoint / Effect Ledger]
|
||||
REC --> RT[既有 runtime task store / event log]
|
||||
RUN --> OUTPUT[answer / artifact / progress / diagnostic]
|
||||
RES --> VIEW[DocumentView / ResourceView]
|
||||
OUTPUT --> APP
|
||||
VIEW --> APP
|
||||
```
|
||||
|
||||
## 2. 现状与增量
|
||||
|
||||
| 已有能力 | 本轮不能误判为缺失 | 要补的具体语义 |
|
||||
|---|---|---|
|
||||
| runtime.task_control_v2/event_replay/read_lane | 不再新建一套队列与 SSE 框架 | Agent 步骤恢复与传输重放的边界 |
|
||||
| resume_save/load | 不再新建平行 checkpoint 命名空间 | 原子提交、版本摘要、作用域、效应关联 |
|
||||
| CALL、ARGS、READ、plan patch | 不废弃文档式编排 | 类型化 schema、校验错误、有限修复、稳定 call_id |
|
||||
| MCP stdio client | 不从零重写连接器 | timeout/cancel、resource、远程 transport 的能力探测 |
|
||||
| DocumentViewV1 / Locator | 不另写 `.mq.md` 解析器 | 普通资源类型、明确 renderer、历史与相对资源处理 |
|
||||
| LLM messages/usage | 不把 provider 行为交给产品猜测 | 输出预算、原生工具适配探测、角色边界与分通道 |
|
||||
|
||||
新模块路径及协议名以下均是**建议落点**。首批开发先固化 schema 与 fixture,再开放 feature bit;不得只更新 capabilities 把未实现能力标真。
|
||||
|
||||
## 3. 错误和恢复合同
|
||||
|
||||
### 3.1 ErrorV1
|
||||
|
||||
现有字符串错误保留兼容适配,新工具与 Agent 结果返回:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "AgentErrorV1",
|
||||
"code": "tool.timeout",
|
||||
"category": "transient",
|
||||
"stage": "tool",
|
||||
"message": "读取资源超时",
|
||||
"retryable": true,
|
||||
"retry_after_ms": 1200,
|
||||
"effect_state": "none",
|
||||
"recovery_action": "retry_step",
|
||||
"call_id": "call_01",
|
||||
"details_ref": "resource:diagnostic_01"
|
||||
}
|
||||
```
|
||||
|
||||
`retryable` 仅是候选信息,最终动作由可信策略结合效应、截止时间、预算、取消状态决定。远端工具声称 readOnly/idempotent 不构成授权依据。凭证和原始堆栈不进入公开 message。
|
||||
|
||||
| 错误类别 / 例子 | 框架动作 | 不允许的动作 |
|
||||
|---|---|---|
|
||||
| transient:429、部分 5xx、连接失败 | 只在安全步骤重试,遵守 Retry-After | 无限重试、复用同一流片段直接追加 |
|
||||
| invalid_arguments:schema 不匹配 | 把字段级错误作为 observation,最多 1 次参数修复 | 启动工具后才校验;静默丢弃字段 |
|
||||
| validation_failed:编译/文档校验 | 保留候选与诊断,局部修复后重新验证 | 将未验证草稿标为成功 |
|
||||
| context_overflow | 压缩可重建上下文,保留任务/限制/未决效应 | 丢失最新用户指令、授权或版本 |
|
||||
| auth / forbidden | 直接 needs_input/blocked,交应用处理 | 换身份、扩大范围或匿名绕过 |
|
||||
| conflict / stale_revision | 重新读取版本并重新规划 | 覆盖用户修改、对写操作盲重试 |
|
||||
| capability_unavailable | 返回可行替代操作清单 | 假称已搜索、已渲染或已执行 |
|
||||
| policy_denied | 显式拒绝该动作 | 把限制当工具错误让模型规避 |
|
||||
| budget_exhausted | 保存状态与已完成结果,返回 partial | 自动增预算或伪造 completed |
|
||||
| unknown_effect / worker_crash | 对账、查询外部状态,不能判断则 needs_reconciliation | 重跑整个工作簿 |
|
||||
| cancelled | 停止新调用、保留可读产物、传播取消 | 在后台悄悄修复或恢复 |
|
||||
|
||||
初始建议:传输最多 2 次重试(总 3 次尝试),指数退避基准 0.5/1 秒并加 jitter;有 Retry-After 时使用服务端建议并受总 deadline 限制。参数修复最多 1 次,内容修复最多 2 次。预算由应用传入,框架只提供默认上限与执行机制。所有重试、修复和摘要模型调用计入同一成本账本。
|
||||
|
||||
错误指纹 `hash(tool_id, normalized_arguments, error_code, relevant_revision)` 连续出现 2 次且没有新证据时中止该策略,允许一次有依据的替代路径;不能通过改写错误文本绕过循环检测。
|
||||
|
||||
### 3.2 四类“恢复”必须分开
|
||||
|
||||
1. **续订事件**:客户端从 cursor 重放和订阅同一个 task,不发起新推理。
|
||||
2. **重试步骤**:相同逻辑 operation、同幂等键;新 invocation attempt,复用成功的前序结果。
|
||||
3. **恢复工作流**:从兼容 checkpoint 建立新 runtime task / attempt,关联 `resumed_from`,不把旧终态改回 running。
|
||||
4. **重新生成答案**:新的输出版本;已有成功工具结果可复用,用户明确重新执行才重新产生效应。
|
||||
|
||||
当前 TaskState 把 interrupted/needs_reconciliation 视为终态,沿用此约定。新增 `AgentRun` 逻辑关联只管理目标和 attempt 链;调度权仍在原 runtime。恢复请求带 `expected_revision`、checkpoint digest 与幂等键,相同请求返回同一新 attempt,冲突返回 409 对应的框架错误。
|
||||
|
||||
执行失败不必丢掉有用内容:`outcome.status = succeeded | partial | failed | needs_input | cancelled`,伴随 `answer_ref/artifacts/error/recovery_options`。HTTP 调用成功与 Agent 业务 succeeded 不再混为同一字段。
|
||||
|
||||
## 4. 检查点与工具效应
|
||||
|
||||
### 4.1 CheckpointV2
|
||||
|
||||
在 `plugins/agent/src/resume.rs` 扩展已有实现,统一根目录由宿主提供,客户端不能传任意绝对路径。保留 legacy load,但旧 checkpoint 无效应信息时默认 `manual_review`,不能自动重做写操作。
|
||||
|
||||
| 字段组 | 内容 |
|
||||
|---|---|
|
||||
| 身份 | schema、checkpoint_id、logical_run_id、task_id、attempt_id、step_id、parent_checkpoint_id |
|
||||
| 作用域 | principal_ref、workspace_id、session_id、授权 scope digest |
|
||||
| 输入 | task revision、workbook revision/hash、输入资源 revisions、knowledge generation |
|
||||
| 运行配置 | ModelSnapshot public ref、tool registry digest、policy revision、runtime compatibility |
|
||||
| 进度 | completed_steps、pending_operation、observation refs、context summary ref |
|
||||
| 约束 | 剩余 token/tool/time/cost 预算、deadline、cancel epoch |
|
||||
| 一致性 | effect ledger watermark、event seq、manifest digest、committed_at |
|
||||
|
||||
写入协议:写唯一临时文件→flush/fsync→原子 rename→提交 checkpoint 指针;持久化配置下目录 fsync 按平台支持实现。所有引用 blob 在指针提交前落盘。损坏记录报 `checkpoint.corrupt` 并尝试上一个完整版本,不能读取半个 JSON 当无历史。
|
||||
|
||||
恢复前校验身份/授权、源版本、工具 schema 与模型兼容性。权限撤销立即生效;checkpoint 不是永久授权。源文件改变返回 `checkpoint.input_changed`,重新计划受影响步骤,不能偷偷使用最新文件继续旧验证结果。
|
||||
|
||||
### 4.2 Effect ledger
|
||||
|
||||
```text
|
||||
prepared → started → committed
|
||||
↘ failed_no_effect
|
||||
↘ unknown → reconciled_committed | reconciled_no_effect | needs_input
|
||||
```
|
||||
|
||||
每个 operation 持久记录 `operation_id/idempotency_key/args_digest/resource_scope/expected_revision/effect_class/result_ref`。幂等键作用域是 principal + workspace + logical_run + step + operation,不用随机 attempt ID 替换稳定 operation key。对同 key 不同 payload 拒绝。
|
||||
|
||||
读工具可重试;确定幂等写由服务端去重;可补偿写仅在补偿明确安全时执行;未知写先查询 receipt/版本。对外部系统不能承诺通用 exactly-once。关键 crash 窗口是“远端已成功、本地尚未记账”,必须返回 unknown 并对账。
|
||||
|
||||
本地文件使用 expected revision + staging + atomic replace;变更提交后检查点失败也靠 operation receipt 恢复。`tool.completed` 必须在结果与 receipt 可读取后提交,再对外发事件。事件发送失败不回滚已发生的业务效应。
|
||||
|
||||
不要复制 runtime 事件存储:Agent ledger 与 artifact manifest 是步骤事实,现有 runtime event log 是传输/调度事实;以 ID 关联,采用可重建投影与 outbox。跨 DB/文件不能声称一个事务完成,使用 manifest 提交点和恢复扫描补齐索引。
|
||||
|
||||
## 5. 结构化工具与调度
|
||||
|
||||
### 5.1 ToolSpecV1
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "ToolSpecV1",
|
||||
"id": "workspace.read",
|
||||
"version": "1",
|
||||
"description": "读取授权资源的指定行范围",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"resource_id": {"type": "string"},
|
||||
"revision": {"type": "string"},
|
||||
"start_line": {"type": "integer", "minimum": 1},
|
||||
"max_lines": {"type": "integer", "minimum": 1, "maximum": 500}
|
||||
},
|
||||
"required": ["resource_id"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"output_schema_ref": "ResourceReadV1",
|
||||
"effect": "read",
|
||||
"scopes": ["workspace:read"],
|
||||
"timeout_ms": 10000,
|
||||
"max_result_bytes": 65536,
|
||||
"idempotency": "safe",
|
||||
"concurrency_group": "resource-read",
|
||||
"availability": "ready"
|
||||
}
|
||||
```
|
||||
|
||||
定义由宿主/可信模块注册;用户认证、scope 与资源根由宿主注入,不接受模型提供 principal 或任意 root。`read`、`write`、`execute`、`external_write` 是策略分类,不是靠工具命名推断。
|
||||
|
||||
ToolCallV1:`call_id/tool_id/tool_version/arguments/operation_id/parent_span_id`。ToolResultV1:`ok/status/summary/structured_content/resources/citations/error/usage/truncated/next_cursor/effect_receipt_ref`。超限结果先写 artifact,再返回摘要和范围读取句柄;不把几十 MB 文本送进 prompt。
|
||||
|
||||
### 5.2 一条执行通道
|
||||
|
||||
```text
|
||||
发现 → 加载 schema → schema 校验 → 授权与范围解析 → 预算/取消检查
|
||||
→ 幂等查询 → 执行 → 结果 schema/大小校验 → receipt/结果提交 → 事件 → observation
|
||||
```
|
||||
|
||||
provider 原生 function call 与文档 `CALL:name + ARGS:{...}` 都归一到该通道。保留旧白名单适配,已有 `.mq.md ##` 可根据形参推导初始 schema,再允许显式描述收紧。转换不允许调用未暴露的框架内部函数。禁止对模型参数使用 eval/任意代码反序列化。
|
||||
|
||||
`tools.search` 返回已授权且当前可用的工具摘要;`tools.describe` 按 ID 返回完整 schema。核心 5–8 个工具可常驻,其余按需加载;此数量是产品起始配置,不写死框架。每次真正调用仍重新检查授权;搜索索引可列 unavailable 原因供诊断,但不能让未授权工具名字泄露私有连接信息。
|
||||
|
||||
初始支持 JSON Schema 的明确子集并公开能力;不支持的 schema 拒绝注册,不静默放宽。原生工具能力、并行工具调用、图片输入、结构化输出分别探测;纯文本兼容 provider 继续走 CALL,但并发和可靠性承诺按测试能力开放。
|
||||
|
||||
### 5.3 并发、MCP 与程序化执行
|
||||
|
||||
独立只读调用可并发,默认最多 4 个;修改同一资源的调用串行且用 revision CAS。依赖链、审批与取消检查保持顺序。一个并发调用失败不丢弃其他已成功结果。
|
||||
|
||||
MCP 扩展现有 client:先完善 timeout/cancel、错误映射、工具分页与 schema、resource read/link,再按 capability 增加远程 Streamable HTTP 与连接认证。stdio 不应允许模型选择任意宿主可执行文件。连接元数据归应用,凭证通过 secret handle 注入 worker,工具输出和 trace 不记录密钥。
|
||||
|
||||
工具 annotations 只是提示,可信策略确定真实效应。外部结果全部为数据,不能提升为 standing/system。MCP resource URI 转成宿主授权 ResourceRef,不允许前端直接请求任意 file:// 或内部网地址。
|
||||
|
||||
程序化工具编排和子 Agent 为后续切片:在沙箱中仅暴露注册 dispatcher,逐次执行同样的授权、预算和效应日志;禁止通过一段脚本绕过逐工具控制。子任务继承更小权限和共享总预算,不默认开放写同一资源。
|
||||
|
||||
## 6. 输出通道与上下文
|
||||
|
||||
### 6.1 AgentOutputV1
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "AgentOutputV1",
|
||||
"status": "succeeded",
|
||||
"answer": {"format": "markdown", "text": "已整理为三点,完整笔记见右侧。"},
|
||||
"artifacts": [{"resource_id": "res_note_01", "revision": "sha256:example"}],
|
||||
"citations": [],
|
||||
"validation": {"status": "passed", "report_ref": "resource:verify_01"},
|
||||
"finish_reason": "completed",
|
||||
"recovery_options": []
|
||||
}
|
||||
```
|
||||
|
||||
这是结果传输协议,不要求模型每次逐字生成这张 JSON。文档式编排仍可产出 `.mq.md`,框架把稳定产物、摘要与事件归一化。普通文本回答可没有 artifact 和 validation;未运行的内容标 `not_applicable/not_run`,不能填 passed。
|
||||
|
||||
channel 至少区分 `answer/artifact/progress/diagnostic`。只有 answer delta 可追加到聊天正文;工作簿 token 和工具日志进入其他通道。进度是可观察动作摘要,不要求或泄露模型内部思维链。
|
||||
|
||||
应用传 `ResponsePolicy`:detail level、目标长度、最大输出 tokens、是否允许 artifact、用户显式需求。框架负责 budget 和 finish_reason,中文字符数与最终文案由应用决定。达到 token 上限返回 `length_limited` 和完整/不完整标记,不能裁剪 JSON/代码后假装成功。
|
||||
|
||||
优先从已验证产物结构投影摘要。必要时增加一次受预算控制的摘要模型调用;摘要不得新增未经证据支持的结论。修复成功后 accepted candidate、结果正文、artifact revision 与 verification digest 必须全部一致。
|
||||
|
||||
### 6.2 ContextPackV2
|
||||
|
||||
按优先级组装:可信运行规则→最新用户目标/约束→用户选中资源范围→未决操作/成功步骤摘要→检索证据→近期对话→可回读历史。保持角色结构,不把用户/工具文本简单拼进系统指令。
|
||||
|
||||
每个 item 携带 `origin/trust/resource_ref/revision/token_estimate/priority`。预算初值建议保留模型窗口 20% 给输出与协议开销;上下文使用达到可用预算 70% 时触发整理,具体比例配置化。优先剔除可重读的完整工具输出,保留摘要、引用和失败恢复锚点。
|
||||
|
||||
Compaction 必须保留:目标、用户限制、当前计划、已完成步骤、待决写入、授权范围、输入版本、已接受产物与引用、余量。summary 绑定其 source event range 和 digest,不能当成新的已验证知识。工具输入过时须重读,不能依赖压缩摘要执行写入。
|
||||
|
||||
已有 EKC/agent memory 继续提供知识发现和复用。长期用户记忆的创建/修改/删除政策归 qdagent;网页中的指令不能自动变成长期偏好。
|
||||
|
||||
## 7. 统一资源与只读视图
|
||||
|
||||
### 7.1 ResourceRefV1 / ResourceViewV1
|
||||
|
||||
ResourceRef 统一携带 `resource_id/workspace_id/kind/media_type/revision/digest/title/size/origin/locator`;opaque ID 由宿主映射,不将浏览器 path 直接当文件权限。revision 是内容版本,generation 是知识发布代,两者不混用。
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "ResourceViewV1",
|
||||
"resource": {
|
||||
"resource_id": "res_01", "workspace_id": "ws_01", "kind": "file",
|
||||
"revision": "sha256:example", "media_type": "text/markdown", "title": "设计说明"
|
||||
},
|
||||
"renderer": {"id": "markdown", "version": "1", "trust": "sanitized"},
|
||||
"render": {"format": "sanitized_html", "fragment": "<p>正文</p>"},
|
||||
"outline": [],
|
||||
"locators": [],
|
||||
"completeness": "complete",
|
||||
"no_exec": true,
|
||||
"diagnostics": [],
|
||||
"next_cursor": null
|
||||
}
|
||||
```
|
||||
|
||||
继续把 `.mq.md` 交 `view.document`,ResourceView 只包装/扩展,不能另解释一套 Marqdo 语义。普通 `.md/.txt/代码/JSON/CSV` 走媒体类型处理,避免全仓语义编译。浏览器组件由 qdagent 实现,框架只提供可读内容、受控片段或短期授权 blob 引用。
|
||||
|
||||
`fragment`、`text`、`blob_ref` 是互斥载荷,按严格 format enum 解码。未知格式返回 unsupported,不能用 `format.includes('html')` 猜测 HTML。为空与加载失败必须有独立状态。
|
||||
|
||||
### 7.2 格式矩阵
|
||||
|
||||
| 格式 | 框架提供 | 前端消费 | 未具备能力时 |
|
||||
|---|---|---|---|
|
||||
| `.mq.md` | 静态 DocumentView、outline、节点/引用 | 正文、语义链接、源码切换 | 只读源码 + 明确诊断,不执行 |
|
||||
| Markdown / 笔记正文 | 共享静态 Markdown→受控结构/HTML、相对资源引用 | 标题、列表、表格、代码、链接 | 纯文本 |
|
||||
| 文本/代码 | 分页行片段、编码信息、行定位 | 行号与语法高亮 | 编码提示/下载 |
|
||||
| JSON / CSV | 有界解析、路径/行列定位、分页 | 树/表/源码 | 原始文本,标格式错误 |
|
||||
| 图片 | 授权 blob、尺寸/元信息 | 缩放/旋转/替代文本 | 下载,不暗示已做 OCR |
|
||||
| PDF | blob、页元信息,可选提取插件 | PDF.js 页渲染与定位 | 文件卡;提取不可用不阻止支持的视觉预览 |
|
||||
| DOCX/XLSX/PPTX | 受限转换插件,可选 PDF/HTML 派生物 | 按派生格式预览 | 未安装转换器,保留下载 |
|
||||
| HTML/SVG | 默认源码;允许时经严格清洗或受限派生物 | 隔离预览 | 不以内联可执行页面显示 |
|
||||
| 未知二进制/压缩包 | 元数据 | 下载卡 | 不自动解压/执行 |
|
||||
|
||||
提取、预览、执行各有独立 capability。办公转换、OCR 使用受限 worker,禁宏与网络、限制时间/内存/页数/解压膨胀比;结果绑定原件 digest、转换器版本和参数。首期不承诺保真编辑。
|
||||
|
||||
### 7.3 安全、定位、缓存
|
||||
|
||||
所有读取先校验宿主授予的 workspace/resource scope,canonicalize 并处理 symlink 逃逸;读取句柄与校验对象一致,防路径检查后替换。读取原件的身份权限不可被缓存绕过。
|
||||
|
||||
预览绝不调用 `run/verify` 或有执行副作用的 view 路径;正文中的 Marqdo 调用链接只展示语义与导航,执行需单独工具调用与授权。远程图片、相对链接、资源 URL 经受控解析;下载入口不允许任意内网 fetch。
|
||||
|
||||
HTML 清洗在框架产出侧和浏览器插入侧都有明确边界;用成熟 allowlist 实现而非正则。禁止 script/event handler、危险 URI、任意 style/form/嵌入对象。前端配 CSP;对需要交互的产物另建无权限 sandbox,不复用文档渲染器。
|
||||
|
||||
缓存 key:workspace + resource ID + revision + renderer version + render options + policy version;命中前重新授权。历史 revision 不存在就返回 `version_unavailable`,不能默默展示当前版。locator 支持 node/span/line/page/cell,失效时标 stale 并提供用户主动查看最新版的操作。
|
||||
|
||||
建议首期限制:文本片段 256 KiB、代码单次 500 行、CSV 单次 200 行;更大内容分页,完整文件保留。数值需压测校准,不作为现有系统性能声明。
|
||||
|
||||
## 8. 事件、可观测性与能力协商
|
||||
|
||||
复用 RuntimeEventV2 的 task/run/attempt/seq/event_id。拟增加事件类型:`agent.step.started/completed`、`tool.started/completed/failed`、`recovery.scheduled/exhausted`、`checkpoint.committed`、`artifact.updated`、`answer.delta/committed`、`context.compacted`。现有 `message.delta` 由兼容适配映射 answer channel,不能同时重复消费两套事件。
|
||||
|
||||
终态提交包含 accepted output manifest、last_seq、result digest;断流客户端能通过快照补齐最终答案。stream delta 是预览,最终 committed revision 是权威。统计每次尝试的 usage,摘要和修复调用分别标 purpose;未知计量保持 unknown。
|
||||
|
||||
拟新增 capability:`agent.error_v1`、`agent.checkpoint_v2`、`agent.tool_schema_v1`、`agent.output_channels_v1`、`agent.context_pack_v2`、`resource.view_v1`,以及各 media renderer 的独立版本/可用状态。保持现有 probe,新增字段向后兼容;未知 feature 当不可用。
|
||||
|
||||
trace 记录因果 ID、耗时、错误类别、参数摘要、资源版本和 effect receipt。正文与密钥按配置脱敏,不默认把用户全文件复制进日志。输出中不包含模型私有推理;可审计的是实际动作与结果。
|
||||
|
||||
## 9. 分阶段实现与文件落点
|
||||
|
||||
| 包 | 修改/新增建议 | 交付内容 | 依赖 / 退出条件 |
|
||||
|---|---|---|---|
|
||||
| M-A0 | `src/workbench/capabilities.rs`、`tests/workbench_contracts.rs`、新增合同 fixtures | 上述协议、旧适配、状态映射 | 先冻结跨仓字段,未知字段/版本可降级 |
|
||||
| M-A1 | `ext/ai/agent.mq.md`、`plugins/agent/src/resume.rs`、新 recovery/effects 支撑模块 | 分类错误、有界修复、accepted candidate、原子检查点 | M-A0;故障注入不重复写、旧记录不冒险自动恢复 |
|
||||
| M-A2 | `plugins/agent/src/lib.rs`、`mcp_client.rs`、新 tool registry/dispatcher | schema 工具、调用统一、分页与结果资源 | M-A0;未授权调用为 0,失败结果结构正确 |
|
||||
| M-A3 | `src/workbench/document_view.rs`、新 resource_view、媒体适配模块 | ResourceRef/View、普通格式静态阅读、no-exec | M-A0;资源定位与安全语料通过 |
|
||||
| M-A4 | `ext/ai/agent.mq.md`、`plugins/llm/src/lib.rs`、`src/workbench/run_context.rs` | 输出分通道、预算、ContextPackV2 | M-A1/2;原生和纯文本 provider fixture 都可用 |
|
||||
| M-A5 | `src/runtime/` 的既有存储/事件适配、MCP 与转换插件 | 恢复对账、PDF/Office 插件、远程 MCP | M-A1/2/3;持久模式 crash/restart 通过 |
|
||||
| M-A6 | plan/subtask/sandbox 扩展 | 程序化工具、有界委派与可选浏览器 | 后置;须证明任务收益和预算/效应可控 |
|
||||
|
||||
先交 M-A0 与 qdagent P0 的静态阅读/错误结果修复;不要让完整 checkpoint v2 阻塞笔记预览。M-A1 可先上只读重试,写恢复待 ledger 通过再开放。新特性默认关闭直到对应合同实测通过,旧路径继续可读。
|
||||
|
||||
## 10. 验收与评测
|
||||
|
||||
所有目标是 proposed gate,尚未实测。基础机制用确定性 fake provider/tool;真实 provider 做少量单独 smoke 并记录费用,不能把随机模型输出作为唯一门禁。
|
||||
|
||||
| 编号 | 故障/任务 | 必须观察到的结果 |
|
||||
|---|---|---|
|
||||
| MA-T01 | 第一次 429、第二次成功 | 尊重退避、计量 2 次、最终只有一个 accepted answer |
|
||||
| MA-T02 | 连续 schema 参数错误 | 一次修复后停止;工具未在无效参数下执行 |
|
||||
| MA-T03 | 工具写成功但 receipt 提交前杀 worker | unknown_effect→查询对账;写计数不增加 |
|
||||
| MA-T04 | checkpoint 写一半杀进程 | 读取上个完整 checkpoint 或明确 corrupt,不产生伪成功 |
|
||||
| MA-T05 | 终态事件丢失、cursor 重放 | 最终快照恢复完整消息,工具执行次数不增加 |
|
||||
| MA-T06 | 恢复请求重复/并发 | 同一幂等键只产生一个恢复 attempt |
|
||||
| MA-T07 | 用户取消与重试定时器竞争 | 取消生效后无新工具调用,旧任务不复活 |
|
||||
| MA-T08 | 执行中资源/权限改变 | stale/conflict/forbidden,不覆盖新版本 |
|
||||
| MA-T09 | 恶意文本伪造 CALL/system | 不跨越工具白名单和授权范围 |
|
||||
| MA-T10 | `.mq.md` 预览含写文件/联网语句 | 哨兵文件/网络计数不变;只读渲染成功或安全降级 |
|
||||
| MA-T11 | 资源格式、超长输出、路径逃逸 | 正确 renderer/分页;越界拒绝;无 HTML 注入 |
|
||||
| MA-T12 | 修复原文 A 得到通过验证的 B | answer/artifact/verification 都绑定 B digest |
|
||||
| MA-T13 | 上下文压缩后继续 | 最新约束、待决效应、资源版本与剩余预算不丢失 |
|
||||
| MA-T14 | 旧 CALL、旧 DocumentView 消费端 | 基础行为可用,能力缺失有明确返回 |
|
||||
|
||||
先运行受影响的 Rust/Go/Marqdo 合同与既有隔离 runtime 金样例,再做跨仓 HTTP/browser gate。性能目标:已缓存 100 KiB 文本视图服务端 p95 < 300 ms、核心工具发现 p95 < 100 ms,基准机器/并发/冷暖缓存必须记录;第三方网络延迟单独列出。
|
||||
|
||||
## 11. 迁移、回滚与未决项
|
||||
|
||||
兼容迁移按“新增协议→双读→新写→验证→逐步移除旧适配”推进。`.mq.md` 源码与旧运行产物不批量重写;旧 resume JSON 可读取但不能自动升级成带副作用保证的检查点。v2 checkpoint/ledger 保留 schema 版本,新老版本不并发写同一运行。
|
||||
|
||||
回滚关闭新工具自动发现、输出通道或转换器时,已提交资源和最终答案仍可读;执行恢复中的任务先 drain/停止接纳,不能切回会盲重做写操作的旧路径。迁移前备份可变索引与 manifest,回滚不删除用户产物。
|
||||
|
||||
实现前验证项:现有 runtime 持久存储可提供的原子提交边界、各平台 fsync/rename 行为、LLM adapter 原生 tool call 流式兼容、转换插件部署体积。协议优先按本文保守语义实现,验证不能满足则 feature 保持不可用,不阻塞已经完成的只读预览切片。
|
||||
@@ -1,5 +1,7 @@
|
||||
# 语义评估:参数能否从上下文推断(含叙述式调用)
|
||||
|
||||
> **2026-10-23 审阅裁决**:保留参数来源可见、避免隐藏调用者环境的边界;下文已被否决的 L0 建议不可重新启用。自然语言只连接和解释,不能成为隐式执行触发器;当前指导见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **§3 的 L0 方案已被否决**(违反「只执行标记」原则);L1/L2 结论仍有效 |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 评定:`[描述](语句)` —— 方括号读作「人话」,圆括号读作「机器话」
|
||||
|
||||
> **2026-10-23 审阅裁决**:本文的 Classic 冲突统计和兼容取舍属于迁移历史;v2.1 默认采用描述包裹 payload,不能据旧冲突恢复默认 `[键](集合)`。保留标签的阅读/检索/步骤价值,不强制所有子表达式命名,也不把标签当作目标或权限依据。参见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **已决策:走「路线二 纯度」**(落地见 [label-freshness-and-repair.md](label-freshness-and-repair.md)) |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 调用实参:具名与位置
|
||||
|
||||
> **2026-10-23 适用范围校正**:本文保留早期调用尾部规则和历史示例,不是 v2.1 全部调用语法。当前已有 `f(args)` 及具名实参;默认紧邻 `](` 是描述包裹 payload,不再是下文所述的 Classic 取元。语法审阅与开发取舍见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓),当前行为须由默认模式测试复核。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | 已定(v0.1 增补;2026-09-14:方括号标记调用) |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 语法设计:标记如何构成调用(在「只执行标记」原则下)
|
||||
|
||||
> **2026-10-23 审阅裁决**:保留显式标记和正文默认安全;性能数值是旧实验,不作为当前基准。持久跨文档接口优先具名参数,但不禁止位置参数;反引号声明与顺序改变可能是接口变化。当前调用与开发方向见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 待决策**(未改代码) |
|
||||
|
||||
@@ -3,22 +3,24 @@
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · 核心语法封闭集合(CI 守卫锁定)** |
|
||||
| 日期 | 2026-09-22 |
|
||||
| 日期 | 2026-09-22;2026-10-23 校正 v2.1 默认链接语义 |
|
||||
| 守卫 | `tests/core_surface.rs` ⇄ `src/parse/mod.rs` `CORE_CONSTRUCTS`(改动任一侧不同步 → CI 红) |
|
||||
| 相关 | [markdown-mapping-v0.3.md](markdown-mapping-v0.3.md)(语义宪法)· [layers.md](layers.md)(准入戒律)· [three-problems.md](../roadmap/three-problems.md) 问题一 · [three-problems-plan.md](../roadmap/three-problems-plan.md) T1.1/T1.3 |
|
||||
|
||||
> **戒律总纲**:核心标记是**封闭集合**(约 19 个构造)。新增任何一行必须走 ADR(见 [layers.md](layers.md) 准入戒律)并回答「为什么现有标记不够」;生态需求的默认答案是**进 ext**。本表 id 列与守卫测试强一致:改表不同步 `CORE_CONSTRUCTS`(或反之)、样例文件缺失,一律 CI 红。
|
||||
|
||||
> **版本边界**:构造 ID 是历史守卫标识,不是固定不变的语义说明。`link-index` ID 暂保留以匹配注册表,但默认 `[]()` 已是描述包裹 payload;旧索引语义仅属于显式 Classic 兼容路径。未来步骤/知识投影见[文档原生指导](document-native-development-guide.md),不要把目标能力当作当前已实现。核心数量之外,还要审查上下文例外和作者负担。
|
||||
|
||||
## 1. 行构造
|
||||
|
||||
| id | 外形 | 语义 | 戒律 | 样例 |
|
||||
|----|------|------|------|------|
|
||||
| `narrative` | 无标记文字 | 叙述(注释) | 段落内延续到空行;**默认安全**——永远不存在「叙述被执行」的路径 | `tests/markup-v03/soft-emphasis.mq.md` |
|
||||
| `narrative` | 无标记文字 | 叙述(不执行) | 段落内延续到空行;普通叙述不触发执行;未来知识引用投影也不改变此权限边界 | `tests/markup-v03/soft-emphasis.mq.md` |
|
||||
| `blank` | 空行 | 段落边界 | 叙述分段的唯一手段;不得被赋予其它语义 | `tests/markup-v03/inc-prose.mq.md` |
|
||||
| `heading-unit` | `#` / `##` | 对象 / 函数单元 | 标题即单元边界,按深度嵌套;正文禁止用 `##` 做版式(会变成函数) | `tests/markup-v03/decl-ident-only.mq.md` |
|
||||
| `call-line` | `>` | 调用行 | 输出等副作用 = **普通函数**,不是架构标记 | `tests/markup-v03/dual-call.mq.md` |
|
||||
| `branch-list` | 体内 `1.` 递增 | 分支 | 自 `1.` 起序号递增构成一个判断;重出 `1.` 开新判断;`N. *` = else;**不是所有有序列表都是 if** | `tests/markup-v03/greet-default.mq.md` |
|
||||
| `loop-list` | `-` | 循环 | 条件形 `` - `条件` `` 或遍历形 `- [项](集合)` | `tests/structure/loop.mq.md` |
|
||||
| `loop-list` | `-` | 循环 | 条件形 `` - `条件` `` 或遍历形 `- [说明](遍历 集合 为 项)`;描述不是绑定变量名 | `tests/structure/loop.mq.md` |
|
||||
| `param` | `` + `名` `` | 形参(兼容形) | 首选叙述 `` `名` `` 升参(宪法 §8);`+` 形仅兼容保留 | `tests/markup-v03/greet-default.mq.md` |
|
||||
| `table` | GFM 表 | 集合 | 几何即类型([tables-maps-footnotes.md](../roadmap/tables-maps-footnotes.md));**绑定 = 数据,未绑定 = 文档表** | `tests/structure/collection.mq.md` |
|
||||
| `fence` | ` ```lang ` | 外联 | 语言块原样外联,不进 Marqdo 语义 | `tests/lib/foreign-python.mq.md` |
|
||||
@@ -30,12 +32,12 @@
|
||||
| id | 外形 | 语义 | 戒律 | 样例 |
|
||||
|----|------|------|------|------|
|
||||
| `ident` | `` `名` `` | 声明 / 引用 | 首次未赋值 = 声明;叙述与程序共用同一视觉习惯 | `tests/markup-v03/decl-ident-only.mq.md` |
|
||||
| `bold-code` | `**…**` | 代码(赋值 / 调用 / 表达式) | 只有粗体进入执行;判别序:赋值 → 调用 → 表达式 | `tests/markup-v03/multi-bold.mq.md` |
|
||||
| `bold-code` | `**…**` | 代码(赋值 / 调用 / 表达式) | 粗体内代码判别序:赋值 → 调用 → 表达式;不取消调用行、返回等其它显式执行入口 | `tests/markup-v03/multi-bold.mq.md` |
|
||||
| `italic-return` | `*…*` | 返回 | 改变控制流(**返回是架构**);返回即结束函数体 | `tests/markup-v03/inc-prose.mq.md` |
|
||||
| `empty-return` | `*None*` / `****` | 空返回 | 返回 `None` 并结束函数体 | `tests/markup-v03/empty-return-none.mq.md` |
|
||||
| `link-index` | `[键](集合)` | 取元 / foreach | **主取元语法**;与 bracket-call 靠 `]` 后是否紧邻 `(` 消歧 | `tests/lib/table-collections.mq.md` |
|
||||
| `bracket-call` | `修饰… [函数] 实参…` | 方括号标记调用 | 前置修饰 → `名=True`;与 link-index 消歧同上 | `tests/structure/bracket-call.mq.md` |
|
||||
| `footnote-index` | `集合[^键]` | 过渡取元 | **仅兼容**;新代码一律写 link-index | `tests/structure/footnote-index.mq.md` |
|
||||
| `link-index` | `[描述](表达式)` | 描述包裹 payload(历史 ID) | 默认 Occlusion;取元可写 `[说明](access(集合, 键))`,不按描述取键;循环载荷见上 | `tests/occlusion/access-and-loop.mq.md` |
|
||||
| `bracket-call` | `修饰… [函数] 实参…` | 方括号标记调用 | 前置修饰 → `名=True`;紧邻 `](` 的默认语义为 payload 遮蔽,不是旧索引 | `tests/structure/bracket-call.mq.md` |
|
||||
| `footnote-index` | `集合[^键]` | 兼容取元 | 保留既有能力;新代码优先 `access(...)`,需要解释时包裹描述,不再推荐旧 `[键](集合)` | `tests/structure/footnote-index.mq.md` |
|
||||
| `soft-emphasis` | `*仅*` / `**说明**` | 忽略 | 非代码形强调**不执行**——默认安全的关键消歧规则 | `tests/markup-v03/soft-emphasis.mq.md` |
|
||||
|
||||
## 3. 词法大类(LineKind,`src/lex/mod.rs`)
|
||||
|
||||
@@ -0,0 +1,284 @@
|
||||
# Marqdo 文档原生开发指导:让知识的表达、执行与复用成为同一件事
|
||||
|
||||
| 项目 | 约定 |
|
||||
|---|---|
|
||||
| 状态 | 下一阶段开发指导与规划修订;不表示下述目标接口已经实现 |
|
||||
| 日期 / 审阅基线 | 2026-10-23 / `main`,`7674ebb`;继承缺口审计的 `434be9f` 代码证据 |
|
||||
| 原始意图 | [009](../next/009.md)、[附属设计](../next/Marqdo_Executable_Knowledge_Design.md)、[附属示例](../next/Marqdo_Executable_Knowledge_Example.md) |
|
||||
| 配套实施文件 | [详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md)、[缺口与交付路线](../roadmap/executable-knowledge-gaps-2026-10-23.md) |
|
||||
| 本轮边界 | 校正开发方向、作者体验、AI 工作协议和验收;不新增语法,不宣称实现新能力 |
|
||||
|
||||
## 1. 审阅结论与规范关系
|
||||
|
||||
Marqdo 的优势不在于给传统程序换一套 Markdown 符号,而在于:**读者解释知识、作者修改知识、程序执行知识、AI 查找并复用知识,能够指向同一份有结构的文档和其中的同一步。** 标题、叙述、表格、链接描述、输入输出、来源及执行行为不应分散在彼此手工同步的系统中。
|
||||
|
||||
现有规划没有放弃这一目标,但工程排序存在偏移:把身份、图、证据、锁、句柄、晋升全部铺完,再验证人和 AI 是否更容易使用。这样可能交付一个功能齐全的制品管理系统,却让文档作者承担另一套注册、声明、审核和配置工作。需要改变交付方法,不必推翻确定性编译、绑定和权限机制。
|
||||
|
||||
本指导规定**为何开发、作者维护什么、怎样证明值得开发**;详细规范规定语义模型与执行约束;路线图规定当前缺口、切片与完成证据。三者不是三套平行设计。遇到冲突,依次采用:`009` 的产品意图 → 本指导对作者负担、场景分级和交付次序的修订 → 详细规范的工程规则 → 附属概念示例。当前代码与测试证明“已实现什么”,不能把代码局限升级为永久语言禁令;原始讨论保留,不据其重新引入已被否决的方案。
|
||||
|
||||
### 1.1 方向修订清单
|
||||
|
||||
| 容易偏移的做法 | 修订后的要求 | 必须保留的技术约束 |
|
||||
|---|---|---|
|
||||
| 先做完整知识治理平台,再给作者使用 | 从一篇可读、可运行、可定位步骤的文档交付,逐层扩展 | 每个已支持路径都有准确的目标和效应边界 |
|
||||
| 为进入知识库再写 YAML、注册表和索引 | 从源码推导;只有不能推导的领域事实才需作者补充 | 生成物有版本,可重建,可检查来源 |
|
||||
| 所有逻辑先封装成可登记函数/类 | 一次性逻辑直接写;需要命名、解释或复用时逐步提取 | 作用域、形参、返回与副作用明确 |
|
||||
| 所有调用先审核、锁定、签发远程凭据 | 本地探索、项目复用、严格业务、远程执行分级 | 精确目标、运行时权限不能被标签或模型绕过 |
|
||||
| AI 只能搜文档、填工具参数 | AI 可以读步骤、组合、选择候选、局部修改、提取和验证 | 模型的选择必须转成显式目标,由编译器校验 |
|
||||
| 为“独特”拒绝普通调用或新增更多标记 | 保留简单 payload;让 Markdown 结构提供程序之外的信息 | 无第二套表达式文法,无裸词隐式执行 |
|
||||
| 核心构造数量不增长就是简单 | 同时检查上下文例外、记忆负担和诊断成本 | 新构造需 ADR;不靠把新规则藏进旧构造绕过审查 |
|
||||
|
||||
### 1.2 四个不能互相替代的完成条件
|
||||
|
||||
1. **读得懂**:不打开生成图,也能从源文件看出目的、关键步骤、参数角色和限制。
|
||||
2. **执行同一件事**:标签表达意图,payload 决定计算;运行 trace 能回到源码步骤,不能执行另一个同名目标。
|
||||
3. **改得动**:人或 AI 能修改一个明确步骤,看到影响与验证结果,不必重写整份文档或同步多份签名。
|
||||
4. **复用得对**:找到已有能力后实际调用它;不满足条件时给出具体拒绝原因,而非用复制源码掩盖失败。
|
||||
|
||||
“更适合 AI”是待验证的产品假设,不是 Markdown 文件后缀自动赋予的性质。正确性和作者成本优先于 token 下降;错误复用更便宜不算优化。
|
||||
|
||||
## 2. 语法审查的保留、修正与暂缓
|
||||
|
||||
以下裁决针对审查结论的适用范围,不重写历史实验记录。旧文档中的原型、性能数值和当时的 `Value`/AST 限制,不能直接当作 v2.1 的规范或测试结果。
|
||||
|
||||
| 审查材料 | 保留的洞见 | 需要修正的外推 / 本轮处置 |
|
||||
|---|---|---|
|
||||
| [函数即命名](function-as-naming-philosophy.md) | 从正文计算出发,命名应降低理解和复用成本;避免无意义包装 | “具名给值缺失”是历史观察,当前有具名参数;替换模型是教学解释,不意味着禁止递归、前向引用或必须文本展开 |
|
||||
| [方括号作为标签](bracket-as-label-evaluation.md) | 描述与 payload 共址;标签帮助解释、检索和拆步骤 | Classic 冲突是迁移历史,不据此恢复默认 `[键](集合)`;不强制每个常量或子表达式都有标签 |
|
||||
| [圆括号与可读调用](paren-call-and-readable-invocation.md) | 调用标记应明确;保留 `f(args)` 的局部可读性 | 原型撤回不代表现有版本缺少该能力;Marqdo 的独特性不依赖消灭熟悉的表达式形式 |
|
||||
| [从标记推导调用](call-syntax-from-marks.md) | 语义成分有明确标记;自然语言是连接与解释,不参与猜测调用 | 持久跨文档接口优先具名实参;短而稳定的位置参数仍合法。推断参数的声明次序变化属于接口变化,不承诺任意润色都无影响 |
|
||||
| [环境参数](ambient-parameters-evaluation.md) | 参数来源应可见,避免调用者隐式环境影响复用 | 文档顶部已否决 L0 裸“执行”触发,正文旧建议不能重新生效;文件级常量也须有明确绑定与依赖,不变成动态作用域 |
|
||||
| [函数作为值](functions-as-values-evaluation.md) | 区分名称数据、一等函数、有限查表;避免无需求增加核心复杂度 | “匿名函数必不能索引”“动态调用必然破坏契约”不是普遍定理。将来可研究稳定节点身份与运行时契约;本轮不引入闭包,也不永久禁止 |
|
||||
| [函数作为表格](function-as-table-semantics.md) | 数据表几何有既定含义;契约表不可另造一份参数声明 | 不把表格全部变成函数,也不把“现有实现不支持”写成“表格不能计算”;有限决策表先评估库方案,只有明确角色和歧义测试后才讨论语言扩展 |
|
||||
| [核心表面](core-surface.md)、[分层](layers.md) | 核心小、标记可解释、构造有测试与准入流程 | 清单中的旧取元/循环说明需随迁移更新;固定数量不等于低认知成本,守卫通过也不证明文档语义正确 |
|
||||
| [链接设计](link-syntax.md)、[告警疲劳](warning-fatigue-control.md) | 单一 payload 文法;标签漂移可提示;默认安静、按需严格 | 标签不是纯粹可丢弃的注释,也不是事实证明;不做每次运行重复的批准仪式,不靠词汇相似度证明描述与代码一致 |
|
||||
|
||||
### 2.1 继续冻结的语法边界
|
||||
|
||||
- 默认 `[]()` 是描述包裹 payload;旧取元只能在显式历史兼容路径下解释。现行取元用 `access(...)` 等已支持形式;循环使用明确遍历载荷。
|
||||
- `[]()` 不等于“仅函数调用”:计算、取元、调用、遍历、叙述引用都需要各自语义角色,不能只收集 call 边就宣称完成知识步骤。
|
||||
- 正文引用与可执行位置有不同调度权限。普通句子不因出现“执行”“发送”或工具名而运行;正文解析成引用也不自动求值。
|
||||
- 不把所有 `#` 解释成某种知识对象,或把继承悄悄变成实例关系。标题的既有语义必须兼容,IR 可区分 Type、Entity、Fact、Function。
|
||||
- 不给同一参数增加正文声明、签名表、工具 schema 三套作者真相。规范签名和工具 schema 是投影;契约表补充类型、约束和说明,不偷偷增加形参。
|
||||
- 可读源文件是硬要求;普通 Markdown 渲染器是否把含空格/嵌套表达式的链接渲染成可点击链接是另一件事。不能承诺全部标准渲染器理解执行语义。
|
||||
|
||||
### 2.2 哪些编辑不能当成“只是改文案”
|
||||
|
||||
不含语义标记的普通叙述润色,应保持执行结果和实现摘要;标签改名应保留 payload 行为,但更新展示/检索内容。反引号声明、首次出现顺序、标题名称/层级、粗斜体、表格几何、契约、来源和适用范围的改变则可能影响程序或知识可信性。编辑器必须展示真实差异,不能统一标为 prose-only。
|
||||
|
||||
标签漂移诊断应说明“描述或实现可能需要复核”,不能自动替作者改写业务意图。自然语言一致性检查可作辅助建议,确定性检查仍需可定位的符号、类型、契约或测试证据。
|
||||
|
||||
## 3. 作者工作流:先写清楚,再获得复用能力
|
||||
|
||||
### 3.1 一次性计算不需要预先建知识模型
|
||||
|
||||
下面是当前默认语法的完整最小程序。无模式开关、无项目注册、无知识 ID、无锁文件。
|
||||
|
||||
```markdown
|
||||
# main
|
||||
|
||||
这是演示订单,单价与数量直接写在计算处。
|
||||
|
||||
**小计 = [两件商品的小计](2 * 30)**
|
||||
> print text=`小计`
|
||||
```
|
||||
|
||||
可读意图和精确计算共址。这个标签值得保留,因为它说明本次计算的业务角色;不要求把 `2`、`30`、乘法分别命名。
|
||||
|
||||
### 3.2 需要解释或复用时才提取命名能力
|
||||
|
||||
```markdown
|
||||
## 商品小计
|
||||
|
||||
根据购买数量`数量`和商品单价`单价`计算小计。
|
||||
|
||||
*[按数量与单价计算](数量 * 单价)*
|
||||
|
||||
# main
|
||||
|
||||
计算同一笔演示订单。
|
||||
|
||||
**小计 = [两件商品的小计](商品小计(数量=2, 单价=30))**
|
||||
> print text=`小计`
|
||||
```
|
||||
|
||||
两个完整程序在本轮本地现有 `target/debug/marqdo` 上分别以 Tree 和 Bytecode 执行,均输出 `60`。这仅验证基础语法,不证明步骤身份、重构工具或 Agent 闭环已经实现;也不是全仓测试报告。可将各块保存为 `.mq.md` 后用 `marqdo run 文件 --backend tree` / `--backend bytecode` 复核。
|
||||
|
||||
**后续工具目标**:从选中表达式提取为标题单元,推导自由变量为参数、保持求值顺序与 effect 次数、生成具名调用,并给出可审阅补丁。反向内联也应保留契约和来源关联;无法证明安全的捕获变量、递归、控制流、动态目标或有副作用表达式,先拒绝自动变换并说明原因,不伪称等价。
|
||||
|
||||
### 3.3 跨文档复用与事实说明分别增长
|
||||
|
||||
当第二篇文档需要这段计算,才提取模块并显式 import;不要为十行单文件强制创建 `corpus/knowledge/apps` 目录树。项目索引按约定发现,明确边界或外部来源时再加入配置。
|
||||
|
||||
当“商品小计”依赖政策、时间、币种等条件,再就地补契约与来源。简单乘法不要求假造现实来源;“某商店当前折扣为九折”则不能仅凭函数返回 `0.9` 声称真实。
|
||||
|
||||
正文可以说明结果为何成立,目标是引用同一能力和同一次受控执行的结果,而非手抄另一份公式。正文引用 AST、结果投影和绑定身份尚需开发;完整业务链使用[目标示例](../next/Marqdo_Executable_Knowledge_Example.md)验收,不把它冒充当前全部可运行的教程。
|
||||
|
||||
### 3.4 维护时看到知识变化,而不只是文件变化
|
||||
|
||||
目标维护流程:修改上游规则 → 显示受影响的计算、正文陈述和消费者 → 执行相关测试 → 展示结果/证据/接口各自变化 → 按使用策略接受更新。普通显示文案变更不应触发全库业务审批;证据过期也不必使无关纯工具全部失效。
|
||||
|
||||
## 4. 人维护语义,编译器维护机械关联
|
||||
|
||||
| 信息 | 作者责任 | 编译器 / 工具责任 |
|
||||
|---|---|---|
|
||||
| 目的与步骤 | 有帮助的标题、叙述、标签;避免空泛重复 | 保留源位置和层级,生成步骤投影,不要求手工节点注册 |
|
||||
| 输入输出 | 一份参数来源和必要契约;领域单位不能靠猜 | 推断规范签名、检查调用、生成 schema 和签名视图 |
|
||||
| 目标与依赖 | 显式 import、可读名称、必要消歧 | 绑定唯一 SymbolId,生成依赖边与闭包;不要求手写 ID |
|
||||
| 效应 | 扩展作者声明宿主能力;业务作者必要时收窄允许范围 | 汇总直接及传递效应;未知不得推断 pure;运行时执行能力上限 |
|
||||
| 事实与来源 | 原始资料、范围、有效期、人工审阅等不可推导信息 | 关联来源、记录验证事件、传播失效;不将文件路径视为事实证明 |
|
||||
| 身份与版本 | 有意移动/更名时确认必要迁移 | 区分身份与内容摘要;报告歧义,不靠猜测维持假稳定 |
|
||||
| view / AI 索引 | 不维护第二份可执行描述 | 同一语义模型投影;显示生成代、验证政策和缺失字段 |
|
||||
|
||||
生成信息不等于免费:首次编译、缓存修复、错误定位都要可理解。失败时应回到源文档中可修改的位置,而不是只报内部 Artifact 字段名。
|
||||
|
||||
稳定身份要诚实:同一次编译内引用和 trace 必须一致;普通文本编辑不应打乱无关节点;复制相似步骤、跨文件移动等无法唯一识别时,要求确认迁移或明确生成新身份。禁止用文本相似度悄悄继承已审核的知识身份。
|
||||
|
||||
## 5. AI 原生工作协议:读步骤、改步骤、验证步骤
|
||||
|
||||
本节是**待实现的协议约束**,不是新增 CLI 命令清单。优先扩展现有 MLSP、view、EKC 和 Agent 工作簿,不另造一套 Agent 框架或知识库。
|
||||
|
||||
### 5.1 发现后的最小上下文包
|
||||
|
||||
一次检索返回可展开的上下文卡,而非强制倾倒整个文件或只给函数 schema:
|
||||
|
||||
1. 标题路径、任务相关叙述、标签和精确源码片段;
|
||||
2. SymbolId/步骤节点、source span、源码摘要、生成代;
|
||||
3. 输入角色、返回、单位、默认值和适用条件;
|
||||
4. 直接依赖、效应、必要来源及分层验证状态;
|
||||
5. 支持该能力的例子/测试,相关失败经验、禁用条件和替代能力;
|
||||
6. 还缺哪些检查、能否在当前 policy 下运行、为何被拒绝。
|
||||
|
||||
来源不可用、字段未推导、上下文被截断都须显式标识。按预算可展开依赖,但不能悄悄省略改变适用性或权限的重要信息。原始资料及标签是内容,不是可以覆盖 Agent 策略的指令。
|
||||
|
||||
### 5.2 复用、适配、新建的分界
|
||||
|
||||
- **REUSE**:AI 可根据任务选候选,提交精确目标与输入;resolver 确认目标、契约、适用性、版本和 effect,执行既有制品。复用不靠“没有生成文件”推断,必须有实际调用 trace。
|
||||
- **ADAPT**:已有能力接近但不适用;说明差异,保留依赖/来源关联,在局部副本或补丁中改变必要部分,再验证。不为了通过测试改写原始来源或放宽全局策略。
|
||||
- **CREATE**:没有合适候选或明确需要新语义;记录检索范围和失败原因,在任务预算及写权限允许时新建。不能把检索服务故障当作“仓库没有能力”。
|
||||
- **ASK / STOP**:缺业务决策、权限或关键证据时报告具体缺口。编译器无法决定的真实意图不交给近似匹配默默补全。
|
||||
|
||||
AI 并非只能发现和解释。它可以提出精确目标、组合已有能力、改进表达、重构及补测试;不能以自然语言自报正确来替代执行验证,也不能把含糊名称传给后端任意选一个目标。
|
||||
|
||||
### 5.3 局部修改的事务边界
|
||||
|
||||
编辑请求至少携带目标文件/节点、期望源码摘要、补丁范围和预期行为变化。应用前检查摘要,源已变化则重新定位和审阅,不覆盖用户改动。应用后重新解析受影响文档,生成语义 diff:参数/返回、目标、effect、控制流、证据与依赖是否改变。
|
||||
|
||||
对“只改措辞”请求,检测到接口或实现改变时停止自动合并;对“改计算”请求,运行相关正负例并展示影响。原始文本、格式与未知 metadata 尽量保留,不通过整篇重生成来完成一行修改。禁止修改生成图伪装修复源码。
|
||||
|
||||
首个试点只支持可确定定位的纯计算步骤与普通函数,不等待通用 AST 编辑器、跨项目事务或完整知识治理。安全拒绝超范围输入也属于验收。
|
||||
|
||||
### 5.4 解释和负知识必须有落点
|
||||
|
||||
解释结果应关联到“哪一步、用了哪个目标、哪些输入、哪份来源、哪次验证”,区分已执行观察和模型推断。运行 trace 可以证明程序路径,不能证明任意自然语言论证正确。
|
||||
|
||||
失败经验需要保存适用范围、触发条件、关联能力、原因与失效时间/版本。例如“此折扣不适用于退款”应在查询定价能力时可见,而不是埋在另一份聊天记录里。先由普通文档与结构化关联承载,不先新增通用推理语言;过期负知识也要撤销,不能永久拦截修复后的能力。
|
||||
|
||||
## 6. 渐进严格:不把治理门槛塞进每一行正文
|
||||
|
||||
下面是目标政策分级,不是当前已有四种 CLI 模式。政策由项目/宿主配置和调用上下文选定,源文档保持同一种语法。
|
||||
|
||||
| 场景 | 必需条件 | 默认不要求 |
|
||||
|---|---|---|
|
||||
| 本地探索、单文件运行 | 明确语法与目标、已有运行时契约、宿主能力限制;如实显示未知状态 | lock、来源、ready 晋升、远程签名句柄 |
|
||||
| 项目内自动复用 | 精确绑定、输入/适用性检查、效应许可、状态满足所选政策、执行 trace | 所有纯工具都有人审证据、每次独立确认 |
|
||||
| 严格业务/可复现任务 | 相关依赖与证据锁定、必要验证新鲜度、受控环境和审计;明确不可重放部分 | 对无关文案变化重审整个项目 |
|
||||
| 远程/多租户/不受信调用方 | 上述相关约束,加可信授权主体、范围、服务端能力上限、撤销/期限与防伪设计 | 把客户端提交的 `allow_effects` 当授权本身 |
|
||||
|
||||
`ready` 是“对某项 policy 和输入/上下文已具备复用条件”,不是永久万能认证。未验证的本地草稿可以被检索、查看和按明确权限试跑;不能冒充满足严格政策的可复用事实。测试通过与事实证实分别报告。
|
||||
|
||||
正文 `check/view static` 不执行外部效应。用户主动打开的纯预览可复用项目或会话已有政策,不要求每个链接弹窗;effectful 运行始终受宿主能力和明确执行上下文控制。锁和摘要是完整性机制,不自动等于远程授权;可信本地引用无需先搭建凭据服务。
|
||||
|
||||
## 7. 交付切片:每片都有文档作者与 AI 能看见的结果
|
||||
|
||||
编号与[缺口路线图 §7](../roadmap/executable-knowledge-gaps-2026-10-23.md#7-依赖顺序与可合并切片)一致;G 编号沿用现状证据,M0–M6 保留为工程能力映射,不再被理解为必须全部串行完成的墙。
|
||||
|
||||
### S0 作者基线与最小语料
|
||||
|
||||
- 选定单文件计算、提取函数、两文档引用、表格取元/遍历、业务政策及拒绝复用六类任务;记录当前写法、输出、诊断和作者重复维护项。
|
||||
- 优先整理 `doc/README`、核心语法卡片、教程与运行示例的版本状态;不要先扩展通用库数量。
|
||||
- 为最小样例建立无网络门禁。100 单元及更大语料逐步增长,不阻塞首个可读文档切片。
|
||||
- 完成证据:别人按当前入口能写出正确默认语法;已实现与预期失败分开报告,不能把待实现测试跳过计为成功。
|
||||
|
||||
### S1 可读步骤与唯一目标
|
||||
|
||||
- 主要落点:`src/parse/`、`src/ast.rs`、`src/sema/`、`src/knowledge/compile.rs`、`src/view/`。
|
||||
- 在最小故事内联合完成 G01/G03:保留 label、payload、上下文和跨度;每类步骤有输入输出与依赖投影;旧 EKC 查询索引不再覆盖语义图。
|
||||
- 普通导航、正文引用、可执行表达式、循环分别处理,正文保持惰性。先确保支持范围内同一引用在源码、view、图中是同一个目标,再扩展覆盖范围。
|
||||
- 完成证据:作者能从渲染步骤返回对应源码;同名目标不误连;计算/取元/循环不是伪造的 call;重复构建可对照;未支持上下文明示。
|
||||
|
||||
### S2 确定执行与早期 AI 复用试点
|
||||
|
||||
- 主要落点:`src/sema/`、`src/interp.rs`、`src/bytecode/`、`src/mlsp.rs` 与现有 Agent 入口。
|
||||
- 完成静态普通函数、显式 import、关联函数的 BoundCall 主链;G02 与 G06 的效应安全下限不可延期。小范围可先用明确纯操作白名单,未知路径必须拒绝纯证明,而非猜 pure。
|
||||
- 实现最小上下文卡与本地绑定引用;以 stub/no-LLM 测试验证协议,然后用现有 Agent 跑相同故事。无需先完成远程鉴权平台或全面来源审核。
|
||||
- 完成证据:同一需求实际调用既有函数,不新增同义实现;两个后端 trace 对上图中目标;改目标、错误输入、未知效应、权限不足不能静默执行。
|
||||
|
||||
### S3 局部编辑、提取与库可读性
|
||||
|
||||
- 主要落点:MLSP 编辑/定位、相关 sema/检查入口、`lib/`、`ext/`、教程与测试。
|
||||
- 按 §5.3 实现受摘要保护的局部修改和语义 diff;实现受限纯表达式提取,或明确标为待实现,不用人工演示冒充工具能力。
|
||||
- 迁移标准库/扩展时删掉重复包装与噪音标签,保留有意义的业务步骤、契约和负例;中英镜像含义一致。兼容旧调用接口,不用增加旧语法解释救迁移遗漏。
|
||||
- 完成证据:同义润色不改执行;改反引号声明能检测签名变化;表达式提取前后结果/effect 次数一致;冲突补丁拒绝;CLI、库、扩展与 view 既有能力继续通过。
|
||||
|
||||
### S4 业务知识的来源与适用性
|
||||
|
||||
- 主要落点:契约、SourceRef/VerificationRecord、验证事件、Agent 候选解释;完成 G05 与 G07 的项目政策部分。
|
||||
- 选定书店案例的币种、数量边界、会员范围与政策有效期;给 schema、拒绝诊断及可运行负例,不泛化为全能业务规则语言。
|
||||
- 同步补齐传递效应与相关负知识;生成契约/状态展示,减少作者重复填写。
|
||||
- 完成证据:同一计算因输入或证据不同而正确通过/拒绝;来源过期不被 `tests=passed` 覆盖;普通纯工具无需虚构 SourceRef。
|
||||
|
||||
### S5 变化传播与严格复用
|
||||
|
||||
- 主要落点:`src/sema/lock.rs`、图/impact、产物提交与消费者;完成 G04/G08 和严格 handle 复核。
|
||||
- 区分身份/接口/实现/证据/闭包,给出可读升级 diff;整代快照保证消费者不会读取混代数据。并发读写一旦开放,该一致性就是前置条件,不能等到此片才承认风险。
|
||||
- 完成证据:修改依赖政策后,下游计算和正文解释进入影响集合;文案变化不误判实现变化;并发编辑或构建崩溃不导致错目标执行;锁不声称重放网络/随机/LLM。
|
||||
|
||||
### S6 生态验收与发布
|
||||
|
||||
- G10/G11 从 S0 起每片执行;S6 汇总兼容矩阵、真实工作流和打包后冒烟,不把测试留到最后。
|
||||
- G12 随实际规模需求推进;远程凭据/多租户是有条件的独立交付。它们不阻塞受信本地试点,也不得被本地通过的测试覆盖为“已安全支持”。
|
||||
- 完成证据:当前默认语法的 lib/ext/CLI/MLSP/view/WASM 等受影响消费者有明确结果;目标示例中的已发布子集实际可运行,未发布部分保留状态;发布声明与证据一一对应。
|
||||
|
||||
## 8. 验证 Marqdo 优势,而不只验证组件存在
|
||||
|
||||
### 8.1 必须收集的产品指标
|
||||
|
||||
| 维度 | 记录方法 | 失败信号 |
|
||||
|---|---|---|
|
||||
| 作者负担 | 完成同一任务所需手写声明、重复字段、配置、跳转和确认次数 | 简单本地算式也必须填写制品清单 |
|
||||
| 阅读与定位 | 给定问题,定位正确步骤/限制的准确率、耗时与上下文量 | 只看 label 会误导,必须打开生成 JSON 才理解源文档 |
|
||||
| 可修改性 | 修改范围、无关 diff、语义变化检测率、冲突覆盖数 | 为改一步而重生成全文件;措辞调整偷偷改变接口 |
|
||||
| 复用质量 | 已有能力发现率、实际调用率、重复实现率、误用率 | 检索命中却重新写代码;正确拒绝算成任务失败而被优化掉 |
|
||||
| 知识可追溯性 | 结果关联目标、步骤、输入、来源和验证的覆盖率 | `tested` 被当作事实正确;解释引用未执行的步骤 |
|
||||
| 总成本 | 正确完成任务的 token、工具调用、轮次、总耗时及编译成本 | 只报模型 tokens,隐藏额外注册/审核/修复成本 |
|
||||
|
||||
### 8.2 对照实验与门槛
|
||||
|
||||
1. 固定任务、可访问能力、输入数据、模型版本、工具能力、预算与超时;比较普通 Markdown+代码/RAG、当前 EKC 和新方案。不能给新方案更多现成实现再声称语法更优。
|
||||
2. 任务包括成功复用、同名冲突、近似但不适用、未知 effect、过期来源、正文不执行、局部修改及库升级。加入自然语言诱导绕过 policy 的负例。
|
||||
3. 先测确定性协议,再测多次 AI 运行,保留种子(支持时)、原始 trace、版本、失败样本与分布;小样本不夸大收益。
|
||||
4. 正确目标、无越权和正文无外部 effect 是硬门槛。原有兼容测试不得降低;效率阈值先测基线再登记,不捏造提升百分比。
|
||||
5. 若作者负担增加或误用率上升,回到交互/语义设计分析原因,不能只用“更多治理更安全”或“模型更便宜”解释过去。
|
||||
|
||||
### 8.3 发布声明分层
|
||||
|
||||
- “支持新语法”需语法、迁移、双后端与库兼容证据,不意味着完整知识复用已经交付。
|
||||
- “本地确定性知识复用”需 S1/S2 支持范围内的端到端与拒绝用例;范围外路径显式不支持,不能回落为模糊执行。
|
||||
- “严格业务知识复用”还需相关 S4/S5 来源、适用性、失效、锁和一致性证据。
|
||||
- “安全远程知识执行”需额外的授权、能力上限、凭据/撤销及隔离测试;本地 JSON handle 不等于远程安全。
|
||||
|
||||
## 9. 每个开发变更必须回答的问题
|
||||
|
||||
提交设计或实现时给出一份短而具体的审阅记录:
|
||||
|
||||
- 用户要读/写/改/复用的知识是什么?附源文档前后对照,不只给内部结构图。
|
||||
- 利用了哪种文档原生信息?若只增加通用框架能力,与主线的必要关系是什么?
|
||||
- 新增了哪些作者必须手工维护的信息?为什么不能从现有语义推导?
|
||||
- 是否改变标记、作用域、求值顺序、形参顺序、表格几何或正文安全?分别有哪些测试?
|
||||
- 人、CLI、view、图和 Agent 是否看到同一目标与状态?失败能否回到源码修复?
|
||||
- AI 的正确复用/局部编辑如何证明?哪些部分仍需模型判断,哪些由编译器决定?
|
||||
- 标准库和扩展是否先迁移?是否保住既有公开接口、输出与 effect?
|
||||
- 这是已实现、受限试点还是设计目标?文档和发布声明是否一致?
|
||||
|
||||
**准入原则**:内部可以使用成熟编译器、数据库和协议技术;外部必须减少知识在“说明、程序、索引、工具描述”之间的重复翻译。独特性由这条链的实际收益体现,不由新符号数量或与其他语言不同的程度体现。
|
||||
@@ -51,22 +51,25 @@ Marqdo 把仓库投影为 **可查询、可验证、可复用** 的工程知识
|
||||
|
||||
`implements` · `uses` · `depends_on` · `tested_by` · `documented_by` · `constrained_by` · `decided_by` · `supersedes` · `conflicts_with` · `deprecated_by` · `derived_from` · `failed_by` · `related_to`
|
||||
|
||||
落盘:`.marqdo/graph/graph.json` · `edges.json` · `index.json`。
|
||||
落盘(G01 分离后):
|
||||
- **权威语义图**:`.marqdo/graph/edges.json` · `nodes.json`(`SemanticProject`,含 `authority` 字段)· `steps.json`
|
||||
- **EKC 检索投影**(非执行依据):`.marqdo/ekc/graph/graph.json` · `edges.json` · `index.json` · **`ir.json`**(完整 IR 快照,供查询热路径)
|
||||
|
||||
## 5. CLI
|
||||
|
||||
```bash
|
||||
marqdo knowledge [PATH] -o .marqdo # compile(默认)
|
||||
marqdo knowledge preflight "task"
|
||||
marqdo knowledge [PATH] -o .marqdo # compile(默认;写投影 + ir.json)
|
||||
marqdo knowledge preflight "task" # 暖机读 ir.json;缺图才 compile
|
||||
marqdo knowledge preflight "task" --refresh # 强制全量重编译后再预检
|
||||
marqdo knowledge record REUSE
|
||||
marqdo knowledge learn --task ... --failure ...
|
||||
marqdo find "authentication"
|
||||
marqdo find "authentication" # 同上:缺图才 compile;--refresh 强制
|
||||
marqdo reuse "implement oauth login"
|
||||
marqdo impact path/to/file.mq.md
|
||||
marqdo conflicts | stale | duplicate [--fail] | verify
|
||||
```
|
||||
|
||||
统一 `--json`。`duplicate` 默认阈值 `0.75`;`--fail` 作 CI Gate。
|
||||
统一 `--json`。查询类命令(`preflight` / `find` / `reuse` / `impact` / `conflicts` / `stale` / `duplicate` / `verify`)默认 **读已编译投影**;CI 或源变更后应先 `marqdo knowledge` 或加 `--refresh`。`duplicate` 默认阈值 `0.75`;`--fail` 作 CI Gate。
|
||||
|
||||
## 6. Reuse 协议
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 语法设计哲学:函数在「代码即文档」里为何水土不服,以及用什么语义取代
|
||||
|
||||
> **2026-10-23 审阅裁决**:保留“正文优先、按需命名”的方向;下文语料统计和“具名给值缺失”等判断属于当时快照。当前已有具名实参;“调用即代入”不规定文本展开实现,也不禁止递归。下一阶段以[文档原生指导 §2–3](document-native-development-guide.md#2-语法审查的保留修正与暂缓)的范围和验收为准,本文不是替换整个函数模型的已批准语法提案。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 哲学层讨论**(不涉及实现,不含语法提案细节) |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 语义探析:代码函数能否解释为一张 GFM 表
|
||||
|
||||
> **2026-10-23 审阅裁决**:下文 v1.5.2 实验不替代当前回归。保留现有数据表几何与契约表角色,不重复声明参数;“现有实现不支持函数表”不等于表格不能表达计算。有限决策表和生成式签名视图按真实用例评估,见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 待决策**(未修改任何代码) |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 方案评估:函数能否作为变量的一种(函数即表)
|
||||
|
||||
> **2026-10-23 审阅裁决**:以下是旧实现约束与方案权衡,不是“匿名函数无法索引”或“动态调用无法检查契约”的普遍结论。一等函数/闭包本轮暂缓而非永久禁止;有限决策表先评估现有库表达,文中建议的新辅助函数不因此成为已实现能力。参见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 待决策**(未修改任何代码) |
|
||||
|
||||
@@ -0,0 +1,326 @@
|
||||
# 隔离执行与响应式 Web 运行时:详细开发设计
|
||||
|
||||
| 项目 | 约定 |
|
||||
|---|---|
|
||||
| 状态 | **Verified**(execute 路径);HTTP `execute` / `sse-live` / `stream:true` 默认走 Supervisor worker 池(不再经父进程 `CallLibPath` / `INVOKE_LOCK`);`runtime.isolated_execution=1`;MR-01 金样例 `tests/isolated_execute_mr01.rs`;未声明非流式仍 `legacy_serial`(ADR 0008) |
|
||||
| 基线 | Marqdo `1b11fe0` / 2.1.3;qdagent `64b97e5`;2026-10-26 |
|
||||
| 问题依据 | [单解释器缺陷](web-invoke-serialization.md)、[ADR 0008](../adr/0008-web-invoke-serialization.md)、[本轮跨库复核 R01–R14](../../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md) |
|
||||
| 产品配套 | [qdagent 可靠工作台开发设计](../../../qdagent/doc/design/reliable-workbench-development-2026-10-26.md) |
|
||||
| 继承合同 | [工作台 M00–M06](qdagent-workbench-runtime-and-view-2026-10-25.md) 中 ModelSelection、ModelSnapshot、Usage、DocumentView、Locator;保留文档原生作者面 |
|
||||
|
||||
## 1. 目标、边界与不变量
|
||||
|
||||
目标是让用户在长对话、导入和构建过程中持续打开模型列表、读取来源、修改下一轮设置并可靠停止任务。高并发容量按资源预算显式限制;不能让所有请求进入一把解释器锁再超时。
|
||||
|
||||
Marqdo 负责通用执行隔离、资源调度、事件/取消、语义分析和制品校验。qdagent 负责用户/工作区授权、业务状态与发布政策。产品仍通过 `.mq.md` 调用 Marqdo Agent,不增加第二套模型循环或知识编译器。
|
||||
|
||||
必须保持的系统不变量:
|
||||
|
||||
1. 同一个可变 HostContext 同时最多一个求值者;不删除锁、不增加虚假的 `unsafe impl Sync` 来获得并行。
|
||||
2. 一个执行租约只属于一个 run/attempt;凭证、事件、取消、cwd、权限和模型选择不可流入另一请求。
|
||||
3. 控制与读取具有独立容量,不被 LLM 等待占满;停止不需要重新进入正在执行的解释器。
|
||||
4. 事件可以重投,外部副作用不默认重做;不宣称跨网络 exactly-once。
|
||||
5. 已确认终态不能被晚到完成/取消覆盖;取消不回滚已经完成的外部写入。
|
||||
6. 阅读不执行 `.mq.md`,缓存不绕过授权或版本校验;缓存命中必须实际复用结果。
|
||||
7. 正文、依赖、证据和编译代属于同一不可变快照;不会把两个代的结果拼成一次阅读或执行。
|
||||
8. 任何能力开关都以实际探测和测试为依据;新 API/schema 在落地前都属于本文提案。
|
||||
|
||||
## 2. 架构选择
|
||||
|
||||
| 方案 | 优点 | 当前代价 | 本轮选择 |
|
||||
|---|---|---|---|
|
||||
| 继续全局锁,只增加 Go handler | 改动小 | 长任务依然阻塞所有 Endpoint,串扰变量也未消失 | 只保留作兼容模式 |
|
||||
| 同进程多 HostContext | 潜在内存复用较好 | GLOBAL_HOST、插件回调、环境、cwd、事件 sink 生命周期需系统重做 | 后续优化,不是首个交付门槛 |
|
||||
| 独立 Marqdo worker 进程 + 控制/读取通道 | 隔离现有全局状态,可强制回收卡死任务,复用现有语言与 Agent | IPC、进程启动、内存、发布写协调 | **首选实现** |
|
||||
| 引入外部通用队列和新业务 Runtime | 容易扩到集群 | 运维复杂度及第二套编排风险 | 单机阶段不引入 |
|
||||
|
||||
首期采用**预热的一次性执行进程池**:进程可预装运行时和非用户特定模块,分配任务后独占,结束即退出并异步补充。这样不用假设清空部分字段就能安全复用 HostContext。长寿命多租约 worker 只有通过状态泄漏、插件资源与 RSS 回归后才开启。只读分析服务可以长寿命运行,但不得加载有副作用的求值插件。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
UI[浏览器 / CLI / MCP] --> WEB[Go HTTP / 认证入口 / 限流]
|
||||
WEB --> CTRL[任务控制与事件服务]
|
||||
WEB --> SHORT[短业务请求容量]
|
||||
WEB --> READ[只读分析服务]
|
||||
SHORT --> POLICY[qdagent .mq.md 授权与业务规则]
|
||||
POLICY --> CTRL
|
||||
CTRL --> DB[(任务索引 / outbox / 事件日志)]
|
||||
CTRL --> SUP[Rust Supervisor / 有界调度]
|
||||
SUP --> A[Marqdo worker A / 一个租约]
|
||||
SUP --> B[Marqdo worker B / 一个租约]
|
||||
A --> AGENT[既有 Marqdo Agent / LLM / sandbox]
|
||||
B --> AGENT
|
||||
A --> STAGE[每任务暂存目录]
|
||||
B --> STAGE
|
||||
STAGE --> COMMIT[按资源串行提交与 revision CAS]
|
||||
COMMIT --> SNAP[(不可变知识代 / 发布 manifest)]
|
||||
READ --> SNAP
|
||||
A --> CTRL
|
||||
B --> CTRL
|
||||
CTRL --> UI
|
||||
```
|
||||
|
||||
“短业务请求容量”依旧可执行产品 `.mq.md`,但有独立 worker 预算和短 deadline。控制服务只做通用接纳、状态 CAS、取消与日志投递,不把模型选型、知识晋升等业务逻辑搬进 Go。读取服务接收经过授权的资源句柄;不得直接相信浏览器提交的绝对路径。
|
||||
|
||||
## 3. 路由分类与接纳
|
||||
|
||||
### 3.1 执行类别
|
||||
|
||||
在 Endpoint 装配中新增可选 metadata(字段名待 schema 固定):
|
||||
|
||||
| 类别 | 示例 | 资源预算与规则 |
|
||||
|---|---|---|
|
||||
| `control` | 任务查询、事件订阅、取消、health | 不执行用户代码;认证后访问任务存储/租约注册表 |
|
||||
| `read` | view.document、引用解析、已发布模型列表投影 | 只读快照;不写用户源码;可缓存 |
|
||||
| `short` | 保存设置、创建导入、接纳任务 | 无 LLM 长等待;默认 deadline 2s,较慢动作转为任务 |
|
||||
| `execute` | ASK/BUILD/生成/验证 | 有界队列、专属进程、显式任务 deadline |
|
||||
| `commit` | 切换知识代、合并受控文件变更 | 按 workspace/kb 的短临界区;执行前检查 revision |
|
||||
|
||||
旧 Endpoint 未声明时仍走 legacy_serial;不能自动猜测任意函数是纯读。纯度来自静态能力和注册配置,实际写权限仍由宿主限制。文件上传字节传输不占解释器,应先落隔离存储,再进入短接纳路径。
|
||||
|
||||
### 3.2 接纳步骤
|
||||
|
||||
1. HTTP 层解析受限请求,验证身份、格式、速率与连接数量。
|
||||
2. 产品短入口解析 workspace/model/policy;授权版本与模型配置 revision 一并冻结。返回规范化任务请求,不执行 Agent。
|
||||
3. 控制服务在同一数据库事务中登记作用域化 idempotency key、payload hash、task 和 dispatch outbox;重复请求读回已有 task,同键异内容返回 409。
|
||||
4. 提交事务后才返回 202/Location。调度器消费 outbox 分配租约,租约带 attempt/epoch。
|
||||
5. 对同步兼容 API,先完成接纳,再挂接结果;等待客户端超时不等于安全重做。
|
||||
|
||||
队列入库前容量不足返回 429 + Retry-After;服务不可用返回 503;接纳失败不发送 started。控制通道须保留槽位,不能因为数据流连接占满而无法取消。
|
||||
|
||||
## 4. Supervisor、生命周期与调度
|
||||
|
||||
### 4.1 Worker 生命周期
|
||||
|
||||
`starting → ready → leased → running → draining → exited`;启动失败/心跳丢失走 `unhealthy → terminating → exited`。任务状态由控制服务维护,进程状态不能直接当作业务成功。
|
||||
|
||||
- worker 启动握手报告 CLI/ext/native、protocol_version、能力、构建摘要;不匹配不进入 ready。
|
||||
- 预热阶段不读取用户凭证、不挂载工作区写权限。分配租约后挂载当前任务的 capability roots、只读快照和独立 staging。
|
||||
- 每任务具有随机 lease_id、run_id、attempt_id、epoch、绝对 deadline 和资源限制。
|
||||
- Supervisor 持有进程句柄、子进程组、事件通道与取消句柄。worker 不能自行修改其他 worker 状态。
|
||||
- 返回业务结果后仍须 drain 事件/用量并提交最终 checkpoint,控制服务持久化终态后确认 ACK,之后释放租约。
|
||||
- 超时或断连清理不能遗留孤儿子任务;正常优雅退出后再删除临时文件。异常由独立 janitor 按租约过期回收。
|
||||
|
||||
### 4.2 有界、公平与资源预算
|
||||
|
||||
初始验证配置建议 4 vCPU/8 GiB、执行槽 2、短请求槽 1、读取服务 1。具体容量由每类任务的 RSS p95 和 CPU profile 决定,不能把这个数当所有机器的默认。执行槽上限:
|
||||
|
||||
`min(CPU约束, floor((可用内存 - 系统/控制/缓存预留) / worker_RSS_p95))`。
|
||||
|
||||
运行队列按 workspace 轮转,再按 session FIFO;一个 session 默认只允许一个活动 turn,跨 session 可并行。相同知识库的生成可并行,发布只占短提交锁。interactive 优先于后台索引,但后台使用老化提升或最低服务配额,防永久饥饿。
|
||||
|
||||
全局队列初始上限 64、单工作区 16、单会话待处理 4,均可配置;队列超限及时返回,不能悄悄丢任务。显示队列位置或“前方有任务”,没有可靠完成时间估计时不显示虚构倒计时。
|
||||
|
||||
### 4.3 副作用与重试
|
||||
|
||||
任务声明 `read_only / staged_write / external_effect`,由工具权限校验而非模型自报。read_only 可按政策重试;staged_write 丢弃未提交 staging 后重试;external_effect 出现超时/失联时标记 `effect_unknown`,等待核对,不自动重复支付、发信或写入。
|
||||
|
||||
稳定的操作键使用 `task_id + logical_step_id`,attempt_id 只标记传输尝试,不应导致同一外部动作生成新的去重键。工具不支持幂等查询时,只能明确报告不确定。工作簿记录已完成与未知副作用;“已停止”不意味着“全部撤销”。
|
||||
|
||||
## 5. IPC 与请求上下文
|
||||
|
||||
首期使用本机 Unix socket 或匿名管道,采用长度前缀 JSON 控制帧与独立事件通道。协议禁止无上限 JSON/单行日志;控制帧初始上限 1 MiB,大文档通过授权 blob handle 传递。普通 stdout/stderr 仅诊断,不与协议混流。当前 NDJSON 文件桥保留为 legacy fallback。
|
||||
|
||||
新增 `InvokeRequestV2` 示例(提案):
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "InvokeRequestV2",
|
||||
"schema_version": 2,
|
||||
"task_id": "task_01",
|
||||
"run_id": "run_01",
|
||||
"attempt_id": "attempt_01",
|
||||
"lease": {"id": "lease_01", "epoch": 3},
|
||||
"entry": {"module": "agent/qdagent.mq.md", "function": "run"},
|
||||
"args_ref": "blob:sha256:...",
|
||||
"context": {
|
||||
"workspace_id": "ws_01",
|
||||
"snapshot_id": "generation_01",
|
||||
"selection_ref": "model-snapshot_01",
|
||||
"capabilities_ref": "grant_01",
|
||||
"deadline_at": "2026-10-26T08:10:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`grant_01` 是服务端验证的短寿命授权句柄,内容包括 principal、scope、根目录、允许效应、授权 revision、失效时间。worker 不使用浏览器任意传入的 fn/path。权限撤回后,新的工具调用必须重新验证 lease/grant,已开始的外部调用记录撤回时间与实际结果。
|
||||
|
||||
`RunContextV1` 保留原有字段与序列化兼容,内部增补显式 Context 对象;扩展 V2 由能力协商启用。event_sink、usage_sink、cancel_token 使用句柄,不使用外部可控路径。子任务通过 IPC/Command.env 显式继承,禁止父进程 `setenv`/`set_current_dir` 承担请求状态。
|
||||
|
||||
凭证在租约启动时通过受保护 IPC 或继承 FD 注入;不落任务 JSON、argv、日志或通用事件。模型快照可含 credential_ref,不含 api_key。即使选择相同 provider,不同请求的授权也不得合并。
|
||||
|
||||
## 6. 取消协议与状态 CAS
|
||||
|
||||
任务基础状态:`accepted → queued → running → finalizing → succeeded/failed`;queued/running 可进入 `cancel_requested → cancelled`;worker 失联进入 `interrupted`,副作用不明时进入 `needs_reconciliation`。这些是提案,产品可以将 succeeded 映射为 completed。
|
||||
|
||||
取消顺序:
|
||||
|
||||
1. 验证调用者对 task 的权限,CAS 写 cancel_requested、revision 与 requested_at;终态返回原状态,未知 ID 返回 404。
|
||||
2. 向匹配 lease/epoch 的控制通道发送 Cancel;不经任务解释器锁。queued 任务原子移出待派发队列。
|
||||
3. worker 在每次 LLM/工具调用、重试、子任务启动和提交之前检查 token;网络请求使用可中断句柄,不能仅在一行输出结束后才检查。
|
||||
4. 收到 ACK 仍不等于任务停止。等待 worker 报 stopped、落未完成副作用和用量,再 CAS 到 cancelled。
|
||||
5. 默认 2s 协作宽限、额外 3s 进程退出宽限,之后回收进程组。强制退出记 interrupted/needs_reconciliation,不能伪造干净 cancelled。
|
||||
|
||||
完成与取消竞争的线性化点是控制存储的 CAS:终态先提交则 cancel 返回 already_terminal;取消先提交则晚到 result 只可作为证据追加,不能重新变 succeeded。若外部动作已发生但响应晚到,记录 effect outcome,不撤销取消事实。
|
||||
|
||||
兼容文件 token 统一:不存在/空/`0` 为未取消,`1`/`cancelled` 为取消;读取失败返回诊断并按明确策略处理。删除 token 只能发生在持有者退出之后。移除全局 `MARQDO_CANCEL` 对多个任务的隐式作用域。
|
||||
|
||||
## 7. 事件、SSE 与背压
|
||||
|
||||
### 7.1 事件权威
|
||||
|
||||
生产者发出 `(run_id, attempt_id, local_seq)`,控制服务去重、持久化并分配 task 内单调 seq/event_id;客户端游标来自持久化日志,不是 handler 重连后从 1 开始计数。终态事件唯一,usage 可以在终态后以更高 usage revision 补录,但不能再次结束任务。
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "RuntimeEventV2",
|
||||
"schema_version": 2,
|
||||
"event_id": "task_01:42",
|
||||
"seq": 42,
|
||||
"task_id": "task_01",
|
||||
"run_id": "run_01",
|
||||
"attempt_id": "attempt_01",
|
||||
"type": "message.delta",
|
||||
"payload": {"message_id": "msg_01", "offset": 24, "text": "结果"}
|
||||
}
|
||||
```
|
||||
|
||||
不要直接破坏 RuntimeEventV1;新增字段/新版本由能力协商确定。普通用户可见 progress 只描述工具、阶段、文件等结构化事实;模型内部 reasoning 不作为完成证据,也不默认写入公开事件。
|
||||
|
||||
### 7.2 App SSE
|
||||
|
||||
新增任务 API 的事件订阅用 `GET /tasks/{id}/events`(最终产品前缀见配套设计),发送 SSE `id`/`event`/`data`;每个 data 是一个完整 JSON。客户端支持 Last-Event-ID、CRLF、多行 data、任意 UTF-8 分块和 EOF 未完成帧。重复事件按 ID 去重;message offset 检查缺口,缺口时恢复快照。
|
||||
|
||||
心跳 15s,客户端空闲探测建议 45s;二者可配置。事件保留建议 24h,最终结果/审计按项目政策更长保留。游标早于保留边界返回 410 `event.cursor_expired` 和快照入口,不能默默从最新开始丢字。
|
||||
|
||||
订阅断开默认只关闭订阅。App 显式停止通过 cancel API,切页/移动网络重连可恢复同一任务。传统同步 OpenAI 调用可保持 disconnect=cancel 的兼容策略,但必须显式区分,不能统一把所有浏览器断流当取消。
|
||||
|
||||
### 7.3 OpenAI 兼容流
|
||||
|
||||
`/v1/chat/completions stream=true` 输出 OpenAI chunk 与 `[DONE]`;App 运行事件由独立入口或协商的协议提供。禁止仅凭任意 JSON body 中 stream=true 将不相关路由自动改为 RuntimeEvent 协议。已发 header 后的错误使用该协议允许的错误帧;发 header 前可用 429/503/401 等正确状态码。
|
||||
|
||||
token delta 与最终完整答复分开:最终事件持有结果引用/摘要和 finish reason,不再次追加整篇正文。无 token 能力的适配器报告 `evented_batch`,界面用阶段状态,不把整包人为拆字伪装成实时。
|
||||
|
||||
### 7.4 背压和旧桥修复
|
||||
|
||||
worker→事件服务与服务→订阅者均有界。建议每订阅者 256 KiB 待发送内存,超出后断开慢消费者并允许按游标重连;事件正文已持久化所以不能静默丢 token。允许合并短时间 delta,必须保留 message offset;usage/终态/错误不丢弃。磁盘压力时停止接纳新任务,已有任务使用保留终态空间并按策略中断,不能无限写事件。
|
||||
|
||||
legacy NDJSON tailer 只消费换行已完整到达的字节;保留残片,offset 按真实消费字节移动。超长行、I/O error、文件旋转显式报协议错误。测试至少分三次写一个 JSON、中文多字节边界、恰好 EOF、超过上限、取消与尾帧竞争。禁止吞掉 Scanner.Err() 继续宣告成功。
|
||||
|
||||
## 8. 只读快照与真正有效的缓存
|
||||
|
||||
### 8.1 快照一致性
|
||||
|
||||
分析服务构建 SnapshotKey:规范化 project identity、允许扫描范围、源文件闭包摘要、依赖/来源/配置摘要、compiler/lib/ext 版本、policy/schema、解析选项。输入由编译器枚举,不另造“前 200 个根目录文件”规则。
|
||||
|
||||
读取一次快照内的正文、语义和来源,完成后复核输入 manifest;文件在分析中发生变化则重试一次或返回 `snapshot.changed`。对不可变发布代直接读其 manifest。mtime/size 只用于快速发现候选变更,不能代替内容摘要;watcher 丢事件后以完整扫描修复。
|
||||
|
||||
### 8.2 缓存结构
|
||||
|
||||
- `AnalysisCache<SnapshotKey, Arc<Compilation>>`:实际编译结果。
|
||||
- `DocumentCache<(SnapshotKey, document_digest, renderer_version, view_options), DocumentView>`:安全 fragment、outline、nodes。
|
||||
- 同 key single-flight 的 follower 等待同一个 Result,不再次 compile。
|
||||
- RAII guard 在 error/panic/cancel 时清除 inflight 并唤醒 waiters;等待者有自己的 deadline。
|
||||
- compile 失败只返回给当前等待者,短暂错误缓存需显式 TTL;不得调用“finish_success”标记失败结果可复用。
|
||||
- LRU 按字节而非条数上限管理;初始预算按部署内存的 10% 且设硬上限,长文档超限不缓存或分页;记录淘汰原因。
|
||||
- 每次读取先授权再查缓存;权限敏感的响应按 scope/policy 分区。全局缓存只共享不含用户秘密且访问已验证的不可变语义数据。
|
||||
|
||||
性能证据必须统计 `compile_invocations`、输入字节、缓存结果命中、leader/follower、wait_ms、evictions;现有 hit counter 不能证明减少编译。
|
||||
|
||||
### 8.3 HTML 与来源定位
|
||||
|
||||
沿用 DocumentViewV1,明确 `render.format=plain_text|sanitized_html`。HTML 通过解析器白名单清洗标签/属性/URL scheme,剥离脚本、事件属性、危险 CSS/嵌入与可执行 SVG;相对资源经受限资源端点解析。纯文本始终转义,不以 `<` 字符判断类型。
|
||||
|
||||
node_id、revision、generation 必须从定位到读取到浏览器滚动完整传递。不存在历史快照返回 version_unavailable;返回当前版是用户显式操作且标明版本变化。no_exec 测试分别验证“无语言副作用”和“浏览器无脚本执行”,二者不能互相替代。
|
||||
|
||||
### 8.4 首次编译的性能改造
|
||||
|
||||
本轮 `v3_corpus_1000_compile_smoke` 在 debug 下 243.012s,超过既有 180s 门禁;缓存不能缩短一个全新项目的第一次编译。先为 `knowledge::compile` 的语义编译、语义落盘、检索 extract、检索落盘、manifest 分别加 span,再细分 `ProjectIndex::build`、源枚举/解析、导入挂接、绑定、效应传播、闭包摘要。记录文件数/总字节、节点/边数、parse 次数与序列化字节量。
|
||||
|
||||
源码中 `sema::compile_project` 在每个文件查找 module path、每个 module 过滤全量 symbols,属于值得测量的重复扫描;未取得 profile 前不将其认定为 243s 的唯一根因。优化候选按证据选择:建立 path→module 与 module→symbols 索引;同一输入快照共享 parsed module;绑定/依赖图用稳定索引;效应传播用 worklist;闭包按 SCC DAG 计算并保留循环依赖语义;减少每制品重复序列化和小文件写入。
|
||||
|
||||
任何优化须保持排序确定性、SymbolId/ArtifactId 与证据摘要规则,禁止通过跳过来源校验、删掉复杂 fixture 或改大超时过关。用 100/1000/10000 单元、无依赖/链/菱形/循环四种图分别测时间与 RSS;给出随 V/E 增长的曲线。release 和 debug 分开;已有 debug 门禁恢复通过只是第一步,§12 的 release 预算仍须独立验收。
|
||||
|
||||
## 9. 写入、发布与数据库边界
|
||||
|
||||
执行 worker 只写任务 staging;需要写工程时通过沙箱产生文件 diff。发布以 kb 资源锁 + expected_generation CAS 提交。锁只覆盖校验当前版本和切换 manifest,不覆盖 LLM、全文编译或网络调用。
|
||||
|
||||
完整发布代先写临时目录、校验摘要、完成必要 fsync,再原子 rename 到不可变 generation 目录;最后在同一文件系统原子替换 CURRENT。源码、编译投影和证据在该 manifest 下对齐。SQLite 中的产品“published”是投影,不参与虚构的文件/数据库跨系统原子事务。通过 publish journal + outbox 恢复:CURRENT 已切换但 DB 未更新时重放;未切换时旧代继续有效。
|
||||
|
||||
project_id 与逻辑文档 ID 不随 staging/generation 的物理目录名变化。manifest 保存稳定 project identity、逻辑相对路径和实际 blob/generation 映射;导入引用按项目规则解析,不能因复制到发布目录而重新生成全部符号身份。框架若尚不能区分逻辑身份与物理根,先补该能力,再启用 generation 目录发布;迁移 fixture 必须断言未修改的知识 ID 和跨文档绑定保持稳定。
|
||||
|
||||
任务表/幂等/CAS/outbox 需要通用事务能力:先审计 `ext/data` 是否支持事务、条件更新、受影响行数、复合唯一键;缺少则在 Marqdo 数据层补齐,不能用 `.mq.md` 的先读后写模拟原子性。SQLite 使用短事务和有界 busy timeout;WAL/连接池是否开启需按现有插件适配测试确认。单主本地盘为首期部署范围,跨主机共享 SQLite/NFS 不在首期承诺内。
|
||||
|
||||
## 10. 临时缓解与迁移
|
||||
|
||||
### 10.1 进程池完成前
|
||||
|
||||
- 保留 1500ms busy 预算和 Retry-After;展示繁忙,而非无限 spinner。
|
||||
- 在接纳任何 SSE 之前取得覆盖整个 bridge 生命周期的 legacy 单飞许可,第二条流在修改环境变量/发送 200 前失败;普通请求不得绕过许可观察被污染的状态。
|
||||
- 断连只标记该任务取消,清理由监督者等待执行结束后负责;无法确认停止时报告 stopping/interrupted。
|
||||
- 上述步骤只降低 R02/R03 风险,仍不算并行 Endpoint。
|
||||
|
||||
### 10.2 功能开关(新增提案)
|
||||
|
||||
`runtime.isolated_execution`、`runtime.task_control_v2`、`runtime.event_replay`、`view.analysis_cache_v2`、`runtime.read_lane`。探测返回 feature version 和限制。二进制版本相同不等于所有 native 插件都有该能力。
|
||||
|
||||
先离线 fixture → 本地假模型 → 新 workspace → 只读 ASK → staging BUILD → 真正发布。禁止对有副作用的请求做双执行 shadow;允许 shadow 只读分析并比较摘要。
|
||||
|
||||
回滚先停接新任务、drain 已接纳任务、保留任务/事件和新 schema;不能将活动任务交给不认识 lease 的旧进程。只读当前发布代可以保留,写入能力按版本协商关闭。ADR 0008 仅在并行金样例与回滚演练通过后由新 ADR 正式替代。
|
||||
|
||||
## 11. 开发包与依赖
|
||||
|
||||
下列新模块路径是建议,不代表文件已存在。
|
||||
|
||||
| 包 | 建议落点与改动 | 前置 | 验收 |
|
||||
|---|---|---|---|
|
||||
| M-R0 基线/缓解 | `host/plugin.rs`、`httpx/routes_invoke.go`;固定并发屏障、旧桥许可、取消清理 | 无 | 双流 busy 不改写 A 的桥;断连无提前清理 |
|
||||
| M-R1 请求上下文 | `workbench/run_context.rs`、`host/subtask.rs`、LLM 插件;统一 token 与显式 sink | M-R0 | 父子运行模型/事件/取消不串;空 token 不取消 |
|
||||
| M-R2 Worker/IPC | 新 `runtime/{supervisor,worker,protocol,lease}.rs`;独立进程与握手 | M-R1 | 两任务并行、超时回收、子进程无孤儿 |
|
||||
| M-R3 接纳/控制 | 新 `runtime/{task_store,scheduler,control}.rs` 与 ext/data 事务 primitives | M-R2 | 原子幂等、队列公平、取消与结果 CAS |
|
||||
| M-R4 事件协议 | `workbench/event_sink.rs`、Web stream/IPC、协议适配器 | M-R3 | 断点恢复/去重/半帧/慢消费/磁盘错误 |
|
||||
| M-R5 读服务/缓存 | `analysis_cache.rs`、`document_view.rs`、`locator.rs`、MLSP transport | M-R1,可独立推进 | 100 同 key 真正只编译一次;嵌套变更正确失效 |
|
||||
| M-R5b 首次编译性能 | `knowledge/compile.rs`、`sema/ir.rs`、ProjectIndex;分阶段 profile 与按热点优化 | 基线语义回归 | 现有 1000 单元门禁恢复;release 与规模曲线报告,身份和语义不变 |
|
||||
| M-R6 提交底座 | `knowledge` generation/manifest、publish journal、条件更新 | M-R3 | 崩溃点恢复,旧代不被半发布污染 |
|
||||
| M-R7 整合与发布 | 版本探测、CI、release、文档/ADR | M-R4–R6 + 产品接入 | §12 全部必要门禁与回滚演练 |
|
||||
|
||||
先以本地文件和单机进程解决明确缺陷;不要把集群、远程 worker、复杂 Actor 系统设为第一阶段依赖。
|
||||
|
||||
## 12. 验收、性能预算与可观测性
|
||||
|
||||
### 12.1 正确性矩阵
|
||||
|
||||
| 编号 | 场景 | 必须观察到的结果 |
|
||||
|---|---|---|
|
||||
| MR-01 | A/B 两个 60s 假模型流 + 20 次/s 模型/设置读取 | 两流不混字、不混模型;短请求 200;无 interpreter busy |
|
||||
| MR-02 | B 在 A 运行时接纳失败/断连 | A 事件和取消句柄完全不变 |
|
||||
| MR-03 | 同时取消 A、完成 B;取消重复/晚到 | A 正确停;B 成功;终态不回退;审计只结算一次 |
|
||||
| MR-04 | 帧拆字节、多行、EOF 半帧、重连 | 无丢字/重字;协议错误可见;游标恢复精确 |
|
||||
| MR-05 | 慢消费者、磁盘满、worker 崩溃 | 内存有界;事件不静默丢失;状态可恢复 |
|
||||
| MR-06 | 100 个相同读取并发 | 实际编译一次;后续结果复用;失败 leader 不挂死 follower |
|
||||
| MR-07 | 改子目录依赖、第 201 个文件、配置、来源 | SnapshotKey 改变;缓存失效;旧业务句柄拒绝 |
|
||||
| MR-08 | 上传有写文件/网络代码、HTML 事件属性 | 阅读不求值;浏览器脚本不执行;正文可读 |
|
||||
| MR-09 | 发布每个持久化边界 kill | 仅旧完整代或新完整代可见;DB 投影最终追上 |
|
||||
| MR-10 | 并发幂等请求、取消队列、权限撤回 | 单一 task;同键异内容 409;无越权派发 |
|
||||
| MR-11 | kill 后副作用未知 | 不自动重放,显示 needs_reconciliation |
|
||||
| MR-12 | 降级/升级、混合 CLI/ext/native | 缺能力明确禁用,不报已支持 |
|
||||
|
||||
### 12.2 性能目标(待测验收预算)
|
||||
|
||||
基准机:Linux 4 vCPU/8 GiB SSD;release 构建;1000 个知识单元、文档总量 10 MiB、500 条历史 run;假上游固定 60s、每秒 20 个增量;30s 预热、5min 采样、3 轮,报告 p50/p95/p99、错误率、CPU、RSS、磁盘。另测 10k 单元和 100 MiB 以确定容量上限,不能直接沿用小数据阈值。
|
||||
|
||||
| 指标 | 初始预算 | 说明 |
|
||||
|---|---|---|
|
||||
| 短读/任务接纳 | p95 ≤200ms,p99 ≤500ms | 两个长任务运行时;过载拒绝另统计 |
|
||||
| 取消受理 | p95 ≤200ms | 独立于任务停止耗时 |
|
||||
| 协作取消收敛 | p95 ≤2s | 假上游/可中断工具;不可中断外部动作按不确定处理 |
|
||||
| 已就绪 worker 调度 | p95 ≤100ms | 不含排队、模型首 token 时间 |
|
||||
| 事件到订阅者新增延迟 | p95 ≤100ms | 从已收到 provider delta 计,不承诺上游 TTFT |
|
||||
| 1000 单元首次分析 | p95 ≤2s | 候选目标,超标先剖析,不降低正确性 |
|
||||
| 热 DocumentView | p95 ≤100ms | 真实复用结果;长文档另测分页 |
|
||||
| 慢消费者缓冲 | 每订阅 ≤256 KiB | 总连接数也必须受限 |
|
||||
| 压测结束内存 | 稳态可回收、无持续增长 | 记录每次重复轮次 RSS;不能只看一次峰值 |
|
||||
|
||||
新指标至少有 queue_depth/wait、worker_active/recycle、admission_rejected、cancel_ack/stop、orphan_count、event_lag/replay/duplicate、compile_invocations/cache_result_hit、publish_conflict/recovery。指标标签禁止 task_id、原始路径、prompt 等高基数/敏感值;trace 中可记录脱敏 task/attempt/lease 与构建摘要。
|
||||
|
||||
现有测试继续保留:`cargo test --offline --locked --test workbench_contracts --test executable_knowledge`、LLM 插件单测、Go httpx 测试。新增集成门禁必须启动真实 HTTP→IPC→worker 路径并使用本地假模型;不能以手写 fixture 或写 token 文件代替端到端行为。本轮执行结果统一记录在[跨库复核 §5](../../../qdagent/doc/gaps/04-workbench-runtime-and-product-audit-2026-10-26.md)。
|
||||
@@ -1,5 +1,7 @@
|
||||
# 调用写法评估:`[函数名](参数)` 与更接近人读的调用形式
|
||||
|
||||
> **2026-10-23 审阅裁决**:“原型回退”仅描述当时实验,当前 payload 已支持 `f(args)` 和具名实参。旧 Classic 索引冲突不再决定默认语法;保留熟悉的调用形态与文档原生描述并不矛盾。参见[文档原生指导 §2](document-native-development-guide.md#2-语法审查的保留修正与暂缓),不要把本文备选形式当作已批准的新语法。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Proposed · 待决策**(原型已验证并已回退,未合入代码) |
|
||||
|
||||
@@ -0,0 +1,435 @@
|
||||
# qdagent 知识工作台所需的多模型运行与 View 嵌入支持
|
||||
|
||||
日期:2026-10-25。状态:Partial / in progress(M00–M06 合同门禁 + F02 agent.step/resume 假模型 e2e + USTC live selection smoke + DocumentView 浏览器只读核对已落地;F 矩阵未全绿,不得整文宣称 Verified;原生非兼容适配器仍 unavailable)。
|
||||
|
||||
源码基线:Marqdo `dc853fb`(2.1.2),qdagent `643dd31`,已同步远端。应用侧配套:[qdagent 多提供商与知识工作台](../../../qdagent/doc/design/multi-provider-and-knowledge-workbench-2026-10-25.md)。跨仓相对链接以兄弟仓库工作区布局为准。
|
||||
|
||||
沿用 [文档原生开发指导](document-native-development-guide.md)、[可执行知识规范](../next/Marqdo_Executable_Knowledge_Development.md) 与 [View 阅读体验设计](view-knowledge-reading-experience-2026-10-24.md)。本规格补充嵌入、调用身份和运行观测,不另造语言或第二套知识真相。
|
||||
|
||||
## 1. 框架目标与责任边界
|
||||
|
||||
让任何宿主应用可以按请求构造模型、追踪所有真实模型调用、以只读 View 展示知识文档,并从引用定位到版本一致的知识与来源。qdagent 的设置页、模型目录持久化、账单展示、会话导航、工作区授权和画像管理由应用实现。
|
||||
|
||||
| 能力 | Marqdo 拥有 | qdagent 拥有 |
|
||||
|---|---|---|
|
||||
| 多提供商 | 协议适配、模型 handle、能力协商、调用参数、归一化错误 | 提供商账户/端点/模型 CRUD、默认选择、凭证管理 |
|
||||
| 模型路由 | Runtime 执行显式选择和允许的辅助模型策略 | 用户选择、工作区模型允许列表、路由授权 |
|
||||
| 用量 | 每次请求/重试的身份、usage 来源与完整性、子任务传播 | 持久化账本、价格版本、费用估算、筛选/导出/预算策略 |
|
||||
| 实时运行 | 逐事件输出、取消传播、背压、终态 | HTTP 会话鉴权、轮次映射、持久化事件重放、断线 UX |
|
||||
| View | 文档语义、正文/目录/步骤、纯读取、稳定定位、可嵌入输出 | 容器/主题/深链/阅读历史/产品操作 |
|
||||
| 知识引用 | 解析/歧义/版本/来源定位、验证多维事实 | 消息引用标识、授权、历史资料保存与入口 |
|
||||
| 工作区 | 按显式 project 编译/查询,缓存及输出隔离 | 工作区 registry、root 解析、用户权限、文件管理 |
|
||||
| 画像与笔记 | 通用文档渲染、来源关联、验证/编译原语 | 内容策略、提案/确认、生命周期、展示和上下文选择 |
|
||||
|
||||
不在语言核心添加 provider/workspace/profile 关键字;沿用 `.mq.md` 函数、结构化参数、扩展插件和 MLSP。不得让“可读”取决于用户手填 hash、node id、source span 等机器字段。
|
||||
|
||||
## 2. 2.1.2 已完成的基础与剩余缺口
|
||||
|
||||
### 2.1 当前可复用基础
|
||||
|
||||
| 源码 | 已有内容 | 本规格的增量 |
|
||||
|---|---|---|
|
||||
| [ext/ai/llm](../../ext/ai/llm.mq.md) | `create(model,base_url,api_key,backend,name)`、命名 handle、ask/stream | 无需重做基础 handle;增加隔离策略、协议能力、完整消息与调用身份 |
|
||||
| [LLM plugin](../../plugins/llm/src/lib.rs) | 兼容 chat completions HTTP、SSE 文本读取、非流 usage 三项 | SSE usage/异常终态/每次调用观测、可取消持续输出 |
|
||||
| [Agent](../../ext/ai/agent.mq.md) | Agent 唯一 loop、router_model、EFI、绑定/发布门闩 | 主/路由/修复/生成工作簿的模型及计量上下文传播 |
|
||||
| [语义 IR](../../src/sema/ir.rs) | Artifact、KnowledgeStep、KnowledgeRef、SourceLocation、BoundCall | View 面向阅读的完整投影及 ID/版本映射 |
|
||||
| [MLSP](../../src/mlsp.rs) | `artifact.inspect`、`link.resolve`、verify/bind/execute、document.extract | 只读 document/引用解析及 capabilities 版本握手 |
|
||||
| [KnowledgePresentation](../../src/view/knowledge_present.rs) | schema v1,artifact nodes、依赖、generation/page token | 保留兼容,新增正文/目录/refs/source spans/结构化诊断 |
|
||||
| [View output](../../src/view/output.rs) | `--no-exec` 在构建前跳过 capture | 将纯读取能力暴露给嵌入客户端,不复用会执行的 live 默认路径 |
|
||||
| [document extract](../../src/document_extract.rs) | TXT/MD,partial,PDF/DOCX/OCR 显式不支持 | 能力查询及稳定来源定位;不要求本阶段重造 OCR |
|
||||
| [数据资源](../../ext/data/db.mq.md) | SQLite migrate/exec/事务、唯一/索引声明 | 应用可直接承载 registry/账本/索引;不需另加产品专用数据库框架 |
|
||||
|
||||
2.1.2 已把知识辅助块移到正文后并折叠,跳过 module/type 卡片,显示“结构步骤非实际执行时序”。旧 View 截图中的最初顺序问题已有改进;但这不意味着已提供能直接放进 qdagent 右栏的独立阅读 SDK。
|
||||
|
||||
### 2.2 直接影响集成的事实
|
||||
|
||||
1. `llm.complete_result` 当前仍走 `openai.chat_completions`,Ollama 也是兼容路径;`backend` 名称不是任意原生协议适配器。
|
||||
2. `plugins/llm::chat_completions` 从 `prompt` 构造一条 user 消息,没有完整消息列表契约;不能靠模型选择 UI 弥补历史丢失。
|
||||
3. `usage_from` 缺字段填零;Agent `model_turn(stream=True)` 直接 `tokens=0`,最终 `token_cost=tokens`。现有输出无法支撑准确货币或流式用量显示。
|
||||
4. 原生 LLM SSE parser 返回事件数组;`ext/ai` 有流事件机制,但应用所用 [web invoke](../../plugins/web/internal/httpx/routes_invoke.go) 在 `CallLibPath` 返回后才写 `{sse}`。需要打通实际增量路径,不能只换 Content-Type。
|
||||
5. Agent 生成子工作簿的模板仍能构造默认 `llm.llm`;只传主 agent handle 不足以证明嵌套调用沿用了用户选定模型。
|
||||
6. `build_file_view` 默认传 `execute=true`;live GET 路径使用它。嵌入阅读绝不能直接暴露这个入口。
|
||||
7. Presentation 目前按 artifacts 投影,缺全量 prose/heading/step/ref 的统一阅读树;`render_knowledge_html` 另从 steps/refs 渲染,容易与 JSON 事实发生覆盖差异。
|
||||
8. `source_digest` 当前由语义 artifact hashes 汇总,`semantic_generation` 是语义散列截短,`page_token` 来自原文件。它们不能直接分别冒充原文 revision、`.marqdo/CURRENT` generation 和跨请求完整版本。
|
||||
9. `sse_accepts_packet` 对缺 token 的 legacy 包放行;正式嵌入客户端必须采用严格版本匹配,而不是继承这项兼容宽容。
|
||||
10. `artifact.inspect/link.resolve` 每次走当前 semantic_project;现有入口不保证可以读任意历史 generation。历史引用需要明确的保留与解析合同。
|
||||
|
||||
## 3. 共用合同、版本与能力发现
|
||||
|
||||
合同名字固定为 `ModelSelectionV1`、`ModelCallUsageV1`、`RuntimeEventV1`、`DocumentViewV1`、`KnowledgeLocatorV1`;它们是这次扩展的 v1,与当前 KnowledgePresentation `schema_version:1` 不混用。新增 read model 不强行给已有 JSON 改含义。
|
||||
|
||||
新增 MLSP `capabilities`(拟议):
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"runtime_version": "implementation-version",
|
||||
"features": {
|
||||
"llm.explicit_selection": 1,
|
||||
"llm.call_usage": 1,
|
||||
"runtime.event_stream": 1,
|
||||
"runtime.cancel": 1,
|
||||
"view.document": 1,
|
||||
"view.reference_resolve": 1
|
||||
},
|
||||
"adapters": [{"id": "openai-compatible", "available": true}],
|
||||
"extract_formats": [{"format": "text", "available": true}],
|
||||
"loaded_plugins": [{"name": "llm", "capabilities": ["call_usage_v1"]}]
|
||||
}
|
||||
```
|
||||
|
||||
只返回实际探测且加载可用的功能。不能仅从 CLI 版本推断 native plugin 符号存在;字段未知时应用降级或解释不支持。协议允许新增可选字段;删除/改变状态语义必须提升 major contract version。能力查询不调用供应商、不执行项目代码。
|
||||
|
||||
MLSP 继续使用现有 `{id,ok,result|error}` envelope;qdagent `/api/app` 转为自己的 `{ok,schema_version,data|error}`,不要要求它们共享同一层 envelope。所有合同提供成功、partial、unknown、unsupported、stale、cancelled、错误 fixtures 与 JSON Schema;应用和框架从相同样本跑消费测试。
|
||||
|
||||
## 4. M01:请求级模型身份与适配器
|
||||
|
||||
### 4.1 显式选择合同
|
||||
|
||||
应用将配置解析成可信 `ModelSelectionV1` 后传入 runtime,核心字段:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"selection_id": "sel-1",
|
||||
"provider_id": "provider-1",
|
||||
"model_id": "model-2",
|
||||
"provider_revision": 3,
|
||||
"model_revision": 5,
|
||||
"adapter": "openai-compatible",
|
||||
"endpoint_ref": "endpoint-local-1",
|
||||
"credential_ref": "credential-local-1",
|
||||
"credential_version": 2,
|
||||
"upstream_model_id": "example-model",
|
||||
"parameters": {"max_output_tokens": 2048},
|
||||
"fallback_policy": "disabled"
|
||||
}
|
||||
```
|
||||
|
||||
endpoint/credential 引用由宿主 resolver 解析为调用局部配置;短期直接向现有 `llm.create` 传参数亦可,但这不是可序列化的 public snapshot。API key 不能出现在 events、debug JSON、生成工作簿、prompt、错误 URI 或持久化模型对象中。
|
||||
|
||||
当前显式 args 优先的能力保留;新增 strict selection 模式:字段缺失报错,不回退 process env/frontmatter。无凭证端点通过 `auth_mode:none` 显式表达,避免 falsy key 触发环境 fallback。legacy 命令行未传 selection 时仍可沿用原有 env 行为;本地便利兼容不扩散到应用请求。
|
||||
|
||||
每个运行绑定 selection snapshot,后续设置改变不修改它。主模型与 router_model 分开但同属受控 selection 集;manual 模式所有需要 LLM 的步骤固定选择,REUSE 无模型调用仍允许。fallback 必须由宿主明确授予、列出允许目标,不能因额度/网络错误静默转到另一个供应商。
|
||||
|
||||
### 4.2 子任务和工作簿传播
|
||||
|
||||
统一 `RunContextV1 = run_id,parent_run_id?,turn_id?,workspace_scope,selection_context,usage_sink,event_sink,cancellation_token`。进入 ask/step/plan、repair、summary、learn、子工作簿时显式继承;新 run/call id 关联父节点。
|
||||
|
||||
生成工作簿只包含不可重放滥用的运行上下文引用,不写密钥;执行时宿主注入/解析。独立导出工作簿在缺凭证/选择时清晰要求配置,不偷偷取默认模型。在任何 child/spawn/resume 路径遗漏 selection 都应由 strict 模式拒绝并给出定位诊断。
|
||||
|
||||
同时审查环境变量不只是凭证:project root、temp request、输出目录、event sink 也必须请求局部,不能在并发运行中依靠进程可变状态串联。
|
||||
|
||||
### 4.3 适配器与消息
|
||||
|
||||
适配器最小职责:验证 selection、报告能力、序列化请求、解析增量结果、usage 归一化、取消传输、错误归一化。可选 discovery/probe;无模型列表 API 仍允许手工配置。
|
||||
|
||||
V1 消息包含 role 和 content parts,parts 最低 text,附件通过受控引用;tool call/result 要保持配对及 role。不支持的 part/parameter 返回 `llm.unsupported_input/parameter`,不能悄悄转成空字符串。适配器决定协议格式,Agent 决定行为;qdagent 不拥有供应商协议代码。
|
||||
|
||||
阶段一验证两个不同兼容端点,分别至少两个模型,以及无密钥本地兼容服务。阶段二增加原生非兼容协议:每个适配器需具名能力、流式/非流式/错误/取消/usage fixtures 和独立接入说明后才能在 capabilities 中置 available。不能把所有厂商默认走兼容接口后统称全支持。
|
||||
|
||||
统一参数采用 `max_output_tokens,temperature?,stop?,reasoning?` 等语义键,能力不支持时明确拒绝;不随意透传客户端未知参数。实际 upstream model 若与请求 id 不同,在 result 中保留 requested/resolved 两者。供应商版本与价格随时变化,本文仅锁定内部合同;实施原生适配器时另附当时官方协议核验记录。
|
||||
|
||||
## 5. M02:用量、事件流与取消
|
||||
|
||||
### 5.1 ModelCallUsageV1
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"call_id": "call-1",
|
||||
"attempt_id": "attempt-1",
|
||||
"parent_call_id": null,
|
||||
"run_id": "run-1",
|
||||
"turn_id": "turn-1",
|
||||
"workspace_scope": "ws-1",
|
||||
"purpose": "answer",
|
||||
"selection_id": "sel-1",
|
||||
"provider_id": "provider-1",
|
||||
"model_id": "model-2",
|
||||
"requested_model": "example-model",
|
||||
"resolved_model": "example-model",
|
||||
"status": "succeeded",
|
||||
"usage_state": "final",
|
||||
"measurement_source": "provider",
|
||||
"input_tokens": 1200,
|
||||
"output_tokens": 300,
|
||||
"total_tokens": 1500,
|
||||
"cached_input_tokens": 800,
|
||||
"reasoning_output_tokens": null,
|
||||
"revision": 2,
|
||||
"provider_request_id": "opaque-provider-id",
|
||||
"started_at": "2026-10-25T01:00:00Z",
|
||||
"finished_at": "2026-10-25T01:00:03Z"
|
||||
}
|
||||
```
|
||||
|
||||
字段要求:tokens 非负整数或 null;`usage_state = pending|partial|final|unknown`;`measurement_source = provider|estimated|none`。估算可作为单独 estimate 字段返回,不能覆盖实测字段。供应商没有报告时用 null/unknown,传输失败不能自动解释成零消耗。
|
||||
|
||||
`status = started|succeeded|failed|cancelled|interrupted`,status 和 usage_state 独立。每个真实发出的 HTTP attempt 一个唯一 id;预检拒绝只产生 call-rejected 事件,不伪造已发送请求。框架重试必须可观测:所有 attempts 包括失败/有部分输出的 attempt;重复 delivery 的 usage revision 不算新请求。
|
||||
|
||||
累计 SSE usage 在同 attempt 内按 revision 发布完整快照,不让上层猜是 delta 还是 total。provider 总量与 input/output 不一致时保留原始读数及 `usage.inconsistent` 诊断,不自行强制改账;细分字段要标 inclusion 关系以避免缓存、reasoning 与总量重复相加。只保留脱敏的必要 provider usage 扩展字段,不把完整原始响应当 public usage。
|
||||
|
||||
纯知识复用用 run summary 中 `llm_calls=0`、`measurement_source=none` 表示,而不是伪造一条耗费为零的模型请求。框架输出 `call_count,known_tokens,unknown_call_count`,宿主据此显示覆盖程度。
|
||||
|
||||
### 5.2 事件流合同
|
||||
|
||||
```text
|
||||
run.started
|
||||
→ model.resolved
|
||||
→ model.call.started
|
||||
→ message.delta / tool.progress / usage.updated(零到多次)
|
||||
→ model.call.finished(每个 attempt 各一条)
|
||||
→ document.produced / citation.produced(仅有真实对象时)
|
||||
→ run.completed | run.failed | run.cancelled | run.interrupted
|
||||
```
|
||||
|
||||
`RuntimeEventV1` 公共字段:`schema_version,event_id,seq,run_id,parent_run_id?,turn_id?,workspace_scope,timestamp,type,payload`。seq 每个 run 单调;不同子 run 可以并发,宿主按所属 turn 另分配产品流序号。runtime terminal 事件仅一次;后续补全计量用有 revision 的 `usage.updated`,不能改变内容终态。
|
||||
|
||||
事件 sink 支持有界队列和背压;慢客户端不能无限堆积 token。丢弃可合并 progress 时记录 gap;内容与终态事件不可无声丢失。连接终止不会自动重跑;持久化/重放由应用,但框架需在实时回调后仍提供可核对的 final result/call summaries。
|
||||
|
||||
同一真实调用所有数据使用同一 run/call identity,不能流与最终结果各生成一套 id。工具参数/结果按 policy 脱敏,不把 secret、完整私有文件、模型内部推理链写入默认进度事件。
|
||||
|
||||
### 5.3 打通传输而不重造 Agent
|
||||
|
||||
需要同时打通 LLM provider reader → Agent event sink → host callback → web streaming handler → 浏览器。现有 `sys.stream_publish` 可以作为底座,但要验证它是否绑定正确请求且在 `CallLibPath` 返回前可写出;不能用进程全局事件广播混流。
|
||||
|
||||
可实施路径:新增 web 流式 invoke/job primitive,由框架注册 run handle 和 event sink;调用返回 handle 后宿主可订阅/取消。采用插件 ABI 增量接口及能力检测,旧 ABI 未支持时拒绝新 streaming 模式,保留旧完整返回。qdagent 只挂产品鉴权、event id 持久化和 HTTP 端点,不添加另一个推理循环。
|
||||
|
||||
SSE parser 必须处理任意 TCP 分块、UTF-8 跨块、CRLF、多行 data、注释 heartbeat、尾帧、usage-only 帧、[DONE]、连接在结束前关闭。未知可选事件可忽略且记诊断;已知事件无效、异常截断不能静默当成功完成。公共 framing 不包含供应商私有认证信息。
|
||||
|
||||
### 5.4 取消与重试
|
||||
|
||||
取消 token 传到 HTTP reader、循环、工具边界、子任务。取消后不得启动新模型调用/新工具;正在进行的不可抢占操作返回 `cancelling` 和是否仍运行,结束后才报告 cancelled。明确“停止生成”不能撤销已经完成的外部写入。
|
||||
|
||||
调用超时、连接错误、鉴权失败、限流、协议错误分开;错误字段包含 `code,retryable,retry_after_ms?,call_id,attempt_id,usage_state`。不要将原始包含密钥/完整 prompt 的错误直接传到 UI。限流/短暂网络错误按有界策略重试;已产生输出的重试不能接在同一条文字后假装连续,也不能把未知结果的外部工具操作自动重做。
|
||||
|
||||
费用价格由应用管理。框架保留 legacy `tokens/token_cost` 并标注 deprecated/单位为 token,新功能只消费 ModelCallUsageV1,避免破坏旧脚本的同时停止误用。
|
||||
|
||||
## 6. M03:无执行的嵌入式 View
|
||||
|
||||
### 6.1 新入口与纯度约束
|
||||
|
||||
新增内部公共服务 `analyze_document → project_read_model → render_fragment`;MLSP 暴露 `view.document`,可选 CLI `view document --json --no-exec` 与它共享实现。不要让新入口先进入 `build_file_view` 再隐藏按钮/清空输出。
|
||||
|
||||
读取允许:授权范围内原文/已编译快照/已有验证记录。读取禁止:运行模块入口、加载文档声明插件、执行外国代码、LLM、自动抓取 URL、写回、重新验证执行测试、隐式发布/重编译写盘。静态分析/内存投影允许;缺完整依赖时明确 partial。任何需要执行的行为必须另走显式 execute/verify 产品流程。
|
||||
|
||||
当前 `compile_project` 声明不加载插件/不求值/不抓取来源,可复用;缓存写入若需要,作为单独的宿主管理路径,不能把“读文档”隐含写项目目录。
|
||||
|
||||
### 6.2 DocumentViewV1 结构
|
||||
|
||||
请求由宿主传入受控 project 和文档定位,宿主必须已授权;框架仍做路径边界校验。
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "view.document",
|
||||
"params": {
|
||||
"project": "/authorized/project",
|
||||
"document": "knowledge/example.mq.md",
|
||||
"revision": "sha256:requested-source",
|
||||
"generation": "generation-id-or-null",
|
||||
"mode": "read",
|
||||
"node_id": "optional-node",
|
||||
"include": ["fragment", "outline", "nodes", "refs", "diagnostics"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "DocumentViewV1",
|
||||
"schema_version": 1,
|
||||
"document": {
|
||||
"id": "project-relative-document-id",
|
||||
"title": "知识标题",
|
||||
"source_revision": "sha256:raw-bytes-digest",
|
||||
"semantic_digest": "sha256:semantic-facts",
|
||||
"generation_id": null,
|
||||
"presentation_token": "opaque-token"
|
||||
},
|
||||
"execution": {"state": "not_run", "run_id": null},
|
||||
"render": {"format": "sanitized_html", "fragment": "<article class=\"mq-document\">…</article>", "renderer_version": "1"},
|
||||
"outline": [],
|
||||
"nodes": [],
|
||||
"refs": [],
|
||||
"diagnostics": [],
|
||||
"completeness": "complete",
|
||||
"capabilities": {"read_only": true, "has_semantics": true}
|
||||
}
|
||||
```
|
||||
|
||||
`generation_id` 来自实际编译投影;即时分析但未编译发布时为 null,同时给 semantic_digest。`source_revision` 是原始文件字节摘要,不能复用当前基于 artifacts 的 `source_digest`;`presentation_token` 覆盖 source revision、generation/semantic digest、依赖版本、renderer/schema、读取策略与请求范围。应用缓存还必须含用户授权范围。
|
||||
|
||||
`completeness = complete|partial`;语法损坏时允许返回原文与结构化诊断,但不能展示猜测出来的正常知识图。失败用 error envelope,不能空字符串伪装“文档无内容”。大文档 response 支持 section cursor 和 truncated 标识,限制要可配置并在响应中标明。
|
||||
|
||||
### 6.3 统一正文、目录、节点与引用
|
||||
|
||||
节点包含 `id,parent_id,kind,label,order,source_span,artifact_id?,ref_ids[],children[]`。kind 包括 heading/prose/table/code/observe/derive/call/iterate/narrate/navigation;正文树来自 parser/classifier/sema 的共同投影,不能只枚举 functions 后把每个步骤标 call。
|
||||
|
||||
outline 描述读者文档层级,不简单把函数和对象全铺成同一级。HTML 中节点带 `data-mq-node`,引用带 `data-mq-ref`;实现 payload 按需展开,说明作为正文。动态调用/无法解析引用保持原文并解释原因,不虚构执行边。
|
||||
|
||||
首批 renderer 共享 `src/view/document.rs` 的文档表面,standalone/full page 和嵌入模式调用同一 read model;只改变容器和导航外观,不产生第二种语义解释。替换当前 `KnowledgePresentation` 与 `render_knowledge_html` 双路径重复组装。
|
||||
|
||||
稳定 ID 的保证范围要写明:
|
||||
|
||||
- 同一 revision 再次读取,node/ref id 必须相同。
|
||||
- 仅前插空行/叙述时,已有命名符号保持 symbol id;步骤 id 由结构身份映射,不能只靠行号。
|
||||
- 同一行多个节点需要 column/offset/span 区分,不能都映射成 `stmt-N`。
|
||||
- 匿名步骤重排/重复同名内容可能无法唯一映射;返回 unmapped/ambiguous,不能声称任意编辑下永恒稳定。
|
||||
- source span 明确 1-based line/column 和 UTF-8 byte offset;JavaScript UTF-16 偏移不直接当 Rust 字节偏移。
|
||||
|
||||
旧 `#stmt-{line}` 可保留为当前版本兼容锚点,但不得作为跨版本唯一身份。
|
||||
|
||||
### 6.4 嵌入渲染、主题与访问性
|
||||
|
||||
fragment 不包含 html/body、应用导航、运行按钮、内联可执行 script、全局 CSS;CSS 以 `.mq-document` namespace 限定。主题暴露 `--mq-bg/surface/text/muted/border/accent/font-body/font-code`,宿主映射自己的设计 tokens;不把独立 View 的全局 `*`/body 样式注入 qdagent。
|
||||
|
||||
首批推荐 fragment + 结构化引用 + 宿主事件代理;无需为 iframe 和 React 各重做 parser。可选 iframe embedded shell 必须只读、无重复导航,宿主/子页通信校验 origin/source/nonce/schema,仅允许 open-reference/selection/ready 等已定义消息,不能从内容发任意运行指令。
|
||||
|
||||
所有文本/属性转义,允许的 Markdown HTML 白名单净化;链接 scheme 限制;外部资源不默认跨网加载;代码/公式不得执行。静态导出、独立页面、嵌入三者都测试恶意文本/闭合标签/危险 URL。不要把 KnowledgePresentation 内 JSON 的转义当作所有正文 XSS 防护。
|
||||
|
||||
标题/列表/表格用语义 HTML;说明链接有可访问名称,焦点可见,视觉排序和键盘顺序一致。折叠只隐藏技术细节,不隐藏理解结论所需的正文。320–720px 容器宽、暗色、200% 缩放和 print 都要能读,不出现遮住正文的关系连线。无 JS 时仍能读正文和普通链接。
|
||||
|
||||
## 7. M04:知识、来源与历史引用定位
|
||||
|
||||
### 7.1 KnowledgeLocatorV1
|
||||
|
||||
统一 locator:`project_id,document_id,source_revision,generation_id?,artifact_id?,node_id?,source_ref_id?,span?,relation`。relation 区分 cites/supports/derived_from/executes。普通链接导航、NarrativeRef 和 BoundCall 都能定位,但语义不同;点击说明永不成为 BoundCall 的执行。
|
||||
|
||||
新增 `view.reference_resolve`:接受 locator 或当前文档 ref_id,返回 `resolution = resolved|stale|unresolved|ambiguous|version_unavailable|unsupported`、目标 locator、候选(若有)、定位精度、诊断、读者标签。框架内部路径不作为浏览器 href,应用根据授权映射为 document/source route。
|
||||
|
||||
扩展 `artifact.inspect` 的版本参数,旧无版本调用仍查当前;新调用带 expected source/generation mismatch 时不能自动降到当前。来源读取接口可由宿主取实际文件,再调用框架 locator 映射;框架不自动 fetch 外部网页。`mq://` 必须按框架规则解析,不能交给浏览器或把 URI 直接当文件路径。
|
||||
|
||||
### 7.2 历史版本与保留责任
|
||||
|
||||
生成 artifact/generation 元数据不等于保存了原文历史。框架须输出可保存的 snapshot manifest:原文 revision、依赖 revision、source refs、artifact hashes、renderer/schema version;应用决定哪些源版本随引用保留及保留周期。已有 Git 或内容寻址存储可提供 snapshot resolver;不强制应用部署 Git。
|
||||
|
||||
source revision 定位到存储快照后,框架基于该快照解析;缺失任何必要版本时返回 version_unavailable/partial,不读取最新文件填充旧快照。如果只留 artifact 元数据、原始资料已删除,仍可展示历史元数据但明确“原文不可读取”。
|
||||
|
||||
知识改名、移动或 source anchor 改变可返回候选映射,只有唯一可靠匹配才自动导航;否则用户选择最新版/候选。保持旧引用与新引用两个对象,防止历史回答证据在后台悄悄改变。
|
||||
|
||||
### 7.3 验证和运行事实
|
||||
|
||||
节点详情至少携带 purpose/description、signature、applicability、provenance、effects、dependencies,以及 parse/resolve/contract/tests/evidence/execution 各自状态与记录来源。sourced 只能说“已关联来源”;static verify 成功不表示执行成功或适用业务正确。
|
||||
|
||||
真实执行证据独立结构 `run_id,artifact_digest,closure_digest,source_revision,generation_id,inputs_digest,executed_at,result_state,evidence_ref`,展示时匹配版本。旧运行结果不涂成当前文档的绿色通过。知识复用必须存在实际 bind/execute trace,不能凭 find 命中显示“已执行”。
|
||||
|
||||
View 不运行验证测试以填满状态。若用户点“验证”,应用走已有 verify/policy 门禁并记录新 evidence,之后读取才刷新详情。
|
||||
|
||||
### 7.4 错误与授权协作
|
||||
|
||||
| 内部结果 | 应用建议呈现 | 不能做 |
|
||||
|---|---|---|
|
||||
| unresolved | 无法定位该知识,可读原引用 | 跳转到第一个同名 artifact |
|
||||
| ambiguous | 显示已授权候选和位置 | 自动选择或暴露不可见候选 |
|
||||
| stale | 该知识/来源已变化,提供版本选择 | 静默替换旧引用 |
|
||||
| version_unavailable | 回答所用版本不可用,可查看最新版 | 用最新版伪装历史证据 |
|
||||
| partial | 部分依赖或来源缺失,说明范围 | 默认绿色通过 |
|
||||
| permission denied(应用) | 无权访问 | 在错误中泄漏标题、路径、片段 |
|
||||
|
||||
用户权限归应用;框架服务在调用边界接受 scoped project/source resolver,不把传来一个 root 视为可遍历整个服务器。所有路径经过 canonicalize/symlink 检查;缓存的语义事实可共享,带原文的响应必须按权限复核,不能只靠 artifact id 缓存授权。
|
||||
|
||||
## 8. M05:工作区、笔记与导入所需的通用底座
|
||||
|
||||
不为 qdagent 画像加语言特殊节点。通用能力应支持普通文档元信息读取、来源范围、内容修订、diff/patch 的 expected digest、提取格式 capability。
|
||||
|
||||
document.extract 继续使用 DocumentSource→ExtractedDocument,报告 format/status/segments/source locator/diagnostics。TXT/MD 可支持;PDF/DOCX/OCR 没有适配器时明确 unsupported/failed,不能将空文本当成功。partial 保留已提取片段及原文范围、遗漏原因;发布政策由应用决定。
|
||||
|
||||
同一资料可产生多个知识单元,provenance 必须到具体片段,不能所有函数只指向文件根。原始资料 digest 改动的失效由现有 provenance/closure 保证;应用用户画像也可用普通来源引用,但不自动进入公共知识发布。
|
||||
|
||||
ProjectRoot/CompileOptions 显式传递,工作区 A/B 的 source id、输出 generation、缓存、临时目录必须隔离。避免每读一个节点就全项目编译:引入按 project snapshot 复用的分析服务,针对来源/依赖变化失效;并发同 snapshot 请求合并,不能分别写同一个 CURRENT。
|
||||
|
||||
上传 multipart/大小限制已有 web 底座,框架只提供安全提取和可消费结果;浏览器上传、作业生命周期、发布确认、权限和历史保存仍在 qdagent。多文件任务桥接采用结构化参数,禁止拼接用户文件名进 shell/JSON 或共用固定请求临时文件。
|
||||
|
||||
## 9. 实现落点与兼容策略
|
||||
|
||||
| 模块 | 改动方向 | 兼容要求 |
|
||||
|---|---|---|
|
||||
| `plugins/llm/src/lib.rs` | adapter boundary、消息数组、usage、event sink、取消 | 保留已有符号;新能力有 probe 和合同版本 |
|
||||
| `ext/ai/llm.mq.md` 与 `ext/ai/llm/*` | strict selection、结构化 result、public snapshot | 无 selection 的旧调用仍可运行;不得回显密钥 |
|
||||
| `ext/ai/agent.mq.md` | run context、子工作簿传播、逐调用观测 | 不复制 Agent loop,EFI/verify/publish_gate 不被绕过 |
|
||||
| `plugins/agent/src/*` | 子任务/恢复上下文、cancel、event identity | resume 不重发已结算请求;不可恢复说明 interrupted |
|
||||
| `src/host/*`、插件 SDK、web invoke | 请求局部 event/cancel 桥接 | ABI 旧宿主不伪支持;并发请求不混流 |
|
||||
| `src/view/knowledge_present.rs` | DocumentView read model/兼容 adapter | 旧 presentation v1 不被新字段语义污染 |
|
||||
| `src/view/document.rs,render.rs,html.rs,mod.rs,output.rs` | 共享纯读取、fragment、scope CSS、定位 | live/debug 与 read 明确分离;旧 no-exec 门禁保留 |
|
||||
| `src/mlsp.rs` | capabilities、view/document/reference、版本 inspect | 使用现有 envelope;未知 feature 可探测 |
|
||||
| `src/sema/ir.rs,provenance.rs` | span、snapshot/locator、依赖与来源失效 | 不改变核心语法,不把读请求变为执行 |
|
||||
| `src/document_extract.rs` | capability 和片段定位完整性 | 保留不支持格式的显式失败 |
|
||||
|
||||
测试分层对应 plugin HTTP fixtures、Agent 假模型、MLSP read contracts、web 持续流集成、独立/嵌入 DOM 和浏览器交互。应用共享样本文件必须注明 schema 版本;两仓 PR 在各自文档索引相互链接对应实现说明。
|
||||
|
||||
迁移顺序为新增合同/接口 → 旧输出 adapter → qdagent 按 capability 接入 → 切默认 → 后续版本再考虑弃用。不要同时移除 legacy 行为且要求应用一次完成全链升级。发布包对 CLI/ext/native 插件做组合测试,缺插件给功能不可用诊断而非 unknown function 直到用户点击才暴露。
|
||||
|
||||
## 10. 交付切片与验收门槛
|
||||
|
||||
| 切片 | 前置 | 验收输出 | 支撑应用任务 |
|
||||
|---|---|---|---|
|
||||
| M00 合同冻结/探测 | 无 | schemas、fixtures、capabilities、版本和插件清单 | Q00、所有升级探测 |
|
||||
| M01 多模型隔离 | M00 | 兼容适配器、strict handles、history、子任务选择一致 | Q01/Q02 |
|
||||
| M02 调用观测与流 | M01 | 每 attempt usage、真流式、取消、错误/重放一致 | Q03/Q06 |
|
||||
| M03 只读嵌入 View | M00,可与模型线独立 | DocumentViewV1、fragment、目录/节点、无执行证明 | Q04 |
|
||||
| M04 引用与快照 | M03 | 历史 locator、来源详情、版本失效/歧义 | Q05 |
|
||||
| M05 文档工作区底座 | M03/M04 + 已有 extract | 范围隔离、提取定位、并发与缓存 | Q07/Q08 |
|
||||
| M06 集成与性能 | 全部所选切片 | 旧门禁 + 新 contracts + 浏览器与负载报告 | Q09 |
|
||||
|
||||
M01 支持多个兼容提供商不等待原生协议全做完;但原生适配器需另列完成状态。M03 不等待计费系统。M04 未完成历史快照前,应用可以显示当前文档,但必须把历史引用标“版本不可用”,不能宣布引用链闭合。
|
||||
|
||||
每个切片的 Definition of Done:实际接口可调用、失败合同覆盖、无副作用/隔离测试、文档示例由测试消费、版本能力可探测、qdagent integration fixture 通过。只有可运行的验收证据才可将 proposed 改为 verified。
|
||||
|
||||
## 11. 框架测试矩阵
|
||||
|
||||
| 编号 | 用例 | 断言 |
|
||||
|---|---|---|
|
||||
| F01 | 两端点各两模型、并行 A/B、无密钥本地端点 | 请求 payload/端点/鉴权正确,环境不串扰 |
|
||||
| F02 | step/plan/router/repair/child/resume | strict selection 和 parent identity 完整,不退回默认模型 |
|
||||
| F03 | multi-turn、system/tool roles、超限/不支持模态 | 顺序和边界完整,不吞最新 user,不静默降级 |
|
||||
| F04 | 非流/流式/usage-only/缺 usage/错误后部分 usage | null 与 0 区分,终态和用量独立 |
|
||||
| F05 | 相同累计 usage 多次、重放、新重试 | 同 attempt revision 覆盖,不重复相加;真重试新增 attempt |
|
||||
| F06 | 慢供应商、慢消费者、断连、取消、超时 | 完成前有 delta;队列有界;取消不启动后续工具/模型 |
|
||||
| F07 | 外部工具已写后取消、恢复未知状态 | 不声称撤销,不自动重做可能有副作用的动作 |
|
||||
| F08 | 带写文件/网络/foreign/插件的文档被阅读 | read API 不加载/运行这些效果,无项目写回 |
|
||||
| F09 | prose/heading/6 类步骤、同一行多引用、中文 span | HTML/outline/节点/ref 同源,定位唯一准确 |
|
||||
| F10 | 仅说明变化、依赖变化、旧 SSE 包、无 token | token 正确失效,嵌入 strict 拒绝过期/无身份包 |
|
||||
| F11 | 来源改删、历史 generation 未留原文、同名歧义 | 明确 stale/unavailable/ambiguous,无猜测替代 |
|
||||
| F12 | no-js/390px/暗色/长表格/键盘 | 正文可读,可返回,有清晰标题和焦点 |
|
||||
| F13 | 原始 HTML/危险 URL/闭合 script/越界路径 | 无脚本执行/越权读取/泄漏;不靠应用猜净化 |
|
||||
| F14 | 两项目同相对路径/并发 100 读取 | scope 缓存隔离,同 snapshot 合并分析 |
|
||||
| F15 | 编译成功但验证/证据未知、旧执行记录 | 各状态独立;不把 static/sourced 标 verified |
|
||||
| F16 | CLI/ext/native 版本错配 | capability 早发现不可用,而非运行时静默失败 |
|
||||
|
||||
性能目标需实测而非发版文字:固定机器与 release/debug 分开记录,1000 单元分析冷/热、单文档 fragment 体积/时间、并发读取 P95/内存、UI 首屏。应用目标见 Q 文档;如果全量 compile 超时,优先分析重复全库扫描、依赖计算与缓存,不通过删除 fixture 或抬高阈值宣布可用。
|
||||
|
||||
## 12. 本次验证结果与已知阻碍
|
||||
|
||||
### 12.1 M00–M06 落地证据(2026-10-25 续)
|
||||
|
||||
命令(本机):
|
||||
|
||||
```bash
|
||||
cargo test --offline --locked --test workbench_contracts
|
||||
cargo test --offline --locked -p marqdo_plugin_llm
|
||||
cargo test --offline --locked --test executable_knowledge f1_agent_eng_bind_gate_mq_offline -- --exact
|
||||
```
|
||||
|
||||
已交付并通过自动化子集(**非** F01–F16 全绿):
|
||||
|
||||
| 切片 | 证据 |
|
||||
|---|---|
|
||||
| M00 | fixtures + JSON Schema 消费;`capabilities`;`force_unloaded` / `force_*_off`(F16) |
|
||||
| M01 | strict selection;`RunContextV1` 并行不串扰;llm plugin messages/roles;workbook 禁 `llm.llm`;**F02** `m01_f02_agent_step_selection_and_resume`(mock HTTP + `llm.create` 抗 env poison + resume save/load) |
|
||||
| M02 | usage null≠0;revision 替换(F05);cancel token(F06/F07);event_sink;web `sse-live` 断连 cancel |
|
||||
| M03 | `view.document`;CJK outline/nodes;path escape;静态 a11y 子集;F08 无副作用标记;**浏览器** `workbench-docview.html`(`article.mq-document`、0 script、无 `javascript:`) |
|
||||
| M04 | stale / version_unavailable / **ambiguous**;`SnapshotManifestV1` 输出(retention=application);`artifact.version_mismatch`;embed strict SSE |
|
||||
| M05 | extract `:B`;analysis_cache begin/finish;**F14 并发 100** 同 snapshot + 跨 project 隔离 |
|
||||
| M06 | CI 预构建 agent+llm + `workbench_contracts`;Go `drainForward` / cancel-token 桥 |
|
||||
| Live | `tests/workbench_live.rs`:USTC OpenAI-compatible(`OPENAI_*` env only)`qwen3.8-chat` / `deepseek-flash` selection ask → `pong` + usage;`MARQDO_EXT` 指向仓内 `ext/` 避免 `~/.marqdo/ext` 陈旧遮蔽 |
|
||||
|
||||
已知限制(发版前须书面接受或继续补):
|
||||
|
||||
- F02 repair/plan **全路径**假模型仍薄于 step+resume;RunContext 三级继承 + skeleton 禁 `llm.llm` + 子任务 env 已有。
|
||||
- 历史正文由应用按 SnapshotManifest 保留;框架缺快照 → `version_unavailable`(不静默降级)。
|
||||
- 完整宿主嵌入 F12 / 产品鉴权留给 qdagent;原生适配器 / PDF/OCR 仍 unavailable。
|
||||
|
||||
### 12.2 历史基线记录(设计起草时)
|
||||
|
||||
起草时 `executable_knowledge` 曾出现 `f1` 缺插件与 `v3_corpus_1000` debug 超时;其后 `ensure_agent_plugin_built` + CI 预构建已修复 f1 路径。1000 单元性能仍按剖面单独记录。
|
||||
@@ -0,0 +1,463 @@
|
||||
# View 可执行知识阅读体验重设计
|
||||
|
||||
| 项目 | 说明 |
|
||||
|---|---|
|
||||
| 日期 | 2026-10-24 |
|
||||
| 状态 | 详细设计建议,待实施;不是已交付能力声明 |
|
||||
| 审计基线 | `b8a4ddd` / Marqdo 2.1.1;使用当前源码构建的 debug CLI 复核 |
|
||||
| 输入 | 用户提供的 view 截图:Executable Knowledge 在 Structure 前,Steps、Target、Kind、Signature、Effects、Verification、Lock digest 等字段大量铺开 |
|
||||
| 范围 | Live view、静态导出、正文结构、知识说明、运行结果、来源与状态呈现;不改语言语义,不增加作者必填的展示 DSL |
|
||||
| 关联 | [文档原生开发指导](document-native-development-guide.md)、[View 总体设计](view.md)、[文档面与标注](view-document-annotations.md)、[求道相关框架缺口](../roadmap/qdagent-knowledge-support-gaps-2026-10-24.md) |
|
||||
|
||||
## 1. 设计决策
|
||||
|
||||
**View 首先呈现知识文档本身,结构帮助读者理解和定位,机器信息解释当前所读内容。** 正文、程序结构、知识关系与执行证据应由同一份源文档关联起来,而不是顺序展示几份互相脱节的报告。
|
||||
|
||||
确定以下方向:
|
||||
|
||||
1. 移除正文之前整块展开的 Executable Knowledge。主阅读流首先出现作者的标题、介绍和内容;不把完整机器报告简单搬到正文后继续平铺。
|
||||
2. 保留并改进现有 Structure 的叙述保真文档面,面向读者命名为“文档”。正文、表格、公式、函数和可执行步骤在原位置阅读,不生成一份模型改写的平行说明书。
|
||||
3. 右侧“本文结构”负责章节与知识单元定位;选中知识片段后,在同一辅助栏展示它的解释、输入输出、来源和验证依据。源码标识、摘要与完整协议放在可展开的技术详情中。
|
||||
4. 运行结果就地关联到对应步骤,明确运行实例与输入;完整日志、运行控制和源码放在正文后。看见一个数值时,读者必须能知道它是文档声明、计算结果还是历史快照。
|
||||
5. 静态分析、程序运行、测试断言、来源关联和领域确认分别表达。不能用一个 `ok`、`pure` 或绿色“已验证”覆盖不同事实。
|
||||
6. 功能渐进增强:无 JavaScript、静态导出、移动端、打印时仍能读完文档、定位知识和查看必要依据。普通短文不变成复杂仪表盘。
|
||||
|
||||
这些决策恢复 [view.md](view.md) 原有“文档面为主、Browse 与 Debug 分工”的方向,同时修订过度依赖引出线和默认展开技术面板的呈现方式。
|
||||
|
||||
## 2. 现状审计:为什么截图难读
|
||||
|
||||
### 2.1 可定位的实现依据
|
||||
|
||||
| 观察 | 当前代码 | 对阅读的影响 |
|
||||
|---|---|---|
|
||||
| 知识面板位于正文前 | [html.rs / page_file](../../src/view/html.rs) 的模板顺序为 notebook → knowledge → Structure → Execution → Source | 必须先越过辅助资料才能看到文档;模块越多越严重 |
|
||||
| 所有制品一律展开 | [render.rs / render_knowledge_html](../../src/view/render.rs) 按文件筛选 artifacts,并逐项输出完整 `<dl>` | module/type/function 同等突出,容器元信息挤占知识内容 |
|
||||
| Steps 脱离所属结构 | 同函数把当前文件所有 steps 放在单一 `<ol>` 中,展示 kind、label、完整目标 ID | 静态步骤容易被误读为实际执行顺序;调用/循环/正文引用混在一起 |
|
||||
| 内部字段未经阅读转译 | URI、签名、effects、verification 拼接文本、完整 sha256 默认可见 | 读者仍不知道这项知识解决什么问题、怎样用、依据在哪里 |
|
||||
| “Lock digest”名实不符 | 该字段实际显示 `artifact.hashes.artifact`;[mod.rs](../../src/view/mod.rs) 编译使用 `LockMode::Off` | 制品摘要被误读为已经锁定且通过复现检查 |
|
||||
| 静态说明与运行脱节 | 每张卡固定输出 `Not executed by knowledge preview.`;实际 run capture 在另一条路径 | 即使页面有运行结果,也很难理解“未执行”到底指什么 |
|
||||
| 分析失败静默消失 | `build_file_view` 对语义编译结果使用 `unwrap_or_default` | 无知识、分析失败、尚未分析在界面上都可能表现为空 |
|
||||
| 标题与层级表达较弱 | `page_file` 标题取文件名;[document.rs / render_fun_heading](../../src/view/document.rs)统一用 h3 | 作者 frontmatter 标题未用于页标题,嵌套结构主要靠其它线索判断 |
|
||||
| 知识组件缺少专用布局 | `knowledge-*` 在 HTML 中有类名,但 `html.rs` 没有与面板匹配的专用样式规则 | 大量使用默认列表/定义列表排版,空白多、关系弱,和正文主题缺少一致约束 |
|
||||
| 新旧皮肤叠加 | `html.rs` 内基础 view CSS 与 `notebook_ui` 大块 notebook 主题共存 | 正文、注释、知识区的字号和密度不一致;不能只修一个卡片颜色 |
|
||||
| 验收主要看元素存在 | [executable_knowledge.rs](../../tests/executable_knowledge.rs) 的 view 用例检查 step 类名、kind 和 `#stmt-` 链接 | 没检查正文优先、跳转目标存在、可读文案和实际布局 |
|
||||
|
||||
已有能力应保留:[document.rs](../../src/view/document.rs) 的叙述/表格保真、Occlusion 描述与实现分离、函数大纲、调用帧/循环帧选择、[annot.rs](../../src/view/annot.rs) 的真实执行标注。重新设计应基于这些能力,而不是退回 AST 卡片浏览器。
|
||||
|
||||
### 2.2 对截图的解释边界
|
||||
|
||||
截图可直接确认:机器字段先于正文、Steps 只有 `call print`、长 URI/摘要显著占位、英文状态串缺少解释,以及深色底上的孤立黑色短线。
|
||||
|
||||
黑色短线的具体来源尚未在浏览器中复现。代码中存在 annotation leader、布局重算和主题覆盖,可能需要检查,但本文件不把它们断言为唯一根因。实施时应使用用户截图对应页面复现,检查实际 DOM、computed style 和几何位置;不能用全局隐藏所有 SVG/边框来掩盖问题。
|
||||
|
||||
### 2.3 本轮实际复核
|
||||
|
||||
把 [public/features/01-phase-i-treewalk.mq.md](../../public/features/01-phase-i-treewalk.mq.md) 复制到临时目录,使用 debug 2.1.1 执行静态导出并检查生成 HTML:knowledge 确实在 structure 前,共 3 个 `knowledge-card`(含 Steps)和 2 个完整 sha256。该短文的核心只是介绍 tree-walk 流程并打印一句话,当前辅助内容明显过重。
|
||||
|
||||
另用隔离临时文件复核 `view output --no-exec`:嵌入的 `fs.write_text` 仍创建了临时标记文件。原因是 [output.rs / write_static](../../src/view/output.rs) 先调用会执行的 `build_file_view`,再在 `opts.no_exec` 分支清空结果。**这是已复现的执行/呈现不一致,必须修复,不能仅把标签改成“未运行”。** §9 给出新的构建边界。未执行线上浏览器像素比对,本文线框为目标设计。
|
||||
|
||||
## 3. 读者任务与设计成效
|
||||
|
||||
| 读者任务 | 默认应看到 | 按需展开 | 成功判据 |
|
||||
|---|---|---|---|
|
||||
| 理解这篇文档 | 作者标题、介绍、正文和核心表格/步骤 | 完整源码 | 不滚过机器报告就能开始阅读 |
|
||||
| 看懂文档结构 | 章节、类/规则/函数的真实层级,当前位置 | 参数和局部步骤树 | 能判断一个步骤属于哪个知识单元 |
|
||||
| 理解某项知识 | 描述、简洁公式/调用、条件、返回含义 | payload、契约、精确目标 | 能解释它做什么、需要什么、产生什么 |
|
||||
| 判断结果是否可信 | 来源状态和所选运行的明确结果 | 测试、来源片段、版本和诊断 | 能区分“可以运行”和“事实成立” |
|
||||
| 跟踪跨文档复用 | 友好目标名、所属文档、输入输出 | 依赖关系、完整标识 | 能到达定义并返回原阅读位置 |
|
||||
| 检查某次运行 | 本次输入、选中调用/迭代、对应结果 | 日志、历史帧 | 不把不同运行的值混在一起 |
|
||||
|
||||
不要求所有读者理解 SymbolId、IR、hash 或 effect 协议。开发者仍能找到这些信息,但它们不再成为阅读的前置条件。
|
||||
|
||||
## 4. 页面信息架构
|
||||
|
||||
### 4.1 默认桌面布局
|
||||
|
||||
```text
|
||||
┌ Marqdo 项目 / 文档路径 搜索 主题 ┐
|
||||
├───────────┬───────────────────────────────┬────────────┤
|
||||
│ 文件目录 │ Phase I — tree-walk interpreter│ 本文结构 │
|
||||
│ │ Lex · parse · check … │ 介绍 │
|
||||
│ 当前文件 │ [运行记录:尚未运行] [详情] │ main 入口 │
|
||||
│ │ │ │
|
||||
│ │ Milestone: Phase I … │ │
|
||||
│ │ Pipeline: load → … → eval │ 选中后: │
|
||||
│ │ │ 当前知识 │
|
||||
│ │ main · 入口 │ 作用 │
|
||||
│ │ 输出说明 print text=… [说明] │ 输入/返回 │
|
||||
│ │ │ 来源/条件 │
|
||||
│ │ ▸ 运行与结果 │ 检查/运行 │
|
||||
│ │ ▸ 完整源码 │ 技术详情 ▸ │
|
||||
└───────────┴───────────────────────────────┴────────────┘
|
||||
```
|
||||
|
||||
布局是文件导航、主文档、一个辅助栏,不能增加第四条永久侧栏。右栏上部保留可收起的“本文结构”;用户选中片段后,下部显示“当前知识”。空间不足时以当前知识替换下部内容,并提供“返回目录”操作。打开/关闭详情不改变正文宽度,避免重排导致阅读位置跳动。
|
||||
|
||||
主文档的默认顺序:
|
||||
|
||||
1. 文档标题、已有 description、简短状态入口;title 缺失时退回文件名。
|
||||
2. 源文档的正文、结构和轻量就地知识说明,保持作者顺序。
|
||||
3. “运行与结果”:运行按钮、输入、所选运行结果、输出图形/日志;已有重要失败有就地入口。
|
||||
4. “完整源码”及“文档技术信息”,默认折叠。
|
||||
|
||||
不在顶部强制生成内容摘要、全量函数清单、巨大状态矩阵或知识关系图。没有 description 时不调用 LLM 补写。结构大纲始终可达,正文不因折叠辅助栏而变成无结构的文章。
|
||||
|
||||
### 4.2 尺寸与响应式规则(设计目标)
|
||||
|
||||
| 可用宽度 | 布局 |
|
||||
|---|---|
|
||||
| ≥1280 CSS px | 文件栏约 220–260px;正文弹性;辅助栏约 280–320px;正文中文行长约 32–44 字 |
|
||||
| 900–1279px | 文件栏可收起;正文优先,辅助栏可切换为有标题的侧抽屉 |
|
||||
| <900px | 单列文档;“目录”“当前知识”作为明确按钮打开抽屉或块内 details |
|
||||
| 320–390px 手机 | 标题、正文、状态不横向溢出;长表格/代码局部滚动,整页不左右拖动 |
|
||||
| 打印 | 正文展开,必要来源/验证摘要转为节后附注;交互按钮和导航隐藏;技术原始字段按导出选项包含 |
|
||||
|
||||
在 1440×900、100% 缩放的短文验收样例中,首屏应包含至少一段正文和首个主要结构单元;状态头不得因每个 artifact 增长而把正文推出首屏。此约束不截断作者自己的长标题或介绍。
|
||||
|
||||
### 4.3 普通文档与复杂知识文档采用同一布局
|
||||
|
||||
- 普通说明页只有 main/print:显示原文与输出操作,技术区保留模块信息;不强制展示“1 项可执行知识”卡。
|
||||
- 可复用函数库:目录列出函数;函数标题下出现紧凑签名和“说明”入口;即使无 main,也能正常阅读。
|
||||
- 事实/规则文档:来源、条件与验证紧邻相应知识;来源问题不能藏进页尾。
|
||||
- 教程/Notebook:保留源文档的代码单元与结果;全局单元运行条收进运行区域或显式工作模式,不能把所有文章永久变为 notebook 工具台。
|
||||
- 大文档:目录按真实层级和顺序组织,可搜索;正文不因知识数量膨胀为另一份报告。
|
||||
|
||||
## 5. 文档与结构呈现规范
|
||||
|
||||
### 5.1 标题、类与函数层级
|
||||
|
||||
页标题优先来自解析后的 frontmatter `title`,其次使用合适的源文档标题,再回退文件名;不要自行用正则重解析 YAML。保留原文标题内容,不擅自把技术名称改成模型生成的名称。
|
||||
|
||||
“本文结构”同时服务阅读和程序定位,但明确区分:
|
||||
|
||||
- 作者章节按实际文档关系展示。
|
||||
- 可执行的类/类型、函数、入口附轻量类型词,例如“规则”“函数”“入口”;类型词来自编译结果或已声明元数据,不由标题猜业务类型。
|
||||
- 同一节点既是章节又是函数时只有一个目录项,旁边标类型,避免两棵树重复。
|
||||
- 嵌套函数、分支、循环在对应父节点下展开;语义归属与 Markdown 可视层级分别保留,不把所有内容都扁平化为 h3。
|
||||
- 普通段落引用不提升为函数;module 容器不自动成为一张与业务函数同权的卡。
|
||||
|
||||
DOM 标题使用连续、可访问的层级;实际语法的 `#`/`##` 层数在源码面保留。用户复制源码时取得原始合法语法,而非为了 HTML 标题级别改写过的内容。
|
||||
|
||||
### 5.2 可执行步骤:就地解释并保留上下文
|
||||
|
||||
| 编译器步骤 | 默认阅读文案 | 展开后 |
|
||||
|---|---|---|
|
||||
| observe | “读取:商品单价” | 集合、取元表达式、字段说明、当前调用下的值 |
|
||||
| derive | “计算:商品小计” | 公式、输入来源、计算条件、当前结果 |
|
||||
| call | 优先作者标签;无标签时“调用 商品小计” | 目标、所属文档、实参映射、返回和运行记录 |
|
||||
| iterate | “逐项处理:订单商品” | 集合、循环体、迭代数、所选迭代 |
|
||||
| narrate | 正文保留作者描述,附“知识引用”入口 | 定义与来源;不当作已执行步骤 |
|
||||
| navigation | 标准文档链接 | 正常导航,不显示虚构的执行结果 |
|
||||
| 空 payload / stub | 作者描述 + “待实现” | 缺失的实现和已有诊断,不能展示成功数值 |
|
||||
|
||||
名称来自当前节点的 label、声明说明、目标可读名,依次回退;没有业务描述时就显示“调用 print”,不得自动声称它是“验证知识”或“输出最终结论”。普通 print 在所属文档位置解释输出作用,放在“输出”类信息中,不作为高价值知识卡突出。
|
||||
|
||||
“步骤”列表只在当前知识详情中显示,标为“文档中的步骤”或“结构步骤”。静态序号不能暗示实际执行时序;实际轨迹另按运行事件呈现,分支和循环保留嵌套。
|
||||
|
||||
### 5.3 描述、实现与结果的三层关系
|
||||
|
||||
以 `**总价 = [计算订单小计](pricing.小计(数量, 单价))**` 为说明示意:
|
||||
|
||||
```text
|
||||
计算订单小计 [说明]
|
||||
总价 = pricing.小计(数量, 单价) ← 简短实现,次一级视觉
|
||||
本次示例:数量 2,单价 30 → 60 ← 仅有匹配运行记录时出现
|
||||
```
|
||||
|
||||
作者的上下文句、返回说明、单位和条件仍在原位。短公式/实现可次级显示,长 payload 默认折叠到详情;显式“展开实现”控制可对当前知识单元生效。不得把所有代码完全藏掉,使“代码即文档”退化成只有自然语言标签。
|
||||
|
||||
复制操作区分“复制说明”和“复制源码”;源码来自源 span,保留必要定界符、参数和 import 依赖提示。显示为漂亮文本的结果不能被复制成伪造的可执行代码。
|
||||
|
||||
### 5.4 表格、条件与关系
|
||||
|
||||
数据表按作者结构呈现,表头固定语义,单位与列说明常显;契约表按输入/返回/约束组织。显示层不能把 Marqdo 列向集合悄悄换成行向记录;需要另一种预览时明确标为预览,并能查看真实形状。
|
||||
|
||||
前提条件与失败分支不得因“简洁”隐藏。默认显示全部分支结构,所选运行只突出实际命中的分支,继续遵守 [调用帧与结构保真](view-frame-selection.md)。
|
||||
|
||||
跨文档关系默认用可读链接,例如“使用:标准税率 · tax.mq.md”,而非 `mq://…`。直接依赖与被引用者各列前几项并可展开;可选局部关系图放在详情层,不作为默认阅读画布。关系图必须有等价文字列表,并显示未知/动态关系,不能补画不存在的边。
|
||||
|
||||
## 6. 当前知识详情:回答问题,而非列出字段
|
||||
|
||||
### 6.1 内容顺序
|
||||
|
||||
```text
|
||||
商品小计 函数
|
||||
根据数量与单价计算商品金额。 ← 仅引用已有描述
|
||||
|
||||
输入 数量:整数;单价:金额 ← 类型/单位未知则明确未知
|
||||
返回 金额
|
||||
条件 数量非负 ← 仅当已声明且可定位
|
||||
实现 数量 × 单价
|
||||
|
||||
来源 定价说明 / 第 2 节 [打开]
|
||||
检查 契约未检查;测试未运行 [查看依据]
|
||||
运行记录 示例 #2:2 × 30 → 60 [定位输出]
|
||||
使用 … [查看定义]
|
||||
|
||||
▸ 技术详情:精确标识、完整签名、效应、摘要、诊断 JSON
|
||||
```
|
||||
|
||||
此处“金额”“数量非负”等只用于解释布局,不宣称当前样例具有这些元数据。没有信息就省略非关键行或写“未声明”,不得推断成“无限制”“没有来源也可靠”。缺失影响可用性时常显;纯工具没有业务来源并不自动是错误。
|
||||
|
||||
### 6.2 当前字段的去向
|
||||
|
||||
| 当前字段 | 新的主要位置与文案 |
|
||||
|---|---|
|
||||
| Target / `mq://…` | 标题处可读定义链接;完整 ID 在技术详情,可复制 |
|
||||
| Kind / callable | 标题旁简短类型词;module 信息进入文档技术信息 |
|
||||
| Signature | 输入/返回紧凑表;完整规范签名可展开 |
|
||||
| Effects | “涉及的操作”,说明文件读写、网络等;未知明确显示,非纯不等于失败 |
|
||||
| Verification 长串 | 分项检查和证据表,见 §7 |
|
||||
| Lock digest | 更名“制品摘要”,技术详情内显示;锁状态单独来自真实 lock report |
|
||||
| Depends on | 可读目标列表,指向定义与引用位置;未分析不能写“无依赖” |
|
||||
| Execution 固定句 | 匹配运行记录,或“尚无该知识的运行记录” |
|
||||
| SourceRef / applicability | 当前遗漏的关键内容:放入“来源”和“适用条件” |
|
||||
|
||||
默认不铺完整 hash。技术详情可显示缩略摘要并提供全值/复制;缩略值不用于识别唯一性。没有读取锁报告时写“未检查锁定状态”,不能把拥有摘要显示成“已锁定”。
|
||||
|
||||
### 6.3 交互、跳转与返回
|
||||
|
||||
1. 点击正文的“说明”或目录知识项,选中同一 `node_id`;正文轻量高亮,右栏同步。键盘和触屏具有等价入口,不依赖 hover。
|
||||
2. 跳转到定义时保留来源文件、锚点、滚动位置和所选运行;返回恢复阅读位置。
|
||||
3. 文件、符号与语句不能只靠显示名称匹配。同名函数的详情显示所在文档和父结构。
|
||||
4. 默认不自动抢焦点、不因滚动不停替换用户正在阅读的详情。滚动只更新目录当前位置;点击选择才改变详情。
|
||||
5. 手机抽屉有标题、关闭、返回焦点和滚动管理;无 JS 时使用正文附近的原生 details 或页尾知识附注锚点。
|
||||
6. 当前 URI 不在导出范围内时显示“定义未包含在此次导出”,提供可读路径和复制引用;禁止输出必然失效的 `mq://` 或 live `/file` 链接。
|
||||
|
||||
## 7. 状态设计:读者必须知道什么被证明了
|
||||
|
||||
### 7.1 分离三个状态来源
|
||||
|
||||
| 状态来源 | 可以说明 | 不能说明 |
|
||||
|---|---|---|
|
||||
| 编译/静态分析 | 结构解析、目标绑定、契约检查结果、已建模效应 | 程序已运行、答案正确、资料真实 |
|
||||
| 运行记录 | 指定输入、环境、源码版本下的执行与输出 | 覆盖所有输入、测试通过、来源可信 |
|
||||
| 证据/验证记录 | 哪个断言/测试/来源核验通过,何时失效 | 未执行或未核验的其它部分也可靠 |
|
||||
|
||||
顶部仅给紧凑事实,例如“静态分析完成 · 尚无运行记录”,或“有 1 项来源问题 · 查看”。普通 unknown/unchecked 不应制造全页红色警报;与当前结论直接相关的失败、过期、未解析必须就地提示且可定位。
|
||||
|
||||
### 7.2 分项文案映射
|
||||
|
||||
| 机器状态 | 阅读文案 |
|
||||
|---|---|
|
||||
| parse=passed | 文档结构可解析 |
|
||||
| resolve=passed | 此项检查报告目标已解析;覆盖范围见详情 |
|
||||
| contract=unchecked | 契约尚未检查 |
|
||||
| execution=not_run | 尚无执行记录 |
|
||||
| tests=unknown | 尚无测试记录 |
|
||||
| evidence=sourced | 已关联来源,内容尚未核验 |
|
||||
| evidence=unverified | 来源内容尚未核验;结合来源列表判断是否已关联 |
|
||||
| missing_source / expired | 来源缺失 / 已过适用期 |
|
||||
| domain=unreviewed | 尚无领域审阅记录 |
|
||||
| effects=pure | 静态分析未发现已建模的外部操作;展开可看范围 |
|
||||
| effects=unknown | 涉及的操作尚不能完全确定 |
|
||||
| 无对应字段 | 未提供该项检查结果 |
|
||||
|
||||
分项取值不能自动升级成综合“可信”。只有宿主选定的政策实际给出聚合结论,才显示“满足 ×× 政策”,同时附政策名称、版本与证据;业务条件还须明确展示。
|
||||
|
||||
### 7.3 结果来源与失效
|
||||
|
||||
每个运行值绑定 `run_id + source_digest + node_id + call_id + loop/iteration` 中适用的字段,展示本次输入或简短上下文。无节点映射时只展示整次运行输出,不硬塞到某个函数旁边。
|
||||
|
||||
静态导出有快照时标“导出快照”,附生成时间/输入;读者切换调用帧是在看历史结果,不叫“重新计算”。源码、依赖或来源与记录不匹配时标“历史结果,版本已变化”,不继续用成功样式绑定到新正文。
|
||||
|
||||
正文里的字面量可以显示为“文档声明值”;解释器实际执行得到的值才叫“运行结果”。没有值、返回 None、stdout 为空、尚未运行、执行失败必须分开,不能统一成 `(no stdout)`。
|
||||
|
||||
### 7.4 空态与错误态
|
||||
|
||||
| 情况 | 处理 |
|
||||
|---|---|
|
||||
| 没有可识别知识单元 | 正常显示文档,不插空知识面板 |
|
||||
| 语义分析失败 | 保留可读正文/源码,显示“知识关系暂不可用”及可定位诊断 |
|
||||
| 局部未解析 | 仅对应片段显示问题,保留其它已知结果 |
|
||||
| 无运行或 no-exec | 运行区说明原因,使用中性状态,无绿色运行成功标记 |
|
||||
| 运行失败 | 错误摘要定位到源片段;日志在运行区,静态知识仍可读 |
|
||||
| 证据过期/冲突 | 相关知识常显问题、来源与影响;不静默选择一个结论 |
|
||||
| 分析被截断/超过扫描范围 | 显示范围和未覆盖项;“未发现”不能替代“没有” |
|
||||
|
||||
## 8. 视觉、密度与可访问性
|
||||
|
||||
### 8.1 排版与主题
|
||||
|
||||
正文建议 16–18px、行高约 1.65;辅助说明 13–14px、行高约 1.5;代码使用等宽字体,全文不因进入 notebook 主题缩为密集工具面板。主次依赖字号、间距和位置共同表达,不能仅靠颜色。
|
||||
|
||||
使用现有系统字体和主题机制,建立统一的 view tokens:正文/辅助文字、背景、细分隔、焦点、成功/警告/错误。知识组件只依赖 tokens,不硬编码黑色线条或白底。支持明暗主题,首次尊重系统/显式设置,不为这次改版再写第三套皮肤。
|
||||
|
||||
表格与长公式允许局部滚动;ID、路径在详情中可换行;主文无整页水平滚动。重要来源和适用条件不使用极浅灰隐藏。对比度验收目标:普通文字至少 4.5:1,大字和必要非文本边界至少 3:1;这些是实现验收目标,尚非本轮测量结果。
|
||||
|
||||
### 8.2 注释和连线
|
||||
|
||||
默认使用紧邻内容的短结果行或有标签的注释块,优先通过“同一节点名 + 明确位置”关联。长 L-leader 不再是默认必需元素;复杂节点可保留短连线,必须满足端点存在、在同一块内、不交叉、不覆盖文字。
|
||||
|
||||
当目标折叠、帧过滤、元素隐藏或几何测量失败时,不绘制连线。主题/字体/容器宽度变化后只重算当前可见块;采用可取消的局部布局更新,不全页不断测量。窄屏、打印、reduced-motion 下直接使用有编号的注释。截图中的孤立线条归入专门视觉回归场景。
|
||||
|
||||
### 8.3 键盘、屏幕阅读器与退化
|
||||
|
||||
- 目录用 nav,正文用 main/article,详情用有名字的 aside/section;标题层级连续。
|
||||
- 展开按钮有 `aria-expanded/controls`;选中状态、失败和待验证都有文字,不依赖颜色或图标。
|
||||
- 深链接滚动有 `scroll-margin-top`,聚焦目标不被固定工具条遮住;关闭详情后焦点回到触发点。
|
||||
- 不使用只有鼠标悬停才能看到的来源、代码或错误。大表格有标题与表头关系,日志可复制。
|
||||
- 无 JS 时正文、目录锚点、来源链接、原生 details 和静态结果仍工作;只失去实时联动与运行操作。
|
||||
- 选择帧不隐藏源结构;隐藏实现也不能删除文档含义和必需条件。
|
||||
|
||||
## 9. 技术设计与数据契约
|
||||
|
||||
### 9.1 从预拼 HTML 改为可关联的展示模型
|
||||
|
||||
当前 `FileViewModel.knowledge_html` 是独立 HTML 字符串,难以与正文、trace、诊断同步。建议新增结构化 `KnowledgePresentation`,在 Rust 端形成同一份模型,由正文、目录、详情共同消费。兼容阶段可保留旧字段,但不让它继续成为唯一知识接口。
|
||||
|
||||
```text
|
||||
ViewDocument
|
||||
document_id, source_digest, semantic_generation, title, description
|
||||
blocks[] 保真正文结构及 source span
|
||||
outline[] 标题/知识单元父子关系
|
||||
knowledge_nodes[] 对应 artifact/ref/step,不复制语义解释器
|
||||
diagnostics[] 范围、严重性、可读说明、定位
|
||||
execution_state 明确未运行/运行中/成功/失败/快照
|
||||
run_records[] 与文档版本相匹配的证据
|
||||
|
||||
KnowledgeNode
|
||||
node_id, artifact_id?, symbol_id?, ref_id?, step_id?
|
||||
owner_id?, source_span, display_name, authored_description?
|
||||
kind, signature?, payload?, dependencies[], provenance[]
|
||||
applicability?, verification[], effects, coverage
|
||||
execution_refs[]
|
||||
```
|
||||
|
||||
这些字段为建议契约,未承诺当前 IR 已提供所有信息。语义事实来自 `SemanticProject`,文档叙述来自解析后的源结构,执行事实来自 capture;缺少关系时展示 unknown,不让 UI 用名称模糊匹配或调用模型编造。
|
||||
|
||||
### 9.2 数据流
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
S[源文档与项目边界] --> P[解析与语义分析]
|
||||
P --> B[正文块与结构]
|
||||
P --> K[知识节点与诊断]
|
||||
R[显式运行或匹配的历史快照] --> T[执行记录]
|
||||
B --> V[统一展示模型]
|
||||
K --> V
|
||||
T --> V
|
||||
V --> D[正文与就地说明]
|
||||
V --> O[目录与当前知识]
|
||||
V --> E[运行区域与技术详情]
|
||||
```
|
||||
|
||||
知识模型不通过“render HTML → 再抓取文本”生成。页面分析失败不可用空字符串吞掉,应保留有状态的结果;部分成功与整体失败分开。
|
||||
|
||||
### 9.3 源位置、稳定身份与锚点
|
||||
|
||||
目前 step 跳转用 `#stmt-{line}`,函数用 `#fn-{line}`。一行多个表达式、正文段落合并、同名函数、源码插行都会使单纯行号不够。
|
||||
|
||||
新增内部节点锚点注册表,将文档块、函数、语义 step/ref 与真实 DOM id 映射;使用 artifact/node 身份和源 span 生成页内唯一 ID,不把未经编码的 URI 当 DOM id。当前编译器没有稳定 SyntaxNodeId 的地方,退化为“源码版本 + span”,明确跨版本不可保证稳定;不能只把行号哈希一下就宣称稳定身份。
|
||||
|
||||
兼容现有 `#structure`、`#fn-N`、`#stmt-N` 链接,保留别名或定位映射;所有生成链接都验目标存在。跨文件 href 复用 [html.rs 的 LinkMode/href_file](../../src/view/html.rs),静态导出保持相对路径;路径展示解码与 URL 安全编码分离。
|
||||
|
||||
### 9.4 分离分析与执行,修正 no-exec
|
||||
|
||||
将当前 `build_file_view` 拆成可组合路径(名称为建议):
|
||||
|
||||
1. `analyze_document`:读取、解析、语义检查、生成静态展示数据,不执行目标程序。
|
||||
2. `capture_execution`:只有执行策略明确允许时调用 run/cell runtime,返回真实状态与源版本。
|
||||
3. `assemble_view`:关联正文、语义和匹配的结果,生成同一个展示模型。
|
||||
|
||||
`--no-exec` 必须在第 2 步之前分支,不能先跑后清空;snapshot 模式使用匹配快照,不为了构造 UI 又运行一次。Live 现有自动执行兼容行为在首轮不静默改变,必须通过显式 execution policy 传入并如实标注;若后续将 live 默认改为阅读时不执行,作为独立 CLI 行为变更发布和迁移。点击详情、查看来源、展开关系等阅读操作绝不能触发执行。
|
||||
|
||||
知识详情不要求通过运行得出静态信息。实现参数预览/试跑时,使用既有受控执行路径,不因“pure”字样绕过宿主能力边界。本文不新增一套求值器或权限系统。
|
||||
|
||||
### 9.5 Live 更新一致性
|
||||
|
||||
当前 SSE 会替换 structure_html/bindings_html/trace_json,而知识面板来自另一次静态编译,容易新旧混用。建议更新包包含文档摘要、语义代、run_id 和变更节点;前端仅合并兼容版本的数据。
|
||||
|
||||
运行写回导致源码变化时,重新分析后一次更新相互关联的正文、目录、知识与运行映射。原选中节点仍存在则保留选择/焦点/滚动;已消失则明确提示,不能悄悄指向同名节点。旧请求返回晚于新请求时按版本丢弃,不覆盖较新阅读状态。
|
||||
|
||||
### 9.6 性能与导出体积
|
||||
|
||||
避免每个页面重复编译整个项目。一次静态导出共享同一项目语义结果,逐页投影;live 按项目源指纹、编译器版本、配置和适用性时钟缓存,读取使用一致的语义代。来源/有效期变化必须使相关状态失效,不能无限缓存 sourced。
|
||||
|
||||
正文服务端生成,不等待 JS 初始化或全部知识图完成才出现。默认只嵌入当前文档必要知识,深层依赖按需取;静态导出则包含保证离线可用的局部数据与明确范围。大量结果采用折叠/分页,但不能让屏幕阅读器和查找失去正文。
|
||||
|
||||
性能验收目标:100 个知识单元的本地选择/展开交互 p95 ≤100ms;1000 单元目录可搜索而不阻塞正文滚动;导出项目的语义编译次数不随页面数线性重复。记录硬件、样例和体积,目标未达时先优化数据重复和布局测量,不以截断关键来源达标。
|
||||
|
||||
## 10. 实施文件与兼容策略
|
||||
|
||||
| 位置 | 建议变更 |
|
||||
|---|---|
|
||||
| [src/view/render.rs](../../src/view/render.rs) | 提取知识展示模型;替换全量 dl 面板;关联步骤归属、友好名称与诊断 |
|
||||
| [src/view/document.rs](../../src/view/document.rs) | 源块/知识节点映射、正文就地入口、层级标题、可复制原始 span |
|
||||
| [src/view/html.rs](../../src/view/html.rs) | 正文优先布局、辅助栏、详情/目录交互、状态组件、统一 tokens;保留 LinkMode |
|
||||
| [src/view/mod.rs](../../src/view/mod.rs) | 分离分析/执行、保留诊断、组装统一模型、项目分析缓存 |
|
||||
| [src/view/output.rs](../../src/view/output.rs) | 提前应用 no-exec/snapshot 策略、共享项目分析、静态详情与相对跳转 |
|
||||
| [src/view/annot.rs](../../src/view/annot.rs) | 运行标注与节点/帧/版本关联,减少重复输出;缺映射不伪绑定 |
|
||||
| [src/view/debug_page.rs](../../src/view/debug_page.rs) | 仅回归检查共享组件;不把阅读布局强加给 debug |
|
||||
| [tests/executable_knowledge.rs](../../tests/executable_knowledge.rs)、view 单测 | 从字符串存在升级到顺序、锚点、状态和执行边界断言 |
|
||||
| 文档与 public 示例 | 更新旧 View 布局说明、截图和行为描述,区分历史默认与新设计 |
|
||||
|
||||
默认展示不要求作者补特殊 frontmatter;title/description、函数说明、契约和来源已有时直接受益。尚无信息的旧文档保真显示,并在详情中如实缺省。
|
||||
|
||||
与现有文档的关系:保留 [view-document-annotations](view-document-annotations.md) 的叙述主导和真实结果,修订“长 L-leader 为默认”;保留 [view-frame-selection](view-frame-selection.md) 的源码常显与帧隔离;[view-markup-surface](view-markup-surface.md) 的历史 Classic 例子需按当前默认语法同步,不能在新 UI 继续教授旧取元。外联/WASM 交互沿既有独立路线发展,不作为完成本次阅读改版的前置。
|
||||
|
||||
## 11. 分阶段交付
|
||||
|
||||
### V0:先纠正顺序、误导状态和执行边界
|
||||
|
||||
正文提前;完整知识报告收进明确的辅助入口,module/type 容器默认不铺卡;“Lock digest”改为准确的摘要标签;状态不再共用一个 ok。修复 no-exec 先执行问题,分析错误可见。暂不等待完整节点模型,就能显著改善截图场景。
|
||||
|
||||
V0 不算最终完成:仅把面板搬到后面/折叠,仍未解决正文与知识的关联。
|
||||
|
||||
### V1:完成阅读与结构理解的主闭环
|
||||
|
||||
统一展示模型、标题/目录层级、就地说明、当前知识详情、友好定义跳转与返回。完成函数、计算、来源、空态与基本运行结果的正负例;live/static 共用。
|
||||
|
||||
### V2:贯通运行、来源和跨文档关系
|
||||
|
||||
匹配运行实例与版本、调用/循环帧、来源定位/失效、局部依赖关系、SSE 原子更新。上游证据能力尚不完整的地方明确显示未知,不能以 UI 补出“验证通过”。
|
||||
|
||||
### V3:移动端、离线和规模验收
|
||||
|
||||
完成键盘/读屏、无 JS、打印、暗色/浅色、100/1000 单元性能、视觉回归与读者测试。发布前同步 public 示例与说明,保留旧锚点兼容。
|
||||
|
||||
## 12. 验收矩阵
|
||||
|
||||
### 12.1 必测文档
|
||||
|
||||
| 样例 | 必须验证 |
|
||||
|---|---|
|
||||
| 截图对应 Phase I 页 | 正文先出现;无 full URI/hash 瀑布;print 在所属正文解释;无孤立线条 |
|
||||
| 纯叙述/少量 Markdown 页 | 阅读无知识空卡、无误导运行徽章 |
|
||||
| 单个纯函数,无 main | 参数/返回与调用方式清楚,不因无入口标全文失败 |
|
||||
| 订单定价、多文档 import | 小计/税率/总额归属和依赖可追踪;输入和结果可对应 |
|
||||
| 带来源的事实与过期/缺失来源 | sourced 不显示已证实;问题在对应知识旁可见 |
|
||||
| 同名函数、多次调用和循环分支 | 定义唯一;切换帧只改结果和高亮,不隐藏结构 |
|
||||
| 空 payload、动态调用、未解析引用 | 待实现/动态/未解析准确可辨,不能展示虚构结果 |
|
||||
| 一行多个链接、嵌套表达式、中英文/长路径 | span 对应、锚点唯一、文本不丢失、不水平溢出 |
|
||||
| 运行失败、分析失败、快照过期 | 三种情况分开,仍能读到正文与具体诊断 |
|
||||
| 外联单元、变量大表、长日志 | 单层主体与对应结果,不重复三份输出;局部滚动 |
|
||||
|
||||
### 12.2 自动化判据
|
||||
|
||||
1. DOM 中文档主体先于全量知识技术详情,标题来自既有元数据;短文没有隐藏正文的默认展开报告。
|
||||
2. 每个知识/目录链接都能定位真实节点;一行多节点和同名跨模块不冲突;live/static 路径均有效。
|
||||
3. 验证、运行、来源状态按固定真值表渲染;无证据时绝不输出综合已验证。
|
||||
4. no-exec 用隔离目录的副作用计数测试证明没有执行;snapshot 不重跑;展开详情也不执行。
|
||||
5. 帧选择和 SSE 更新保留正文结构、选中身份与焦点,拒绝过期结果包。
|
||||
6. 明暗主题/320、390、1024、1440 宽度/200% 缩放下无遮挡、截断关键条件和整页横向溢出;孤立 leader 数为零。
|
||||
7. 无 JS、file 静态打开、打印预览都保留正文、定义/来源导航和必要证据说明。
|
||||
8. 输出中的作者内容、label、URI/属性与嵌入 JSON 按原上下文转义;资料中的 HTML/脚本不能因为新详情组件被直接执行。
|
||||
|
||||
字符串断言只能覆盖结构,不能替代浏览器布局验收;需截图基线和真实键盘交互测试。可复用现有测试基础,避免测试只镜像组件实现。
|
||||
|
||||
### 12.3 读者任务测试
|
||||
|
||||
建议用至少 5 位未参与实现的读者,分别完成三个任务,并与当前页面对照:
|
||||
|
||||
- 在 30 秒内说清文档目的并找到主要规则/函数。
|
||||
- 在 60 秒内找到某个结果的输入、定义和原始来源。
|
||||
- 正确区分“来源已关联但未核验”“本次运行成功”“未运行/历史结果”,不得误认为都已验证。
|
||||
|
||||
目标为至少 4/5 完成前两项,第三项无误导性判断;小样本仅作为可用性门槛,不声称统计显著。记录失败位置并修正信息架构,不能只问“是否好看”。
|
||||
|
||||
## 13. 完成定义与本次交付边界
|
||||
|
||||
完成的 view 应让读者先读懂文档,再自然看到其中的知识与执行结构;需要深入时能沿着一个具体片段追到实现、输入、结果和证据。全量技术字段仍可查,但不要求读者先理解编译器才能读文档。
|
||||
|
||||
本次只提交设计文档与索引,不修改渲染实现。实证包括用户截图、当前模板/渲染路径、静态 HTML 顺序检查和临时 no-exec 反例;布局尺寸、交互、性能和读者任务结果均为待实施的验收目标。
|
||||
@@ -0,0 +1,163 @@
|
||||
# Web 热路径:单解释器串行化(设计缺陷说明)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted**(缺陷记录 + 缓解已落地) |
|
||||
| 日期 | 2026-10-26 |
|
||||
| 规范决策 | [ADR 0008](../adr/0008-web-invoke-serialization.md) |
|
||||
| 相关 | [ext-web-go-rewrite.md](ext-web-go-rewrite.md) · [ext-abi.md](ext-abi.md) · [ADR 0004](../adr/0004-web-plugin-go.md) · [ADR 0007](../adr/0007-web-document-endpoint.md) |
|
||||
|
||||
本文把「页面卡死 / 像单线程」拆成可验证的架构事实、错误语义与演进边界。
|
||||
**结论先行:HTTP 是多线程的;卡住是因为所有 Endpoint 共享一把解释器锁。**
|
||||
|
||||
增量实施提案见 [隔离执行与响应式 Web 运行时(2026-10-26)](isolated-web-runtime-development-2026-10-26.md):独立进程池、控制/读取容量、请求级事件与取消、缓存和恢复。提案通过验收前,本文件及 ADR 0008 的现行单飞约束不变。
|
||||
|
||||
---
|
||||
|
||||
## 1. 问题陈述
|
||||
|
||||
### 1.1 用户可见症状
|
||||
|
||||
1. 发起一次长对话(OpenAI 兼容 `POST …/v1/chat/completions` + `stream:true` / `sse-live`)。
|
||||
2. 对话尚未结束时,打开设置、拉 `/v1/models`、或其它短 JSON API。
|
||||
3. **旧行为**:浏览器请求一直 pending,整页像死机;只能杀 `marqdo run serve…` 恢复。
|
||||
4. **目标行为**:短请求在约 1.5s 内得到 **HTTP 503**,错误体标明 `interpreter busy`;对话 SSE 继续;结束后短请求立刻恢复 200。
|
||||
|
||||
### 1.2 错误归因(不要再说错)
|
||||
|
||||
| 错误说法 | 正确说法 |
|
||||
|----------|----------|
|
||||
| 「web 插件是单线程的」 | Go `net/http` 与 SSE 刷写是多 goroutine |
|
||||
| 「忘记启动进程所以卡死」 | 进程在跑;是 **锁争用** 导致短请求永不返回 |
|
||||
| 「把 SSE 的等待时间调大就能修好」 | 进程级拉长 `MARQDO_INVOKE_WAIT_MS` 会让短请求一起卡死 |
|
||||
| 「去掉 INVOKE_LOCK 即可并行」 | `HostContext` 非 Sync,无锁并行会数据竞争 / 未定义行为 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 调用栈与锁边界
|
||||
|
||||
```
|
||||
Browser
|
||||
│ GET /v1/models POST /v1/chat/completions (SSE)
|
||||
▼
|
||||
Go plugins/web (多 goroutine)
|
||||
handleInvoke / handleInvokeLiveSSE
|
||||
│
|
||||
▼ C ABI host_query("call_lib_path", {path, args})
|
||||
Rust src/host/plugin.rs
|
||||
┌─────────────────────────────┐
|
||||
│ INVOKE_LOCK (进程级 Mutex) │ ← 同一时刻只允许一个 worker 进入
|
||||
└─────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
HostContext:装载/求值作者 .mq.md、写回、事件总线、LLM 插件回调…
|
||||
```
|
||||
|
||||
要点:
|
||||
|
||||
1. **锁在宿主,不在 Go 业务路由表。** 增加更多 Go handler 不会自动并行执行 Marqdo。
|
||||
2. **SSE 持锁时间 ≈ 整次 Endpoint 求值时间**(含多次 LLM 往返),可达数十秒到数分钟。
|
||||
3. **事件转发**(`MARQDO_EVENT_FORWARD` NDJSON)与 **取消**(`MARQDO_CANCEL_TOKEN`)在持锁期间仍可由其它 goroutine 写文件/读桥;它们不进入解释器,故不受 `INVOKE_LOCK` 保护,也不应误设长等待。
|
||||
|
||||
---
|
||||
|
||||
## 3. 为何不能简单「多线程跑 Marqdo」
|
||||
|
||||
`HostContext` 持有(非穷尽):
|
||||
|
||||
- 当前模块图、函数表、写回缓冲与 FS 根;
|
||||
- 插件注册与 `CURRENT_HOST` / 全局 host 指针;
|
||||
- Agent / LLM 运行时与 stream 发布钩子;
|
||||
- 与文档写回、会话态相关的可变结构。
|
||||
|
||||
这些状态在实现上按**单线程求值**建模。把它标成 `Sync` 或去掉 `INVOKE_LOCK` 而没有隔离运行时,属于未定义行为,不是性能优化。
|
||||
|
||||
**真并行的合法方向(未实现):**
|
||||
|
||||
1. **解释器池**:每请求或每会话一个 `HostContext`(或可重置的 isolate),路由层选闲置实例;
|
||||
2. **只读快照 + 写队列**:热路径只读克隆,写回串行合并(适合纯查询 Endpoint,不适合带写回的 agent);
|
||||
3. **把长任务移出解释器**:Marqdo 只做编排,重活在独立进程/队列(与「代码即文档」主路径权衡后再定)。
|
||||
|
||||
在上述任一方案落地并经金样例验收前,**产品必须以「单飞解释器」为约束设计 UX**。
|
||||
|
||||
---
|
||||
|
||||
## 4. 缓解契约(已实现)
|
||||
|
||||
### 4.1 宿主:限时争锁
|
||||
|
||||
文件:`src/host/plugin.rs`
|
||||
|
||||
- 仅当从 worker 线程进入(`CURRENT_HOST` 为空、走全局 host)时争 `INVOKE_LOCK`。
|
||||
- `acquire_invoke_lock`:`try_lock` 循环,预算默认 **1500ms**(`MARQDO_INVOKE_WAIT_MS`,最小 1)。
|
||||
- 超时:`set_err("interpreter busy: another request holds the Marqdo invoke lock (waited …ms)")`,返回非 0。
|
||||
- **不要**在 SSE 入口把该环境变量设成数分钟:它是进程级的,会污染并发短请求。
|
||||
|
||||
### 4.2 Web 插件:503
|
||||
|
||||
文件:`plugins/web/internal/httpx/routes_invoke.go`
|
||||
|
||||
- 普通 JSON invoke:错误信息含 `interpreter busy` → **503** + `Retry-After: 2` + `{ok:false, error:…}`。
|
||||
- Live SSE:若启动时即 busy,以失败事件/错误结束该流,**不得**无限挂起握手。
|
||||
|
||||
### 4.3 客户端:超时与文案
|
||||
|
||||
短 API(模型列表、设置、registry 探测等)必须:
|
||||
|
||||
- 设置合理 `fetch` 超时(秒级,例如 8s);
|
||||
- 将 503 / `interpreter busy` 展示为「服务正忙(对话进行中)」类可读提示,而不是空白转圈。
|
||||
|
||||
(qdagent 前端 `frontend/src/lib/api.ts` 已按此契约处理。)
|
||||
|
||||
### 4.4 验收探针
|
||||
|
||||
在本地 serve 上:
|
||||
|
||||
```bash
|
||||
# 终端 A:启动长 SSE 对话(持锁)
|
||||
curl -N -m 120 -H 'Authorization: Bearer …' -H 'Content-Type: application/json' \
|
||||
-d '{"stream":true,…}' http://127.0.0.1:PORT/v1/chat/completions
|
||||
|
||||
# 终端 B:对话进行中
|
||||
curl -sS -w 'http=%{http_code} time=%{time_total}\n' --max-time 5 \
|
||||
-H 'Authorization: Bearer …' http://127.0.0.1:PORT/v1/models
|
||||
# 期望:http=503,time≈1.5,body 含 interpreter busy
|
||||
|
||||
# 对话结束后再打同一 URL → http=200,time≪1s
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 运维与调参
|
||||
|
||||
| 变量 | 默认 | 含义 |
|
||||
|------|------|------|
|
||||
| `MARQDO_INVOKE_WAIT_MS` | `1500` | worker 进入解释器前最长等待;超时 → busy |
|
||||
|
||||
建议:
|
||||
|
||||
- 开发机 / 产品默认保持 1–3s。
|
||||
- **禁止**为「让第二个聊天排队」而把默认调到分钟级——那等于恢复卡死。
|
||||
- 需要排队时,应在 **应用层**(会话队列、UI 禁用「再发一条」),而不是拉长宿主锁等待。
|
||||
|
||||
---
|
||||
|
||||
## 6. 与其它机制的边界
|
||||
|
||||
| 机制 | 与 INVOKE_LOCK 的关系 |
|
||||
|------|----------------------|
|
||||
| `MARQDO_EVENT_FORWARD` | SSE 期间写 NDJSON;不替代争锁;不延长等待 |
|
||||
| `MARQDO_CANCEL_TOKEN` | 断开连接时可取消 LLM;释放锁仍取决于解释器何时回到 FFI |
|
||||
| `web.serve` / 静态资源 | 通常不经 `call_lib_path`,不受本锁影响 |
|
||||
| 插件内部 Tokio/Go 并发 | 可以;只要不再次 `call_lib_path` 进同一 HostContext |
|
||||
|
||||
---
|
||||
|
||||
## 7. 演进检查清单(根治前勿宣称「已支持并行 Endpoint」)
|
||||
|
||||
- [ ] 多 `HostContext` 或 isolate 的生命周期与 ABI(装载、shutdown、写回冲突)有设计稿与金样例。
|
||||
- [ ] 池化后 `call_lib_path` 不再依赖进程级单锁,或锁粒度降到实例级。
|
||||
- [ ] 两个并发长 SSE + 一个短 `/v1/models` 的压力测试:短请求稳定 200,长请求互不污染会话写回。
|
||||
- [ ] 更新本文件与 ADR 0008 Status(例如改为 Superseded by ADR NNNN)。
|
||||
|
||||
在清单完成前,对外与文档中应明确:**Marqdo Web Endpoint 求值在单进程内串行;HTTP 层并发,解释器层单飞。**
|
||||
+124
-1
@@ -450,4 +450,127 @@ AI 优先使用既有知识,而非反复解释和重新生成
|
||||
|
||||
如何让一个 AI 在面对新任务时,能够像编译器复用已有函数一样,直接复用知识库中已经编译的知识,而不必重新理解和生成它?
|
||||
|
||||
如果 Marqdo 能真正解决这个问题,你原来的三句话就不再只是设计理念,而是一个可以通过实验验证的语言能力。
|
||||
如果 Marqdo 能真正解决这个问题,你原来的三句话就不再只是设计理念,而是一个可以通过实验验证的语言能力。
|
||||
|
||||
# Marqdo 可执行知识体系:设计规范与完整范例
|
||||
|
||||
我已经基于 Marqdo 当前仓库的语法规范、`[]()` 设计、知识分层、对象模型和 EKC,整理了两份详细文档。
|
||||
|
||||
这次设计保留了你的原始理念:
|
||||
|
||||
> 代码即文档,文档即知识库,知识即可执行。
|
||||
|
||||
核心目标不是让 AI 更容易生成 Marqdo 代码,而是让 AI 尽可能不必重新生成已经存在的知识,转而通过编译器确定性地发现、引用、验证和执行既有知识。
|
||||
|
||||
第一份:完整架构设计规范
|
||||
|
||||
涵盖语义制品、知识引用、编译器架构、解析优先级、效应安全、版本管理、EKC 集成、Agent 复用协议和实施验收。
|
||||
|
||||
阅读设计文档
|
||||
|
||||
第二份:多文件可执行知识范例
|
||||
|
||||
以虚构书店定价为例,从原始政策文档开始,逐步构造知识定义、对象、计算能力、业务入口、AI 问答、知识复用和错误验证。
|
||||
|
||||
阅读完整示例
|
||||
|
||||
两份文档打包
|
||||
|
||||
下载 ZIP
|
||||
|
||||
## 一、这份设计最重要的三个决定
|
||||
|
||||
# Marqdo 可执行知识体系:核心设计纲领
|
||||
|
||||
**核心理念:代码即文档,文档即知识库,知识即可执行。**
|
||||
|
||||
## 1. 知识是可编译的语义制品
|
||||
|
||||
文档经过 Marqdo 编译器处理后,不只产生可执行逻辑,还应产生具有稳定身份、类型、签名、契约、依赖关系、来源和验证状态的语义制品(Semantic Artifact)。
|
||||
|
||||
知识可以表现为事实、函数、能力、约束、数据集或类型。它们无需全部被强制编码为类。
|
||||
|
||||
## 2. `[]()` 是知识使用的统一入口
|
||||
|
||||
`[描述](目标)` 保留 Markdown 的可读性。
|
||||
|
||||
- `[]` 面向人类,描述意图、意义或用途。
|
||||
- `()` 面向编译器,包含既有 Marqdo 表达式或者可解析的知识目标。
|
||||
- 描述不改变执行结果,也不能作为可信性依据。
|
||||
- 编译器负责将目标解析为确定的符号身份,并验证使用条件。
|
||||
|
||||
既支持隐藏不易阅读的代码,也支持引用已经编译的知识。
|
||||
|
||||
## 3. AI 复用知识,而不是重新解释知识
|
||||
|
||||
AI 面对新任务时,应该遵循:
|
||||
|
||||
**Discover → Resolve → Verify → Reuse → Execute → Trace**
|
||||
|
||||
当知识缺失或不适用时,才进入 ADAPT 或 CREATE 流程。
|
||||
|
||||
检索可以模糊,但最终绑定必须确定。验证必须区分程序可执行性、契约符合性和现实事实正确性。
|
||||
|
||||
## 4. 编译器是知识系统的基础
|
||||
|
||||
AST、Semantic IR、符号索引、链接关系和知识验证应当由同一套语义体系产生。
|
||||
|
||||
解释器、字节码 VM、View、EKC 和 Agent 消费统一的编译结果,而不是各自重新解析文档。
|
||||
|
||||
## 5. 最终验收标准
|
||||
|
||||
一个 AI 能够发现现有知识、确定其身份、检查适用条件、直接执行既有能力、引用结果和来源,并在知识发生变化时识别受影响的调用者。
|
||||
|
||||
只有这条链条成立,Marqdo 才真正实现知识的可执行复用。
|
||||
|
||||
## 二、示例将展示什么
|
||||
|
||||
我选择了一个具有明确规则和复用价值的业务案例,而不只是使用“李白出生年份”这样的静态事实。
|
||||
|
||||
用户购买两件单价 60 元的商品,金卡折扣为 90%,满 100 元免运费。
|
||||
|
||||
AI 要处理的业务请求
|
||||
|
||||
> 请计算这笔金卡订单的最终应付款,并解释依据。
|
||||
|
||||
传统路径
|
||||
|
||||
检索政策 → AI 阅读规则 → 重新推导计算 → 生成新代码或答案
|
||||
|
||||
Marqdo 目标路径
|
||||
|
||||
发现定价能力 → 绑定已编译函数 → 检查输入与政策 → 执行 → 引用结果
|
||||
|
||||
目标语法示意(v2.1+,不是 v2.0.0 实测)
|
||||
|
||||
```
|
||||
**应付 = [订单最终应付款](定价.计算应付金额(60, 2, "金卡"))**
|
||||
```
|
||||
|
||||
根据演示政策计算
|
||||
|
||||
# ¥108
|
||||
|
||||
这套例子进一步展示了参数和返回值、类与继承、表格、条件分支、循环、跨文档调用、自然语言遮蔽,以及知识来源改变时的影响分析。
|
||||
|
||||
## 三、与当前版本的兼容边界
|
||||
|
||||
我特别保留了一个重要区分:设计目标不等于当前已经实现的功能。
|
||||
|
||||
根据这次读取的仓库内容,v2.0.0 已支持本地表达式遮蔽、对象、方法、控制流、表格集合、导入以及现有 EKC 基础;但无实例类函数调用、跨文档知识自动解析、完整 LinkInfo 和语义制品绑定仍需要进一步开发。
|
||||
|
||||
因此示例文档明确标注了目标语法和现有兼容写法,没有将尚未验证的代码描述为已经跑通。
|
||||
|
||||
## 四、我的主要架构建议
|
||||
|
||||
我认为接下来的开发不应该从 `[]()` 的外观继续扩展,而应该先实现:
|
||||
|
||||
统一符号索引 → 稳定知识身份 → 引用绑定 → 契约验证 → EKC 复用。
|
||||
|
||||
还有一点值得特别强调:我不赞成为了建立知识系统,再单独实现一个与 Marqdo 解释器并列的“知识编译器”。那样会重新制造你希望消除的表示碎片。
|
||||
|
||||
更合适的架构是让现有 Marqdo 编译管线产出统一的语义中间表示,然后由运行时、View、EKC 和 Agent 各自消费。
|
||||
|
||||
这意味着 Marqdo 的知识体系不是一个外挂的 RAG 框架,而是语言本身的语义能力。
|
||||
|
||||
我认为这才是整个设计中最值得投入资源,也最有可能形成独立研究贡献的部分。
|
||||
+374
@@ -0,0 +1,374 @@
|
||||
请你重新阅读marqdo的官方库,掌握marqdo的最新进度。在我对marqdo项目的仔细审视之下我发现有一点没有闭环,我终于找到了,考虑现在智能体开发最常用的SKILL,这是一个markdown文件写明了智能体该干什么,我想要更推荐一步,因为marqdo使用markdown语法,支持叙述和代码相互混合,这就使得marqdo满足这样一个条件,当分章节的SKILL撰写下来,只要经过调整就能成为一份marqdo代码其中包含若干函数,我打算开发新的CLI命令marqdo > 函数名 函数参数或者marqdo call 函数名 函数参数 此时SKILL的不同章节就变成了不同的函数,智能体就真正的成为了将知识复现的作用,这让代码即文档、文档即知识,知识即可执行推进了一步变成了知识即工具,真正发挥了marqdo的优势,同样这样的执行还以为着智能体能够自行改进SKILL并接受确定的问题后修改而不会出错,这让自我进化的智能体成为了具备理论支撑和清晰边界的可能
|
||||
|
||||
# Marqdo 从“可执行知识”走向“知识即工具”:最新仓库审查与架构建议
|
||||
|
||||
我认为你这次找到了 Marqdo 一个非常重要、而且比单纯增加 `marqdo call` 命令更深层的突破口。
|
||||
|
||||
你实际上提出了一个新的统一关系:
|
||||
|
||||
> SKILL 是人和 AI 都能阅读的操作知识;Marqdo 可以让其中具有确定性执行语义的章节,直接成为可调用的工具。
|
||||
|
||||
这样,Agent 的职责就有机会从“阅读说明、理解步骤、临时生成实现”,转向“定位已有能力、填入参数、执行和验证结果”。
|
||||
|
||||
这与 Marqdo 原来的三句话是连续的,不需要推翻原来的定位:
|
||||
|
||||
代码即文档 → 文档即知识库 → 知识即可执行 → 知识即可作为工具使用。
|
||||
|
||||
不过我建议把“知识即工具”视为可执行知识的对外使用形态,而不是再创造一套与语言、知识库平行的 Skill 运行系统。
|
||||
|
||||
## 一、重新检查仓库后的最新情况
|
||||
|
||||
本次核对的是 [cflmy/marqdo](https://github.com/cflmy/marqdo?utm_source=chatgpt.com) 的 `main` 分支,HEAD 为 `4018e992`,`Cargo.toml` 版本为 2.1.5。最近的 v2.1.5 版本记录日期是 2026 年 10 月 27 日。
|
||||
|
||||
与我们之前讨论 v2.0.0 时相比,项目已经完成了不少与你的构想直接相关的基础建设。
|
||||
|
||||
| 相关能力 | 最新仓库证据 | 判断 |
|
||||
| --------------------- | ---------------------------------------------------------------- | -------- |
|
||||
| 跨文档确定性函数绑定 | `BoundCall`、`SymbolId`,Tree/Bytecode 双后端测试 | 已有核心基础 |
|
||||
| 已编译知识的执行复用 | MLSP `artifact.bind`、`artifact.execute`、`artifact.reuse_execute` | 已有子集 |
|
||||
| Agent 复用已有函数 | `agent_eng_bind_execute`、发布门禁 | 已有子集 |
|
||||
| 指定函数执行 | `marqdo run --call-lib-path`、JSON 参数、隔离 worker | 已有底层入口 |
|
||||
| 智能体工具规范 | `ToolSpecV1`、效应分类、权限检查与恢复设计 | 已有框架基础 |
|
||||
| 章节式 SKILL 自动成为 CLI 工具 | 目前没有独立的 `marqdo call` 或 Skill 编译命令 | 待开发 |
|
||||
| SKILL 自修改、自动测试和受控发布 | 已有局部修改、验证与沙盒组件 | 尚未形成完整闭环 |
|
||||
|
||||
因此,现在不是重新开发一个 Skill 执行器,而是将已存在的 Marqdo 函数、知识制品、工具协议和 Agent 机制统一起来。
|
||||
|
||||
还有一个与你的新构想直接相关的问题值得先修复:
|
||||
|
||||
最新官方 `skills/marqdo/SKILL.md` 的核心语法说明,与仓库中的现行解析器、v0.3 语法规范和可执行测试存在冲突。
|
||||
|
||||
目前解析器与测试使用的是:
|
||||
|
||||
- `**…**`:代码、赋值、调用等语句;
|
||||
- `*…*`:返回值。
|
||||
|
||||
但官方 Skill 顶部却把两者反过来描述。这不是小的文档笔误:如果 Agent 按照错误规则自我修改 Skill,就可能产生与实际运行时不一致的程序。
|
||||
|
||||
这个真实存在的不一致,恰好证明了你所说的闭环为什么重要——SKILL 不能仅靠自然语言说明自己的正确性,它需要由语言编译器和实际测试提供独立验证。
|
||||
|
||||
## 二、你真正提出的是一种新的 Skill 体系
|
||||
|
||||
我建议把它暂时命名为 Executable Skill(可执行技能)。
|
||||
|
||||
它不应被理解为“给 Markdown Skill 增加一点代码”,而应当满足一个更严格的定义:
|
||||
|
||||
> Executable Skill 是一份以 Markdown 为人类可读接口、以 Marqdo 函数为确定性执行单元、以编译后的语义制品为工具身份、能够由 Agent 发现、调用、验证和改进的技能文档。
|
||||
|
||||
传统 Skill 与你的目标之间,存在一个非常重要的区别。
|
||||
|
||||
| 维度 | 普通 Markdown Skill | Marqdo Executable Skill |
|
||||
| ---- | ----------------------- | ----------------------- |
|
||||
| 技能说明 | 自然语言 | 自然语言 |
|
||||
| 操作步骤 | Agent 阅读后自行实施 | 可以绑定已有 Marqdo 函数 |
|
||||
| 函数接口 | 通常需要额外编写工具代码 | 从 `.mq.md` 的函数和契约生成 |
|
||||
| 执行 | 依赖 Agent 选择工具、生成代码或操作软件 | 对已实现的步骤直接执行 |
|
||||
| 输出验证 | 往往需要单独建立 | 可连接编译器、契约和测试 |
|
||||
| 改进 | 编辑说明或脚本 | 编辑同源的知识与实现,验证后发布 |
|
||||
| 可追踪性 | 主要依赖工具日志 | 可关联 SymbolId、源码、版本和执行结果 |
|
||||
|
||||
但不是每个 Skill 章节都应该自动变成可执行函数。
|
||||
|
||||
比如“适用场景”“注意事项”“禁止操作”是知识或约束;“检查订单”“生成报价”“汇总报告”才可能成为可调用能力。编译器应区分文档章节与可执行章节,不能把所有二级标题不加判断地暴露成工具。
|
||||
|
||||
### 这实际上使 Agent 的结构发生变化
|
||||
|
||||
SKILL.md
|
||||
|
||||
目标 · 流程 · 函数 · 契约 · 示例 · 约束
|
||||
|
||||
Marqdo Skill Compiler
|
||||
|
||||
章节解析 · 符号绑定 · 类型检查 · 效应分析
|
||||
|
||||
Knowledge View
|
||||
|
||||
Agent 阅读技能与限制
|
||||
|
||||
Tool Registry
|
||||
|
||||
Agent 调用已编译函数
|
||||
|
||||
Agent:选择、组合、执行、观察与改进
|
||||
|
||||
由语言运行时执行确定性部分,由模型处理非确定性判断
|
||||
|
||||
特别值得注意的是:这不仅是执行接口的统一,还是知识交付接口的统一。
|
||||
|
||||
同一份 Skill 能够投影成可读说明、CLI 函数、Agent Tool Schema、MCP 工具描述,以及知识库中的能力节点。实现这些投影的系统仍是同一个 Marqdo 编译器。
|
||||
|
||||
## 三、CLI 应采用 `marqdo call`,不要采用 `marqdo >`
|
||||
|
||||
你提出了两个命令形式:
|
||||
|
||||
```
|
||||
marqdo > 函数名 函数参数
|
||||
marqdo call 函数名 函数参数
|
||||
```
|
||||
|
||||
我明确建议采用第二种。
|
||||
|
||||
原因是 `>` 在 Bash、PowerShell、CMD 等 Shell 中已经具有输出重定向语义。`marqdo > xxx` 会被 Shell 先行解释,无法可靠地作为传递给 Marqdo 的函数调用语法。
|
||||
|
||||
推荐设计为:
|
||||
|
||||
```
|
||||
# 按函数名调用
|
||||
marqdo call "订单计算.商品小计" \
|
||||
--skill ./skills/order-calculator \
|
||||
--arg 单价=30 \
|
||||
--arg 数量=2
|
||||
|
||||
# 结构化调用,供 Agent 使用
|
||||
marqdo call "订单计算.商品小计" \
|
||||
--skill ./skills/order-calculator \
|
||||
--args-json inputs.json \
|
||||
--output json \
|
||||
--policy pure
|
||||
```
|
||||
|
||||
上述是建议新增的 CLI 接口,不是 v2.1.5 已支持的命令。
|
||||
|
||||
我还建议在 CLI 层保留一个便利别名:`marqdo skill call`。它与 `marqdo call` 进入同一个 dispatcher,不应分别实现两套调用逻辑。
|
||||
|
||||
### 最小必要的 CLI 能力
|
||||
|
||||
| 命令 | 作用 |
|
||||
| ---------------------------- | ----------------------- |
|
||||
| `marqdo skill inspect PATH` | 解析 Skill 文档、章节、可调用函数和依赖 |
|
||||
| `marqdo skill list PATH` | 列出对 Agent 暴露的工具 |
|
||||
| `marqdo skill check PATH` | 检查语法、绑定、签名、效应和工具导出规则 |
|
||||
| `marqdo call TARGET` | 调用指定 Marqdo 函数并返回结果 |
|
||||
| `marqdo skill test PATH` | 执行 Skill 的测试与契约检查 |
|
||||
| `marqdo skill export PATH` | 生成通用工具描述或 MCP 暴露接口 |
|
||||
| `marqdo skill trial PATH` | 对修改后的候选版本进行隔离验证 |
|
||||
| `marqdo skill promote PATH` | 在门禁通过后发布新版本 |
|
||||
| `marqdo skill rollback PATH` | 恢复到指定已接受版本 |
|
||||
|
||||
不过不建议一次性实现这九项命令。最小可用产品只需要 `inspect`、`call`、`check`、`test` 四项,版本发布与改进机制可以随后补齐。
|
||||
|
||||
最重要的技术决策是:
|
||||
|
||||
`marqdo call` 必须调用现有编译器绑定的函数,不得重新用字符串搜索源码、猜测章节位置再执行。
|
||||
|
||||
当前 [src/mlsp.rs](https://github.com/cflmy/marqdo/blob/main/src/mlsp.rs?utm_source=chatgpt.com) 已有 `artifact.bind` 与 `artifact.execute`,能够检查制品指纹、依赖闭包、来源摘要和效应策略。新的 CLI 应复用相同的应用服务和语义校验,而不是通过进程内调用 MLSP JSON 接口制造另一层重复协议。
|
||||
|
||||
还有一个现有实现细节:`marqdo run --call-lib-path` 已支持隔离 worker 通过 JSON 参数调用函数,但当前 `src/lib.rs` 的这个入口只在树遍历分支中处理,Bytecode 分支仍按普通程序入口执行。新的公开 `call` 命令需要补齐这一差异,或者明确在第一版只支持 Tree backend,不能静默忽略所选后端。
|
||||
|
||||
## 四、完整示例:让一份 Skill 同时成为文档、知识和工具
|
||||
|
||||
我建议第一版尽量保持作者体验简单:人只维护一份 Markdown 源文件,编译器自动生成工具描述、函数索引和知识制品。
|
||||
|
||||
例如,我们建立一个订单计算 Skill。
|
||||
|
||||
下面是目标设计示例。其中 `runtime` 和 `tools` 是我建议为 Skill 编译增加的元数据约定,目前不能将这份 `SKILL.md` 直接交给 v2.1.5 的 CLI,期待自动注册工具。
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
name: order-calculator
|
||||
description: 订单价格计算,包括商品小计和含税金额
|
||||
runtime: marqdo
|
||||
tools:
|
||||
- 订单计算.商品小计
|
||||
- 订单计算.含税金额
|
||||
---
|
||||
|
||||
# 订单计算
|
||||
|
||||
这份技能用于计算订单金额。
|
||||
|
||||
所有金额使用同一货币单位。税率使用小数表示,
|
||||
例如 0.10 表示 10%。
|
||||
|
||||
## 商品小计
|
||||
|
||||
给定商品单价`单价`和购买数量`数量`,
|
||||
计算未税商品小计。
|
||||
|
||||
*[商品小计](单价 * 数量)*
|
||||
|
||||
## 含税金额
|
||||
|
||||
给定商品单价`单价`、购买数量`数量`
|
||||
和适用税率`税率`,计算含税金额。
|
||||
|
||||
**小计 = [计算商品小计](订单计算.商品小计(单价=单价, 数量=数量))**
|
||||
|
||||
*[最终含税金额](小计 * (1 + 税率))*
|
||||
|
||||
```
|
||||
|
||||
这里的计算是演示规则。正式业务技能还需要校验数量范围、金额精度、货币单位、税率适用条件与舍入规则;这些不能由函数名字自动推断为正确。
|
||||
|
||||
### 这份 Skill 会产生三种投影
|
||||
|
||||
给人阅读: 它是一份正常的 Markdown 文档。读者能理解每个步骤的目的,以及所需输入和返回值。
|
||||
|
||||
给 Marqdo 编译器: 两个 `##` 单元构成可解析的函数;第二个函数可以确定性地调用第一个函数;符号索引能够记录它们的依赖关系。
|
||||
|
||||
给 Agent 使用: 编译器输出两个工具的结构化描述。Agent 不需要重新生成乘法计算程序,只需选择工具并提供参数。
|
||||
|
||||
例如,编译后的工具描述可以是:
|
||||
|
||||
```
|
||||
{
|
||||
"id": "order-calculator::订单计算.商品小计",
|
||||
"description": "计算未税商品小计",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"单价": {"type": "number"},
|
||||
"数量": {"type": "number"}
|
||||
},
|
||||
"required": ["单价", "数量"]
|
||||
},
|
||||
"output_schema": {
|
||||
"type": "number"
|
||||
},
|
||||
"effect": "pure"
|
||||
}
|
||||
```
|
||||
|
||||
但这个 Schema 必须由明确的类型契约和编译器检查生成。若源文件没有足够的类型信息,应标记为未知或要求补全,不能因为变量参加了乘法就无条件断言它一定是 `number`。
|
||||
|
||||
需要强调:不是 `##` 标题自动等于一个公开工具。 `##` 让章节具备 Marqdo 函数的结构,`tools` 则声明哪些函数允许对外暴露。这样才能避免“安全说明”一类章节被意外注册成工具,也避免内部辅助函数被 Agent 随意调用。
|
||||
|
||||
### 为什么采用显式的 `runtime: marqdo`?
|
||||
|
||||
因为普通 Skill 的 Markdown 本来就可能包含 `**加粗内容**`、编号列表、代码块和二级标题。这些在现有 Marqdo 中可能具有执行语义。
|
||||
|
||||
如果直接把所有 `SKILL.md` 当成 `.mq.md` 执行,很可能发生严重的语义误读。
|
||||
|
||||
所以我建议:
|
||||
|
||||
- 未明确声明 Marqdo 运行时的 `SKILL.md`,仍然是普通 Markdown 文档。
|
||||
- 明确声明 `runtime: marqdo` 的 Skill 才进入 Marqdo 编译流程。
|
||||
- 内嵌示例代码块不应因展示而执行。
|
||||
- 编辑器应对具有执行意义的 Markdown 标记提供可视化提示。
|
||||
- 导出的工具必须经过独立检查和授权。
|
||||
|
||||
未来还可以支持从已有普通 Skill 中自动提取候选函数,但那应当是生成可审阅的转换补丁,而不是默认把自然语言步骤当成已经实现的代码。
|
||||
|
||||
## 五、这次最有价值的部分:自我改进的 Skill
|
||||
|
||||
你提出的第二层意义,我认为甚至比 `marqdo call` 更值得研究。
|
||||
|
||||
传统 Agent 修改一个 Skill 时,常见问题是:它修改了一段说明或脚本,却无法可靠判断其他步骤是否受到影响。
|
||||
|
||||
Marqdo 已经具备一些适合解决这个问题的基础,例如 `SymbolId`、依赖闭包、`artifact.patch`、`artifact.verify` 和知识影响分析。
|
||||
|
||||
但是,不能由此推出“智能体修改 Skill 就不会出错”。
|
||||
|
||||
更精确的目标是:智能体可以提出改进,系统能够判定一组明确、可测试的性质是否被保持,并且只有通过门禁的版本才有资格替换原有 Skill。
|
||||
|
||||
我建议采用以下演化协议:
|
||||
|
||||
已发布 Skill v1
|
||||
|
||||
现有稳定版本与完整依赖
|
||||
|
||||
1\. 发现确定的问题
|
||||
|
||||
失败测试、诊断、复现步骤或用户提出的明确修改要求
|
||||
|
||||
2\. AI 生成局部修改候选
|
||||
|
||||
只修改获准的章节或函数,记录版本与修改意图
|
||||
|
||||
3\. 编译与静态验证
|
||||
|
||||
语法、符号、参数、返回类型、效应、依赖关系
|
||||
|
||||
4\. 隔离测试与回归验证
|
||||
|
||||
目标问题修复、历史行为保持、异常输入、性能与权限边界
|
||||
|
||||
5\. 发布门禁
|
||||
|
||||
仅当策略允许且必要验证通过时晋升新版本;否则保留旧版本
|
||||
|
||||
Skill v2 · 已接受版本
|
||||
|
||||
可追溯、可回滚、后续调用绑定到明确版本
|
||||
|
||||
### 举一个真正的自我改进案例
|
||||
|
||||
原 Skill 中存在:
|
||||
|
||||
```
|
||||
## 商品小计
|
||||
|
||||
商品单价`单价`,购买数量`数量`。
|
||||
|
||||
*[小计](单价 * 数量)*
|
||||
```
|
||||
|
||||
现在发现一个确定的问题:输入 `数量=-1` 时,函数返回负数,但业务要求数量必须大于零。
|
||||
|
||||
Agent 不应该只是把说明修改成“数量必须大于零”。这不会改变执行行为。
|
||||
|
||||
它应当在候选版本中增加可执行的校验逻辑,并补充至少三个测试:
|
||||
|
||||
| 测试 | 输入 | 期望 |
|
||||
| ---- | ----------- | ------------- |
|
||||
| 正常订单 | 单价 30,数量 2 | 返回 60 |
|
||||
| 非法数量 | 单价 30,数量 -1 | 返回结构化参数错误 |
|
||||
| 边界数量 | 单价 30,数量 0 | 根据业务契约决定接受或拒绝 |
|
||||
|
||||
如果 Marqdo 尚不具备合适的语言级错误返回或异常处理方式,这个案例还会暴露一个语言能力缺口:需要一种由 Marqdo 程序自身构造、传递和处理错误的机制,而不是只能依赖 Rust 宿主报错。
|
||||
|
||||
这与此前我们讨论的通用语言能力重构是直接关联的。
|
||||
|
||||
### 自我进化的边界必须比普通代码修改更严格
|
||||
|
||||
我建议把以下约束固定下来:
|
||||
|
||||
1. 修改不能自动扩大权限。 原 Skill 只有纯计算权限,新版本不能因为 Agent 自己添加网络或文件操作就获得这些权限。
|
||||
2. 候选版不能直接覆盖运行中的正式版本。 需要隔离工作区、原子发布和固定引用版本。
|
||||
3. 必须证明目标问题被修复。 不允许通过删除测试、放宽断言或隐藏失败来宣称成功。
|
||||
4. 测试通过不是绝对正确的证明。 未覆盖的业务情况、外部事实和依赖服务仍可能出错。
|
||||
5. 高风险修改需要更严格的发布决策。 尤其是支付、外部写入、删除、凭证和安全策略变更,不能由被修改的 Skill 自己批准。
|
||||
|
||||
最新仓库的 `artifact.patch` 已经提供基于摘要的修改和有限回滚,但它还不是完整的候选版本事务与测试后晋升系统。现有 `HostCaps` 的默认值也允许文件写入、执行、网络和插件,因此新 `call` 的安全默认值应当收紧,不能简单继承普通本地运行的全部宿主权限。
|
||||
|
||||
这就是“有界自我进化”的含义:不保证 Agent 永不犯错,而是让它能够在可验证、可回滚、可限制权限的范围内持续改进。
|
||||
|
||||
## 六、我建议如何安排开发
|
||||
|
||||
| 阶段 | 目标 | 关键验收 |
|
||||
| ---------------- | ---------------------------------------- | ------------------------ |
|
||||
| S0 · Skill 语义一致性 | 修复官方 Skill 与解析器的标记冲突,建立 Skill 语法同步测试 | Skill 记载的每条核心语法都有可运行金样例 |
|
||||
| S1 · 函数调用 CLI | `marqdo call` 调用已绑定的普通函数、类函数和跨文档函数 | JSON 参数正确绑定;错误路径和双后端行为明确 |
|
||||
| S2 · Skill 编译 | 对显式启用的 `SKILL.md` 解析章节和可执行函数 | 自动产生 SymbolId、签名、依赖、导出清单 |
|
||||
| S3 · 知识即工具 | 将公开函数编译为 ToolSpec,并接入现有 Agent dispatcher | Agent 不重新生成实现即可调用并取得结果 |
|
||||
| S4 · Skill 测试 | 增加 `skill check/test`、负例、效应门禁和回归测试 | 未通过验证的 Skill 不得自动发布 |
|
||||
| S5 · Skill 演化 | 实现候选修改、隔离验证、影响分析、版本晋升与回滚 | 修改失败不影响已发布版本,权限不能自行扩大 |
|
||||
|
||||
这里还需要一项对你的研究目标尤其重要的对照实验。
|
||||
|
||||
让相同模型分别使用普通 Markdown Skill、Markdown Skill 加外部脚本工具、以及 Marqdo Executable Skill 完成相同的一组任务。分别测量正确率、模型需要重新生成的代码量、工具调用参数错误、修改后的回归失败率、token 消耗和完成时间。
|
||||
|
||||
只有与“Markdown + 现有工具调用”进行公平对照,才能证明 Marqdo 的优势来自文档、程序与知识的统一,而不只是因为它额外提供了一个执行器。
|
||||
|
||||
## 七、我对这项构想的最终评价
|
||||
|
||||
我认为这次构想具有三个非常值得肯定的性质。
|
||||
|
||||
第一,它有明确的语言基础。`#`、`##`、自然语言叙述、函数参数、`[]()`、契约表和跨文档符号绑定并不是为了 Skill 重新创造的。Skill 可以成为现有语言的一种应用形态。
|
||||
|
||||
第二,它有真实的工程基础。当前 Marqdo 已经具备部分确定性绑定、制品验证、Agent 复用与隔离运行能力。你需要做的是将这些能力收敛到统一的公开接口,而不是重新实现一套 Agent 框架。
|
||||
|
||||
第三,它能够引出一个更有研究价值的问题:自然语言描述的操作知识,如何逐步转化为可调用、可检查、可维护,并可由 Agent 在受限条件下自行改进的程序制品?
|
||||
|
||||
但是,“SKILL 章节即函数”本身还不足以证明新的编程范式。Markdown 章节映射函数,与文学编程、文档驱动工作流及现有工具包装有明显的共同点。
|
||||
|
||||
更有价值的是把三个原本需要单独维护的对象——Skill 说明、函数实现、Agent Tool Schema——统一为一个可编译的源事实,并展示这种统一确实降低了错误、维护成本和重复生成成本。
|
||||
|
||||
我建议为 Marqdo 保留原来的核心理念,并将这一阶段的技术方向明确为:
|
||||
|
||||
> 知识即工具(Knowledge as Tool):将可执行知识直接编译为 Agent 能够发现、调用和验证的工具,并允许智能体在明确的语义、测试和权限边界内持续改进这些知识。
|
||||
|
||||
这可能是 Marqdo 从“AI 可以使用的编程语言”,进一步走向“AI 能够使用并维护的可执行知识体系”的关键一步。
|
||||
|
||||
备注,这里实际上有一点叙述错误,marqdo支持mq.md文件,所以最好针对SKILL也是SKILL.mq.md这样会比较统一。
|
||||
+249
@@ -0,0 +1,249 @@
|
||||
SKILL的名称应该是.mq.md有一点必须提及,SKILL其实命名不一定必须得是SKILL,平常的一个marqdo文件也可以是一份skill这里的设计太复杂了没有抽离出来核心本质,你看对于一个平常的marqdo文件,完全也可以和SKILL一样,这都是没有区别的,所谓的知识即工具作为高度抽象本质就是带参数调用函数,所以你的inspect等等命令其实完全可以对传统的marqdo也生效而不是非要绑定SKILL对吧,而且这样处理还有好处是方面智能体做单元测试,单个函数是否可以带参数直接运行,换句话说,你仔细思考就会发现这些好处是marqdo的设计哲学在增加一点点call自然派生的优势,所以不需要繁复的设计
|
||||
|
||||
你说得对。这次我认为你抓住了真正的核心,而且我的上一版设计确实把一个本来很自然的语言能力复杂化了。
|
||||
|
||||
Marqdo 根本不需要引入一种叫作 Executable Skill 的新概念,也不需要专门建立 Skill 编译器。
|
||||
|
||||
因为在 Marqdo 的设计哲学下:
|
||||
|
||||
> 一个 `.mq.md` 文件本身就是文档、知识与程序。只要允许从外部直接调用其中的函数,它自然也就能够成为 Skill。
|
||||
|
||||
这不是为 Skill 开发一套新系统,而是为 Marqdo 补齐一个非常基础的语言使用入口。
|
||||
|
||||
## 一、真正缺少的只有一件事:函数级执行入口
|
||||
|
||||
Marqdo 当前已经支持函数定义、参数、返回值、模块引用,以及确定性符号绑定。
|
||||
|
||||
但是日常使用时,我们往往仍然站在“运行整个文件”的角度:
|
||||
|
||||
```
|
||||
marqdo run example.mq.md
|
||||
```
|
||||
|
||||
你现在提出的是:
|
||||
|
||||
```
|
||||
marqdo call example.mq.md 函数名 参数
|
||||
```
|
||||
|
||||
这实际上意味着:
|
||||
|
||||
Marqdo 的最小可执行单位,从文件进一步扩展为文件中的任意可调用函数。
|
||||
|
||||
而这一点会自然派生大量优势。
|
||||
|
||||
普通 `.mq.md` 文件
|
||||
|
||||
自然语言叙述 + 函数 + 参数 + 返回值
|
||||
|
||||
marqdo call
|
||||
|
||||
指定函数 + 绑定参数 + 直接执行
|
||||
|
||||
知识复用
|
||||
|
||||
调用已存在的实现
|
||||
|
||||
Agent Skill
|
||||
|
||||
直接暴露函数为工具
|
||||
|
||||
单元测试
|
||||
|
||||
逐个函数验证结果
|
||||
|
||||
自我改进
|
||||
|
||||
修改函数并重新测试
|
||||
|
||||
这里不存在四套实现,只有一个底层能力的四种使用方式。
|
||||
|
||||
## 二、一个普通的 Marqdo 文件就足够了
|
||||
|
||||
比如,文件 `calculator.mq.md`:
|
||||
|
||||
```
|
||||
# 计算器
|
||||
|
||||
这是一个普通的 Marqdo 文档。
|
||||
|
||||
其中包含一些可以复用的数学计算知识。
|
||||
|
||||
## 加法
|
||||
|
||||
计算两个数字`a`和`b`的和。
|
||||
|
||||
*[计算结果](a + b)*
|
||||
|
||||
## 乘法
|
||||
|
||||
计算两个数字`a`和`b`的积。
|
||||
|
||||
*[计算结果](a * b)*
|
||||
```
|
||||
|
||||
它既是可阅读的文档,也包含两个函数。
|
||||
|
||||
现在只需要支持:
|
||||
|
||||
```
|
||||
marqdo call calculator.mq.md 计算器.加法 a=2 b=3
|
||||
```
|
||||
|
||||
返回:
|
||||
|
||||
```
|
||||
5
|
||||
```
|
||||
|
||||
再比如:
|
||||
|
||||
```
|
||||
marqdo call calculator.mq.md 计算器.乘法 a=4 b=6
|
||||
```
|
||||
|
||||
返回:
|
||||
|
||||
```
|
||||
24
|
||||
```
|
||||
|
||||
以上 `marqdo call` 是拟新增的命令形式;函数示例遵循当前 `*…*` 返回、`[]()` 遮蔽的语义。
|
||||
|
||||
这时候,一个 Agent 只要能够读取这个 `.mq.md` 文档,就可以理解函数的用途;只要能够调用 Marqdo CLI,就能执行其中的能力。
|
||||
|
||||
它根本不需要关心这个文件是不是 Skill,也不需要一份特殊的 `SKILL.md`。
|
||||
|
||||
如果 Agent 把它当成 Skill 阅读和使用,它就是 Skill;如果开发者把它当成程序库调用,它就是程序库;如果研究者把它作为知识库中的一部分,它就是可执行知识。
|
||||
|
||||
文件的身份没有变化,变化的是使用方式。
|
||||
|
||||
## 三、这项能力会自然解决三个问题
|
||||
|
||||
### 1. 让知识成为工具
|
||||
|
||||
传统 Skill:
|
||||
|
||||
Agent 阅读操作说明,决定如何完成任务。
|
||||
|
||||
Marqdo:
|
||||
|
||||
Agent 阅读函数说明,找到适合的函数,传入参数,直接执行。
|
||||
|
||||
这就将原来的:
|
||||
|
||||
知识 → Agent 理解 → 重新实现 → 执行
|
||||
|
||||
缩短为:
|
||||
|
||||
知识 → 函数定位 → 参数调用 → 执行
|
||||
|
||||
但模型仍然需要负责判断应该调用哪个函数、传入哪些参数。确定性的是被调用函数的语言语义,而不是整个 Agent 决策过程。
|
||||
|
||||
### 2. 让单元测试成为语言的天然能力
|
||||
|
||||
我非常赞同你提到的单元测试价值。
|
||||
|
||||
有了 `marqdo call`,可以直接测试:
|
||||
|
||||
```
|
||||
marqdo call calculator.mq.md 计算器.加法 a=2 b=3
|
||||
# 期望 5
|
||||
|
||||
marqdo call calculator.mq.md 计算器.加法 a=-2 b=3
|
||||
# 期望 1
|
||||
|
||||
marqdo call calculator.mq.md 计算器.乘法 a=0 b=100
|
||||
# 期望 0
|
||||
```
|
||||
|
||||
不必为了测试一个函数,再创建一个 `# main`,再编写调用、打印和结果检查的包装程序。
|
||||
|
||||
未来即使增加 `marqdo test`,底层仍然可以复用同一个函数调用接口。
|
||||
|
||||
### 3. 让 Agent 可以针对知识进行局部改进
|
||||
|
||||
这是你提出的最有意义的延伸。
|
||||
|
||||
假设一个 Agent 发现 `计算器.乘法` 对某类输入存在错误。
|
||||
|
||||
它可以:
|
||||
|
||||
1. 阅读相应函数及其上下文。
|
||||
2. 修改这个 `.mq.md` 文件中的函数实现。
|
||||
3. 直接用 `marqdo call` 传入错误输入验证。
|
||||
4. 再运行该函数的历史测试。
|
||||
5. 如果检查通过,就提交候选修改;否则保留原有实现。
|
||||
|
||||
这使得 Agent 的改进对象不再仅仅是自然语言提示词,而是有明确输入、输出和可执行语义的知识单元。
|
||||
|
||||
当然,测试不能证明函数在所有输入上都正确;正式发布和权限控制仍需要遵循已有的项目验证机制。但不需要为此创造一套 Skill 专属的演化协议。
|
||||
|
||||
## 四、真正的 CLI 设计可以非常简单
|
||||
|
||||
我建议第一阶段只做一个命令:
|
||||
|
||||
```
|
||||
marqdo call <文件> <函数> [具名参数...]
|
||||
```
|
||||
|
||||
比如:
|
||||
|
||||
```
|
||||
marqdo call book.mq.md 订单.计算总价 单价=30 数量=2
|
||||
```
|
||||
|
||||
它只需要完成五件事:
|
||||
|
||||
| 步骤 | 职责 |
|
||||
| -- | -------------------- |
|
||||
| 1 | 加载指定 `.mq.md` 文件及其导入 |
|
||||
| 2 | 解析并唯一定位目标函数 |
|
||||
| 3 | 根据现有函数签名绑定、校验参数 |
|
||||
| 4 | 使用现有运行时执行该函数 |
|
||||
| 5 | 返回函数结果和结构化错误 |
|
||||
|
||||
额外提供一个 `--json` 输出选项就足够方便 Agent 调用。
|
||||
|
||||
如果未来需要 `marqdo inspect`,它也应该针对任何 `.mq.md` 文件,列出其中函数、参数和文档描述,而非限定 Skill。
|
||||
|
||||
此外,当前仓库已经有 `--call-lib-path` 和 `artifact.execute` 的相关基础,因此实现时应尽量复用现有函数定位与执行机制,而不是重复开发解释器。
|
||||
|
||||
唯一值得坚持的边界是:外部调用仍必须经过现有权限检查,不能因为新增了 `call` 就默认允许 Agent 对任意函数执行文件删除、网络写入等操作。
|
||||
|
||||
## 五、我认为你的设计哲学现在更加清晰了
|
||||
|
||||
你原来的理念:
|
||||
|
||||
> 代码即文档,文档即知识库,知识即可执行。
|
||||
|
||||
实际上已经足够完整。
|
||||
|
||||
“知识即工具”不是额外的设计理念,而是“知识即可执行”在函数级调用能力成熟后自然出现的结果。
|
||||
|
||||
进一步看,这件事还有一个很漂亮的对称性:
|
||||
|
||||
| 参与者 | 对同一个 `.mq.md` 的使用方式 |
|
||||
| ---------- | ------------------- |
|
||||
| 人类 | 阅读文档、理解知识 |
|
||||
| 编译器 | 解析并验证函数 |
|
||||
| 开发者 | 调用函数、组合程序 |
|
||||
| Agent | 发现知识、调用工具 |
|
||||
| 测试系统 | 输入参数、检查结果 |
|
||||
| 维护者或 Agent | 修改实现、验证改进 |
|
||||
|
||||
这些角色并没有要求作者维护六份不同的东西。
|
||||
|
||||
同一份文件,同一套语义,同一个函数调用接口。
|
||||
|
||||
这也恰恰是我认为 Marqdo 相比传统“Markdown 文档 + 外部工具”组合最值得证明的优势:知识描述与执行实现共址,而且能够直接由语言工具链消费,避免维护重复的工具包装和操作说明。
|
||||
|
||||
所以,我现在的建议与上一轮不同:
|
||||
|
||||
不要开发 Executable Skill 系统。不要设计 Skill 专属语法。不要引入新的 Skill 注册、编译和执行机制。
|
||||
|
||||
先完成 `marqdo call`,使普通 `.mq.md` 中的可调用函数都能被精确定位、传参执行,并用现有语义系统验证。
|
||||
|
||||
这是一个相对很小的接口变化,却可能自然打通 Marqdo 的编程、知识复用、Agent 工具调用和函数级测试四个场景。
|
||||
|
||||
我认为这才是你所说的:让 Marqdo 自己的设计哲学发挥作用,而不是不断增加外部框架来证明它的价值。
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **实施基线(Draft for Implementation)** |
|
||||
| 状态 | **目标实施规范;按文档原生指导修订交付与政策分级** |
|
||||
| 日期 | 2026-10-23 |
|
||||
| 适用版本 | v2.1 及后续兼容演进 |
|
||||
| 真实意图来源 | [`009.md`](009.md)、[`Marqdo_Executable_Knowledge_Design.md`](Marqdo_Executable_Knowledge_Design.md)、[`Marqdo_Executable_Knowledge_Example.md`](Marqdo_Executable_Knowledge_Example.md) |
|
||||
@@ -10,17 +10,17 @@
|
||||
|
||||
## 0. 文档地位与执行规则
|
||||
|
||||
本文把上述三份文档中的真实产品意图整理为可编码、可测试、可迁移、可验收的开发规范。开发本轮能力时,应以本文和三份意图来源为准;仓库中更早的链接、遮蔽、知识分层、类模型、OKF 或 EKC 文档若与本文冲突,不得据其改变本轮设计。
|
||||
本文把上述三份文档中的真实产品意图整理为可编码、可测试、可迁移、可验收的开发规范。2026-10-23 的[文档原生开发指导](../design/document-native-development-guide.md)进一步约束作者负担、AI 编辑与复用、场景分级和交付顺序;本文的内部结构不能反向要求作者重复填写注册表、工具 schema 或知识图。仓库中更早的链接、遮蔽、知识分层、类模型、OKF 或 EKC 文档若与上述方向冲突,不得据其改变本轮设计。
|
||||
|
||||
优先级从高到低如下:
|
||||
|
||||
1. `009.md` 中确认的产品目标与边界;
|
||||
2. 本文冻结的工程决策、数据结构和验收条件;
|
||||
3. 两份附属文档中的概念设计与完整示例;
|
||||
4. 当前代码的兼容事实;
|
||||
2. [文档原生开发指导](../design/document-native-development-guide.md)对作者体验、政策分级和交付排序的修订;
|
||||
3. 本文的工程决策、数据结构和验收条件;
|
||||
4. 两份附属文档中的概念设计与完整示例;
|
||||
5. 其他旧文档,仅可作为历史背景,不是规范依据。
|
||||
|
||||
本文不宣称目标能力已经实现。表格中“当前”表示仓库 v2.0.0 的实际行为,“目标”表示完成相应里程碑后的行为。任何阶段只有在对应的自动化测试、双后端一致性测试和迁移测试全部通过后,才可从“目标”改标为“已实现”。
|
||||
本文不宣称目标能力已经实现。未另行标日期的“当前”表格与 §2 缺口保留最初 v2.0.0 审计背景,不能继续当作 v2.1 的实时状态:默认 Occlusion、线程局部解析作用域、关联函数与零参路径等已经演进。当前实现判断见[2026-10-23 缺口审计](../roadmap/executable-knowledge-gaps-2026-10-23.md)及对应源码/测试;代码事实证明实现范围,不自动取代产品目标。任何阶段只有在对应的自动化测试、双后端一致性测试和迁移测试全部通过后,才可改标为“已实现”。CLI/API 示意与结构定义仍是目标协议,使用前须核对实现。
|
||||
|
||||
---
|
||||
|
||||
@@ -56,7 +56,10 @@ Marqdo 不是“能运行 Markdown 中几段代码”的解释器,也不是“
|
||||
- 代码、依赖、来源或验证状态变化时,影响范围能够被程序找出;
|
||||
- Tree Interpreter 与 Bytecode VM 对同一已绑定程序产生一致结果和诊断;
|
||||
- Agent 命中已有且适用的制品时,得到的是可调用句柄,不是让模型复制源码再写一遍;
|
||||
- 系统从不把 `compiled`、`tested` 或 `source_present` 误报成“现实世界事实已证明正确”。
|
||||
- 系统从不把 `compiled`、`tested` 或 `source_present` 误报成“现实世界事实已证明正确”;
|
||||
- 作者能从正文计算开始,按需要提取命名能力与跨文档复用,不先建立知识治理工程;
|
||||
- 人和 AI 能按标题/步骤获得最小上下文、局部修改并查看语义影响,不必整篇重生成;
|
||||
- 每个交付切片同时证明阅读、修改或复用的实际收益,而不只证明内部字段存在。
|
||||
|
||||
### 1.3 不在本轮目标内
|
||||
|
||||
@@ -69,9 +72,17 @@ Marqdo 不是“能运行 Markdown 中几段代码”的解释器,也不是“
|
||||
- 不把自然语言标签当作授权、契约或事实正确性的依据;
|
||||
- 不让 EKC、view、check、Agent 各自再次解析源码并形成不同理解。
|
||||
|
||||
### 1.4 作者成本与渐进严格
|
||||
|
||||
标题、叙述、表格、链接描述和执行语句是作者的主要工作面;符号 ID、步骤 ID、规范签名、效应闭包、工具 schema、图和摘要由工具生成。不能推导的领域条件、来源和许可由作者明确补充,不要求每个纯工具函数都填写事实证据。
|
||||
|
||||
本地探索允许无锁运行草稿,但不绕过宿主能力限制;项目自动复用按所选 policy 校验;严格业务再增加必要证据、版本锁和新鲜度;远程/多租户另有可信授权与隔离门槛。这是同一语法上的政策分级,不是新的语法模式。纯预览可复用已授予的项目/会话政策,不做逐链接重复确认。详情见[指导 §6](../design/document-native-development-guide.md#6-渐进严格不把治理门槛塞进每一行正文)。
|
||||
|
||||
自然语言标签是发现、解释和局部编辑的重要信息,但不证明 payload 正确。AI 可以选候选、组合、适配、重构及补测试;编译器确定实际绑定、类型和效应,宿主决定能力上限。不得把“不用模型决定语义”误读成“模型只能搜索”。
|
||||
|
||||
---
|
||||
|
||||
## 2. 当前实现基线与真实缺口
|
||||
## 2. 初始实现基线与缺口背景(非 v2.1 实时清单)
|
||||
|
||||
### 2.1 可以复用的现有基础
|
||||
|
||||
@@ -890,6 +901,10 @@ TASK
|
||||
|
||||
REUSE 的判据不是“搜索分数超过阈值”,而是候选已经精确绑定、输入满足契约、effect 获得许可、版本策略通过。搜索分数只用于候选排序。
|
||||
|
||||
`ready` 是对所选 policy、输入与上下文的判断,不是所有知识必须先获得的全局认证。AI 可以从候选中选择明确目标;歧义不能交给执行器按扫描顺序消解。ADAPT/CREATE 必须遵守任务写权限和预算,缺少业务决策时请求补充,而非擅自改变适用条件。草稿可被发现并在明确权限下探索执行,但不得冒充已通过严格政策的事实。
|
||||
|
||||
发现结果应按需包含标题/步骤、源码片段、输入角色、契约、效应、来源、验证缺口和相关失败经验。局部编辑须携带期望源码摘要,应用后检查语义 diff 与相关测试;不以重写整份文档作为默认修复方式。协议及受限试点范围见[指导 §5](../design/document-native-development-guide.md#5-ai-原生工作协议读步骤改步骤验证步骤)。
|
||||
|
||||
### 12.4 BoundArtifactHandle
|
||||
|
||||
```rust
|
||||
@@ -906,17 +921,21 @@ pub struct BoundArtifactHandle {
|
||||
|
||||
Agent 执行必须携带 handle 和显式输入;runtime 再次核对摘要和权限,防止发现后到执行前目标变化。handle 是能力引用,不包含可由模型修改后悄悄执行的源码副本。
|
||||
|
||||
上述结构表达绑定内容,不独自构成远程授权凭据。受信本地调用可先采用精确引用和摘要复核;不受信/远程调用必须另有服务端认可的主体、能力范围、防伪及撤销/期限机制,客户端自报 `accepted_effects` 不能扩大宿主许可。handle 由工具产生,不要求作者维护;是否检查 lock 和来源新鲜度取决于明确的 policy。
|
||||
|
||||
### 12.5 Promote 门槛
|
||||
|
||||
Agent 新建或适配的知识进入可复用索引前,至少完成:
|
||||
Agent 新建或适配的知识进入**共享自动复用的 ready 集合**前,按目标 policy 至少完成:
|
||||
|
||||
- Parse/Resolve/Type/Contract 通过;
|
||||
- 声明 effect;
|
||||
- effect 已由静态绑定与宿主/扩展元数据推导;不能推导时保留 unknown 并按政策拒绝或要求明确能力许可,不要求业务作者抄写闭包;
|
||||
- pure 逻辑有测试,外部 effect 有 mock/沙箱证明或人工批准;
|
||||
- 来源型知识附 SourceRef;
|
||||
- 生命周期从 `draft` 开始;
|
||||
- 生成者、输入、工具版本、变更 diff 可审计;
|
||||
- 未达到门槛的内容只在 quarantine/draft index 中可见。
|
||||
- 未达到门槛的内容保持 draft/quarantine 状态;可在同一权威索引中过滤展示,不另造语义身份。
|
||||
|
||||
这是晋升政策,不是普通 `.mq.md` 文件被索引、查看或本地执行的前置注册流程。简单纯计算不强制现实来源;声称业务事实成立的知识不能用测试通过替代所需证据。
|
||||
|
||||
---
|
||||
|
||||
@@ -1117,7 +1136,7 @@ View 至少展示:
|
||||
|
||||
## 17. 分阶段开发计划
|
||||
|
||||
每个里程碑都必须形成可独立合并的垂直切片;不得先大规模重构再等待最后一次性验收。
|
||||
M0–M6 保留为工程能力与验收项目,不再作为顺序排期;部分迁移能力已经完成,状态见[现状审计](../roadmap/executable-knowledge-gaps-2026-10-23.md)。实际工作按[文档原生 S0–S6](../design/document-native-development-guide.md#7-交付切片每片都有文档作者与-ai-能看见的结果)形成可独立合并的垂直切片:早期联动 M1/M2/M3/M5 的安全子集,交付可读步骤和真实复用,不能等 M4 全部做完才让作者或 Agent 使用。100 单元生成器逐步增长,不阻塞最小故事;不得先大规模重构再等待最后一次性验收。
|
||||
|
||||
### M0:语义冻结与测试生成器
|
||||
|
||||
@@ -1218,7 +1237,7 @@ View 至少展示:
|
||||
- 图中 calls 边与实际运行 trace 的静态目标一致;
|
||||
- Agent 命中 ready 制品时执行 handle,不生成新函数;
|
||||
- 只有模糊候选时不能直接执行;
|
||||
- 未验证、过期、effect 不允许的制品不会被标记为 REUSE ready。
|
||||
- 不满足所选验证政策、已过相关有效期或 effect 不允许的制品不会被标记为该 policy 下的 REUSE ready;本地探索不冒充严格验证。
|
||||
|
||||
### M6:迁移默认值与发布
|
||||
|
||||
@@ -1241,6 +1260,8 @@ View 至少展示:
|
||||
|
||||
## 18. 文件级实施清单
|
||||
|
||||
下表是目标职责映射,不是“所有文件尚未创建”的任务表。`src/project/`、`src/sema/` 和部分 API 已存在,先检查对应缺口与测试,再增量修改;不要依照旧“新建”措辞重复搭建模块或移走仍被消费者使用的能力。
|
||||
|
||||
| 文件/模块 | 修改 |
|
||||
|---|---|
|
||||
| `src/ast.rs` | 增加 SyntaxNodeId;`Occluded` 迁为 `Named`;承载 Narrative AST 的最小引用 |
|
||||
@@ -1372,6 +1393,8 @@ View 至少展示:
|
||||
- 总 token、工具调用、修复轮次和耗时;
|
||||
- 回答能否追溯到 SymbolId、digest、输入和来源。
|
||||
|
||||
同时记录作者手写声明/重复字段/配置数量、阅读定位准确率、语义步骤修改范围、无关 diff、并发补丁冲突和首次写作成本。比较时各方案应获得同等现有能力与输入信息,不向新方案额外提供答案;将编译和人工准备成本计入总成本。先通过正确性与权限硬门槛,再讨论 token 和速度收益,参见[指导 §8](../design/document-native-development-guide.md#8-验证-marqdo-优势而不只验证组件存在)。
|
||||
|
||||
### 20.3 发布门槛
|
||||
|
||||
- 核心确定性套件 100% 通过;
|
||||
@@ -1385,6 +1408,8 @@ View 至少展示:
|
||||
|
||||
不要只报告平均值;保存失败样本、任务定义、环境、置信区间和版本摘要。
|
||||
|
||||
以上按发布宣称的能力范围执行。语法兼容、本地确定性复用、严格业务复用、远程执行分别有独立证据,不以一个总体 `done` 替代。M0–M6 保留为工程能力检查项,实际交付顺序采用[修订路线 S0–S6](../roadmap/executable-knowledge-gaps-2026-10-23.md#7-依赖顺序与可合并切片):早期完成一篇文档的读/执行/复用闭环,再增加治理深度;任何已开放执行路径都不能延后安全门禁。
|
||||
|
||||
---
|
||||
|
||||
## 21. 书店示例的最终验收链
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
# 智能体能力调研与求道差距复核(2026-10-27)
|
||||
|
||||
| 项目 | 说明 |
|
||||
|---|---|
|
||||
| 状态 | Research / 设计输入;不是功能交付声明 |
|
||||
| 调研日期 | 2026-10-27,Asia/Shanghai |
|
||||
| 源码基线 | Marqdo `16f26d7d60ce9fbc62feb0e8e83f93951f7b53fe`;qdagent `68b757e02ebfb64833b5262b0c8a683512581e8b` |
|
||||
| 框架设计 | [Agent 恢复、工具与资源协议](../design/agent-reliability-tools-resources-2026-10-27.md) |
|
||||
| 产品设计 | [求道智能体能力与预览体验](../../../qdagent/doc/design/agent-capabilities-and-preview-2026-10-27.md)(同级仓库) |
|
||||
|
||||
## 1. 方法与结论边界
|
||||
|
||||
先以 `git pull --ff-only` 同步两仓,再追踪真实调用链与前端入口;对比公开官方文档,不以营销介绍、工具数量或旧设计中的 done 标签代替实现证据。本次是静态源码复核与官方资料研究,没有启动部署实例,没有调用付费模型,没有进行竞品账号实测。因此,下表“源码确认”仅代表代码路径可见,“设计判断”与“待实测”不能写成已复现或已交付。
|
||||
|
||||
主流能力的共同方向是:可恢复的执行、结构化工具与资源、按需上下文、可追溯结果、独立产物展示。不同产品和模型支持范围不同;这里不是市场占有率调查,也不声称所有智能体都具备全部能力。求道应保留文档原生、EKC、验证后沉淀的定位,吸收机制,不引入第二套 Agent Runtime。
|
||||
|
||||
## 2. 官方资料与可借鉴机制
|
||||
|
||||
以下页面均在调研日实际打开;MCP 特意引用固定 `2025-06-18` 版本,作为兼容基线,不宣称它是最新协议。厂商专有能力必须探测,不能因为端点兼容 `/v1` 就认为全部可用。
|
||||
|
||||
| 编号 | 官方来源 | 已核实内容 | 对求道的设计启示 |
|
||||
|---|---|---|---|
|
||||
| S01 | [OpenAI Agents:集成与可观测性](https://developers.openai.com/api/docs/guides/agents/integrations-observability) | MCP 接入与运行、模型、工具等 tracing | 统一 call/attempt/span 关联;产品只显示阶段摘要,详细事件可展开 |
|
||||
| S02 | [OpenAI:Memory 与 Compaction](https://developers.openai.com/cookbook/examples/agents_sdk/building_reliable_agents_memory_compaction) | 区分当前任务上下文压缩、跨运行记忆、可审阅产物 | 历史不要无限拼接;完整产物不等于聊天全文 |
|
||||
| S03 | [LangGraph Persistence](https://docs.langchain.com/oss/python/langgraph/persistence) | checkpointer 保存 thread 状态,store 支撑跨 thread 数据 | 恢复执行状态和用户长期知识是两种生命周期 |
|
||||
| S04 | [LangGraph Interrupts](https://docs.langchain.com/oss/python/langgraph/interrupts) | 恢复时节点代码可能重新执行 | 检查点不能单独保证写操作只执行一次,必须记录效应与幂等键 |
|
||||
| S05 | [Claude Tool search](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool) | 工具定义按需进入模型上下文 | 工具多不代表每轮注入全部定义;检索后再校验权限 |
|
||||
| S06 | [Claude Programmatic tool calling](https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling) | 在执行环境中编排工具、筛选结果 | 大批量只读处理可减少模型往返;后置于基础工具可靠性 |
|
||||
| S07 | [Deep Agents overview](https://docs.langchain.com/oss/python/deepagents/overview) | 文件系统、上下文管理、子任务与可选规划能力 | 规划应按任务启用;简单问答不强制复杂工作流 |
|
||||
| S08 | [Gemini Tools](https://ai.google.dev/gemini-api/docs/tools) | 检索、URL、文件搜索、代码执行等工具面 | 覆盖“找资料—读内容—计算—形成产物”,不能只有工程检索 |
|
||||
| S09 | [Gemini File search](https://ai.google.dev/gemini-api/docs/file-search) | 文件检索与引用信息 | 命中要能定位原文,检索摘要不能代替源文件版本 |
|
||||
| S10 | [MCP Tools](https://modelcontextprotocol.io/specification/2025-06-18/server/tools) | schema、结构化结果、资源链接;annotations 有信任边界 | 工具结果包含短摘要和资源引用;注解不是权限凭证 |
|
||||
| S11 | [MCP Resources](https://modelcontextprotocol.io/specification/2025-06-18/server/resources) | URI 资源、读取和应用决定的上下文注入 | “预览”“加入上下文”“执行”必须是独立动作 |
|
||||
| S12 | [Claude Artifacts](https://support.claude.com/en/articles/17153992-what-are-artifacts-and-how-do-i-use-them) | 产物在会话旁独立打开,可继续查看与编辑 | 右栏应承载正文;不应只提供离开当前会话的入口 |
|
||||
| S13 | [Mozilla PDF.js](https://mozilla.github.io/pdf.js/) | Web PDF 解析和渲染 | PDF 视觉预览和 OCR/知识提取分开交付 |
|
||||
| S14 | [DOMPurify 官方仓库](https://github.com/cure53/DOMPurify) | DOM 清洗、标签/属性配置;插入前清洗 | 替换前端正则“清洗器”,配合同源资源代理和 CSP |
|
||||
|
||||
上述设计启示是本次推导,相关性能收益、成本下降比例不照搬厂商基准。工程路线不绑定某个最新模型或套餐。
|
||||
|
||||
## 3. 源码事实清单
|
||||
|
||||
下列 qdagent 路径相对该仓库;Marqdo 路径相对本仓库。基线 commit 已固定,后续实现需按符号再定位。
|
||||
|
||||
| 编号 | 当前实现证据 | 结论与限制 |
|
||||
|---|---|---|
|
||||
| F01 | qdagent `agent/qdagent.mq.md::契约提示`、`ask` | ASK 和 BUILD 都要求完整六段式 `.mq.md`,包含 main、验证和元数据;普通回答变长有明确结构性诱因。不能断言这是唯一原因 |
|
||||
| F02 | 同文件 `ask` 的门禁修复分支 | `r2` 送入第二次门禁后,成功分支仍 `result = raw`;可能展示修复前内容,设计必须绑定同一 accepted candidate |
|
||||
| F03 | 同文件 `ask` 返回表 | `gated_error` 路径仍可能保留 `status=ok` 且返回 `ok=True`;必须明确业务失败与输送成功的不同含义 |
|
||||
| F04 | 同文件 `eng_tools` | 默认白名单是 `tool_find/reuse/bind/verify/summarize` 五个工程工具;不是整个框架只有五个工具,也不代表已覆盖网页、笔记、文件编辑 |
|
||||
| F05 | `ext/ai/agent.mq.md` | 有 step/plan/run、READ、CALL、观察压缩、工作簿和恢复接口;应扩展这些原语,不另造循环 |
|
||||
| F06 | `plugins/agent/src/resume.rs::resume_save` | JSON 直接 `fs::write`;现有恢复接口不等于具备原子检查点、工具效应对账及任意步骤自动续跑 |
|
||||
| F07 | `src/runtime/`、`src/workbench/capabilities.rs` | 已有隔离执行、取消、事件回放与读通道;这一轮是 Agent 级恢复补足,不重新规划整个进程池 |
|
||||
| F08 | qdagent `session-store.ts::resumeTurn` | 终态 completed/failed/interrupted 直接返回;已有事件续订,但终态消息补齐与中断后的执行恢复需分别处理 |
|
||||
| F09 | qdagent `DesktopFrame.tsx::RightNotesPanel` | 实际只有“查看全部笔记”导航至 `/notes`,没有侧栏笔记正文 |
|
||||
| F10 | 同文件 `RightFilesPanel`、`WorkspacePage.tsx::openFile` | 文件点击已调 `openTarget`;右栏取首个 workspace,根目录最多 40 个非目录项。不能笼统写“所有按钮都跳转” |
|
||||
| F11 | `DesktopFrame.tsx` 的 reader/rightbar 条件 | `DocumentReader` 与右侧 tab 容器互斥,且使用两套开关/宽度状态;不是统一的资源浏览面板 |
|
||||
| F12 | qdagent `NotesPage.tsx` detail | 剥离 frontmatter 后以 `whiteSpace: pre-wrap` 显示文本;没有 Markdown/Marqdo 正文结构渲染 |
|
||||
| F13 | qdagent `DocumentReader.tsx` | 已有 DocumentView、引用版本传递、请求代数防迟到覆盖;`profile_item` target 未进入读取分支,格式不是通用资源分派 |
|
||||
| F14 | 同文件 `sanitizeHtmlFragment` | 主要靠正则移除 script/事件/javascript URL;不能当作完整 HTML 白名单安全边界。未在本次做漏洞利用实验 |
|
||||
| F15 | `src/workbench/document_view.rs` | 有只读文档视图,注释与实现明确避开执行路径;已解析语义应继续复用 |
|
||||
| F16 | `src/workbench/capabilities.rs` | text/markdown 提取可用,pdf/docx/ocr 标 unavailable;提取能力与浏览器 PDF 预览能力须分别探测 |
|
||||
| F17 | qdagent `lib/web_search.mq.md` | 既有网页搜索桥使用 legacy 脚本和固定 `last_web_search.json`;存在并发覆盖设计风险,不能直接作为新工具通道 |
|
||||
| F18 | `plugins/agent/src/mcp_client.rs` | 已有 stdio connect/list/call/close;不能把本轮描述为“从零添加 MCP”,远程连接和资源能力需另验 |
|
||||
| F19 | qdagent `compose_messages` / `ask` | 对话按角色标记拼成字符串注入 standing;会话语义、上下文预算和外部内容信任层级仍需强化 |
|
||||
| F20 | qdagent `lib/workbench.mq.md::documents_read` | 临时文件名以秒和环境 RANDOM 构造,RANDOM 缺失又退到秒;设计仍需真正唯一临时文件。workspace 缺省 `.` 不应作为新授权资源接口规则 |
|
||||
|
||||
这些是本轮优先级依据。10-26 release notes 所列修复作为已有基础保留,但“Implemented(本仓)”不能替代浏览器、真实 HTTP、进程中断与模型输出的联合验收。
|
||||
|
||||
## 4. 能力覆盖与差距优先级
|
||||
|
||||
| 能力 | 当前基础 | 本轮目标 | 优先级 / 归属 |
|
||||
|---|---|---|---|
|
||||
| 自动纠错 | 一次文档门禁修复、plan patch | 分类错误、有限修复、正确结果替换、可恢复失败 | P0 / 两仓 |
|
||||
| 执行恢复 | 任务隔离、取消、事件回放、JSON checkpoint | 传输续订与执行续跑分离;效应对账 | P0 / Marqdo 原语,qdagent UX |
|
||||
| 回答长度 | 六段式文档、完整历史 | 直接回答、摘要+产物、显式详述三种表达 | P0 / qdagent 策略,Marqdo 输出通道 |
|
||||
| 资源阅读 | DocumentView / Locator | 笔记、工作文件、引用、产物统一入口 | P0 / 两仓 |
|
||||
| 侧栏预览 | 两套状态与导航入口 | 单一右栏,点击即预览,聊天不中断 | P0 / qdagent |
|
||||
| 工具使用 | 五工具默认白名单、MCP/库原语 | schema 注册、按需发现、资源读取/搜索/写入/验证 | P1 / 两仓 |
|
||||
| 联网研究 | legacy 搜索脚本 | search+fetch、来源时间、限量正文、可定位引用 | P1 / 两仓 |
|
||||
| 上下文与记忆 | EFI pack、profile、history | 分层上下文、预算、可编辑记忆、资源 pin | P1 / 两仓 |
|
||||
| 多格式 | 文档视图、文本提取 | text/md/mq/code/json/csv/image;PDF 后续 | P0→P1 / 两仓 |
|
||||
| 长任务与计划 | plan 工作簿 | 里程碑、预算、暂停/继续、产物进度 | P1 / 两仓 |
|
||||
| 多智能体与浏览器 | 不作为当前完成前提 | 有界委派、隔离执行、受控浏览器工具 | P2 / 框架主导 |
|
||||
| 可观测评测 | usage、事件与既有测试 | 工具/恢复指标、实际任务语料、回归门禁 | P0 起持续 / 两仓 |
|
||||
|
||||
## 5. 采用与暂缓的决策
|
||||
|
||||
采用:有界失败恢复;工具结果摘要+资源引用;结构化参数;步骤效应账本;产物与聊天投影分离;资源预览与“加入上下文”分离;用户可检查的引用;按需工具发现。原有源码、知识、工作簿与验证证据仍是权威,衍生视图可重建。
|
||||
|
||||
暂缓:全自动跨站电脑操作、任意外部 MCP 自动信任、默认多智能体协作、所有 Office 格式保真编辑、每次问答都规划/运行代码。它们不能先于当前恢复、工具授权和预览闭环。
|
||||
|
||||
不采用:把长回答直接 substring 截断、把失败原文当最终成功、前端断流即重发任务、把所有文件当 HTML、用网页内容改写 system 指令、把预览等同于执行可执行文档。
|
||||
|
||||
## 6. 必须补做的动态验证
|
||||
|
||||
1. 在真实聊天中验证六段式输出与门禁失败/修复的最终消息和持久产物是否一致。
|
||||
2. 丢弃最后一帧、断网、刷新、worker 崩溃分别注入,确认事件重放不重做工具副作用。
|
||||
3. 使用真实 HTTP 入口验证右栏笔记、嵌套文件、历史引用、跨 workspace 资源;不只调用兄弟辅助函数。
|
||||
4. 用恶意/畸形 HTML、路径与符号链接、过大文件验证预览隔离;no-exec 通过哨兵副作用测试。
|
||||
5. 对无凭证模型、text-only provider、不可用转换器做明确降级;记录版本与环境,禁止以 mock 成功声称上线成功。
|
||||
|
||||
后续设计文档中的时间、轮数、字符数、成功率阈值均为拟定验收目标,需用该语料校准。
|
||||
@@ -0,0 +1,194 @@
|
||||
# 跨仓 Agent 可靠性开发计划(2026-10-27)
|
||||
|
||||
| 项目 | 约定 |
|
||||
|---|---|
|
||||
| 状态 | **Proposed / planning**;本文是排期与验收编排,**不是**已实现声明 |
|
||||
| 日期 | 2026-10-27(Asia/Shanghai) |
|
||||
| 基线 | Marqdo `6ffe5ce` / 2.1.4;qdagent `c4c976c`(设计文档基线曾为 `16f26d7` / `68b757e`;排期以当前 HEAD 为准) |
|
||||
| 协议真源 | [智能体能力补足设计](../design/agent-reliability-tools-resources-2026-10-27.md) |
|
||||
| 产品真源 | [qdagent 智能体能力与预览](../../../qdagent/doc/design/agent-capabilities-and-preview-2026-10-27.md)(同级 checkout) |
|
||||
| 证据 | [官方调研与 F01–F20](../research/agent-capabilities-2026-10-27.md) |
|
||||
| 产品侧镜像 | [qdagent 开发计划镜像](../../../qdagent/doc/roadmap/agent-capabilities-development-2026-10-27.md) |
|
||||
|
||||
本文只编排依赖、落点、门禁与发布次序。**不发明第二套 Agent Framework**;不修改 `doc/next/`;不把 capability bit 在合同实测前标真。协议字段以设计文档为准,路线图只引用不另造 schema。
|
||||
|
||||
## 1. 目标与架构边界
|
||||
|
||||
让同一套文档原生 Agent 在模型错误、工具失败、网络中断后有界恢复;工具可发现、可验证、可审计;结果同时服务模型上下文与用户阅读。保留 `.mq.md` 工作簿、EKC 与验证证据的权威地位。
|
||||
|
||||
| 议题 | Marqdo 所有 | qdagent 所有 |
|
||||
|---|---|---|
|
||||
| 恢复 | ErrorV1、预算、CheckpointV2、效应账本、运行/事件语义 | 恢复按钮、会话关联、凭证修复入口、产品政策 |
|
||||
| 工具 | ToolSpec/Call/Result、schema、权限执行、并发、MCP 传输 | 业务工具、默认集合、连接配置、用户授权范围 |
|
||||
| 回答 | AgentOutput 分通道、输出预算、ContextPackV2 | 简洁/标准/详细策略、摘要与产物卡、ASK/BUILD |
|
||||
| 资源 | ResourceRef/View、只读预览、转换插件接口 | 笔记/文件/产物 ID 映射、owner 权限、预览布局 |
|
||||
| 知识 | preflight/reuse/verify/compile、记忆原语 | 何时提议沉淀、用户画像与可见管理 |
|
||||
| 调试 | span、结构化事件、用量与错误码 | 可读过程、运行详情、产品评测语料 |
|
||||
|
||||
## 2. 现状(不再重做)
|
||||
|
||||
| 已落地基础 | 说明 |
|
||||
|---|---|
|
||||
| Agent A0–A4、O1–O5 | 文档原生 step/plan/run、沙盒与优化波次;扩展 `ext/ai/agent`,不另造循环 |
|
||||
| 隔离 Web 运行时(ADR 0009) | `src/runtime/` Verified;本轮补 Agent 步骤语义,不重建进程池 |
|
||||
| Workbench 合同切片 | DocumentView / Locator / capabilities;Partial,随本轮补 F 矩阵 |
|
||||
| 可执行知识 S0–S6 | 本地确定性子集已门禁;**不重开 S0–S6** |
|
||||
|
||||
本轮增量(设计已锁、代码待做):`AgentErrorV1`、`CheckpointV2` + effect ledger、`ToolSpecV1` 统一调度、`ResourceViewV1`、`AgentOutputV1` 通道、`ContextPackV2`。
|
||||
|
||||
**产品措辞约束**:本地门禁绿 ≠ 远程/多租户安全;≠「知识验证 + Agent 复用闭环」全产品完成。
|
||||
|
||||
## 3. 跨仓依赖与日历顺序
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
MA0[M-A0 contracts] --> QA0[Q-A0 fixtures]
|
||||
MA0 --> MA3[M-A3 ResourceView]
|
||||
MA0 --> MA1[M-A1 Error and Checkpoint]
|
||||
MA0 --> MA2[M-A2 ToolSpec]
|
||||
QA0 --> QA1[Q-A1 answer outcome]
|
||||
QA0 --> QA2[Q-A2 right panel]
|
||||
MA3 --> QA2
|
||||
QA1 --> QA3[Q-A3 recovery UX]
|
||||
MA1 --> QA3
|
||||
MA2 --> QA4[Q-A4 product tools]
|
||||
MA1 --> MA4[M-A4 channels ContextPack]
|
||||
MA2 --> MA4
|
||||
MA4 --> QA4
|
||||
MA3 --> QA5[Q-A5 formats]
|
||||
MA5[M-A5 ledger PDF MCP] --> QA5
|
||||
MA4 --> QA6[Q-A6 plan memory]
|
||||
MA5 --> QA7[Q-A7 connectors]
|
||||
MA6[M-A6 subagents] --> QA7
|
||||
```
|
||||
|
||||
### 3.1 强制顺序原则
|
||||
|
||||
1. 先 **M-A0 ∥ Q-A0** 冻结跨仓字段与缺陷 fixtures。
|
||||
2. 立刻 **Q-A1 + Q-A2 + M-A3(文本/md/mq)**——**不得**被完整 CheckpointV2 阻塞。
|
||||
3. **M-A1** 可先上只读重试 + 原子 checkpoint;**Q-A3** 终态快照可先行,写恢复等 effect ledger。
|
||||
4. 再 **M-A2 ∥ Q-A4** → **M-A4 / Q-A6** → **M-A5 / Q-A5** → **M-A6 / Q-A7**。
|
||||
5. 四类恢复必须分开:事件续订 / 步骤重试 / 工作流 resume(新 attempt + `resumed_from`)/ 答案再生成。
|
||||
|
||||
## 4. 阶段包明细
|
||||
|
||||
### Wave 0 — 合同
|
||||
|
||||
| 包 | 修改/新增建议 | 交付内容 | 退出门槛 |
|
||||
|---|---|---|---|
|
||||
| **M-A0** | `src/workbench/capabilities.rs`、`tests/workbench_contracts.rs`、新增合同 fixtures | Error/Tool/Resource/Output/Context 协议、旧适配、状态映射;capability **默认 false** | 跨仓字段冻结;未知字段/版本可降级 |
|
||||
| **Q-A0** | 缺陷样例、HTTP 路由清单、answer/resource fixtures;设计入口 | 固化 F01–F20 相关 fixtures;`mock`/`live` 证据区分 | 与 M-A0 字段一致;不把 mock 当 live |
|
||||
|
||||
### Wave 1 — 可见痛点
|
||||
|
||||
| 包 | 修改/新增建议 | 交付内容 | 退出门槛 |
|
||||
|---|---|---|---|
|
||||
| **Q-A1** | `agent/qdagent.mq.md`、`gateway/openai`、api 映射 | accepted candidate;`outcome` ≠ HTTP `ok`;摘要投影;ASK 不强制六节 | 修复后只展示 B;未通过不标成功(F01/F02/F03) |
|
||||
| **M-A3(首批)** | `src/workbench/document_view.rs`、新 `resource_view`、媒体适配 | ResourceRef/View;md/txt/code/mq 静态阅读;no-exec | 资源定位与安全语料通过(F15) |
|
||||
| **Q-A2** | `DesktopFrame`、panel store、`DocumentReader`、ResourcePanel/Preview | 统一右栏;笔记/文件点击不导航;同 renderer | 所有入口同预览(F09–F14) |
|
||||
|
||||
### Wave 2 — 恢复
|
||||
|
||||
| 包 | 修改/新增建议 | 交付内容 | 退出门槛 |
|
||||
|---|---|---|---|
|
||||
| **M-A1** | `ext/ai/agent.mq.md`、`plugins/agent/src/resume.rs`、recovery/effects 模块 | ErrorV1、有界修复、accepted candidate 绑定、CheckpointV2 原子写 | 故障注入不重复写;旧记录不冒险自动恢复(F06) |
|
||||
| **Q-A3** | `execution/turns`、session store、`api.ts` | 终态快照补齐、恢复动作、错误卡 | 事件恢复 0 工具调用;写恢复须 ledger(F08) |
|
||||
|
||||
### Wave 3 — 工具与通道
|
||||
|
||||
| 包 | 修改/新增建议 | 交付内容 | 退出门槛 |
|
||||
|---|---|---|---|
|
||||
| **M-A2** | `plugins/agent/src/lib.rs`、`mcp_client.rs`、tool registry/dispatcher | ToolSpec/Call/Result;CALL+native 单通道;分页与结果资源 | 未授权调用为 0;失败结果结构正确(F04/F05/F18) |
|
||||
| **M-A4** | `ext/ai/agent.mq.md`、`plugins/llm`、`src/workbench/run_context.rs` | AgentOutput 通道、预算、ContextPackV2 | 原生与纯文本 provider fixture 可用(F01/F19) |
|
||||
| **Q-A4** | `agent/tools/`、工具目录、ContextAttachment | 业务工具;保留旧五工程工具别名 | 找→读→改→验证→预览真实闭环 |
|
||||
| **Q-A6** | 计划折叠、预算 UI、profile 忘记、候选沉淀 | ResponsePolicy UX、记忆边界 | 压缩与恢复不丢用户约束 |
|
||||
|
||||
### Wave 4 — 持久与格式
|
||||
|
||||
| 包 | 修改/新增建议 | 交付内容 | 退出门槛 |
|
||||
|---|---|---|---|
|
||||
| **M-A5** | `src/runtime/` 存储/事件适配、MCP 与转换插件 | ledger↔runtime 对账、PDF/Office 插件、远程 MCP | crash/restart 通过;诚实探测(F16–F18) |
|
||||
| **Q-A5** | JSON/CSV/图片/PDF UI、网页来源与页/行引用 | 格式扩展与转换进度 | 缺插件可读降级;PDF 不假称 OCR |
|
||||
| **M-A6 / Q-A7** | plan/subtask/sandbox;MCP 配置与委派 | 程序化工具、有界委派、可选浏览器 | **后置**;须证明收益与预算/效应可控 |
|
||||
|
||||
## 5. F01–F20 → 包映射
|
||||
|
||||
| ID | 摘要 | Marqdo | qdagent | 主要验收 |
|
||||
|---|---|---|---|---|
|
||||
| F01 | ASK/BUILD 强制六节 | M-A4 输出通道 | Q-A1 ResponsePolicy | QA-01、MA-T12 |
|
||||
| F02 | 修复后仍返回 raw | M-A1 accepted 绑定 | Q-A1 | QA-03、MA-T12 |
|
||||
| F03 | `gated_error` 仍 `status=ok` | M-A0/M-A1 ErrorV1 | Q-A1 envelope | QA-04 |
|
||||
| F04 | 默认仅五工程工具 | M-A2 registry | Q-A4 目录 | QA-17 |
|
||||
| F05 | 已有 step/plan/run | **扩展** agent.mq.md | — | 不另造 loop |
|
||||
| F06 | `resume_save` 裸 `fs::write` | M-A1 CheckpointV2 | Q-A3 UX | MA-T03/T04 |
|
||||
| F07 | runtime 已有 | 复用,不重建 | 接线恢复 API | — |
|
||||
| F08 | `resumeTurn` 终态直返 | 事件 vs 再执行分离 | Q-A3 快照补齐 | QA-13、MA-T05 |
|
||||
| F09–F12 | 右栏/笔记预览缺陷 | M-A3 ResourceView | Q-A2 | QA-06–QA-09 |
|
||||
| F13 | DocumentReader 非通用分派 | DocumentView 保留 mq;ResourceView 其它 | 预览分派 | QA-09 |
|
||||
| F14 | 正则 HTML 清洗 | 生产侧 allowlist | DOMPurify + CSP | QA-11、MA-T11 |
|
||||
| F15 | DocumentView no-exec | 保持;ResourceView 禁 run/verify | — | MA-T10 |
|
||||
| F16 | pdf/docx/ocr unavailable | M-A5 插件 + 诚实探测 | Q-A5 | QA-10 |
|
||||
| F17 | legacy web_search 临时文件 | 优先框架 net 工具 | 迁移工具 | QA-17、F20 |
|
||||
| F18 | MCP stdio 已有 | M-A2/5 timeout/cancel/remote | 连接器配置 | MA-T07 |
|
||||
| F19 | 历史字符串 standing | M-A4 ContextPackV2 | compose_messages | QA-19、MA-T13 |
|
||||
| F20 | 弱临时名;workspace `.` | 安全临时文件 + ResourceRef | 资源解析 | QA-08、MA-T11 |
|
||||
|
||||
## 6. 验收与发布
|
||||
|
||||
### 6.1 Marqdo 门禁(MA-T01–T14)
|
||||
|
||||
以设计文档 §10 为准;基础机制用确定性 fake provider/tool;真实 provider 少量单独 smoke 并记费用。摘要:
|
||||
|
||||
| 编号 | 必须观察到 |
|
||||
|---|---|
|
||||
| MA-T01 | 429 后成功:退避、计量、单一 accepted answer |
|
||||
| MA-T02 | schema 连续错:一次修复后停;无效参数不执行工具 |
|
||||
| MA-T03 | 写成功但 receipt 前提杀:unknown_effect→对账;写计数不增 |
|
||||
| MA-T04 | checkpoint 写一半杀进程:完整上一检查点或明确 corrupt |
|
||||
| MA-T05 | 终态事件丢失:快照恢复完整消息;工具次数不增 |
|
||||
| MA-T06 | 重复/并发恢复:同一幂等键一个 attempt |
|
||||
| MA-T07 | 取消 vs 重试:取消后无新工具调用 |
|
||||
| MA-T08 | 权限/版本变更:stale/conflict/forbidden |
|
||||
| MA-T09 | 伪造 CALL/system:不越白名单 |
|
||||
| MA-T10 | `.mq.md` 预览含写/联网:哨兵不变;只读或安全降级 |
|
||||
| MA-T11 | 格式/超长/路径逃逸:正确 renderer;越界拒绝;无注入 |
|
||||
| MA-T12 | 修复 A→B:answer/artifact/verification 均绑定 B digest |
|
||||
| MA-T13 | 上下文压缩后:约束、待决效应、版本、预算不丢 |
|
||||
| MA-T14 | 旧 CALL/DocumentView:基础可用;缺失能力明确返回 |
|
||||
|
||||
先跑受影响的 Rust/Go/Marqdo 合同与隔离 runtime 金样例,再做跨仓 HTTP/browser gate。性能目标(须记录机器/并发/冷暖缓存):已缓存 100 KiB 文本视图服务端 p95 < 300 ms;核心工具发现 p95 < 100 ms。
|
||||
|
||||
### 6.2 qdagent 门禁(QA-01–QA-20)
|
||||
|
||||
以产品设计 §13 为准;路径必须 `serve → api → 权威业务函数 →` 真实前端。零容忍:重复写效应、预览执行副作用、跨 workspace 越权。
|
||||
|
||||
实现时:`python3 scripts/check-contracts.py`、相关 workbench 测试、前端 `npm test` / `npm run build`,再跑 Playwright;无服务时 E2E 须明确 skip/失败原因。
|
||||
|
||||
### 6.3 Feature flags 与灰度
|
||||
|
||||
建议开关:`agent_output_v2`、`resource_panel_v2`、`agent_tools_v1`、`agent_recovery_v2`。启用取决于 runtime capability + 服务端政策,不由前端单独决定。
|
||||
|
||||
灰度:内部单 workspace → 多 workspace → 真 provider/断网 → 默认开启。关闭恢复时必须停止接纳新恢复请求并保留 attempt 状态,**禁止**切回会盲重做写操作的旧路径。
|
||||
|
||||
兼容迁移:新增协议 → 双读 → 新写 → 验证 → 逐步移除旧适配。旧 resume JSON 可读但不能自动升级成带副作用保证的检查点。
|
||||
|
||||
## 7. 伴生轨(登记,不阻塞主线)
|
||||
|
||||
| 轨 | 动作 |
|
||||
|---|---|
|
||||
| 可执行知识 S0–S6 | **不重开**;消费 BoundCall/EKC;话术保持「本地子集」 |
|
||||
| 知识支撑 F0(M02/M03/M05) | **并行伴生**:provenance/closure invalidate、真 verify API、Skill 无冲突 |
|
||||
| Workbench Partial | 随 M-A0/M-A3 补 F 矩阵与 adapters;不重写 M00–M05 |
|
||||
| 语言 next-phase P3 / Wave B1 评测 | **延后** |
|
||||
| Web Document/Endpoint | 遵守 ADR 0007;禁止 compose_/json bag 回退 |
|
||||
|
||||
## 8. 下一步开工(第一刀实现切片)
|
||||
|
||||
确认本路线图入库后,**另开实现回合**(本文件仅文档):
|
||||
|
||||
1. **M-A0 + Q-A0**:合同与 fixtures
|
||||
2. **Q-A1**:raw→accepted、outcome 一致性
|
||||
3. **Q-A3 子集**:终态快照补齐
|
||||
4. **Q-A2 + M-A3 子集**:右栏真实预览(笔记 / md / mq / txt / code)
|
||||
|
||||
发布判据是 MA-T* / QA-* 与真实 HTTP/browser 证据,不是接口返回 200 或代码中出现某符号名。
|
||||
@@ -0,0 +1,230 @@
|
||||
# Marqdo 现状与规划缺口:从可执行文档到可复用知识
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | 现状审计与下一阶段规划;不是已实现能力声明 |
|
||||
| 审计基线 | `main` / `434be9f`,2026-10-23;后续改动须重新核验对应判断 |
|
||||
| 产品意图 | [009.md](../next/009.md) · [详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md) · [完整示例](../next/Marqdo_Executable_Knowledge_Example.md) |
|
||||
| 目标 | **代码即文档,文档即知识库,知识库可执行,可执行知识可复用** |
|
||||
| 方向修订 | 2026-10-23:按[文档原生开发指导](../design/document-native-development-guide.md)重排交付,G01–G12 保留为缺口证据,不等于作者必须经历的治理流程 |
|
||||
|
||||
## 1. 本文解决什么问题
|
||||
|
||||
仓库已经拥有语言运行时、双后端、标准库、扩展、文档站、EKC、Agent 和一批语义制品结构。现有[详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md)定义了目标架构和 M0–M6,但它主要写于目标能力尚未落地之时;现在照它逐项勾选,容易把“结构存在”“CLI 可调用”“目标闭环成立”混为一谈。本文只补**现状与目标之间尚未被规划成可验证闭环的工作**。语法和产品目标仍以前述意图文档为准;旧文档中的 Classic 链接取元、单体知识类、或另一套知识索引不应反向规定新语义。
|
||||
|
||||
本轮判断以源码、可定位的测试和文档状态为证据。`done` 表示有可运行入口和回归测试;`partial` 表示有结构或单一路径,仍不满足产品判据;`design` 表示主要存在于文档和目标示例。**不能把能解析、能运行、能检索、能验证事实视作同一个结论。**
|
||||
|
||||
### 1.1 对上一版规划排序的修正
|
||||
|
||||
上一版把内部工程完整度放在主要位置,直到 S6/S7 才验收 Agent 复用和作者体验,容易造成“先建治理平台,再证明文档有用”的偏差。本版改为从可读步骤、确定执行和早期本地复用向外扩展;每片同时交付源码示例、诊断和消费者可见结果。工程基础按切片所需完成,不降低已开放路径的安全要求。
|
||||
|
||||
P0/P1/P2 分类改为缺口领域,不再按编号整组串行:G06 的未知效应安全下限必须进入最早执行试点,G09 的本地复用也不再等锁和远程授权全部完成。复杂业务的证据、严格锁和远程凭据按使用场景增加,不能强加给每个单文件程序。每片还须记录作者重复声明数、局部修改范围和实际复用结果,指标见[指导 §8](../design/document-native-development-guide.md#8-验证-marqdo-优势而不只验证组件存在)。
|
||||
|
||||
## 2. 设计思想落实为四个可证伪承诺
|
||||
|
||||
| 承诺 | 必须由什么证明 | 当前判定 |
|
||||
|---|---|---|
|
||||
| 代码即文档 | 一份 `.mq.md` 的叙述、契约、示例、可执行语句由同一解析结果驱动;改变语义会让说明/示例失配可见 | 运行与 view 已有基础;正文引用尚未进入完整语义 AST(partial) |
|
||||
| 文档即知识库 | 可寻址的事实、规则、步骤、来源、适用范围和关系由编译器生成;不同消费者读取同一身份和图 | 项目索引与 Artifact 投影已起步;双图和占位字段仍存在(partial) |
|
||||
| 知识库可执行 | 引用先绑定、检查参数/契约/效应/版本,再由运行时执行**绑定的目标**;正文默认不触发副作用 | CLI 可解析、运行时可执行,但两条路径未闭合(partial) |
|
||||
| 可执行知识可复用 | Agent 发现候选后取得有条件、可失效的句柄,实际调用既有制品;trace 记录符号、摘要、输入和结果 | MLSP 有初版 handle;EKC/Agent 尚未统一到此协议(partial) |
|
||||
|
||||
这四条有语义依赖,但应在受限故事中逐步贯通,不是四个必须全部做完才能进入下一层的大项目。阅读体验和检索召回不能代替绑定;绑定不能证明知识内容正确;执行成功不能证明已经复用既有知识。
|
||||
|
||||
## 3. 仓库能力盘点与规范 M0–M6 的真实进度
|
||||
|
||||
| 既有里程碑 | 可复用成果 | 仍未满足的完成条件 |
|
||||
|---|---|---|
|
||||
| M0 语义与测试 | Occlusion 已为默认;[语法测试规程](../next/Marqdo_Executable_Knowledge_Testing.md)和部分金样存在;标准库、扩展库已经迁移 | `tests/executable_knowledge/` 系统夹具、100 单元生成器、导航/正文/效应负例矩阵未形成独立门禁 |
|
||||
| M1 项目身份 | [`src/project/mod.rs`](../../src/project/mod.rs)确定项目根和扫描顺序;[`src/sema/symbols.rs`](../../src/sema/symbols.rs)产生 ModuleId/SymbolId;CLI `knowledge inspect/resolve` 可用 | 身份保留、移动/重命名、symlink/重复扫描、大项目增量与错误聚合还需完整策略和测试 |
|
||||
| M2 统一链接与调用 | 裸零参路径和关联函数已有 Tree/Bytecode、CLI 金样;`Expr::Occluded` 已承载标签和 payload | [`src/ast.rs`](../../src/ast.rs)仍无统一 Named/正文引用节点;[`src/sema/ir.rs`](../../src/sema/ir.rs)只提取执行体部分调用;运行时仍独立按名称寻找调用目标 |
|
||||
| M3 语义制品 | SemanticArtifact、EffectSet、SourceRef、VerificationRecord、JSON 投影已存在 | 适用性、来源证据、单位、真实验证状态、完整效应闭包与规范摘要仍缺;字段存在不代表语义已实现 |
|
||||
| M4 锁与影响 | `knowledge lock/impact` 可用;[`src/sema/lock.rs`](../../src/sema/lock.rs)支持原子写入和反向依赖查询 | 锁只比较 artifact 摘要;目标、接口、实现、证据、传递闭包变更未独立报告;正文/证据边不在影响闭包内 |
|
||||
| M5 消费者 | view 显示部分语义信息;MLSP 提供 `artifact.*`;EKC/Agent 有各自检索与复用能力 | 同一图/同一绑定尚未贯通;Agent 对 ready 的判定与执行句柄未被强制;trace 不足以证明实际复用 |
|
||||
| M6 迁移与发布 | 全仓新语法迁移、`--legacy-markup` 入口、v2.1.0 发布已完成 | 发布不等于 M0–M5 已完成;目标示例仍注明“尚不能完整运行”,100 单元与 Agent 对照实验未过 |
|
||||
|
||||
`examples/executable-knowledge/README.md` 是目标示例,仍明确说明类函数、统一 `Named<T>` 和步骤图的完整运行是设计目标。它不得作为“产品闭环已完成”的发版证据。
|
||||
|
||||
### 3.1 相邻路线如何进入本规划
|
||||
|
||||
| 仓库区域 | 当前路线状态 | 与可执行知识主线的连接点 |
|
||||
|---|---|---|
|
||||
| Interpreter / 标准库 | Tree、Bytecode、`lib/` 与中英库有大量金样;[Mid2](stdlib-mid2.md)已完成 | 作为 BoundCall、效应声明、签名与迁移回归的基础;不为本路线重做通用库 |
|
||||
| Web / 浏览器 | [Web](ext-web.md) W 波与 [WASM](browser-wasm.md) C 波已落地 | 作为 view/静态投影和宿主能力消费者;知识引用不能在静态导出或浏览器预览中偷跑服务端 effect |
|
||||
| Agent / LLM / 沙盒 | [Agent](ext-agent-optimize.md) Wave B 有局部待项;[优化 O 波](ext-ai-optimize.md)已完成 | 工作簿、OKF、沙盒可复用;S2 接本地 bound handle,S4/S5 增强政策;不另建平行知识身份 |
|
||||
| 外联 / Notebook | [外联 F4](foreign-v2.md)和[会话 D4–D5](notebook-session.md)尚有余项 | 外部内核、持久状态和重放需显式建模为 effect/环境依赖;不能把会话结果当纯制品缓存 |
|
||||
| 领域扩展 | [Quantum Q8c](ext-quantum.md)等剩余项属领域体验;[宿主 H4b](ext-hosting.md)为可选 HTTP 能力 | 不阻塞语义主链;但扩展元数据与插件调用必须参加 G06/G10 的 effect 和兼容性门禁 |
|
||||
|
||||
这些路线各有自己的完成定义。本文不将其余项全部吸收到下一版核心范围;只规定它们与统一语义层交互时必须遵守的契约。
|
||||
|
||||
## 4. 语义一致性缺口:消除错误的语义真相
|
||||
|
||||
### G01 两份图与同名输出路径
|
||||
|
||||
**证据。** [`src/knowledge/compile.rs`](../../src/knowledge/compile.rs)先调用 `compile_project` 并写语义投影,随后重新用 `extract_graph` 从源码提取旧 EKC 图。后者的 `write_projections` 再写 `graph/edges.json`;[`src/knowledge/graph.rs`](../../src/knowledge/graph.rs)的调用边按短名选首个匹配符号。由此“语义边”和“检索边”既不是同一事实,也会争用同一个文件名。
|
||||
|
||||
**规划缺口。** 详细规范要求 EKC 是 `SemanticProject` 投影,却未给出**并行迁移期间的文件命名、读取优先级、冲突检测与废弃时点**。只新增语义图不解决消费者继续读旧边的问题。
|
||||
|
||||
**决策与交付。** 明确一个权威 `SemanticProject`;图边按稳定 `SymbolId/ArtifactId` 生成。旧检索索引仅含查询优化数据,不再自行确定 calls/uses。若迁移期保留旧 JSON,放入独立版本化路径并声明其非执行依据。`knowledge compile`、`view`、`check`、`impact`、MLSP、Agent 对同一引用应报告同一目标及诊断。
|
||||
|
||||
**验收。** 制造两个模块同名函数并让调用只 import 其中一个;语义图、CLI resolve、Tree/Bytecode trace、view 和 Agent handle 必须指向同一 ID。编译两次输出字节一致;生成物中没有同名文件被后续投影覆盖。
|
||||
|
||||
### G02 绑定没有成为执行的前置条件
|
||||
|
||||
**证据。** [`src/main.rs`](../../src/main.rs)的 `knowledge resolve` 走项目索引;[`src/mlsp.rs`](../../src/mlsp.rs)的 `link.resolve` 通过 artifact 名字/后缀匹配;[`src/interp.rs`](../../src/interp.rs)仍有自身的 `eval_path_call` 和函数查找;[`src/bytecode/`](../../src/bytecode/)也有独立调用实现。`compile_project` 目前只将同模块无接收者调用识别为静态依赖,其余记为 `external_or_dynamic`。这几条路径存在真实分叉。
|
||||
|
||||
**规划缺口。** 现有规范提出 Bound IR,但缺少**最小可合并的执行切面**以及动态调用的明确边界。不能先宣称图边覆盖执行调用,再等待后端改造。
|
||||
|
||||
**决策与交付。** 先让显式 import、普通函数、关联函数这三个静态路径通过唯一 resolver,生成 `BoundCall { target, signature, source_span, required_effects }`。Tree 和 Bytecode 只消费 BoundCall;确需动态解析的路径使用独立 `DynamicCall`,在图/诊断中显式标记,不能假装已绑定。`run/check/knowledge resolve/MLSP` 使用相同 import 范围和歧义规则。
|
||||
|
||||
**验收。** 同名跨模块、重命名、缺 import、实例方法误用、动态目标分别有稳定诊断;任一静态 trace 中执行的 SymbolId 与编译图边一致。错误不得因后端不同而变成另一目标。
|
||||
|
||||
### G03 `[]()` 只有标签,还没有完整知识步骤
|
||||
|
||||
**证据。** [`src/ast.rs`](../../src/ast.rs)保留 `Expr::Occluded`;[`src/parse/prose.rs`](../../src/parse/prose.rs)扫描粗体、斜体和反引号,但没有完整 Markdown 正文链接 AST。[`src/sema/ir.rs`](../../src/sema/ir.rs)当前 KnowledgeRef/KnowledgeStep 主要由调用点提取,步骤 `kind` 为 `call`;取元、推导、遍历、正文、导航尚未同等进入图。目标示例列出的 observe/derive/iterate/narrate 不能由现有投影证明。
|
||||
|
||||
**规划缺口。** 缺少正文 inline 的位置、逃逸、代码围栏、标准 URL 与知识引用的**一套无副作用解析协议**,以及 `[]()` 在表达式、循环和正文的统一节点身份。
|
||||
|
||||
**决策与交付。** 保留现有 payload 表达式解析器;为上下文建立稳定 SyntaxNodeId 与 `Named<T>`/等价 lowering。明确 `NavigationLink`、`NarrativeRef`、`ExecutableNamed`、`LoopNamed` 的互斥分类及 span。正文引用只解析和验证;主动纯预览可使用项目/会话既有政策,不要求逐链接确认;外部 effect 仍需明确执行上下文与宿主能力许可。各类型步骤的输入、输出、依赖和角色由编译器推导;标签是阅读、检索和局部编辑的上下文,但不参与授权和寻址。作者不手工填写步骤 ID 或生成图。
|
||||
|
||||
**验收。** 一份文档同时含 URL、锚点、带空格/嵌套括号载荷、围栏、行内代码、取元、计算、循环、正文引用;生成的 ref/step 数、分类和跨度稳定,正文 `run/check/view static` 不发生外部 effect。
|
||||
|
||||
### G04 生成物写入不是整代快照
|
||||
|
||||
**证据。** [`src/sema/ir.rs`](../../src/sema/ir.rs)将语义文件写到临时目录后,逐个删除并移动 `project.json`、`symbols.json`、`refs.json`、`artifacts/`、`graph/`、`verification/`;[`src/knowledge/compile.rs`](../../src/knowledge/compile.rs)再写旧投影。这不是消费者可原子读取的整代快照,构建中途可能看到混合版本。
|
||||
|
||||
**规划缺口。** 规范写了“原子写入”,未规定**整代提交、读者快照、失败恢复和旧版本保留**。
|
||||
|
||||
**决策与交付。** 使用带 schema/build-id 的代目录与原子指针或等价快照机制;所有消费者一次选择一代,读完整个结果后才释放。生成物可删除重建;崩溃后只能看到旧完整代或新完整代。
|
||||
|
||||
**验收。** 在写每个投影文件时注入故障并并发读取;不得出现 `project` 属于 A 代而 `edges` 属于 B 代,失败后旧代仍可读。
|
||||
|
||||
## 5. 可信执行缺口:按场景落实验证门槛
|
||||
|
||||
### G05 来源、适用性和验证状态仍是占位投影
|
||||
|
||||
**证据。** [`src/sema/ir.rs`](../../src/sema/ir.rs)给函数固定 `"no explicit applicability constraints"`,`SourceRef` 主要指向定义它的源码,`evidence` 摘要来自契约文本;验证记录固定 `contract=unchecked`、`tests=unknown`、`evidence=unverified`、`domain=unreviewed`。现有 [`src/knowledge/lifecycle.rs`](../../src/knowledge/lifecycle.rs)的 `verify` 是另一套 EKC 检查。因此“有 SourceRef”并不表示政策原文被关联或核验。
|
||||
|
||||
**规划缺口。** 规范列出字段和七层状态,但没有定义**作者如何写来源/适用性、如何证明测试确实覆盖此制品、谁可将 domain 从 unreviewed 提升、何时失效**。不确定现实真假的结论必须保留 unknown。
|
||||
|
||||
**决策与交付。** 用现有 Markdown/frontmatter/契约表表达可选的 `source`、有效区间、地域/单位/输入前提,形成可解析 schema;把编译、静态检查、实跑测试、来源内容、人工领域审阅作为独立证据事件。定义状态迁移、撤销和失效规则;生成物记录证据文件/测试运行摘要,不允许 CLI/Agent 自报 `verified`。
|
||||
|
||||
**验收。** 政策原文改变而函数未改时 evidence 状态下降;来源删除、过期、输入不满足分别给不同诊断;无来源的纯工具函数可运行,但不能被宣称为已证实业务事实。
|
||||
|
||||
### G06 效应识别存在“未知即纯”的漏洞
|
||||
|
||||
**证据。** [`src/sema/ir.rs`](../../src/sema/ir.rs)的 `effects_for_call` 依据名字前缀猜测 `fs/net/llm` 等,其余返回 `pure`;`direct_effects` 只汇总直接调用。未知插件、别名、包装函数和传递调用可能因此被标作纯。[`src/mlsp.rs`](../../src/mlsp.rs)以该集合做 `artifact.execute` 的效应拒绝。
|
||||
|
||||
**规划缺口。** 详细规范定义 EffectSet,却没有给出**跨宿主函数、标准库、扩展 ABI、动态调用的权威效应注册表和未知效应策略**。
|
||||
|
||||
**决策与交付。** 内核/标准库/插件元数据声明 effects,绑定后做调用图闭包;动态或缺元数据为 `unknown`,需要显式能力授予,不得降为 pure。`view` 静态、`verify pure`、Agent ready 门槛使用同一集合。实际宿主能力限制仍在执行时强制。
|
||||
|
||||
**验收。** 用自定义名字包住写文件/网络/插件调用;静态 effect 必须传递到入口,默认 pure 策略拒绝执行。循环调用图收敛;未知动态调用拒绝 pure 证明。
|
||||
|
||||
### G07 绑定句柄未表达可复用的授权与适用性
|
||||
|
||||
**证据。** [`src/mlsp.rs`](../../src/mlsp.rs)的 `artifact.bind` 主要检查目标唯一及可选 digest,返回 `artifact_id/digest/project/policy/effects`;它没有要求 ready、契约、适用性或来源状态。`artifact.execute` 重编译比对摘要后按 `display_name` 最后一段调用解释器;调用者提交的 `allow_effects` 会变为宿主能力。现阶段这是实验性执行接口,不是可信复用协议。
|
||||
|
||||
**规划缺口。** 缺少句柄的**发行者、主体、策略、输入约束、能力上限、过期/撤销、并发源变更**模型;也缺少“绑定到精确嵌套函数”在 runtime 的保证。
|
||||
|
||||
**决策与交付。** `bind` 必须验证 SymbolId、签名、输入、已声明适用性、所选验证政策及 effect 上限;政策要求锁时检查 lock。`execute` 复核同一目标与状态并按绑定符号调用。受信本地调用先采用精确绑定引用和摘要复核,不强制搭建签发服务;远程/多租户/不受信调用再加入可信发行者、主体、范围、期限/撤销及防伪机制。客户端参数不是远程授权来源;运行时能力上限始终由宿主强制。响应如实显示政策和未知状态,作者不手写 handle。
|
||||
|
||||
**验收。** 本地先覆盖改参数、改依赖、改效应策略、同名嵌套函数和并发源码变更,不得错调用或越权;严格政策增加来源与锁变更,远程增加伪造/过期/撤销测试。`ready` 必须注明相应 policy 与条件,不是全局永久认证。
|
||||
|
||||
### G08 锁定与影响分析未覆盖知识变化
|
||||
|
||||
**证据。** [`src/sema/lock.rs`](../../src/sema/lock.rs)的 lock 文件虽然写出 interface/implementation/evidence/dependencies,但检查时只比较 `artifact` 摘要。该摘要在 [`src/sema/ir.rs`](../../src/sema/ir.rs)只组合本地 interface/implementation/evidence,没有传递依赖闭包;impact 的边也只来自已识别的本地调用。`knowledge impact` 与语义 impact 是两套命令/图。
|
||||
|
||||
**规划缺口。** 未定义**锁的审阅单位、变更分类、依赖 SCC、证据变更如何传播到正文、升级冲突处理**。
|
||||
|
||||
**决策与交付。** 分别持久化 target/interface/implementation/evidence/closure 摘要,SCC 按确定规则求闭包;strict 对未审阅目标变化失败。impact 用同一图沿 calls/depends_on/supported_by/narrates 反向传播,并说明为何被影响。每次 lock update 留下人可审阅的差异报告。
|
||||
|
||||
**验收。** 本地函数源码不变而下游税率规则变了时 closure 变化;只改描述不混同事实证据变化;来源变更影响客服说明;同一个查询在 CLI、MLSP、view 展示相同受影响 ID 集合。
|
||||
|
||||
## 6. 作者与生态缺口:从早期切片持续验证
|
||||
|
||||
### G09 Agent 仍可能把候选当作可执行知识
|
||||
|
||||
**证据。** [`src/knowledge/reuse.rs`](../../src/knowledge/reuse.rs)、[`plugins/agent/`](../../plugins/agent/)和 `ext/ai/agent.mq.md` 有各自的检索、near-match、工作簿/Skill 固化流程;[`src/mlsp.rs`](../../src/mlsp.rs)另有 artifact handle。现有 [Agent 路线图](ext-agent-optimize.md)主要跟踪工作簿、路由和评测,没有把“发现→精确绑定→验证→调用既有制品→记账”设成强制路径。
|
||||
|
||||
**规划缺口。** 需要明确 **REUSE、ADAPT、CREATE 的进入条件与失败后分流**,以及“避免重复生成”用什么客观指标衡量。
|
||||
|
||||
**决策与交付。** near/soft-match 只能供候选发现;Agent 可以选择并提交精确候选,REUSE 前必须经同一 resolver 和 policy 得到 bound handle。不可用则记录原因,按权限进入 ADAPT/CREATE 或请求补充决策。新草稿可被发现、查看和在本地明确权限下试跑,进入共享自动复用的 ready 集合前按政策验证晋升。先提供含步骤、源码、约束、负知识和验证状态的最小上下文卡,再支持带期望摘要的局部补丁和语义 diff,避免整篇再生成。trace 包含任务、候选、选定 SymbolId、所有相关摘要、输入摘要、policy、实际执行结果及因果依赖;私密输入只存可审计脱敏摘要。
|
||||
|
||||
**验收。** 已有适用能力时实际调用而不新建同义函数;同名未消歧、违反所选政策的过期/未验证候选、权限不足不能自动执行。局部修改不覆盖并发编辑;只改措辞却改变签名必须可见。同样任务与预算下报告重复实现率、错误复用率、发现率、成功率、编辑范围、作者维护成本及失败样本。
|
||||
|
||||
### G10 标准库、扩展库与发布物缺少跨层一致性门禁
|
||||
|
||||
**证据。** [语法测试规程](../next/Marqdo_Executable_Knowledge_Testing.md)覆盖了部分零参路径金样;仓库有大量 `lib/`、`ext/`、`public/` 和示例。`.github/workflows/` 目前只有 release/pages,没有对每次提交运行可执行知识项目夹具的 CI;[目标示例](../../examples/executable-knowledge/README.md)仍不可端到端运行。现有 [lib/ext 路线图](lib-ext-code-as-docs.md)的历史“链接取元”已被 v2.1 默认遮蔽语义取代,`done` 不代表今天的知识引用集成已完成。
|
||||
|
||||
**规划缺口。** 对新语义没有固定的**全仓兼容矩阵、跨平台发布前检查、示例可运行状态标签、回滚规则**。
|
||||
|
||||
**决策与交付。** 分层门禁:parser/sema → Tree/Bytecode → 标准库中英镜像 → 扩展插件装载 → CLI/MLSP/view/静态导出 → 一体化书店/订单示例。每个负例也断言诊断码和副作用计数。每次语义变化先迁移 `lib/ext` 并保持 stdout 与 effect,不通过扩展旧语义来掩盖库的过时写法。CI 在 main 的 PR/push 执行无网络核心套件;需要插件/浏览器的平台测试单列结果;发布前检查包内标准库/扩展与二进制版本匹配。
|
||||
|
||||
**验收。** `cargo test --locked`、gold 单线程、可执行知识项目夹具、CLI JSON schema 和每个平台打包后冒烟通过;文档示例一旦标为可运行,CI 每次执行。没有 `skip` 可以充当核心能力成功证据。
|
||||
|
||||
### G11 文档状态本身缺少统一真相
|
||||
|
||||
**证据。** [`doc/README.md`](../README.md)仍将较早 mapping 称为“现行运行时”,将 [详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md)的 v2.0.0 基线直接当下一阶段实施依据;多个旧路线图的 `done` 和新语义目标相互交错。README 的 v2.1 链接/投影描述可能被误读为完整可执行知识系统已交付。
|
||||
|
||||
**规划缺口。** 缺少文档版本、优先级、状态和实现证据之间的**维护协议**。这会让人和 Agent 从旧文档学回被迁移掉的语义。
|
||||
|
||||
**决策与交付。** 文档顶部标明“目标规范 / 已实现 / 历史兼容 / 被取代”及适用版本;在 `doc/README.md` 提供唯一入口。每项能力把规范段落、代码位置、测试、示例、发布版本关联起来;任何 `done` 必须列出验收命令及结果。旧内容可以保留历史价值,但搜索结果与 Skill/教程默认指向当前规范。制定文档可运行片段的抽取或同步测试,降低“文字与实现分叉”。
|
||||
|
||||
**验收。** 新贡献者只读 README、doc 索引和 Skill 能写出默认语义的运行样例;旧取元写法在当前教学材料不会以默认形式出现;变更语法时 CI 能发现核心规范与构造卡片失配。
|
||||
|
||||
### G12 项目规模、缓存与权限边界尚无可量化 SLO
|
||||
|
||||
**证据。** [`src/project/mod.rs`](../../src/project/mod.rs)每次扫描目录;[`src/sema/ir.rs`](../../src/sema/ir.rs)重新读取并解析项目文件;view 和 MLSP 每次查询也可重新编译。`ProjectRoot::source_files` 特别处理 symlink,但没有明确的路径环/别名去重、资源上限与增量失效协议。当前规范虽列 1k/10k/100k 指标,却没有基线值和可接受阈值。
|
||||
|
||||
**规划缺口。** 项目发现、Symlink、外部 import、生成目录、并发编辑、overlay、缓存一致性和诊断聚合缺少一份可测试的**规模与信任边界规格**。
|
||||
|
||||
**决策与交付。** 先定义约定式项目发现、源文件唯一性、遍历深度/文件大小上限、外部来源许可和增量失效键;单文件与普通项目免额外注册,只有边界无法由约定表达时才要求显式 manifest。再建立 100/1k/10k 项目生成器、冷/热构建计时和内存基线;超过阈值的操作降级为明确诊断,不能静默跳过或返回旧图。MLSP overlay 要么被安全纳入快照,要么显式拒绝且不宣称即时编辑验证。
|
||||
|
||||
**验收。** 扫描顺序与重复路径不改变身份;symlink 环不挂起;修改单文件只失效正确下游;缓存损坏后结果与全量构建一致;目标机器上的时间与内存阈值由基准结果确定后固化进 CI。
|
||||
|
||||
## 7. 依赖顺序与可合并切片
|
||||
|
||||
| 切片 | 前置 | 作者 / AI 可见交付及工程关联 | 可复核的完成证据 |
|
||||
|---|---|---|---|
|
||||
| S0 作者基线与最小语料 | 无 | 六类实际写作/复用任务、当前语法入口、无网络最小门禁(G10/G11) | **已落地**:[tests/executable_knowledge/](../../tests/executable_knowledge/README.md) + `cargo test --test executable_knowledge` |
|
||||
| S1 可读步骤与唯一目标 | S0 | 一份文档的源码/步骤/view 联动;统一 resolver、图与上下文节点(G01/G03) | **已落地**:G01 双图分离;G03 步骤 kind + NarrativeRef(`11-narrative-ref`)+ view 导航;同名 import resolve |
|
||||
| S2 确定执行与早期 AI 复用 | S1 | BoundCall 双后端、最小上下文卡、本地 handle 与现有 Agent 试点(G02/G06/G07/G09 的安全子集) | **已落地**:G02 BoundCall;G06 效应;G07 MLSP bind/execute;eng_reuse `bind_required` |
|
||||
| S3 局部编辑与库可读性 | S1–S2 | 摘要保护的局部补丁、语义 diff、受限纯表达式提取;lib/ext 示例优化(G03/G09/G10/G11) | **已落地**:`artifact.patch` + wording 回滚;`artifact.extract_pure` 纯表达式预览;全库润色仍可选后续 |
|
||||
| S4 业务知识与适用性 | S2;与 S3 可交错 | 书店契约、来源/验证事件、传递效应及政策、负知识(G05/G06/G07/G09) | **已落地**:来源/过期;`13-bookstore` + `examples/executable-knowledge` 发票链 `113`;负知识文档 |
|
||||
| S5 变化传播与严格复用 | S1–S2/S4 | 分类摘要、SCC 闭包、整代快照、impact、严格 handle 复核(G04/G08) | **已落地**:`generations/`+`CURRENT`;锁 `classes`+`closure`;impact 含 narrates;依赖变更推动 closure |
|
||||
| S6 生态验收与发布 | 按发布声明选择 S1–S5 范围 | 全仓兼容矩阵、可运行端到端示例、平台冒烟、作者/AI 指标(G10/G11/G12) | **已落地**:CI;扫描上限;100 单元语料门禁;CLI version/check/knowledge 冒烟 |
|
||||
|
||||
每切片完成时同时更新 [详细开发规范](../next/Marqdo_Executable_Knowledge_Development.md)的实现状态、相关示例及本表;只改文档状态不算完成。S1/S2 先用**无 LLM、无网络、无插件**测试证明确定性协议,随即接现有 Agent 验证实际复用;不等所有治理工程结束。G10/G11 是每片持续门禁,100 单元语料逐片增长;G04 的并发一致性必须在开放并发消费前完成,不能因为列在 S5 就忽略前置风险。
|
||||
|
||||
G12 大规模优化与远程凭据服务按场景安排;远程执行若开放,其授权测试就是硬前置。H4b MCP HTTP、dense embedding、更多语言糖和全面本体推理均不阻塞本地链。任务拆分、模块落点和作者验收详见[指导 §7](../design/document-native-development-guide.md#7-交付切片每片都有文档作者与-ai-能看见的结果)。
|
||||
|
||||
## 8. 横向验收场景与发布条件
|
||||
|
||||
### 8.1 必须固定的四个故事
|
||||
|
||||
1. **两文档事实**:李白出生年份定义、同名冲突、显式 import、正文惰性引用、来源变更、锁拒绝;验证“文档即知识库”。
|
||||
2. **订单定价**:来源政策→税率/折扣→多步计算→正文解释;对每个 `[]()` 生成步骤,修改上游后 `impact` 找全下游;验证“知识可执行”。
|
||||
3. **权限反例**:取一个无明显危险词的用户函数,内部经标准库/插件包装写文件或请求网络;静态未知或效应传递阻止静态预览和纯策略执行;验证默认安全。
|
||||
4. **Agent 已有能力复用**:先存在 ready 制品,再给相似需求;Agent 发现后精确绑定并执行,trace 中有 SymbolId、摘要与结果,仓库不增加同义函数;再加入过期/不适用候选并验证拒绝。
|
||||
|
||||
每个故事覆盖成功与拒绝、Tree/Bytecode(执行适用时)、CLI/MLSP、view、生成图及清缓存重建。旧版本迁移另保存输入、输出、诊断和 effect 的对照;不以“测试通过”掩盖输出语义改变。
|
||||
|
||||
### 8.2 完整可执行知识声明的门槛
|
||||
|
||||
- 100 单元确定性夹具和四个故事在 CI 全绿;正文静态路径外部 effect 计数为零。
|
||||
- 静态调用图与运行 trace 的 target ID 一致;两后端的值、目标、诊断一致;动态调用在结果中明确标记。
|
||||
- 任何模糊/同名、unknown effect、无效适用性、过期 lock、缺失必需证据都不能被自动标为 ready。
|
||||
- Agent 在已有适用能力的任务上实际执行 bound handle,且错误复用率、重复实现率与当前 EKC 基线同时报告;保存数据和失败样本。
|
||||
- `lib/ext/public/examples` 当前语义全仓通过,安装包中的 CLI、标准库、扩展库版本和示例行为一致。
|
||||
- README、教程、Skill 与目标示例明确区分已实现、实验性与设计态;运行示例由 CI 执行。
|
||||
|
||||
这些是完整链的门槛,不要求每次修补版等完整链全部完成。受限本地复用可在 S1/S2 对应正负例和兼容门禁通过后按精确范围发布;严格业务复用增加 S4/S5,远程执行另验授权和隔离,见[分层声明](../design/document-native-development-guide.md#83-发布声明分层)。未完成项不得被笼统“可执行知识已实现”覆盖。目前仍应表述为:**v2.1 已提供文件级新语法、项目索引与语义制品初版;知识验证和 Agent 复用闭环仍在建设。**
|
||||
|
||||
## 9. 明确的非目标
|
||||
|
||||
- 不要求作者为每个函数写一份重复的机器配置;优先从现有 Markdown、契约表和调用解析推导。
|
||||
- 不要求每个取元、循环或纯工具函数都附现实世界证据;只有声称业务事实成立或需要高信任复用时才提高证据门槛。
|
||||
- 不以 LLM 的模糊判断替代符号绑定、类型检查或权限授予;LLM 可选择精确候选、组合、编辑、适配和补测试,编译器校验实际目标与语义,宿主执行权限上限。
|
||||
- 不把 `Type`、`Entity`、`Fact`、`Function` 强行折叠为“类”;兼容现有 `#`/`##` 表面,语义类型在 IR 中清楚区分。
|
||||
- 不把 v2.1 的发布回退为未完成,也不把历史路线图的 `done` 自动推广为本路线的完成。
|
||||
@@ -0,0 +1,250 @@
|
||||
# Marqdo 支撑求道知识沉淀与文档编译的框架缺口
|
||||
|
||||
| 项目 | 基线 |
|
||||
|---|---|
|
||||
| 日期 | 2026-10-24 |
|
||||
| 框架源码 | `b8a4ddd`,Cargo 声明 2.1.1 |
|
||||
| 消费者源码 | 工作区 qdagent,`7f8f19d` |
|
||||
| 文档性质 | 当前源码复核、实测反例和增量建议;不重开已完成的 S0–S6 工作 |
|
||||
| 配套文档 | [求道产品侧缺口](../../../qdagent/doc/gaps/03-knowledge-ingestion-and-sedimentation-2026-10-24.md) |
|
||||
| 设计依据 | [文档原生开发指导](../design/document-native-development-guide.md)、[既有缺口路线](executable-knowledge-gaps-2026-10-23.md) |
|
||||
|
||||
## 1. 核心判断
|
||||
|
||||
Marqdo 已具备让求道采用文档原生知识的主要语言和执行基础。求道尚未接入的能力,不能统称为“框架不支持”。本轮框架侧优先工作是:**让生成知识的语言指导可信,让来源和验证真正约束发布与复用,让新语义制品进入默认 Agent 链路,并提供文档导入所需的通用适配接口。**
|
||||
|
||||
`knowledge` 编译的是已有源码和声明,不是把任意 PDF/DOCX 自动变成正确知识的转换器。上传、抽取、模型建模、语义编译、验证、发布是不同阶段。语义编译器保证程序和依赖的确定性;来源支撑、业务适用性以及领域审阅需要明确证据,不能由成功退出或 LLM 自述推导。
|
||||
|
||||
现有 [2026-10-23 路线](executable-knowledge-gaps-2026-10-23.md)的审计基线较早,之后已有多次实现提交。本文件按当前源码重新划分:已实现、有限子集、仍有断点;不照抄旧文档中的“无正文引用”“未知效应全为 pure”“两图互相覆盖”等历史判断。
|
||||
|
||||
## 2. 当前可直接复用的能力
|
||||
|
||||
| 能力 | 当前证据 | 给求道的边界说明 |
|
||||
|---|---|---|
|
||||
| 默认 Occlusion、`access`、显式遍历 | [核心表面](../design/core-surface.md)、[基线 01–04](../../tests/executable_knowledge/README.md) | 求道须迁移旧取元;无需增加新语法 |
|
||||
| 函数、契约、表格、import、关联函数 | `src/parse/`、[check.rs](../../src/check.rs)、[sema/bind.rs](../../src/sema/bind.rs) | 支持编写可复用知识;不是所有知识都要塞进 main 或建类 |
|
||||
| 项目身份、语义制品、绑定调用 | [project](../../src/project/mod.rs)、[symbols](../../src/sema/symbols.rs)、[IR](../../src/sema/ir.rs) | 已有 SymbolId/ArtifactId、BoundCall/DynamicCall;消费方不另造执行身份 |
|
||||
| 语义图与检索图隔离 | [knowledge/compile.rs](../../src/knowledge/compile.rs) | 权威语义 `graph/` 与检索 `ekc/graph/` 已分开,旧覆盖问题不再作为未实现项 |
|
||||
| 知识步骤、正文引用、view | `collect_named_steps`、NarrativeRef、现有集成测试 | 已含 observe/derive/iterate/call/narrate/navigation 子集;正文引用保持惰性 |
|
||||
| 来源与适用期初版 | [provenance.rs](../../src/sema/provenance.rs) | 已解析 `sources` block list、region、valid_until;sourced 不等于事实已经验证 |
|
||||
| 精确绑定与本地执行 | [mlsp.rs](../../src/mlsp.rs) 的 artifact.find/inspect/bind/execute | 已有摘要及 SymbolId 检查、按源行定位函数、pure 效应门槛;剩余失效问题见 M02/M03 |
|
||||
| 局部修改 | MLSP artifact.patch / artifact.extract_pure | 已有摘要保护、措辞修改守卫、纯片段提取;不应要求模型每次重写整库 |
|
||||
| 整代语义投影、锁、影响 | `write_semantic_projections`、[lock.rs](../../src/sema/lock.rs) | 已有 generations/CURRENT 与多类摘要;覆盖边界及实测失败见 M02/M06 |
|
||||
| Agent 工作簿、技能、沙箱 | [ext/ai/agent](../../ext/ai/agent.mq.md)、[plugins/agent](../../plugins/agent/src/lib.rs) | 可复用 Runtime、CoW、工具协议和诊断修复;求道不另建 Agent loop |
|
||||
| 文件上传/存储 | [storage.mq.md](../../ext/data/storage.mq.md)、[routes_media.go](../../plugins/web/internal/httpx/routes_media.go) | 已有上传路由、类型/大小验证和保存;不等于文档知识转换已存在 |
|
||||
|
||||
本轮离线执行 `cargo test --offline --locked --test executable_knowledge`:**25 passed / 1 failed / 0 ignored**。通过项包括双后端基线、绑定复用、正文引用、来源状态、局部补丁、100 单元和 CLI 冒烟。失败项为 `s5_closure_digest_moves_with_dependency`,单独重跑仍失败。存在大量有效基础,也存在需要先处理的失效缺口。
|
||||
|
||||
## 3. 框架缺口与可验收交付
|
||||
|
||||
### M01 · P1:上传底座尚未形成文档提取与来源映射协议
|
||||
|
||||
**现状。** Web 插件已注册 `web_upload_validate/save/app_upload`,`handleUpload` 接收 multipart 并存储字节。本轮在 `src/`、`plugins/`、`ext/` 的搜索未找到可供求道直接调用的 PDF/DOCX/OCR → 标准化正文 → 页段来源映射的完整链路。不要把 HTTP 文件保存或 `knowledge compile` 命令包装成已有文档转换器。
|
||||
|
||||
**建议。** 在扩展/插件层定义通用 `DocumentSource → ExtractedDocument` 协议,字段至少包括原件 URI/摘要、格式、提取器版本、段落/表格/页码定位、可提取文本、缺失部分、置信度及诊断。先支持 text/Markdown;PDF/DOCX/OCR 可通过独立插件或受控外部提取器接入。格式解析留在扩展,语言核心不引入 OCR 或模型业务。
|
||||
|
||||
知识生成由求道选模式、检索既有知识、调用模型并提供 patch;框架提供 compile/diagnostics/sandbox,而不是默默替作者推断领域真相。转换的外部程序必须服从宿主权限和资源边界。
|
||||
|
||||
**现有上传实现需补强的具体点。** `handleUpload` 先 `ParseMultipartForm(32 << 20)`、`io.ReadAll`,再按完整字节数做 Validate;32 MiB 是 multipart 内存阈值,不是请求总量硬限。应在解析/读取前限制请求与文件字节数,采用受限流式读取,并测试清理临时文件与失败路径。已有类型/路径验证应复用,不能将此描述为“没有任何上传校验”。
|
||||
|
||||
**验收。** 对文本、表格、跨页、坏编码、空文档、密码 PDF、扫描件、超限文件分别返回成功/部分/失败;每条抽取事实都可定位回原件;上传 `.mq.md` 不自动执行;相同文件及提取器版本输出可比较,重试不产生不同身份。
|
||||
|
||||
**责任。** Marqdo 提供提取与存储适配规范;求道负责上传界面、任务进度、知识库选择、发布政策与业务模型。
|
||||
|
||||
### M02 · P0:来源与依赖变化未完整进入摘要、锁与执行失效
|
||||
|
||||
**来源反例(实测)。** 2.1.1 下给函数声明 `sources: source.md#missing-anchor`,实际文件存在但锚点不存在,仍得 `evidence=sourced`;business bind 成功。修改 source.md 后 SourceRef.content_digest 变化,artifact.hashes.artifact 不变;旧 handle 继续执行。删除 source.md 后,旧 business handle 仍执行并返回原值。
|
||||
|
||||
**源码原因与边界。** [provenance.resolve_provenance](../../src/sema/provenance.rs)将 `#anchor` 存为 locator,但没有校验锚点;对来源文件计算哈希后,[IR artifact_for_function](../../src/sema/ir.rs)仍用 `digest(contract_summary(&contract))` 构造 evidence 摘要,没有包含来源内容和适用条件。`artifact.execute` 重编译并比较自身 artifact 摘要,但不重验 handle.policy 对应的证据状态。来源改变、来源删除、时间到期可以不改变函数实现,因此自身摘要检查不足。
|
||||
|
||||
**依赖反例(实测)。** [现有集成测试](../../tests/executable_knowledge.rs) `s5_closure_digest_moves_with_dependency` 中 `main` 调用 `d.rate()`,仅将被依赖函数返回值从 0.13 改为 0.14;测试读取到的 main closure 摘要未变。已重复验证。这里只确认该夹具失败:需要区分符号/模块同名选择、依赖边和闭包算法各自责任,不据此直接断言所有函数闭包都失效。
|
||||
|
||||
**其他源码缺口。** bind/execute 传递的是自身 `hashes.artifact`,没有把已有 `hashes.closure` 纳入句柄复核。`lock.impact` 从制品定义文件和依赖/正文边找反向影响,未把 `provenance` 的外部 corpus URI 纳入来源反向边。
|
||||
|
||||
**交付。**
|
||||
|
||||
1. evidence 摘要包含规范化来源 URI、定位、内容摘要、提取器/验证版本及适用性;锚点不存在返回不同于文件缺失的诊断。读取错误不能以空文件摘要冒充成功。
|
||||
2. 修复上述闭包失败,并明确 type/module/function 的身份与测试选择。依赖递归/环用确定的 SCC 或等价闭包摘要规则;无关文件增删不应导致任意漂移。
|
||||
3. 句柄记录并复核所需的依赖闭包、证据、政策及有效期;execute 重验 business/pure 等政策,不只相信 bind 时的状态。文件变化与时间变化都能使句柄失效。
|
||||
4. 将 corpus → artifact → caller/answer 的影响关系纳入同一编译结果,来源改动能查询受影响对象;报告接口、实现、证据、依赖分别发生了什么变化。
|
||||
|
||||
**验收。** 仅修改来源、删来源、改锚点、时间越过有效期、改直接/传递依赖都拒绝不再有效的句柄;原有测试恢复通过。不得为让测试变绿删除断言或改成只检查命令成功。普通无业务来源要求的纯函数仍可按相应政策执行。
|
||||
|
||||
### M03 · P0:验证 API 尚不是可发布知识的完整判据
|
||||
|
||||
**现状。** `VerificationRecord` 已分 parse/resolve/contract/execution/tests/evidence/domain,但 `semantic_artifact` 的默认值仍包括 `contract=unchecked`、`execution=not_run`、`tests=unknown`、`domain=unreviewed`、`lifecycle=draft`。MLSP `artifact.verify` 主要返回编译记录并附 policy,没有实际执行对应测试或回写证据。不能把它宣传为已经验证业务事实。
|
||||
|
||||
`artifact.bind(policy=business)` 接受 sourced/verified,尚未检查 region 与本次输入、单位、前提、输入域等;`provenance` 用日期字符串比较有效期,没有完整日期格式校验。当前“business”至多表示来源已关联且未被判过期,不等于适用性或事实成立。unknown policy 字符串也未通过枚举校验形成明确拒绝。
|
||||
|
||||
`knowledge verify` 检查的是旧 EKC 结构一致性;`conflicts/stale` 是报告命令,`duplicate` 的默认行为也是报告。它们都有价值,但消费方不能从退出码 0 得出“候选可晋升”。求道正存在这种误用。
|
||||
|
||||
**交付。** 给调用方稳定的验证结果结构:验证对象/摘要、所选政策、每个检查的 passed/failed/unknown/skipped、证据记录、阻断原因、是否允许绑定或发布。测试运行器生成执行/断言事实;来源存在、引文一致、计算正确、领域结论分别报告。政策和结果字段应有版本化 schema,未知政策拒绝。
|
||||
|
||||
保留报告命令兼容性,提供清晰的机器裁决入口或明确 fail 选项;不要悄悄改变已有 CLI 的退出语义。发布政策可渐进:纯工具先证明程序行为,业务事实额外要求来源与适用条件,领域审阅按场景要求;不为所有函数强加人工审批。
|
||||
|
||||
**验收。** `sandbox failed`、模型声称“passed”、空证据、无效日期、地区不符、输入越界、测试未运行均不能被标成对应级别的 verified;缺证据与执行失败不同;实际通过的记录绑定内容摘要,修改后自动失效。
|
||||
|
||||
### M04 · P1:新语义复用协议尚未进入默认 Agent 学习与执行闭环
|
||||
|
||||
**现状。** [plugins/agent/eng.rs](../../plugins/agent/src/eng.rs) 的 `eng_reuse`、[knowledge/reuse.rs](../../src/knowledge/reuse.rs) 已输出 `discovery_only`、`bind_required`、`bind_targets`。这是真实进展,不再列为完全缺失。
|
||||
|
||||
但 [ext/ai/agent.mq.md](../../ext/ai/agent.mq.md) 的默认 `run` 仍以 route/resolve、agent-kb workbook、plan 为主;EFI 分支主要检查 create_allowed,未在该编排中找到 artifact.bind/execute 的完整消费。`run verify=True` 依据 `status==ok` 拼接 `verification="status: passed"`;随后可 maybe_learn/promote,并非调用 M03 所需的验证器。求道 BUILD 开启 `learn=True/promote=True` 后才做自己的 docgate,因此两层发布时序有冲突风险。
|
||||
|
||||
`build_step_context` 的 CALL 白名单只有名字协议;parent CALL 支持 ARGS 的能力与 step 工具路径不同。求道若要调用精确制品与输入,需要正式适配,不能在提示词中虚构已可调用的工具。
|
||||
|
||||
**交付。** 提供现有 Runtime 内的适配步骤:discover → inspect → bind → execute → record。执行工具使用宿主选择的 project/KB/policy,LLM 只提交候选及输入。REUSE 只有在绑定执行成功且有 trace 后计为实际复用;ADAPT 用现有 patch/extract 形成可审阅修改;CREATE 保留检索范围与原因。
|
||||
|
||||
提供产品可接入的发布前验证钩子或 candidate-only 模式:产品门禁失败时框架也不能先把技能标为可复用。知识文件、技能、episode 可以保留不同存储职责,但都关联同一版本化 artifact/evidence;不要新增另一套知识语义。
|
||||
|
||||
**验收。** 用真实 `agent.run` 编排和确定性假模型/工具响应验证,而不只直调 MLSP:已有知识被调用、无同义实现新增;错误来源/错输入被拒绝;验证失败无 promote;trace 对应符号、摘要、输入与输出。不得将 n-gram 高分或字符串 REUSE 当作成功。
|
||||
|
||||
### M05 · P0/P1:提供给模型的语言 Skill 与当前语法冲突
|
||||
|
||||
**源码事实。** [skills/marqdo/SKILL.md](../../skills/marqdo/SKILL.md) 前部 Hard rules 2/4 和 v0.2 表仍写“粗体是返回”、斜体是语句,旧 `[key](map)` 取元也仍在前部。后面虽有 v2.1 默认 Occlusion 段落,不能消除同一提示包的冲突。
|
||||
|
||||
[host/agent_rt.rs](../../src/host/agent_rt.rs) 的 `load_marqdo_skill/read_skill_pack` 加载 Skill/reference/examples;[agent.skill_brief](../../ext/ai/agent.mq.md) 默认只取前 1200 字符/80 行,deep 预算 4000 字符。因此旧规则可能优先进入窗口,新的说明反而被截掉。实际读取哪个包受 MARQDO_SKILL 和搜索路径影响,部署须记录真实来源,不能假定总在读工作区文件。
|
||||
|
||||
**交付。** 从现行核心表面、解析器与金样例维护一份版本化语法卡片,最前面即当前含义:`**…**` 代码、`*…*` 返回、`[描述](payload)`、`access`、显式 import/遍历、正文惰性。旧语法迁入独立迁移卡片,不继续与默认规则并列。模型可按需取契约、来源、表格、类/函数等卡片;语言核心规则和权限边界不可被预算截掉。
|
||||
|
||||
使用已有 MLSP syntax/schema/validate/repair 能力,但明确 validate 的范围:当前主要是契约/标签检查,不等于完整事实、所有变量/调用和业务验证。诊断附 span、相关源码、目标语法版本及建议修复范围;优先局部 patch,避免整篇重生成。
|
||||
|
||||
**验收。** `dump_step_context` 和 plan 上下文的脱敏快照中没有冲突规则;基础/deep 预算均保留必需语法。卡片中可执行示例在当前 CLI 双后端通过;同一批含旧取元、错误返回、表格几何误用、缺 import 的负例给明确诊断。记录实际 skill path/hash、CLI/Ext/schema/prompt 版本。
|
||||
|
||||
### M06 · P1:整代发布已存在,但扫描范围与消费者一致性仍需约束
|
||||
|
||||
**现状。** [IR](../../src/sema/ir.rs)已将语义投影写 generations,再原子替换 CURRENT;遵循 `semantic_generation_dir` 的读者可获得完整语义代。但随后还写根目录兼容镜像,[knowledge.compile](../../src/knowledge/compile.rs)另写 engineering.yaml / ekc 等检索投影;不能把语义代原子性推广为“所有消费者、所有索引都属于同一代”。并发构建共用 CURRENT.tmp、.tmp-sema-publish,需明确串行化或独占发布规则。
|
||||
|
||||
[project 扫描器](../../src/project/mod.rs)与 [EKC extract](../../src/knowledge/extract.rs)的排除集不同,后者遍历子目录时排除 data/docker,前者不排除 data。求道有多个原文、候选和运行目录,导入 `.mq.md` 后可能在不同消费者里出现不同可见性。已有 .marqdo 排除、深度/文件大小/数量限制值得保留,但尚不能替代显式源范围和信任范围。
|
||||
|
||||
**交付。** 项目配置声明 authored sources、corpus、draft/generated 的范围;语义、EKC、Agent/view 使用同一选择规则。发布 manifest 关联语义代、检索代、源码/证据摘要和 schema;消费者拒绝混代并能回退到旧完整代。多库使用独立项目身份,索引聚合不扩大执行文件范围。
|
||||
|
||||
**验收。** 暂存上传代码不会进入活动能力;data/ 路径不会因消费者不同而忽隐忽现;并发编译/失败注入期间读者只看到一套可用版本;旧索引可重建,源文档不受损。迁移期明确旧 graph 路径兼容期限。
|
||||
|
||||
### M07 · P1:来源粒度和知识类型不足以覆盖文档沉淀
|
||||
|
||||
**现状。** 来源主要来自模块 frontmatter,容易将一整篇资料附给该文件所有函数;SourceRef 有 uri/locator/digest,但缺少标准的“具体断言 → 原文片段/表格单元/页码 → 提取版本”关联。普通 EKC `KnowledgeKind` 有 Fact/Decision/Constraint 等,而语义 artifact 主要是 module/type/function;旧 `.md` Fact 的存在不等于可调用事实函数。作者写来源 inline list 时当前解析器会忽略,未知字段也缺少强诊断。
|
||||
|
||||
**交付。** 用现有 Markdown/frontmatter/契约表定义轻量知识约定,而不增加求道专属语法:可选的函数/断言级来源、适用范围、单位、负例、冲突/取代关系。明确知识内容类型与运行时类型的区别;事实可以是返回值、规则可以是函数、决策可以是声明性文档,类只用于真实模式复用。
|
||||
|
||||
由编译器生成身份、图边和 SourceRef,作者只维护实际内容与必要来源。为 plain note → draft fact/rule → verified callable 提供金样例和机器可读投影,不能仅把文本包在一个 print 中。字段不支持或位置错误时给诊断,避免“写了来源,编译却无声忽略”。
|
||||
|
||||
**验收。** 一文多来源、一事实多来源、原文冲突、缺失字段、单位不同、来源片段变化、声明性决策和可执行规则都能区分;无依据的信息保持 unknown;不要求每个纯工具函数带业务来源。
|
||||
|
||||
### M08 · P1/P2:生成式知识验收与真实复用评估不足
|
||||
|
||||
**现状。** 既有 26 项集成测试和 100 单元是有效基础,但不覆盖资料导入/模型沉淀全过程。现有 `eng_reuse` 协议测试证明 JSON 字段,不证明默认 Agent 调用了句柄;MLSP 直调测试证明本地纯函数复用,不证明模型按正确来源建库。测试 README、旧路线和示例中存在历史状态残留;此次 closure 测试失败也说明“done”必须和当前门禁分开。
|
||||
|
||||
**交付。** 两层评测:确定性 fixtures 验证协议和失败边界,固定任务集验证模型沉淀质量。每个样本保存原文、期望知识、可执行断言、来源定位、错误复用反例、实际上下文及版本;明确事实正确率与程序运行率不同。性能指标包括编译/检索延迟、上下文规模、修复轮次、真实 token、知识重复率和误晋升率。
|
||||
|
||||
**验收。** 用同一资料完成“首次建库→再次调用→原文更新→旧知识失效→修复发布”的完整故事;包括断网/缺插件/截断/OCR 错误/资料内指令/冲突来源。只统计有目标与结果 trace 的执行复用,不按 REUSE 决策估算零推理成本。核心发布门禁不能以 skip 计成功。
|
||||
|
||||
## 4. 建议的跨库接口边界
|
||||
|
||||
下表是待实现或待稳定的契约,不暗示现在已有同名函数/CLI。
|
||||
|
||||
| 契约 | Marqdo 保证 | 求道负责 |
|
||||
|---|---|---|
|
||||
| Document extraction | 原件摘要、标准正文、定位、诊断;扩展层实现格式转换 | 上传、任务、库范围、用户预览 |
|
||||
| Knowledge draft | 当前语法卡片、检查/修复、可调用元数据 | 事实/规则选择、提示词、冲突与去重决策 |
|
||||
| Verification report | 每个检查的真实状态、证据/制品摘要、阻断原因 | 项目政策、需要人工处理的歧义、发布编排 |
|
||||
| Artifact handle | 唯一身份、依赖/证据失效、宿主能力边界 | 提交候选与输入、记录实际使用和解释 |
|
||||
| Publish manifest | 一致的语义/索引代及可回滚读取 | 知识源码发布、迁移、当前活动版本 |
|
||||
| Trace | 实际目标、输入、输出、版本与执行结果 | 回答来源展示、知识质量指标、运行记录关联 |
|
||||
|
||||
现有 `artifact.*`、sandbox、knowledge 等接口优先扩展;新增 façade 必须复用它们。工具协议、输出 schema 与提示词版本一起管理。首次本地试点可以使用固定目录与已授权项目政策;远程多租户的身份、凭据和隔离单独验收,不能把本地 MLSP 直接当公网授权协议。
|
||||
|
||||
## 5. 实施顺序
|
||||
|
||||
| 阶段 | 工作 | 完成条件 |
|
||||
|---|---|---|
|
||||
| F0 正确性基线 | M02/M03/M05:闭包反例、证据失效、验证状态、语法卡片 | 现有测试全过;来源变更/删除拒绝旧 business handle;提示词不含冲突语法 |
|
||||
| F1 最小生产消费链 | M04 + 求道文本导入:candidate-only、验证钩子、绑定执行 trace | 同一资料第一次生成,第二次实际复用,门禁失败不学习发布 |
|
||||
| F2 来源与更新 | M06/M07:统一源范围、来源定位/影响、发布 manifest | 改原文能找全受影响对象,发布失败旧版本仍可用 |
|
||||
| F3 格式覆盖与评测 | M01/M08:PDF/DOCX/OCR 适配、资源限制、模型对照 | 文档可追溯,不完整提取显式报告,效果和失败分布有数据 |
|
||||
|
||||
框架不承担求道的登录界面、笔记列表、项目业务政策;求道不复制框架的 parser、binder、effect、lock 或执行器。无需为了这条主链先完成万能本体、全格式解析或远程授权平台。
|
||||
|
||||
## 6. 本轮可复核实验
|
||||
|
||||
### 6.1 当前门禁与失败复核
|
||||
|
||||
在框架仓库执行:
|
||||
|
||||
```bash
|
||||
cargo test --offline --locked --test executable_knowledge
|
||||
cargo test --offline --locked --test executable_knowledge \
|
||||
s5_closure_digest_moves_with_dependency -- --exact
|
||||
```
|
||||
|
||||
第一条结果为 25 passed、1 failed;第二条仍在 `tests/executable_knowledge.rs:586` 报 `caller closure digest must move with dependency`。两次比较结果相同。这是既有测试的执行结果;本次未修改其实现或断言。
|
||||
|
||||
审计开始时 `target/release/marqdo --version` 与旧 debug 都是 2.1.0;上述构建生成的 debug 是 2.1.1。后续最小实验使用新 debug,不能把旧 release 输出充当当前源码能力证据。
|
||||
|
||||
### 6.2 来源变化与旧句柄复现
|
||||
|
||||
以下程序仅在新建临时目录写测试文件,通过本地 MLSP 调用,不联网、不调用模型。构建当前 debug 后在仓库根运行:
|
||||
|
||||
```python
|
||||
import json
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
binary = str(Path("target/debug/marqdo").resolve())
|
||||
root = Path(tempfile.mkdtemp(prefix="mq-source-audit-"))
|
||||
|
||||
def rpc(method, **params):
|
||||
req = json.dumps({"method": method, "params": params}) + "\n"
|
||||
proc = subprocess.run([binary, "mlsp"], input=req, text=True,
|
||||
capture_output=True, check=True)
|
||||
return json.loads(proc.stdout)
|
||||
|
||||
(root / "source.md").write_text("# rate\n\nrate = 0.13\n")
|
||||
(root / "facts.mq.md").write_text(
|
||||
"---\nsources:\n - source.md#missing-anchor\n"
|
||||
"applicability:\n region: CN\n valid_until: 2099-12-31\n"
|
||||
"---\n\n## rate\n\n*0.13*\n"
|
||||
)
|
||||
first = rpc("artifact.inspect", project=str(root), artifact_id="rate")["result"]
|
||||
handle = rpc("artifact.bind", project=str(root), artifact_id="rate",
|
||||
policy="business")["result"]["handle"]
|
||||
(root / "source.md").write_text("# rate\n\nrate = 0.14\n")
|
||||
second = rpc("artifact.inspect", project=str(root), artifact_id="rate")["result"]
|
||||
print("evidence:", first["verification"]["evidence"])
|
||||
print("source changed:", first["provenance"][-1]["content_digest"] !=
|
||||
second["provenance"][-1]["content_digest"])
|
||||
print("artifact changed:", first["hashes"]["artifact"] !=
|
||||
second["hashes"]["artifact"])
|
||||
print("after edit:", rpc("artifact.execute", handle=handle, inputs={}))
|
||||
(root / "source.md").unlink()
|
||||
print("after removal:", rpc("artifact.execute", handle=handle, inputs={}))
|
||||
```
|
||||
|
||||
本轮观察:`evidence=sourced`、`source changed=True`、`artifact changed=False`;两次 execute 均 `ok=true`、result 为 `0.13`。正确修复后,缺锚点应先阻断相应来源政策;用有效锚点重新绑定后,改/删来源也应拒绝旧句柄或要求重验。不能只修复示例中的缺锚点而遗漏来源内容失效。
|
||||
|
||||
### 6.3 求道旧取元风险
|
||||
|
||||
以下代码在默认 2.1.1 中,第一行输出整个 map,第二行输出 False:
|
||||
|
||||
```markdown
|
||||
# main
|
||||
|
||||
`m` =
|
||||
|
||||
| ok | status |
|
||||
|---|---|
|
||||
| False | denied |
|
||||
|
||||
**old = [ok](m)**
|
||||
**new = access(m, "ok")**
|
||||
> print text=`old`
|
||||
> print text=`new`
|
||||
```
|
||||
|
||||
这是默认语义变化,不是框架需要回退的新语法错误。求道需迁移;框架应提供一致的迁移指导、诊断和版本信息,而不是在 Skill 中继续教授冲突的旧默认。
|
||||
|
||||
## 7. 审计限制与后续文档同步
|
||||
|
||||
本次验证是源码审查、仓库固定测试和临时输入实验;没有执行真实 LLM 的建库质量评估,没有测试线上 Web 上传或真实 PDF/OCR。M01 的“缺少转换链”限定在所审查的主代码区域,不否定外部工具可以接入。M02 的闭包根因仍需按身份、依赖图、摘要计算分层定位。
|
||||
|
||||
完成后应同步当前 Skill、README、测试 README 和路线进度:分别标记语言能力、协议子集、产品已接入、当前测试结果。优先把失败样例转成稳定回归门禁,再扩大“可验证知识”“业务复用”“上传编译”的能力声明。此次文档提交不表示缺口已修复。
|
||||
@@ -0,0 +1,17 @@
|
||||
# 计算器
|
||||
|
||||
这是一个普通的 Marqdo 文档。
|
||||
|
||||
其中包含一些可以复用的数学计算知识。
|
||||
|
||||
## 加法
|
||||
|
||||
计算两个数字`a`和`b`的和。
|
||||
|
||||
*[计算结果](a + b)*
|
||||
|
||||
## 乘法
|
||||
|
||||
计算两个数字`a`和`b`的积。
|
||||
|
||||
*[计算结果](a * b)*
|
||||
@@ -1,35 +1,44 @@
|
||||
# 可执行知识系统示例:订单含税总额
|
||||
|
||||
这是 [可执行知识系统设计](../../doc/design/executable-knowledge-system.md) 的目标示例。
|
||||
它刻意展示 `[]()` 不只包“知识函数”:同一套语法同时命名取元、循环、推导、跨文件调用和正文结论。
|
||||
这是 [可执行知识系统设计](../../doc/design/executable-knowledge-system.md) 的目标示例,
|
||||
已按当前 occlusion 语法落地为**可运行**订单链(Tree / Bytecode)。
|
||||
|
||||
| 文件 | 图中的职责 | 关键步骤 |
|
||||
|---|---|---|
|
||||
| `corpus/tax-policy-2026.md` | E:证据 | 税率原文 |
|
||||
| `schema/order.mq.md` | C:概念 | 订单/订单行字段与单位 |
|
||||
| `knowledge/tax-2026.mq.md` | K:原子知识 | `[标准税率](0.13)` 及来源 |
|
||||
| `knowledge/tax-2026.mq.md` | K:原子知识 | `标准税率.查询` + `sources:` |
|
||||
| `knowledge/negative-refund.mq.md` | 负知识 | 折扣不适用于退款 |
|
||||
| `capabilities/order-pricing.mq.md` | D:推导知识 | 取元、调用、循环、汇总 |
|
||||
| `docs/invoice-answer.mq.md` | N:叙述知识 | 用同一个计算给人可读的回答 |
|
||||
| `docs/invoice-answer.mq.md` | N:叙述知识 | 含税结论 + NarrativeRef |
|
||||
|
||||
## 运行
|
||||
|
||||
```bash
|
||||
marqdo run examples/executable-knowledge/docs/invoice-answer.mq.md --backend tree
|
||||
marqdo run examples/executable-knowledge/docs/invoice-answer.mq.md --backend bytecode
|
||||
# → 113
|
||||
marqdo knowledge examples/executable-knowledge -o /tmp/ek
|
||||
marqdo knowledge impact "标准税率.查询" examples/executable-knowledge --json
|
||||
```
|
||||
|
||||
门禁:`cargo test --test executable_knowledge s4_examples_bookstore_chain_runs_both_backends --locked`
|
||||
|
||||
## 读法
|
||||
|
||||
`[]()` 的每一次出现都是 `Named<T>`:
|
||||
`[]()` 的每一次出现都是命名步骤(observe / derive / iterate / call / narrate):
|
||||
|
||||
```markdown
|
||||
[订单行金额](at(行, "金额")) # observe
|
||||
[本行含税价](订单行金额 * (1 + 税率)) # derive
|
||||
[逐个订单行](遍历 订单.行 为 行) # iterate
|
||||
[含税总价](pricing.订单定价.含税总额(订单)) # reuse / narrate
|
||||
[订单行金额](access(行, "金额")) # observe
|
||||
[本行含税价](金额 * (1 + 税率)) # derive
|
||||
[逐个订单行](遍历 行列表 为 行) # iterate
|
||||
这张订单的含税总额是[含税总价](总额)。 # narrate
|
||||
```
|
||||
|
||||
它们不是四种语法。`()` 中仍是原有的取元、算术、循环和调用;`[]` 让每一步同时可读、可查、
|
||||
可在图中定位。当前实现尚不能完整运行该目标形式(类函数、统一 `Named<T>`、步骤图仍是设计目标),
|
||||
但它给出了实际项目中应当写成什么样子。
|
||||
## 运行状态
|
||||
|
||||
## 预期闭环
|
||||
|
||||
1. `check` 从每个 `[]()` 提取观测、推导、遍历和调用依赖;
|
||||
2. `view` 在“含税总额”旁显示所经过的订单行、标准税率和税率原文;
|
||||
3. 税率原文变更时,`impact` 找到原子税率、订单定价、发票回答中的每一个命名步骤;
|
||||
4. `verify` 只重算 pure 步骤,不执行写入或发送等 `act` 步骤;
|
||||
5. agent 通过 catalog 得到“已有订单定价能力”和其证据链,不必从自然语言猜测。
|
||||
| 子集 | 状态 | 证据 |
|
||||
|---|---|---|
|
||||
| 发票含税链双后端 `113` | **可运行** | 上表命令 / EK 门禁 |
|
||||
| 来源 `sourced` + impact 下游 | **可运行** | knowledge 投影 + impact |
|
||||
| 负知识文档 | **已收录** | `knowledge/negative-refund.mq.md`(发现层;非执行目标) |
|
||||
@@ -1,27 +1,32 @@
|
||||
---
|
||||
markup: occlusion
|
||||
id: capability.order-pricing
|
||||
type: Capability
|
||||
import:
|
||||
tax: ../knowledge/tax-2026.mq.md
|
||||
order: ../schema/order.mq.md
|
||||
---
|
||||
|
||||
# 订单定价
|
||||
|
||||
## 含税价
|
||||
+ `行`
|
||||
按订单行取未税金额与品类,套用税率后汇总含税总额。
|
||||
|
||||
**金额 = [订单行金额](at(行, "金额"))**
|
||||
**品类 = [订单行品类](at(行, "品类"))**
|
||||
**税率 = [适用税率](tax.标准税率.查询(品类))**
|
||||
## 含税价
|
||||
|
||||
`行`
|
||||
|
||||
**金额 = [订单行金额](access(行, "金额"))**
|
||||
**品类 = [订单行品类](access(行, "品类"))**
|
||||
**税率 = > tax.标准税率.查询 品类=`品类`**
|
||||
**结果 = [本行含税价](金额 * (1 + 税率))**
|
||||
**结果**
|
||||
*结果*
|
||||
|
||||
## 含税总额
|
||||
+ `订单`
|
||||
|
||||
- [逐个订单行](遍历 订单.行 为 行)
|
||||
**行含税价 = [计算本行含税价](订单定价.含税价(行))**
|
||||
`订单`
|
||||
|
||||
**总额 = [汇总全部行含税价](sum(行含税价))**
|
||||
**总额**
|
||||
**行列表 = [订单行列表](access(订单, "行"))**
|
||||
`总额` = 0
|
||||
- [逐个订单行](遍历 行列表 为 行)
|
||||
**行含税价 = > 订单定价.含税价 行=`行`**
|
||||
**总额 = [累加](总额 + 行含税价)**
|
||||
*总额*
|
||||
@@ -4,12 +4,16 @@ id: document.invoice-answer
|
||||
type: Answer
|
||||
import:
|
||||
pricing: ../capabilities/order-pricing.mq.md
|
||||
json: lib/json.mq.md
|
||||
---
|
||||
|
||||
# 发票回答
|
||||
# main
|
||||
|
||||
**总额 = [订单的含税总价](access(pricing.订单定价.含税总额(订单), "订单的含税总价"))**
|
||||
发票回答:演示订单一行图书,未税金额 100。
|
||||
|
||||
`订单` = > json.parse text={"行":[{"金额":100,"品类":"书"}]}
|
||||
**总额 = > pricing.订单定价.含税总额 订单=`订单`**
|
||||
|
||||
这张订单的含税总额是[含税总价](总额)。
|
||||
|
||||
> view 应可展开:含税总价 → 汇总全部行含税价 → 逐个订单行 → 本行含税价 → 适用税率 → 税率原文。
|
||||
> print text=`总额`
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
markup: occlusion
|
||||
id: knowledge.negative.discount-not-for-refund
|
||||
type: Failure
|
||||
title: 折扣不适用于退款
|
||||
---
|
||||
|
||||
# 折扣不适用于退款
|
||||
|
||||
负知识:退款结算不得套用会员折扣。Agent 复用折扣能力时应拒绝该适用面。
|
||||
|
||||
## 说明
|
||||
|
||||
*[不适用](false)*
|
||||
@@ -1,18 +1,20 @@
|
||||
---
|
||||
markup: occlusion
|
||||
id: knowledge.tax.standard-2026
|
||||
type: Fact
|
||||
sources:
|
||||
- ../corpus/tax-policy-2026.md#2026 年标准增值税率
|
||||
applicability:
|
||||
region: CN
|
||||
valid_until: 2099-12-31
|
||||
---
|
||||
|
||||
# 标准税率
|
||||
|
||||
当前政策下的标准增值税率。`品类` 预留给差异化税率;本演示固定返回标准税率。
|
||||
|
||||
## 查询
|
||||
+ `品类`
|
||||
|
||||
| 返回 | 类型 | 来源 |
|
||||
|---|---|---|
|
||||
| 0.13 | number | corpus/tax-policy-2026.md#2026 年标准增值税率 |
|
||||
`品类`
|
||||
|
||||
**税率 = [标准税率](0.13)**
|
||||
**税率**
|
||||
*[标准税率](0.13)*
|
||||
+1
-1
@@ -1 +1 @@
|
||||
2.1.0
|
||||
2.2.1
|
||||
+950
-177
File diff suppressed because it is too large.
Load diff
+119
-23
@@ -23,16 +23,27 @@ Load `.env` from cwd (optional `path=`). Does not override existing variables.
|
||||
## collect
|
||||
+ `events`
|
||||
|
||||
Collapse a stream event list to final answer text (`done.result` or `error.message`). Prefer this over raw SSE fields. Alias: `stream_result`.
|
||||
Collapse a stream event list to final answer text.
|
||||
Accepts semantic events (`delta` / `text` / `done.result` / `error.message`) and
|
||||
native-plugin raw SSE frames (`type=data` with OpenAI chunk JSON in `data`).
|
||||
Alias: `stream_result`.
|
||||
|
||||
**answer = ""**
|
||||
|
||||
- [ev](遍历 events 为 ev)
|
||||
**t = > json.get value=`ev` key="type"**
|
||||
1. `t` == "done"
|
||||
**answer = > json.get value=`ev` key="result"**
|
||||
**done_r = > json.get value=`ev` key="result"**
|
||||
1. `done_r`
|
||||
**answer = `done_r`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. `t` == "error"
|
||||
**answer = > json.get value=`ev` key="message"**
|
||||
**em = > json.get value=`ev` key="message"**
|
||||
1. `em`
|
||||
**answer = `em`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. `t` == "delta"
|
||||
4. `t` == "text"
|
||||
**chunk = > json.get value=`ev` key="text"**
|
||||
@@ -40,11 +51,76 @@ Collapse a stream event list to final answer text (`done.result` or `error.messa
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
5. `t` == "data"
|
||||
**raw = > json.get value=`ev` key="data"**
|
||||
1. `raw`
|
||||
1. `raw` == "[DONE]"
|
||||
**_ = 1**
|
||||
2. *
|
||||
**parsed = > json.parse text=`raw`**
|
||||
1. `parsed`
|
||||
**choices = > json.get value=`parsed` key="choices"**
|
||||
1. `choices`
|
||||
**c0 = [1](access(choices, "1"))**
|
||||
**delta = > json.get value=`c0` key="delta"**
|
||||
1. `delta`
|
||||
**piece = > json.get value=`delta` key="content"**
|
||||
1. `piece`
|
||||
**answer = answer + piece**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**msg = > json.get value=`c0` key="message"**
|
||||
1. `msg`
|
||||
**piece = > json.get value=`msg` key="content"**
|
||||
1. `piece`
|
||||
**answer = answer + piece**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
6. *
|
||||
**_ = 1**
|
||||
|
||||
1. not `answer`
|
||||
**answer = ""**
|
||||
*answer*
|
||||
|
||||
## collect_usage
|
||||
+ `events`
|
||||
|
||||
Best-effort `total_tokens` from stream frames (requires `stream_options.include_usage`).
|
||||
|
||||
**tokens = None**
|
||||
1. `events`
|
||||
- [ev](遍历 events 为 ev)
|
||||
**t = > json.get value=`ev` key="type"**
|
||||
**raw = None**
|
||||
1. `t` == "data"
|
||||
**raw = > json.get value=`ev` key="data"**
|
||||
2. *
|
||||
**u0 = > json.get value=`ev` key="usage"**
|
||||
1. `u0`
|
||||
**tt0 = > json.get value=`u0` key="total_tokens"**
|
||||
1. `tt0` != None
|
||||
**tokens = `tt0`**
|
||||
1. `raw`
|
||||
1. not `raw` == "[DONE]"
|
||||
**parsed = > json.parse text=`raw`**
|
||||
1. `parsed`
|
||||
**u = > json.get value=`parsed` key="usage"**
|
||||
1. `u`
|
||||
**tt = > json.get value=`u` key="total_tokens"**
|
||||
1. `tt` != None
|
||||
**tokens = `tt`**
|
||||
*tokens*
|
||||
|
||||
## stream_result
|
||||
+ `events`
|
||||
|
||||
@@ -156,6 +232,7 @@ Prompt artifact helper: `path=` loads a file; `text=` strips `type: prompt` fron
|
||||
|
||||
Module convenience: create a default handle and return **answer text**. Pass `path=` to load a prompt document, `prompt=` text, or omit both to use the entry `.mq.md` as a prompt artifact (`type: prompt` body via `sys.module_source`).
|
||||
|
||||
**p = `prompt`**
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
@@ -268,20 +345,26 @@ Prefer declaring `model` / `api_key` / `base_url` in frontmatter Binding, then u
|
||||
## ask
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
+ `messages`=None
|
||||
+ `selection`=None
|
||||
+ `strict_selection`=False
|
||||
|
||||
Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`).
|
||||
Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`). Pass `messages=` for multi-turn history (M01). `selection=` + `strict_selection=True` freeze request identity without env fallback.
|
||||
|
||||
1. `path`
|
||||
**p = `prompt`**
|
||||
1. `messages`
|
||||
**p = None**
|
||||
2. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
3. `prompt`
|
||||
**p = `prompt`**
|
||||
3. *
|
||||
4. *
|
||||
**raw = > sys.module_source**
|
||||
1. not `raw`
|
||||
**p = ""**
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
**pack = > self.complete_result prompt=`p` stream=False echo=False**
|
||||
**pack = > self.complete_result prompt=`p` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=False echo=False**
|
||||
**text = > json.get value=`pack` key="text"**
|
||||
**usage = > json.get value=`pack` key="usage"**
|
||||
**finish = > json.get value=`pack` key="finish"**
|
||||
@@ -312,11 +395,16 @@ inside the table is unsafe.
|
||||
**nm = > json.get value=`self` key="name"**
|
||||
1. `usage`
|
||||
**tt = > json.get value=`usage` key="total_tokens"**
|
||||
**ustate = > json.get value=`usage` key="usage_state"**
|
||||
2. *
|
||||
**tt = 0**
|
||||
1. not `tt`
|
||||
**tt = 0**
|
||||
2. *
|
||||
**tt = None**
|
||||
**ustate = "unknown"**
|
||||
1. `ustate` == "unknown"
|
||||
Keep null — never coerce missing usage to zero tokens (M02 / F04).
|
||||
**tt = None**
|
||||
2. not `tt`
|
||||
**tt = None**
|
||||
3. *
|
||||
**_ = 1**
|
||||
1. `err`
|
||||
**calls = 0**
|
||||
@@ -325,9 +413,9 @@ inside the table is unsafe.
|
||||
|
||||
`result` =
|
||||
|
||||
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error |
|
||||
|------|-------|--------|---------|-------|------|-----------|--------|------------|-------|
|
||||
| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | `calls` | `tt` | `tc` | `err` |
|
||||
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error | selection |
|
||||
|------|-------|--------|---------|-------|------|-----------|--------|------------|-------|-----------|
|
||||
| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | `calls` | `tt` | `tc` | `err` | `selection` |
|
||||
|
||||
*result*
|
||||
|
||||
@@ -335,8 +423,10 @@ inside the table is unsafe.
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
Semantic stream: return the event list (use `llm.collect` to reduce to text). Omit `prompt=` / `path=` to use the entry document.
|
||||
`stream_kind=plan` tags EventBus frames for the parent plan protocol (process trail).
|
||||
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
@@ -349,30 +439,35 @@ Semantic stream: return the event list (use `llm.collect` to reduce to text). Om
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
|
||||
*> self.complete prompt=`p` stream=True echo=`echo`*
|
||||
*> self.complete prompt=`p` stream=True echo=`echo` stream_kind=`stream_kind`*
|
||||
|
||||
## complete_result
|
||||
+ `prompt`
|
||||
+ `prompt`=None
|
||||
+ `messages`=None
|
||||
+ `selection`=None
|
||||
+ `strict_selection`=False
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`.
|
||||
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`. Prefer `messages=` for multi-turn; optional `selection=` (ModelSelectionV1) with `strict_selection=True` for request-scoped identity (M01).
|
||||
|
||||
*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` stream=`stream` echo=`echo` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))*
|
||||
*> openai.chat_completions base_url=[base_url](access(self, "base_url")) api_key=[api_key](access(self, "api_key")) model=[model](access(self, "model")) prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` stream=`stream` echo=`echo` stream_kind=`stream_kind` suffix=[suffix](access(self, "suffix")) bearer=[bearer](access(self, "bearer"))*
|
||||
|
||||
## complete
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
Internal / compatibility primitive. Prefer `ask` / `stream` in new code.
|
||||
|
||||
1. `stream`
|
||||
**pack = > self.complete_result prompt=`prompt` stream=True echo=`echo`**
|
||||
**pack = > self.complete_result prompt=`prompt` stream=True echo=`echo` stream_kind=`stream_kind`**
|
||||
**_ret = > json.get value=`pack` key="events"**
|
||||
*_ret*
|
||||
2. *
|
||||
**pack = > self.complete_result prompt=`prompt` stream=False echo=`echo`**
|
||||
**pack = > self.complete_result prompt=`prompt` stream=False echo=`echo` stream_kind=`stream_kind`**
|
||||
**_ret = > json.get value=`pack` key="text"**
|
||||
*_ret*
|
||||
|
||||
@@ -380,7 +475,8 @@ Internal / compatibility primitive. Prefer `ask` / `stream` in new code.
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
|
||||
Alias for `complete` (compat).
|
||||
|
||||
*> self.complete prompt=`prompt` stream=`stream` echo=`echo`*
|
||||
*> self.complete prompt=`prompt` stream=`stream` echo=`echo` stream_kind=`stream_kind`*
|
||||
+24
-6
@@ -73,28 +73,39 @@ Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list).
|
||||
+ `base_url`
|
||||
+ `api_key`
|
||||
+ `model`
|
||||
+ `prompt`
|
||||
+ `prompt`=None
|
||||
+ `messages`=None
|
||||
+ `selection`=None
|
||||
+ `strict_selection`=False
|
||||
+ `auth_mode`=None
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `stream_kind`="delta"
|
||||
+ `suffix`=/chat/completions
|
||||
+ `bearer`="Bearer "
|
||||
|
||||
POST `/chat/completions` (or stream SSE). Prefers `plugins/llm` ABI (`llm_chat_completions`) when the native plugin is resolvable; otherwise `lib/net` HTTP/SSE. Non-stream → `{text,usage,finish,tool_calls}`; stream → `{events,finish}`. Soft transport / HTTP failures return `finish=error` packs (never abort the VM).
|
||||
|
||||
Pass either `messages=` (role/content list) or legacy `prompt=` (single user message). Optional `selection=` is a ModelSelectionV1 snapshot; `strict_selection=True` refuses env fallbacks inside the plugin.
|
||||
`stream_kind` tags live EventBus frames: `delta` (answer tokens) or `plan` (parent protocol — UI process trail, not the bubble).
|
||||
|
||||
**p = > plugin.native_path name="llm"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`*
|
||||
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` messages=`messages` selection=`selection` strict_selection=`strict_selection` auth_mode=`auth_mode` stream=`stream` echo=`echo` stream_kind=`stream_kind` suffix=`suffix` bearer=`bearer`*
|
||||
2. *
|
||||
**url = base_url + suffix**
|
||||
**auth = bearer + api_key**
|
||||
**headers = > table.put in=None at="Authorization" value=`auth`**
|
||||
|
||||
`messages` =
|
||||
1. not `messages`
|
||||
`messages` =
|
||||
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `prompt` |
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `prompt` |
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
`req` =
|
||||
|
||||
@@ -104,6 +115,13 @@ POST `/chat/completions` (or stream SSE). Prefers `plugins/llm` ABI (`llm_chat_c
|
||||
|
||||
1. `stream`
|
||||
**req = > table.put in=`req` at="stream" value=True**
|
||||
`stream_opts` =
|
||||
|
||||
| include_usage |
|
||||
|---------------|
|
||||
| True |
|
||||
|
||||
**req = > table.put in=`req` at="stream_options" value=`stream_opts`**
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post_sse url=`url` body=`body` headers=`headers` echo=`echo`**
|
||||
**st = > json.get value=`resp` key="status"**
|
||||
|
||||
+50
-3
@@ -20,14 +20,25 @@ description: >-
|
||||
## 收集
|
||||
+ `事件`
|
||||
|
||||
Collapse stream events to text. Supports semantic `delta`/`done.result` and
|
||||
native-plugin raw frames (`type=data` + OpenAI chunk JSON).
|
||||
|
||||
**answer = ""**
|
||||
|
||||
- [ev](遍历 事件 为 ev)
|
||||
**t = [type](access(ev, "type"))**
|
||||
1. `t` == "done"
|
||||
**answer = [result](access(ev, "result"))**
|
||||
**done_r = [result](access(ev, "result"))**
|
||||
1. `done_r`
|
||||
**answer = `done_r`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. `t` == "error"
|
||||
**answer = [message](access(ev, "message"))**
|
||||
**em = [message](access(ev, "message"))**
|
||||
1. `em`
|
||||
**answer = `em`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. `t` == "delta"
|
||||
4. `t` == "text"
|
||||
**chunk = [text](access(ev, "text"))**
|
||||
@@ -35,9 +46,45 @@ description: >-
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
5. `t` == "data"
|
||||
**raw = [data](access(ev, "data"))**
|
||||
1. `raw`
|
||||
1. `raw` == "[DONE]"
|
||||
**_ = 1**
|
||||
2. *
|
||||
**parsed = > json.parse text=`raw`**
|
||||
1. `parsed`
|
||||
**choices = [choices](access(parsed, "choices"))**
|
||||
1. `choices`
|
||||
**c0 = [1](access(choices, "1"))**
|
||||
**delta = [delta](access(c0, "delta"))**
|
||||
1. `delta`
|
||||
**piece = [content](access(delta, "content"))**
|
||||
1. `piece`
|
||||
**answer = answer + piece**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**msg = [message](access(c0, "message"))**
|
||||
1. `msg`
|
||||
**piece = [content](access(msg, "content"))**
|
||||
1. `piece`
|
||||
**answer = answer + piece**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
6. *
|
||||
**_ = 1**
|
||||
|
||||
1. not `answer`
|
||||
**answer = ""**
|
||||
*answer*
|
||||
|
||||
## 流式结果
|
||||
|
||||
@@ -86,6 +86,19 @@ Like `select` but returns `{ rows, total }` — the total counts rows matching `
|
||||
**url = [url](access(self, "url"))**
|
||||
*> web_db_update url=`url` table=`table` id=`id` row=`row` txn=`txn`*
|
||||
|
||||
## update_if
|
||||
+ `table`
|
||||
+ `id`
|
||||
+ `row`
|
||||
+ `expect_col`
|
||||
+ `expect_val`
|
||||
+ `txn`=None
|
||||
|
||||
Conditional update (CAS) for task/revision control: `UPDATE … WHERE id=? AND expect_col=expect_val`. Returns `{ ok, updated, matched }`; `updated=0` means conflict — do not simulate atomicity with read-then-write in `.mq.md`.
|
||||
|
||||
**url = [url](access(self, "url"))**
|
||||
*> web_db_update_if url=`url` table=`table` id=`id` row=`row` expect_col=`expect_col` expect_val=`expect_val` txn=`txn`*
|
||||
|
||||
## delete
|
||||
+ `table`
|
||||
+ `id`
|
||||
|
||||
+7
-4
@@ -16,15 +16,17 @@ import:
|
||||
+ `request`="json"
|
||||
+ `response`="json"
|
||||
+ `auth`=""
|
||||
+ `class`=""
|
||||
|
||||
Declared route handle. Prefer `web.route` / Artifact `type: endpoint` discovery via `web.serve`.
|
||||
Optional `class`: `execute` | `short` | `read` | `control` (empty → heuristic / `legacy_serial`).
|
||||
|
||||
> cap.load
|
||||
`out` =
|
||||
|
||||
| _type | app | path | method | page | fn | request | response | auth |
|
||||
|-------|-----|------|--------|------|----|---------|----------|------|
|
||||
| route | `app` | `path` | `method` | `page` | `fn` | `request` | `response` | `auth` |
|
||||
| _type | app | path | method | page | fn | request | response | auth | class |
|
||||
|-------|-----|------|--------|------|----|---------|----------|------|-------|
|
||||
| route | `app` | `path` | `method` | `page` | `fn` | `request` | `response` | `auth` | `class` |
|
||||
|
||||
*out*
|
||||
|
||||
@@ -57,6 +59,7 @@ Register this declared route on an app handle.
|
||||
**method = [method](access(self, "method"))**
|
||||
**request = [request](access(self, "request"))**
|
||||
**response = [response](access(self, "response"))**
|
||||
*> web_app_invoke app=`a` path=`path` method=`method` fn=`fn` body=`request` return=`response`*
|
||||
**class = [class](access(self, "class"))**
|
||||
*> web_app_invoke app=`a` path=`path` method=`method` fn=`fn` body=`request` return=`response` class=`class`*
|
||||
3. *
|
||||
*> table.put in=None at="ok" value=False*
|
||||
+3
-1
@@ -38,10 +38,12 @@ Construct a Document handle. Prefer `type: web` Artifact files discovered by `se
|
||||
+ `request`="json"
|
||||
+ `response`="json"
|
||||
+ `auth`=""
|
||||
+ `class`=""
|
||||
|
||||
Register a declared Document page or Endpoint (`fn` = `lib.member`).
|
||||
Optional `class`: `execute` | `short` | `read` | `control`.
|
||||
|
||||
**r = > route_mod.route app=`app` path=`path` method=`method` page=`page` fn=`fn` request=`request` response=`response` auth=`auth`**
|
||||
**r = > route_mod.route app=`app` path=`path` method=`method` page=`page` fn=`fn` request=`request` response=`response` auth=`auth` class=`class`**
|
||||
*> `r`.register app=`app`*
|
||||
|
||||
## render
|
||||
|
||||
@@ -0,0 +1,266 @@
|
||||
//! ContextPackV2 assemble + compact (M-A4).
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn item_tokens(item: &Value) -> u64 {
|
||||
item.get("token_estimate")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or_else(|| {
|
||||
item.get("text")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|s| (s.chars().count() as u64).div_ceil(2))
|
||||
.unwrap_or(32)
|
||||
})
|
||||
}
|
||||
|
||||
fn normalize_item(raw: &Value, default_priority: i64) -> Result<Value, String> {
|
||||
let origin = raw
|
||||
.get("origin")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("unknown");
|
||||
let trust = raw
|
||||
.get("trust")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(match origin {
|
||||
"system" | "policy" => "trusted",
|
||||
"user" => "user",
|
||||
"tool" => "tool",
|
||||
"web" => "untrusted",
|
||||
_ => "unknown",
|
||||
});
|
||||
// External / web content must never be trust=trusted.
|
||||
if matches!(origin, "web" | "tool") && trust == "trusted" {
|
||||
return Err("tool/web origin cannot have trust=trusted".into());
|
||||
}
|
||||
Ok(json!({
|
||||
"origin": origin,
|
||||
"trust": trust,
|
||||
"resource_ref": raw.get("resource_ref").cloned().unwrap_or(Value::Null),
|
||||
"revision": raw.get("revision").cloned().unwrap_or(Value::Null),
|
||||
"token_estimate": item_tokens(raw),
|
||||
"priority": raw.get("priority").and_then(|v| v.as_i64()).unwrap_or(default_priority),
|
||||
"role": raw.get("role").cloned().unwrap_or(json!(origin)),
|
||||
"text": raw.get("text").cloned().unwrap_or(Value::Null),
|
||||
"kind": raw.get("kind").cloned().unwrap_or(json!("content")),
|
||||
"retain": raw.get("retain").and_then(|v| v.as_bool()).unwrap_or(false),
|
||||
}))
|
||||
}
|
||||
|
||||
/// Build ContextPackV2 from items + optional budget.
|
||||
pub fn context_pack_build(args: &Value) -> Result<Value, String> {
|
||||
let raw_items = args
|
||||
.get("items")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
let mut items = Vec::new();
|
||||
for (i, raw) in raw_items.iter().enumerate() {
|
||||
items.push(normalize_item(raw, i as i64)?);
|
||||
}
|
||||
// Stable priority order: lower priority number first (more important).
|
||||
items.sort_by(|a, b| {
|
||||
let pa = a.get("priority").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
let pb = b.get("priority").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
pa.cmp(&pb)
|
||||
});
|
||||
let budget = args
|
||||
.get("budget_tokens")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(8_000);
|
||||
let reserve_pct = args
|
||||
.get("output_reserve_pct")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(20)
|
||||
.min(50);
|
||||
let available = budget.saturating_mul(100 - reserve_pct) / 100;
|
||||
let used: u64 = items.iter().map(item_tokens).sum();
|
||||
Ok(json!({
|
||||
"schema": "ContextPackV2",
|
||||
"schema_version": 2,
|
||||
"items": items,
|
||||
"budget_tokens": budget,
|
||||
"available_tokens": available,
|
||||
"used_tokens": used,
|
||||
"output_reserve_pct": reserve_pct,
|
||||
"compact_threshold_pct": args.get("compact_threshold_pct").and_then(|v| v.as_u64()).unwrap_or(70),
|
||||
"goal": args.get("goal").cloned().unwrap_or(Value::Null),
|
||||
"constraints": args.get("constraints").cloned().unwrap_or_else(|| json!([])),
|
||||
}))
|
||||
}
|
||||
|
||||
/// Compact pack when over threshold; must retain anchors listed in design §6.2.
|
||||
pub fn context_pack_compact(args: &Value) -> Result<Value, String> {
|
||||
let mut pack = if args.get("schema").and_then(|v| v.as_str()) == Some("ContextPackV2") {
|
||||
args.clone()
|
||||
} else {
|
||||
context_pack_build(args)?
|
||||
};
|
||||
let available = pack
|
||||
.get("available_tokens")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(6400);
|
||||
let threshold_pct = pack
|
||||
.get("compact_threshold_pct")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(70);
|
||||
let threshold = available.saturating_mul(threshold_pct) / 100;
|
||||
let used = pack
|
||||
.get("used_tokens")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(0);
|
||||
if used <= threshold {
|
||||
if let Some(obj) = pack.as_object_mut() {
|
||||
obj.insert("compacted".into(), json!(false));
|
||||
obj.insert("reason".into(), json!("under_threshold"));
|
||||
}
|
||||
return Ok(pack);
|
||||
}
|
||||
|
||||
let items = pack
|
||||
.get("items")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
// Retain: goal/constraints (pack fields), retain=true, high priority (<=2),
|
||||
// pending writes, failures — drop bulky tool dumps first (kind=tool_output, low priority).
|
||||
let mut kept = Vec::new();
|
||||
let mut dropped = Vec::new();
|
||||
for item in items {
|
||||
let retain = item.get("retain").and_then(|v| v.as_bool()).unwrap_or(false);
|
||||
let priority = item.get("priority").and_then(|v| v.as_i64()).unwrap_or(99);
|
||||
let kind = item.get("kind").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let trust = item.get("trust").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let must = retain
|
||||
|| priority <= 2
|
||||
|| matches!(kind, "goal" | "constraint" | "pending_write" | "plan" | "failure")
|
||||
|| trust == "trusted";
|
||||
if must {
|
||||
kept.push(item);
|
||||
} else if kind == "tool_output" || priority >= 8 {
|
||||
dropped.push(item);
|
||||
} else {
|
||||
kept.push(item);
|
||||
}
|
||||
}
|
||||
// If still over budget, drop lowest-priority non-retain from kept.
|
||||
let mut kept_tokens: u64 = kept.iter().map(item_tokens).sum();
|
||||
if kept_tokens > threshold {
|
||||
kept.sort_by(|a, b| {
|
||||
let pa = a.get("priority").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
let pb = b.get("priority").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
pb.cmp(&pa) // high priority number first for dropping
|
||||
});
|
||||
let mut next = Vec::new();
|
||||
for item in kept.into_iter().rev() {
|
||||
// rebuild ascending later
|
||||
let retain = item.get("retain").and_then(|v| v.as_bool()).unwrap_or(false);
|
||||
let priority = item.get("priority").and_then(|v| v.as_i64()).unwrap_or(99);
|
||||
let t = item_tokens(&item);
|
||||
if (retain || priority <= 1) || kept_tokens <= threshold {
|
||||
next.push(item);
|
||||
} else if kept_tokens > threshold {
|
||||
kept_tokens = kept_tokens.saturating_sub(t);
|
||||
dropped.push(item);
|
||||
} else {
|
||||
next.push(item);
|
||||
}
|
||||
}
|
||||
next.sort_by(|a, b| {
|
||||
let pa = a.get("priority").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
let pb = b.get("priority").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
pa.cmp(&pb)
|
||||
});
|
||||
kept = next;
|
||||
kept_tokens = kept.iter().map(item_tokens).sum();
|
||||
}
|
||||
|
||||
let digest_src = format!(
|
||||
"kept={} dropped={} used={}",
|
||||
kept.len(),
|
||||
dropped.len(),
|
||||
kept_tokens
|
||||
);
|
||||
let digest = format!("sha256-lite:{}", simple_hash(&digest_src));
|
||||
|
||||
if let Some(obj) = pack.as_object_mut() {
|
||||
obj.insert("items".into(), json!(kept));
|
||||
obj.insert("used_tokens".into(), json!(kept_tokens));
|
||||
obj.insert("compacted".into(), json!(true));
|
||||
obj.insert(
|
||||
"compaction".into(),
|
||||
json!({
|
||||
"dropped_count": dropped.len(),
|
||||
"source_event_range": args.get("source_event_range").cloned().unwrap_or(Value::Null),
|
||||
"digest": digest,
|
||||
"retained": ["goal", "constraints", "pending_write", "plan", "trusted_policy"],
|
||||
}),
|
||||
);
|
||||
// Goal / constraints fields stay untouched.
|
||||
if obj.get("goal").map(|v| v.is_null()).unwrap_or(true) {
|
||||
if let Some(g) = opt_text(args, "goal") {
|
||||
obj.insert("goal".into(), json!(g));
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(pack)
|
||||
}
|
||||
|
||||
fn simple_hash(s: &str) -> u64 {
|
||||
let mut h: u64 = 0xcbf29ce484222325;
|
||||
for b in s.as_bytes() {
|
||||
h ^= u64::from(*b);
|
||||
h = h.wrapping_mul(0x100000001b3);
|
||||
}
|
||||
h
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn build_sorts_and_rejects_trusted_web() {
|
||||
let pack = context_pack_build(&json!({
|
||||
"budget_tokens": 1000,
|
||||
"items": [
|
||||
{"origin": "user", "text": "goal", "priority": 0, "kind": "goal", "retain": true},
|
||||
{"origin": "tool", "text": "big dump", "priority": 9, "kind": "tool_output", "token_estimate": 500}
|
||||
]
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(pack["schema"], "ContextPackV2");
|
||||
assert_eq!(pack["items"][0]["priority"], 0);
|
||||
|
||||
let bad = context_pack_build(&json!({
|
||||
"items": [{"origin": "web", "trust": "trusted", "text": "x"}]
|
||||
}));
|
||||
assert!(bad.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn compact_keeps_goal_drops_tool_output() {
|
||||
let pack = context_pack_build(&json!({
|
||||
"budget_tokens": 200,
|
||||
"output_reserve_pct": 20,
|
||||
"compact_threshold_pct": 50,
|
||||
"goal": "finish task",
|
||||
"items": [
|
||||
{"origin": "user", "text": "do it", "priority": 0, "kind": "goal", "retain": true, "token_estimate": 20},
|
||||
{"origin": "tool", "text": "x".repeat(400), "priority": 9, "kind": "tool_output", "token_estimate": 200}
|
||||
]
|
||||
}))
|
||||
.unwrap();
|
||||
let compacted = context_pack_compact(&pack).unwrap();
|
||||
assert_eq!(compacted["compacted"], true);
|
||||
let items = compacted["items"].as_array().unwrap();
|
||||
assert!(items.iter().any(|i| i["kind"] == "goal"));
|
||||
assert!(!items.iter().any(|i| i["kind"] == "tool_output"));
|
||||
assert_eq!(compacted["goal"], "finish task");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,623 @@
|
||||
//! EffectLedgerRecordV1 — operation receipts for write-safe resume (M-A5 subset).
|
||||
//!
|
||||
//! State machine (design §4.2):
|
||||
//! prepared → started → committed
|
||||
//! ↘ failed_no_effect
|
||||
//! ↘ unknown → reconciled_committed | reconciled_no_effect | needs_input
|
||||
|
||||
use std::fs::{self, File, OpenOptions};
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
const DEFAULT_LEDGER_DIR: &str = ".marqdo/agent-effects";
|
||||
|
||||
const STATES: &[&str] = &[
|
||||
"prepared",
|
||||
"started",
|
||||
"committed",
|
||||
"failed_no_effect",
|
||||
"unknown",
|
||||
"reconciled_committed",
|
||||
"reconciled_no_effect",
|
||||
"needs_input",
|
||||
];
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
opt_text(args, key).ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn now_rfc3339() -> String {
|
||||
chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, true)
|
||||
}
|
||||
|
||||
fn resolve_path(rel: &str) -> PathBuf {
|
||||
let p = PathBuf::from(rel);
|
||||
if p.is_absolute() {
|
||||
return p;
|
||||
}
|
||||
if p.exists() {
|
||||
return p;
|
||||
}
|
||||
let base = (|| {
|
||||
let q = crate::host_query_json("cwd")?;
|
||||
q.as_str()
|
||||
.map(PathBuf::from)
|
||||
.ok_or_else(|| "cwd not text".to_string())
|
||||
})()
|
||||
.or_else(|_| std::env::current_dir().map_err(|e| e.to_string()))
|
||||
.unwrap_or_else(|_| PathBuf::from("."));
|
||||
base.join(p)
|
||||
}
|
||||
|
||||
fn ledger_root(args: &Value) -> Result<PathBuf, String> {
|
||||
let rel = opt_text(args, "ledger_dir").unwrap_or(DEFAULT_LEDGER_DIR);
|
||||
let p = PathBuf::from(rel);
|
||||
if p.is_absolute() {
|
||||
return Err(
|
||||
"ledger_dir must be host-relative; absolute client paths are rejected".into(),
|
||||
);
|
||||
}
|
||||
if rel.contains("..") {
|
||||
return Err("ledger_dir must not contain '..'".into());
|
||||
}
|
||||
Ok(resolve_path(rel))
|
||||
}
|
||||
|
||||
fn safe_id(id: &str) -> String {
|
||||
id.chars()
|
||||
.map(|c| {
|
||||
if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
|
||||
c
|
||||
} else {
|
||||
'_'
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn record_path(root: &Path, operation_id: &str) -> PathBuf {
|
||||
root.join(format!("{}.json", safe_id(operation_id)))
|
||||
}
|
||||
|
||||
fn ensure_dir(root: &Path) -> Result<(), String> {
|
||||
fs::create_dir_all(root).map_err(|e| format!("effects mkdir: {e}"))
|
||||
}
|
||||
|
||||
fn atomic_write_json(path: &Path, doc: &Value) -> Result<(), String> {
|
||||
let parent = path
|
||||
.parent()
|
||||
.ok_or_else(|| "ledger path has no parent".to_string())?;
|
||||
ensure_dir(parent)?;
|
||||
let tmp_name = format!(
|
||||
".{}.tmp-{}-{}",
|
||||
path.file_name()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("op.json"),
|
||||
std::process::id(),
|
||||
now_rfc3339().replace(':', "")
|
||||
);
|
||||
let tmp = parent.join(tmp_name);
|
||||
let text = serde_json::to_string_pretty(doc).map_err(|e| e.to_string())?;
|
||||
{
|
||||
let mut f = OpenOptions::new()
|
||||
.create(true)
|
||||
.write(true)
|
||||
.truncate(true)
|
||||
.open(&tmp)
|
||||
.map_err(|e| format!("effects temp open: {e}"))?;
|
||||
f.write_all(text.as_bytes())
|
||||
.map_err(|e| format!("effects temp write: {e}"))?;
|
||||
f.flush().map_err(|e| format!("effects temp flush: {e}"))?;
|
||||
f.sync_all()
|
||||
.map_err(|e| format!("effects temp fsync: {e}"))?;
|
||||
}
|
||||
if let Ok(dir) = File::open(parent) {
|
||||
let _ = dir.sync_all();
|
||||
}
|
||||
fs::rename(&tmp, path).map_err(|e| {
|
||||
let _ = fs::remove_file(&tmp);
|
||||
format!("effects atomic rename: {e}")
|
||||
})?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn load_record(path: &Path) -> Result<Option<Value>, String> {
|
||||
if !path.is_file() {
|
||||
return Ok(None);
|
||||
}
|
||||
let text = fs::read_to_string(path).map_err(|e| format!("effects read: {e}"))?;
|
||||
let doc: Value = serde_json::from_str(&text).map_err(|e| format!("effects corrupt: {e}"))?;
|
||||
Ok(Some(doc))
|
||||
}
|
||||
|
||||
fn valid_transition(from: &str, to: &str) -> bool {
|
||||
matches!(
|
||||
(from, to),
|
||||
("prepared", "started")
|
||||
| ("started", "committed")
|
||||
| ("started", "failed_no_effect")
|
||||
| ("started", "unknown")
|
||||
| ("unknown", "reconciled_committed")
|
||||
| ("unknown", "reconciled_no_effect")
|
||||
| ("unknown", "needs_input")
|
||||
// Idempotent no-ops / crash retries of the same terminal.
|
||||
| ("committed", "committed")
|
||||
| ("failed_no_effect", "failed_no_effect")
|
||||
| ("reconciled_committed", "reconciled_committed")
|
||||
| ("reconciled_no_effect", "reconciled_no_effect")
|
||||
| ("needs_input", "needs_input")
|
||||
// Re-prepare of same op is handled separately via idempotency.
|
||||
| ("prepared", "prepared")
|
||||
)
|
||||
}
|
||||
|
||||
fn effect_class_ok(c: &str) -> bool {
|
||||
matches!(c, "read" | "write" | "execute" | "external_write" | "none")
|
||||
}
|
||||
|
||||
/// Prepare (or return existing) an EffectLedgerRecordV1.
|
||||
pub fn effects_prepare(args: &Value) -> Result<Value, String> {
|
||||
let operation_id = arg_text(args, "operation_id")?;
|
||||
let idempotency_key = arg_text(args, "idempotency_key")?;
|
||||
let args_digest = arg_text(args, "args_digest")?;
|
||||
let effect_class = opt_text(args, "effect_class").unwrap_or("write");
|
||||
if !effect_class_ok(effect_class) {
|
||||
return Err(format!("invalid effect_class={effect_class}"));
|
||||
}
|
||||
let root = ledger_root(args)?;
|
||||
ensure_dir(&root)?;
|
||||
let path = record_path(&root, operation_id);
|
||||
|
||||
if let Some(existing) = load_record(&path)? {
|
||||
let same_key = existing
|
||||
.get("idempotency_key")
|
||||
.and_then(|v| v.as_str())
|
||||
== Some(idempotency_key);
|
||||
let same_digest = existing.get("args_digest").and_then(|v| v.as_str()) == Some(args_digest);
|
||||
if same_key && same_digest {
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"deduped": true,
|
||||
"record": existing,
|
||||
}));
|
||||
}
|
||||
if same_key && !same_digest {
|
||||
return Err(
|
||||
"idempotency_key collision: same key with different args_digest".into(),
|
||||
);
|
||||
}
|
||||
// Different operation_id path can't collide; same path different key is corrupt reuse.
|
||||
return Err("operation_id already exists with different idempotency_key".into());
|
||||
}
|
||||
|
||||
// Also scan for same idempotency_key under other operation_ids in this ledger dir.
|
||||
if let Ok(rd) = fs::read_dir(&root) {
|
||||
for ent in rd.flatten() {
|
||||
let p = ent.path();
|
||||
if p.extension().and_then(|e| e.to_str()) != Some("json") {
|
||||
continue;
|
||||
}
|
||||
if let Ok(Some(doc)) = load_record(&p) {
|
||||
if doc.get("idempotency_key").and_then(|v| v.as_str()) == Some(idempotency_key) {
|
||||
let dig = doc.get("args_digest").and_then(|v| v.as_str());
|
||||
if dig == Some(args_digest) {
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"deduped": true,
|
||||
"record": doc,
|
||||
}));
|
||||
}
|
||||
return Err(
|
||||
"idempotency_key collision: same key with different args_digest".into(),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let doc = json!({
|
||||
"schema": "EffectLedgerRecordV1",
|
||||
"schema_version": 1,
|
||||
"operation_id": operation_id,
|
||||
"idempotency_key": idempotency_key,
|
||||
"args_digest": args_digest,
|
||||
"resource_scope": args.get("resource_scope").cloned().unwrap_or(Value::Null),
|
||||
"expected_revision": args.get("expected_revision").cloned().unwrap_or(Value::Null),
|
||||
"effect_class": effect_class,
|
||||
"result_ref": args.get("result_ref").cloned().unwrap_or(Value::Null),
|
||||
"state": "prepared",
|
||||
"logical_run_id": args.get("logical_run_id").cloned().unwrap_or(Value::Null),
|
||||
"step_id": args.get("step_id").cloned().unwrap_or(Value::Null),
|
||||
"attempt_id": args.get("attempt_id").cloned().unwrap_or(Value::Null),
|
||||
"prepared_at": now_rfc3339(),
|
||||
"updated_at": now_rfc3339(),
|
||||
});
|
||||
atomic_write_json(&path, &doc)?;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"deduped": false,
|
||||
"record": doc,
|
||||
"path": path.to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
|
||||
/// Transition an existing record to a new state.
|
||||
pub fn effects_transition(args: &Value) -> Result<Value, String> {
|
||||
let operation_id = arg_text(args, "operation_id")?;
|
||||
let to = arg_text(args, "state")?;
|
||||
if !STATES.contains(&to) {
|
||||
return Err(format!("invalid state={to}"));
|
||||
}
|
||||
let root = ledger_root(args)?;
|
||||
let path = record_path(&root, operation_id);
|
||||
let mut doc = load_record(&path)?
|
||||
.ok_or_else(|| format!("operation_id not found: {operation_id}"))?;
|
||||
let from = doc
|
||||
.get("state")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("prepared")
|
||||
.to_string();
|
||||
if !valid_transition(&from, to) {
|
||||
return Err(format!("illegal transition {from} → {to}"));
|
||||
}
|
||||
if let Some(obj) = doc.as_object_mut() {
|
||||
obj.insert("state".into(), json!(to));
|
||||
obj.insert("updated_at".into(), json!(now_rfc3339()));
|
||||
if let Some(r) = args.get("result_ref") {
|
||||
if !r.is_null() {
|
||||
obj.insert("result_ref".into(), r.clone());
|
||||
}
|
||||
}
|
||||
if let Some(n) = args.get("note") {
|
||||
if !n.is_null() {
|
||||
obj.insert("note".into(), n.clone());
|
||||
}
|
||||
}
|
||||
if to == "unknown" || to.starts_with("reconciled") {
|
||||
obj.insert(
|
||||
"reconcile_hint".into(),
|
||||
args.get("reconcile_hint")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!("check receipt / resource revision")),
|
||||
);
|
||||
}
|
||||
}
|
||||
atomic_write_json(&path, &doc)?;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"from": from,
|
||||
"to": to,
|
||||
"record": doc,
|
||||
}))
|
||||
}
|
||||
|
||||
pub fn effects_get(args: &Value) -> Result<Value, String> {
|
||||
let operation_id = arg_text(args, "operation_id")?;
|
||||
let root = ledger_root(args)?;
|
||||
let path = record_path(&root, operation_id);
|
||||
match load_record(&path)? {
|
||||
Some(doc) => Ok(json!({
|
||||
"ok": true,
|
||||
"found": true,
|
||||
"record": doc,
|
||||
"path": path.to_string_lossy(),
|
||||
})),
|
||||
None => Ok(json!({
|
||||
"ok": true,
|
||||
"found": false,
|
||||
"operation_id": operation_id,
|
||||
"path": path.to_string_lossy(),
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn effects_list(args: &Value) -> Result<Value, String> {
|
||||
let root = ledger_root(args)?;
|
||||
let mut records = Vec::new();
|
||||
if root.is_dir() {
|
||||
let mut paths: Vec<_> = fs::read_dir(&root)
|
||||
.map_err(|e| e.to_string())?
|
||||
.flatten()
|
||||
.map(|e| e.path())
|
||||
.filter(|p| p.extension().and_then(|e| e.to_str()) == Some("json"))
|
||||
.collect();
|
||||
paths.sort();
|
||||
for p in paths {
|
||||
if let Ok(Some(doc)) = load_record(&p) {
|
||||
if let Some(run) = opt_text(args, "logical_run_id") {
|
||||
if doc.get("logical_run_id").and_then(|v| v.as_str()) != Some(run) {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if let Some(st) = opt_text(args, "state") {
|
||||
if doc.get("state").and_then(|v| v.as_str()) != Some(st) {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
records.push(doc);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"count": records.len(),
|
||||
"records": records,
|
||||
"ledger_dir": root.to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
|
||||
/// Compute a watermark digest over committed / reconciled records for a run.
|
||||
pub fn effects_watermark(args: &Value) -> Result<Value, String> {
|
||||
let listed = effects_list(args)?;
|
||||
let records = listed
|
||||
.get("records")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
let mut parts = Vec::new();
|
||||
let mut open_unknown = 0u64;
|
||||
let mut pending_writes = 0u64;
|
||||
for r in &records {
|
||||
let state = r.get("state").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let oid = r.get("operation_id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let cls = r.get("effect_class").and_then(|v| v.as_str()).unwrap_or("");
|
||||
parts.push(format!("{oid}:{state}"));
|
||||
if state == "unknown" || state == "needs_input" {
|
||||
open_unknown += 1;
|
||||
}
|
||||
if matches!(cls, "write" | "external_write" | "execute")
|
||||
&& matches!(state, "prepared" | "started" | "unknown" | "needs_input")
|
||||
{
|
||||
pending_writes += 1;
|
||||
}
|
||||
}
|
||||
parts.sort();
|
||||
let joined = parts.join("|");
|
||||
let digest = format!("sha256-lite:{:x}", fnv64(&joined));
|
||||
let write_auto_resume_safe = open_unknown == 0 && pending_writes == 0;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"schema": "EffectLedgerWatermarkV1",
|
||||
"schema_version": 1,
|
||||
"watermark": digest,
|
||||
"record_count": records.len(),
|
||||
"open_unknown": open_unknown,
|
||||
"pending_writes": pending_writes,
|
||||
"write_auto_resume_safe": write_auto_resume_safe,
|
||||
"logical_run_id": args.get("logical_run_id").cloned().unwrap_or(Value::Null),
|
||||
}))
|
||||
}
|
||||
|
||||
/// Reconcile an `unknown` record after inspecting the external receipt / revision.
|
||||
pub fn effects_reconcile(args: &Value) -> Result<Value, String> {
|
||||
let outcome = arg_text(args, "outcome")?; // committed | no_effect | needs_input
|
||||
let state = match outcome {
|
||||
"committed" => "reconciled_committed",
|
||||
"no_effect" => "reconciled_no_effect",
|
||||
"needs_input" => "needs_input",
|
||||
other => return Err(format!("invalid reconcile outcome={other}")),
|
||||
};
|
||||
let mut targs = args.clone();
|
||||
if let Some(obj) = targs.as_object_mut() {
|
||||
obj.insert("state".into(), json!(state));
|
||||
}
|
||||
effects_transition(&targs)
|
||||
}
|
||||
|
||||
fn fnv64(s: &str) -> u64 {
|
||||
let mut h: u64 = 0xcbf29ce484222325;
|
||||
for b in s.as_bytes() {
|
||||
h ^= u64::from(*b);
|
||||
h = h.wrapping_mul(0x100000001b3);
|
||||
}
|
||||
h
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn scratch(tag: &str) -> PathBuf {
|
||||
let dir = std::env::temp_dir().join(format!(
|
||||
"mq-effects-{}-{}-{}",
|
||||
tag,
|
||||
std::process::id(),
|
||||
now_rfc3339().replace(':', "")
|
||||
));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
fs::create_dir_all(&dir).unwrap();
|
||||
dir
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prepare_dedupe_and_collision() {
|
||||
let dir = scratch("prep");
|
||||
let prev = std::env::current_dir().unwrap();
|
||||
std::env::set_current_dir(&dir).unwrap();
|
||||
let a = effects_prepare(&json!({
|
||||
"operation_id": "op1",
|
||||
"idempotency_key": "idem1",
|
||||
"args_digest": "sha256:a",
|
||||
"effect_class": "write",
|
||||
"ledger_dir": "led",
|
||||
"logical_run_id": "run1",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(a["deduped"], false);
|
||||
assert_eq!(a["record"]["state"], "prepared");
|
||||
|
||||
let again = effects_prepare(&json!({
|
||||
"operation_id": "op1",
|
||||
"idempotency_key": "idem1",
|
||||
"args_digest": "sha256:a",
|
||||
"ledger_dir": "led",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(again["deduped"], true);
|
||||
|
||||
let clash = effects_prepare(&json!({
|
||||
"operation_id": "op2",
|
||||
"idempotency_key": "idem1",
|
||||
"args_digest": "sha256:b",
|
||||
"ledger_dir": "led",
|
||||
}));
|
||||
assert!(clash.unwrap_err().contains("collision"));
|
||||
|
||||
std::env::set_current_dir(prev).unwrap();
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn transition_and_watermark_gates_write_resume() {
|
||||
let dir = scratch("wm");
|
||||
let prev = std::env::current_dir().unwrap();
|
||||
std::env::set_current_dir(&dir).unwrap();
|
||||
effects_prepare(&json!({
|
||||
"operation_id": "op_w",
|
||||
"idempotency_key": "idem_w",
|
||||
"args_digest": "sha256:w",
|
||||
"effect_class": "write",
|
||||
"ledger_dir": "led",
|
||||
"logical_run_id": "run1",
|
||||
}))
|
||||
.unwrap();
|
||||
effects_transition(&json!({
|
||||
"operation_id": "op_w",
|
||||
"state": "started",
|
||||
"ledger_dir": "led",
|
||||
}))
|
||||
.unwrap();
|
||||
// Crash window: unknown — write auto-resume must stay closed.
|
||||
effects_transition(&json!({
|
||||
"operation_id": "op_w",
|
||||
"state": "unknown",
|
||||
"ledger_dir": "led",
|
||||
}))
|
||||
.unwrap();
|
||||
let wm1 = effects_watermark(&json!({
|
||||
"ledger_dir": "led",
|
||||
"logical_run_id": "run1",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(wm1["write_auto_resume_safe"], false);
|
||||
assert_eq!(wm1["open_unknown"], 1);
|
||||
|
||||
effects_reconcile(&json!({
|
||||
"operation_id": "op_w",
|
||||
"outcome": "committed",
|
||||
"ledger_dir": "led",
|
||||
"result_ref": "receipt:1",
|
||||
}))
|
||||
.unwrap();
|
||||
let wm2 = effects_watermark(&json!({
|
||||
"ledger_dir": "led",
|
||||
"logical_run_id": "run1",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(wm2["write_auto_resume_safe"], true);
|
||||
assert_eq!(wm2["open_unknown"], 0);
|
||||
assert!(wm2["watermark"].as_str().unwrap().starts_with("sha256-lite:"));
|
||||
|
||||
std::env::set_current_dir(prev).unwrap();
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ma_t03_unknown_then_reconcile_no_second_commit_path() {
|
||||
// Write "succeeds" locally only after reconcile; retry while unknown must not
|
||||
// look write_auto_resume_safe (MA-T03 watermark gate).
|
||||
let dir = scratch("mat03");
|
||||
let prev = std::env::current_dir().unwrap();
|
||||
std::env::set_current_dir(&dir).unwrap();
|
||||
effects_prepare(&json!({
|
||||
"operation_id": "op_kill",
|
||||
"idempotency_key": "run|s|write_probe|sha256-lite:1",
|
||||
"args_digest": "sha256-lite:1",
|
||||
"effect_class": "write",
|
||||
"ledger_dir": "led",
|
||||
"logical_run_id": "run_kill",
|
||||
}))
|
||||
.unwrap();
|
||||
effects_transition(&json!({
|
||||
"operation_id": "op_kill",
|
||||
"state": "started",
|
||||
"ledger_dir": "led",
|
||||
}))
|
||||
.unwrap();
|
||||
// Crash before commit:
|
||||
effects_transition(&json!({
|
||||
"operation_id": "op_kill",
|
||||
"state": "unknown",
|
||||
"ledger_dir": "led",
|
||||
}))
|
||||
.unwrap();
|
||||
let wm = effects_watermark(&json!({
|
||||
"ledger_dir": "led",
|
||||
"logical_run_id": "run_kill",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(wm["write_auto_resume_safe"], false);
|
||||
|
||||
// Idempotent re-prepare returns same unknown record (no new op).
|
||||
let again = effects_prepare(&json!({
|
||||
"operation_id": "op_kill",
|
||||
"idempotency_key": "run|s|write_probe|sha256-lite:1",
|
||||
"args_digest": "sha256-lite:1",
|
||||
"effect_class": "write",
|
||||
"ledger_dir": "led",
|
||||
"logical_run_id": "run_kill",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(again["deduped"], true);
|
||||
assert_eq!(again["record"]["state"], "unknown");
|
||||
|
||||
effects_reconcile(&json!({
|
||||
"operation_id": "op_kill",
|
||||
"outcome": "committed",
|
||||
"ledger_dir": "led",
|
||||
"result_ref": "effect:op_kill",
|
||||
}))
|
||||
.unwrap();
|
||||
let wm2 = effects_watermark(&json!({
|
||||
"ledger_dir": "led",
|
||||
"logical_run_id": "run_kill",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(wm2["write_auto_resume_safe"], true);
|
||||
assert_eq!(wm2["pending_writes"], 0);
|
||||
|
||||
std::env::set_current_dir(prev).unwrap();
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn illegal_skip_started() {
|
||||
let dir = scratch("bad");
|
||||
let prev = std::env::current_dir().unwrap();
|
||||
std::env::set_current_dir(&dir).unwrap();
|
||||
effects_prepare(&json!({
|
||||
"operation_id": "op_x",
|
||||
"idempotency_key": "idem_x",
|
||||
"args_digest": "sha256:x",
|
||||
"ledger_dir": "led",
|
||||
}))
|
||||
.unwrap();
|
||||
let err = effects_transition(&json!({
|
||||
"operation_id": "op_x",
|
||||
"state": "committed",
|
||||
"ledger_dir": "led",
|
||||
}))
|
||||
.unwrap_err();
|
||||
assert!(err.contains("illegal transition"));
|
||||
std::env::set_current_dir(prev).unwrap();
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
+241
-7
@@ -77,7 +77,10 @@ fn resolve_marqdo_root(start: &str) -> PathBuf {
|
||||
}
|
||||
for _ in 0..8 {
|
||||
let cand = cur.join(".marqdo");
|
||||
if cand.join("engineering.yaml").exists() || cand.join("graph/graph.json").exists() {
|
||||
if cand.join("engineering.yaml").exists()
|
||||
|| cand.join("ekc/graph/graph.json").exists()
|
||||
|| cand.join("graph/graph.json").exists()
|
||||
{
|
||||
return cand;
|
||||
}
|
||||
if !cur.pop() {
|
||||
@@ -289,7 +292,10 @@ pub fn eng_preflight(args: &Value) -> Result<Value, String> {
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| resolve_marqdo_root(root));
|
||||
|
||||
if !marqdo.join("engineering.yaml").exists() && !marqdo.join("graph/graph.json").exists() {
|
||||
if !marqdo.join("engineering.yaml").exists()
|
||||
&& !marqdo.join("ekc/graph/graph.json").exists()
|
||||
&& !marqdo.join("graph/graph.json").exists()
|
||||
{
|
||||
return Ok(json!({
|
||||
"status": "missing_knowledge",
|
||||
"create_allowed": false,
|
||||
@@ -424,18 +430,246 @@ pub fn eng_preflight(args: &Value) -> Result<Value, String> {
|
||||
}))
|
||||
}
|
||||
|
||||
/// `agent_eng_reuse` — alias of preflight focused on decision payload.
|
||||
/// `agent_eng_reuse` — discovery + REUSE|ADAPT|CREATE decision.
|
||||
///
|
||||
/// Scores and capability names are **not** executable targets (G02/G07).
|
||||
/// When decision is REUSE/ADAPT, callers must `artifact.bind` a
|
||||
/// `bind_targets[]` entry before `artifact.execute`.
|
||||
pub fn eng_reuse(args: &Value) -> Result<Value, String> {
|
||||
let mut pf = eng_preflight(args)?;
|
||||
if let Some(obj) = pf.as_object_mut() {
|
||||
obj.insert(
|
||||
"protocol".into(),
|
||||
json!("REUSE|ADAPT|CREATE"),
|
||||
);
|
||||
obj.insert("protocol".into(), json!("REUSE|ADAPT|CREATE"));
|
||||
obj.insert("discovery_only".into(), json!(true));
|
||||
let decision = obj
|
||||
.get("decision")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_ascii_uppercase();
|
||||
let bind_required = matches!(decision.as_str(), "REUSE" | "ADAPT");
|
||||
obj.insert("bind_required".into(), json!(bind_required));
|
||||
let mut targets = Vec::new();
|
||||
if let Some(cands) = obj.get("candidates").and_then(|v| v.as_array()) {
|
||||
for c in cands {
|
||||
if let Some(impls) = c.get("implemented_by").and_then(|v| v.as_array()) {
|
||||
for t in impls {
|
||||
if let Some(s) = t.as_str() {
|
||||
if !s.is_empty() {
|
||||
targets.push(json!({
|
||||
"query": s,
|
||||
"via": "implemented_by",
|
||||
"capability": c.get("name"),
|
||||
"note": "pass query to artifact.bind then artifact.execute",
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(id) = c.get("id").and_then(|v| v.as_str()) {
|
||||
if id.contains(':') || id.contains('.') {
|
||||
targets.push(json!({
|
||||
"query": id,
|
||||
"via": "candidate_id",
|
||||
"capability": c.get("name"),
|
||||
"note": "pass query to artifact.bind then artifact.execute",
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// Dedup by query
|
||||
let mut seen = std::collections::HashSet::new();
|
||||
targets.retain(|t| {
|
||||
t.get("query")
|
||||
.and_then(|q| q.as_str())
|
||||
.map(|q| seen.insert(q.to_string()))
|
||||
.unwrap_or(false)
|
||||
});
|
||||
obj.insert("bind_targets".into(), json!(targets));
|
||||
if bind_required {
|
||||
let mut reasons = obj
|
||||
.get("reason")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!([]));
|
||||
if let Some(arr) = reasons.as_array_mut() {
|
||||
arr.push(json!(
|
||||
"scores are discovery-only; bind SymbolId/handle before execute (G02/G07)"
|
||||
));
|
||||
}
|
||||
obj.insert("reason".into(), reasons);
|
||||
}
|
||||
}
|
||||
Ok(pf)
|
||||
}
|
||||
|
||||
/// M04: discover → inspect → bind → execute → record (actual reuse, not n-gram REUSE).
|
||||
///
|
||||
/// Requires host_query `mlsp`. Returns trace with artifact_id, digests, inputs, result.
|
||||
/// Does **not** promote skills — callers pass `candidate_only=true` to skip learn/promote.
|
||||
pub fn eng_bind_execute(args: &Value) -> Result<Value, String> {
|
||||
let project = args
|
||||
.get("project")
|
||||
.or_else(|| args.get("root"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(".");
|
||||
let policy = args
|
||||
.get("policy")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("pure");
|
||||
let inputs = args.get("inputs").cloned().unwrap_or(json!({}));
|
||||
let query = if let Some(q) = args.get("artifact_id").or_else(|| args.get("query")).and_then(|v| v.as_str()) {
|
||||
q.to_string()
|
||||
} else {
|
||||
// Fall back to first bind_target from eng_reuse.
|
||||
let reuse = eng_reuse(args)?;
|
||||
let targets = reuse
|
||||
.get("bind_targets")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
targets
|
||||
.first()
|
||||
.and_then(|t| t.get("query").and_then(|q| q.as_str()))
|
||||
.ok_or_else(|| {
|
||||
"eng_bind_execute: no artifact_id/query and eng_reuse returned no bind_targets"
|
||||
.to_string()
|
||||
})?
|
||||
.to_string()
|
||||
};
|
||||
|
||||
let inspect = crate::host_query_json_args(
|
||||
"mlsp",
|
||||
&json!({"method":"artifact.inspect","params":{"project":project,"artifact_id":&query}}),
|
||||
)?;
|
||||
if inspect.get("ok") != Some(&json!(true)) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"stage": "inspect",
|
||||
"error": inspect.get("error").cloned().unwrap_or(json!("inspect failed")),
|
||||
"query": query,
|
||||
}));
|
||||
}
|
||||
let artifact_id = inspect["result"]["artifact_id"]
|
||||
.as_str()
|
||||
.unwrap_or(&query)
|
||||
.to_string();
|
||||
|
||||
let bound = crate::host_query_json_args(
|
||||
"mlsp",
|
||||
&json!({"method":"artifact.bind","params":{"project":project,"artifact_id":&artifact_id,"policy":policy}}),
|
||||
)?;
|
||||
if bound.get("ok") != Some(&json!(true)) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"stage": "bind",
|
||||
"error": bound.get("error").cloned().unwrap_or(json!("bind failed")),
|
||||
"artifact_id": artifact_id,
|
||||
"discovery_only": false,
|
||||
}));
|
||||
}
|
||||
let handle = bound["result"]["handle"].clone();
|
||||
|
||||
let executed = crate::host_query_json_args(
|
||||
"mlsp",
|
||||
&json!({"method":"artifact.execute","params":{"handle":&handle,"inputs":inputs}}),
|
||||
)?;
|
||||
if executed.get("ok") != Some(&json!(true)) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"stage": "execute",
|
||||
"error": executed.get("error").cloned().unwrap_or(json!("execute failed")),
|
||||
"artifact_id": artifact_id,
|
||||
"handle": handle,
|
||||
}));
|
||||
}
|
||||
|
||||
// Record only after successful bind+execute (actual reuse).
|
||||
let _ = eng_record(&json!({
|
||||
"decision": "REUSE",
|
||||
"root": project,
|
||||
"marqdo_dir": args.get("marqdo_dir"),
|
||||
}));
|
||||
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"stage": "executed",
|
||||
"discovery_only": false,
|
||||
"bind_required": false,
|
||||
"actual_reuse": true,
|
||||
"artifact_id": artifact_id,
|
||||
"symbol_id": handle.get("symbol_id"),
|
||||
"digest": handle.get("digest"),
|
||||
"evidence_digest": handle.get("evidence_digest"),
|
||||
"closure_digest": handle.get("closure_digest"),
|
||||
"policy": policy,
|
||||
"inputs": inputs,
|
||||
"result": executed.get("result"),
|
||||
"candidate_only": args.get("candidate_only").and_then(|v| v.as_bool()).unwrap_or(false),
|
||||
"trace": {
|
||||
"artifact_id": artifact_id,
|
||||
"symbol_id": handle.get("symbol_id"),
|
||||
"digest": handle.get("digest"),
|
||||
"inputs": inputs,
|
||||
"stdout": executed["result"]["stdout"],
|
||||
"value": executed["result"]["result"],
|
||||
},
|
||||
}))
|
||||
}
|
||||
|
||||
/// Gate promote/learn: when `candidate_only` or may_bind=false, refuse promotion.
|
||||
pub fn eng_publish_gate(args: &Value) -> Result<Value, String> {
|
||||
let candidate_only = matches!(args.get("candidate_only"), Some(Value::Bool(true)));
|
||||
let product_ok = !matches!(args.get("product_gate_ok"), Some(Value::Bool(false)));
|
||||
let project = args
|
||||
.get("project")
|
||||
.or_else(|| args.get("root"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(".");
|
||||
let policy = args
|
||||
.get("policy")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("business");
|
||||
let artifact = args
|
||||
.get("artifact_id")
|
||||
.or_else(|| args.get("query"))
|
||||
.and_then(|v| v.as_str());
|
||||
|
||||
if candidate_only {
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"may_promote": false,
|
||||
"reason": "candidate_only — framework must not promote",
|
||||
}));
|
||||
}
|
||||
if !product_ok {
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"may_promote": false,
|
||||
"reason": "product gate failed — framework must not promote",
|
||||
}));
|
||||
}
|
||||
if let Some(id) = artifact {
|
||||
let adj = crate::host_query_json_args(
|
||||
"mlsp",
|
||||
&json!({"method":"artifact.verify","params":{"project":project,"artifact_id":id,"policy":policy}}),
|
||||
)?;
|
||||
let may_bind = adj
|
||||
.pointer("/result/may_bind")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
return Ok(json!({
|
||||
"ok": true,
|
||||
"may_promote": may_bind,
|
||||
"verification": adj.get("result"),
|
||||
"reason": if may_bind { "verification allows bind" } else { "verification denied bind" },
|
||||
}));
|
||||
}
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"may_promote": product_ok,
|
||||
"reason": "no artifact_id; deferred to product_gate_ok",
|
||||
}))
|
||||
}
|
||||
|
||||
/// `agent_eng_record` — append reuse metrics under `.marqdo/agent/episodes`.
|
||||
pub fn eng_record(args: &Value) -> Result<Value, String> {
|
||||
let decision = arg_text(args, "decision")?;
|
||||
|
||||
@@ -0,0 +1,274 @@
|
||||
//! AgentErrorV1 — classified errors and bounded recovery decisions (M-A1).
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
/// Build an AgentErrorV1 document from caller fields (compat: missing → defaults).
|
||||
pub fn make_error(args: &Value) -> Result<Value, String> {
|
||||
let code = args
|
||||
.get("code")
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.ok_or_else(|| "missing text `code`".to_string())?;
|
||||
let category = args
|
||||
.get("category")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or_else(|| infer_category(code));
|
||||
let stage = args
|
||||
.get("stage")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("agent");
|
||||
let message = args
|
||||
.get("message")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(code);
|
||||
let retryable = args
|
||||
.get("retryable")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or_else(|| default_retryable(category));
|
||||
let effect_state = args
|
||||
.get("effect_state")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("none");
|
||||
let recovery_action = args
|
||||
.get("recovery_action")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or_else(|| default_recovery(category, effect_state));
|
||||
|
||||
let mut out = json!({
|
||||
"schema": "AgentErrorV1",
|
||||
"schema_version": 1,
|
||||
"code": code,
|
||||
"category": category,
|
||||
"stage": stage,
|
||||
"message": message,
|
||||
"retryable": retryable,
|
||||
"effect_state": effect_state,
|
||||
"recovery_action": recovery_action,
|
||||
});
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
if let Some(v) = args.get("retry_after_ms") {
|
||||
obj.insert("retry_after_ms".into(), v.clone());
|
||||
} else {
|
||||
obj.insert("retry_after_ms".into(), Value::Null);
|
||||
}
|
||||
for key in ["call_id", "details_ref", "fingerprint"] {
|
||||
if let Some(v) = args.get(key) {
|
||||
if !v.is_null() {
|
||||
obj.insert(key.into(), v.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Decide whether a step may auto-retry (read-only / no-effect only in M-A1).
|
||||
///
|
||||
/// Returns `{ok, action, reason, attempts_used, max_attempts}`.
|
||||
pub fn decide_retry(args: &Value) -> Result<Value, String> {
|
||||
let error = args
|
||||
.get("error")
|
||||
.cloned()
|
||||
.ok_or_else(|| "missing map `error`".to_string())?;
|
||||
let attempts = args
|
||||
.get("attempts")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(0) as u32;
|
||||
let max_transport = args
|
||||
.get("max_transport_attempts")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(3) as u32; // 1 initial + 2 retries
|
||||
let max_arg_repair = args
|
||||
.get("max_arg_repairs")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(1) as u32;
|
||||
let effect_class = args
|
||||
.get("effect_class")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("read");
|
||||
let cancelled = args
|
||||
.get("cancelled")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
let fingerprint_repeats = args
|
||||
.get("fingerprint_repeats")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(0) as u32;
|
||||
|
||||
if cancelled {
|
||||
return Ok(decision("stop", "cancelled", attempts, max_transport));
|
||||
}
|
||||
|
||||
let category = error
|
||||
.get("category")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("unknown");
|
||||
let retryable = error
|
||||
.get("retryable")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
let effect_state = error
|
||||
.get("effect_state")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("none");
|
||||
let recovery = error
|
||||
.get("recovery_action")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("none");
|
||||
|
||||
// Write / unknown effect: never auto-retry in M-A1 (ledger required).
|
||||
if effect_class != "read" || matches!(effect_state, "unknown" | "started" | "committed") {
|
||||
return Ok(decision(
|
||||
"manual_review",
|
||||
"write_or_unknown_effect_requires_ledger",
|
||||
attempts,
|
||||
max_transport,
|
||||
));
|
||||
}
|
||||
|
||||
if fingerprint_repeats >= 2 {
|
||||
return Ok(decision(
|
||||
"stop",
|
||||
"error_fingerprint_loop",
|
||||
attempts,
|
||||
max_transport,
|
||||
));
|
||||
}
|
||||
|
||||
match category {
|
||||
"transient" if retryable && attempts + 1 < max_transport => Ok(decision(
|
||||
"retry_step",
|
||||
"transient_read_safe",
|
||||
attempts,
|
||||
max_transport,
|
||||
)),
|
||||
"invalid_arguments" if recovery == "repair_args" && attempts < max_arg_repair => Ok(
|
||||
decision("repair_args", "one_arg_repair", attempts, max_arg_repair),
|
||||
),
|
||||
"auth" | "forbidden" | "policy_denied" => Ok(decision(
|
||||
"needs_input",
|
||||
category,
|
||||
attempts,
|
||||
max_transport,
|
||||
)),
|
||||
"budget_exhausted" => Ok(decision("partial", category, attempts, max_transport)),
|
||||
"cancelled" => Ok(decision("stop", "cancelled", attempts, max_transport)),
|
||||
"unknown_effect" | "worker_crash" => Ok(decision(
|
||||
"manual_review",
|
||||
category,
|
||||
attempts,
|
||||
max_transport,
|
||||
)),
|
||||
_ => Ok(decision("stop", "not_auto_retryable", attempts, max_transport)),
|
||||
}
|
||||
}
|
||||
|
||||
fn decision(action: &str, reason: &str, attempts: u32, max_attempts: u32) -> Value {
|
||||
json!({
|
||||
"ok": true,
|
||||
"schema": "AgentRetryDecisionV1",
|
||||
"schema_version": 1,
|
||||
"action": action,
|
||||
"reason": reason,
|
||||
"attempts_used": attempts,
|
||||
"max_attempts": max_attempts,
|
||||
"auto": matches!(action, "retry_step" | "repair_args"),
|
||||
})
|
||||
}
|
||||
|
||||
fn infer_category(code: &str) -> &'static str {
|
||||
let c = code.to_ascii_lowercase();
|
||||
if c.contains("timeout") || c.contains("429") || c.contains("5xx") || c.contains("connect") {
|
||||
"transient"
|
||||
} else if c.contains("schema") || c.contains("argument") {
|
||||
"invalid_arguments"
|
||||
} else if c.contains("auth") {
|
||||
"auth"
|
||||
} else if c.contains("forbid") {
|
||||
"forbidden"
|
||||
} else if c.contains("policy") {
|
||||
"policy_denied"
|
||||
} else if c.contains("budget") {
|
||||
"budget_exhausted"
|
||||
} else if c.contains("cancel") {
|
||||
"cancelled"
|
||||
} else if c.contains("conflict") || c.contains("stale") {
|
||||
"conflict"
|
||||
} else if c.contains("effect") || c.contains("crash") {
|
||||
"unknown_effect"
|
||||
} else {
|
||||
"unknown"
|
||||
}
|
||||
}
|
||||
|
||||
fn default_retryable(category: &str) -> bool {
|
||||
matches!(category, "transient")
|
||||
}
|
||||
|
||||
fn default_recovery(category: &str, effect_state: &str) -> &'static str {
|
||||
if effect_state == "unknown" {
|
||||
return "reconcile";
|
||||
}
|
||||
match category {
|
||||
"transient" => "retry_step",
|
||||
"invalid_arguments" => "repair_args",
|
||||
"auth" | "forbidden" | "policy_denied" | "needs_input" => "needs_input",
|
||||
"budget_exhausted" => "return_partial",
|
||||
"cancelled" => "stop",
|
||||
"unknown_effect" | "worker_crash" => "manual_review",
|
||||
_ => "stop",
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn make_error_defaults() {
|
||||
let err = make_error(&json!({"code": "tool.timeout", "message": "t"})).unwrap();
|
||||
assert_eq!(err["schema"], "AgentErrorV1");
|
||||
assert_eq!(err["category"], "transient");
|
||||
assert_eq!(err["retryable"], true);
|
||||
assert_eq!(err["recovery_action"], "retry_step");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_transient_retries_once() {
|
||||
let err = make_error(&json!({"code": "tool.timeout"})).unwrap();
|
||||
let d = decide_retry(&json!({
|
||||
"error": err,
|
||||
"attempts": 0,
|
||||
"effect_class": "read"
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(d["action"], "retry_step");
|
||||
assert_eq!(d["auto"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn write_never_auto_retries() {
|
||||
let err = make_error(&json!({"code": "tool.timeout"})).unwrap();
|
||||
let d = decide_retry(&json!({
|
||||
"error": err,
|
||||
"attempts": 0,
|
||||
"effect_class": "write"
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(d["action"], "manual_review");
|
||||
assert_eq!(d["auto"], false);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fingerprint_loop_stops() {
|
||||
let err = make_error(&json!({"code": "tool.timeout"})).unwrap();
|
||||
let d = decide_retry(&json!({
|
||||
"error": err,
|
||||
"attempts": 0,
|
||||
"effect_class": "read",
|
||||
"fingerprint_repeats": 2
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(d["action"], "stop");
|
||||
}
|
||||
}
|
||||
+154
-1
@@ -1,14 +1,19 @@
|
||||
//! Agent plugin (C ABI v2): layout helpers + session bag + context via host_query + agent-kb + v2 memory + adaptive route.
|
||||
|
||||
mod context_pack;
|
||||
mod corpus;
|
||||
mod effects;
|
||||
mod eng;
|
||||
mod error;
|
||||
mod kb;
|
||||
mod mcp_client;
|
||||
mod mcp_server;
|
||||
mod memory;
|
||||
mod output;
|
||||
mod resume;
|
||||
mod route;
|
||||
mod sandbox;
|
||||
mod tools;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::ffi::CString;
|
||||
@@ -667,7 +672,7 @@ unsafe extern "C" fn agent_marqdo_skill(
|
||||
}
|
||||
}
|
||||
|
||||
fn tool_names(tools: &serde_json::Value) -> Vec<String> {
|
||||
pub(crate) fn tool_names(tools: &serde_json::Value) -> Vec<String> {
|
||||
let mut out = Vec::new();
|
||||
let rows = match tools {
|
||||
serde_json::Value::Array(a) => a.as_slice(),
|
||||
@@ -829,8 +834,90 @@ kb_ffi!(agent_resume_save, resume::resume_save, "agent_resume_save");
|
||||
kb_ffi!(agent_resume_load, resume::resume_load, "agent_resume_load");
|
||||
kb_ffi!(agent_resume_clear, resume::resume_clear, "agent_resume_clear");
|
||||
kb_ffi!(agent_resume_list, resume::resume_list, "agent_resume_list");
|
||||
kb_ffi!(agent_error_make, error::make_error, "agent_error_make");
|
||||
kb_ffi!(agent_error_decide_retry, error::decide_retry, "agent_error_decide_retry");
|
||||
kb_ffi!(agent_tools_register, tools::tools_register, "agent_tools_register");
|
||||
kb_ffi!(agent_tools_clear, tools::tools_clear, "agent_tools_clear");
|
||||
kb_ffi!(agent_tools_search, tools::tools_search, "agent_tools_search");
|
||||
kb_ffi!(agent_tools_describe, tools::tools_describe, "agent_tools_describe");
|
||||
kb_ffi!(agent_tools_prepare_call, tools::tools_prepare_call, "agent_tools_prepare_call");
|
||||
kb_ffi!(agent_tools_wrap_result, tools::tools_wrap_result, "agent_tools_wrap_result");
|
||||
kb_ffi!(
|
||||
agent_tools_args_digest,
|
||||
tools::tools_args_digest,
|
||||
"agent_tools_args_digest"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_tools_needs_ledger,
|
||||
tools::tools_needs_ledger,
|
||||
"agent_tools_needs_ledger"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_tools_register_eng_defaults,
|
||||
tools::tools_register_eng_defaults,
|
||||
"agent_tools_register_eng_defaults"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_tools_seed_allowlist,
|
||||
tools::seed_from_allowlist_args,
|
||||
"agent_tools_seed_allowlist"
|
||||
);
|
||||
kb_ffi!(agent_output_build, output::output_build, "agent_output_build");
|
||||
kb_ffi!(
|
||||
agent_output_apply_policy,
|
||||
output::output_apply_policy,
|
||||
"agent_output_apply_policy"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_output_channel_event,
|
||||
output::output_channel_event,
|
||||
"agent_output_channel_event"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_output_project_conclusion,
|
||||
output::output_project_conclusion,
|
||||
"agent_output_project_conclusion"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_context_pack_build,
|
||||
context_pack::context_pack_build,
|
||||
"agent_context_pack_build"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_context_pack_compact,
|
||||
context_pack::context_pack_compact,
|
||||
"agent_context_pack_compact"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_effects_prepare,
|
||||
effects::effects_prepare,
|
||||
"agent_effects_prepare"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_effects_transition,
|
||||
effects::effects_transition,
|
||||
"agent_effects_transition"
|
||||
);
|
||||
kb_ffi!(agent_effects_get, effects::effects_get, "agent_effects_get");
|
||||
kb_ffi!(
|
||||
agent_effects_list,
|
||||
effects::effects_list,
|
||||
"agent_effects_list"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_effects_watermark,
|
||||
effects::effects_watermark,
|
||||
"agent_effects_watermark"
|
||||
);
|
||||
kb_ffi!(
|
||||
agent_effects_reconcile,
|
||||
effects::effects_reconcile,
|
||||
"agent_effects_reconcile"
|
||||
);
|
||||
kb_ffi!(agent_eng_preflight, eng::eng_preflight, "agent_eng_preflight");
|
||||
kb_ffi!(agent_eng_reuse, eng::eng_reuse, "agent_eng_reuse");
|
||||
kb_ffi!(agent_eng_bind_execute, eng::eng_bind_execute, "agent_eng_bind_execute");
|
||||
kb_ffi!(agent_eng_publish_gate, eng::eng_publish_gate, "agent_eng_publish_gate");
|
||||
kb_ffi!(agent_eng_record, eng::eng_record, "agent_eng_record");
|
||||
|
||||
#[no_mangle]
|
||||
@@ -876,6 +963,70 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
|
||||
("agent_resume_load", "id,resume_dir", agent_resume_load),
|
||||
("agent_resume_clear", "id,resume_dir", agent_resume_clear),
|
||||
("agent_resume_list", "resume_dir", agent_resume_list),
|
||||
("agent_error_make", "code,message,category", agent_error_make),
|
||||
("agent_error_decide_retry", "error,attempts,effect_class", agent_error_decide_retry),
|
||||
("agent_tools_register", "spec,specs", agent_tools_register),
|
||||
("agent_tools_clear", "", agent_tools_clear),
|
||||
("agent_tools_search", "query,tools", agent_tools_search),
|
||||
("agent_tools_describe", "id,tools", agent_tools_describe),
|
||||
("agent_tools_prepare_call", "tool_id,arguments,tools", agent_tools_prepare_call),
|
||||
("agent_tools_wrap_result", "ok,status,summary", agent_tools_wrap_result),
|
||||
("agent_tools_args_digest", "arguments", agent_tools_args_digest),
|
||||
("agent_tools_needs_ledger", "effect", agent_tools_needs_ledger),
|
||||
(
|
||||
"agent_tools_register_eng_defaults",
|
||||
"",
|
||||
agent_tools_register_eng_defaults,
|
||||
),
|
||||
("agent_tools_seed_allowlist", "tools", agent_tools_seed_allowlist),
|
||||
("agent_output_build", "status,text,validation", agent_output_build),
|
||||
(
|
||||
"agent_output_apply_policy",
|
||||
"status,text,policy",
|
||||
agent_output_apply_policy,
|
||||
),
|
||||
(
|
||||
"agent_output_channel_event",
|
||||
"channel,delta",
|
||||
agent_output_channel_event,
|
||||
),
|
||||
(
|
||||
"agent_output_project_conclusion",
|
||||
"document",
|
||||
agent_output_project_conclusion,
|
||||
),
|
||||
(
|
||||
"agent_context_pack_build",
|
||||
"items,budget_tokens",
|
||||
agent_context_pack_build,
|
||||
),
|
||||
(
|
||||
"agent_context_pack_compact",
|
||||
"items,budget_tokens",
|
||||
agent_context_pack_compact,
|
||||
),
|
||||
(
|
||||
"agent_effects_prepare",
|
||||
"operation_id,idempotency_key,args_digest",
|
||||
agent_effects_prepare,
|
||||
),
|
||||
(
|
||||
"agent_effects_transition",
|
||||
"operation_id,state",
|
||||
agent_effects_transition,
|
||||
),
|
||||
("agent_effects_get", "operation_id", agent_effects_get),
|
||||
("agent_effects_list", "ledger_dir", agent_effects_list),
|
||||
(
|
||||
"agent_effects_watermark",
|
||||
"ledger_dir",
|
||||
agent_effects_watermark,
|
||||
),
|
||||
(
|
||||
"agent_effects_reconcile",
|
||||
"operation_id,outcome",
|
||||
agent_effects_reconcile,
|
||||
),
|
||||
("agent_kb_add_alias", "kb_dir,slug,alias", agent_kb_add_alias),
|
||||
("agent_kb_promote", "kb_dir,goal,workbook", agent_kb_promote),
|
||||
("agent_kb_quarantine", "kb_dir,slug", agent_kb_quarantine),
|
||||
@@ -900,6 +1051,8 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
|
||||
("agent_maybe_compile_policy", "memory_dir,min_evidence", agent_maybe_compile_policy),
|
||||
("agent_eng_preflight", "task", agent_eng_preflight),
|
||||
("agent_eng_reuse", "task", agent_eng_reuse),
|
||||
("agent_eng_bind_execute", "project,query,policy", agent_eng_bind_execute),
|
||||
("agent_eng_publish_gate", "project,artifact_id,candidate_only", agent_eng_publish_gate),
|
||||
("agent_eng_record", "decision", agent_eng_record),
|
||||
("sandbox_open", "", sandbox::sandbox_open),
|
||||
("sandbox_write", "", sandbox::sandbox_write),
|
||||
|
||||
@@ -0,0 +1,289 @@
|
||||
//! AgentOutputV1 + ResponsePolicy (M-A4).
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
/// Normalize / build an AgentOutputV1 document.
|
||||
pub fn output_build(args: &Value) -> Result<Value, String> {
|
||||
let status = opt_text(args, "status").unwrap_or("succeeded");
|
||||
if !matches!(
|
||||
status,
|
||||
"succeeded" | "partial" | "failed" | "needs_input" | "cancelled"
|
||||
) {
|
||||
return Err(format!("invalid status={status}"));
|
||||
}
|
||||
let answer = args.get("answer").cloned().unwrap_or_else(|| {
|
||||
json!({
|
||||
"format": opt_text(args, "format").unwrap_or("markdown"),
|
||||
"text": opt_text(args, "text").unwrap_or(""),
|
||||
})
|
||||
});
|
||||
let validation = args.get("validation").cloned().unwrap_or_else(|| {
|
||||
json!({
|
||||
"status": "not_run",
|
||||
"report_ref": Value::Null
|
||||
})
|
||||
});
|
||||
let vstatus = validation
|
||||
.get("status")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("not_run");
|
||||
if !matches!(
|
||||
vstatus,
|
||||
"passed" | "failed" | "not_applicable" | "not_run"
|
||||
) {
|
||||
return Err(format!("invalid validation.status={vstatus}"));
|
||||
}
|
||||
// Never claim passed without an explicit report when status says passed.
|
||||
if vstatus == "passed" && validation.get("report_ref").map(|v| v.is_null()).unwrap_or(true) {
|
||||
if args.get("allow_passed_without_report").and_then(|v| v.as_bool()) != Some(true) {
|
||||
return Err("validation.status=passed requires report_ref".into());
|
||||
}
|
||||
}
|
||||
|
||||
let finish = opt_text(args, "finish_reason").unwrap_or(match status {
|
||||
"succeeded" => "completed",
|
||||
"partial" => "length_limited",
|
||||
"cancelled" => "cancelled",
|
||||
"needs_input" => "needs_input",
|
||||
_ => "error",
|
||||
});
|
||||
|
||||
Ok(json!({
|
||||
"schema": "AgentOutputV1",
|
||||
"schema_version": 1,
|
||||
"status": status,
|
||||
"answer": answer,
|
||||
"artifacts": args.get("artifacts").cloned().unwrap_or_else(|| json!([])),
|
||||
"citations": args.get("citations").cloned().unwrap_or_else(|| json!([])),
|
||||
"validation": validation,
|
||||
"finish_reason": finish,
|
||||
"recovery_options": args.get("recovery_options").cloned().unwrap_or_else(|| json!([])),
|
||||
"channels": args.get("channels").cloned().unwrap_or_else(|| json!({
|
||||
"answer": true,
|
||||
"artifact": true,
|
||||
"progress": true,
|
||||
"diagnostic": true
|
||||
})),
|
||||
}))
|
||||
}
|
||||
|
||||
/// Apply ResponsePolicy budgets to an answer text → possibly partial output.
|
||||
pub fn output_apply_policy(args: &Value) -> Result<Value, String> {
|
||||
let mut out = if args.get("schema").and_then(|v| v.as_str()) == Some("AgentOutputV1") {
|
||||
args.clone()
|
||||
} else {
|
||||
output_build(args)?
|
||||
};
|
||||
let policy = args.get("policy").cloned().unwrap_or(json!({}));
|
||||
let detail = policy
|
||||
.get("detail")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("standard");
|
||||
let max_chars = match policy.get("max_chars").and_then(|v| v.as_u64()) {
|
||||
Some(n) if n > 0 => n as usize,
|
||||
_ => match detail {
|
||||
"concise" => 250,
|
||||
"detailed" => 8_000,
|
||||
_ => 700,
|
||||
},
|
||||
};
|
||||
let max_tokens = policy
|
||||
.get("max_output_tokens")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(0) as usize;
|
||||
let artifact_allowed = policy
|
||||
.get("artifact_allowed")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(true);
|
||||
|
||||
let text = out
|
||||
.pointer("/answer/text")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
// Soft char budget (CJK-aware enough: count Unicode scalars).
|
||||
let chars: Vec<char> = text.chars().collect();
|
||||
let mut finish = out
|
||||
.get("finish_reason")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("completed")
|
||||
.to_string();
|
||||
let mut status = out
|
||||
.get("status")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("succeeded")
|
||||
.to_string();
|
||||
let mut answer_text = text.clone();
|
||||
if chars.len() > max_chars {
|
||||
answer_text = chars.into_iter().take(max_chars).collect();
|
||||
// Prefer not to cut mid-code fence if possible — soft trim at last newline.
|
||||
if let Some(pos) = answer_text.rfind('\n') {
|
||||
if pos > max_chars / 2 {
|
||||
answer_text.truncate(pos);
|
||||
}
|
||||
}
|
||||
finish = "length_limited".into();
|
||||
if status == "succeeded" {
|
||||
status = "partial".into();
|
||||
}
|
||||
}
|
||||
// Token budget is advisory here (chars/4 approx) when max_output_tokens set.
|
||||
if max_tokens > 0 {
|
||||
let approx = answer_text.chars().count().div_ceil(2); // rough CJK token estimate
|
||||
if approx > max_tokens {
|
||||
let keep = max_tokens.saturating_mul(2);
|
||||
answer_text = answer_text.chars().take(keep).collect();
|
||||
finish = "length_limited".into();
|
||||
status = "partial".into();
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("status".into(), json!(status));
|
||||
obj.insert("finish_reason".into(), json!(finish));
|
||||
if let Some(ans) = obj.get_mut("answer").and_then(|v| v.as_object_mut()) {
|
||||
ans.insert("text".into(), json!(answer_text));
|
||||
}
|
||||
if !artifact_allowed {
|
||||
obj.insert("artifacts".into(), json!([]));
|
||||
}
|
||||
obj.insert(
|
||||
"response_policy".into(),
|
||||
json!({
|
||||
"detail": detail,
|
||||
"max_chars": max_chars,
|
||||
"max_output_tokens": max_tokens,
|
||||
"artifact_allowed": artifact_allowed,
|
||||
}),
|
||||
);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Split a channel event. Only `answer` is meant for chat body append.
|
||||
pub fn output_channel_event(args: &Value) -> Result<Value, String> {
|
||||
let channel = opt_text(args, "channel").ok_or_else(|| "missing channel".to_string())?;
|
||||
if !matches!(channel, "answer" | "artifact" | "progress" | "diagnostic") {
|
||||
return Err(format!("invalid channel={channel}"));
|
||||
}
|
||||
let delta = opt_text(args, "delta").unwrap_or("");
|
||||
Ok(json!({
|
||||
"schema": "AgentChannelEventV1",
|
||||
"schema_version": 1,
|
||||
"channel": channel,
|
||||
"delta": delta,
|
||||
"append_to_chat": channel == "answer",
|
||||
"resource_id": args.get("resource_id").cloned().unwrap_or(Value::Null),
|
||||
"meta": args.get("meta").cloned().unwrap_or(json!({})),
|
||||
}))
|
||||
}
|
||||
|
||||
/// Project `# 结论` (or first non-empty paragraph) from a marqdo document for chat.
|
||||
pub fn output_project_conclusion(args: &Value) -> Result<Value, String> {
|
||||
let doc = opt_text(args, "document")
|
||||
.or_else(|| opt_text(args, "text"))
|
||||
.unwrap_or("");
|
||||
let mut in_conclusion = false;
|
||||
let mut buf = String::new();
|
||||
for line in doc.lines() {
|
||||
let t = line.trim();
|
||||
if t == "# 结论" || t.eq_ignore_ascii_case("# conclusion") {
|
||||
in_conclusion = true;
|
||||
continue;
|
||||
}
|
||||
if in_conclusion {
|
||||
if t.starts_with("# ") {
|
||||
break;
|
||||
}
|
||||
if t.is_empty() {
|
||||
if !buf.is_empty() {
|
||||
buf.push('\n');
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if !buf.is_empty() {
|
||||
buf.push('\n');
|
||||
}
|
||||
buf.push_str(t);
|
||||
}
|
||||
}
|
||||
if buf.trim().is_empty() {
|
||||
// Fallback: first non-frontmatter paragraph.
|
||||
let mut past_fm = !doc.starts_with("---");
|
||||
let mut fm_done = past_fm;
|
||||
for line in doc.lines() {
|
||||
if !fm_done {
|
||||
if line.trim() == "---" {
|
||||
if past_fm {
|
||||
fm_done = true;
|
||||
} else {
|
||||
past_fm = true;
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
let t = line.trim();
|
||||
if t.is_empty() || t.starts_with('#') || t.starts_with('|') {
|
||||
continue;
|
||||
}
|
||||
buf = t.to_string();
|
||||
break;
|
||||
}
|
||||
}
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"text": buf.trim(),
|
||||
"channel": "answer",
|
||||
"source": "conclusion_projection",
|
||||
}))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn build_and_policy_limits() {
|
||||
let out = output_build(&json!({
|
||||
"status": "succeeded",
|
||||
"text": "短答",
|
||||
"validation": {"status": "not_applicable", "report_ref": null}
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(out["schema"], "AgentOutputV1");
|
||||
assert_eq!(out["answer"]["text"], "短答");
|
||||
|
||||
let long: String = "字".repeat(400);
|
||||
let limited = output_apply_policy(&json!({
|
||||
"status": "succeeded",
|
||||
"text": long,
|
||||
"validation": {"status": "not_applicable"},
|
||||
"policy": {"detail": "concise", "max_chars": 100}
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(limited["status"], "partial");
|
||||
assert_eq!(limited["finish_reason"], "length_limited");
|
||||
assert!(limited["answer"]["text"].as_str().unwrap().chars().count() <= 100);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn channel_answer_only_appends_chat() {
|
||||
let e = output_channel_event(&json!({"channel": "progress", "delta": "检索中"})).unwrap();
|
||||
assert_eq!(e["append_to_chat"], false);
|
||||
let a = output_channel_event(&json!({"channel": "answer", "delta": "hi"})).unwrap();
|
||||
assert_eq!(a["append_to_chat"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn project_conclusion() {
|
||||
let doc = "---\ntitle: t\n---\n\n# 任务\n问\n\n# 结论\n答案是 42。\n\n# main\nprint\n";
|
||||
let p = output_project_conclusion(&json!({"document": doc})).unwrap();
|
||||
assert!(p["text"].as_str().unwrap().contains("42"));
|
||||
}
|
||||
}
|
||||
+348
-16
@@ -1,6 +1,8 @@
|
||||
//! Agent plan/run resume checkpoints under `.marqdo/agent-resume/`.
|
||||
//! M-A1: CheckpointV2 atomic write (temp→fsync→rename); legacy load → manual_review.
|
||||
|
||||
use std::fs;
|
||||
use std::fs::{self, File, OpenOptions};
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
@@ -27,7 +29,9 @@ fn resolve_path(rel: &str) -> PathBuf {
|
||||
}
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
@@ -43,9 +47,19 @@ fn opt_i64(args: &Value, key: &str, default: i64) -> i64 {
|
||||
}
|
||||
}
|
||||
|
||||
fn resume_root(args: &Value) -> PathBuf {
|
||||
/// Host-relative resume roots only — reject absolute client paths (M-A1).
|
||||
fn resume_root(args: &Value) -> Result<PathBuf, String> {
|
||||
let rel = opt_text(args, "resume_dir").unwrap_or(DEFAULT_RESUME_DIR);
|
||||
resolve_path(rel)
|
||||
let p = PathBuf::from(rel);
|
||||
if p.is_absolute() {
|
||||
return Err(
|
||||
"resume_dir must be host-relative; absolute client paths are rejected".into(),
|
||||
);
|
||||
}
|
||||
if rel.contains("..") {
|
||||
return Err("resume_dir must not contain '..'".into());
|
||||
}
|
||||
Ok(resolve_path(rel))
|
||||
}
|
||||
|
||||
fn checkpoint_path(root: &Path, id: &str) -> PathBuf {
|
||||
@@ -70,11 +84,106 @@ fn ensure_dir(root: &Path) -> Result<(), String> {
|
||||
fs::create_dir_all(root).map_err(|e| format!("resume mkdir: {e}"))
|
||||
}
|
||||
|
||||
pub fn resume_save(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
ensure_dir(&root)?;
|
||||
let path = checkpoint_path(&root, id);
|
||||
/// Atomic durable write: unique temp → write → flush/fsync → rename.
|
||||
fn atomic_write_json(path: &Path, doc: &Value) -> Result<(), String> {
|
||||
let parent = path
|
||||
.parent()
|
||||
.ok_or_else(|| "checkpoint path has no parent".to_string())?;
|
||||
ensure_dir(parent)?;
|
||||
let tmp_name = format!(
|
||||
".{}.tmp-{}-{}",
|
||||
path.file_name()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("ckpt.json"),
|
||||
std::process::id(),
|
||||
now_rfc3339().replace(':', "")
|
||||
);
|
||||
let tmp = parent.join(tmp_name);
|
||||
let text = serde_json::to_string_pretty(doc).map_err(|e| e.to_string())?;
|
||||
{
|
||||
let mut f = OpenOptions::new()
|
||||
.create(true)
|
||||
.write(true)
|
||||
.truncate(true)
|
||||
.open(&tmp)
|
||||
.map_err(|e| format!("resume temp open: {e}"))?;
|
||||
f.write_all(text.as_bytes())
|
||||
.map_err(|e| format!("resume temp write: {e}"))?;
|
||||
f.flush().map_err(|e| format!("resume temp flush: {e}"))?;
|
||||
f.sync_all()
|
||||
.map_err(|e| format!("resume temp fsync: {e}"))?;
|
||||
}
|
||||
// Best-effort directory fsync (platform may ignore).
|
||||
if let Ok(dir) = File::open(parent) {
|
||||
let _ = dir.sync_all();
|
||||
}
|
||||
fs::rename(&tmp, path).map_err(|e| {
|
||||
let _ = fs::remove_file(&tmp);
|
||||
format!("resume atomic rename: {e}")
|
||||
})?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn wants_checkpoint_v2(args: &Value) -> bool {
|
||||
matches!(
|
||||
args.get("schema").and_then(|v| v.as_str()),
|
||||
Some("CheckpointV2")
|
||||
) || args.get("schema_version").and_then(|v| v.as_u64()) == Some(2)
|
||||
|| args.get("checkpoint_v2").and_then(|v| v.as_bool()) == Some(true)
|
||||
}
|
||||
|
||||
fn build_checkpoint_v2(args: &Value, id: &str) -> Value {
|
||||
let committed = now_rfc3339();
|
||||
let mut doc = json!({
|
||||
"schema": "CheckpointV2",
|
||||
"schema_version": 2,
|
||||
"checkpoint_id": args.get("checkpoint_id").cloned().unwrap_or_else(|| json!(id)),
|
||||
"logical_run_id": args.get("logical_run_id").cloned().unwrap_or_else(|| json!(id)),
|
||||
"task_id": args.get("task_id").cloned().unwrap_or_else(|| json!(id)),
|
||||
"attempt_id": args.get("attempt_id").cloned().unwrap_or_else(|| json!("attempt_1")),
|
||||
"step_id": args.get("step_id").cloned().unwrap_or(Value::Null),
|
||||
"parent_checkpoint_id": args.get("parent_checkpoint_id").cloned().unwrap_or(Value::Null),
|
||||
"principal_ref": args.get("principal_ref").cloned().unwrap_or(Value::Null),
|
||||
"workspace_id": args.get("workspace_id").cloned().unwrap_or(Value::Null),
|
||||
"session_id": args.get("session_id").cloned().unwrap_or(Value::Null),
|
||||
"completed_steps": args.get("completed_steps").cloned().unwrap_or_else(|| json!([])),
|
||||
"pending_operation": args.get("pending_operation").cloned().unwrap_or(Value::Null),
|
||||
"deadline": args.get("deadline").cloned().unwrap_or(Value::Null),
|
||||
"committed_at": args.get("committed_at").cloned().unwrap_or_else(|| json!(committed)),
|
||||
// Effect watermark — absent/null means no ledger yet (write auto-resume closed).
|
||||
"effect_ledger_watermark": args.get("effect_ledger_watermark").cloned().unwrap_or(Value::Null),
|
||||
"event_seq": args.get("event_seq").cloned().unwrap_or(Value::Null),
|
||||
"manifest_digest": args.get("manifest_digest").cloned().unwrap_or(Value::Null),
|
||||
});
|
||||
if let Some(obj) = doc.as_object_mut() {
|
||||
// Preserve legacy author fields for dual-read.
|
||||
for key in [
|
||||
"goal", "task", "status", "workbook", "events", "route", "result", "round", "meta",
|
||||
"id",
|
||||
] {
|
||||
if let Some(v) = args.get(key) {
|
||||
if !v.is_null() {
|
||||
obj.insert(key.to_string(), v.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
if !obj.contains_key("id") {
|
||||
obj.insert("id".into(), json!(id));
|
||||
}
|
||||
if !obj.contains_key("updated_at") {
|
||||
obj.insert("updated_at".into(), json!(committed.clone()));
|
||||
}
|
||||
if !obj.contains_key("round") {
|
||||
obj.insert("round".into(), json!(opt_i64(args, "round", 0)));
|
||||
}
|
||||
if !obj.contains_key("status") {
|
||||
obj.insert("status".into(), json!("running"));
|
||||
}
|
||||
}
|
||||
doc
|
||||
}
|
||||
|
||||
fn build_legacy_doc(args: &Value, id: &str) -> Value {
|
||||
let mut doc = json!({
|
||||
"id": id,
|
||||
"updated_at": now_rfc3339(),
|
||||
@@ -96,19 +205,84 @@ pub fn resume_save(args: &Value) -> Result<Value, String> {
|
||||
obj.insert("status".into(), json!("running"));
|
||||
}
|
||||
}
|
||||
let text = serde_json::to_string_pretty(&doc).map_err(|e| e.to_string())?;
|
||||
fs::write(&path, text).map_err(|e| format!("resume save: {e}"))?;
|
||||
doc
|
||||
}
|
||||
|
||||
pub fn resume_save(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args)?;
|
||||
ensure_dir(&root)?;
|
||||
let path = checkpoint_path(&root, id);
|
||||
let v2 = wants_checkpoint_v2(args);
|
||||
let doc = if v2 {
|
||||
build_checkpoint_v2(args, id)
|
||||
} else {
|
||||
build_legacy_doc(args, id)
|
||||
};
|
||||
atomic_write_json(&path, &doc)?;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"id": id,
|
||||
"path": path.to_string_lossy(),
|
||||
"updated_at": doc.get("updated_at").cloned().unwrap_or(Value::Null),
|
||||
"schema": doc.get("schema").cloned().unwrap_or(Value::Null),
|
||||
"schema_version": doc.get("schema_version").cloned().unwrap_or(Value::Null),
|
||||
"updated_at": doc.get("updated_at").or_else(|| doc.get("committed_at")).cloned().unwrap_or(Value::Null),
|
||||
"atomic": true,
|
||||
}))
|
||||
}
|
||||
|
||||
fn watermark_allows_write_auto(wm: &Value) -> bool {
|
||||
match wm {
|
||||
Value::Null => false,
|
||||
Value::String(s) => !s.is_empty(),
|
||||
Value::Object(obj) => {
|
||||
// Prefer explicit safety bit from effects_watermark; bare digest objects
|
||||
// without the bit are treated as unsafe (open unknown / pending writes).
|
||||
match obj.get("write_auto_resume_safe").and_then(|v| v.as_bool()) {
|
||||
Some(b) => b,
|
||||
None => obj
|
||||
.get("watermark")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|s| !s.is_empty())
|
||||
.unwrap_or(false)
|
||||
&& obj.get("open_unknown").and_then(|v| v.as_u64()).unwrap_or(1) == 0
|
||||
&& obj.get("pending_writes").and_then(|v| v.as_u64()).unwrap_or(1) == 0,
|
||||
}
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn annotate_recovery(mut doc: Value) -> Value {
|
||||
let is_v2 = doc.get("schema").and_then(|v| v.as_str()) == Some("CheckpointV2")
|
||||
&& doc.get("schema_version").and_then(|v| v.as_u64()) == Some(2);
|
||||
let wm = doc
|
||||
.get("effect_ledger_watermark")
|
||||
.cloned()
|
||||
.unwrap_or(Value::Null);
|
||||
let ledger_safe = watermark_allows_write_auto(&wm);
|
||||
let (recovery_mode, write_auto_resume) = if is_v2 && ledger_safe {
|
||||
("checkpoint_v2", true)
|
||||
} else if is_v2 {
|
||||
// V2 without a safe ledger watermark: read resume OK; write auto-resume closed.
|
||||
("checkpoint_v2_read_only", false)
|
||||
} else {
|
||||
// Legacy: no effect info → manual_review; never auto redo writes.
|
||||
("manual_review", false)
|
||||
};
|
||||
if let Some(obj) = doc.as_object_mut() {
|
||||
obj.insert("recovery_mode".into(), json!(recovery_mode));
|
||||
obj.insert("write_auto_resume".into(), json!(write_auto_resume));
|
||||
if !is_v2 {
|
||||
obj.insert("legacy_adaptation".into(), json!("manual_review"));
|
||||
}
|
||||
}
|
||||
doc
|
||||
}
|
||||
|
||||
pub fn resume_load(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
let root = resume_root(args)?;
|
||||
let path = checkpoint_path(&root, id);
|
||||
if !path.is_file() {
|
||||
return Ok(json!({
|
||||
@@ -119,7 +293,31 @@ pub fn resume_load(args: &Value) -> Result<Value, String> {
|
||||
}));
|
||||
}
|
||||
let text = fs::read_to_string(&path).map_err(|e| format!("resume load: {e}"))?;
|
||||
let mut doc: Value = serde_json::from_str(&text).map_err(|e| format!("resume json: {e}"))?;
|
||||
let parsed: Value = match serde_json::from_str(&text) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"found": true,
|
||||
"id": id,
|
||||
"path": path.to_string_lossy(),
|
||||
"error": {
|
||||
"schema": "AgentErrorV1",
|
||||
"schema_version": 1,
|
||||
"code": "checkpoint.corrupt",
|
||||
"category": "unknown_effect",
|
||||
"stage": "checkpoint",
|
||||
"message": format!("corrupt checkpoint json: {e}"),
|
||||
"retryable": false,
|
||||
"effect_state": "unknown",
|
||||
"recovery_action": "manual_review",
|
||||
},
|
||||
"recovery_mode": "manual_review",
|
||||
"write_auto_resume": false,
|
||||
}));
|
||||
}
|
||||
};
|
||||
let mut doc = annotate_recovery(parsed);
|
||||
if let Some(obj) = doc.as_object_mut() {
|
||||
obj.insert("ok".into(), json!(true));
|
||||
obj.insert("found".into(), json!(true));
|
||||
@@ -130,7 +328,7 @@ pub fn resume_load(args: &Value) -> Result<Value, String> {
|
||||
|
||||
pub fn resume_clear(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
let root = resume_root(args)?;
|
||||
let path = checkpoint_path(&root, id);
|
||||
let existed = path.is_file();
|
||||
if existed {
|
||||
@@ -145,7 +343,7 @@ pub fn resume_clear(args: &Value) -> Result<Value, String> {
|
||||
}
|
||||
|
||||
pub fn resume_list(args: &Value) -> Result<Value, String> {
|
||||
let root = resume_root(args);
|
||||
let root = resume_root(args)?;
|
||||
if !root.is_dir() {
|
||||
return Ok(json!({ "ok": true, "items": [], "resume_dir": root.to_string_lossy() }));
|
||||
}
|
||||
@@ -156,6 +354,14 @@ pub fn resume_list(args: &Value) -> Result<Value, String> {
|
||||
if p.extension().and_then(|e| e.to_str()) != Some("json") {
|
||||
continue;
|
||||
}
|
||||
// Skip leftover temp files from crashed writers.
|
||||
if p.file_name()
|
||||
.and_then(|s| s.to_str())
|
||||
.map(|s| s.starts_with('.'))
|
||||
.unwrap_or(false)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let id = p
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
@@ -179,3 +385,129 @@ pub fn resume_list(args: &Value) -> Result<Value, String> {
|
||||
"resume_dir": root.to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
fn scratch(label: &str) -> PathBuf {
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap()
|
||||
.as_nanos();
|
||||
let dir = std::env::temp_dir().join(format!("marqdo-resume-{label}-{nanos}"));
|
||||
fs::create_dir_all(&dir).unwrap();
|
||||
dir
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn absolute_resume_dir_rejected() {
|
||||
let err = resume_root(&json!({"resume_dir": "/tmp/evil"})).unwrap_err();
|
||||
assert!(err.contains("host-relative"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn atomic_v2_roundtrip_and_legacy_manual_review() {
|
||||
let dir = scratch("v2");
|
||||
// Use relative path via env cwd — chdir into scratch for this test.
|
||||
let prev = std::env::current_dir().unwrap();
|
||||
std::env::set_current_dir(&dir).unwrap();
|
||||
let rel = "ckpts";
|
||||
let saved = resume_save(&json!({
|
||||
"id": "run1",
|
||||
"schema": "CheckpointV2",
|
||||
"schema_version": 2,
|
||||
"goal": "ping",
|
||||
"status": "running",
|
||||
"round": 1,
|
||||
"resume_dir": rel,
|
||||
"attempt_id": "a1",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(saved["ok"], true);
|
||||
assert_eq!(saved["atomic"], true);
|
||||
assert_eq!(saved["schema"], "CheckpointV2");
|
||||
|
||||
let loaded = resume_load(&json!({"id": "run1", "resume_dir": rel})).unwrap();
|
||||
assert_eq!(loaded["found"], true);
|
||||
assert_eq!(loaded["schema"], "CheckpointV2");
|
||||
assert_eq!(loaded["recovery_mode"], "checkpoint_v2_read_only");
|
||||
assert_eq!(loaded["write_auto_resume"], false);
|
||||
|
||||
// Legacy document without schema.
|
||||
let _ = resume_save(&json!({
|
||||
"id": "legacy1",
|
||||
"goal": "pong",
|
||||
"status": "interrupted",
|
||||
"round": 2,
|
||||
"resume_dir": rel,
|
||||
}))
|
||||
.unwrap();
|
||||
let legacy = resume_load(&json!({"id": "legacy1", "resume_dir": rel})).unwrap();
|
||||
assert_eq!(legacy["recovery_mode"], "manual_review");
|
||||
assert_eq!(legacy["write_auto_resume"], false);
|
||||
assert_eq!(legacy["legacy_adaptation"], "manual_review");
|
||||
|
||||
// V2 + safe watermark → write_auto_resume opens.
|
||||
let _ = resume_save(&json!({
|
||||
"id": "run_safe",
|
||||
"schema": "CheckpointV2",
|
||||
"schema_version": 2,
|
||||
"status": "running",
|
||||
"round": 3,
|
||||
"resume_dir": rel,
|
||||
"attempt_id": "a2",
|
||||
"effect_ledger_watermark": {
|
||||
"schema": "EffectLedgerWatermarkV1",
|
||||
"watermark": "sha256-lite:abc",
|
||||
"write_auto_resume_safe": true,
|
||||
"open_unknown": 0,
|
||||
"pending_writes": 0
|
||||
}
|
||||
}))
|
||||
.unwrap();
|
||||
let safe = resume_load(&json!({"id": "run_safe", "resume_dir": rel})).unwrap();
|
||||
assert_eq!(safe["recovery_mode"], "checkpoint_v2");
|
||||
assert_eq!(safe["write_auto_resume"], true);
|
||||
|
||||
// Unsafe watermark (open unknown) keeps write auto-resume closed.
|
||||
let _ = resume_save(&json!({
|
||||
"id": "run_unsafe",
|
||||
"schema": "CheckpointV2",
|
||||
"schema_version": 2,
|
||||
"status": "running",
|
||||
"round": 4,
|
||||
"resume_dir": rel,
|
||||
"attempt_id": "a3",
|
||||
"effect_ledger_watermark": {
|
||||
"watermark": "sha256-lite:def",
|
||||
"write_auto_resume_safe": false,
|
||||
"open_unknown": 1
|
||||
}
|
||||
}))
|
||||
.unwrap();
|
||||
let unsafe_ck = resume_load(&json!({"id": "run_unsafe", "resume_dir": rel})).unwrap();
|
||||
assert_eq!(unsafe_ck["recovery_mode"], "checkpoint_v2_read_only");
|
||||
assert_eq!(unsafe_ck["write_auto_resume"], false);
|
||||
|
||||
std::env::set_current_dir(prev).unwrap();
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn corrupt_checkpoint_reports_error() {
|
||||
let dir = scratch("corrupt");
|
||||
let prev = std::env::current_dir().unwrap();
|
||||
std::env::set_current_dir(&dir).unwrap();
|
||||
fs::create_dir_all("ckpts").unwrap();
|
||||
fs::write("ckpts/bad.json", "{not-json").unwrap();
|
||||
let loaded = resume_load(&json!({"id": "bad", "resume_dir": "ckpts"})).unwrap();
|
||||
assert_eq!(loaded["found"], true);
|
||||
assert_eq!(loaded["ok"], false);
|
||||
assert_eq!(loaded["error"]["code"], "checkpoint.corrupt");
|
||||
assert_eq!(loaded["recovery_mode"], "manual_review");
|
||||
std::env::set_current_dir(prev).unwrap();
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,794 @@
|
||||
//! ToolSpecV1 registry + call validation (M-A2).
|
||||
//! Discovery → schema validate → authorize. Execution of Marqdo `##` stays in subtask;
|
||||
//! this module never evals model-supplied code.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Mutex, OnceLock};
|
||||
|
||||
use serde_json::{json, Map, Value};
|
||||
|
||||
fn registry() -> &'static Mutex<HashMap<String, Value>> {
|
||||
static REG: OnceLock<Mutex<HashMap<String, Value>>> = OnceLock::new();
|
||||
REG.get_or_init(|| Mutex::new(HashMap::new()))
|
||||
}
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
opt_text(args, key).ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
/// Supported JSON Schema subset for tool input (reject unsupported constructs).
|
||||
fn validate_schema_subset(schema: &Value) -> Result<(), String> {
|
||||
let obj = schema
|
||||
.as_object()
|
||||
.ok_or_else(|| "input_schema must be object".to_string())?;
|
||||
let ty = obj
|
||||
.get("type")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("object");
|
||||
if ty != "object" {
|
||||
return Err(format!("unsupported root type={ty}; only object"));
|
||||
}
|
||||
if let Some(props) = obj.get("properties") {
|
||||
let props = props
|
||||
.as_object()
|
||||
.ok_or_else(|| "properties must be object".to_string())?;
|
||||
for (name, prop) in props {
|
||||
let p = prop
|
||||
.as_object()
|
||||
.ok_or_else(|| format!("property {name} must be object"))?;
|
||||
let pty = p.get("type").and_then(|v| v.as_str()).unwrap_or("string");
|
||||
match pty {
|
||||
"string" | "integer" | "number" | "boolean" | "array" | "object" => {}
|
||||
other => {
|
||||
return Err(format!(
|
||||
"unsupported property type {other} for {name}; refuse register"
|
||||
))
|
||||
}
|
||||
}
|
||||
// Reject constructs we do not implement.
|
||||
for banned in ["$ref", "oneOf", "anyOf", "allOf", "not", "if", "then", "else"] {
|
||||
if p.contains_key(banned) {
|
||||
return Err(format!(
|
||||
"unsupported schema keyword {banned} on {name}; refuse register"
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for banned in ["$ref", "oneOf", "anyOf", "allOf", "not", "if", "then", "else"] {
|
||||
if obj.contains_key(banned) {
|
||||
return Err(format!(
|
||||
"unsupported schema keyword {banned}; refuse register"
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn normalize_spec(mut spec: Value) -> Result<Value, String> {
|
||||
let obj = spec
|
||||
.as_object_mut()
|
||||
.ok_or_else(|| "spec must be object".to_string())?;
|
||||
obj.insert("schema".into(), json!("ToolSpecV1"));
|
||||
obj.entry("schema_version".to_string())
|
||||
.or_insert(json!(1));
|
||||
let id = obj
|
||||
.get("id")
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.ok_or_else(|| "ToolSpecV1 requires id".to_string())?
|
||||
.to_string();
|
||||
obj.entry("version".to_string()).or_insert(json!("1"));
|
||||
obj.entry("description".to_string())
|
||||
.or_insert(json!(""));
|
||||
let input = obj
|
||||
.get("input_schema")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!({"type": "object", "properties": {}, "additionalProperties": false}));
|
||||
validate_schema_subset(&input)?;
|
||||
obj.insert("input_schema".into(), input);
|
||||
obj.entry("effect".to_string()).or_insert(json!("read"));
|
||||
let effect = obj.get("effect").and_then(|v| v.as_str()).unwrap_or("read");
|
||||
if !matches!(effect, "read" | "write" | "execute" | "external_write") {
|
||||
return Err(format!("invalid effect={effect}"));
|
||||
}
|
||||
obj.entry("scopes".to_string()).or_insert(json!([]));
|
||||
obj.entry("timeout_ms".to_string()).or_insert(json!(10_000));
|
||||
obj.entry("max_result_bytes".to_string())
|
||||
.or_insert(json!(65_536));
|
||||
obj.entry("idempotency".to_string())
|
||||
.or_insert(json!("safe"));
|
||||
obj.entry("availability".to_string())
|
||||
.or_insert(json!("ready"));
|
||||
let _ = id;
|
||||
Ok(spec)
|
||||
}
|
||||
|
||||
fn validate_args_against_schema(schema: &Value, arguments: &Value) -> Result<(), String> {
|
||||
let args = arguments
|
||||
.as_object()
|
||||
.ok_or_else(|| "arguments must be object".to_string())?;
|
||||
let sch = schema.as_object().ok_or_else(|| "input_schema invalid".to_string())?;
|
||||
let props = sch
|
||||
.get("properties")
|
||||
.and_then(|v| v.as_object())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
let additional = sch
|
||||
.get("additionalProperties")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(true);
|
||||
if !additional {
|
||||
for key in args.keys() {
|
||||
if !props.contains_key(key) {
|
||||
return Err(format!("unknown argument `{key}`"));
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(req) = sch.get("required").and_then(|v| v.as_array()) {
|
||||
for r in req {
|
||||
let name = r.as_str().unwrap_or("");
|
||||
if name.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if !args.contains_key(name) || args.get(name).map(|v| v.is_null()).unwrap_or(true) {
|
||||
return Err(format!("missing required argument `{name}`"));
|
||||
}
|
||||
}
|
||||
}
|
||||
for (key, val) in args {
|
||||
if let Some(prop) = props.get(key) {
|
||||
if let Some(pty) = prop.get("type").and_then(|v| v.as_str()) {
|
||||
let ok = match pty {
|
||||
"string" => val.is_string(),
|
||||
"integer" => val.as_i64().is_some() || val.as_u64().is_some(),
|
||||
"number" => val.as_f64().is_some(),
|
||||
"boolean" => val.is_boolean(),
|
||||
"array" => val.is_array(),
|
||||
"object" => val.is_object(),
|
||||
_ => true,
|
||||
};
|
||||
if !ok {
|
||||
return Err(format!("argument `{key}` type mismatch (want {pty})"));
|
||||
}
|
||||
}
|
||||
if let Some(min) = prop.get("minimum").and_then(|v| v.as_f64()) {
|
||||
if let Some(n) = val.as_f64() {
|
||||
if n < min {
|
||||
return Err(format!("argument `{key}` below minimum {min}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
if let Some(max) = prop.get("maximum").and_then(|v| v.as_f64()) {
|
||||
if let Some(n) = val.as_f64() {
|
||||
if n > max {
|
||||
return Err(format!("argument `{key}` above maximum {max}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn authorized(allowlist: Option<&Value>, tool_id: &str) -> bool {
|
||||
match allowlist {
|
||||
None => true, // registry-only call sites may omit; still need registered spec
|
||||
Some(tools) => crate::tool_names(tools).iter().any(|n| n == tool_id),
|
||||
}
|
||||
}
|
||||
|
||||
/// Register one ToolSpecV1 (or list). Host/trusted only — model cannot invent principal/root.
|
||||
pub fn tools_register(args: &Value) -> Result<Value, String> {
|
||||
let specs = if let Some(arr) = args.get("specs").and_then(|v| v.as_array()) {
|
||||
arr.clone()
|
||||
} else if args.get("spec").is_some() || args.get("id").is_some() {
|
||||
let raw = args.get("spec").cloned().unwrap_or_else(|| args.clone());
|
||||
vec![raw]
|
||||
} else {
|
||||
return Err("tools.register requires spec or specs[]".into());
|
||||
};
|
||||
let mut registered = Vec::new();
|
||||
let mut guard = registry()
|
||||
.lock()
|
||||
.map_err(|_| "tools registry lock poisoned".to_string())?;
|
||||
for raw in specs {
|
||||
let spec = normalize_spec(raw)?;
|
||||
let id = spec
|
||||
.get("id")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap()
|
||||
.to_string();
|
||||
guard.insert(id.clone(), spec);
|
||||
registered.push(id);
|
||||
}
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"registered": registered,
|
||||
"count": registered.len(),
|
||||
}))
|
||||
}
|
||||
|
||||
pub fn tools_clear(args: &Value) -> Result<Value, String> {
|
||||
let _ = args;
|
||||
let mut guard = registry()
|
||||
.lock()
|
||||
.map_err(|_| "tools registry lock poisoned".to_string())?;
|
||||
let n = guard.len();
|
||||
guard.clear();
|
||||
Ok(json!({"ok": true, "cleared": n}))
|
||||
}
|
||||
|
||||
pub fn tools_search(args: &Value) -> Result<Value, String> {
|
||||
let query = opt_text(args, "query").unwrap_or("").to_ascii_lowercase();
|
||||
let allow = args.get("tools");
|
||||
let guard = registry()
|
||||
.lock()
|
||||
.map_err(|_| "tools registry lock poisoned".to_string())?;
|
||||
let mut items = Vec::new();
|
||||
for (id, spec) in guard.iter() {
|
||||
let avail = spec
|
||||
.get("availability")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("ready");
|
||||
let auth = authorized(allow, id);
|
||||
if !auth {
|
||||
// Do not leak unauthorized private connector names in search hits.
|
||||
continue;
|
||||
}
|
||||
if !query.is_empty() {
|
||||
let desc = spec
|
||||
.get("description")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("");
|
||||
if !id.to_ascii_lowercase().contains(&query)
|
||||
&& !desc.to_ascii_lowercase().contains(&query)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
}
|
||||
items.push(json!({
|
||||
"id": id,
|
||||
"version": spec.get("version").cloned().unwrap_or(json!("1")),
|
||||
"description": spec.get("description").cloned().unwrap_or(json!("")),
|
||||
"effect": spec.get("effect").cloned().unwrap_or(json!("read")),
|
||||
"availability": avail,
|
||||
"authorized": true,
|
||||
}));
|
||||
}
|
||||
items.sort_by(|a, b| {
|
||||
let ai = a.get("id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let bi = b.get("id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
ai.cmp(bi)
|
||||
});
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"schema": "ToolSearchResultV1",
|
||||
"schema_version": 1,
|
||||
"items": items,
|
||||
"count": items.len(),
|
||||
}))
|
||||
}
|
||||
|
||||
pub fn tools_describe(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let allow = args.get("tools");
|
||||
if !authorized(allow, id) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"found": false,
|
||||
"authorized": false,
|
||||
"error": {
|
||||
"schema": "AgentErrorV1",
|
||||
"schema_version": 1,
|
||||
"code": "tool.forbidden",
|
||||
"category": "forbidden",
|
||||
"stage": "tool",
|
||||
"message": format!("tool `{id}` not in allowlist"),
|
||||
"retryable": false,
|
||||
"effect_state": "none",
|
||||
"recovery_action": "needs_input",
|
||||
}
|
||||
}));
|
||||
}
|
||||
let guard = registry()
|
||||
.lock()
|
||||
.map_err(|_| "tools registry lock poisoned".to_string())?;
|
||||
match guard.get(id) {
|
||||
Some(spec) => Ok(json!({
|
||||
"ok": true,
|
||||
"found": true,
|
||||
"authorized": true,
|
||||
"spec": spec,
|
||||
})),
|
||||
None => Ok(json!({
|
||||
"ok": false,
|
||||
"found": false,
|
||||
"authorized": true,
|
||||
"error": {
|
||||
"schema": "AgentErrorV1",
|
||||
"schema_version": 1,
|
||||
"code": "tool.not_found",
|
||||
"category": "capability_unavailable",
|
||||
"stage": "tool",
|
||||
"message": format!("tool `{id}` not registered"),
|
||||
"retryable": false,
|
||||
"effect_state": "none",
|
||||
"recovery_action": "stop",
|
||||
}
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
/// Validate + authorize a ToolCallV1. Does not execute side effects.
|
||||
pub fn tools_prepare_call(args: &Value) -> Result<Value, String> {
|
||||
let tool_id = arg_text(args, "tool_id")
|
||||
.or_else(|_| arg_text(args, "id"))
|
||||
.or_else(|_| arg_text(args, "name"))?;
|
||||
let allow = args.get("tools");
|
||||
if !authorized(allow, tool_id) {
|
||||
return Ok(tool_result_err(
|
||||
"tool.forbidden",
|
||||
"forbidden",
|
||||
format!("unauthorized call to `{tool_id}`"),
|
||||
false,
|
||||
));
|
||||
}
|
||||
let guard = registry()
|
||||
.lock()
|
||||
.map_err(|_| "tools registry lock poisoned".to_string())?;
|
||||
let spec = match guard.get(tool_id) {
|
||||
Some(s) => s.clone(),
|
||||
None => {
|
||||
return Ok(tool_result_err(
|
||||
"tool.not_found",
|
||||
"capability_unavailable",
|
||||
format!("tool `{tool_id}` not registered"),
|
||||
false,
|
||||
));
|
||||
}
|
||||
};
|
||||
drop(guard);
|
||||
|
||||
let avail = spec
|
||||
.get("availability")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("ready");
|
||||
if avail != "ready" {
|
||||
return Ok(tool_result_err(
|
||||
"tool.unavailable",
|
||||
"capability_unavailable",
|
||||
format!("tool `{tool_id}` availability={avail}"),
|
||||
false,
|
||||
));
|
||||
}
|
||||
|
||||
let arguments = match args.get("arguments") {
|
||||
None | Some(Value::Null) => json!({}),
|
||||
Some(v) => v.clone(),
|
||||
};
|
||||
let schema = spec
|
||||
.get("input_schema")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!({"type":"object"}));
|
||||
if let Err(e) = validate_args_against_schema(&schema, &arguments) {
|
||||
return Ok(tool_result_err(
|
||||
"tool.invalid_arguments",
|
||||
"invalid_arguments",
|
||||
e,
|
||||
true,
|
||||
));
|
||||
}
|
||||
|
||||
let call_id = opt_text(args, "call_id")
|
||||
.map(|s| s.to_string())
|
||||
.unwrap_or_else(|| format!("call_{}", std::process::id()));
|
||||
let operation_id = opt_text(args, "operation_id")
|
||||
.map(|s| s.to_string())
|
||||
.unwrap_or_else(|| format!("op_{tool_id}_{call_id}"));
|
||||
let version = spec
|
||||
.get("version")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("1");
|
||||
let effect = spec
|
||||
.get("effect")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("read");
|
||||
let digest = args_digest(&arguments);
|
||||
let logical_run = opt_text(args, "logical_run_id").unwrap_or("-");
|
||||
let step = opt_text(args, "step_id").unwrap_or("-");
|
||||
let idempotency_key = opt_text(args, "idempotency_key")
|
||||
.map(|s| s.to_string())
|
||||
.unwrap_or_else(|| format!("{logical_run}|{step}|{tool_id}|{digest}"));
|
||||
let needs_ledger = matches!(effect, "write" | "execute" | "external_write");
|
||||
|
||||
let call = json!({
|
||||
"schema": "ToolCallV1",
|
||||
"schema_version": 1,
|
||||
"call_id": call_id,
|
||||
"tool_id": tool_id,
|
||||
"tool_version": version,
|
||||
"arguments": arguments,
|
||||
"operation_id": operation_id,
|
||||
"parent_span_id": args.get("parent_span_id").cloned().unwrap_or(Value::Null),
|
||||
});
|
||||
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"status": "prepared",
|
||||
"call": call,
|
||||
"spec": {
|
||||
"id": tool_id,
|
||||
"effect": effect,
|
||||
"timeout_ms": spec.get("timeout_ms").cloned().unwrap_or(json!(10000)),
|
||||
"max_result_bytes": spec.get("max_result_bytes").cloned().unwrap_or(json!(65536)),
|
||||
},
|
||||
"args_digest": digest,
|
||||
"idempotency_key": idempotency_key,
|
||||
"needs_ledger": needs_ledger,
|
||||
"summary": format!("prepared authorized call to `{tool_id}`"),
|
||||
}))
|
||||
}
|
||||
|
||||
/// Stable digest of tool arguments for effect ledger idempotency (M-A5a).
|
||||
pub fn tools_args_digest(args: &Value) -> Result<Value, String> {
|
||||
let arguments = args
|
||||
.get("arguments")
|
||||
.cloned()
|
||||
.or_else(|| args.get("value").cloned())
|
||||
.unwrap_or_else(|| json!({}));
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"args_digest": args_digest(&arguments),
|
||||
}))
|
||||
}
|
||||
|
||||
fn args_digest(arguments: &Value) -> String {
|
||||
let text = serde_json::to_string(arguments).unwrap_or_else(|_| "{}".into());
|
||||
format!("sha256-lite:{:x}", fnv1a64(&text))
|
||||
}
|
||||
|
||||
fn fnv1a64(s: &str) -> u64 {
|
||||
let mut h: u64 = 0xcbf29ce484222325;
|
||||
for b in s.as_bytes() {
|
||||
h ^= u64::from(*b);
|
||||
h = h.wrapping_mul(0x100000001b3);
|
||||
}
|
||||
h
|
||||
}
|
||||
|
||||
/// True when effect_class requires EffectLedger before/after execution.
|
||||
pub fn tools_needs_ledger(args: &Value) -> Result<Value, String> {
|
||||
let effect = opt_text(args, "effect")
|
||||
.or_else(|| opt_text(args, "effect_class"))
|
||||
.unwrap_or("read");
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"needs_ledger": matches!(effect, "write" | "execute" | "external_write"),
|
||||
"effect": effect,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Wrap a successful / failed tool execution into ToolResultV1.
|
||||
pub fn tools_wrap_result(args: &Value) -> Result<Value, String> {
|
||||
let ok = args.get("ok").and_then(|v| v.as_bool()).unwrap_or(false);
|
||||
let status = opt_text(args, "status").unwrap_or(if ok { "succeeded" } else { "failed" });
|
||||
let summary = opt_text(args, "summary").unwrap_or("");
|
||||
let mut out = json!({
|
||||
"schema": "ToolResultV1",
|
||||
"schema_version": 1,
|
||||
"ok": ok,
|
||||
"status": status,
|
||||
"summary": summary,
|
||||
"structured_content": args.get("structured_content").cloned().unwrap_or(Value::Null),
|
||||
"resources": args.get("resources").cloned().unwrap_or_else(|| json!([])),
|
||||
"citations": args.get("citations").cloned().unwrap_or_else(|| json!([])),
|
||||
"error": args.get("error").cloned().unwrap_or(Value::Null),
|
||||
"usage": args.get("usage").cloned().unwrap_or(Value::Null),
|
||||
"truncated": args.get("truncated").and_then(|v| v.as_bool()).unwrap_or(false),
|
||||
"next_cursor": args.get("next_cursor").cloned().unwrap_or(Value::Null),
|
||||
"effect_receipt_ref": args.get("effect_receipt_ref").cloned().unwrap_or(Value::Null),
|
||||
});
|
||||
// Truncate oversized summary into artifact hint (do not dump MB into prompt).
|
||||
let max_bytes = args
|
||||
.get("max_result_bytes")
|
||||
.and_then(|v| v.as_u64())
|
||||
.unwrap_or(65_536) as usize;
|
||||
if summary.len() > max_bytes {
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("truncated".into(), json!(true));
|
||||
obj.insert(
|
||||
"summary".into(),
|
||||
json!(format!(
|
||||
"{}… [truncated; use resource handle]",
|
||||
&summary[..max_bytes.min(summary.len())]
|
||||
)),
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Seed the five engineering tools used by qdagent (aliases remain CALL:tool_*).
|
||||
pub fn tools_register_eng_defaults(args: &Value) -> Result<Value, String> {
|
||||
let _ = args;
|
||||
let specs = eng_default_specs();
|
||||
tools_register(&json!({"specs": specs}))
|
||||
}
|
||||
|
||||
fn eng_default_specs() -> Vec<Value> {
|
||||
vec![
|
||||
json!({
|
||||
"id": "tool_find",
|
||||
"version": "1",
|
||||
"description": "Engineering find via EKC (read-only)",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {"query": {"type": "string"}},
|
||||
"required": ["query"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"effect": "read",
|
||||
"scopes": ["knowledge:read"],
|
||||
"availability": "ready"
|
||||
}),
|
||||
json!({
|
||||
"id": "tool_reuse",
|
||||
"version": "1",
|
||||
"description": "Engineering reuse / preflight discovery (read-only)",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {"task": {"type": "string"}},
|
||||
"required": ["task"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"effect": "read",
|
||||
"scopes": ["knowledge:read"],
|
||||
"availability": "ready"
|
||||
}),
|
||||
json!({
|
||||
"id": "tool_bind",
|
||||
"version": "1",
|
||||
"description": "Bind and execute a verified knowledge artifact",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {"type": "string"},
|
||||
"policy": {"type": "string"},
|
||||
"project": {"type": "string"}
|
||||
},
|
||||
"required": ["query"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"effect": "execute",
|
||||
"scopes": ["knowledge:execute"],
|
||||
"availability": "ready"
|
||||
}),
|
||||
json!({
|
||||
"id": "tool_verify",
|
||||
"version": "1",
|
||||
"description": "Run L3 knowledge verification checks",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"effect": "read",
|
||||
"scopes": ["knowledge:verify"],
|
||||
"availability": "ready"
|
||||
}),
|
||||
json!({
|
||||
"id": "tool_summarize",
|
||||
"version": "1",
|
||||
"description": "Summarize a Context Pack / preflight map",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {"preflight": {"type": "object"}},
|
||||
"required": ["preflight"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"effect": "read",
|
||||
"scopes": ["knowledge:read"],
|
||||
"availability": "ready"
|
||||
}),
|
||||
]
|
||||
}
|
||||
|
||||
fn tool_result_err(code: &str, category: &str, message: String, retryable: bool) -> Value {
|
||||
let recovery = if category == "invalid_arguments" {
|
||||
"repair_args"
|
||||
} else if category == "forbidden" {
|
||||
"needs_input"
|
||||
} else {
|
||||
"stop"
|
||||
};
|
||||
json!({
|
||||
"ok": false,
|
||||
"status": "failed",
|
||||
"summary": message,
|
||||
"schema": "ToolResultV1",
|
||||
"schema_version": 1,
|
||||
"structured_content": null,
|
||||
"resources": [],
|
||||
"citations": [],
|
||||
"error": {
|
||||
"schema": "AgentErrorV1",
|
||||
"schema_version": 1,
|
||||
"code": code,
|
||||
"category": category,
|
||||
"stage": "tool",
|
||||
"message": message,
|
||||
"retryable": retryable,
|
||||
"retry_after_ms": null,
|
||||
"effect_state": "none",
|
||||
"recovery_action": recovery,
|
||||
},
|
||||
"usage": null,
|
||||
"truncated": false,
|
||||
"next_cursor": null,
|
||||
"effect_receipt_ref": null,
|
||||
})
|
||||
}
|
||||
|
||||
/// Seed stub specs from a CALL allowlist (args.tools).
|
||||
pub fn seed_from_allowlist_args(args: &Value) -> Result<Value, String> {
|
||||
let tools = args
|
||||
.get("tools")
|
||||
.ok_or_else(|| "tools.seed_allowlist requires tools".to_string())?;
|
||||
seed_from_allowlist(tools)
|
||||
}
|
||||
|
||||
fn seed_from_allowlist(tools: &Value) -> Result<Value, String> {
|
||||
let names = crate::tool_names(tools);
|
||||
let mut specs = Vec::new();
|
||||
for name in names {
|
||||
// Minimal stub if not already registered — CALL whitelist compatibility.
|
||||
let already = {
|
||||
let guard = registry()
|
||||
.lock()
|
||||
.map_err(|_| "tools registry lock poisoned".to_string())?;
|
||||
guard.contains_key(&name)
|
||||
};
|
||||
if already {
|
||||
continue;
|
||||
}
|
||||
let mut m = Map::new();
|
||||
m.insert("id".into(), json!(name));
|
||||
m.insert("version".into(), json!("1"));
|
||||
m.insert(
|
||||
"description".into(),
|
||||
json!(format!("Allowlist-derived tool `{name}`")),
|
||||
);
|
||||
m.insert(
|
||||
"input_schema".into(),
|
||||
json!({"type":"object","properties":{},"additionalProperties":true}),
|
||||
);
|
||||
m.insert("effect".into(), json!("execute"));
|
||||
m.insert("scopes".into(), json!(["call:allowlist"]));
|
||||
m.insert("availability".into(), json!("ready"));
|
||||
specs.push(Value::Object(m));
|
||||
}
|
||||
if specs.is_empty() {
|
||||
return Ok(json!({"ok": true, "registered": [], "count": 0}));
|
||||
}
|
||||
tools_register(&json!({"specs": specs}))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn register_search_describe_and_forbid() {
|
||||
let _ = tools_clear(&json!({}));
|
||||
let reg = tools_register_eng_defaults(&json!({})).unwrap();
|
||||
assert_eq!(reg["count"], 5);
|
||||
|
||||
let allow = json!(["tool_find", "tool_reuse"]);
|
||||
let search = tools_search(&json!({"tools": allow, "query": "find"})).unwrap();
|
||||
assert_eq!(search["count"], 1);
|
||||
assert_eq!(search["items"][0]["id"], "tool_find");
|
||||
|
||||
// Unauthorized tool must not appear.
|
||||
let search2 = tools_search(&json!({"tools": allow, "query": "bind"})).unwrap();
|
||||
assert_eq!(search2["count"], 0);
|
||||
|
||||
let denied = tools_describe(&json!({"id": "tool_bind", "tools": allow})).unwrap();
|
||||
assert_eq!(denied["authorized"], false);
|
||||
assert_eq!(denied["ok"], false);
|
||||
|
||||
let ok = tools_describe(&json!({"id": "tool_find", "tools": allow})).unwrap();
|
||||
assert_eq!(ok["found"], true);
|
||||
assert_eq!(ok["spec"]["schema"], "ToolSpecV1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prepare_rejects_bad_args_and_unauthorized() {
|
||||
let _ = tools_clear(&json!({}));
|
||||
tools_register_eng_defaults(&json!({})).unwrap();
|
||||
let allow = json!(["tool_find"]);
|
||||
|
||||
let bad = tools_prepare_call(&json!({
|
||||
"tool_id": "tool_find",
|
||||
"tools": allow,
|
||||
"arguments": {}
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(bad["ok"], false);
|
||||
assert_eq!(bad["error"]["category"], "invalid_arguments");
|
||||
|
||||
let unauth = tools_prepare_call(&json!({
|
||||
"tool_id": "tool_bind",
|
||||
"tools": allow,
|
||||
"arguments": {"query": "x"}
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(unauth["ok"], false);
|
||||
assert_eq!(unauth["error"]["category"], "forbidden");
|
||||
|
||||
let good = tools_prepare_call(&json!({
|
||||
"tool_id": "tool_find",
|
||||
"tools": allow,
|
||||
"arguments": {"query": "foo"},
|
||||
"call_id": "c1"
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(good["ok"], true);
|
||||
assert_eq!(good["call"]["schema"], "ToolCallV1");
|
||||
assert_eq!(good["call"]["tool_id"], "tool_find");
|
||||
assert_eq!(good["needs_ledger"], false);
|
||||
assert!(good["args_digest"].as_str().unwrap().starts_with("sha256-lite:"));
|
||||
|
||||
let allow2 = json!(["tool_bind"]);
|
||||
let bind = tools_prepare_call(&json!({
|
||||
"tool_id": "tool_bind",
|
||||
"tools": allow2,
|
||||
"arguments": {"query": "x"},
|
||||
"call_id": "c2",
|
||||
"logical_run_id": "run1",
|
||||
"step_id": "s1"
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(bind["ok"], true);
|
||||
assert_eq!(bind["needs_ledger"], true);
|
||||
assert_eq!(bind["spec"]["effect"], "execute");
|
||||
assert!(bind["idempotency_key"]
|
||||
.as_str()
|
||||
.unwrap()
|
||||
.starts_with("run1|s1|tool_bind|"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn needs_ledger_only_for_mutating_effects() {
|
||||
assert_eq!(
|
||||
tools_needs_ledger(&json!({"effect": "read"})).unwrap()["needs_ledger"],
|
||||
false
|
||||
);
|
||||
assert_eq!(
|
||||
tools_needs_ledger(&json!({"effect": "write"})).unwrap()["needs_ledger"],
|
||||
true
|
||||
);
|
||||
assert_eq!(
|
||||
tools_needs_ledger(&json!({"effect": "execute"})).unwrap()["needs_ledger"],
|
||||
true
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn refuse_unsupported_schema_keywords() {
|
||||
let err = tools_register(&json!({
|
||||
"id": "bad",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"oneOf": [{"type": "object"}]
|
||||
}
|
||||
}));
|
||||
assert!(err.is_err());
|
||||
}
|
||||
}
|
||||
+485
-24
@@ -44,21 +44,40 @@ fn read_limited(mut r: impl Read) -> Result<String, String> {
|
||||
String::from_utf8(buf).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Prefer null over zero when the provider omitted a field (ModelCallUsageV1 / F04).
|
||||
fn token_or_null(v: Option<i64>) -> Value {
|
||||
match v {
|
||||
Some(n) if n >= 0 => json!(n),
|
||||
_ => Value::Null,
|
||||
}
|
||||
}
|
||||
|
||||
fn usage_from(data: &Value) -> Value {
|
||||
let u = data.get("usage").cloned().unwrap_or(Value::Null);
|
||||
let pt = u.get("prompt_tokens").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
let ct = u
|
||||
.get("completion_tokens")
|
||||
.and_then(|v| v.as_i64())
|
||||
.unwrap_or(0);
|
||||
if u.is_null() || !u.is_object() {
|
||||
return json!({
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
});
|
||||
}
|
||||
let pt = u.get("prompt_tokens").and_then(|v| v.as_i64());
|
||||
let ct = u.get("completion_tokens").and_then(|v| v.as_i64());
|
||||
let tt = u
|
||||
.get("total_tokens")
|
||||
.and_then(|v| v.as_i64())
|
||||
.unwrap_or(pt + ct);
|
||||
.or_else(|| match (pt, ct) {
|
||||
(Some(a), Some(b)) => Some(a + b),
|
||||
_ => None,
|
||||
});
|
||||
json!({
|
||||
"prompt_tokens": pt,
|
||||
"completion_tokens": ct,
|
||||
"total_tokens": tt,
|
||||
"prompt_tokens": token_or_null(pt),
|
||||
"completion_tokens": token_or_null(ct),
|
||||
"total_tokens": token_or_null(tt),
|
||||
"usage_state": "final",
|
||||
"measurement_source": "provider",
|
||||
})
|
||||
}
|
||||
|
||||
@@ -72,10 +91,101 @@ fn tool_calls_from(data: &Value) -> Value {
|
||||
.unwrap_or_else(|| json!([]))
|
||||
}
|
||||
|
||||
fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, String> {
|
||||
fn cancel_requested() -> bool {
|
||||
// Prefer request-scoped token handle. Empty / "0" ≠ cancel (M-R1).
|
||||
// When a token path is set, ignore process-global MARQDO_CANCEL so parallel
|
||||
// tasks cannot cancel each other via shared env.
|
||||
if let Ok(tok) = std::env::var("MARQDO_CANCEL_TOKEN") {
|
||||
if !tok.is_empty() {
|
||||
let path = std::path::Path::new(&tok);
|
||||
if path.is_file() {
|
||||
if let Ok(body) = std::fs::read_to_string(path) {
|
||||
let t = body.trim();
|
||||
return t == "1" || t.eq_ignore_ascii_case("cancelled");
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
// Legacy single-task CLI only when no token handle is bound.
|
||||
std::env::var("MARQDO_CANCEL")
|
||||
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// Mirror host `net::push_openai_delta_events`: append semantic frames to
|
||||
/// `MARQDO_EVENT_FORWARD` so subtask/web sse-live can flush tokens before the
|
||||
/// lib call returns (OpenAI-style incremental SSE).
|
||||
fn forward_event_bus_frame(ev: &Value) {
|
||||
let Ok(path) = std::env::var("MARQDO_EVENT_FORWARD") else {
|
||||
return;
|
||||
};
|
||||
if path.is_empty() {
|
||||
return;
|
||||
}
|
||||
use std::io::Write;
|
||||
if let Ok(mut f) = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(&path)
|
||||
{
|
||||
let _ = writeln!(f, "{ev}");
|
||||
let _ = f.flush();
|
||||
}
|
||||
}
|
||||
|
||||
/// Publish `{type:delta|plan|reasoning|error, text|message}` from one OpenAI chunk.
|
||||
/// `content_type` is usually `delta` (answer tokens) or `plan` (parent protocol LLM).
|
||||
fn forward_openai_chunk(data: &str, content_type: &str) {
|
||||
let data = data.trim();
|
||||
if data.is_empty() || data == "[DONE]" {
|
||||
return;
|
||||
}
|
||||
let Ok(v) = serde_json::from_str::<Value>(data) else {
|
||||
return;
|
||||
};
|
||||
if let Some(msg) = v.get("error").and_then(|e| {
|
||||
e.get("message")
|
||||
.and_then(|m| m.as_str())
|
||||
.or_else(|| e.as_str())
|
||||
}) {
|
||||
forward_event_bus_frame(&json!({"type": "error", "message": msg}));
|
||||
return;
|
||||
}
|
||||
let reasoning = v
|
||||
.pointer("/choices/0/delta/reasoning_content")
|
||||
.and_then(|c| c.as_str())
|
||||
.unwrap_or("");
|
||||
if !reasoning.is_empty() {
|
||||
forward_event_bus_frame(&json!({"type": "reasoning", "text": reasoning}));
|
||||
}
|
||||
let content = v
|
||||
.pointer("/choices/0/delta/content")
|
||||
.and_then(|c| c.as_str())
|
||||
.unwrap_or("");
|
||||
if !content.is_empty() {
|
||||
let ty = if content_type.is_empty() {
|
||||
"delta"
|
||||
} else {
|
||||
content_type
|
||||
};
|
||||
forward_event_bus_frame(&json!({"type": ty, "text": content}));
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_sse_events(
|
||||
reader: impl BufRead,
|
||||
echo: bool,
|
||||
content_type: &str,
|
||||
) -> Result<Vec<Value>, String> {
|
||||
let mut events = Vec::new();
|
||||
let mut data_lines: Vec<String> = Vec::new();
|
||||
let mut accumulated = String::new();
|
||||
for line in reader.lines() {
|
||||
if cancel_requested() {
|
||||
events.push(json!({"type": "cancelled", "message": "MARQDO_CANCEL"}));
|
||||
return Ok(events);
|
||||
}
|
||||
let line = line.map_err(|e| e.to_string())?;
|
||||
if echo {
|
||||
eprint!("{line}\n");
|
||||
@@ -85,9 +195,27 @@ fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, Stri
|
||||
let data = data_lines.join("\n");
|
||||
data_lines.clear();
|
||||
if data.trim() == "[DONE]" {
|
||||
// Parent owns run lifecycle `done`; only forward for answer streams.
|
||||
if content_type == "delta" || content_type.is_empty() {
|
||||
let done = json!({"type": "done", "result": accumulated});
|
||||
forward_event_bus_frame(&done);
|
||||
}
|
||||
events.push(json!({"type": "done", "data": "[DONE]"}));
|
||||
break;
|
||||
}
|
||||
forward_openai_chunk(&data, content_type);
|
||||
if let Ok(v) = serde_json::from_str::<Value>(&data) {
|
||||
if let Some(c) = v
|
||||
.pointer("/choices/0/delta/content")
|
||||
.and_then(|x| x.as_str())
|
||||
{
|
||||
accumulated.push_str(c);
|
||||
}
|
||||
// Preserve provider usage as a first-class event for llm.collect_usage.
|
||||
if let Some(u) = v.get("usage").filter(|u| u.is_object()) {
|
||||
events.push(json!({"type": "usage", "usage": u}));
|
||||
}
|
||||
}
|
||||
events.push(json!({"type": "data", "data": data}));
|
||||
}
|
||||
continue;
|
||||
@@ -99,21 +227,123 @@ fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, Stri
|
||||
if !data_lines.is_empty() {
|
||||
let data = data_lines.join("\n");
|
||||
if data.trim() == "[DONE]" {
|
||||
if content_type == "delta" || content_type.is_empty() {
|
||||
let done = json!({"type": "done", "result": accumulated});
|
||||
forward_event_bus_frame(&done);
|
||||
}
|
||||
events.push(json!({"type": "done", "data": "[DONE]"}));
|
||||
} else {
|
||||
forward_openai_chunk(&data, content_type);
|
||||
events.push(json!({"type": "data", "data": data}));
|
||||
}
|
||||
}
|
||||
Ok(events)
|
||||
}
|
||||
|
||||
fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
let base_url = arg_text(args, "base_url")?;
|
||||
let api_key = arg_text(args, "api_key").unwrap_or("");
|
||||
let model = arg_text(args, "model")?;
|
||||
fn build_messages(args: &Value) -> Result<Value, String> {
|
||||
if let Some(arr) = args.get("messages").and_then(|v| v.as_array()) {
|
||||
if arr.is_empty() {
|
||||
return Err("messages array must not be empty".into());
|
||||
}
|
||||
// Validate minimal role/content shape; reject unsupported part types loudly.
|
||||
for (i, m) in arr.iter().enumerate() {
|
||||
let role = m.get("role").and_then(|r| r.as_str()).unwrap_or("");
|
||||
if role.is_empty() {
|
||||
return Err(format!("llm.unsupported_input: messages[{i}].role missing"));
|
||||
}
|
||||
if let Some(parts) = m.get("content").and_then(|c| c.as_array()) {
|
||||
for (j, p) in parts.iter().enumerate() {
|
||||
let ty = p.get("type").and_then(|t| t.as_str()).unwrap_or("text");
|
||||
if ty != "text" {
|
||||
return Err(format!(
|
||||
"llm.unsupported_input: messages[{i}].content[{j}] type={ty}"
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return Ok(Value::Array(arr.clone()));
|
||||
}
|
||||
let prompt = arg_text(args, "prompt")?;
|
||||
Ok(json!([{"role": "user", "content": prompt}]))
|
||||
}
|
||||
|
||||
fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
if cancel_requested() {
|
||||
return Ok(json!({
|
||||
"text": "",
|
||||
"usage": {
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
},
|
||||
"finish": "cancelled",
|
||||
"error": "cancelled",
|
||||
"tool_calls": [],
|
||||
"transport": "plugin",
|
||||
}));
|
||||
}
|
||||
// Strict selection mode: refuse env fallback when selection.strict is set.
|
||||
let strict = opt_bool(args, "strict_selection", false)
|
||||
|| args
|
||||
.get("selection")
|
||||
.and_then(|s| s.get("strict"))
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
let selection = args.get("selection").cloned();
|
||||
if strict {
|
||||
if selection.is_none() {
|
||||
return Err("llm.selection_missing:selection".into());
|
||||
}
|
||||
let sel = selection.as_ref().unwrap();
|
||||
for key in ["selection_id", "provider_id", "model_id", "upstream_model_id"] {
|
||||
if sel
|
||||
.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.is_none()
|
||||
{
|
||||
return Err(format!("llm.selection_missing:{key}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let base_url = match arg_text(args, "base_url") {
|
||||
Ok(u) => u,
|
||||
Err(_) if strict => return Err("llm.selection_missing:base_url".into()),
|
||||
Err(e) => return Err(e),
|
||||
};
|
||||
let auth_mode = opt_text(args, "auth_mode")
|
||||
.or_else(|| {
|
||||
selection
|
||||
.as_ref()
|
||||
.and_then(|s| s.get("auth_mode"))
|
||||
.and_then(|v| v.as_str())
|
||||
})
|
||||
.unwrap_or("api_key");
|
||||
let api_key = if auth_mode == "none" {
|
||||
""
|
||||
} else {
|
||||
arg_text(args, "api_key").unwrap_or("")
|
||||
};
|
||||
if strict && auth_mode != "none" && api_key.is_empty() {
|
||||
return Err("llm.selection_missing:api_key".into());
|
||||
}
|
||||
let model = selection
|
||||
.as_ref()
|
||||
.and_then(|s| s.get("upstream_model_id"))
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(|s| s.to_string())
|
||||
.or_else(|| arg_text(args, "model").ok().map(|s| s.to_string()))
|
||||
.ok_or_else(|| "missing text `model`".to_string())?;
|
||||
let messages = build_messages(args)?;
|
||||
let stream = opt_bool(args, "stream", false);
|
||||
let echo = opt_bool(args, "echo", false);
|
||||
// Answer tokens → "delta"; parent plan protocol → "plan" (UI process trail).
|
||||
let stream_kind = opt_text(args, "stream_kind").unwrap_or("delta");
|
||||
let suffix = opt_text(args, "suffix").unwrap_or("/chat/completions");
|
||||
let bearer = opt_text(args, "bearer").unwrap_or("Bearer ");
|
||||
|
||||
@@ -123,13 +353,27 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
}
|
||||
|
||||
let mut req_body = Map::new();
|
||||
req_body.insert("model".into(), Value::String(model.to_string()));
|
||||
req_body.insert(
|
||||
"messages".into(),
|
||||
json!([{"role": "user", "content": prompt}]),
|
||||
);
|
||||
req_body.insert("model".into(), Value::String(model.clone()));
|
||||
req_body.insert("messages".into(), messages);
|
||||
if let Some(params) = selection
|
||||
.as_ref()
|
||||
.and_then(|s| s.get("parameters"))
|
||||
.and_then(|p| p.as_object())
|
||||
{
|
||||
if let Some(max) = params.get("max_output_tokens").and_then(|v| v.as_u64()) {
|
||||
req_body.insert("max_tokens".into(), json!(max));
|
||||
}
|
||||
if let Some(temp) = params.get("temperature") {
|
||||
req_body.insert("temperature".into(), temp.clone());
|
||||
}
|
||||
}
|
||||
if stream {
|
||||
req_body.insert("stream".into(), Value::Bool(true));
|
||||
// OpenAI / DeepSeek / compatible: final SSE chunk carries usage only when requested.
|
||||
req_body.insert(
|
||||
"stream_options".into(),
|
||||
json!({ "include_usage": true }),
|
||||
);
|
||||
}
|
||||
let body = Value::Object(req_body).to_string();
|
||||
let auth = format!("{bearer}{api_key}");
|
||||
@@ -162,7 +406,8 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
"transport": "plugin",
|
||||
}));
|
||||
}
|
||||
let events = parse_sse_events(BufReader::new(r.into_reader()), echo)?;
|
||||
let events =
|
||||
parse_sse_events(BufReader::new(r.into_reader()), echo, stream_kind)?;
|
||||
Ok(json!({
|
||||
"events": events,
|
||||
"finish": "stop",
|
||||
@@ -193,11 +438,19 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
if status != 200 {
|
||||
return Ok(json!({
|
||||
"text": "",
|
||||
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
|
||||
"usage": {
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
},
|
||||
"finish": "error",
|
||||
"error": format!("HTTP {status}: {text}"),
|
||||
"tool_calls": [],
|
||||
"transport": "plugin",
|
||||
"requested_model": model,
|
||||
"resolved_model": model,
|
||||
}));
|
||||
}
|
||||
let data: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?;
|
||||
@@ -208,9 +461,12 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
.and_then(|c0| c0.get("message"))
|
||||
.cloned()
|
||||
.unwrap_or(json!({}));
|
||||
// Some gateways (reasoning models) return null content + reasoning_content.
|
||||
let content = msg
|
||||
.get("content")
|
||||
.and_then(|v| v.as_str())
|
||||
.filter(|s| !s.is_empty())
|
||||
.or_else(|| msg.get("reasoning_content").and_then(|v| v.as_str()))
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let fr = data
|
||||
@@ -221,32 +477,62 @@ fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("stop")
|
||||
.to_string();
|
||||
let resolved = data
|
||||
.get("model")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(model.as_str())
|
||||
.to_string();
|
||||
Ok(json!({
|
||||
"text": content,
|
||||
"usage": usage_from(&data),
|
||||
"finish": fr,
|
||||
"tool_calls": tool_calls_from(&data),
|
||||
"transport": "plugin",
|
||||
"requested_model": model,
|
||||
"resolved_model": resolved,
|
||||
"selection": selection.as_ref().map(|s| {
|
||||
let mut snap = s.clone();
|
||||
if let Some(obj) = snap.as_object_mut() {
|
||||
obj.remove("api_key");
|
||||
}
|
||||
snap
|
||||
}),
|
||||
}))
|
||||
}
|
||||
Err(ureq::Error::Status(code, r)) => {
|
||||
let text = read_limited(r.into_reader()).unwrap_or_default();
|
||||
Ok(json!({
|
||||
"text": "",
|
||||
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
|
||||
"usage": {
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
},
|
||||
"finish": "error",
|
||||
"error": format!("HTTP {code}: {text}"),
|
||||
"tool_calls": [],
|
||||
"transport": "plugin",
|
||||
"requested_model": model,
|
||||
"resolved_model": model,
|
||||
}))
|
||||
}
|
||||
Err(e) => Ok(json!({
|
||||
"text": "",
|
||||
"usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
|
||||
"usage": {
|
||||
"prompt_tokens": Value::Null,
|
||||
"completion_tokens": Value::Null,
|
||||
"total_tokens": Value::Null,
|
||||
"usage_state": "unknown",
|
||||
"measurement_source": "none",
|
||||
},
|
||||
"finish": "error",
|
||||
"error": format!("http POST {url}: {e}"),
|
||||
"tool_calls": [],
|
||||
"transport": "plugin",
|
||||
"requested_model": model,
|
||||
"resolved_model": model,
|
||||
})),
|
||||
}
|
||||
}
|
||||
@@ -274,5 +560,180 @@ unsafe extern "C" fn llm_ping(
|
||||
|
||||
marqdo_plugin!(marqdo_plugin_sdk::ABI_VERSION, [
|
||||
("llm_ping", "", llm_ping),
|
||||
("llm_chat_completions", "base_url,api_key,model,prompt,stream,echo,suffix,bearer", llm_chat_completions),
|
||||
("llm_chat_completions", "base_url,api_key,model,prompt,messages,selection,strict_selection,auth_mode,stream,echo,suffix,bearer", llm_chat_completions),
|
||||
]);
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn f03_messages_preserve_roles_and_reject_image() {
|
||||
let msgs = build_messages(&json!({
|
||||
"messages": [
|
||||
{"role": "system", "content": "rules"},
|
||||
{"role": "user", "content": "你好"},
|
||||
{"role": "assistant", "content": "先前"},
|
||||
{"role": "user", "content": "最新用户"}
|
||||
]
|
||||
}))
|
||||
.unwrap();
|
||||
let arr = msgs.as_array().unwrap();
|
||||
assert_eq!(arr.len(), 4);
|
||||
assert_eq!(arr[3]["role"], "user");
|
||||
assert_eq!(arr[3]["content"], "最新用户");
|
||||
|
||||
let err = build_messages(&json!({
|
||||
"messages": [{"role": "user", "content": [{"type": "image", "url": "x"}]}]
|
||||
}))
|
||||
.unwrap_err();
|
||||
assert!(err.contains("unsupported_input"), "{err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f01_strict_selection_does_not_invent_env_model() {
|
||||
std::env::remove_var("MARQDO_CANCEL");
|
||||
std::env::remove_var("MARQDO_CANCEL_TOKEN");
|
||||
let err = chat_completions(&json!({
|
||||
"strict_selection": true,
|
||||
"base_url": "http://127.0.0.1:9",
|
||||
"api_key": "k",
|
||||
"prompt": "hi"
|
||||
// missing selection → must fail before any HTTP
|
||||
}))
|
||||
.unwrap_err();
|
||||
assert!(err.contains("selection_missing"), "{err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f06_cancel_short_circuits_before_http() {
|
||||
// Prefer token file over process-global MARQDO_CANCEL to avoid parallel test races.
|
||||
let tok = std::env::temp_dir().join(format!("llm-cancel-{}", std::process::id()));
|
||||
std::fs::write(&tok, "cancelled").unwrap();
|
||||
std::env::set_var("MARQDO_CANCEL_TOKEN", &tok);
|
||||
let out = chat_completions(&json!({
|
||||
"base_url": "http://127.0.0.1:9",
|
||||
"api_key": "k",
|
||||
"model": "m",
|
||||
"prompt": "hi"
|
||||
}));
|
||||
std::env::remove_var("MARQDO_CANCEL_TOKEN");
|
||||
let _ = std::fs::remove_file(&tok);
|
||||
let out = out.unwrap();
|
||||
assert_eq!(out["finish"], "cancelled");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f06_empty_cancel_token_is_not_cancelled() {
|
||||
// sse-live creates an empty token file; that must NOT abort the turn.
|
||||
let tok = std::env::temp_dir().join(format!("llm-cancel-empty-{}", std::process::id()));
|
||||
std::fs::write(&tok, "").unwrap();
|
||||
std::env::set_var("MARQDO_CANCEL_TOKEN", &tok);
|
||||
std::env::remove_var("MARQDO_CANCEL");
|
||||
assert!(!cancel_requested());
|
||||
std::env::remove_var("MARQDO_CANCEL_TOKEN");
|
||||
let _ = std::fs::remove_file(&tok);
|
||||
}
|
||||
|
||||
/// F01: two local endpoints receive distinct models/auth — no cross-talk.
|
||||
#[test]
|
||||
fn f01_dual_endpoint_isolation() {
|
||||
use std::io::{Read, Write};
|
||||
use std::net::TcpListener;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::thread;
|
||||
|
||||
fn spawn_mock(tag: &'static str, seen: Arc<Mutex<Vec<String>>>) -> String {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
thread::spawn(move || {
|
||||
if let Ok((mut stream, _)) = listener.accept() {
|
||||
let mut raw = Vec::new();
|
||||
let mut buf = [0u8; 4096];
|
||||
loop {
|
||||
let n = stream.read(&mut buf).unwrap_or(0);
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
raw.extend_from_slice(&buf[..n]);
|
||||
if let Some(pos) = raw.windows(4).position(|w| w == b"\r\n\r\n") {
|
||||
let header = String::from_utf8_lossy(&raw[..pos]);
|
||||
let cl = header
|
||||
.lines()
|
||||
.find_map(|l| l.strip_prefix("Content-Length:"))
|
||||
.and_then(|v| v.trim().parse::<usize>().ok())
|
||||
.unwrap_or(0);
|
||||
let body_start = pos + 4;
|
||||
while raw.len() < body_start + cl {
|
||||
let n = stream.read(&mut buf).unwrap_or(0);
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
raw.extend_from_slice(&buf[..n]);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
let req = String::from_utf8_lossy(&raw).to_string();
|
||||
seen.lock().unwrap().push(format!("{tag}:{req}"));
|
||||
let body = r#"{"choices":[{"message":{"content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1,"total_tokens":2}}"#;
|
||||
let resp = format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}",
|
||||
body.len(),
|
||||
body
|
||||
);
|
||||
let _ = stream.write_all(resp.as_bytes());
|
||||
}
|
||||
});
|
||||
format!("http://{addr}")
|
||||
}
|
||||
|
||||
let seen_a = Arc::new(Mutex::new(Vec::new()));
|
||||
let seen_b = Arc::new(Mutex::new(Vec::new()));
|
||||
let url_a = spawn_mock("A", seen_a.clone());
|
||||
let url_b = spawn_mock("B", seen_b.clone());
|
||||
|
||||
let out_a = chat_completions(&json!({
|
||||
"base_url": url_a,
|
||||
"api_key": "key-a",
|
||||
"model": "model-a",
|
||||
"auth_mode": "api_key",
|
||||
"messages": [{"role":"user","content":"from-a"}],
|
||||
"selection": {
|
||||
"selection_id": "sel-a",
|
||||
"provider_id": "p-a",
|
||||
"model_id": "m-a",
|
||||
"upstream_model_id": "model-a",
|
||||
"strict": true
|
||||
},
|
||||
"strict_selection": true
|
||||
}))
|
||||
.unwrap();
|
||||
let out_b = chat_completions(&json!({
|
||||
"base_url": url_b,
|
||||
"api_key": "key-b",
|
||||
"model": "model-b",
|
||||
"auth_mode": "api_key",
|
||||
"messages": [{"role":"user","content":"from-b"}],
|
||||
"selection": {
|
||||
"selection_id": "sel-b",
|
||||
"provider_id": "p-b",
|
||||
"model_id": "m-b",
|
||||
"upstream_model_id": "model-b",
|
||||
"strict": true
|
||||
},
|
||||
"strict_selection": true
|
||||
}))
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(out_a["finish"], "stop", "{out_a}");
|
||||
assert_eq!(out_b["finish"], "stop", "{out_b}");
|
||||
let a = seen_a.lock().unwrap().join("\n");
|
||||
let b = seen_b.lock().unwrap().join("\n");
|
||||
assert!(a.contains("model-a"), "A saw wrong model: {a}");
|
||||
assert!(a.contains("key-a") || a.contains("Authorization"), "{a}");
|
||||
assert!(!a.contains("model-b"), "A must not see B's model: {a}");
|
||||
assert!(b.contains("model-b"), "B saw wrong model: {b}");
|
||||
assert!(!b.contains("model-a"), "B must not see A's model: {b}");
|
||||
}
|
||||
}
|
||||
@@ -63,6 +63,11 @@ func ParseFrontmatter(src string) (meta map[string]string, body string) {
|
||||
body = body[i+3:]
|
||||
body = strings.TrimPrefix(body, "\r")
|
||||
body = strings.TrimPrefix(body, "\n")
|
||||
// Drop blank line(s) between closing --- and first content.
|
||||
for strings.HasPrefix(body, "\n") || strings.HasPrefix(body, "\r\n") {
|
||||
body = strings.TrimPrefix(body, "\r\n")
|
||||
body = strings.TrimPrefix(body, "\n")
|
||||
}
|
||||
}
|
||||
for _, line := range strings.Split(fm, "\n") {
|
||||
line = strings.TrimSpace(strings.TrimSuffix(line, "\r"))
|
||||
@@ -254,6 +259,10 @@ func SystemRoutes() []map[string]any {
|
||||
return []map[string]any{
|
||||
{"method": "GET", "path": "/_part/*", "kind": "system"},
|
||||
{"method": "GET|POST", "path": "/_form/*", "kind": "system"},
|
||||
{"method": "POST", "path": "/_runtime/tasks", "kind": "system"},
|
||||
{"method": "GET", "path": "/_runtime/tasks/{id}", "kind": "system"},
|
||||
{"method": "POST", "path": "/_runtime/tasks/{id}/cancel", "kind": "system"},
|
||||
{"method": "GET", "path": "/_runtime/tasks/{id}/events", "kind": "system"},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -275,8 +284,20 @@ func InspectBag(app map[string]any, declared []ArtifactFile) map[string]any {
|
||||
}
|
||||
}
|
||||
if inv, ok := app["invoke_routes"].(map[string]any); ok {
|
||||
for key := range inv {
|
||||
dec = append(dec, map[string]any{"path": key, "kind": "endpoint"})
|
||||
for key, v := range inv {
|
||||
row := map[string]any{"path": key, "kind": "endpoint"}
|
||||
if m, ok := v.(map[string]any); ok {
|
||||
if c, ok := m["class"].(string); ok && c != "" {
|
||||
row["class"] = c
|
||||
}
|
||||
if method, ok := m["method"].(string); ok && method != "" {
|
||||
row["method"] = method
|
||||
}
|
||||
if fn, ok := m["fn"].(string); ok && fn != "" {
|
||||
row["fn"] = fn
|
||||
}
|
||||
}
|
||||
dec = append(dec, row)
|
||||
}
|
||||
}
|
||||
if art, ok := app["artifact_routes"].(map[string]any); ok {
|
||||
|
||||
@@ -968,6 +968,54 @@ func Update(url, tableName, id string, row any, txnID ...string) (map[string]any
|
||||
return map[string]any{"ok": true, "updated": float64(n)}, nil
|
||||
}
|
||||
|
||||
// UpdateIf is a conditional update (CAS): SET row WHERE id=? AND expect_col=expect_val.
|
||||
// Returns {"ok":true,"updated":N,"matched":N}. Callers treat updated==0 as conflict.
|
||||
// Optional txnID routes through an open Begin transaction.
|
||||
func UpdateIf(url, tableName, id string, row any, expectCol string, expectVal any, txnID ...string) (map[string]any, error) {
|
||||
tn, err := ident(tableName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ecol, err := ident(expectCol)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
obj, ok := row.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("row must be a map")
|
||||
}
|
||||
c, err := connFor(url, optTxn(txnID))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var sets []string
|
||||
var vals []any
|
||||
for k, v := range obj {
|
||||
if k == "id" {
|
||||
continue
|
||||
}
|
||||
if _, err := ident(k); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sets = append(sets, fmt.Sprintf(`"%s" = ?`, k))
|
||||
vals = append(vals, toSQLArg(v))
|
||||
}
|
||||
if len(sets) == 0 {
|
||||
return nil, fmt.Errorf("nothing to update")
|
||||
}
|
||||
vals = append(vals, id, toSQLArg(expectVal))
|
||||
sqlStmt := fmt.Sprintf(
|
||||
`UPDATE "%s" SET %s WHERE "id" = ? AND "%s" = ?`,
|
||||
tn, strings.Join(sets, ", "), ecol,
|
||||
)
|
||||
res, err := c.Exec(sqlStmt, vals...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
n, _ := res.RowsAffected()
|
||||
return map[string]any{"ok": true, "updated": float64(n), "matched": float64(n)}, nil
|
||||
}
|
||||
|
||||
// Delete removes a row by id. Returns {"ok":true,"deleted":N}.
|
||||
// Optional txnID routes through an open Begin transaction.
|
||||
func Delete(url, tableName, id string, txnID ...string) (map[string]any, error) {
|
||||
|
||||
@@ -100,6 +100,24 @@ func TestInitInsertSelectGet(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpdateIfCAS(t *testing.T) {
|
||||
url := setupArticles(t)
|
||||
ok, err := db.UpdateIf(url, "articles", "1", map[string]any{"title": "CAS"}, "title", "Alpha")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ok["updated"] != float64(1) {
|
||||
t.Fatalf("cas hit=%v", ok)
|
||||
}
|
||||
miss, err := db.UpdateIf(url, "articles", "1", map[string]any{"title": "Nope"}, "title", "Alpha")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if miss["updated"] != float64(0) {
|
||||
t.Fatalf("cas miss should be 0, got %v", miss)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpdateDelete(t *testing.T) {
|
||||
url := setupArticles(t)
|
||||
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// F06: client-disconnect cancel bridge writes token file + MARQDO_CANCEL for llm plugin.
|
||||
func TestPropagateCancelSignal(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
tok := filepath.Join(dir, "cancel.tok")
|
||||
if err := os.WriteFile(tok, []byte("cancelled"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("MARQDO_CANCEL_TOKEN", tok)
|
||||
t.Setenv("MARQDO_CANCEL", "1")
|
||||
|
||||
if os.Getenv("MARQDO_CANCEL") != "1" {
|
||||
t.Fatalf("MARQDO_CANCEL not set")
|
||||
}
|
||||
body, err := os.ReadFile(tok)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(body) != "cancelled" {
|
||||
t.Fatalf("token body=%q", body)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestDrainForwardReadsIncrementalLines(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "evt.ndjson")
|
||||
if err := os.WriteFile(path, []byte("{\"type\":\"a\"}\n{\"type\":\"b\"}\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var lines []string
|
||||
off := drainForward(path, 0, func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 2 {
|
||||
t.Fatalf("got %d lines %#v", len(lines), lines)
|
||||
}
|
||||
if off <= 0 {
|
||||
t.Fatalf("offset=%d", off)
|
||||
}
|
||||
// Append after first drain — second call with offset must only see new line.
|
||||
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.WriteString("{\"type\":\"c\"}\n"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = f.Close()
|
||||
var more []string
|
||||
off2 := drainForward(path, off, func(line string) {
|
||||
more = append(more, line)
|
||||
})
|
||||
if len(more) != 1 || more[0] != "{\"type\":\"c\"}" {
|
||||
t.Fatalf("incremental got %#v off=%d→%d", more, off, off2)
|
||||
}
|
||||
}
|
||||
@@ -198,6 +198,35 @@ func invokeArgsFromPayload(payload map[string]any) map[string]any {
|
||||
return args
|
||||
}
|
||||
|
||||
// payloadWantsStream detects OpenAI-compatible stream:true on the request body.
|
||||
func payloadWantsStream(payload map[string]any) bool {
|
||||
if payload == nil {
|
||||
return false
|
||||
}
|
||||
if v, ok := payload["stream"]; ok {
|
||||
return boolishAny(v)
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func boolishAny(v any) bool {
|
||||
switch t := v.(type) {
|
||||
case bool:
|
||||
return t
|
||||
case string:
|
||||
s := strings.ToLower(strings.TrimSpace(t))
|
||||
return s == "1" || s == "true" || s == "yes" || s == "on"
|
||||
case float64:
|
||||
return t != 0
|
||||
case int:
|
||||
return t != 0
|
||||
case int64:
|
||||
return t != 0
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func readJSONBody(r *http.Request, max int64) (map[string]any, error) {
|
||||
r.Body = http.MaxBytesReader(nil, r.Body, max)
|
||||
defer r.Body.Close()
|
||||
|
||||
@@ -239,6 +239,7 @@ func NewHandler(appBag map[string]any, entryDir string) (http.Handler, error) {
|
||||
st.mountGalleryRoutes(mux)
|
||||
st.mountProxyRoutes(mux)
|
||||
st.mountInvokeRoutes(mux)
|
||||
st.mountRuntimeRoutes(mux)
|
||||
st.mountArtifactRoutes(mux)
|
||||
st.mountAuthRoutes(mux)
|
||||
|
||||
@@ -799,6 +800,13 @@ func (g *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
|
||||
return nil, nil, fmt.Errorf("underlying ResponseWriter does not support hijacking")
|
||||
}
|
||||
|
||||
func (g *gzipResponseWriter) Flush() {
|
||||
_ = g.w.Flush()
|
||||
if f, ok := g.ResponseWriter.(http.Flusher); ok {
|
||||
f.Flush()
|
||||
}
|
||||
}
|
||||
|
||||
type statusRecorder struct {
|
||||
http.ResponseWriter
|
||||
status int
|
||||
@@ -816,6 +824,14 @@ func (s *statusRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
|
||||
return nil, nil, fmt.Errorf("underlying ResponseWriter does not support hijacking")
|
||||
}
|
||||
|
||||
// Flush must be forwarded so sse-live / EventBus token streams are not buffered
|
||||
// behind access-log wrapping (otherwise clients see 0 bytes until the invoke ends).
|
||||
func (s *statusRecorder) Flush() {
|
||||
if f, ok := s.ResponseWriter.(http.Flusher); ok {
|
||||
f.Flush()
|
||||
}
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||
w.WriteHeader(code)
|
||||
|
||||
@@ -5,16 +5,40 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/plugin"
|
||||
)
|
||||
|
||||
// legacySSEBridgeMu serializes the process-global MARQDO_EVENT_FORWARD /
|
||||
// MARQDO_CANCEL_TOKEN bridge for the legacy_serial path only. Isolated execute
|
||||
// never acquires this lock (ADR 0009).
|
||||
var legacySSEBridgeMu sync.Mutex
|
||||
|
||||
// legacySSEBridgeActive is 1 while a holder has mutated process env for the bridge.
|
||||
var legacySSEBridgeActive atomic.Int32
|
||||
|
||||
func tryAcquireLegacySSEBridge() bool {
|
||||
return legacySSEBridgeMu.TryLock()
|
||||
}
|
||||
|
||||
func releaseLegacySSEBridge() {
|
||||
legacySSEBridgeActive.Store(0)
|
||||
legacySSEBridgeMu.Unlock()
|
||||
}
|
||||
|
||||
type invokeRoute struct {
|
||||
Method string
|
||||
FnPath string
|
||||
BodyMode string
|
||||
ReturnMode string
|
||||
Class string
|
||||
Path string
|
||||
}
|
||||
|
||||
func parseInvokeRoute(v any) invokeRoute {
|
||||
@@ -26,11 +50,16 @@ func parseInvokeRoute(v any) invokeRoute {
|
||||
if fn == "" {
|
||||
fn = strOpt(m, "function", "")
|
||||
}
|
||||
class := strings.ToLower(strOpt(m, "class", ""))
|
||||
if class == "" {
|
||||
class = strings.ToLower(strOpt(m, "类别", ""))
|
||||
}
|
||||
return invokeRoute{
|
||||
Method: method,
|
||||
FnPath: fn,
|
||||
BodyMode: body,
|
||||
ReturnMode: ret,
|
||||
Class: class,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -38,6 +67,7 @@ func (st *state) mountInvokeRoutes(mux *http.ServeMux) {
|
||||
for _, key := range sortedKeys(st.invokeRoutes) {
|
||||
cfg := parseInvokeRoute(st.invokeRoutes[key])
|
||||
path := routePathKey(key)
|
||||
cfg.Path = path
|
||||
method := cfg.Method
|
||||
if method == "" {
|
||||
method = http.MethodPost
|
||||
@@ -102,7 +132,6 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
|
||||
}
|
||||
}
|
||||
|
||||
// Bind common auth / client headers for OpenAI-compatible and API-key surfaces.
|
||||
if auth := strings.TrimSpace(r.Header.Get("Authorization")); auth != "" {
|
||||
if _, exists := payload["authorization"]; !exists {
|
||||
payload["authorization"] = auth
|
||||
@@ -113,12 +142,77 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
|
||||
payload["api_key"] = key
|
||||
}
|
||||
}
|
||||
if last := strings.TrimSpace(r.Header.Get("Last-Event-ID")); last != "" {
|
||||
if _, exists := payload["last_event_id"]; !exists {
|
||||
payload["last_event_id"] = last
|
||||
}
|
||||
}
|
||||
// Merge Go ServeMux path wildcards (e.g. /turns/{id}/events → id / turn_id).
|
||||
if route.Path != "" {
|
||||
pathVals := pathValuesOf(r, route.Path)
|
||||
for k, v := range pathVals {
|
||||
if _, exists := payload[k]; !exists {
|
||||
payload[k] = v
|
||||
}
|
||||
}
|
||||
if id, ok := pathVals["id"]; ok {
|
||||
if _, exists := payload["turn_id"]; !exists {
|
||||
payload["turn_id"] = id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
args := invokeArgsFromPayload(payload)
|
||||
|
||||
liveSSE := route.ReturnMode == "sse-live" || route.ReturnMode == "sse_live"
|
||||
if !liveSSE && payloadWantsStream(payload) {
|
||||
liveSSE = true
|
||||
}
|
||||
|
||||
class := resolveInvokeClass(route, liveSSE)
|
||||
switch class {
|
||||
case "execute":
|
||||
// Isolated workers: do not take parent INVOKE_LOCK (ADR 0009).
|
||||
st.handleInvokeIsolated(w, r, route, args, liveSSE, class)
|
||||
return
|
||||
case "short", "read", "control":
|
||||
// Stay on parent: execute already runs off-process, so parent lock is free.
|
||||
// Putting short/read/control on the 2-slot execute worker pool caused
|
||||
// systemic no_slot (and zombies held slots after observer disconnect).
|
||||
// Product cancel/events prefer /_runtime/* (lock-free).
|
||||
st.handleInvokeParent(w, r, route, args)
|
||||
return
|
||||
default:
|
||||
// legacy_serial: parent CallLibPath (ADR 0008). Never isolate undeclared non-stream.
|
||||
st.handleInvokeParent(w, r, route, args)
|
||||
}
|
||||
}
|
||||
|
||||
func resolveInvokeClass(route invokeRoute, liveSSE bool) string {
|
||||
c := strings.ToLower(strings.TrimSpace(route.Class))
|
||||
switch c {
|
||||
case "execute", "short", "read", "control":
|
||||
return c
|
||||
case "legacy", "legacy_serial":
|
||||
return "legacy_serial"
|
||||
}
|
||||
if liveSSE {
|
||||
return "execute"
|
||||
}
|
||||
return "legacy_serial"
|
||||
}
|
||||
|
||||
func (st *state) handleInvokeParent(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any) {
|
||||
_ = r
|
||||
value, err := plugin.CallLibPath(route.FnPath, args)
|
||||
if err != nil {
|
||||
code := http.StatusInternalServerError
|
||||
msg := err.Error()
|
||||
if strings.Contains(msg, "interpreter busy") {
|
||||
w.Header().Set("Retry-After", "2")
|
||||
jsonErr(w, http.StatusServiceUnavailable, msg)
|
||||
return
|
||||
}
|
||||
if strings.Contains(msg, "unknown library") ||
|
||||
strings.Contains(msg, "call_lib_path: need") ||
|
||||
strings.Contains(msg, "not allowed") ||
|
||||
@@ -133,8 +227,6 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
|
||||
return
|
||||
}
|
||||
|
||||
// OpenAI-compatible handlers may return `{sse: "..."}` to force event-stream
|
||||
// even when the route's declared return mode is json.
|
||||
if m, ok := value.(map[string]any); ok {
|
||||
if s, ok := m["sse"].(string); ok && strings.TrimSpace(s) != "" {
|
||||
writeSSE(w, m)
|
||||
@@ -176,3 +268,495 @@ func (st *state) handleInvoke(w http.ResponseWriter, r *http.Request, route invo
|
||||
writeJSON(w, code, body)
|
||||
}
|
||||
}
|
||||
|
||||
// handleInvokeIsolated runs admits on a Supervisor worker (no parent
|
||||
// CallLibPath / Setenv / legacySSEBridgeMu). admissionClass is execute|short|read|control.
|
||||
func (st *state) handleInvokeIsolated(w http.ResponseWriter, r *http.Request, route invokeRoute, args map[string]any, liveSSE bool, admissionClass string) {
|
||||
if admissionClass == "" {
|
||||
admissionClass = "execute"
|
||||
}
|
||||
taskID := fmt.Sprintf("exec_%d", time.Now().UnixNano())
|
||||
adm, err := runtimeQuery("admit", map[string]any{
|
||||
"task_id": taskID,
|
||||
"idempotency_key": taskID,
|
||||
"class": admissionClass,
|
||||
"payload": map[string]any{"fn": route.FnPath},
|
||||
})
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("admit: %v", err))
|
||||
return
|
||||
}
|
||||
if ok, _ := adm["ok"].(bool); !ok {
|
||||
status := http.StatusServiceUnavailable
|
||||
if s, ok := adm["status"].(float64); ok && s >= 400 {
|
||||
status = int(s)
|
||||
}
|
||||
w.Header().Set("Retry-After", "2")
|
||||
jsonErr(w, status, fmt.Sprint(adm["error"]))
|
||||
return
|
||||
}
|
||||
cancelTok, _ := adm["cancel_token"].(string)
|
||||
eventPath, _ := adm["event_path"].(string)
|
||||
|
||||
entryModule, fsRoot, err := resolveSiteEntry(st)
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
argsFile, err := materializeArgsJSON(args)
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("args: %v", err))
|
||||
return
|
||||
}
|
||||
defer os.Remove(argsFile)
|
||||
|
||||
disp, err := runtimeQuery("dispatch", map[string]any{
|
||||
"task_id": taskID,
|
||||
"entry": map[string]any{
|
||||
"module": entryModule,
|
||||
"function": route.FnPath,
|
||||
},
|
||||
"fs_root": fsRoot,
|
||||
"args_json": argsFile,
|
||||
})
|
||||
if err != nil {
|
||||
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
|
||||
jsonErr(w, http.StatusInternalServerError, fmt.Sprintf("dispatch: %v", err))
|
||||
return
|
||||
}
|
||||
if ok, _ := disp["ok"].(bool); !ok {
|
||||
status := http.StatusTooManyRequests
|
||||
if s, ok := disp["status"].(float64); ok && s >= 400 {
|
||||
status = int(s)
|
||||
}
|
||||
if ra, ok := disp["Retry-After"]; ok {
|
||||
w.Header().Set("Retry-After", fmt.Sprint(ra))
|
||||
} else {
|
||||
w.Header().Set("Retry-After", "2")
|
||||
}
|
||||
jsonErr(w, status, fmt.Sprint(disp["error"]))
|
||||
return
|
||||
}
|
||||
if ep, ok := disp["event_path"].(string); ok && ep != "" {
|
||||
eventPath = ep
|
||||
}
|
||||
if tok, ok := disp["cancel_token"].(string); ok && tok != "" {
|
||||
cancelTok = tok
|
||||
}
|
||||
emitResult, _ := disp["emit_result"].(string)
|
||||
|
||||
if liveSSE {
|
||||
st.streamIsolatedSSE(w, r, route, taskID, eventPath, cancelTok, emitResult)
|
||||
return
|
||||
}
|
||||
st.waitIsolatedJSON(w, r, route, taskID, eventPath, cancelTok, emitResult)
|
||||
}
|
||||
|
||||
// cancelOnDisconnect reports whether the client opted into legacy
|
||||
// disconnect=cancel (header X-Marqdo-Cancel-On-Disconnect: 1|true).
|
||||
// Default is false: subscription close must not cancel the task (ADR 0009 / design §7.2).
|
||||
func cancelOnDisconnect(r *http.Request) bool {
|
||||
v := strings.TrimSpace(strings.ToLower(r.Header.Get("X-Marqdo-Cancel-On-Disconnect")))
|
||||
return v == "1" || v == "true" || v == "yes"
|
||||
}
|
||||
|
||||
func (st *state) streamIsolatedSSE(w http.ResponseWriter, r *http.Request, route invokeRoute, taskID, eventPath, cancelTok, emitResult string) {
|
||||
w.Header().Set("Content-Type", "text/event-stream; charset=utf-8")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
w.Header().Set("X-Accel-Buffering", "no")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
flusher, _ := w.(http.Flusher)
|
||||
writeEvent := func(payload any) {
|
||||
b, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
_, _ = io.WriteString(w, "data: "+string(b)+"\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
writeEvent(map[string]any{
|
||||
"schema": "RuntimeEventV1",
|
||||
"schema_version": 1,
|
||||
"type": "run.started",
|
||||
"payload": map[string]any{
|
||||
"fn": route.FnPath,
|
||||
"bridge": "isolated",
|
||||
"task_id": taskID,
|
||||
},
|
||||
})
|
||||
|
||||
offset := int64(0)
|
||||
carry := ""
|
||||
seq := uint64(1)
|
||||
poll := time.NewTicker(25 * time.Millisecond)
|
||||
defer poll.Stop()
|
||||
ctx := r.Context()
|
||||
|
||||
finish := func(clientGone bool) {
|
||||
offset, carry = drainForwardCarry(eventPath, offset, carry, func(line string) {
|
||||
seq++
|
||||
writeEvent(normalizeLiveSSELine(seq, line))
|
||||
})
|
||||
if !clientGone {
|
||||
result := readEmitResult(emitResult)
|
||||
if result == nil {
|
||||
if g, err := runtimeQuery("get", map[string]any{"task_id": taskID}); err == nil {
|
||||
if t, ok := g["task"].(map[string]any); ok {
|
||||
result = t["result"]
|
||||
}
|
||||
}
|
||||
}
|
||||
writeEvent(map[string]any{
|
||||
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.completed",
|
||||
"payload": map[string]any{"result": result, "task_id": taskID},
|
||||
})
|
||||
_, _ = io.WriteString(w, "data: [DONE]\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
doCancelOnDisconnect := cancelOnDisconnect(r)
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
// Default: observer gone ≠ cancel. Task keeps running; client may
|
||||
// reattach via GET /_runtime/tasks/{id}/events (Last-Event-ID).
|
||||
if doCancelOnDisconnect {
|
||||
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
|
||||
_ = writeCancelToken(cancelTok)
|
||||
writeEvent(map[string]any{
|
||||
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.interrupted",
|
||||
"payload": map[string]any{
|
||||
"reason": "client_disconnect",
|
||||
"cancel_propagated": cancelTok != "",
|
||||
"status": "stopping",
|
||||
"task_id": taskID,
|
||||
},
|
||||
})
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for time.Now().Before(deadline) {
|
||||
if workerDone(taskID, emitResult) {
|
||||
break
|
||||
}
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
}
|
||||
} else {
|
||||
writeEvent(map[string]any{
|
||||
"schema": "RuntimeEventV1", "schema_version": 1, "type": "run.interrupted",
|
||||
"payload": map[string]any{
|
||||
"reason": "client_disconnect",
|
||||
"cancel_propagated": false,
|
||||
"status": "observer_closed",
|
||||
"task_id": taskID,
|
||||
"resume": "/_runtime/tasks/" + taskID + "/events",
|
||||
},
|
||||
})
|
||||
// Observer gone ≠ cancel, but still reap when the worker exits so
|
||||
// execute slots are not held forever by zombies.
|
||||
go reapIsolatedTask(taskID, emitResult, 10*time.Minute)
|
||||
}
|
||||
finish(true)
|
||||
return
|
||||
case <-poll.C:
|
||||
offset, carry = drainForwardCarry(eventPath, offset, carry, func(line string) {
|
||||
seq++
|
||||
writeEvent(normalizeLiveSSELine(seq, line))
|
||||
})
|
||||
if workerDone(taskID, emitResult) {
|
||||
finish(false)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (st *state) waitIsolatedJSON(w http.ResponseWriter, r *http.Request, route invokeRoute, taskID, eventPath, cancelTok, emitResult string) {
|
||||
_ = eventPath
|
||||
poll := time.NewTicker(25 * time.Millisecond)
|
||||
defer poll.Stop()
|
||||
ctx := r.Context()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
|
||||
_ = writeCancelToken(cancelTok)
|
||||
jsonErr(w, http.StatusRequestTimeout, "client disconnected")
|
||||
return
|
||||
case <-poll.C:
|
||||
if !workerDone(taskID, emitResult) {
|
||||
continue
|
||||
}
|
||||
value := readEmitResult(emitResult)
|
||||
if value == nil {
|
||||
jsonErr(w, http.StatusInternalServerError, "worker produced no result")
|
||||
return
|
||||
}
|
||||
if m, ok := value.(map[string]any); ok {
|
||||
if s, ok := m["sse"].(string); ok && strings.TrimSpace(s) != "" {
|
||||
writeSSE(w, m)
|
||||
return
|
||||
}
|
||||
}
|
||||
switch route.ReturnMode {
|
||||
case "text", "html", "text/html":
|
||||
text := fmt.Sprint(value)
|
||||
if s, ok := value.(string); ok {
|
||||
text = s
|
||||
}
|
||||
ct := "text/plain; charset=utf-8"
|
||||
if route.ReturnMode == "html" || route.ReturnMode == "text/html" {
|
||||
ct = "text/html; charset=utf-8"
|
||||
}
|
||||
w.Header().Set("Content-Type", ct)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = io.WriteString(w, text)
|
||||
case "status":
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
default:
|
||||
code, body, err := jsonWrapInvokeValue(value)
|
||||
if err != nil {
|
||||
jsonErr(w, code, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, code, body)
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// reapIsolatedTask polls runtime get until the worker exits (or timeout),
|
||||
// so supervisor/scheduler slots are released after observer disconnect.
|
||||
func reapIsolatedTask(taskID, emitResult string, budget time.Duration) {
|
||||
deadline := time.Now().Add(budget)
|
||||
for time.Now().Before(deadline) {
|
||||
if workerDone(taskID, emitResult) {
|
||||
return
|
||||
}
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
}
|
||||
// Last-chance cancel so slots can free.
|
||||
_, _ = runtimeQuery("cancel", map[string]any{"task_id": taskID})
|
||||
for i := 0; i < 25; i++ {
|
||||
if workerDone(taskID, emitResult) {
|
||||
return
|
||||
}
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
func workerDone(taskID, emitResult string) bool {
|
||||
// Always runtime.get when probing completion: get runs try_reap and
|
||||
// releases execute slots. Do not return early on emit_result alone —
|
||||
// that left zombie workers holding both pool slots (no_slot).
|
||||
if taskID == "" {
|
||||
return false
|
||||
}
|
||||
g, err := runtimeQuery("get", map[string]any{"task_id": taskID})
|
||||
if err != nil {
|
||||
// Fallback: result file present (get may fail during early boot).
|
||||
if emitResult != "" {
|
||||
if st, err := os.Stat(emitResult); err == nil && st.Size() > 0 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
if dir := os.Getenv("MARQDO_RUNTIME_DIR"); dir != "" {
|
||||
p := filepath.Join(dir, "events", taskID+".result.json")
|
||||
if st, err := os.Stat(p); err == nil && st.Size() > 0 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
if _, ok := g["exit_code"]; ok {
|
||||
return true
|
||||
}
|
||||
if t, ok := g["task"].(map[string]any); ok {
|
||||
switch fmt.Sprint(t["state"]) {
|
||||
case "succeeded", "failed", "cancelled", "interrupted":
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func readEmitResult(path string) any {
|
||||
if path == "" {
|
||||
return nil
|
||||
}
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil || len(b) == 0 {
|
||||
return nil
|
||||
}
|
||||
var v any
|
||||
if err := json.Unmarshal(b, &v); err != nil {
|
||||
return string(b)
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
func materializeArgsJSON(args map[string]any) (string, error) {
|
||||
f, err := os.CreateTemp("", "mq-args-*.json")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
path := f.Name()
|
||||
enc := json.NewEncoder(f)
|
||||
if err := enc.Encode(args); err != nil {
|
||||
_ = f.Close()
|
||||
_ = os.Remove(path)
|
||||
return "", err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
_ = os.Remove(path)
|
||||
return "", err
|
||||
}
|
||||
return path, nil
|
||||
}
|
||||
|
||||
func resolveSiteEntry(st *state) (module string, fsRoot string, err error) {
|
||||
fsRoot = st.entryDir
|
||||
if raw, qerr := plugin.Query("entry_dir", "{}"); qerr == nil {
|
||||
var s string
|
||||
if json.Unmarshal([]byte(raw), &s) == nil && s != "" {
|
||||
fsRoot = s
|
||||
}
|
||||
}
|
||||
if raw, qerr := plugin.Query("entry_path", "{}"); qerr == nil {
|
||||
var s string
|
||||
if json.Unmarshal([]byte(raw), &s) == nil && s != "" {
|
||||
module = s
|
||||
}
|
||||
}
|
||||
if module == "" {
|
||||
// Fallback: common serve entry names under fs_root.
|
||||
for _, name := range []string{"serve.mq.md", "index.mq.md", "main.mq.md"} {
|
||||
p := filepath.Join(fsRoot, name)
|
||||
if st, e := os.Stat(p); e == nil && !st.IsDir() {
|
||||
module = p
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if module == "" {
|
||||
return "", "", fmt.Errorf("isolated execute: cannot resolve site entry module")
|
||||
}
|
||||
if !filepath.IsAbs(module) {
|
||||
abs, aerr := filepath.Abs(module)
|
||||
if aerr == nil {
|
||||
module = abs
|
||||
}
|
||||
}
|
||||
if fsRoot == "" {
|
||||
fsRoot = filepath.Dir(module)
|
||||
}
|
||||
if !filepath.IsAbs(fsRoot) {
|
||||
abs, aerr := filepath.Abs(fsRoot)
|
||||
if aerr == nil {
|
||||
fsRoot = abs
|
||||
}
|
||||
}
|
||||
return module, fsRoot, nil
|
||||
}
|
||||
|
||||
// normalizeLiveSSELine maps Marqdo EventBus frames onto RuntimeEventV1.
|
||||
func normalizeLiveSSELine(seq uint64, line string) map[string]any {
|
||||
base := map[string]any{
|
||||
"schema": "RuntimeEventV1", "schema_version": 1, "seq": seq,
|
||||
}
|
||||
var obj map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &obj); err != nil || obj == nil {
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = map[string]any{"raw": line}
|
||||
return base
|
||||
}
|
||||
if t, _ := obj["type"].(string); t == "message.delta" || t == "turn.started" || t == "model.resolved" || t == "usage.updated" || t == "turn.completed" || t == "turn.failed" {
|
||||
for k, v := range obj {
|
||||
base[k] = v
|
||||
}
|
||||
base["seq"] = seq
|
||||
return base
|
||||
}
|
||||
if t, _ := obj["type"].(string); t == "delta" || t == "plan" || t == "reasoning" {
|
||||
text, _ := obj["text"].(string)
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = map[string]any{"delta": text, "kind": t}
|
||||
return base
|
||||
}
|
||||
if t, _ := obj["type"].(string); t == "decompose" || t == "decision" || t == "await" || t == "round" || t == "done" || t == "tool_start" || t == "tool_end" || t == "plan_act" {
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = obj
|
||||
return base
|
||||
}
|
||||
if text, ok := obj["text"].(string); ok && text != "" {
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = map[string]any{"delta": text}
|
||||
return base
|
||||
}
|
||||
base["type"] = "message.delta"
|
||||
base["payload"] = obj
|
||||
return base
|
||||
}
|
||||
|
||||
func drainForward(path string, offset int64, onLine func(string)) int64 {
|
||||
off, _ := drainForwardCarry(path, offset, "", onLine)
|
||||
return off
|
||||
}
|
||||
|
||||
const drainForwardMaxLine = 1024 * 1024
|
||||
|
||||
func drainForwardCarry(path string, offset int64, carry string, onLine func(string)) (int64, string) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return offset, carry
|
||||
}
|
||||
defer f.Close()
|
||||
if _, err := f.Seek(offset, io.SeekStart); err != nil {
|
||||
return offset, carry
|
||||
}
|
||||
chunk, err := io.ReadAll(io.LimitReader(f, drainForwardMaxLine+1))
|
||||
if err != nil && err != io.EOF {
|
||||
onLine(fmt.Sprintf(`{"type":"protocol.error","error":%q}`, err.Error()))
|
||||
return offset, carry
|
||||
}
|
||||
if len(chunk) == 0 {
|
||||
return offset, carry
|
||||
}
|
||||
offset += int64(len(chunk))
|
||||
buf := carry + string(chunk)
|
||||
for {
|
||||
i := strings.IndexByte(buf, '\n')
|
||||
if i < 0 {
|
||||
if len(buf) > drainForwardMaxLine {
|
||||
onLine(`{"type":"protocol.error","error":"ndjson line exceeds 1MiB"}`)
|
||||
return offset, ""
|
||||
}
|
||||
return offset, buf
|
||||
}
|
||||
lineBytes := buf[:i]
|
||||
line := strings.TrimSpace(strings.TrimSuffix(lineBytes, "\r"))
|
||||
buf = buf[i+1:]
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
onLine(line)
|
||||
}
|
||||
}
|
||||
|
||||
func legacySSEBridgeSnapshot() (active bool, forward string, cancelTok string) {
|
||||
return legacySSEBridgeActive.Load() != 0, os.Getenv("MARQDO_EVENT_FORWARD"), os.Getenv("MARQDO_CANCEL_TOKEN")
|
||||
}
|
||||
|
||||
func writeCancelToken(path string) error {
|
||||
if path == "" {
|
||||
return nil
|
||||
}
|
||||
return os.WriteFile(path, []byte("cancelled"), 0o600)
|
||||
}
|
||||
@@ -81,7 +81,16 @@ func (st *state) handleUpload(w http.ResponseWriter, r *http.Request, routePath
|
||||
_, hadCookie := session.IDFromCookie(r.Header.Get("Cookie"))
|
||||
sid, _, setCookie := resolveSession(r)
|
||||
|
||||
if err := r.ParseMultipartForm(32 << 20); err != nil {
|
||||
// M01: hard-limit request body before multipart parse (32MiB was only in-memory
|
||||
// form budget, not a request total). Cap at configured MaxBytes + 1MiB overhead.
|
||||
maxBody := int64(cfg.MaxBytes)
|
||||
if maxBody <= 0 {
|
||||
maxBody = 32 << 20
|
||||
}
|
||||
overhead := int64(1 << 20)
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxBody+overhead)
|
||||
|
||||
if err := r.ParseMultipartForm(maxBody + overhead); err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("multipart: %v", err))
|
||||
return
|
||||
}
|
||||
@@ -109,11 +118,20 @@ func (st *state) handleUpload(w http.ResponseWriter, r *http.Request, routePath
|
||||
return
|
||||
}
|
||||
defer f.Close()
|
||||
b, err := io.ReadAll(f)
|
||||
// Stream-capped read: never buffer more than MaxBytes.
|
||||
lim := maxBody
|
||||
if lim <= 0 {
|
||||
lim = 32 << 20
|
||||
}
|
||||
b, err := io.ReadAll(io.LimitReader(f, lim+1))
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("read file: %v", err))
|
||||
return
|
||||
}
|
||||
if int64(len(b)) > lim {
|
||||
jsonErr(w, http.StatusRequestEntityTooLarge, "file exceeds max_bytes")
|
||||
return
|
||||
}
|
||||
fileBytes = b
|
||||
} else {
|
||||
// Fallback: walk File map keys.
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestHandleUploadRejectsOversizedBeforeFullBuffer(t *testing.T) {
|
||||
st := &state{
|
||||
uploadRoutes: map[string]any{
|
||||
"/upload": map[string]any{
|
||||
"field": "file",
|
||||
"storage_url": "memory://t",
|
||||
"prefix": "u",
|
||||
"max_bytes": float64(64),
|
||||
},
|
||||
},
|
||||
}
|
||||
cfg := parseUploadRoute(st.uploadRoutes["/upload"])
|
||||
var body bytes.Buffer
|
||||
w := multipart.NewWriter(&body)
|
||||
part, err := w.CreateFormFile("file", "big.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := part.Write([]byte(strings.Repeat("x", 200))); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = w.Close()
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/upload", &body)
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
rr := httptest.NewRecorder()
|
||||
st.handleUpload(rr, req, "/upload", cfg)
|
||||
if rr.Code == http.StatusOK {
|
||||
t.Fatalf("expected reject for oversized upload, got %d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if rr.Code != http.StatusRequestEntityTooLarge && rr.Code != http.StatusBadRequest {
|
||||
t.Fatalf("expected 413 or 400, got %d body=%s", rr.Code, rr.Body.String())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,207 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/plugin"
|
||||
)
|
||||
|
||||
// System runtime control routes (M-R3 / M-R4). Cancel and event replay do not
|
||||
// enter the Marqdo interpreter lock.
|
||||
func (st *state) mountRuntimeRoutes(mux *http.ServeMux) {
|
||||
mux.HandleFunc("POST /_runtime/tasks", st.handleRuntimeAdmit)
|
||||
mux.HandleFunc("GET /_runtime/tasks/{id}", st.handleRuntimeGet)
|
||||
mux.HandleFunc("POST /_runtime/tasks/{id}/cancel", st.handleRuntimeCancel)
|
||||
mux.HandleFunc("GET /_runtime/tasks/{id}/events", st.handleRuntimeEvents)
|
||||
}
|
||||
|
||||
func runtimeQuery(op string, extra map[string]any) (map[string]any, error) {
|
||||
payload := map[string]any{"op": op}
|
||||
for k, v := range extra {
|
||||
payload[k] = v
|
||||
}
|
||||
raw, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out, err := plugin.Query("runtime", string(raw))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal([]byte(out), &m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func (st *state) handleRuntimeAdmit(w http.ResponseWriter, r *http.Request) {
|
||||
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 1<<20))
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
var req map[string]any
|
||||
if len(body) > 0 {
|
||||
if err := json.Unmarshal(body, &req); err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("json: %v", err))
|
||||
return
|
||||
}
|
||||
}
|
||||
if req == nil {
|
||||
req = map[string]any{}
|
||||
}
|
||||
taskID, _ := req["task_id"].(string)
|
||||
if taskID == "" {
|
||||
taskID = fmt.Sprintf("task_%d", time.Now().UnixNano())
|
||||
req["task_id"] = taskID
|
||||
}
|
||||
if _, ok := req["idempotency_key"]; !ok {
|
||||
req["idempotency_key"] = taskID
|
||||
}
|
||||
if _, ok := req["class"]; !ok {
|
||||
req["class"] = "execute"
|
||||
}
|
||||
m, err := runtimeQuery("admit", req)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "conflict") || strings.Contains(fmt.Sprint(m["error"]), "conflict") {
|
||||
jsonErr(w, http.StatusConflict, err.Error())
|
||||
return
|
||||
}
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
if ok, _ := m["ok"].(bool); !ok {
|
||||
jsonErr(w, http.StatusConflict, fmt.Sprint(m["error"]))
|
||||
return
|
||||
}
|
||||
w.Header().Set("Location", "/_runtime/tasks/"+taskID)
|
||||
writeJSON(w, http.StatusAccepted, m)
|
||||
}
|
||||
|
||||
func (st *state) handleRuntimeGet(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
m, err := runtimeQuery("get", map[string]any{"task_id": id})
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
|
||||
jsonErr(w, http.StatusNotFound, "task not found")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
}
|
||||
|
||||
func (st *state) handleRuntimeCancel(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
m, err := runtimeQuery("cancel", map[string]any{"task_id": id})
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
|
||||
jsonErr(w, http.StatusNotFound, "task not found")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
}
|
||||
|
||||
func (st *state) handleRuntimeEvents(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
after := int64(0)
|
||||
if s := r.URL.Query().Get("after_seq"); s != "" {
|
||||
fmt.Sscanf(s, "%d", &after)
|
||||
}
|
||||
if last := strings.TrimSpace(r.Header.Get("Last-Event-ID")); last != "" {
|
||||
// Accept "task:seq" or bare seq.
|
||||
if i := strings.LastIndexByte(last, ':'); i >= 0 {
|
||||
fmt.Sscanf(last[i+1:], "%d", &after)
|
||||
} else {
|
||||
fmt.Sscanf(last, "%d", &after)
|
||||
}
|
||||
}
|
||||
wantSSE := strings.Contains(r.Header.Get("Accept"), "text/event-stream") ||
|
||||
r.URL.Query().Get("stream") == "1"
|
||||
|
||||
if !wantSSE {
|
||||
m, err := runtimeQuery("events", map[string]any{"task_id": id, "after_seq": after})
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
if st, _ := m["status"].(float64); st == 404 || fmt.Sprint(m["error"]) == "not_found" {
|
||||
jsonErr(w, http.StatusNotFound, "task not found")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/event-stream; charset=utf-8")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
flusher, _ := w.(http.Flusher)
|
||||
cursor := after
|
||||
ticker := time.NewTicker(25 * time.Millisecond)
|
||||
defer ticker.Stop()
|
||||
heartbeat := time.NewTicker(15 * time.Second)
|
||||
defer heartbeat.Stop()
|
||||
ctx := r.Context()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-heartbeat.C:
|
||||
_, _ = io.WriteString(w, ": ping\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
case <-ticker.C:
|
||||
m, err := runtimeQuery("events", map[string]any{"task_id": id, "after_seq": cursor})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if st, _ := m["status"].(float64); st == 404 {
|
||||
_, _ = io.WriteString(w, "event: error\ndata: {\"error\":\"not_found\"}\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
return
|
||||
}
|
||||
evs, _ := m["events"].([]any)
|
||||
for _, raw := range evs {
|
||||
b, err := json.Marshal(raw)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
var obj map[string]any
|
||||
_ = json.Unmarshal(b, &obj)
|
||||
eid := ""
|
||||
if s, ok := obj["event_id"].(string); ok {
|
||||
eid = s
|
||||
}
|
||||
if seq, ok := obj["seq"].(float64); ok {
|
||||
cursor = int64(seq)
|
||||
}
|
||||
etype, _ := obj["type"].(string)
|
||||
if etype == "" {
|
||||
etype = "message"
|
||||
}
|
||||
if eid != "" {
|
||||
_, _ = io.WriteString(w, "id: "+eid+"\n")
|
||||
}
|
||||
_, _ = io.WriteString(w, "event: "+etype+"\ndata: "+string(b)+"\n\n")
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSystemRuntimeRoutesListed(t *testing.T) {
|
||||
// Compile-time presence: handlers must be addressable.
|
||||
var st state
|
||||
_ = st.handleRuntimeAdmit
|
||||
_ = st.handleRuntimeCancel
|
||||
_ = st.handleRuntimeEvents
|
||||
_ = st.handleRuntimeGet
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// M-R0: second stream must fail before rewriting A's forward/cancel env.
|
||||
func TestLegacySSEBridgeSingleFlight(t *testing.T) {
|
||||
if !tryAcquireLegacySSEBridge() {
|
||||
t.Fatal("expected first acquire to succeed")
|
||||
}
|
||||
fwdA := filepath.Join(t.TempDir(), "a.ndjson")
|
||||
tokA := filepath.Join(t.TempDir(), "a.tok")
|
||||
if err := os.WriteFile(fwdA, nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(tokA, nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("MARQDO_EVENT_FORWARD", fwdA)
|
||||
t.Setenv("MARQDO_CANCEL_TOKEN", tokA)
|
||||
legacySSEBridgeActive.Store(1)
|
||||
|
||||
if tryAcquireLegacySSEBridge() {
|
||||
releaseLegacySSEBridge()
|
||||
t.Fatal("second acquire must fail while bridge held")
|
||||
}
|
||||
active, fwd, tok := legacySSEBridgeSnapshot()
|
||||
if !active {
|
||||
t.Fatal("bridge should remain active for holder A")
|
||||
}
|
||||
if fwd != fwdA || tok != tokA {
|
||||
t.Fatalf("rejected second stream rewrote env: fwd=%q tok=%q", fwd, tok)
|
||||
}
|
||||
|
||||
releaseLegacySSEBridge()
|
||||
if tryAcquireLegacySSEBridge() {
|
||||
releaseLegacySSEBridge()
|
||||
} else {
|
||||
t.Fatal("bridge should be free after release")
|
||||
}
|
||||
}
|
||||
|
||||
// M-R0: cancel-token file must not be deleted while bridge holder waits.
|
||||
// (sse-live observer disconnect no longer cancels by default — see cancelOnDisconnect.)
|
||||
func TestCancelTokenSurvivesUntilHolderExits(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
tok := filepath.Join(dir, "cancel.tok")
|
||||
if err := os.WriteFile(tok, nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := writeCancelToken(tok); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body, err := os.ReadFile(tok)
|
||||
if err != nil {
|
||||
t.Fatalf("token must still exist after cancel mark: %v", err)
|
||||
}
|
||||
if string(body) != "cancelled" {
|
||||
t.Fatalf("body=%q", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDrainForwardCarryHalfFrames(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "evt.ndjson")
|
||||
if err := os.WriteFile(path, []byte(`{"t":"部`), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var lines []string
|
||||
off, carry := drainForwardCarry(path, 0, "", func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 0 {
|
||||
t.Fatalf("incomplete UTF-8 frame must stay in carry, got %#v", lines)
|
||||
}
|
||||
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.WriteString("分\"}\n{\"t\":\"ok\"}\n"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = f.Close()
|
||||
off, carry = drainForwardCarry(path, off, carry, func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 2 {
|
||||
t.Fatalf("got %d lines %#v carry=%q off=%d", len(lines), lines, carry, off)
|
||||
}
|
||||
if carry != "" {
|
||||
t.Fatalf("carry should be empty, got %q", carry)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDrainForwardCarryCRLFAndSplit(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "evt.ndjson")
|
||||
part1 := []byte("{\"a\":1}\r\n{\"b\":")
|
||||
if err := os.WriteFile(path, part1, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var lines []string
|
||||
off, carry := drainForwardCarry(path, 0, "", func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 1 || lines[0] != `{"a":1}` {
|
||||
t.Fatalf("first frame %#v", lines)
|
||||
}
|
||||
f, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.WriteString("2}\n"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = f.Close()
|
||||
_, carry = drainForwardCarry(path, off, carry, func(line string) {
|
||||
lines = append(lines, line)
|
||||
})
|
||||
if len(lines) != 2 || lines[1] != `{"b":2}` {
|
||||
t.Fatalf("split frame %#v carry=%q", lines, carry)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacySSEBridgeConcurrentReject(t *testing.T) {
|
||||
var wg sync.WaitGroup
|
||||
acquired := 0
|
||||
var mu sync.Mutex
|
||||
for i := 0; i < 8; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if tryAcquireLegacySSEBridge() {
|
||||
mu.Lock()
|
||||
acquired++
|
||||
mu.Unlock()
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
releaseLegacySSEBridge()
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if acquired < 1 {
|
||||
t.Fatal("expected at least one acquirer")
|
||||
}
|
||||
}
|
||||
@@ -16,9 +16,10 @@ type Options struct {
|
||||
Fn string
|
||||
Body string
|
||||
Return string
|
||||
Class string
|
||||
}
|
||||
|
||||
// RoutesFromTable normalizes `|路径|方法|函数|正文|返回|` into invoke_routes map keys.
|
||||
// RoutesFromTable normalizes `|路径|方法|函数|正文|返回|类别|` into invoke_routes map keys.
|
||||
func RoutesFromTable(tableV any) map[string]any {
|
||||
rows, _ := table.AsRows(tableV).([]any)
|
||||
out := map[string]any{}
|
||||
@@ -47,7 +48,8 @@ func RoutesFromTable(tableV any) map[string]any {
|
||||
if ret == "" {
|
||||
ret = "json"
|
||||
}
|
||||
out[path] = routeEntry(method, fnPath, body, ret)
|
||||
class := strings.ToLower(strings.TrimSpace(col(m, "类别", "class", "Class")))
|
||||
out[path] = routeEntry(method, fnPath, body, ret, class)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -78,12 +80,13 @@ func Register(appBag map[string]any, opts Options) (map[string]any, error) {
|
||||
if ret == "" {
|
||||
ret = "json"
|
||||
}
|
||||
class := strings.ToLower(strings.TrimSpace(opts.Class))
|
||||
out := clone(appBag)
|
||||
routes := map[string]any{}
|
||||
if r, ok := out["invoke_routes"].(map[string]any); ok {
|
||||
routes = clone(r)
|
||||
}
|
||||
routes[key] = routeEntry(method, fnPath, body, ret)
|
||||
routes[key] = routeEntry(method, fnPath, body, ret, class)
|
||||
out["invoke_routes"] = routes
|
||||
return out, nil
|
||||
}
|
||||
@@ -101,22 +104,28 @@ func OptionsFromArgs(args map[string]any) (Options, error) {
|
||||
method, _ := argStr(args, "method", "方法")
|
||||
body, _ := argStr(args, "body", "正文")
|
||||
ret, _ := argStr(args, "return", "返回")
|
||||
class, _ := argStr(args, "class", "类别")
|
||||
return Options{
|
||||
Path: path,
|
||||
Method: method,
|
||||
Fn: fnPath,
|
||||
Body: body,
|
||||
Return: ret,
|
||||
Class: class,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func routeEntry(method, fnPath, body, ret string) map[string]any {
|
||||
return map[string]any{
|
||||
func routeEntry(method, fnPath, body, ret, class string) map[string]any {
|
||||
m := map[string]any{
|
||||
"method": method,
|
||||
"fn": fnPath,
|
||||
"body": body,
|
||||
"return": ret,
|
||||
}
|
||||
if class != "" {
|
||||
m["class"] = class
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func col(m map[string]any, keys ...string) string {
|
||||
|
||||
+37
-1
@@ -80,6 +80,7 @@ extern int web_compose_form(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_app_new(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_app_route(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_update(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_update_if(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_delete(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_query(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_db_count(char *args_json, char **out_json, char **err_msg);
|
||||
@@ -183,6 +184,7 @@ static int register_core(void) {
|
||||
if (host_register((char *)"web_db_select", (char *)"url,table,where,limit,order,offset,txn", web_db_select) != 0) return 1;
|
||||
if (host_register((char *)"web_db_get", (char *)"url,table,id,txn", web_db_get) != 0) return 1;
|
||||
if (host_register((char *)"web_db_update", (char *)"url,table,id,row,txn", web_db_update) != 0) return 1;
|
||||
if (host_register((char *)"web_db_update_if", (char *)"url,table,id,row,expect_col,expect_val,txn", web_db_update_if) != 0) return 1;
|
||||
if (host_register((char *)"web_db_delete", (char *)"url,table,id,txn", web_db_delete) != 0) return 1;
|
||||
if (host_register((char *)"web_db_exec", (char *)"url,sql,args,txn", web_db_exec) != 0) return 1;
|
||||
if (host_register((char *)"web_db_query", (char *)"url,sql,args,txn", web_db_query) != 0) return 1;
|
||||
@@ -264,7 +266,7 @@ static int register_core(void) {
|
||||
if (host_register((char *)"web_cache_exists", (char *)"url,key", web_cache_exists) != 0) return 1;
|
||||
if (host_register((char *)"web_cache_ttl", (char *)"url,key", web_cache_ttl) != 0) return 1;
|
||||
if (host_register((char *)"web_app_proxy", (char *)"app,path,upstream,stream,strip_prefix,methods,headers_from_env,timeout_ms", web_app_proxy) != 0) return 1;
|
||||
if (host_register((char *)"web_app_invoke", (char *)"app,path,method,fn,body,return", web_app_invoke) != 0) return 1;
|
||||
if (host_register((char *)"web_app_invoke", (char *)"app,path,method,fn,body,return,class", web_app_invoke) != 0) return 1;
|
||||
if (host_register((char *)"web_api_key_check", (char *)"key,authorization,pepper,keys", web_api_key_check) != 0) return 1;
|
||||
if (host_register((char *)"web_render_nodes", (char *)"page,nodes,db,title", web_render_nodes) != 0) return 1;
|
||||
if (host_register((char *)"web_render_document", (char *)"page,body,db,title", web_render_document) != 0) return 1;
|
||||
@@ -994,6 +996,40 @@ func web_db_update(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
return replyJSON(outJSON, errMsg, out, err)
|
||||
}
|
||||
|
||||
//export web_db_update_if
|
||||
func web_db_update_if(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
url, err := dbURLOf(args)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
tableName, err := argStrReq(args, "table")
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
id, err := argStrReq(args, "id")
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
expectCol, err := argStrReq(args, "expect_col")
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
expectVal, ok := args["expect_val"]
|
||||
if !ok {
|
||||
return replyJSON(outJSON, errMsg, nil, fmt.Errorf("missing expect_val"))
|
||||
}
|
||||
row := args["row"]
|
||||
if row == nil {
|
||||
row = map[string]any{}
|
||||
}
|
||||
out, err := db.UpdateIf(url, tableName, id, row, expectCol, expectVal, txnOpt(args))
|
||||
return replyJSON(outJSON, errMsg, out, err)
|
||||
}
|
||||
|
||||
//export web_db_delete
|
||||
func web_db_delete(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
|
||||
@@ -17,7 +17,9 @@ The language is still early: structural markers, English/Chinese builtin aliases
|
||||
|
||||
中文文档请看同目录的 [`00-欢迎.mq.md`](00-欢迎.mq.md)(叙述与内置均用中文)。
|
||||
|
||||
Want a quick hello? Calls only run when they sit in the entry `# main` — `##` children are methods until something calls them. Try **greet who="marqdo"**. Names can include spaces: **greet who="marqdo user"**.
|
||||
Want a quick hello? `marqdo run` executes the entry `# main`. Try **greet who="marqdo"**. Names can include spaces: **greet who="marqdo user"**.
|
||||
|
||||
This version also runs one function on its own. `marqdo call` takes the file, the function and named arguments. The page you read is the program that runs. What that is for, and why it matters, is in [features/22-call.mq.md](features/22-call.mq.md).
|
||||
|
||||
Beyond the basics, Marqdo ships table, time, text, files, system, math, foreign, and more. Math and foreign deserve a special mention.
|
||||
|
||||
|
||||
+3
-1
@@ -17,7 +17,9 @@ AI 偏爱 Markdown 结构的输出。我当时的念头很简单——何必再
|
||||
|
||||
English docs: [`00-welcome.mq.md`](00-welcome.mq.md)(English prose and English builtins).
|
||||
|
||||
想先打个招呼?调用写在入口 `# main` 里才会真的跑——子标题里的方法要等你叫到它。比如:**问候 谁="marqdo"**。名字里可以带空格:**问候 谁="marqdo user"**。
|
||||
想先打个招呼?`marqdo run` 会运行入口 `# main`。比如:**问候 谁="marqdo"**。名字里可以带空格:**问候 谁="marqdo user"**。
|
||||
|
||||
从这一版起,不必为了试用一个函数再包一层入口。`marqdo call` 直接运行文件里的某个函数,参数写在命令上。同一页仍然是给人读的文档。作用和价值写在 [features/22-函数调用.mq.md](features/22-函数调用.mq.md)。
|
||||
|
||||
除了这些基本面,标准库里还有表、时间、文本、文件、系统、数学、外联等。我特别想提数学和外联。
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@
|
||||
| [`features/19-foreign-view-cells.mq.md`](features/19-foreign-view-cells.mq.md) / [`19-外联视图单元.mq.md`](features/19-外联视图单元.mq.md) | 外联 v2 完整 Python 示例:probe / JSON call / 并行 / 持久内核 / 状态桥 / view 单元 |
|
||||
| [`features/20-okf-ekc.mq.md`](features/20-okf-ekc.mq.md) / [`20-OKF与EKC.mq.md`](features/20-OKF与EKC.mq.md) | 代码即文档、文档即知识:OKF 形态与 EKC 工程知识编译器 |
|
||||
| [`features/21-persistence.mq.md`](features/21-persistence.mq.md) / [`21-持久化.mq.md`](features/21-持久化.mq.md) | 外联结果持久化:静态快照与 writeback |
|
||||
| [`features/22-call.mq.md`](features/22-call.mq.md) / [`22-函数调用.mq.md`](features/22-函数调用.mq.md) | `marqdo call`:同一份文档按函数执行,知识可以直接当工具 |
|
||||
|
||||
官方扩展(非 stdlib):`ext/llm`、`ext/agent`、`ext/web`、`ext/quantum`、`ext/linalg` — 见 `features/05-*.mq.md`、`06-*.mq.md`、`07-linalg.mq.md` / `07-线性代数.mq.md` 与 `doc/design/ext-*.md`。
|
||||
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
markup: occlusion
|
||||
title: Call one function: the same document is the tool
|
||||
description: v2.2.0 marqdo call runs a function inside any .mq.md. The explanation, the implementation and the call are one page.
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
v2.2.0 adds one command: `marqdo call`. It adds no Markdown marker and no skill system. An ordinary .mq.md file is already a document, a piece of knowledge and a program. What was missing is a way to call one of its functions from outside.
|
||||
|
||||
The usual command runs the whole file. Functions inside it run only when the entry calls them. Trying one calculation meant writing an entry, a print and a check. The smallest unit you can execute is now a function.
|
||||
|
||||
```text
|
||||
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3
|
||||
```
|
||||
|
||||
Standard output is 5. `--json` is for programs and agents: success carries the result, failure carries a diagnostic code, and the process exits non-zero. `--backend tree` and `--backend bytecode` both run that function. Another entry in the file is not used as a fallback.
|
||||
|
||||
The second half of this page is itself a calculator. This command calls this page:
|
||||
|
||||
```text
|
||||
marqdo call public/features/22-call.mq.md calculator.add a=2 b=3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
The change is how the file is used.
|
||||
|
||||
A person reads the sentences next to the function and sees what add computes. The compiler reads the same markers and checks parameters and the return. A developer composes the function into another program. A test passes inputs to this function and looks at the output, without a wrapper whose only job is to print. An agent reads what the function is for and runs the implementation that is already there, instead of reading the page and rewriting it.
|
||||
|
||||
Those four uses are one call. Knowledge becomes a tool because knowledge can already run, now at function scope. The file does not need a special name, and it does not need to declare itself a skill.
|
||||
|
||||
---
|
||||
|
||||
The command keeps a boundary.
|
||||
|
||||
An external call runs a pure function. Writing files, starting a process, using the network or loading a plugin is refused before the function runs, so the side effect does not happen. Wider permission stays on the checks the project already has. An instance method needs its receiver; this command does not invent one.
|
||||
|
||||
The function must resolve to exactly one declaration. No match, or one short name that hits two functions, is a structured error. The command does not pick the first hit. Arguments bind through the signature and contract that the function already has. Missing, unknown and ill-typed arguments are diagnostics.
|
||||
|
||||
After editing a function, call it again with the inputs that failed. A passing check is a reason to keep the edit. A test does not prove every input. It gives the edit a check that can be repeated.
|
||||
|
||||
# calculator
|
||||
|
||||
This is an ordinary Marqdo document.
|
||||
|
||||
It holds a little reusable arithmetic.
|
||||
|
||||
## add
|
||||
|
||||
Add the numbers `a` and `b`.
|
||||
|
||||
*[result](a + b)*
|
||||
|
||||
## multiply
|
||||
|
||||
Multiply the numbers `a` and `b`.
|
||||
|
||||
*[result](a * b)*
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
markup: occlusion
|
||||
title: 函数级调用:同一份文档成为工具
|
||||
description: v2.2.0 的 marqdo call 让任意 .mq.md 里的函数可以带参数执行。说明、实现和调用是同一页。
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
v2.2.0 增加一条命令:`marqdo call`。它不新增 Markdown 标记,也不另建一套技能系统。一个普通的 .mq.md 本来就是文档、知识和程序。缺的是从外部直接调用其中某个函数。
|
||||
|
||||
以前日常用法是运行整份文件。入口对象跑完,里面的函数要等文件自己去调用。要试验一个计算,往往得再写入口、打印和检查。现在最小可执行单位是函数。
|
||||
|
||||
```text
|
||||
marqdo call examples/calculator.mq.md 计算器.加法 a=2 b=3
|
||||
```
|
||||
|
||||
标准输出是 5。`--json` 给程序和 Agent 读:成功时是带结果的对象,失败时是带诊断码的对象,退出码非 0。`--backend tree` 与 `--backend bytecode` 执行同一个函数。文件里即使还有别的入口,也不会改去跑那个入口。
|
||||
|
||||
本页后半本身就是一份计算器。下面这条命令调用的就是这一页:
|
||||
|
||||
```text
|
||||
marqdo call public/features/22-函数调用.mq.md 计算器.加法 a=2 b=3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
它解决的是使用方式,不是再写一份说明书。
|
||||
|
||||
人打开这一页,读函数旁边的句子,知道加法在算什么。编译器解析同一批标记,检查参数和返回。开发者把函数嵌进别的程序。测试对这个函数传入输入,看输出是不是期望值,不必为它包一层只用来打印的程序。Agent 读到的是函数的用途,执行的是已经写好的实现,而不是读完说明再临时重写一遍。
|
||||
|
||||
这四件事共用一次调用。知识成为工具,是「知识即可执行」在函数这一层上的结果。文件不需要改名,也不需要声明自己是技能。运行时和工具清单都不会把一页普通文档挡在门外。
|
||||
|
||||
---
|
||||
|
||||
边界也写在命令里。
|
||||
|
||||
外部调用默认只执行纯函数。写文件、起进程、访问网络、加载插件,会在运行前被拒绝,副作用不会发生。要放宽权限,仍走项目里已有的验证,而不是在这条命令上新开一个开关。实例方法需要接收者,不能靠这条命令凭空调用。
|
||||
|
||||
函数名必须能唯一对上。没有这个函数,或同一个短名指向两个函数,都会返回结构化错误,不会擅自挑第一个。参数按函数已有的签名和契约绑定:缺了、多了、类型不对,都是诊断,不是静默猜一个值。
|
||||
|
||||
改一个函数之后,用原来的输入再调用一次。结果对了,这个实现就留下来;不对,就还是原来的实现。测试不能证明所有输入都正确。它让修改有一个可以重复的检查点。
|
||||
|
||||
# 计算器
|
||||
|
||||
这是一个普通的 Marqdo 文档。
|
||||
|
||||
其中包含一些可以复用的数学计算知识。
|
||||
|
||||
## 加法
|
||||
|
||||
计算两个数字`a`和`b`的和。
|
||||
|
||||
*[计算结果](a + b)*
|
||||
|
||||
## 乘法
|
||||
|
||||
计算两个数字`a`和`b`的积。
|
||||
|
||||
*[计算结果](a * b)*
|
||||
@@ -27,6 +27,13 @@ need curl curl
|
||||
need tar tar
|
||||
need xz xz-utils
|
||||
|
||||
# /tmp is often a small tmpfs; vendor + Rust extract exceed quota. Prefer disk.
|
||||
if [ -z "${TMPDIR:-}" ] || [ "$TMPDIR" = /tmp ] || [ "$TMPDIR" = /var/tmp ]; then
|
||||
TMPDIR="${HOME}/.cache/marqdo-ppa/work"
|
||||
fi
|
||||
mkdir -p "$TMPDIR"
|
||||
export TMPDIR
|
||||
|
||||
VER="$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)"
|
||||
# Debian revision comes from debian/changelog (e.g. 1.0.2-1ppa2); we append ~series.
|
||||
CHG_VER="$(dpkg-parsechangelog -l debian/changelog -S Version)"
|
||||
@@ -43,7 +50,7 @@ if [[ "$BASE_DEB_VER" != "$VER"-* ]]; then
|
||||
fi
|
||||
fi
|
||||
DEB_VER="${BASE_DEB_VER}~${SERIES}"
|
||||
PKG_ROOT="${TMPDIR:-/tmp}/marqdo-ppa/${VER}"
|
||||
PKG_ROOT="${TMPDIR}/marqdo-ppa/${VER}"
|
||||
BUILD_AREA="${ROOT}/../build-area"
|
||||
GPG_KEY="${MARQDO_PPA_GPG_KEY:-505943294D04C803}"
|
||||
RUST_VER="${MARQDO_PPA_RUST_VERSION:-1.85.0}"
|
||||
@@ -212,18 +219,20 @@ PY
|
||||
exit 1
|
||||
fi
|
||||
# Launchpad builds need only rustc + cargo + rust-std (debian/rules PATH).
|
||||
# Skip docs/json-docs/rustfmt/rls — ~1G of dead weight in the orig tarball.
|
||||
# install.sh's --without is a *single* comma-separated valopt; repeating
|
||||
# --without=… keeps only the last flag, so rust-docs still installed and
|
||||
# blew /tmp quota. Prefer explicit --components= instead.
|
||||
bash "$INSTALL_SH" \
|
||||
--prefix="$PKG_ROOT/third_party/rust" \
|
||||
--without=rust-docs \
|
||||
--without=rust-docs-json-preview \
|
||||
--without=rustfmt-preview \
|
||||
--without=rls-preview \
|
||||
--components="rustc,cargo,rust-std-${RUST_TRIPLE}" \
|
||||
--disable-ldconfig
|
||||
# Belt: docs must never ship in .orig (even if component list drifts).
|
||||
rm -rf "$PKG_ROOT/third_party/rust/share/doc"
|
||||
rm -rf "$RUST_EXTRACT"
|
||||
trap - EXIT
|
||||
"$PKG_ROOT/third_party/rust/bin/rustc" --version
|
||||
"$PKG_ROOT/third_party/rust/bin/cargo" --version
|
||||
du -sh "$PKG_ROOT/third_party/rust"
|
||||
|
||||
cd "$(dirname "$PKG_ROOT")"
|
||||
echo "==> packing ${ORIG_NAME} (vendor + third_party/rust)"
|
||||
|
||||
+57
-56
@@ -14,65 +14,66 @@ description: >-
|
||||
|
||||
Marqdo = **Markdown markers as syntax**. A `.mq.md` file is both documentation and a runnable program. Read this skill **before** inventing syntax from Markdown habits or Python.
|
||||
|
||||
Canonical design (repo): `doc/design/markdown-mapping.md`, `doc/design/keywords.md`. Deeper tables: [reference.md](reference.md). Copy-paste patterns: [examples.md](examples.md).
|
||||
<!-- CORE_SYNTAX_V2.1: must stay at the top so skill_brief budgets never truncate it -->
|
||||
|
||||
**Authoring / library craft** (GFM-first, no json glue, how to develop Marqdo): use **[marqdo-dev](../marqdo-dev/SKILL.md)** whenever writing or refactoring `.mq.md`, `lib/*`, or `ext/*`.
|
||||
## CORE SYNTAX (v2.1 Occlusion — mandatory)
|
||||
|
||||
**Product release** (tag, CHANGELOG, README, public docs, VSIX, GitHub assets): use [marqdo-release](../marqdo-release/SKILL.md) — detect latest version, **ask the user for the next SemVer**, then follow that playbook. Do not invent a version number.
|
||||
These rules are the **current default**. Do **not** teach Classic `[key](map)` index or “bold is return of print”. Classic migration only: [migration-classic.md](migration-classic.md).
|
||||
|
||||
1. **File suffix** `.mq.md`. Print with `> print text=…` / `> 打印 内容=…` — never use `**…**` to print.
|
||||
2. **`**…**` = statement** (assign / call / expression). `*…*` = return. Closing markers touch the last token (no trailing space).
|
||||
3. **`[描述](payload)` = occlusion:** `[]` is prose (never executed); `()` is the only executed part. Empty `()` → `None` stub.
|
||||
4. **Read collections with `access(coll, key)`** (aliases `取` / `取值`). **Never** use `[key](map)` as index under default Occlusion.
|
||||
5. **Foreach:** `- [描述](遍历 coll 为 item)` (`遍历`/`for`, `为`/`as`/`in`). Prefer explicit traversal when generating new code.
|
||||
6. **Import / call:** frontmatter `import:` / `导入:`; call `lib.member`. One language (EN or ZH) per file.
|
||||
7. **Control flow:** `#` object/type, `##`+ function, `+` params, `1.`/`2.` branches, `-` loops, `N. *` else. No invented `if`/`for`/`def`/`return` keywords.
|
||||
8. **Prose vs code:** blank line separates comment paragraphs from executable lines.
|
||||
9. **GFM tables** for data; prefer `table.put` over `json.set` chains. `ext/**` never calls `host_*`.
|
||||
10. Body narrative `[描述](target)` is lazy (NarrativeRef) — reading does not execute side effects.
|
||||
|
||||
| Marker | Meaning (v2.1) |
|
||||
|--------|----------------|
|
||||
| `#` / `##`+ | Object/type · function/method |
|
||||
| `**…**` | Statement (assign / call / expression) |
|
||||
| `*…*` | Return |
|
||||
| `[描述](payload)` | Occlusion step (payload executes) |
|
||||
| `access(c, k)` | Collection read |
|
||||
| `> fn …` | Call |
|
||||
| GFM table | Collection literal |
|
||||
|
||||
Canonical: `doc/design/core-surface.md`, `doc/design/document-native-development-guide.md`. Deeper tables: [reference.md](reference.md). Patterns: [examples.md](examples.md). Authoring craft: [marqdo-dev](../marqdo-dev/SKILL.md). Release: [marqdo-release](../marqdo-release/SKILL.md).
|
||||
|
||||
Executable-knowledge gate: [`tests/executable_knowledge/`](../../tests/executable_knowledge/README.md).
|
||||
|
||||
## Hard rules (do not violate)
|
||||
|
||||
1. **File suffix** must be `.mq.md` (never plain `.md` for executable sources).
|
||||
2. **Output is not `**bold**`.** Print with `> print text=…` (or `> 打印 内容=…`). Bold `**…**` is **return value only**.
|
||||
3. **Prose vs code:** after a blank line, an unmarked first line starts a **comment paragraph**; every following non-blank line stays comment until the next blank line. Put a **blank line** between narration and executable lines.
|
||||
4. **Do not invent keywords** `if` / `else` / `while` / `for` / `def` / `return`. Control flow is Markdown: **`+` params** (under heading), **`1.` `2.` … branches**, **`-` loops**, arm `N. *` = else, `#` = **object/type**, `##`+ = **function/method**, `**…**` = return.
|
||||
5. **Identifiers use backticks** — params `` + `name` ``. **Exemptions:** foreach `` - [item](coll) ``; footnote `` name[^key] ``; **inside italic `*…*` / bold `**…**`**, bare ids are **variables** (Python-style unified namespace) and method-call receivers are bare too. **Drop the backticks on value-expression variable names and method receivers inside `*…*` / `**…**`** — they are redundant once the `*…*`/`**…**` markers mark the segment as code: `*分类 = 分类[^苹果]*`, `**n * 2**`, `*p = > page.主体装配 组件=home*`. **Text literals must be quoted**: `*a = > text.split value=src sep=","*`. Backticks **still required** where a bare word is *not* a variable in **standalone `>` calls** (bare = text there): `` > str `n` ``, `` > `obj`.method ``. **No trailing space** inside `*…*` / `**…**` wrapped code: the closing marker must touch the last token directly (`*a = 1*`, `**n**`), never `` *a = 1 * ``.
|
||||
6. **Structure lines are not wrapped in italics.** `#` `>` `+` `-` `|` lines stand alone; use `*…*` only for general statements (bindings / expressions).
|
||||
7. **Paths:** In bare *expressions* `/` is division. In **call args / param defaults / table cells**, unspaced `a/b` and quoted `".marqdo/agent-kb"` are path text — no `json.parse` needed. Numeric ratios that must stay text use quotes (`"1/5"`, `"16/9"`); bare `1/5` is division.
|
||||
8. Prefer ending side-effect-only function bodies with `*None*` / `*无*` / whole-line `**` / `****` when later lines must belong to an outer function. Lone `---` / `***` are Markdown thematic breaks (skipped), not function end.
|
||||
9. **`ext/**` never calls `host_*`.** Agent/OKF helpers are plugin names (`agent_kb_*`, …) after `plugin.load`. Do **not** add agent/OKF domain code to `src/host/` (core bloat). See `doc/design/ext-agent.md` §4.
|
||||
10. **Browser (route C/D/E/F):** client logic is Marqdo on WASM; official bridge is host glue and **may** implement lists, routing, storage, WebSocket, canvas, file read, observers — authors **must not hand-write business JS**. Use `web.client_embed` / `data-mq-source-url` auto-mount. Client effects: prefer **GFM tables + `lib/browser`** ([marqdo-dev](../marqdo-dev/SKILL.md)).
|
||||
11. **Code-as-documentation / no bag glue:** Prefer **GFM tables** for maps, lists, wire, commands. Prefer **`table.put` / named helpers** over `json.set` / `json.append` chains. `lib/json` is for parse/stringify/quote only — not a dict builder. Unreadable json pipelines are a style bug. Read collections with **`[key](map)`**, not footnote indexes for new code. In `markup: occlusion` files read with **`access(map, key)`** — there `[x](y)` is occlusion, not index. **Positive copy-paste shapes** (result map, soft-error pack, sparse `table.put`): [marqdo-dev](../marqdo-dev/SKILL.md) §2 · [examples.md](examples.md) §6e. Do **not** extend legacy `json.set` in `ext/ai` — refactor to tables.
|
||||
12. **One language per file:** Do not mix EN and ZH API names in the same `.mq.md` (imports, calls, params). Pick English *or* Chinese mirrors for the whole program.
|
||||
13. **Sandbox trial habit (Agent / generated `.mq.md`):** Prefer `agent.trial` / `run sandbox=True` / `sandbox.open`→`run`→`diff`→`commit` before promoting to the host tree. Host files must stay unchanged until a successful commit. Caps live in `sandbox.profile` GFM tables — do not invent a second dialect.
|
||||
1. Obey **CORE SYNTAX** above — Occlusion is default; do not emit Classic index links.
|
||||
2. **Identifiers:** params `` + `name` ``; inside `*…*` / `**…**` bare ids are variables. Quote text literals. Standalone `>` calls: backticks where bare would be text.
|
||||
3. **Structure lines** (`#` `>` `+` `-` `|`) are not wrapped in italics.
|
||||
4. **Paths:** bare `/` in expressions is division; in call args / table cells unspaced `a/b` is path text. Ratios as text need quotes (`"1/5"`).
|
||||
5. End side-effect-only helpers with `*None*` / `*空*` / `**` / `****` when the outer function continues. Lone `---` is a thematic break, not function end.
|
||||
6. **`ext/**` never calls `host_*`.** Prefer tables over `json.set` / `json.append`.
|
||||
7. **One language per file.** Sandbox trial before promoting Agent-generated trees.
|
||||
8. Browser: Marqdo on WASM + `lib/browser` tables; do not hand-write business JS.
|
||||
|
||||
|
||||
## Markup → meaning (v0.2)
|
||||
## Call one function
|
||||
|
||||
| Marker | Meaning |
|
||||
|--------|---------|
|
||||
| `#` | **Object / type** (constructor body). `# main` = entry object. `# Child = > Parent` = inherit (**no** implicit parent constructor; call `` `self` = > Parent … `` explicitly when needed). |
|
||||
| `##` … `######` | **Function / method** (nesting by heading depth). |
|
||||
| `` + `name` `` under heading | Parameter (optional `` `name`=default ``) |
|
||||
| `1.` `2.` … in body | Branch arm (condition or `N. *` else). **Same-indent restart at `1.` = new branch statement** (not more arms of the previous list). |
|
||||
| `- …` inside body | Loop (`while` or `` [item](coll) `` / `` [`item`](`coll`) ``) |
|
||||
| Line `N. *` | Else arm |
|
||||
| `> fn args` | Call (named `k=v` or positional) |
|
||||
| `` > `obj`.method args `` | Method call (`obj` must be a map with `_type`) |
|
||||
| Frontmatter nested `import:` / `导入:` map | Import file (`bind: path.mq.md`) or short name (`bind: lib.member`) |
|
||||
| `> lib.member …` / `> lib.Type.member …` | Call via bare dotted path (instance methods need `` `var`.m ``) |
|
||||
| `*…*` | Statement (bind / expr) — closing `*` touches last token, **no trailing space**; bare ids are variables |
|
||||
| `**…**` | Return value — closing `**` touches last token, **no trailing space**; bare ids are variables |
|
||||
| `****` or `**` + spaces + `**` | Return `None` and end function body |
|
||||
| Lone `---` / `***` | Markdown thematic break (skip; not function end) |
|
||||
| GFM table after empty RHS bind | Collection; get with `[key](coll)` (preferred) or legacy `` `m`[^key] `` |
|
||||
| `- [item](coll)` | Foreach (unchanged) |
|
||||
| `` ```lang `` | Foreign code block (via `lib/foreign`) |
|
||||
| `` `"text"` `` | Quoted string (`\n` `\t` `\\` `\"`; `` `var` `` inside); bare tokens unescaped |
|
||||
| Unmarked prose | Comment |
|
||||
```markdown
|
||||
# 计算器
|
||||
|
||||
Entry: load file (often `index.mq.md`) → collect defs → run `# main`.
|
||||
## 加法
|
||||
|
||||
## Occlusion syntax (v2.0.0)
|
||||
计算两个数字`a`和`b`的和。
|
||||
|
||||
A file with frontmatter `markup: occlusion` (or `标记: 遮蔽`) reads `[x](y)` as **occlusion**, not index — in that file only (`MARQDO_MARKUP=occlusion` sets the process default; classic files are unchanged).
|
||||
*[计算结果](a + b)*
|
||||
```
|
||||
|
||||
- `[描述](实现)` — `[]` is prose (never executed); `()` is the **only** executed part.
|
||||
- `[顾客留存率](0.42)` · `[总价](at(价目表, "苹果") * 数量)` · `[功能开发中]()` → `None` (stub; `label.empty_payload` warning).
|
||||
- Read maps/lists with `access(集合, 键)` (aliases `取` / `取值`): `[第二项](access(列, 2))`.
|
||||
- Foreach: `- [篮子里的每个果子](遍历 篮子 为 果)` (accepts `遍历`/`for` and `为`/`as`/`in`).
|
||||
- Calls inside `()`: `加(1)` / `加(1, 2)` / `加一 号(31)` / `txt.str_trim(" x ")` — `)` terminates, commas optional.
|
||||
- Tooling: `marqdo labels pending` (stub ledger) · `marqdo labels --grep 词 --count` · `marqdo migrate labels . --write` · `marqdo check` reports `label.missing` / `label.empty_payload` / `label.ref_dangling`.
|
||||
```
|
||||
marqdo call calculator.mq.md 计算器.加法 a=2 b=3
|
||||
```
|
||||
|
||||
prints `5`. The same command works on any `.mq.md`; the file does not need a special skill name.
|
||||
|
||||
## Minimal program
|
||||
|
||||
@@ -113,13 +114,13 @@ Chinese builtins (same functions, no import):
|
||||
## Statements, returns, branches
|
||||
|
||||
```markdown
|
||||
*x = 1*
|
||||
*y = x + 2*
|
||||
**x = 1**
|
||||
**y = x + 2**
|
||||
|
||||
## add_one
|
||||
+ `n`
|
||||
|
||||
**n + 1**
|
||||
*n + 1*
|
||||
```
|
||||
|
||||
```markdown
|
||||
@@ -147,9 +148,9 @@ import:
|
||||
|
||||
# main
|
||||
|
||||
*xs = > text.split value="a,b,c" sep=","*
|
||||
**xs = > text.split value="a,b,c" sep=","**
|
||||
> print text=`xs`
|
||||
*t = > clock.now_unix*
|
||||
**t = > clock.now_unix**
|
||||
```
|
||||
|
||||
- Import **English** or **Chinese** library file; call that file’s API names via `lib.member` (do not mix languages).
|
||||
@@ -208,8 +209,8 @@ Install: `marqdo ext add quantum` (ZH id: `量子`). Build: `cargo build --relea
|
||||
**Q8 themes (circuit / probs / bloch):** `theme="dark"|"light"|"bw"` (ZH `主题=`; **default `dark`** — slate lab look, gate-family colors, label chips so wires never cross `q0` text). View plot chrome follows `data-theme` on the SVG. Advanced kinds (hinton/…) still Q8c pending full re-skin.
|
||||
|
||||
```markdown
|
||||
*_ = > `qc`.draw kind="circuit" theme="dark"*
|
||||
*_ = > `qc`.draw kind="probs" theme="light"*
|
||||
**_ = > `qc`.draw kind="circuit" theme="dark"**
|
||||
**_ = > `qc`.draw kind="probs" theme="light"**
|
||||
```
|
||||
|
||||
Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7.md](../../doc/design/ext-quantum-q7.md) · Q8 viz: [ext-quantum-viz-style.md](../../doc/design/ext-quantum-viz-style.md) · examples: [quantum-bell](../../examples/quantum-bell/) · [quantum-entanglement](../../examples/quantum-entanglement/).
|
||||
@@ -240,8 +241,8 @@ Design: [ext-quantum.md](../../doc/design/ext-quantum.md) · Q7: [ext-quantum-q7
|
||||
| `# main` then prose then code with no blank line | Blank line before code |
|
||||
| `if x > 0:` | `1. `x` > 0` |
|
||||
| `* > print text=hi *` | `> print text=hi` |
|
||||
| `` *`a` = 1 * `` (trailing space + backticks) | `*a = 1*` — bare bind, no trailing space before `*` / `**` |
|
||||
| `*xs = > text.split value=a,b sep=,*` (bare text args) | `*xs = > text.split value="a,b" sep=","*` — quote text literals; bare words are variables |
|
||||
| `` *`a` = 1 * `` (trailing space + backticks) | `**a = 1**` — bare bind, no trailing space before `*` / `**` |
|
||||
| `*xs = > text.split value=a,b sep=,*` (italic is return, and bare text args) | `**xs = > text.split value="a,b" sep=","**` — statement; quote text literals |
|
||||
| Forgetting empty return after nested `##` helper when outer continues | Add `*None*` / `**` / `****` (not `---`) |
|
||||
| Import `lib/text` then call bare `split` | `> text.split …` (qualified) |
|
||||
| Import `lib/text` then call `拆分` | Match file language (`text.split`, not 文本) |
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
# Classic markup migration card (not default)
|
||||
|
||||
Use this card **only** when editing files that declare `markup: classic` / `标记: 经典`, or when the CLI is run with `--legacy-markup`. New documents must use Occlusion (see SKILL.md CORE SYNTAX).
|
||||
|
||||
| Classic (legacy) | Occlusion / current |
|
||||
|---|---|
|
||||
| `[key](map)` index read | `access(map, "key")` / `取(map, "key")` |
|
||||
| `- [item](coll)` foreach by index form | `- [描述](遍历 coll 为 item)` |
|
||||
| Teach `**…**` as “the return of print” | Print with `> print`; `**…**` is return/code expression |
|
||||
| Default link = index | Default `[描述](payload)` = occlusion; `[]` never executes |
|
||||
|
||||
Do not place Classic rules in the front of the default Skill pack. After migration, remove `markup: classic` and verify with `marqdo run` / `marqdo check`.
|
||||
+92
-21
@@ -28,16 +28,29 @@ struct FlatFun {
|
||||
path_index: HashMap<Vec<String>, usize>,
|
||||
/// Short name from frontmatter `use` → path segments.
|
||||
uses: HashMap<String, Vec<String>>,
|
||||
/// Module collection scope (G02): bare-name resolve must not cross imports.
|
||||
scope: u32,
|
||||
}
|
||||
|
||||
pub fn compile_module(path: Option<&Path>, module: &Module) -> Result<Program> {
|
||||
compile_with(path, module, true)
|
||||
}
|
||||
|
||||
/// Compile every function even when the file has no single program entry.
|
||||
/// `Program::entry` is then only a placeholder; callers must use [`Vm::call_function`].
|
||||
pub fn compile_for_call(path: Option<&Path>, module: &Module) -> Result<Program> {
|
||||
compile_with(path, module, false)
|
||||
}
|
||||
|
||||
fn compile_with(path: Option<&Path>, module: &Module, require_entry: bool) -> Result<Program> {
|
||||
let mut flat = Vec::new();
|
||||
let _exports = collect_module_tree(module, &mut flat);
|
||||
let mut next_scope = 1u32;
|
||||
let _exports = collect_module_tree(module, &mut flat, 0, &mut next_scope);
|
||||
if flat.is_empty() {
|
||||
bail!("bytecode: no functions to compile");
|
||||
}
|
||||
|
||||
let entry = flat
|
||||
let chosen = flat
|
||||
.iter()
|
||||
.position(|f| f.name == "main" && f.parent.is_none())
|
||||
.or_else(|| {
|
||||
@@ -52,8 +65,14 @@ pub fn compile_module(path: Option<&Path>, module: &Module) -> Result<Program> {
|
||||
} else {
|
||||
None
|
||||
}
|
||||
})
|
||||
.ok_or_else(|| anyhow::anyhow!("bytecode: no `# main` object"))?;
|
||||
});
|
||||
let entry = match chosen {
|
||||
Some(index) => index,
|
||||
None if !require_entry => 0,
|
||||
None => bail!("bytecode: no `# main` object"),
|
||||
};
|
||||
// Imports are compiled first, so a missing entry must take paths from the file's own functions.
|
||||
let path_owner = if chosen.is_some() { entry } else { flat.len() - 1 };
|
||||
|
||||
let mut objects = Vec::new();
|
||||
let mut methods = Vec::new();
|
||||
@@ -92,6 +111,7 @@ pub fn compile_module(path: Option<&Path>, module: &Module) -> Result<Program> {
|
||||
|
||||
let parents: Vec<Option<usize>> = flat.iter().map(|f| f.parent).collect();
|
||||
let children: Vec<Vec<usize>> = flat.iter().map(|f| f.children.clone()).collect();
|
||||
let path_index = flat[path_owner].path_index.clone();
|
||||
|
||||
Ok(Program {
|
||||
functions,
|
||||
@@ -101,15 +121,27 @@ pub fn compile_module(path: Option<&Path>, module: &Module) -> Result<Program> {
|
||||
methods,
|
||||
parents,
|
||||
children,
|
||||
path_index,
|
||||
})
|
||||
}
|
||||
|
||||
/// Collect `module` and its imports into `flat`.
|
||||
/// Returns relative export paths within this module (e.g. `["parse"]`, `["agent","step"]`).
|
||||
fn collect_module_tree(module: &Module, flat: &mut Vec<FlatFun>) -> HashMap<Vec<String>, usize> {
|
||||
///
|
||||
/// Each module gets a distinct `scope` id. Bare-name `resolve_call` only searches
|
||||
/// within the caller's scope so imported same-named functions cannot be silently
|
||||
/// selected (BoundCall / G02). Cross-module calls must use `bind.member` paths.
|
||||
fn collect_module_tree(
|
||||
module: &Module,
|
||||
flat: &mut Vec<FlatFun>,
|
||||
scope: u32,
|
||||
next_scope: &mut u32,
|
||||
) -> HashMap<Vec<String>, usize> {
|
||||
let mut path_index = HashMap::new();
|
||||
for (bind, dep) in &module.import_modules {
|
||||
let dep_exports = collect_module_tree(dep, flat);
|
||||
let child_scope = *next_scope;
|
||||
*next_scope += 1;
|
||||
let dep_exports = collect_module_tree(dep, flat, child_scope, next_scope);
|
||||
for (rel, idx) in dep_exports {
|
||||
let mut full = Vec::with_capacity(rel.len() + 1);
|
||||
full.push(bind.clone());
|
||||
@@ -124,11 +156,23 @@ fn collect_module_tree(module: &Module, flat: &mut Vec<FlatFun>) -> HashMap<Vec<
|
||||
.map(|u| (u.bind.clone(), u.path.clone()))
|
||||
.collect();
|
||||
|
||||
let start = flat.len();
|
||||
let mut exports = HashMap::new();
|
||||
for fun in &module.functions {
|
||||
let id = collect_fun(fun, None, flat, &path_index, &uses);
|
||||
let id = collect_fun(fun, None, flat, &path_index, &uses, scope);
|
||||
export_tree(&mut exports, &[], id, flat);
|
||||
}
|
||||
// Same-module BoundCall paths (`Type.member`) must be visible to siblings.
|
||||
for (rel, idx) in &exports {
|
||||
path_index.insert(rel.clone(), *idx);
|
||||
}
|
||||
for f in &mut flat[start..] {
|
||||
if f.scope == scope {
|
||||
for (rel, idx) in &exports {
|
||||
f.path_index.insert(rel.clone(), *idx);
|
||||
}
|
||||
}
|
||||
}
|
||||
exports
|
||||
}
|
||||
|
||||
@@ -152,6 +196,7 @@ fn collect_fun(
|
||||
flat: &mut Vec<FlatFun>,
|
||||
path_index: &HashMap<Vec<String>, usize>,
|
||||
uses: &HashMap<String, Vec<String>>,
|
||||
scope: u32,
|
||||
) -> usize {
|
||||
let id = flat.len();
|
||||
flat.push(FlatFun {
|
||||
@@ -166,10 +211,11 @@ fn collect_fun(
|
||||
span: fun.span,
|
||||
path_index: path_index.clone(),
|
||||
uses: uses.clone(),
|
||||
scope,
|
||||
});
|
||||
let mut child_ids = Vec::new();
|
||||
for child in &fun.children {
|
||||
child_ids.push(collect_fun(child, Some(id), flat, path_index, uses));
|
||||
child_ids.push(collect_fun(child, Some(id), flat, path_index, uses, scope));
|
||||
}
|
||||
flat[id].children = child_ids;
|
||||
id
|
||||
@@ -323,28 +369,42 @@ impl<'a> FnCompiler<'a> {
|
||||
s
|
||||
}
|
||||
|
||||
/// BoundCall-equivalent bare-name resolve (G02): only the caller's module
|
||||
/// scope. Imported functions are reachable solely via `path_index` paths.
|
||||
fn resolve_call(&self, name: &str) -> Result<usize> {
|
||||
let fun = &self.flat[self.fn_id];
|
||||
let scope = fun.scope;
|
||||
for &cid in &fun.children {
|
||||
if self.flat[cid].name == name {
|
||||
if self.flat[cid].name == name && self.flat[cid].scope == scope {
|
||||
return Ok(cid);
|
||||
}
|
||||
}
|
||||
let mut parent = fun.parent;
|
||||
while let Some(pid) = parent {
|
||||
if self.flat[pid].scope != scope {
|
||||
break;
|
||||
}
|
||||
for &cid in &self.flat[pid].children {
|
||||
if self.flat[cid].name == name {
|
||||
if self.flat[cid].name == name && self.flat[cid].scope == scope {
|
||||
return Ok(cid);
|
||||
}
|
||||
}
|
||||
parent = self.flat[pid].parent;
|
||||
}
|
||||
let mut matches = Vec::new();
|
||||
for (i, f) in self.flat.iter().enumerate() {
|
||||
if f.parent.is_none() && f.name == name {
|
||||
return Ok(i);
|
||||
if f.parent.is_none() && f.scope == scope && f.name == name {
|
||||
matches.push(i);
|
||||
}
|
||||
}
|
||||
Err(self.err(format!("unknown function `{name}`")))
|
||||
match matches.as_slice() {
|
||||
[only] => Ok(*only),
|
||||
[] => Err(self.err(format!("unknown function `{name}`"))),
|
||||
_ => Err(self.err(format!(
|
||||
"ambiguous function `{name}`: {} candidates in this module; use an explicit path",
|
||||
matches.len()
|
||||
))),
|
||||
}
|
||||
}
|
||||
|
||||
fn compile_stmt(&mut self, stmt: &Stmt) -> Result<()> {
|
||||
@@ -470,13 +530,20 @@ impl<'a> FnCompiler<'a> {
|
||||
}
|
||||
|
||||
fn compile_call(&mut self, call: &CallExpr, as_stmt: bool) -> Result<()> {
|
||||
// Mirror `sema::bind::resolve_for_exec` (G02 BoundCall):
|
||||
// static → Op::Call(fid); dynamic → MethodCall / CallFn / PluginCall.
|
||||
let mut call = call
|
||||
.with_modifiers_expanded()
|
||||
.map_err(|m| self.err(m))?;
|
||||
if let Some(recv) = &call.receiver {
|
||||
return self.compile_method_call(recv, &call, as_stmt);
|
||||
}
|
||||
if let Some(path) = &call.path {
|
||||
// Bare dotted callee `a.b`: if `a` is a local variable, it is a method
|
||||
// receiver (Python-style unified namespace); otherwise it's a library path.
|
||||
if path.len() == 2 && self.locals.contains_key(&path[0]) {
|
||||
// Local-var method sugar before import path (same as Tree / resolve_for_exec).
|
||||
if path.len() == 2
|
||||
&& self.locals.contains_key(&path[0])
|
||||
&& !self.flat[self.fn_id].path_index.keys().any(|p| p.first() == Some(&path[0]))
|
||||
{
|
||||
let recv = path[0].clone();
|
||||
return self.compile_method_call(&recv, &call, as_stmt);
|
||||
}
|
||||
@@ -488,10 +555,6 @@ impl<'a> FnCompiler<'a> {
|
||||
call.callee =
|
||||
crate::aliases::normalize_call_callee_and_args(&call.callee, &mut call.args);
|
||||
|
||||
if let Some(recv) = &call.receiver {
|
||||
return self.compile_method_call(recv, &call, as_stmt);
|
||||
}
|
||||
|
||||
match call.callee.as_str() {
|
||||
"print" => {
|
||||
let text_expr = call
|
||||
@@ -648,9 +711,17 @@ impl<'a> FnCompiler<'a> {
|
||||
_ => {}
|
||||
}
|
||||
|
||||
// BoundCall bare name (same-module scope only) or DynamicCall plugin.
|
||||
match self.resolve_call(&call.callee) {
|
||||
Ok(fid) => self.compile_user_call(fid, &call, as_stmt),
|
||||
Err(_) => self.compile_plugin_call(&call, as_stmt),
|
||||
Err(err) => {
|
||||
let msg = format!("{err:#}");
|
||||
if msg.contains("ambiguous function") {
|
||||
return Err(err);
|
||||
}
|
||||
// DynamicCall(Plugin): unresolved bare names may be runtime plugins.
|
||||
self.compile_plugin_call(&call, as_stmt)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+5
-1
@@ -3,9 +3,11 @@
|
||||
mod compile;
|
||||
mod vm;
|
||||
|
||||
pub use compile::compile_module;
|
||||
pub use compile::{compile_for_call, compile_module};
|
||||
pub use vm::Vm;
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
use crate::diagnostics::Span;
|
||||
use crate::value::Value;
|
||||
|
||||
@@ -100,6 +102,8 @@ pub struct Program {
|
||||
pub parents: Vec<Option<usize>>,
|
||||
/// Parallel to `functions`: child fn indices.
|
||||
pub children: Vec<Vec<usize>>,
|
||||
/// Dotted path (`计算器.加法`, `inc.加一`) → function index. Entry module exports plus imports.
|
||||
pub path_index: HashMap<Vec<String>, usize>,
|
||||
}
|
||||
|
||||
impl Program {
|
||||
|
||||
+42
-4
@@ -102,7 +102,6 @@ impl Vm {
|
||||
}
|
||||
|
||||
pub fn run(&mut self, program: &Program) -> Result<Value> {
|
||||
let mut stack: Vec<Value> = Vec::new();
|
||||
let entry = program.entry;
|
||||
let arity = program.functions[entry].params.len();
|
||||
if arity != 0 {
|
||||
@@ -113,14 +112,53 @@ impl Vm {
|
||||
.unwrap_or(Span::new(1, 1));
|
||||
return Err(self.err_at(span, "bytecode: entry function needs arguments"));
|
||||
}
|
||||
self.execute(program, entry, &[])
|
||||
}
|
||||
|
||||
/// Run `fn_idx` with named arguments already bound to local slots.
|
||||
/// Does not fall back to [`Program::entry`].
|
||||
pub fn call_function(
|
||||
&mut self,
|
||||
program: &Program,
|
||||
fn_idx: usize,
|
||||
args: &[(String, Value)],
|
||||
) -> Result<Value> {
|
||||
if fn_idx >= program.functions.len() {
|
||||
return Err(self.err_at(
|
||||
Span::new(1, 1),
|
||||
format!("bytecode: function index {fn_idx} is out of range"),
|
||||
));
|
||||
}
|
||||
self.execute(program, fn_idx, args)
|
||||
}
|
||||
|
||||
fn execute(
|
||||
&mut self,
|
||||
program: &Program,
|
||||
fn_idx: usize,
|
||||
args: &[(String, Value)],
|
||||
) -> Result<Value> {
|
||||
let mut stack: Vec<Value> = Vec::new();
|
||||
let mut slots = vec![Value::None; program.functions[fn_idx].locals.len().max(1)];
|
||||
for (name, value) in args {
|
||||
if let Some(i) = program.functions[fn_idx]
|
||||
.locals
|
||||
.iter()
|
||||
.position(|local| local == name)
|
||||
{
|
||||
if i < slots.len() {
|
||||
slots[i] = value.clone();
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut frames = vec![Frame {
|
||||
fn_idx: entry,
|
||||
fn_idx,
|
||||
ip: 0,
|
||||
slots: vec![Value::None; program.functions[entry].locals.len().max(1)],
|
||||
slots,
|
||||
retag_matrix: None,
|
||||
}];
|
||||
self.host
|
||||
.push_call_frame(&program.functions[entry].name.clone());
|
||||
.push_call_frame(&program.functions[fn_idx].name.clone());
|
||||
|
||||
loop {
|
||||
let frame = frames.last_mut().unwrap();
|
||||
|
||||
+295
@@ -0,0 +1,295 @@
|
||||
//! `marqdo call`: run one already-bound function in a `.mq.md` file.
|
||||
//!
|
||||
//! Location, effects, and execution go through the existing compiler and runtime.
|
||||
//! The command does not scan headings, and it does not run the program entry when
|
||||
//! the named function cannot be compiled.
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use crate::ast::Function;
|
||||
use crate::bytecode::{compile_for_call, Vm};
|
||||
use crate::diagnostics::{Diagnostic, Span};
|
||||
use crate::host::{HostCaps, HostContext};
|
||||
use crate::interp::Interpreter;
|
||||
use crate::load::load_module;
|
||||
use crate::sema::{compile_file_closure, SemanticArtifact, SemanticProject};
|
||||
use crate::value::Value;
|
||||
use crate::Backend;
|
||||
|
||||
/// Call `function` in `path` with named arguments.
|
||||
///
|
||||
/// Non-pure effects are refused before the function runs. Host write, exec, net,
|
||||
/// and plugin capabilities stay off for the call.
|
||||
pub fn call_file(
|
||||
path: &Path,
|
||||
function: &str,
|
||||
args: &[(String, Value)],
|
||||
backend: Backend,
|
||||
) -> Result<Value> {
|
||||
let path = path
|
||||
.canonicalize()
|
||||
.map_err(|e| call_diag(None, 1, "call.compile", format!("failed to read {}: {e}", path.display())))?;
|
||||
let compilation = compile_file_closure(&path).map_err(|e| preserve_diag(&path, e))?;
|
||||
let artifact = locate_artifact(&compilation.semantics, &path, function)?;
|
||||
deny_unsafe(&path, artifact)?;
|
||||
execute_source_function(
|
||||
&path,
|
||||
artifact.source.line,
|
||||
&artifact.display_name,
|
||||
args,
|
||||
tight_caps(),
|
||||
backend,
|
||||
true,
|
||||
)
|
||||
.map(|(value, _stdout)| value)
|
||||
}
|
||||
|
||||
/// Run the function declared at `line` in `path`.
|
||||
///
|
||||
/// `display_name` is the declaration path (`计算器.加法`). The query the caller
|
||||
/// typed is not used as a heading search and is not used as the bytecode path,
|
||||
/// so a unique short name or SymbolId still executes that function.
|
||||
/// `caps` are the host capabilities for this run. `call` passes them all off.
|
||||
pub(crate) fn execute_source_function(
|
||||
path: &Path,
|
||||
line: u32,
|
||||
display_name: &str,
|
||||
args: &[(String, Value)],
|
||||
caps: HostCaps,
|
||||
backend: Backend,
|
||||
strict_args: bool,
|
||||
) -> Result<(Value, String)> {
|
||||
let module = load_module(path)?;
|
||||
let fun = find_function_at_line(&module.functions, line).ok_or_else(|| {
|
||||
call_diag(
|
||||
Some(path),
|
||||
line,
|
||||
"artifact.unresolved",
|
||||
format!("无法在 {}:{} 定位函数 {display_name}", path.display(), line),
|
||||
)
|
||||
})?;
|
||||
let expected = display_name
|
||||
.rsplit('.')
|
||||
.next()
|
||||
.unwrap_or(display_name);
|
||||
if fun.name != expected {
|
||||
return Err(call_diag(
|
||||
Some(path),
|
||||
line,
|
||||
"artifact.unresolved",
|
||||
format!("源码行 {line} 上的函数是 `{}`,与 `{display_name}` 不一致", fun.name),
|
||||
));
|
||||
}
|
||||
let mut interp = Interpreter::with_capture(Some(path), false)
|
||||
.with_host(HostContext::for_capture(Some(path), caps));
|
||||
// `call` checks the signature. `artifact.execute` keeps its previous
|
||||
// environment injection so an input the body reads is not rejected as unknown.
|
||||
let bound;
|
||||
let invoke_args: &[(String, Value)] = if strict_args {
|
||||
bound = interp.bind_external_args(&module, fun, args)?;
|
||||
&bound
|
||||
} else {
|
||||
args
|
||||
};
|
||||
match backend {
|
||||
Backend::Tree => {
|
||||
let value = interp.invoke_bound_function(&module, fun, invoke_args)?;
|
||||
Ok((value, interp.captured_stdout))
|
||||
}
|
||||
Backend::Bytecode => {
|
||||
let program = compile_for_call(Some(path), &module)?;
|
||||
let key: Vec<String> = display_name
|
||||
.split('.')
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(|s| s.to_string())
|
||||
.collect();
|
||||
let idx = program.path_index.get(&key).copied().ok_or_else(|| {
|
||||
call_diag(
|
||||
Some(path),
|
||||
line,
|
||||
"artifact.unresolved",
|
||||
format!("bytecode 没有编译出 `{display_name}`;拒绝改去执行程序入口"),
|
||||
)
|
||||
})?;
|
||||
let mut vm = Vm::with_capture(Some(path))
|
||||
.with_host(HostContext::for_capture(Some(path), caps));
|
||||
let value = vm.call_function(&program, idx, invoke_args)?;
|
||||
Ok((value, vm.captured_stdout))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse `name=value` tokens. Values are integers, decimals, quoted text, or True/False/None.
|
||||
pub fn parse_call_args(tokens: &[String]) -> Result<Vec<(String, Value)>> {
|
||||
let mut out = Vec::new();
|
||||
for token in tokens {
|
||||
let Some((name, raw)) = token.split_once('=') else {
|
||||
return Err(call_diag(
|
||||
None,
|
||||
0,
|
||||
"call.bad_arg",
|
||||
format!("参数 `{token}` 需要写成 名字=值"),
|
||||
));
|
||||
};
|
||||
if name.is_empty() {
|
||||
return Err(call_diag(
|
||||
None,
|
||||
0,
|
||||
"call.bad_arg",
|
||||
format!("参数 `{token}` 缺少名字"),
|
||||
));
|
||||
}
|
||||
let value = parse_literal(raw).map_err(|message| {
|
||||
call_diag(None, 0, "call.bad_value", format!("参数 `{name}`: {message}"))
|
||||
})?;
|
||||
out.push((name.to_string(), value));
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn tight_caps() -> HostCaps {
|
||||
HostCaps {
|
||||
fs_write: false,
|
||||
exec: false,
|
||||
net: false,
|
||||
plugin: false,
|
||||
}
|
||||
}
|
||||
|
||||
fn locate_artifact<'a>(
|
||||
project: &'a SemanticProject,
|
||||
entry: &Path,
|
||||
query: &str,
|
||||
) -> Result<&'a SemanticArtifact> {
|
||||
let in_entry: Vec<&SemanticArtifact> = project
|
||||
.artifacts
|
||||
.iter()
|
||||
.filter(|artifact| same_file(&project.root, &artifact.source.file, entry))
|
||||
.collect();
|
||||
let exact: Vec<&SemanticArtifact> = in_entry
|
||||
.iter()
|
||||
.copied()
|
||||
.filter(|artifact| {
|
||||
artifact.display_name == query
|
||||
|| artifact
|
||||
.symbol_id
|
||||
.as_ref()
|
||||
.is_some_and(|id| id.as_str() == query)
|
||||
|| artifact.artifact_id.0 == query
|
||||
})
|
||||
.collect();
|
||||
let hits = if !exact.is_empty() {
|
||||
exact
|
||||
} else if !query.contains('.') {
|
||||
in_entry
|
||||
.iter()
|
||||
.copied()
|
||||
.filter(|artifact| {
|
||||
artifact.kind == "function"
|
||||
&& artifact.display_name.rsplit('.').next() == Some(query)
|
||||
})
|
||||
.collect()
|
||||
} else {
|
||||
Vec::new()
|
||||
};
|
||||
match hits.len() {
|
||||
1 => Ok(hits[0]),
|
||||
0 => Err(call_diag(
|
||||
Some(entry),
|
||||
1,
|
||||
"artifact.unresolved",
|
||||
format!("无法唯一绑定 `{query}`"),
|
||||
)),
|
||||
_ => Err(call_diag(
|
||||
Some(entry),
|
||||
1,
|
||||
"artifact.ambiguous",
|
||||
format!("无法唯一绑定 `{query}`"),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn deny_unsafe(path: &Path, artifact: &SemanticArtifact) -> Result<()> {
|
||||
if artifact.kind != "function" || artifact.callable.as_deref() == Some("instance_method") {
|
||||
return Err(call_diag(
|
||||
Some(path),
|
||||
artifact.source.line,
|
||||
"artifact.not_executable",
|
||||
"只有非实例函数可由 call 直接执行".to_string(),
|
||||
));
|
||||
}
|
||||
if !artifact.effects.is_pure() {
|
||||
return Err(call_diag(
|
||||
Some(path),
|
||||
artifact.source.line,
|
||||
"artifact.effect_denied",
|
||||
format!(
|
||||
"call 默认只执行纯函数,拒绝效应 {}",
|
||||
artifact.effects.values().join(", ")
|
||||
),
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn same_file(root: &Path, rel: &str, entry: &Path) -> bool {
|
||||
let abs = root.join(rel);
|
||||
match (abs.canonicalize(), entry.canonicalize()) {
|
||||
(Ok(left), Ok(right)) => left == right,
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn find_function_at_line(functions: &[Function], line: u32) -> Option<&Function> {
|
||||
for fun in functions {
|
||||
if fun.span.line == line {
|
||||
return Some(fun);
|
||||
}
|
||||
if let Some(found) = find_function_at_line(&fun.children, line) {
|
||||
return Some(found);
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn parse_literal(raw: &str) -> Result<Value, String> {
|
||||
match raw {
|
||||
"True" | "true" => return Ok(Value::Bool(true)),
|
||||
"False" | "false" => return Ok(Value::Bool(false)),
|
||||
"None" | "none" => return Ok(Value::None),
|
||||
_ => {}
|
||||
}
|
||||
if raw.len() >= 2 {
|
||||
let bytes = raw.as_bytes();
|
||||
if (bytes[0] == b'"' && bytes[raw.len() - 1] == b'"')
|
||||
|| (bytes[0] == b'\'' && bytes[raw.len() - 1] == b'\'')
|
||||
{
|
||||
return Ok(Value::Text(raw[1..raw.len() - 1].to_string()));
|
||||
}
|
||||
}
|
||||
if let Ok(n) = raw.parse::<i64>() {
|
||||
return Ok(Value::Int(n));
|
||||
}
|
||||
if raw.contains('.') {
|
||||
if let Ok(n) = raw.parse::<f64>() {
|
||||
return Ok(Value::Num(n));
|
||||
}
|
||||
}
|
||||
Err(format!(
|
||||
"无法解析 `{raw}`(整数、小数、引号文本、True/False/None)"
|
||||
))
|
||||
}
|
||||
|
||||
fn preserve_diag(path: &Path, err: anyhow::Error) -> anyhow::Error {
|
||||
if Diagnostic::find(&err).is_some() {
|
||||
err
|
||||
} else {
|
||||
call_diag(Some(path), 1, "call.compile", format!("{err:#}"))
|
||||
}
|
||||
}
|
||||
|
||||
fn call_diag(path: Option<&Path>, line: u32, code: &str, message: String) -> anyhow::Error {
|
||||
anyhow::Error::new(Diagnostic::new(path, Span::new(line, 1), message).with_code(code))
|
||||
}
|
||||
@@ -0,0 +1,299 @@
|
||||
//! Document extraction protocol (M01): DocumentSource → ExtractedDocument.
|
||||
//!
|
||||
//! Format parsers live here / in extensions — language core does not OCR.
|
||||
//! First slice: plain text and Markdown. PDF/DOCX plug in later via the same shape.
|
||||
|
||||
use std::fs;
|
||||
use std::path::Path;
|
||||
|
||||
use sha2::{Digest, Sha256};
|
||||
use serde_json::{json, Value};
|
||||
|
||||
pub const EXTRACTOR_VERSION: &str = "marqdo-document-extract/1";
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DocumentSource {
|
||||
pub path: std::path::PathBuf,
|
||||
pub format_hint: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ExtractedSegment {
|
||||
pub id: String,
|
||||
pub kind: String,
|
||||
pub text: String,
|
||||
pub locator: String,
|
||||
pub confidence: f64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ExtractedDocument {
|
||||
pub source_uri: String,
|
||||
pub source_digest: String,
|
||||
pub format: String,
|
||||
pub extractor_version: String,
|
||||
pub text: String,
|
||||
pub segments: Vec<ExtractedSegment>,
|
||||
pub missing: Vec<String>,
|
||||
pub diagnostics: Vec<String>,
|
||||
pub status: String,
|
||||
}
|
||||
|
||||
impl ExtractedDocument {
|
||||
pub fn to_json(&self) -> Value {
|
||||
json!({
|
||||
"schema_version": 1,
|
||||
"source_uri": self.source_uri,
|
||||
"source_digest": self.source_digest,
|
||||
"format": self.format,
|
||||
"extractor_version": self.extractor_version,
|
||||
"status": self.status,
|
||||
"text": self.text,
|
||||
"segments": self.segments.iter().map(|s| json!({
|
||||
"id": s.id,
|
||||
"kind": s.kind,
|
||||
"text": s.text,
|
||||
"locator": s.locator,
|
||||
"confidence": s.confidence,
|
||||
})).collect::<Vec<_>>(),
|
||||
"missing": self.missing,
|
||||
"diagnostics": self.diagnostics,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Extract text/Markdown from a file. Uploaded `.mq.md` is treated as untrusted
|
||||
/// source text and is never executed here.
|
||||
pub fn extract_document(source: &DocumentSource) -> ExtractedDocument {
|
||||
let path = &source.path;
|
||||
let uri = path.to_string_lossy().replace('\\', "/");
|
||||
let ext = path
|
||||
.extension()
|
||||
.and_then(|e| e.to_str())
|
||||
.unwrap_or("")
|
||||
.to_ascii_lowercase();
|
||||
let name = path
|
||||
.file_name()
|
||||
.and_then(|n| n.to_str())
|
||||
.unwrap_or("")
|
||||
.to_ascii_lowercase();
|
||||
|
||||
let format = source
|
||||
.format_hint
|
||||
.clone()
|
||||
.unwrap_or_else(|| {
|
||||
if name.ends_with(".mq.md") || ext == "md" {
|
||||
"markdown".into()
|
||||
} else if matches!(ext.as_str(), "txt" | "text") {
|
||||
"text".into()
|
||||
} else if matches!(ext.as_str(), "pdf" | "docx") {
|
||||
ext.clone()
|
||||
} else {
|
||||
"unknown".into()
|
||||
}
|
||||
});
|
||||
|
||||
if matches!(
|
||||
format.as_str(),
|
||||
"pdf" | "docx" | "ocr" | "png" | "jpg" | "jpeg" | "webp" | "tiff"
|
||||
) || matches!(ext.as_str(), "pdf" | "docx" | "png" | "jpg" | "jpeg" | "webp" | "tiff")
|
||||
{
|
||||
let kind = if matches!(format.as_str(), "ocr")
|
||||
|| matches!(ext.as_str(), "png" | "jpg" | "jpeg" | "webp" | "tiff")
|
||||
{
|
||||
"ocr/image".to_string()
|
||||
} else {
|
||||
format.clone()
|
||||
};
|
||||
return ExtractedDocument {
|
||||
source_uri: uri,
|
||||
source_digest: String::new(),
|
||||
format,
|
||||
extractor_version: EXTRACTOR_VERSION.into(),
|
||||
text: String::new(),
|
||||
segments: vec![],
|
||||
missing: vec!["full_document".into()],
|
||||
diagnostics: vec![format!(
|
||||
"format `{kind}` requires an external extractor plugin (not in core); incomplete extraction is explicit failure"
|
||||
)],
|
||||
status: "failed".into(),
|
||||
};
|
||||
}
|
||||
|
||||
let bytes = match fs::read(path) {
|
||||
Ok(b) => b,
|
||||
Err(e) => {
|
||||
return ExtractedDocument {
|
||||
source_uri: uri,
|
||||
source_digest: String::new(),
|
||||
format,
|
||||
extractor_version: EXTRACTOR_VERSION.into(),
|
||||
text: String::new(),
|
||||
segments: vec![],
|
||||
missing: vec!["full_document".into()],
|
||||
diagnostics: vec![format!("read error: {e}")],
|
||||
status: "failed".into(),
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
if bytes.is_empty() {
|
||||
return ExtractedDocument {
|
||||
source_uri: uri,
|
||||
source_digest: digest_bytes(&bytes),
|
||||
format,
|
||||
extractor_version: EXTRACTOR_VERSION.into(),
|
||||
text: String::new(),
|
||||
segments: vec![],
|
||||
missing: vec!["content".into()],
|
||||
diagnostics: vec!["empty document".into()],
|
||||
status: "failed".into(),
|
||||
};
|
||||
}
|
||||
|
||||
// Resource limit: refuse to fully buffer huge uploads as text (M01).
|
||||
const MAX_EXTRACT_BYTES: usize = 8 << 20;
|
||||
if bytes.len() > MAX_EXTRACT_BYTES {
|
||||
let head = &bytes[..MAX_EXTRACT_BYTES];
|
||||
let lossy = String::from_utf8_lossy(head).into_owned();
|
||||
let mut doc = extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &lossy);
|
||||
doc.missing.push("tail_beyond_limit".into());
|
||||
doc.diagnostics.push(format!(
|
||||
"document truncated at {MAX_EXTRACT_BYTES} bytes (got {}); extraction is partial",
|
||||
bytes.len()
|
||||
));
|
||||
doc.status = "partial".into();
|
||||
return doc;
|
||||
}
|
||||
|
||||
let text = match String::from_utf8(bytes.clone()) {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
let lossy = String::from_utf8_lossy(&bytes).into_owned();
|
||||
let mut doc = extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &lossy);
|
||||
doc.diagnostics
|
||||
.push(format!("invalid utf-8 ({e}); decoded lossy"));
|
||||
doc.status = if doc.text.is_empty() {
|
||||
"failed".into()
|
||||
} else {
|
||||
"partial".into()
|
||||
};
|
||||
return doc;
|
||||
}
|
||||
};
|
||||
|
||||
extract_markdown_or_text(&uri, &digest_bytes(&bytes), &format, &text)
|
||||
}
|
||||
|
||||
fn extract_markdown_or_text(
|
||||
uri: &str,
|
||||
digest: &str,
|
||||
format: &str,
|
||||
text: &str,
|
||||
) -> ExtractedDocument {
|
||||
let mut segments = Vec::new();
|
||||
let mut n = 0u32;
|
||||
let mut byte_off = 0usize;
|
||||
for (i, line) in text.lines().enumerate() {
|
||||
let line_no = i + 1;
|
||||
let line_bytes = line.len();
|
||||
// M05: stable fragment locator with 1-based line + UTF-8 byte offset.
|
||||
let loc = |kind: &str| format!("{kind}:L{line_no}:B{byte_off}");
|
||||
if let Some(rest) = line.trim().strip_prefix('#') {
|
||||
n += 1;
|
||||
let heading = rest.trim_start_matches('#').trim();
|
||||
segments.push(ExtractedSegment {
|
||||
id: format!("seg-{n}"),
|
||||
kind: "heading".into(),
|
||||
text: heading.to_string(),
|
||||
locator: loc("heading"),
|
||||
confidence: 1.0,
|
||||
});
|
||||
} else if line.trim().starts_with('|') {
|
||||
n += 1;
|
||||
segments.push(ExtractedSegment {
|
||||
id: format!("seg-{n}"),
|
||||
kind: "table_row".into(),
|
||||
text: line.to_string(),
|
||||
locator: loc("table_row"),
|
||||
confidence: 0.9,
|
||||
});
|
||||
} else if !line.trim().is_empty() {
|
||||
n += 1;
|
||||
segments.push(ExtractedSegment {
|
||||
id: format!("seg-{n}"),
|
||||
kind: "paragraph".into(),
|
||||
text: line.to_string(),
|
||||
locator: loc("paragraph"),
|
||||
confidence: 1.0,
|
||||
});
|
||||
}
|
||||
// +1 for '\n' when present in original (lines() strips it).
|
||||
byte_off += line_bytes + 1;
|
||||
}
|
||||
ExtractedDocument {
|
||||
source_uri: uri.to_string(),
|
||||
source_digest: digest.to_string(),
|
||||
format: format.to_string(),
|
||||
extractor_version: EXTRACTOR_VERSION.into(),
|
||||
text: text.to_string(),
|
||||
segments,
|
||||
missing: vec![],
|
||||
diagnostics: vec![],
|
||||
status: "ok".into(),
|
||||
}
|
||||
}
|
||||
|
||||
fn digest_bytes(bytes: &[u8]) -> String {
|
||||
let mut h = Sha256::new();
|
||||
h.update(bytes);
|
||||
format!("sha256:{:x}", h.finalize())
|
||||
}
|
||||
|
||||
/// Convenience for tests / CLI: extract if path exists.
|
||||
pub fn extract_path(path: &Path) -> ExtractedDocument {
|
||||
extract_document(&DocumentSource {
|
||||
path: path.to_path_buf(),
|
||||
format_hint: None,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::io::Write;
|
||||
|
||||
#[test]
|
||||
fn extracts_markdown_headings() {
|
||||
let dir = std::env::temp_dir().join(format!("mq-extract-{}", std::process::id()));
|
||||
let _ = fs::create_dir_all(&dir);
|
||||
let path = dir.join("note.md");
|
||||
let mut f = fs::File::create(&path).unwrap();
|
||||
writeln!(f, "# rate\n\nhello").unwrap();
|
||||
let doc = extract_path(&path);
|
||||
assert_eq!(doc.status, "ok");
|
||||
assert!(doc.segments.iter().any(|s| s.kind == "heading" && s.text == "rate"));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pdf_reports_missing_extractor() {
|
||||
let doc = extract_document(&DocumentSource {
|
||||
path: Path::new("x.pdf").into(),
|
||||
format_hint: Some("pdf".into()),
|
||||
});
|
||||
assert_eq!(doc.status, "failed");
|
||||
assert!(!doc.diagnostics.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ocr_image_reports_missing_extractor() {
|
||||
let doc = extract_document(&DocumentSource {
|
||||
path: Path::new("scan.png").into(),
|
||||
format_hint: Some("ocr".into()),
|
||||
});
|
||||
assert_eq!(doc.status, "failed");
|
||||
assert!(doc.missing.contains(&"full_document".into()));
|
||||
assert!(doc.diagnostics.iter().any(|d| d.contains("ocr") || d.contains("extractor")));
|
||||
}
|
||||
}
|
||||
+5
-2
@@ -326,8 +326,11 @@ pub fn apply_patch_blocks(
|
||||
};
|
||||
match apply_patch_blocks_inner(ctx, path, text) {
|
||||
Ok(n) => Ok(n),
|
||||
// soft only swallows unique-FIND miss — never parse / multi-match / whole-file.
|
||||
Err(e) if soft && e.contains("FIND not found") => Ok(Value::Int(0)),
|
||||
// soft swallows unique-FIND miss and empty FIND (model protocol mistakes).
|
||||
// Never softens parse / multi-match / whole-file rewrite.
|
||||
Err(e) if soft && (e.contains("FIND not found") || e.contains("empty FIND")) => {
|
||||
Ok(Value::Int(0))
|
||||
}
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -67,7 +67,7 @@ use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// Host capability flags. Default: all enabled.
|
||||
#[derive(Debug, Clone)]
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct HostCaps {
|
||||
pub fs_write: bool,
|
||||
pub exec: bool,
|
||||
|
||||
+113
-11
@@ -6,7 +6,8 @@ use std::collections::HashMap;
|
||||
use std::ffi::{CStr, CString};
|
||||
use std::os::raw::{c_char, c_int, c_void};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Mutex, OnceLock};
|
||||
use std::sync::{Mutex, MutexGuard, OnceLock};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use libloading::{Library, Symbol};
|
||||
|
||||
@@ -75,6 +76,8 @@ static GLOBAL_HOST: OnceLock<Mutex<SendPtr<HostContext>>> = OnceLock::new();
|
||||
/// Process-wide lib-path hook (same lifetime as GLOBAL_HOST).
|
||||
static GLOBAL_LIB_PATH: OnceLock<Mutex<SendLibPath>> = OnceLock::new();
|
||||
/// Serialize interpreter re-entry from HTTP worker threads.
|
||||
/// HostContext is not Sync — one Marqdo invoke at a time. Short routes must
|
||||
/// fail fast (`interpreter busy`) so a long SSE chat cannot freeze the UI.
|
||||
static INVOKE_LOCK: OnceLock<Mutex<()>> = OnceLock::new();
|
||||
|
||||
fn global_host_slot() -> &'static Mutex<SendPtr<HostContext>> {
|
||||
@@ -89,6 +92,41 @@ fn invoke_lock() -> &'static Mutex<()> {
|
||||
INVOKE_LOCK.get_or_init(|| Mutex::new(()))
|
||||
}
|
||||
|
||||
/// Default wait for short HTTP invokes (`MARQDO_INVOKE_WAIT_MS`, default 1500ms).
|
||||
/// Do not raise this process-wide during SSE — concurrent short routes would hang again.
|
||||
/// See doc/adr/0008-web-invoke-serialization.md.
|
||||
fn invoke_wait_budget() -> Duration {
|
||||
const DEFAULT_MS: u64 = 1500;
|
||||
match std::env::var("MARQDO_INVOKE_WAIT_MS") {
|
||||
Ok(s) => {
|
||||
let ms = s.trim().parse::<u64>().unwrap_or(DEFAULT_MS);
|
||||
Duration::from_millis(ms.max(1))
|
||||
}
|
||||
Err(_) => Duration::from_millis(DEFAULT_MS),
|
||||
}
|
||||
}
|
||||
|
||||
fn acquire_invoke_lock() -> Result<MutexGuard<'static, ()>, String> {
|
||||
let budget = invoke_wait_budget();
|
||||
let deadline = Instant::now() + budget;
|
||||
let lock = invoke_lock();
|
||||
loop {
|
||||
match lock.try_lock() {
|
||||
Ok(g) => return Ok(g),
|
||||
Err(std::sync::TryLockError::Poisoned(p)) => return Ok(p.into_inner()),
|
||||
Err(std::sync::TryLockError::WouldBlock) => {
|
||||
if Instant::now() >= deadline {
|
||||
return Err(format!(
|
||||
"interpreter busy: another request holds the Marqdo invoke lock (waited {}ms)",
|
||||
budget.as_millis()
|
||||
));
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(15));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Hook: resolve bare `lib.member` (and longer paths) in the site module's import tree.
|
||||
#[derive(Clone, Copy)]
|
||||
pub struct LibPathCall {
|
||||
@@ -284,6 +322,42 @@ unsafe extern "C" fn host_query(
|
||||
}
|
||||
};
|
||||
|
||||
let args_owned: Option<String> = if args_json.is_null() {
|
||||
None
|
||||
} else {
|
||||
CStr::from_ptr(args_json)
|
||||
.to_str()
|
||||
.ok()
|
||||
.map(|s| s.to_string())
|
||||
};
|
||||
|
||||
// Control plane (admit/cancel/events) must not take INVOKE_LOCK or require
|
||||
// an evaluating HostContext — otherwise cancel cannot interrupt a busy run.
|
||||
if name == "runtime" {
|
||||
let raw = args_owned.as_deref().unwrap_or("{}");
|
||||
let args: serde_json::Value = match serde_json::from_str(if raw.trim().is_empty() {
|
||||
"{}"
|
||||
} else {
|
||||
raw
|
||||
}) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
set_err(&format!("runtime args: {e}"));
|
||||
return 1;
|
||||
}
|
||||
};
|
||||
match crate::runtime::host_api::handle(&args) {
|
||||
Ok(j) => {
|
||||
set_out(&j.to_string());
|
||||
return 0;
|
||||
}
|
||||
Err(e) => {
|
||||
set_err(&e);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let ctx_ptr = CURRENT_HOST.with(|c| c.get());
|
||||
let ctx_ptr = if ctx_ptr.is_null() {
|
||||
global_host_slot()
|
||||
@@ -300,19 +374,16 @@ unsafe extern "C" fn host_query(
|
||||
let _invoke_guard;
|
||||
let from_worker = CURRENT_HOST.with(|c| c.get().is_null());
|
||||
if from_worker {
|
||||
_invoke_guard = invoke_lock().lock().unwrap_or_else(|e| e.into_inner());
|
||||
match acquire_invoke_lock() {
|
||||
Ok(g) => _invoke_guard = g,
|
||||
Err(msg) => {
|
||||
set_err(&msg);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
let ctx = &mut *ctx_ptr;
|
||||
|
||||
let args_owned: Option<String> = if args_json.is_null() {
|
||||
None
|
||||
} else {
|
||||
CStr::from_ptr(args_json)
|
||||
.to_str()
|
||||
.ok()
|
||||
.map(|s| s.to_string())
|
||||
};
|
||||
|
||||
let result = match name {
|
||||
"module_source" => agent_rt::module_source(ctx).and_then(|v| value_to_json(&v)),
|
||||
"call_site" => agent_rt::call_site(ctx, Some(ctx.current_line)).and_then(|v| value_to_json(&v)),
|
||||
@@ -320,6 +391,19 @@ unsafe extern "C" fn host_query(
|
||||
"cwd" => Ok(serde_json::Value::String(
|
||||
ctx.cwd.to_string_lossy().into_owned(),
|
||||
)),
|
||||
"entry_path" => {
|
||||
let path = ctx.entry_path.clone().unwrap_or_else(|| ctx.cwd.clone());
|
||||
let abs = if path.is_absolute() {
|
||||
path
|
||||
} else {
|
||||
std::env::current_dir()
|
||||
.unwrap_or_else(|_| PathBuf::from("."))
|
||||
.join(path)
|
||||
};
|
||||
Ok(serde_json::Value::String(
|
||||
abs.to_string_lossy().into_owned(),
|
||||
))
|
||||
}
|
||||
"entry_dir" => {
|
||||
// Script directory: parent of entry file (for_run also sets ctx.cwd to this).
|
||||
let dir = ctx
|
||||
@@ -460,6 +544,24 @@ unsafe extern "C" fn host_query(
|
||||
let cap = crate::run_file_capture(&p, &opts).map_err(|e| e.to_string())?;
|
||||
value_to_json(&cap.value)
|
||||
})(),
|
||||
// M04: plugins may drive MLSP artifact.bind/execute without inventing a second protocol.
|
||||
"mlsp" => (|| {
|
||||
let raw = args_owned.as_deref().unwrap_or("{}");
|
||||
let args: serde_json::Value = serde_json::from_str(if raw.trim().is_empty() {
|
||||
"{}"
|
||||
} else {
|
||||
raw
|
||||
})
|
||||
.map_err(|e| format!("mlsp args: {e}"))?;
|
||||
let method = args
|
||||
.get("method")
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or_else(|| "mlsp requires `method`".to_string())?;
|
||||
let params = args.get("params").cloned().unwrap_or(serde_json::json!({}));
|
||||
let line = serde_json::json!({"method": method, "params": params}).to_string();
|
||||
let resp = crate::mlsp::handle_line(&line);
|
||||
serde_json::from_str(&resp).map_err(|e| format!("mlsp response: {e}"))
|
||||
})(),
|
||||
other => Err(format!("host_query: unknown `{other}`")),
|
||||
};
|
||||
|
||||
|
||||
@@ -331,6 +331,7 @@ fn spawn_file(
|
||||
));
|
||||
let _ = std::fs::write(&events_path, "");
|
||||
cmd.env("MARQDO_EVENT_FORWARD", &events_path);
|
||||
apply_run_selection_env(&mut cmd);
|
||||
|
||||
let mut child = cmd
|
||||
.spawn()
|
||||
@@ -693,3 +694,86 @@ fn require<'a>(bound: &'a HashMap<String, Value>, key: &str) -> Result<&'a Value
|
||||
.get(key)
|
||||
.ok_or_else(|| format!("host call missing `{key}`"))
|
||||
}
|
||||
|
||||
/// Propagate run/selection identity onto the *child* Command.env map.
|
||||
/// Reads parent env only as a fallback when an explicit RunContext was not
|
||||
/// applied; never uses parent `setenv`/`set_current_dir` as request state (M-R1).
|
||||
fn apply_run_selection_env(cmd: &mut Command) {
|
||||
for key in [
|
||||
"MARQDO_RUN_ID",
|
||||
"MARQDO_PARENT_RUN_ID",
|
||||
"MARQDO_TURN_ID",
|
||||
"MARQDO_WORKSPACE_SCOPE",
|
||||
"MARQDO_CANCEL_TOKEN",
|
||||
"MARQDO_EVENT_FORWARD",
|
||||
"MARQDO_USAGE_SINK",
|
||||
"MARQDO_CREDENTIAL_REF",
|
||||
"MARQDO_WORKSPACE_ID",
|
||||
"MARQDO_SNAPSHOT_ID",
|
||||
"MARQDO_LEASE_ID",
|
||||
"MARQDO_ATTEMPT_ID",
|
||||
"MARQDO_LLM_MODEL",
|
||||
"MARQDO_LLM_BASE_URL",
|
||||
"OPENAI_MODEL",
|
||||
"OPENAI_BASE_URL",
|
||||
] {
|
||||
if let Ok(v) = std::env::var(key) {
|
||||
cmd.env(key, v);
|
||||
}
|
||||
}
|
||||
// Do not forward process-global MARQDO_CANCEL into children — request cancel
|
||||
// must use MARQDO_CANCEL_TOKEN so leases stay isolated.
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::collections::HashMap;
|
||||
use std::process::Command;
|
||||
|
||||
#[test]
|
||||
fn f02_apply_run_selection_env_forwards_parent_identity() {
|
||||
std::env::set_var("MARQDO_RUN_ID", "run-parent");
|
||||
std::env::set_var("OPENAI_MODEL", "child-model");
|
||||
std::env::set_var("OPENAI_BASE_URL", "http://127.0.0.1:7010");
|
||||
// Must never require writing secrets into child env map for selection.
|
||||
std::env::remove_var("OPENAI_API_KEY");
|
||||
|
||||
let mut cmd = Command::new("true");
|
||||
apply_run_selection_env(&mut cmd);
|
||||
// Command doesn't expose env for inspection portably; re-check via helper contract:
|
||||
let keys: HashMap<&str, String> = [
|
||||
"MARQDO_RUN_ID",
|
||||
"OPENAI_MODEL",
|
||||
"OPENAI_BASE_URL",
|
||||
]
|
||||
.into_iter()
|
||||
.filter_map(|k| std::env::var(k).ok().map(|v| (k, v)))
|
||||
.collect();
|
||||
assert_eq!(keys.get("MARQDO_RUN_ID").map(String::as_str), Some("run-parent"));
|
||||
assert_eq!(keys.get("OPENAI_MODEL").map(String::as_str), Some("child-model"));
|
||||
assert_eq!(
|
||||
keys.get("OPENAI_BASE_URL").map(String::as_str),
|
||||
Some("http://127.0.0.1:7010")
|
||||
);
|
||||
|
||||
std::env::remove_var("MARQDO_RUN_ID");
|
||||
std::env::remove_var("OPENAI_MODEL");
|
||||
std::env::remove_var("OPENAI_BASE_URL");
|
||||
let _ = cmd; // ensure apply didn't panic
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f02_agent_skeleton_has_no_llm_llm() {
|
||||
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
|
||||
let agent = std::fs::read_to_string(root.join("ext/ai/agent.mq.md")).unwrap();
|
||||
assert!(
|
||||
!agent.contains("llm.llm"),
|
||||
"render_workbook_skeleton must not hardcode llm.llm"
|
||||
);
|
||||
assert!(
|
||||
agent.contains("llm.create") && agent.contains("OPENAI_MODEL"),
|
||||
"skeleton should create model from env selection"
|
||||
);
|
||||
}
|
||||
}
|
||||
+278
-129
@@ -15,7 +15,7 @@ use crate::builtin::{
|
||||
};
|
||||
use crate::debug::{emit_trace, DebugController};
|
||||
use crate::diagnostics::{bail_at, Span};
|
||||
use crate::host::{call_host, HostContext, HostFn};
|
||||
use crate::host::{call_host, HostContext};
|
||||
use crate::input_feed::InputFeed;
|
||||
use crate::capture::{
|
||||
BranchHit, CallFrame, LoopFrame, LoopHit, LoopIter, StmtEvent, StmtEventKind, ViewTrace,
|
||||
@@ -151,6 +151,13 @@ impl Interpreter {
|
||||
bail_at(self.path.as_deref(), self.current_span, message)
|
||||
}
|
||||
|
||||
fn err_code(&self, code: &str, message: impl Into<String>) -> anyhow::Error {
|
||||
anyhow::Error::new(
|
||||
crate::diagnostics::Diagnostic::new(self.path.as_deref(), self.current_span, message)
|
||||
.with_code(code),
|
||||
)
|
||||
}
|
||||
|
||||
fn emit_line(&mut self, text: &str) {
|
||||
if self.capture {
|
||||
self.captured_stdout.push_str(text);
|
||||
@@ -349,9 +356,82 @@ impl Interpreter {
|
||||
) -> Result<Value> {
|
||||
let fun = find_function_anywhere(module, name)
|
||||
.ok_or_else(|| self.err(format!("unknown function `{name}`")))?;
|
||||
self.invoke_bound_function(module, fun, args)
|
||||
}
|
||||
|
||||
/// Run an already-bound function node (SymbolId / source-line identity).
|
||||
/// Prefer this over name lookup so nested same-named functions cannot drift (G07).
|
||||
pub fn invoke_bound_function(
|
||||
&mut self,
|
||||
module: &Module,
|
||||
fun: &Function,
|
||||
args: &[(String, Value)],
|
||||
) -> Result<Value> {
|
||||
self.run_function(module, fun, Env::new(), args, false)
|
||||
}
|
||||
|
||||
/// Bind named arguments for an external call. Unknown names, missing
|
||||
/// required parameters, and contract mismatches are errors. Defaults run.
|
||||
pub fn bind_external_args(
|
||||
&mut self,
|
||||
module: &Module,
|
||||
fun: &Function,
|
||||
args: &[(String, Value)],
|
||||
) -> Result<Vec<(String, Value)>> {
|
||||
self.current_span = fun.span;
|
||||
self.host.current_line = fun.span.line;
|
||||
let mut named: HashMap<String, Value> = HashMap::new();
|
||||
for (k, v) in args {
|
||||
if named.insert(k.clone(), v.clone()).is_some() {
|
||||
return Err(self.err_code(
|
||||
"call.duplicate_arg",
|
||||
format!("duplicate named argument `{k}`"),
|
||||
));
|
||||
}
|
||||
}
|
||||
for k in named.keys() {
|
||||
if !fun.params.iter().any(|p| &p.name == k) {
|
||||
return Err(self.err_code(
|
||||
"call.unknown_arg",
|
||||
format!("unknown argument `{k}`"),
|
||||
));
|
||||
}
|
||||
}
|
||||
let ev: Vec<EvArg> = named
|
||||
.into_iter()
|
||||
.map(|(k, v)| EvArg::Named(k, v))
|
||||
.collect();
|
||||
let mut env = Env::new();
|
||||
let bound = bind_function_args(self, module, fun, &mut env, &fun.params, &ev, false)
|
||||
.map_err(|m| {
|
||||
if m.starts_with("missing argument") {
|
||||
self.err_code("call.missing_arg", m)
|
||||
} else if m.starts_with("duplicate named argument") {
|
||||
self.err_code("call.duplicate_arg", m)
|
||||
} else {
|
||||
self.err(m)
|
||||
}
|
||||
})?;
|
||||
if let Some(c) = &fun.contract {
|
||||
if let Some(d) = crate::contract::call_arg_diag(
|
||||
c,
|
||||
&fun.name,
|
||||
&bound,
|
||||
self.path.as_deref(),
|
||||
fun.span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
let mut out = Vec::new();
|
||||
for p in &fun.params {
|
||||
if let Some(v) = bound.get(&p.name) {
|
||||
out.push((p.name.clone(), v.clone()));
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Call a function using (and updating) [`Self::entry_bindings`] from `# main`.
|
||||
/// Used by browser WASM sessions so handlers share state with the entry object.
|
||||
pub fn invoke_in_entry(
|
||||
@@ -534,6 +614,20 @@ impl Interpreter {
|
||||
])
|
||||
}
|
||||
|
||||
/// Pin `module` as site and call `lib.member` (isolated worker / CLI `--call-lib-path`).
|
||||
pub fn run_site_lib_path(
|
||||
&mut self,
|
||||
module: &Module,
|
||||
path: &str,
|
||||
args: Option<&Value>,
|
||||
) -> std::result::Result<Value, String> {
|
||||
let prev = self.site_module;
|
||||
self.site_module = Some(module as *const Module);
|
||||
let out = self.call_site_lib_path(path, args);
|
||||
self.site_module = prev;
|
||||
out
|
||||
}
|
||||
|
||||
/// Resolve `lib.member` on the site (entry) module — used by ABI `call_lib_path`.
|
||||
pub(crate) fn call_site_lib_path(
|
||||
&mut self,
|
||||
@@ -1169,34 +1263,150 @@ impl Interpreter {
|
||||
return self.eval_method_call(module, fun, env, recv_name, &call.callee, &ev_args);
|
||||
}
|
||||
|
||||
// Local-var method sugar (`a.b` when `a` is in env) before BoundCall path resolve.
|
||||
if let Some(path) = &call.path {
|
||||
// Bare dotted callee `a.b`: if `a` is a local variable, it is a method
|
||||
// receiver (Python-style unified namespace); otherwise it's a library path.
|
||||
if path.len() == 2 && env.get(&path[0]).is_some() {
|
||||
if path.len() == 2
|
||||
&& env.get(&path[0]).is_some()
|
||||
&& !module.import_modules.contains_key(&path[0])
|
||||
{
|
||||
let recv = path[0].clone();
|
||||
let method = path[1].clone();
|
||||
return self.eval_method_call(module, fun, env, &recv, &method, &ev_args);
|
||||
}
|
||||
return self.eval_path_call(module, path, &ev_args);
|
||||
}
|
||||
|
||||
if let Some(u) = module.uses.iter().find(|u| u.bind == call.callee) {
|
||||
return self.eval_path_call(module, &u.path, &ev_args);
|
||||
use crate::sema::bind::{resolve_for_exec, DynamicReason, ExecTarget};
|
||||
match resolve_for_exec(module, fun, &call) {
|
||||
ExecTarget::Local(target) => {
|
||||
self.eval_bound_function_call(module, fun, env, target, &ev_args)
|
||||
}
|
||||
ExecTarget::Import {
|
||||
lib: lib_mod,
|
||||
fun: target,
|
||||
..
|
||||
} => self.eval_bound_function_call(lib_mod, fun, env, target, &ev_args),
|
||||
ExecTarget::Builtin(name) => self.eval_builtin_call(name, &ev_args),
|
||||
ExecTarget::Host(hf) => {
|
||||
let mut params: Vec<String> =
|
||||
hf.required_params().iter().map(|s| (*s).to_string()).collect();
|
||||
for p in hf.optional_params() {
|
||||
params.push((*p).to_string());
|
||||
}
|
||||
let bound = bind_args(¶ms, &ev_args, true).map_err(|m| self.err(m))?;
|
||||
for req in hf.required_params() {
|
||||
if !bound.contains_key(*req) {
|
||||
return Err(self.err(format!("{} requires `{req}`", hf.name())));
|
||||
}
|
||||
}
|
||||
call_host(&mut self.host, hf, &bound).map_err(|m| self.err(m))
|
||||
}
|
||||
ExecTarget::Dynamic {
|
||||
reason: DynamicReason::CallFn,
|
||||
..
|
||||
} => {
|
||||
let bound = bind_args(&["name".into()], &ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let name = match bound.get("name") {
|
||||
Some(Value::Text(s)) => s.as_str(),
|
||||
_ => return Err(self.err("call_fn requires text name")),
|
||||
};
|
||||
let target = lookup_function(module, fun, name).ok_or_else(|| {
|
||||
self.err(format!("call_fn: unknown function `{name}`"))
|
||||
})?;
|
||||
if !target.params.is_empty() {
|
||||
return Err(self.err(format!(
|
||||
"call_fn: `{name}` must have no parameters"
|
||||
)));
|
||||
}
|
||||
self.run_function(module, target, Env::new(), &[], false)
|
||||
}
|
||||
ExecTarget::Dynamic {
|
||||
reason: DynamicReason::Plugin,
|
||||
name,
|
||||
} => {
|
||||
if let Some(reg) = self.host.plugins.get(&name).cloned() {
|
||||
let bound =
|
||||
bind_args(®.params, &ev_args, false).map_err(|m| self.err(m))?;
|
||||
return self
|
||||
.call_plugin_registered(&name, &bound)
|
||||
.map_err(|m| self.err(m));
|
||||
}
|
||||
Err(self.err(format!("unknown function `{name}`")))
|
||||
}
|
||||
ExecTarget::Dynamic {
|
||||
reason: DynamicReason::MethodReceiver,
|
||||
name,
|
||||
} => {
|
||||
if let Some(path) = &call.path {
|
||||
if path.len() == 2 && env.get(&path[0]).is_some() {
|
||||
return self.eval_method_call(
|
||||
module,
|
||||
fun,
|
||||
env,
|
||||
&path[0],
|
||||
&path[1],
|
||||
&ev_args,
|
||||
);
|
||||
}
|
||||
}
|
||||
Err(self.err(format!("unresolved dynamic call `{name}`")))
|
||||
}
|
||||
ExecTarget::Unresolved { message, .. } => Err(self.err(message)),
|
||||
}
|
||||
}
|
||||
|
||||
match call.callee.as_str() {
|
||||
/// Invoke a function already resolved via [`crate::sema::bind::resolve_for_exec`].
|
||||
fn eval_bound_function_call(
|
||||
&mut self,
|
||||
module: &Module,
|
||||
caller: &Function,
|
||||
env: &mut Env,
|
||||
target: &Function,
|
||||
ev_args: &[EvArg],
|
||||
) -> Result<Value> {
|
||||
let bound =
|
||||
bind_function_args(self, module, caller, env, &target.params, ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
if let Some(c) = &target.contract {
|
||||
if let Some(d) = crate::contract::call_arg_diag(
|
||||
c,
|
||||
&target.name,
|
||||
&bound,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
let mut call_env = Env::new();
|
||||
for (k, v) in bound {
|
||||
call_env.set(k, v);
|
||||
}
|
||||
self.host.push_call_site_line(self.host.current_line);
|
||||
let result = self.run_function(module, target, call_env, &[], false);
|
||||
self.host.pop_call_site_line();
|
||||
let result = result?;
|
||||
if target.is_object() {
|
||||
Ok(tag_instance(&target.name, result))
|
||||
} else {
|
||||
Ok(result)
|
||||
}
|
||||
}
|
||||
|
||||
fn eval_builtin_call(&mut self, name: &str, ev_args: &[EvArg]) -> Result<Value> {
|
||||
match name {
|
||||
"print" => {
|
||||
let bound = bind_args(&["text".into()], &ev_args, false)
|
||||
let bound = bind_args(&["text".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let text = bound
|
||||
.get("text")
|
||||
.map(|v| v.as_display())
|
||||
.ok_or_else(|| self.err("print requires text (named or positional)"))?;
|
||||
self.emit_line(&text);
|
||||
return Ok(Value::None);
|
||||
Ok(Value::None)
|
||||
}
|
||||
"input" => {
|
||||
let bound = bind_args(&["prompt".into()], &ev_args, true)
|
||||
let bound = bind_args(&["prompt".into()], ev_args, true)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let prompt = bound
|
||||
.get("prompt")
|
||||
@@ -1221,52 +1431,52 @@ impl Interpreter {
|
||||
e
|
||||
}
|
||||
})?;
|
||||
return Ok(Value::Text(line));
|
||||
Ok(Value::Text(line))
|
||||
}
|
||||
"len" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("len requires value"))?;
|
||||
let n = builtin_len(v).map_err(|m| self.err(m))?;
|
||||
return Ok(Value::Int(n));
|
||||
Ok(Value::Int(n))
|
||||
}
|
||||
"str" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("str requires value"))?;
|
||||
return Ok(builtin_str(v));
|
||||
Ok(builtin_str(v))
|
||||
}
|
||||
"int" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("int requires value"))?;
|
||||
let n = builtin_int(v).map_err(|m| self.err(m))?;
|
||||
return Ok(Value::Int(n));
|
||||
Ok(Value::Int(n))
|
||||
}
|
||||
"type" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("type requires value"))?;
|
||||
return Ok(builtin_type(v));
|
||||
Ok(builtin_type(v))
|
||||
}
|
||||
"trim" => {
|
||||
let bound = bind_args(&["value".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let v = bound
|
||||
.get("value")
|
||||
.ok_or_else(|| self.err("trim requires value"))?;
|
||||
return builtin_trim(v).map_err(|m| self.err(m));
|
||||
builtin_trim(v).map_err(|m| self.err(m))
|
||||
}
|
||||
"split" => {
|
||||
let bound = bind_args(&["value".into(), "sep".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into(), "sep".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let value = bound
|
||||
.get("value")
|
||||
@@ -1274,10 +1484,10 @@ impl Interpreter {
|
||||
let sep = bound
|
||||
.get("sep")
|
||||
.ok_or_else(|| self.err("split requires sep"))?;
|
||||
return builtin_split(value, sep).map_err(|m| self.err(m));
|
||||
builtin_split(value, sep).map_err(|m| self.err(m))
|
||||
}
|
||||
"join" => {
|
||||
let bound = bind_args(&["value".into(), "sep".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into(), "sep".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let value = bound
|
||||
.get("value")
|
||||
@@ -1285,10 +1495,10 @@ impl Interpreter {
|
||||
let sep = bound
|
||||
.get("sep")
|
||||
.ok_or_else(|| self.err("join requires sep"))?;
|
||||
return builtin_join(value, sep).map_err(|m| self.err(m));
|
||||
builtin_join(value, sep).map_err(|m| self.err(m))
|
||||
}
|
||||
"at" => {
|
||||
let bound = bind_args(&["value".into(), "index".into()], &ev_args, false)
|
||||
let bound = bind_args(&["value".into(), "index".into()], ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let value = bound
|
||||
.get("value")
|
||||
@@ -1296,80 +1506,9 @@ impl Interpreter {
|
||||
let index = bound
|
||||
.get("index")
|
||||
.ok_or_else(|| self.err("at requires index"))?;
|
||||
return builtin_at(value, index).map_err(|m| self.err(m));
|
||||
builtin_at(value, index).map_err(|m| self.err(m))
|
||||
}
|
||||
"call_fn" => {
|
||||
let bound = bind_args(&["name".into()], &ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
let name = match bound.get("name") {
|
||||
Some(Value::Text(s)) => s.as_str(),
|
||||
_ => return Err(self.err("call_fn requires text name")),
|
||||
};
|
||||
let target = lookup_function(module, fun, name).ok_or_else(|| {
|
||||
self.err(format!("call_fn: unknown function `{name}`"))
|
||||
})?;
|
||||
if !target.params.is_empty() {
|
||||
return Err(self.err(format!(
|
||||
"call_fn: `{name}` must have no parameters"
|
||||
)));
|
||||
}
|
||||
return self.run_function(module, target, Env::new(), &[], false);
|
||||
}
|
||||
other if HostFn::from_name(other).is_some() => {
|
||||
let hf = HostFn::from_name(other).unwrap();
|
||||
let mut params: Vec<String> =
|
||||
hf.required_params().iter().map(|s| (*s).to_string()).collect();
|
||||
for p in hf.optional_params() {
|
||||
params.push((*p).to_string());
|
||||
}
|
||||
let bound = bind_args(¶ms, &ev_args, true).map_err(|m| self.err(m))?;
|
||||
for req in hf.required_params() {
|
||||
if !bound.contains_key(*req) {
|
||||
return Err(self.err(format!("{} requires `{req}`", hf.name())));
|
||||
}
|
||||
}
|
||||
return call_host(&mut self.host, hf, &bound).map_err(|m| self.err(m));
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
if let Some(reg) = self.host.plugins.get(&call.callee).cloned() {
|
||||
let bound = bind_args(®.params, &ev_args, false).map_err(|m| self.err(m))?;
|
||||
return self
|
||||
.call_plugin_registered(&call.callee, &bound)
|
||||
.map_err(|m| self.err(m));
|
||||
}
|
||||
|
||||
let target = lookup_function(module, fun, &call.callee)
|
||||
.ok_or_else(|| self.err(format!("unknown function `{}`", call.callee)))?;
|
||||
|
||||
let bound = bind_function_args(self, module, fun, env, &target.params, &ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
// 契约边界 1(T2.2):调用实参 vs 形参契约。
|
||||
if let Some(c) = &target.contract {
|
||||
if let Some(d) = crate::contract::call_arg_diag(
|
||||
c,
|
||||
&target.name,
|
||||
&bound,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
let mut call_env = Env::new();
|
||||
for (k, v) in bound {
|
||||
call_env.set(k, v);
|
||||
}
|
||||
|
||||
self.host.push_call_site_line(self.host.current_line);
|
||||
let result = self.run_function(module, target, call_env, &[], false);
|
||||
self.host.pop_call_site_line();
|
||||
let result = result?;
|
||||
if target.is_object() {
|
||||
Ok(tag_instance(&target.name, result))
|
||||
} else {
|
||||
Ok(result)
|
||||
other => Err(self.err(format!("unknown builtin `{other}`"))),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1380,36 +1519,8 @@ impl Interpreter {
|
||||
path: &[String],
|
||||
ev_args: &[EvArg],
|
||||
) -> Result<Value> {
|
||||
let (lib, node) = locate_import_function(module, path).map_err(|m| self.err(m))?;
|
||||
let display = path.join(".");
|
||||
if path.len() < 2 {
|
||||
return Err(self.err(format!("library path `{display}`: need at least `lib.member`")));
|
||||
}
|
||||
let lib_name = &path[0];
|
||||
let lib = module.import_modules.get(lib_name).ok_or_else(|| {
|
||||
self.err(format!("unknown library `{lib_name}` (import it in frontmatter)"))
|
||||
})?;
|
||||
|
||||
let mut parent_is_object = false;
|
||||
let mut node = lib
|
||||
.functions
|
||||
.iter()
|
||||
.find(|f| f.name == path[1])
|
||||
.ok_or_else(|| self.err(format!("unknown `{display}`")))?;
|
||||
for seg in &path[2..] {
|
||||
parent_is_object = node.is_object();
|
||||
node = node
|
||||
.children
|
||||
.iter()
|
||||
.find(|c| c.name == *seg)
|
||||
.ok_or_else(|| self.err(format!("unknown `{display}`")))?;
|
||||
}
|
||||
|
||||
if parent_is_object && !node.is_object() && crate::ast::function_uses_receiver(node) {
|
||||
return Err(self.err(format!(
|
||||
"`{display}` is an instance method; call it as `var`.{}",
|
||||
node.name
|
||||
)));
|
||||
}
|
||||
|
||||
let bound = {
|
||||
let mut dummy_env = Env::new();
|
||||
@@ -1543,6 +1654,44 @@ fn tag_instance(type_name: &str, value: Value) -> Value {
|
||||
}
|
||||
}
|
||||
|
||||
/// `lib.member` / `lib.Type.member` against frontmatter imports. Does not execute.
|
||||
pub(crate) fn locate_import_function<'a>(
|
||||
module: &'a Module,
|
||||
path: &[String],
|
||||
) -> Result<(&'a Module, &'a Function), String> {
|
||||
let display = path.join(".");
|
||||
if path.len() < 2 {
|
||||
return Err(format!(
|
||||
"library path `{display}`: need at least `lib.member`"
|
||||
));
|
||||
}
|
||||
let lib = module
|
||||
.import_modules
|
||||
.get(&path[0])
|
||||
.ok_or_else(|| format!("unknown library `{}` (import it in frontmatter)", path[0]))?;
|
||||
let mut parent_is_object = false;
|
||||
let mut node = lib
|
||||
.functions
|
||||
.iter()
|
||||
.find(|f| f.name == path[1])
|
||||
.ok_or_else(|| format!("unknown `{display}`"))?;
|
||||
for seg in &path[2..] {
|
||||
parent_is_object = node.is_object();
|
||||
node = node
|
||||
.children
|
||||
.iter()
|
||||
.find(|c| c.name == *seg)
|
||||
.ok_or_else(|| format!("unknown `{display}`"))?;
|
||||
}
|
||||
if parent_is_object && !node.is_object() && crate::ast::function_uses_receiver(node) {
|
||||
return Err(format!(
|
||||
"`{display}` is an instance method; call it as `var`.{}",
|
||||
node.name
|
||||
));
|
||||
}
|
||||
Ok((lib, node))
|
||||
}
|
||||
|
||||
/// Bind evaluated args onto parameter names. Err is a plain message (caller adds span/path).
|
||||
fn bind_function_args(
|
||||
interp: &mut Interpreter,
|
||||
|
||||
+119
-53
@@ -17,87 +17,141 @@ pub struct KnowledgeOptions {
|
||||
}
|
||||
|
||||
pub fn compile(opts: KnowledgeOptions) -> Result<KnowledgeGraph> {
|
||||
// The semantic projection is the authoritative machine-facing graph. The
|
||||
// older EKC catalog below remains intentionally available as a compatible
|
||||
// human/retrieval projection, but never substitutes for semantic binding.
|
||||
// Authoritative execution/binding graph: SemanticProject under `graph/`.
|
||||
// Legacy EKC retrieval catalog writes only under `ekc/` and must never
|
||||
// overwrite `graph/edges.json` (G01).
|
||||
let semantic_root = if opts.path.is_file() {
|
||||
opts.path.parent().unwrap_or_else(|| Path::new(".")).to_path_buf()
|
||||
} else {
|
||||
opts.path.clone()
|
||||
};
|
||||
let t0 = std::time::Instant::now();
|
||||
let semantic = crate::sema::compile_project(&crate::sema::CompileOptions {
|
||||
project_root: semantic_root,
|
||||
entry: opts.path.is_file().then(|| opts.path.clone()),
|
||||
lock_mode: crate::sema::LockMode::Off,
|
||||
})?;
|
||||
let sema_ms = t0.elapsed().as_millis();
|
||||
let t1 = std::time::Instant::now();
|
||||
crate::sema::write_semantic_projections(&opts.out_dir, &semantic)?;
|
||||
let sema_write_ms = t1.elapsed().as_millis();
|
||||
let extract_opts = ExtractOptions {
|
||||
path: opts.path.clone(),
|
||||
out_dir: opts.out_dir.clone(),
|
||||
};
|
||||
let t2 = std::time::Instant::now();
|
||||
let graph = extract_graph(&extract_opts)?;
|
||||
let extract_ms = t2.elapsed().as_millis();
|
||||
let t3 = std::time::Instant::now();
|
||||
write_projections(&opts.out_dir, &graph)?;
|
||||
let retrieve_write_ms = t3.elapsed().as_millis();
|
||||
let t4 = std::time::Instant::now();
|
||||
write_publish_manifest(&opts.out_dir, &semantic, &graph)?;
|
||||
let manifest_ms = t4.elapsed().as_millis();
|
||||
eprintln!(
|
||||
"marqdo knowledge: {} module(s), {} symbol(s), {} capability(ies), {} knowledge → {}",
|
||||
"marqdo knowledge: {} module(s), {} symbol(s), {} capability(ies), {} knowledge → {} (semantic graph/; retrieval ekc/)",
|
||||
graph.modules.len(),
|
||||
graph.symbols.len(),
|
||||
graph.capabilities.len(),
|
||||
graph.knowledge.len(),
|
||||
opts.out_dir.display()
|
||||
);
|
||||
eprintln!(
|
||||
"marqdo knowledge timing_ms: sema={sema_ms} sema_write={sema_write_ms} extract={extract_ms} retrieve_write={retrieve_write_ms} manifest={manifest_ms} total={}",
|
||||
t0.elapsed().as_millis()
|
||||
);
|
||||
Ok(graph)
|
||||
}
|
||||
|
||||
/// M06: associate semantic generation, retrieval generation, and source digests.
|
||||
fn write_publish_manifest(
|
||||
out_dir: &Path,
|
||||
semantic: &crate::sema::Compilation,
|
||||
graph: &KnowledgeGraph,
|
||||
) -> Result<()> {
|
||||
let current = fs::read_to_string(out_dir.join("CURRENT")).unwrap_or_default();
|
||||
let generation = current.trim().to_string();
|
||||
let source_digest = {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut h = Sha256::new();
|
||||
for a in &semantic.semantics.artifacts {
|
||||
h.update(a.hashes.artifact.0.as_bytes());
|
||||
}
|
||||
format!("sha256:{:x}", h.finalize())
|
||||
};
|
||||
let evidence_digest = {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut h = Sha256::new();
|
||||
for a in &semantic.semantics.artifacts {
|
||||
h.update(a.hashes.evidence.0.as_bytes());
|
||||
}
|
||||
format!("sha256:{:x}", h.finalize())
|
||||
};
|
||||
let manifest = json!({
|
||||
"schema_version": 1,
|
||||
"kind": "publish_manifest",
|
||||
"semantic_generation": generation,
|
||||
"retrieval": {
|
||||
"ekc_graph": "ekc/graph",
|
||||
"modules": graph.modules.len(),
|
||||
"symbols": graph.symbols.len(),
|
||||
"capabilities": graph.capabilities.len(),
|
||||
"knowledge": graph.knowledge.len(),
|
||||
},
|
||||
"source_digest": source_digest,
|
||||
"evidence_digest": evidence_digest,
|
||||
"excluded_dirs": crate::project::EXCLUDED_DIRS,
|
||||
"project_id": semantic.semantics.project_id,
|
||||
});
|
||||
let path = out_dir.join("publish_manifest.json");
|
||||
fs::write(&path, serde_json::to_string_pretty(&manifest)?)
|
||||
.with_context(|| format!("write {}", path.display()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Directory for the legacy EKC retrieval projection (not execution authority).
|
||||
pub fn ekc_graph_dir(out_dir: &Path) -> std::path::PathBuf {
|
||||
out_dir.join("ekc").join("graph")
|
||||
}
|
||||
|
||||
/// Path of the full IR snapshot used by warm query paths (preflight/find/reuse).
|
||||
pub fn ir_snapshot_path(out_dir: &Path) -> std::path::PathBuf {
|
||||
ekc_graph_dir(out_dir).join("ir.json")
|
||||
}
|
||||
|
||||
/// Load a previously compiled KnowledgeGraph without re-extracting sources.
|
||||
/// Prefers `ekc/graph/ir.json`. Does **not** walk the source tree.
|
||||
pub fn load_graph(out_dir: &Path) -> Result<KnowledgeGraph> {
|
||||
let graph_path = out_dir.join("graph").join("graph.json");
|
||||
let edges_path = out_dir.join("graph").join("edges.json");
|
||||
let ir_path = ir_snapshot_path(out_dir);
|
||||
if ir_path.exists() {
|
||||
let raw = fs::read_to_string(&ir_path)
|
||||
.with_context(|| format!("read {}", ir_path.display()))?;
|
||||
let graph: KnowledgeGraph = serde_json::from_str(&raw)
|
||||
.with_context(|| format!("parse {}", ir_path.display()))?;
|
||||
return Ok(graph);
|
||||
}
|
||||
|
||||
let graph_path = ekc_graph_dir(out_dir).join("graph.json");
|
||||
let edges_path = ekc_graph_dir(out_dir).join("edges.json");
|
||||
if !graph_path.exists() {
|
||||
anyhow::bail!(
|
||||
"no knowledge graph at {} — run `marqdo knowledge` first",
|
||||
"no EKC retrieval graph at {} — run `marqdo knowledge` first",
|
||||
graph_path.display()
|
||||
);
|
||||
}
|
||||
// Prefer re-extract for freshness when source path known; for query commands
|
||||
// we rebuild from JSON snapshot fields + re-scan if catalog sibling exists.
|
||||
// Snapshot load: reconstruct minimal IR from written pages + edges.json.
|
||||
|
||||
// Legacy projections without ir.json: edges-only fallback (no source walk).
|
||||
let edges_raw = fs::read_to_string(&edges_path).unwrap_or_else(|_| "{\"edges\":[]}".into());
|
||||
let edges_v: serde_json::Value = serde_json::from_str(&edges_raw)?;
|
||||
let graph_raw = fs::read_to_string(&graph_path)?;
|
||||
let graph_v: serde_json::Value = serde_json::from_str(&graph_raw)?;
|
||||
|
||||
// Re-compile from recorded root if present in graph.json
|
||||
if let Some(root) = graph_v
|
||||
.pointer("/repository/root")
|
||||
.and_then(|v| v.as_str())
|
||||
{
|
||||
let root_path = Path::new(root);
|
||||
if root_path.exists() {
|
||||
return extract_graph(&ExtractOptions {
|
||||
path: root_path.to_path_buf(),
|
||||
out_dir: out_dir.to_path_buf(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: empty-ish graph with edges only
|
||||
let mut edges = Vec::new();
|
||||
if let Some(arr) = edges_v.get("edges").and_then(|e| e.as_array()) {
|
||||
for e in arr {
|
||||
let kind = match e.get("kind").and_then(|k| k.as_str()).unwrap_or("") {
|
||||
"implements" => crate::knowledge::ir::EdgeKind::Implements,
|
||||
"uses" => crate::knowledge::ir::EdgeKind::Uses,
|
||||
"depends_on" => crate::knowledge::ir::EdgeKind::DependsOn,
|
||||
"tested_by" => crate::knowledge::ir::EdgeKind::TestedBy,
|
||||
"documented_by" => crate::knowledge::ir::EdgeKind::DocumentedBy,
|
||||
"constrained_by" => crate::knowledge::ir::EdgeKind::ConstrainedBy,
|
||||
"decided_by" => crate::knowledge::ir::EdgeKind::DecidedBy,
|
||||
"supersedes" => crate::knowledge::ir::EdgeKind::Supersedes,
|
||||
"conflicts_with" => crate::knowledge::ir::EdgeKind::ConflictsWith,
|
||||
"deprecated_by" => crate::knowledge::ir::EdgeKind::DeprecatedBy,
|
||||
"derived_from" => crate::knowledge::ir::EdgeKind::DerivedFrom,
|
||||
"failed_by" => crate::knowledge::ir::EdgeKind::FailedBy,
|
||||
_ => crate::knowledge::ir::EdgeKind::RelatedTo,
|
||||
};
|
||||
let kind = crate::knowledge::ir::EdgeKind::parse(
|
||||
e.get("kind").and_then(|k| k.as_str()).unwrap_or(""),
|
||||
);
|
||||
edges.push(crate::knowledge::ir::Edge {
|
||||
from: e.get("from").and_then(|v| v.as_str()).unwrap_or("").into(),
|
||||
to: e.get("to").and_then(|v| v.as_str()).unwrap_or("").into(),
|
||||
@@ -138,21 +192,27 @@ pub fn load_graph(out_dir: &Path) -> Result<KnowledgeGraph> {
|
||||
})
|
||||
}
|
||||
|
||||
/// Force a full knowledge compile (write path / `--refresh`).
|
||||
pub fn ensure_compiled(path: &Path, out_dir: &Path) -> Result<KnowledgeGraph> {
|
||||
let graph_path = out_dir.join("graph").join("graph.json");
|
||||
if graph_path.exists() {
|
||||
// Always refresh from source when path given
|
||||
return compile(KnowledgeOptions {
|
||||
path: path.to_path_buf(),
|
||||
out_dir: out_dir.to_path_buf(),
|
||||
});
|
||||
}
|
||||
compile(KnowledgeOptions {
|
||||
path: path.to_path_buf(),
|
||||
out_dir: out_dir.to_path_buf(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Query path loader: compile if missing (or `refresh`), else hot-load `ir.json`.
|
||||
pub fn graph_for_query(path: &Path, out_dir: &Path, refresh: bool) -> Result<KnowledgeGraph> {
|
||||
if refresh {
|
||||
return ensure_compiled(path, out_dir);
|
||||
}
|
||||
let has_ir = ir_snapshot_path(out_dir).exists();
|
||||
let has_graph = ekc_graph_dir(out_dir).join("graph.json").exists();
|
||||
if has_ir || has_graph {
|
||||
return load_graph(out_dir);
|
||||
}
|
||||
ensure_compiled(path, out_dir)
|
||||
}
|
||||
|
||||
fn write_projections(out_dir: &Path, graph: &KnowledgeGraph) -> Result<()> {
|
||||
let dirs = [
|
||||
"catalog/modules",
|
||||
@@ -164,7 +224,7 @@ fn write_projections(out_dir: &Path, graph: &KnowledgeGraph) -> Result<()> {
|
||||
"knowledge/patterns",
|
||||
"knowledge/failures",
|
||||
"knowledge/migrations",
|
||||
"graph",
|
||||
"ekc/graph",
|
||||
"agent/policies",
|
||||
"agent/skills",
|
||||
"agent/contexts",
|
||||
@@ -234,20 +294,26 @@ fn write_projections(out_dir: &Path, graph: &KnowledgeGraph) -> Result<()> {
|
||||
fs::write(&path, body)?;
|
||||
}
|
||||
|
||||
// Graph JSON
|
||||
// Retrieval-only EKC graph (must not touch authoritative `graph/`).
|
||||
let ekc = ekc_graph_dir(out_dir);
|
||||
let graph_json = serde_json::to_string_pretty(&graph.to_graph_json())?;
|
||||
fs::write(out_dir.join("graph/graph.json"), graph_json)?;
|
||||
fs::write(ekc.join("graph.json"), graph_json)?;
|
||||
let edges_json = serde_json::to_string_pretty(&graph.to_edges_json())?;
|
||||
fs::write(out_dir.join("graph/edges.json"), edges_json)?;
|
||||
fs::write(ekc.join("edges.json"), edges_json)?;
|
||||
// Full IR snapshot for warm query paths (preflight/find/reuse) — no re-extract.
|
||||
let ir_json = serde_json::to_string(&graph)?;
|
||||
fs::write(ekc.join("ir.json"), ir_json)?;
|
||||
|
||||
// Search index
|
||||
let index = json!({
|
||||
"type": "Marqdo Knowledge Index",
|
||||
"role": "retrieval_only",
|
||||
"note": "Not an execution authority; binding uses graph/edges.json (SemanticProject).",
|
||||
"generated": { "by": graph.generated_by },
|
||||
"entries": graph.index_map(),
|
||||
});
|
||||
fs::write(
|
||||
out_dir.join("graph/index.json"),
|
||||
ekc.join("index.json"),
|
||||
serde_json::to_string_pretty(&index)?,
|
||||
)?;
|
||||
|
||||
|
||||
@@ -164,13 +164,7 @@ fn collect_mq_md(
|
||||
let p = entry.path();
|
||||
if p.is_dir() {
|
||||
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
|
||||
if name == "target"
|
||||
|| name == ".git"
|
||||
|| name == "node_modules"
|
||||
|| name == "docker"
|
||||
|| name == "data"
|
||||
|| name == ".cursor"
|
||||
{
|
||||
if crate::project::is_excluded_dir_name(name) {
|
||||
continue;
|
||||
}
|
||||
if let Ok(canon) = p.canonicalize() {
|
||||
@@ -210,13 +204,7 @@ fn walk_knowledge_md(
|
||||
let p = entry.path();
|
||||
if p.is_dir() {
|
||||
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
|
||||
if name == "target"
|
||||
|| name == ".git"
|
||||
|| name == ".marqdo"
|
||||
|| name == "docker"
|
||||
|| name == "data"
|
||||
|| name == ".cursor"
|
||||
{
|
||||
if crate::project::is_excluded_dir_name(name) {
|
||||
continue;
|
||||
}
|
||||
if let Ok(canon) = p.canonicalize() {
|
||||
|
||||
+67
-9
@@ -1,5 +1,6 @@
|
||||
//! Engineering Knowledge IR (L0–L4) — projections of `.mq.md`, not a second source of truth.
|
||||
|
||||
use serde::{Deserialize, Deserializer, Serialize, Serializer};
|
||||
use serde_json::{json, Value};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
@@ -13,6 +14,19 @@ pub enum Lifecycle {
|
||||
Superseded,
|
||||
}
|
||||
|
||||
impl Serialize for Lifecycle {
|
||||
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
|
||||
serializer.serialize_str(self.as_str())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'de> Deserialize<'de> for Lifecycle {
|
||||
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
|
||||
let s = String::deserialize(deserializer)?;
|
||||
Ok(Self::parse(&s))
|
||||
}
|
||||
}
|
||||
|
||||
impl Lifecycle {
|
||||
pub fn as_str(&self) -> &'static str {
|
||||
match self {
|
||||
@@ -35,7 +49,7 @@ impl Lifecycle {
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Default)]
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct Evidence {
|
||||
pub sources: Vec<String>,
|
||||
pub tests: Vec<String>,
|
||||
@@ -55,7 +69,7 @@ impl Evidence {
|
||||
}
|
||||
|
||||
/// Graded confidence with explanation (not a bare float).
|
||||
#[derive(Debug, Clone)]
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Confidence {
|
||||
pub level: String, // high | medium | low
|
||||
pub implementation: String,
|
||||
@@ -87,7 +101,7 @@ impl Confidence {
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct RepositoryInfo {
|
||||
pub title: String,
|
||||
pub root: String,
|
||||
@@ -98,7 +112,7 @@ pub struct RepositoryInfo {
|
||||
pub knowledge_count: usize,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ModuleInfo {
|
||||
pub id: String,
|
||||
pub resource: String,
|
||||
@@ -111,7 +125,7 @@ pub struct ModuleInfo {
|
||||
pub status: Lifecycle,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct SymbolInfo {
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
@@ -135,7 +149,7 @@ pub struct SymbolInfo {
|
||||
pub confidence: Confidence,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct CapabilityInfo {
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
@@ -215,7 +229,20 @@ impl KnowledgeKind {
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
impl Serialize for KnowledgeKind {
|
||||
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
|
||||
serializer.serialize_str(self.as_str())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'de> Deserialize<'de> for KnowledgeKind {
|
||||
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
|
||||
let s = String::deserialize(deserializer)?;
|
||||
Ok(Self::parse(&s))
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct KnowledgeItem {
|
||||
pub id: String,
|
||||
pub kind: KnowledgeKind,
|
||||
@@ -269,9 +296,40 @@ impl EdgeKind {
|
||||
Self::RelatedTo => "related_to",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse(s: &str) -> Self {
|
||||
match s.trim() {
|
||||
"implements" => Self::Implements,
|
||||
"uses" => Self::Uses,
|
||||
"depends_on" => Self::DependsOn,
|
||||
"tested_by" => Self::TestedBy,
|
||||
"documented_by" => Self::DocumentedBy,
|
||||
"constrained_by" => Self::ConstrainedBy,
|
||||
"decided_by" => Self::DecidedBy,
|
||||
"supersedes" => Self::Supersedes,
|
||||
"conflicts_with" => Self::ConflictsWith,
|
||||
"deprecated_by" => Self::DeprecatedBy,
|
||||
"derived_from" => Self::DerivedFrom,
|
||||
"failed_by" => Self::FailedBy,
|
||||
_ => Self::RelatedTo,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
impl Serialize for EdgeKind {
|
||||
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
|
||||
serializer.serialize_str(self.as_str())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'de> Deserialize<'de> for EdgeKind {
|
||||
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
|
||||
let s = String::deserialize(deserializer)?;
|
||||
Ok(Self::parse(&s))
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Edge {
|
||||
pub from: String,
|
||||
pub to: String,
|
||||
@@ -336,7 +394,7 @@ pub struct ReuseResult {
|
||||
pub recommended: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct KnowledgeGraph {
|
||||
pub repository: RepositoryInfo,
|
||||
pub modules: Vec<ModuleInfo>,
|
||||
|
||||
@@ -52,7 +52,9 @@ pub fn find_stale(graph: &KnowledgeGraph, root: &Path) -> Vec<StaleItem> {
|
||||
if src.exists() {
|
||||
if let (Ok(meta), Ok(proj_meta)) = (
|
||||
fs::metadata(&src),
|
||||
fs::metadata(root.join(".marqdo/graph/graph.json"))
|
||||
fs::metadata(root.join(".marqdo/ekc/graph/graph.json"))
|
||||
.or_else(|_| fs::metadata(root.join(".marqdo/graph/graph.json")))
|
||||
.or_else(|_| fs::metadata(root.join("ekc/graph/graph.json")))
|
||||
.or_else(|_| fs::metadata(root.join("graph/graph.json"))),
|
||||
) {
|
||||
if let (Ok(st), Ok(pt)) = (meta.modified(), proj_meta.modified()) {
|
||||
|
||||
@@ -20,7 +20,10 @@ pub mod preflight;
|
||||
pub mod reuse;
|
||||
pub mod util;
|
||||
|
||||
pub use compile::{compile, ensure_compiled, load_graph, KnowledgeOptions};
|
||||
pub use compile::{
|
||||
compile, ekc_graph_dir, ensure_compiled, graph_for_query, ir_snapshot_path, load_graph,
|
||||
KnowledgeOptions,
|
||||
};
|
||||
pub use duplicate::{find_duplicates, format_duplicates_text};
|
||||
pub use find::{find, find_json, format_find_text};
|
||||
pub use impact::{format_impact_text, impact, impact_json};
|
||||
@@ -32,7 +35,9 @@ pub use lifecycle::{
|
||||
};
|
||||
pub use metrics::{load_metrics, record_decision, save_metrics, ReuseMetrics};
|
||||
pub use policy::compile_policy;
|
||||
pub use preflight::{format_preflight_yaml, preflight, preflight_json, PreflightResult};
|
||||
pub use preflight::{
|
||||
format_preflight_yaml, preflight, preflight_from_reuse, preflight_json, PreflightResult,
|
||||
};
|
||||
pub use reuse::{format_reuse_text, resolve, reuse_json, ReuseBudget};
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
Loaded 100 of 229 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user